School‐based interventions for reducing disciplinary school exclusion: a systematic review

Sara Valdebenito; Manuel Eisner; David P Farrington; Maria M Ttofi; Alex Sutherland

doi:10.4073/csr.2018.1

. 2018 Jan 9;14(1):i–216. doi: 10.4073/csr.2018.1

School‐based interventions for reducing disciplinary school exclusion: a systematic review

Sara Valdebenito, Manuel Eisner, David P Farrington, Maria M Ttofi, Alex Sutherland

PMCID: PMC8533648 PMID: 37131379

Abstract

This Campbell systematic review examines the impact of interventions to reduce exclusion from school. School exclusion, also known as suspension in some countries, is a disciplinary sanction imposed by a responsible school authority, in reaction to students' misbehaviour. Exclusion entails the removal of pupils from regular teaching for a period during which they are not allowed to be present in the classroom (in‐school) or on school premises (out‐of‐school). In some extreme cases the student is not allowed to come back to the same school (expulsion). The review summarises findings from 37 reports covering nine different types of intervention. Most studies were from the USA, and the remainder from the UK.

Included studies evaluated school‐based interventions or school‐supported interventions to reduce the rates of exclusion. Interventions were implemented in mainstream schools and targeted school‐aged children from four to 18, irrespective of nationality or social background. Only randomised controlled trials are included.

The evidence base covers 37 studies. Thirty‐three studies were from the USA, three from the UK, and for one study the country was not clear.

School‐based interventions cause a small and significant drop in exclusion rates during the first six months after intervention (on average), but this effect is not sustained. Interventions seemed to be more effective at reducing some types of exclusion such as expulsion and in‐school exclusion.

Four intervention types – enhancement of academic skills, counselling, mentoring/monitoring, and skills training for teachers – had significant desirable effects on exclusion. However, the number of studies in each case is low, so this result needs to be treated with caution.

There is no impact of the interventions on antisocial behaviour.

Variations in effect sizes are not explained by participants' characteristics, the theoretical basis of the interventions, or the quality of the intervention. Independent evaluator teams reported lower effect sizes than research teams who were also involved in the design and/or delivery of the intervention.

Plain language summary

Interventions can reduce school exclusion but the effect is temporary

Some interventions – enhancement of academic skills, counselling, mentoring/monitoring, and skills training for teachers – appear to have significant effects on exclusion.

The review in brief

Interventions to reduce school exclusion are intended to mitigate the adverse effects of this school sanction. Some approaches, namely those involving enhancement of academic skills, counselling, mentoring/monitoring and those targeting skills training for teachers, have a temporary effect in reducing exclusion. More evaluations are needed to identify the most effective types of intervention; and whether similar effects are also found in different countries.

What is the aim of this review?

This Campbell systematic review examines the impact of interventions to reduce exclusion from school. School exclusion, also known as suspension in some countries, is a disciplinary sanction imposed by a responsible school authority, in reaction to students’ misbehaviour. Exclusion entails the removal of pupils from regular teaching for a period during which they are not allowed to be present in the classroom (in‐school) or on school premises (out‐of‐school). In some extreme cases the student is not allowed to come back to the same school (expulsion). The review summarises findings from 37 reports covering nine different types of intervention. Most studies were from the USA, and the remainder from the UK.

What is this review about?

School exclusion is associated with undesirable effects on developmental outcomes. It increases the likelihood of poor academic performance, antisocial behavior, and poor employment prospects. This school sanction disproportionally affects males, ethnic minorities, those who come from disadvantaged economic backgrounds, and those with special educational needs.

This review assesses the effectiveness of programmes to reduce the prevalence of exclusion.

What are the main findings of this review?

What studies are included?

The evidence base covers 37 studies. Thirty‐three studies were from the USA, three from the UK, and for one study the country was not clear.

Four intervention types ‐ enhancement of academic skills, counselling, mentoring/ monitoring, and skills training for teachers – had significant desirable effects on exclusion. However, the number of studies in each case is low, so this result needs to be treated with caution.

There is no impact of the interventions on antisocial behaviour.

Variations in effect sizes are not explained by participants’ characteristics, the theoretical basis of the interventions, or the quality of the intervention. Independent evaluator teams reported lower effect sizes than research teams who were also involved in the design and/or delivery of the intervention.

What do the findings of this review mean?

School‐based interventions are effective at reducing school exclusion immediately after, and for a few months after, the intervention (6 months on average). Four interventions presented promising and significant results in reducing exclusion, that is, enhancement of academic skills, counselling, mentoring/monitoring, skills training for teachers. However, since the number of studies for each sub‐type of intervention was low, we suggest these results should be treated with caution.

Most of the studies come from the USA. Evaluations are needed from other countries in which exclusion is common. Further research should take advantage of the possibility of conducting cluster‐randomised controlled trials, whilst ensuring that the sample size is sufficiently large.

How up‐to‐date is this review?

The review authors searched for studies published up to December 2015. This Campbell systematic review was published in January 2018.

Executive Summary/Abstract

BACKGROUND

Schools are important institutions of formal social control (Maimon, Antonaccio, & French, 2012). They are, apart from families, the primary social system in which individuals are socialised to follow specific codes of conduct. Violating these codes of conduct may result in some form of punishment. School punishment is normally accepted by families and students as a consequence of transgression, and in that sense school isoften the place where children are first introduced to discipline, justice, or injustice (Whitford & Levine‐Donnerstein, 2014).

A wide range of punishments may be used in schools, from verbal reprimands to more serious actions such as detention, fixed term exclusion or even permanent exclusion from the mainstream education system. It must be said that in some way, these school sanctions resemble the penal system and its array of alternatives to punish those that break the law.

School exclusion, also known as suspension in some countries, is defined as a disciplinary sanction imposed by a responsible school authority, in reaction to students’ misbehaviour. Exclusion entails the removal of pupils from regular teaching for a period during which they are not allowed to be present in the classroom or, in more serious cases, on school premises.Based on the previous definition, this review uses school exclusion and school suspension as synonyms, unless the contrary is explicitly stated.

Most of the available research has found that exclusion correlates with subsequent negative sequels on developmental outcomes. Exclusion or suspension of students is associated with failure within the academic curriculum, aggravated antisocial behaviour, and an increased likelihood of involvement with punitive social control institutions (i.e., the Juvenile Justice System). In the long‐term, opportunities for training and employment seem to be considerably reduced for those who have repeatedly been excluded. In addition to these negative correlated outcomes, previous evidence suggest that the exclusion of students involves a high economic cost for taxpayers and society.

Research from the last 20 years has concluded quite consistently that this disciplinary measure disproportionally targets males, ethnic minorities, those who come from disadvantaged economic backgrounds, and those presenting special educational needs. In other words, suspension affects the most vulnerable children in schools.

Different programmes have attempted to reduce the prevalence of exclusion. Although some of them have shown promising results, so far, no comprehensive systematic review has examined these programmes’ overall effectiveness.

OBJECTIVES

The main goal of the present research is to systematically examine the available evidence for the effectiveness of different types of school‐based interventions aimed at reducing disciplinary school exclusion. Secondary goals include comparing different approaches and identifying those that could potentially demonstrate larger and more significant effects.

The research questions underlying this project are as follows:

Do school‐based programmes reduce the use of exclusionary sanctions in schools?
Are some school‐based approaches more effective than others in reducing exclusionary sanctions?
Do participants’ characteristics (e.g., age, gender, ethnicity) affect the impact of school‐based programmes on exclusionary sanctions in schools?
Do characteristics of the interventions, implementation, and methodology affect the impact of school‐based programmes on exclusionary sanctions in schools?

SEARCH METHODS

The authors conducted a comprehensive search to locate relevant studies reporting on the impact of school‐based interventions on exclusion from 1980 onwards. Twenty‐seven different databases were consulted, including databases that contained both published and unpublished literature. In addition, we contacted researchers in the field of school‐exclusion for further recommendations of relevant studies; we also assessed citation lists from previous systematic and narrative reviews and research reports. Searches were conducted from September 1 to December 1, 2015.

SELECTION CRITERIA

The inclusion and exclusion criteria for manuscripts were defined before we started our searches. To be eligible, studies needed to have: evaluated school‐based interventions or school‐supported interventions intended to reduce the rates of suspension; seen the interventions as an alternative to exclusion; targeted school‐aged children from four to 18 in mainstream schools irrespective of nationality or social background; and reported results of interventions delivered from 1980 onwards. In terms of methodological design, we included randomised controlled trialsonly, with at least one experimental group and onecontrol or placebo group.

DATA COLLECTION AND ANALYSIS

Initial searches produced a total of 42,749 references from 27 different electronic databases. After screening the title, abstract and key words, we kept 1,474 relevant hits. 22 additional manuscripts were identified through other sources (e.g., assessment of citation lists, contribution of authors). After removing duplicates, we ended up with a total of 517 manuscripts. Two independent coders evaluated each report, to determine inclusion or exclusion.

The second round of evaluation excluded 472 papers, with eight papers awaiting classification, and 37 studies kept for inclusion in meta‐analysis. Two independent evaluators assessed all the included manuscripts for risk of quality bias by using EPOC tool.

Due to the broad scope of our targeted programmes, meta‐analysis was conducted under a random‐effect model. We report the impact of the intervention using standardised differences of means, 95% confidence intervals along with the respective forest plots. Sub‐group analysis and meta‐regression were used for examining the impact of the programme. Funnel plots and Duval and Tweedie's trim‐and‐fill analysis were used to explore the effect of publication bias.

RESULTS

Based on our findings, interventions settled in school can produce a small and significant drop in exclusion rates (SMD=.30; 95% CI .20 to .41; p<.001). This means that those participating in interventions are less likely to be suspended than those allocated to control/placebo groups. These results are based on measures of impact collected immediately during the first six months after treatment (on average). When the impact was tested in the long‐term (i.e., 12 or more months after treatment), the effects of the interventions were not sustained. In fact, there was a substantive reduction in the impact of school‐based programmes (SMD=.15; 95%CI ‐.06 to .35), and it was no longer statistically significant.

We ran analysis testing the impact of school‐based interventions on different types of exclusion. Evidence suggests that interventions are more effective at reducing expulsion and in‐school exclusion than out‐of‐school exclusion. In fact, the impact of intervention in out‐of‐school exclusion was close to zero and not statistically significant.

Nine different types of school‐based interventions were identified across the 37 studies included in the review. Four of them presented favourable and significant results in reducing exclusion (i.e., enhancement of academic skills, counselling, mentoring/monitoring, skills training for teachers). Since the number of studies for each sub‐type of intervention was low, we suggest that results should be treated with caution.

A priori defined moderators (i.e., participants’ characteristics, the theoretical basis of the interventions, and quality of the intervention)showed not to be effective at explaining the heterogeneity present in our results. Among three post‐hoc moderators, the role of the evaluator was found to be significant: independent evaluator teams reported lower effect sizes than research teams who were also involved in the design and/or delivery of the intervention.

Two researchers independently evaluated the quality of the evidence involved in this review by using the EPOC tool. Most of the studies did not present enough information for the judgement of quality bias.

AUTHORS’ CONCLUSIONS

The evidence suggests that school‐based interventions are effective at reducing school exclusion immediately after, and for a few months after, the intervention. Some specific types of interventions show more promising and stable results than others, namely those involving mentoring/monitoring and those targeting skills training for teachers. However, based on the number of studies involved in our calculations, we suggest that results must be cautiously interpreted. Implications for policy and practice arising from our results are discussed.

1. Background

1.1 THE PROBLEM, CONDITION OR ISSUE

1.1.1 School discipline

Discipline problems are frequent in schools and they may have a harmful effect on pupils’ learning outcomes. A lack of discipline and the subsequent potential increase in school disorder (e.g. bullying, substance misuse) can seriously threaten the quality of instruction that teachers provide, hamper pupils acquisition of academic skills and subsequently reduce their attachment to the education system (Gottfredson, Cook, & Na, 2012).

As such, discipline represents a serious concern for parents and teachers, demanding significant efforts and resources from schools (Kaplan, Gheen, & Midgley, 2002). The PISA 2009 report (OECD, 2010) stated that schools registering higher levels of disciplinary problems result in teachers spending less time on learning in order to deal with such issues. In its 2012 version, the PISA report asked students about school discipline. Results found that “28% of students reported that teachers had to wait a long time to quiet down every class, or almost all classes”(OECD, 2013). Being more precise, the Teaching and Learning International Survey (TALIS) revealed that teachers spend at least 20% of lesson time dealing with disruption and administrative tasks. In the United Kingdom, the Behaviour Survey 2010 states that 80% of school teachers felt their ability to teach effectively was impaired by students’ poor behaviour (Massey, 2011). On a global level, evidence suggests that 13% of teachers’ time is, on average, spent maintaining order (OECD, 2009).

Schools use different procedures to manage discipline, including a range of punitive responses (e.g., loss of privileges, additional homework or detention). Among these, exclusion is normally seen as one of the most serious punishments. Although types and lengths vary from country to country, school exclusion (also known as school suspension in the United States) ¹ can be broadly defined as a disciplinary sanction imposed in reaction to students’ behaviour (i.e. violations of school policies) by the responsible authority. In concrete terms, exclusion entails a removal from regular teaching for a period of time during which students are also not allowed to be present on school premises. Specifically, fixed‐term exclusions consist of a limited number of hours or days (Cornell, Gregory, & Fan, 2011), whereas permanent exclusion (i.e., expulsion) involves the pupil being transferred to a different school, or educated outside of the regular education system (Spink, 2011; Webb & Vulliamy, 2004).

Even if school policies suggest that exclusion should be used as a last resort, reserved for only the most serious and persistent offences (Gregory & Weinstein, 2008a; Skiba & Peterson, 1999; Skiba, Trachok, Chung, Baker, & Hughes, 2012), research evidence suggests that minor offences can also provoke this type of punishment (Munn, Cullen, Johnstone, & Lloyd, 2001; Skiba, 2014). Fenning et al., (2012) provide a case in point: their research concluded that suspension and expulsion were the most common types of punishment for minor problems such astardiness and school truancy. These findings were also confirmed by Liu (2013) who found that 48% of suspensions lasting a maximum of five days targeted minor disorder or disruptive behaviours.

In terms of prevalence, data provided by the UK Department for Education (academic year 2011/12) shows that in England fixed‐term exclusion affects 3.5% of the school population whereas permanent exclusion applies to only 0.06%. The national figures suggest that students in secondary‐level education (6.8% of the school population) as well as those in special education (14.7%) are the most likely to experience fixed‐term exclusion (DfE, 2013). In the United States, data provided by the Department of Education (academic year 2011/12) concluded that 7.4% (3.5 million) of students were suspended in school, 7% (3.45 million) were suspended out‐of‐school, and less than one per cent were subject to expulsion (around 130,000 students). Black students and those presenting disabilities are, respectively, three and two times more likely to be excluded compared to White and non‐disabled pupils (U.S. Department of Education, 2012).

International comparisons of exclusion prevalence rates are not available in the literature examined. Indeed, differences in use, extent and recording (i.e., unreported exclusions) make an international estimation challenging. In Table 1, the reader will find information regarding the use of exclusion in a sample of high‐ and middle‐income countries. The information is limited to a convenience sample involving twelve different cases to allow an overview of i) the types of exclusion used in these countries, ii) the length of the sanction, iii) the authority responsible for determining this sanction, iv) the behaviours for which school exclusion is permissible, and, in cases where information was available, the table also includes v) the local prevalence of exclusion. This does not claim to be a representative sample of all countries, but as an initial approach will help provide a more complex picture of the phenomenon. In addition, this comparison was intended to help with searches for studies that could be potentially included in the systematic review. For instance, by comparing exclusion in different countries, it was found that the same school sanction had different names in different countries (e.g., “stand‐down” in New Zealand, “exclusion” in the UK and “suspension” in the US).

Table 1.

Comparative description of school exclusion in a sample of high‐ and middle‐income countries

Country	Name given	Type of exclusions	Length (for fixed exclusions)	Who makes the decision?	Legal reasons for exclusion	Prevalence ²
Australia ³ (New South Wales)	Suspension	Short suspensions	4 school days.	School Principal	Continued disobedience and aggressive behaviour	Unknown
		Long suspensions	Up to 20 school days.	School Principal	Physical violence, use or possession of prohibited weapons, firearms or knife, possession, use or supply of a suspected illegal substance, serious criminal behaviour, use a weapon, persistent or serious misbehaviour
	Expulsion	Permanent	Permanent	School Principal	In serious circumstances of misbehaviour, the principal may expel a student of any age from their school. The principal may also expel a student who is over 17 years of age for unsatisfactory participation in learning.
Canada ⁴ (Ontario)	Suspension	Short‐term Long‐term	1 to 20 school days. More than five school days are considered long‐term.	School Principal. Parents must be informed within 24 hours. All suspensions can be appealed to the school board.	Threat to inflict serious bodily harm on another person, possessing alcohol or illegal drugs, being under the influence of alcohol, swearing at a teacher or at another person in a position of authority, committing an act of vandalism that causes extensive damage to school property, or bullying.	2.76% of enrolled students (N= 2,014,407). Academic Year 2013‐2014 ⁵
	Expulsion	From school From all schools (in this case, the students must be offered alternative education)	Indefinite.	School Principal should recommend expulsion to the school board. Parents must be informed within 24 hours. All expulsions can be appealed at a tribunal.	Possessing or using a weapon, physical assault on another person that causes bodily harm requiring treatment by a medical practitioner, sexual assault, trafficking in weapons, trafficking in illegal drugs, robbery, drinking alcohol.	0.02% of enrolled students (N= 2,014,407) Academic Year 2013‐2014 ⁶
Chile ⁷	Suspension	Fixed. Implemented inside the school premises	The law does not limit the duration of fixed suspensions. Each school community issues their own disciplinary code and defines disciplinary sanctions and their duration.	Disciplinary Board	Defined for each school, but it must be used in exceptional cases	Unknown
	Expulsion			School Principal	Conduct that directly threatens the physical or psychological integrity of any member of the school community ⁸	Unknown
Colombia ⁹	Suspension	Fixed	Each school community issues their disciplinary codes and defines disciplinary sanctions and their duration. Normally fixed exclusion lasts 3 days.	Discretionary	Violation to the code of conduct	Unknown
		Definitive			Unknown
Costa Rica ¹⁰	Suspension	Fixed	Up to 8 school days.	School Principal	Not clearly stated	Unknown
		Permanent		School Board	Permanent disruptive/defiant behaviour, non‐compliance with previous sanctions, violence and aggressions towards a member of the school community, lack of moral integrity.
England ¹¹	Exclusion	Fixed (in‐school, out‐of‐school)	1‐45 days per year. After 5 days of fixed out‐of‐school exclusion, the school must provide alternative education.	Discretionary School principal	Repeated failure to follow academic instruction, failure to complete a behavioural sanction (e.g. with a detention, a decision to change the sanction to exclusion would not automatically be unlawful), repeated and persistent breaches of the schools’ behavioural policy.	Academic year 2014‐2015 ¹² 3.8% of students (all schools) 7.51% of students (secondary schools)
		Permanent				0.07% of students (all schools) 0.15% of students (secondary schools)
France ¹³	Exclusion	Temporary exclusion from the classroom	Maximum of 8 days.	Consultation between the various members of the pedagogical and educational team	Serious cases of violence (physical or psychological) against the school community	Unknown
		Temporary exclusion from school	Maximum of 8 days.	School principal or school board
		Definitive exclusion	Permanent	Disciplinary board. The student should be represented on the disciplinary board.
Finland ¹⁴	Exclusion	In‐school exclusion and out‐of‐school exclusion with the school obligated to provide education at home.	In‐school exclusion: remainder of the day. Out‐of‐school exclusion: no more than 3 months. It is a very infrequent measure.	Teacher and school principal using a formal procedure. In cases of out‐of‐school exclusion, a personal plan of education must be provided and local social services should be informed.	Threats or serious violence that would endanger the safety of other members of the school community	Unknown
		Permanent exclusion does not exist in the local law.
Malta	Suspension	Fixed term suspension	Suspension for the rest of the day or for a few days. The number of days is not stated in the law.	Must be applied by the Head of School after the student's parent or guardian has been informed. The National Board for School Behaviour should be consulted.	The law defines 3 levels of misbehaviour. Suspension and expulsion are restricted for level 3, meaning serious offenses only. No further details.	Unknown
	Expulsion	Expulsion	Permanent
Norway ¹⁵	Exclusion	Fixed exclusion, expulsion for the rest of the year and loss of rights to education.	Primary education (level 1‐7): exclusion from specific lessons or for the rest of the day. Secondary education (level 8‐10): maximum of 3 days. Expulsion and loss of rights are defined in the Educational Law but its use is extremely rare.	The school principal in consultation with the pupil's teacher, unless the local authority defines a different procedure.	Exclusion is used as a last resort and can be justified only for serious issues of violence. The law suggests the use of alternatives such as mediation before imposing an exclusion.	Unknown
New Zealand ¹⁶	Stand‐down	Stand‐down	The student is removed from school for 5 school days in a term or 10 school days in a year.	School Principal, through a formal procedure that includes informing the family, the Education Authority and the school board.	Drugs (including substance abuse), continual disobedience and physical assault on other students were the most prevalent causes for stand‐down, suspensions, exclusion and expulsion.	1.5% of school population (2015)
	Suspension	Suspension	The student is removed from school for no more than 7 days.	School Board		0.3% of school population (2015)
	Exclusion	Expulsion	Maximum of 10 days in a year.	School Board		0.1% of the total student population under 16 years old (2015)
	Exclusion	Expulsion	A student under the age of 16 would be excluded from the school, with the requirement that the student enrolls elsewhere	School Board		0.2% of the total student population over 16 years old (2015) ¹⁷
			A student aged 16 or over would be expelled from the school, and the student may or may not enroll at another school.
The US, Washington DC ¹⁸	Suspension	Suspension (short‐term and long‐term) is a restriction in attending school or school activities.	Short‐term suspension: maximum of 10 consecutive days. Long‐term suspension: more than 10 consecutive days.	Certified teachers can decide a suspension but it must be communicated to the school principal. Short‐term suspensions must be formally communicated to the student/parents. Long‐term suspensions and expulsions require a formal process (i.e., written notice by the school district) and should be known by the School Principal.	Violation of school district rules	3.89% of all Washington students have been suspended or expelled (2014–15) The rate of suspensions and expulsions across districts range between nearly 0% to over 10% of students ¹⁹ .
	Expulsion	Expulsion makes this restriction indefinite.	Maximum: 1 calendar year		Violation of school district rules, serious violence, gang activity on school grounds.
		Emergency expulsion	Temporary. The student would go back once the danger ceases		The student's presence poses an immediate and continuing danger to others. The student's presence poses a threat of substantial disruption in the classroom.
The US, Virginia ²⁰	Removal from classes	In‐school		Teacher	Disruptive behaviour	Unknown
	Suspension	Suspension (short‐term and long‐term) is a restriction in attending school or school activities.	Short‐term suspension: 10 consecutive or 10 cumulative school days in a school year Long‐term suspension: more than 10 school days but less than 365 calendar days.	Imposed by the school principal, any assistant principal or, in their absence, any teacher. The suspension should entail a formal process. The student must be heard.	Violation of school code of conduct
	Expulsion	Expulsion makes the restriction last longer.	A student is not permitted to attend school within the school division and is ineligible for readmission for 365 calendar days after expulsion.	Imposed by a committee from the school board. Includes a formal process, written notice and appeal.	Criminal activity, carrying a weapon, drug related offences, or when the pupil presence is a clear threat for the school community.
The US, Texas ²¹	Suspension	In‐school suspension (e.g., seclusion units)	In‐school suspension lasts between 1 class and several days.	Low‐level offences are dealt with on a discretionary basis (according to a defined code of conduct) by the designated administrator (usually the principal or vice principal). Higher‐level offences require mandatory removal from the classroom. Rules for a due process are defined.	Violation of school code of conduct (unruly, disruptive, or abusive behaviours)	9.24% (2014‐2015) ²²
	Expulsion	Out‐of‐school suspension	Out‐of‐school suspension should be no longer than 3 days.	Unknown	Weapon carrying, serious violence or crimes.	4.33% (2014‐2015)
		In the case of serious offences, a student can be expelled from school.	At least 1 year Disciplinary Alternative Education Program (DAEP) for students removed for over 3 days (no maximum period provided).	Unknown		3.39% (2014‐2015)

Open in a new tab

Prevalence was calculated by dividing the number of excluded students per year (numerator) by the total number of students per year (denominator).

Information retrieved from “Suspension and Expulsion of School Students” New South Wales Government. Updated in October 2014 https://www.det.nsw.edu.au/policies/student_serv/discipline/stu_discip_gov/suspol_07.pdf

⁴

http://www.edu.gov.on.ca/eng/safeschools/NeedtoKnowSExp.pdf

⁵

http://www.edu.gov.on.ca/eng/safeschools/statistics.html

⁶

http://www.edu.gov.on.ca/eng/safeschools/statistics.html

⁷

http://www.supereduc.cl/. Additionally, the information can be found in Torche & Mizala (2012)

⁸

http://denuncias.supereduc.cl/cuestionario1/denuncias_tematicos.html

⁹

In Colombia, each school must define school exclusion length. This is established in the Ley General de Educación N° 115, February 1994. http://www.mineducacion.gov.co/1621/articles‐85906_archivo_pdf.pdf. Additional information can be retrieved from http://www.mineducacion.gov.co/1621/article‐86906.html

¹⁰

http://www.mep.go.cr/sites/default/files/Reglamento_General_Establecimientos_Oficiales_de_Educacion_Media.pdf

¹¹

In England, exclusions are regulated by the Education Act of 2002

¹²

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/539704/SFR_26_2016_text.pdf

¹³

In France, school exclusions are regulated by the Code de l'education: http://www.education.gouv.fr/cid56670/sanctions‐scolaires‐reforme‐des‐procedures‐disciplinaires‐dans‐les‐etablissements‐scolaires.html

¹⁴

Basic Education Act 628/1998 (Amendments up to 1136/2010). http://www.finlex.fi/en/laki/kaannokset/1998/en19980628.pdf

¹⁵

LOV 1998‐07‐17 nr 61: Law on Primary and Secondary Education (The Education Act)

¹⁶

In New Zealand, the guidance for suspensions is based on the Education Act of 1989 and the Education Rules 1999 (Stand‐down, Suspension, Exclusion, and Expulsion)

¹⁷

All data referring to prevalence was extracted from a governmental report informing data from academic years 2015. http://www.educationcounts.govt.nz/__data/assets/pdf_file/0011/147764/SSEE‐Indicator‐Report‐2015‐Data.pdf

¹⁸

In the US, procedures and definitions of school suspension vary among states. Here, we use Washington State as an example. For more details, see www.k12.wa.us/Safetycenter/Discipline/pubdocs/Suspension‐expulsion‐rights.pdf

¹⁹

Data extracted from Office of Super Intendent of Education (OSIP), State of Washington. http://www.k12.wa.us/DataAdmin/PerformanceIndicators/DataAnalytics.aspx#discipline

²⁰

See the specific section for Virginia, p. 10‐16 in https://safesupportivelearning.ed.gov/sites/default/files/disciplinecompendium/School%20Discipline%20Laws%20and%20Regulations%20Compendium.pdf

²¹

See the specific section for Texas, p. 14–27 in https://safesupportivelearning.ed.gov/sites/default/files/discipline‐compendium/School%20Discipline%20Laws%20and%20Regulations%20Compendium.pdf

²²

Data extracted from the Texas Education Agency based on categories which count students once. https://rptsvr1.tea.texas.gov/cgi/sas/broker

The comparative data incorporated in the table above suggests heterogeneity in the application of exclusion. For instance, in the US, Norway and England, educational systems distinguish between fixed and permanent exclusion. However, in some educational systems, such as Finland's, the law only permits fixed‐term exclusion. Concerning length, England limits fixed‐term exclusions to a maximum of 45 days per school year while New Zealand's legislation allows exclusions for a maximum of 10 days per year. On the other hand, it is important to note that in some countries – such as France – specific laws define and regulate exclusion, whereas in others – like Chile and Colombia – the ability to set the length of the sanction is granted to each school.

Although the data on prevalence is limited to a few countries, the percentage of in‐school exclusion seems to be larger than out‐of‐school exclusion and expulsion. In New Zealand, the data suggests that the use of exclusion is marginal particularly when compared with some areas in the US and the UK.

1.2 PREDICTORS AND OUTCOMES

The research concerning predictors and outcomes of school exclusion has some limitations it is necessary to address before arriving at any final conclusions. Regarding predictors, only ethnicity seems to have a clear role in predicting exclusion. For other variables of interest such as sex, age or socio‐economic status most of the evidence is limited to bivariate associations.

Regarding the outcomes, while there is a stark link between misbehaviour (e.g., school drop‐out and delinquency) and school exclusion, there is no clear causal relationship. Notwithstanding decades of research on school exclusion and its impact on later behaviour, we are still at an initial stage for testing causal associations in these matters. The association between exclusion and these negative outcomes may simply reflect underlying behavioural tendencies that lead to conduct problems, exclusion and poor outcomes later in life – that is, the antisocial syndrome described by Farrington (1997). In fact, school exclusion and the behaviours outlined here as “negatives” could be explained by the personality traits of the syndrome.

As stated by Sutherland and Eisner (2014) “it is currently unclear whether the disciplinary action itself has a causal effect over and beyond the social, familial and behavioural characteristics of the affected children. To date, studies have used analytical approaches that are unable to reliably establish a robust link between exclusion and outcomes such as criminal behavior.” Some longitudinal studies have attempted to deal with this matter by controlling for previous behavioural characteristics that could alter the impact of the sanction. When this is the case, the methodological details are explicitly presented in this review.

Keeping these reservations in mind, the following section describes variables associated with the prediction of school exclusion, as well as some negative outcomes linked to exclusion.

1.2.1 Predictors of school exclusion

From a normative point of view, school exclusion is a punitive response for misbehaviour. In that sense, behavioural problems seem to be the most obvious empirical predictor for exclusion. Reinke, Herman, Petras, & Ialongo, (2008) illustrate the role of problem behaviour in exclusion by conducting a latent class analysis. Participants in the subclass of boys exhibiting behavioural problems only (i.e., isolating other academic/learning difficulties) were almost 4 times more likely to be suspended (OR = 3.42; 95%CI 1.36 to 8.58; p < .05) than their non‐problematic peers. Similarly, Pas, Bradshaw, Hershfeldt, & Leaf, (2010) found that after controlling for student, teacher, classroom, and school level covariates, the strongest predictor for out‐of‐school suspension was disruptive behaviour (OR = 4.83; 95%CI 4.10 to 5.68; p < .05).

Despite the role of behaviour in school exclusion, research suggests that it is not the sole or even the most prominent predictor. In fact, previous findings show a more complex scenario where exclusion is also strongly predicted by gender, ethnicity, age, economic background, and special educational needs (Costenbader & Markson, 1998a; Mcloughlin & Noltemeyer, 2010; Monroe, 2005; Nickerson & Spears, 2007; Noltemeyer & Mcloughlin, 2010; Skiba et al., 2011; Yudof, 1975). In the following paragraphs, we offer an overview of the role of these variables in predicting school exclusion.

Gender as a predictor of exclusion

Data provided by the Department for Education in England (DfE) 2011/12 suggests that male pupils are around three times more likely to be punished by exclusion than female pupils (DfE, 2013). The same trend can be observed in the study published by Liu (2013) based on longitudinal data from 13,875 American students. The study reports the predominance of males being excluded, but recognises that the proportion of females excluded tends to increase from elementary (23.7%), to secondary (32.7%), to high school (35.2%). More specifically, Bowman‐Perrott et al. (2013, p. 91) concluded that, based on a sample of 2,597 pupils, the predominance of males in exclusion rates (OR = 2.28) was even larger in the case of pupils with learning disabilities (OR = 4.31). ²

Ethnicity

Research outcomes suggest a clear and consistent disproportionality in the prevalence of ethnic minorities as a target for disciplinary exclusion (Anyon et al., 2014; Gregory, Skiba, & Noguera, 2010). In the US, different sources of data show that school exclusion overly affects minorities such as Afro‐Caribbean (Noltemeyer & Mcloughlin, 2010), Latino (Skiba et al., 2011) and American Indian students (Gregory et al., 2010) in comparison with their White peers. In the UK, data from the (DfE, 2012) showed that: “The rate of exclusions was highest for Travellers of Irish Heritage, Black Caribbean and Gypsy/Roman ethnic groups. Black Caribbean pupils were nearly 4 times more likely to receive a permanent exclusion than the school population as a whole and were twice as likely to receive a fixed period exclusion.” Notably, recent multivariate analysis points out that racial disproportionality in exclusion still remains significant after controlling by behaviour, number and type of school offences, age, gender, teacher's ethnicity, and socio‐economic status (Fabelo et al., 2011; Noltemeyer & Mcloughlin, 2010; Rocque & Paternoster, 2011; Skiba, Michael, Nardo, & Peterson, 2002). Consider, for instance, a substantial longitudinal report produced by Fabelo et al., (2011) in Texas (N=928,940), intended to isolate the effect of race alone on disciplinary actions. The study used a multivariate analysis controlling for 83 different variables. The findings suggest that African‐American students were 31% more likely to be removed from classrooms compared to White and Hispanic students. In the same vein, Skiba (2015) has argued that, in the United States at least, racial disproportionality in school discipline is ubiquitous. In his opinion, ethnic minorities are overrepresented in almost all types of school punishment. Even more worryingly, instances of exclusionary discipline among African Americans have continued to increase over the years.

Possible reasons for this overrepresentation of Black students, even when controlling for demographic and risk factors, have been addresses by some scholars, who suggest that a racist bias could explain the phenomenon (Losen, 2011; Skiba et al., 2002; Skiba, 2015). In particular, Simson (2014) asserts that racial stereotyping (conscious or unconscious) as well as a cultural mismatch between teachers and students can explain at least some part of the existing racial disproportionality in school discipline.

It is important to say that, as stated by Theriot, Craun, & Dupper (2010, p.14), “the over‐representation of ethnic minority students, especially African American students, in school suspension and expulsion is one of the most consistent—and perhaps most controversial—findings in the extant literature on school discipline.” In general, studies using solid and strong multivariate models highlight the discrimination against racial minorities compared to White students.

Age as a predictor of exclusion

The likelihood of being punished by exclusion increases with age, being more frequent during adolescence. In England, 52% of permanent exclusions are imposed on pupils aged between 13 and 14 (DfE, 2013). In the case of American students, the results follow a similar trend. In fact, data reported by Liu, (2013) pointed out that suspensions reach a peak in ninth grade (i.e., 14 to 15 years of age). Also based on a sample of American students, Raush & Skiba, (2004) concluded that the number of out‐of‐school suspensions was significantly higher in secondary schools compared to elementary schools.

Socio‐economic status (SES)

Low SES has also been identified as a predictor of high rates of disciplinary exclusion. The UK Department of Education (DfE, 2012) compared the rates of exclusion by eligibility for free school meals (FSM). Those eligible for FSM were 4 times more likely to be punished by a permanent exclusion and around 3 times more likely to get a fixed‐period exclusion than children who were not eligible. In the US, Nichols, (2004) using a sample of 52 schools (37,000 students), found a similar pattern – but the correlation between FSM and exclusion was higher and more significant for pupils in middle school (r = .84; p <. 01) than for elementary (r = ‐.12) or high school pupils (r = .48). In Australia, Hemphill et al., (2010), using multilevel mixed‐effects logistic regression (N = 8,028 students), concluded that pupils settled in low SES neighbourhoods were exposed to higher rates of exclusion (8.7%) when compared with pupils in high SES areas (2.9%).

However, the evidence still seems to be inconclusive in this respect. Recently, Skiba et al., (2012), using a multilevel approach, tested data from 365 schools and a total number of 43,320 students. They concluded that when comparing those students eligible for free or reduced‐cost lunches with their non‐eligible peers, the first were more likely to get out‐of‐school exclusions (OR= 1.27; p<.05). However, contrary to expectations, the eligibility for free or reduced meals resulted in a negative predictor of permanent exclusion (OR= 0.03; p<.05).

Special educational needs (SEN)

Although an increasing amount of research has focused on predictors of school exclusion, analysis of the role of SEN still seems to be limited. In 2007, Achilles, McLaughlin, and Croninger differentiated the role of three different SEN, namely emotional/behavioural disorders (EBD), attention‐deficit/hyperactivity disorders (ADHD), and learning disabilities (LD). Higher rates of exclusion were more likely among those with EBD (OR = 1.49; p<. 001) compared with ADHD (OR = 2.58; p < .001) or LD (OR = 5.44; p < .001). Recently, Bowman‐Perrott et al. (2013), using three waves from the Special Education Elementary Longitudinal Study (SEELS), confirmed that children with emotional or behavioural disorders (OR = 3.95; p <.05) and attention‐deficit or hyperactivity disorders (OR = 4.96; p <.05) were more likely to get suspended or expelled from school than children with learning disabilities (OR = 2.54; p <.05). In a study involving 2,750 students and 39 American schools, Sullivan, Van Norman, & Klingbeil (2014) also observed differences between types of disabilities: those presenting an EBD were at a far greater risk of exclusion (OR = 6.78; SE=0.21) than those presenting other health impairments (i.e., a specific learning disability, intellectual disability, speech and language impairment). When controlling for race and gender, and parents’ education, this trend remained stable and significant. It is important to emphasize that the associations between this disability and exclusion mainly reflect differences in behaviour, respectively psychological or chronic behavioural problems.

1.2.2 Negative outcomes linked to school exclusion

Supporters of zero tolerance policies have pointed out that the use of exclusion can persuade students to account for their behaviour and lead to a decrease in rule‐breaking (Bear, 2012). However, most of the research has consistently documented the negative impact of these types of sanctions (APA Zero Tolerance Task Force, 2008; Chin, Dowdy, Jimerson, & Rime, 2012; Hemphill, Toumbourou, Herrenkohl, McMorris, & Catalano, 2006; Sharkey &Fenning, 2012). In particular, previous research suggests that school exclusion is related to serious negative outcomes in at least three dimensions of young people's development: behavioural, academic, and future social inclusion.

Behaviour

Some literature related to the relationship between exclusionary punishments and behaviour suggests that such harsh punishments could result in a spiral into more defiant behaviour by students. Raffaele‐Mendez, (2003), for instance, found a moderate and significant correlation (r = .39) between out‐of‐school exclusion (grades 4 to 5) and subsequent exclusion (grade 6). Similarly, Theriot, Craun, & Dupper, (2010) found that pupils punished by in‐school and out‐of‐school exclusion were slightly more likely to get the same punishment again (OR_in‐school = 1.25; p < .001; N = 9706 and OR_{out‐of‐school} = 1.32; p < .001; N = 9706).

Using longitudinal data, Arcia, (2006:366) concluded that school dropout was another behavioural consequence of exclusion. In fact, “43% of students who were suspended 21 or more days dropped out 3 years after their ninth‐grade enrolment.” Similarly, Cratty (2012:649) found a positive correlation between out‐of‐school suspensions and dropout rates. In particular, “those who had an early record of multiple exclusions registered 60% dropout during high school” when compared with non‐excluded students.

The use of exclusion, in turn, is linked with more serious behavioural outcomes such as antisocial conduct, delinquency and entry into the juvenile justice system. Longitudinal research carried out by Hemphill et al. (2006:736) argues that “school suspensions significantly increased antisocial behaviour 12 months later, after holding constant established risk and protective factors (OR = 1.5; 95%CI 1.1 to 2.1; p< .05; N = 3655)”In terms of the involvement of school excludees in the criminal justice system, Costenbader & Markson, (1998) found significant differences between excluded students and those never excluded. In their view, “while 6% of the students who had never been suspended reported having been arrested, on probation, or on parole, 32% of the externally suspended subsample and 14% of the internally suspended subsample responded positively to this question. Males reported significantly more involvement with the legal system than did females.” (p.67). Meanwhile, Challen & Walton, (2004), studying a population of males in the criminal justice system, concluded that more than 80% had been previously excluded from school ³ .

Academic achievements

Evidence suggests that periods of exclusion may have detrimental effects on pupils’ learning outcomes. Exclusion is accompanied by missed academic activities, alienation, and demotivation in relation to academic goals (Brown, 2007; Michail, 2011). In particular, Hemphill et al., (2006) found that excluded pupils were slightly more prone to fail in the academic curriculum when compared with non‐excluded students (OR = 1.3, 95% CI 1.1 to 1.5, p< .01). Along similar lines, Arcia, (2006) produced a longitudinal retrospective study regarding the associations between exclusions and achievements from fourth to seventh grade. After three years, non‐excluded students displayed substantially higher reading achievement scores when compared with their non‐excluded peers. In fact, seventh‐grade students who were excluded for 21 days or more achieved scores similar to fourth‐grade students that had not been excluded. Finally, Raffaele‐Mendez, (2003) added that those excluded were also less likely to graduate from high school on schedule.

Future social inclusion

Some studies have pointed out that young people excluded from school can also register a high risk of becoming “Not in Education, Employment, or Training” (NEET) in the future. In 2007, Brookes, Goodall, & Heady stated that students who had been excluded were 37% more likely to be unemployed during adulthood. Spielhofer et al. (2009) showed that among individuals with long‐term status as NEET, the majority had previous experienced of exclusions and truancy. More precisely, Massey (2011) argued that approximately one out of two excluded children will be NEET within two years of their exclusion.

Research has also illustrated the long‐term implications of exclusion for society as a whole. In economic terms, the cost of excluding children from school places a demand on public resources. Although the literature on this matter is still limited, Brookes et al. (2007) produced a report regarding the costs of permanent exclusion in the United Kingdom. The analysis encompasses an estimation of costs for the individual as well as for the educational, health, social and criminal justice services. Overall the cost, in 2005 prices, of permanently excluding a student was estimated at £63,851 per year to society.

While there is a stark link between the aforementioned negative outcomes and school exclusion, these should not be regarded as causal. Notwithstanding decades of research on school exclusion and its impact on later behaviour, we are still at an initial stage for testing causal associations in these matters. The association between exclusion and these negative outcomes may simply reflect underlying behavioural tendencies that lead to conduct problems, exclusion and poor outcomes later in life – that is, the antisocial syndrome depicted by Farrington (1997). In fact, school exclusion and the behaviours described here as “negative outcomes” could be explained by the same underlying factors or personality traits characterising the syndrome.

Despite the lack of empirical support for a causal association, some criminological theories provide a plausible explanatory framework to understand the connection between punishment and the persistence of deviant behaviour. Labelling theory, for example, suggests that those punished (by exclusion) and labelled as “deviant” may start behaving in ways that conform to their newly formed self‐image: by limiting their interactions with integrated students, for example, and shunning conventional social systems such as school (Krohn, Lopes, & Ward, 2014, p. 179). Likewise, Sherman's defiance theory (1993) elucidates the circumstances in which a punishment can produce more antisocial behaviour, such as defiance, instead of compliance with rules. In his view, punishment can increase the prevalence, incidence or seriousness of future offending when offenders deny responsibility, and when they perceive sanctions as unfair, stigmatising and imposed by an illegitimate authority.

Finally, in addition to all these findings and the rationale around the negative outcomes linked to school exclusion, it is important to mention that, so far, there is no evidence demonstrating that exclusion is effective for improving school discipline (Skiba, 2014). What is more, in the short term, exclusion seems to directly deny students’ right to access education as well as reducing adult supervision for those who are most at risk of further deviant behaviour, or most in need of teachers’ support.

1.3 THE INTERVENTION

1.3.1 School‐based programmes

The prevalence of exclusion and its adverse correlated consequences have caught the attention of policy makers and programme developers. As a result, a range of interventions have been designed and implemented to improve school discipline. In the present review, we include different types of school‐based intervention aimed at reducing school exclusion as a punishment for inappropriate behaviour. These interventions include those targeting individual risk factors or school‐related factors, as well as those using a more comprehensive strategy that includes parents, teachers, school administrators, and the community.

Interventions targeting individual risk factors include, for instance, cognitive‐behavioural approaches such as anger management programmes or skills training for children (e.g., Humphrey & Brooks, 2006). Another type of intervention focusing on student behaviour – or, more precisely, students’ skills for conflict resolution – are restorative justice programmes (e.g., Schellenberg & Parks‐Savage, 2007; Shapiro, Burgoon, Welker, & Clough, 2002)In general, these interventions target motivated children and train them in practical skills to deal with anger, solve conflicts or become more assertive in social relationships. Such interventions are normally organised within a curriculum and implemented during school hours. The curriculum involves a package of group or one‐to‐one sessions using a wide range of techniques such as instruction, modelling, role‐play, feedback, and reinforcement, among others (Gottfredson, Cook, & Na, 2012; Schindler & Yoshikawa, 2012).

At the classroom level, interventions may target teachers’ abilities in classroom management (Pane, Rocco, Miller, & Salmon, 2013). The training for teachers encompasses instructional skills, such as guidelines for teaching rules and maintaining attendance, and non‐instructional skills, such as group management techniques, reinforcing positive conduct, and techniques to explain expected behaviour. Both skill sets are aimed at improving the learning process, preventing misbehaviour and encouraging positive participation by pupils (Averdijk, Eisner, Luciano, Valdebenito, & Obsuth, 2014).

Some schools offer mental health services independently or via community agencies. Experienced clinicians are located in schools in order to deliver individual, group, and/or family therapy. Clinicians may also be available for teacher consultation on matters related to students’ behavioural and emotional issues. All these interventions may target a reduction in out‐of‐school exclusion (Bruns, Moore, Stephan, Pruitt, & Weist, 2005).

Alternatively, comprehensive prevention strategies target students, families, teachers and school managers as well as the community as a whole (Bradshaw, Waasdorp, & Leaf, 2012; Flay & Allred, 2003; Colin Pritchard & Williams, 2001; Snyder et al., 2010). A well‐known comprehensive programme is the School‐Wide Positive Behavioural Interventions and Supports (SWPBIS). The programme aims to provide support for positive conduct by building proactive school‐wide disciplinary procedures (i.e., improving school climate and reducing problem behaviours). SWPBIS incorporates a multi‐level approach: from whole school prevention, to group‐based intervention for problematic pupils, and personalised, tailored interventions for high‐risk students. The basic elements of the programme are: i) building a school culture for both social and academic attainment, ii) early prevention of problem behaviours, iii) teaching social skills to all students, iv) using behaviour support practices, and v) actively using data for decision‐making. Research reports promising results, although further and stronger evaluation designs need to be undertaken (Gottfredson et al., 2012; Maag, 2012).

Previous reviews

In 2013/14 we conducted a systematic search of reviews and meta‐analyses assessing the effectiveness of school‐based programmes for promoting early prevention of risks (Averdijk et al., 2014). The results suggested there had been no previous meta‐analysis aimed at assessing the effectiveness of different types of interventions for reducing disciplinary school exclusion. Probably the most similar study is one published by Burrell, Zirbel, & Allen, (2003) who conducted a meta‐analysis on the effectiveness of mediation programmes in educational settings. Among many other outcomes, the analysis suggested that these interventions had a desirable effect (r = ‐.287, k = 17, N = 5,706, p < .05) on administrative suspensions, expulsions and disciplinary actions. However, in this meta‐analysis suspension was reported along with other disciplinary actions, and the study did not compare mediation with any other intervention (as proposed in the present meta‐analysis). The authors also call for a cautious interpretation given the high heterogeneity of primary results. A similar type of analysis was followed by Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, (2011) and Gottfredson, Wilson, & Najaka, (2002). In both studies, school exclusion was coded as an outcome, but the final meta‐analysis did not report on the impact of the intervention specifically in relation to this targeted outcome.

Likewise, Solomon, Klein, Hintze, Cressey, & Peller, (2012) conducted a meta‐analysis exclusively testing the effectiveness of School‐Wide Positive Behavioural Interventions and Supports (SWPBIS) programme. Despite a small number of included studies reporting data on exclusion, the review does not report effect sizes by measuring their increase/decrease. Rather, the review reports effect sizes on the reduction of office discipline referrals and problematic behaviour.

In addition, two narrative reviews have recently been produced looking at intervention as a means of reducing disciplinary exclusion. Spink, (2011) explored qualitative, quantitative and mixed methods studies. Overall, 10 reports were found. The review concluded that multi‐agency interventions were the most frequent and that they could have a positive effect on reducing exclusion of pupils who are at risk. As expected, the study did not report a meta‐analysis of effect sizes. In 2012, Johnson produced another narrative review identifying programmes that may be an alternative for suspension in school systems. The search strategies were not clear enough to allow replication and, again, the nature of the design does not allow for the calculation of effect sizes.

1.4 WHY IT IS IMPORTANT TO DO THE REVIEW

Despite a growing body of research on the negative side effects of exclusion, no previous meta‐analysis based on a comprehensive systematic review has been conducted to synthesize evidence assessing the impact of school‐based interventions in reducing disciplinary exclusion. The current review addresses this gap by meta‐analysing results from existing published and unpublished studies, providing a statistical assessment of the overall effect of school‐based interventions at reducing exclusion.

This meta‐analytic investigation has clear implications for policy making. The results provided by the present study would produce a much‐needed evidence base for school managers, policymakers and researchers alike. These results can contribute to tackling the adverse developmental, social and economic effects of school exclusion mentioned in the previous pages, as well as potentially identifying alternative and less punitive approaches to school discipline.

2. Objectives

The main goal of the present research is to systematically examine the available evidence for the effectiveness of different types of school‐based interventions for reducing disciplinary school exclusion. Secondary goals include comparing different types of interventions (e.g., school‐wide management, classroom management, restorative justice, cognitive‐behavioural interventions) and identifying those that could potentially demonstrate larger and more significant effects.

We also aim – potentially – to run analysis controlling for characteristics of participants (e.g., age, ethnicity, level of risk); interventions (e.g., theoretical bases, components); implementation (e.g., facilitators’ training, doses, quality), and methodology (e.g., research design).

The research questions underlying this project are as follows:

Do school‐based programmes reduce the use of exclusionary sanctions in schools?
Are some school‐based approaches more effective than others in reducing exclusionary sanctions?
Do participants’ characteristics (e.g., age, gender, ethnicity) affect the impact of school‐based programmes on exclusionary sanctions in schools?
Do characteristics of the interventions, implementation, and methodology affect the impact of school‐based programmes on exclusionary sanctions in schools?

3. Methods

3.1 TITLE REGISTRATION AND REVIEW PROTOCOL

The title of the present review was registered in The Campbell Collaboration Library of Systematic Reviews on January 2015. The final version of the review protocol was approved in November 2015. The title registration and the respective protocol are available at: https://www.campbellcollaboration.org/library/reducing‐school‐exclusion‐school‐based‐interventions.html

3.2 CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW

3.2.1 Research design

Our original proposal was to include both randomised controlled trials and high‐quality quasi‐experimental studies (defined as studies using a comparison group, pre‐post testing and a statistical matching approach). To be eligible for inclusion, we stated that manuscripts must clearly report the method used to ensure equivalence between treatment and control groups, taking into account major risk factors (e.g. behavioural measures) and demographic characteristics. ⁴

In this review, we only present results from randomised controlled trials (RCTs). There were three reasons for our decision:

1.
First, even though a number of quasi‐experimental studies initially fulfilling our inclusion criteria were found by our searches, many of them fail to report baseline measures (e.g., Guardino, 2013; Hawkins, Catalano, Kosterman, Abbott, & Hill, 1999; Munoz, Fischetti, & Prather, 2014), the matching procedures were not described (e.g., Hasson, 2011; Risner, Vanderhaar, Muñoz, & Abati, 2005; St James‐Roberts & Singh, 2001) or the balance procedures did not produce statistical equivalence (e.g., Gao, Hallar, & Hartman, 2014).
2.
A number of the school‐based intervention programmes included in this review presented several studies, involving quasi‐experiments as well as RCTs. Some examples involve interventions such as the Positive Action Program or the School‐Wide Positive Behavioural Interventions and Supports (e.g., Lewis et al., 2013; Snyder et al., 2010). In both cases, RCTs (e.g., Lewis et al., 2013; Snyder et al., 2010) were preceded by quasi‐experimental studies (Barrett, Bradshaw, & Lewis‐Palmer, 2008; Flay & Allred, 2003) . In this context, we decided to keep the strongest study design.
3.
RCTs are regarded to be the most compelling methodological design to test the impact of a particular treatment. This type of study has the strengths of isolating confounding factors, reducing the likelihood of alternative explanations for observed effects (Shadish, Cook, & Campbell, 2002; Sherman, Farrington, Welsh, & Mackenzie, 2002). We believe that by selecting only these studies we will achieve a more precise final estimation of the effect of school‐based interventions.

To offer a broad overview of the research testing the impact of school‐based intervention at reducing school exclusion, a list of the quasi‐experimental studies can be provided on request.

Qualitative studies were excluded from the present review as stated in the published protocol.

3.2.2 Types of participants

The present review is focused on the general population of students in primary and secondary schools irrespective of nationality, ethnicity, language, and cultural or socio‐economical background. By targeting primary and secondary schools, participants could theoretically be aged from 4 to 18 years of age.

Reports involving students who presented special education needs, disabilities or learning problems but were educated in mainstream schools were included in this review. However, reports involving students with serious mental disabilities or those in need of special schools were excluded. The rationale for this is that the results of this review are intended to be generalisable to mainstream populations of students in non‐specialised schools from all the included countries.

Students in college or higher levels of education have been excluded. Their exclusion from the review is based on previous evidence suggesting the largest number of exclusions affect pupils aged about 10 to 15 (e.g., Liu, 2013; Raush & Skiba, 2004; DfE, 2012).

3.2.3 Included interventions

We include interventions defined as school‐based: that is, delivered on school premises, or supported by schools with at least one component implemented in the school setting. In the present review, we include interventions explicitly aimed at preventing/reducing school exclusion or those measuring exclusion as an outcome.

Interventions in the present review cover a wide range of psychosocial strategies for targeting students (e.g., Cook et al., 2014), teachers (e.g., Ialongo, Poduska, Werthamer, & Kellam, 2001), or the whole school (e.g., Bradshaw, Waasdorp, & Leaf, 2012). Types of intervention include, for example, those focused on:

instructing students to identify risky behaviours and expanding their alternatives for responding appropriately to risks or harms (e.g., social skills training)
developing teachers’ skills to improve the quality of their classroom management (e.g., reward schemes)
cognitive‐behavioural treatment, such as anger management, counselling, social work, and mentoring programmes;
school‐wide interventions.

Since there was no previous review analysing school‐based prevention programmes for reducing exclusionary discipline, we wanted to include a wide range of school‐based interventions that could be effective for reducing exclusionary practices.

3.2.4 Excluded interventions

We excluded studies where the intervention was not school‐based or school supported. Even though some of these interventions targeted school students, they were community programmes or purely focused on mental health issues without any connection to schools (e.g., Henderson & Green, 2014; Schwartz, Rhodes, Spencer, & Grossman, 2013; Wiggins et al., 2009).

We also excluded interventions designed for children or adolescents who have committed a crime, that is, specialised interventions aimed at reducing reoffending or reconviction. Although suspended students may commit offences, such specialised interventions were excluded from the present review because they exceed the strategies used by schools to prevent misbehaviour and their levels of complexity make them too specific for a general population of students. School‐based prevention programmes targeting outcomes related only to students’ physical health (e.g., AIDS/ HIV prevention programmes, programmes to develop healthy eating programmes) were also excluded.

3.2.5 Types of outcome measures

Primary outcomes

Eligible studies addressed school exclusion as an outcome. As mentioned in the background section, school suspension or exclusion is defined as an official disciplinary sanction imposed by an authority and consisting of the removal of a child from their normal schooling. This removal happens as a reaction to student behaviour that violates the school rules. We included studies testing fixed or permanent, long‐term or short‐term suspension as well as in‐school and out‐of‐school suspensions.

We excluded studies testing other disciplinary sanctions implemented in schools if they do not share the criteria described above. For instance, we excluded disciplinary sanctions such as loss of privileges, extra work, break/lunch detention, and after‐school detentions. These interventions do not involve exclusion from school or exclusion from regular teaching hours, and as such they are not covered by this review.

Secondary outcomes

For any identified study that reported findings on school exclusion as an outcome, we also coded the effects of the intervention on specific behaviour domains, focusing on internalising (e.g., inhibition, social withdrawal, anxiety or depression) and externalising (e.g., defiant or delinquent behaviours or aggressive behaviours such as bullying)problem behaviour (Achenbach & Edelbrock, 1979; Achenbach, 1978; Farrington, 1989)

By coding secondary outcomes, we aimed to assess the extent to which reductions in problem behaviour are a mediator of treatment effects on school exclusion. Indeed, interventions may affect exclusion in two different ways. The first is by improving behaviour that might otherwise lead to an exclusionary measure. The second possibility is that behaviour stays the same, but that the school develops an alternative strategy to deal with the disciplinary problems.

3.2.6 Included literature

Databases and journals were searched from 1980 onwards with the aim of comprising more contemporary interventions or prevention programmes. Eligible studies included both published and unpublished book chapters, journal articles, government reports, and Doctoral theses. When the same data was published in more than one source (e.g., a book chapter and a journal article) we used all the linked manuscripts but the most complete report or the report measuring suspension was defined as the main source of data (see Section 3.4.5). That way we kept as much information as possible from a specific study but avoided over‐estimation of effect sizes. In cases where it was not clear if the manuscripts referred to the same study, we contacted the main author for further information (e.g., email communication with Barnes, Bauza, & Treiber, 2003; Bradshaw et al., 2012; Lewis, Romi, & Roache, 2012; Snyder et al., 2010; Sprague, Biglan, Rusby, Gau, & Vincent, 2016).

3.3 SEARCH METHODS FOR IDENTIFICATION OF STUDIES

The electronic searches were conducted between September 1 and December 1, 2015. In order to reduce the effect of publication bias, an attempt was made to locate the most complete collection of published and unpublished papers.

3.3.1 Electronic searches

Below we list details of the 27 electronic databases searched. As noted above, these databases included both published (e.g., ISI web of knowledge, PsycINFO) and unpublished reports (e.g., Dissertation Abstracts, EThOS) as well as reports from Latin‐American countries (e.g., Scientific Electronic Library Online – SciELO).

Table 2.

Electronic searches

Databases

1.
Australian Education Index (AEI)
2.
British Education Index (BEI)
3.
The Campbell Collaboration Social, Psychological, Educational and Criminological Trials Register (C2‐SPECTR)
4.
BMJ controlled trials
5.
CBCA Education (Canada)
6.
ClinicalTrial.gov
7.
Criminal Justice Abstracts
8.
Cochrane Central Register of Controlled Trials (CENTRAL)
9.
Database of Abstracts of Reviews of Effects (DARE)
10.
Dissertation Abstracts
11.
Educational Resources Information Centre (ERIC)
12.
EThOS (Beta)
13.
EMBASE
14.
Google
15.
Google Scholar
16.
Index to Theses Database
17.
Institute of Education Sciences ‐ What Works Clearinghouse
18.
ISI Web of Knowledge
19.
MEDLINE
20.
The National Dropout Prevention Centre/Network
21.
The Netherlands National Trial Register (NTR)
22.
Open Grey
23.
Psych INFO
24.
Sociological Abstracts
25.
Social Sciences Citation Index (SSCI)
26.
Scientific Electronic Library Online (SciELO). Electronic database collecting scientific production from developing countries (Spanish and Portuguese)
27.
World Health Organisation International Clinical Trials Registry (WHO ICTRP)

Open in a new tab

For each database, we ran pilot searches including the key terms described in Table 3. Four categories of key words were used, including: i) type of study; ii) type of intervention; iii) population; and iv) outcomes. The pilot searches were useful to adjust the terms, synonyms and wildcards as appropriate. They were also helpful in creating combinations of terms that capture relevant sets of studies in each database.

Table 3.

Key words for searches

Type of study

Interventions

Population

Outcomes

Evaluation

Effectiveness

Intervention

Program

Programme

Programme effectiveness

Impact

Effect

Experimental evaluation

Quasi‐experimental evaluation

RCT

Random evaluation

Efficacy trial

Disciplinary methods

Token economy

Classroom management program/ intervention/ strategies

School management

Early interventions

School support projects

Skills training

Schoolchildren

Pupils

Children

Adolescents

School‐aged children

Student

Youth

Adolescent

Young people

School exclusion

Suspension

Out‐of‐school suspension

In‐school suspension

Out‐of‐school exclusion

In‐school exclusion

Suspended

Expelled

Expulsion

Outdoor suspension

Stand‐down

Exclusionary discipline

Discipline

Open in a new tab

We kept a record with the date of searches, number of reports found, number of reports retrieved, key terms included, synonyms, and wildcards used when appropriate. Further details of electronic searches are presented in Section 13.

3.3.2 Other resources searched

As planned, we contacted key authors requesting information on primary studies that could potentially be integrated in this systematic review and meta‐analysis. We also reviewed reference lists of previous primary studies or reviews related to the intervention/outcomes (e.g., Burrell, Zirbel, & Allen, 2003; Gottfredson, Cook, & Na, 2012; Johnson, 2012; Mytton, DiGuiseppi, Gough, Taylor, & Logan, 2006; Wilson, Tanner‐Smith, Lipsey, Steinka‐Fry, & Morrison, 2011).

3.4 DATA COLLECTION AND ANALYSIS

3.4.1 Selection of studies

Eligible studies met the following criteria:

Reported results of interventions from 1980 onwards
Tested the impact of a school‐based intervention on different types of exclusion (e.g., in‐school, out‐of‐school, expulsion)
Included students from primary and secondary school levels settled in mainstream schools
Based on an experimental design, where participants are randomly allocated treatment or control conditions
Reported statistical results for computed an effect size

3.4.2 Data extraction and management

Data extraction was the responsibility of two researchers (AC & SV). Descriptive data of all studies potentially includable in the meta‐analysis was extracted using the data collection instrument presented in Section 12.2. The instrument facilitated the extraction of the following information:

Bibliographical data (e.g., type of publication, year of publication, name of the publication, main author discipline)
Ethics (e.g. declaration of conflicts of interest, use of informed consent)
Research methods (e.g., type of design, units of randomisation, unit of analysis, variables used for matching)
Sample selection (e.g., methods to select sample, attrition)
Primary outcome coding (e.g., type of exclusion, duration of exclusion)
Secondary outcomes coding (e.g., internalising and externalising behaviours, name of the instrument used to measure the outcome data)
Base‐line measurements (e.g., source of data, quantitative measure of the primary outcome)
Programme delivery (e.g., programme deliverer, training, type of intervention, frequency of the intervention)
Post‐intervention and follow‐up measurement (e.g. official records, surveys)
Data for calculation on effect sizes

The same two researchers extracted data for effect‐size calculations. The process was carried out independently. In general, discrepancies were solved by agreement but when the information reported was contentious, we asked for input from the more senior members of the team (ME & DF). Details on the data extracted from each included report can be found in section 9.1, 9.2 and 9.3 of the present review.

When data for calculation of effect sizes was incomplete we used two different strategies. First, we tried to find more details in other sources (e.g., published protocols or reports). Secondly, the lead researcher or members of the research team were contacted regarding the additional data needed.

Endnote X7 software was used to manage references, citations and documents. Data extracted to characterise studies was inputted in STATA v.13 in order to produce inferential/descriptive statistics. Effect sizes were inputted in Version 3.0 of the Comprehensive Meta‐Analysis software.

3.4.3 Strategy to test inter‐rate reliability

To check code consistency across studies, or inter‐rate reliability, we use Cohen's Kappa coefficient (Cohen, 1960). In the event of two coders making inclusion/exclusion decisions, Cohen's Kappa coefficient computes a standardised index across studies based on cross‐tabulated ratings. The index is given by the difference between the observed percentage of agreement in ratings across studies (P_o ), and the probability of expected agreement due to chance (P_e ), divided by 1‐P_e .

Cohen's Kappa was calculated based on the following formula:

k = \frac{P o - P e}{1 - P e}

(1)

To a good approximation, we calculated the standard deviation of Cohen's Kappa, following the expression (Cohen, 1960):

S D_{k} = \sqrt{\frac{P_{o} (1 - P_{o})}{{(1 - P_{e})}^{2}}}

(2)

3.4.4 Assessment of risk of bias in included studies

We planned to assess the risk of bias in included studies by using two different instruments. In the case of RCTs, we intended to use the Cochrane Collaboration risk of bias tool (Higgins & Green, 2011). To analyse risk of bias of studies involving quasi‐experimental designs we planned to use the ACROBAT‐NRSI, another Cochrane Risk of Bias Assessment Tool for Non‐Randomised Studies (Stern, Higgins, & Reeves, 2014). Both instruments were supposed to assist the research team in evaluating the external validity of the included reports.

At the end of January 2016, we began the assessment with the originally proposed instruments. We soon realised that the instruments and their categories seemed more suited to medical trials than school‐based experiments. Therefore, possible alternatives were explored. In consultation with the coordinating group editor, we selected the EPOC risk of bias tool suggested in the methods section of the Campbell Collaboration website (see the tool in section 12.3). The instrument proposes the following eight criteria for the assessment of quality bias:

Sequence generation
Allocation concealment
Baseline outcome equivalence
Baseline characteristics equivalence
Incomplete outcome data
Blinding of outcome assessment
Protection against contamination
Selective outcome reporting

Each of these domains was judged on a 3‐point scale (i.e., low risk, high risk, unclear risk). EPOC tool provides guidance and examples for each domain that facilitate the decision of assigning low, high or unclear risk. Two members of the team performed the assessment of risk of bias (AS & SV). Assessment of bias was performed independently and the final results represent the agreement of both evaluators.

3.4.5 Criteria for determination of independent findings

Since violations of the assumptions of independence in meta‐analysis would lead to incorrect estimates of the variance for pooled effect sizes (Higgins & Green, 2011b; Romano & Kromrey, 2009),we used some strategies to deal with dependency in the data extracted from primary studies.

First, since we included book chapters, journal articles, government reports, and academic PhD theses, we anticipated the case where the same results would be published in more than one source (e.g., a book chapter and a journal article). The protocol stated that in those cases we would code only one outcome (e.g., the most complete, or the most outdated). In practice, we excluded 11 reports whose results were reported in more than one publication. They are grouped in four cases:

The study did not report enough statistical data for effect size calculation. For instance, Vincent, Sprague, Pavel, Tobin, & Gau, (2015) did not report enough data for meta‐calculation. Although the main author was contacted, we were not able to access more details. For that reason we decided to include Sprague, Biglan, Rusby, Gau, & Vincent, (2016). This latter case used the same data but reported enough information for meta‐analysis. There was no overlap.
RCT data was merged with quasi‐experimental data. For instance, Allen & Philliber, (2001) merged the RCT sample with another sample of students. As a result, the study was not an RCT anymore (For further details see Allen & Philliber 2001, p. 641).
The same results were reported in more than one manuscript. That was the case of Panayiotopoulos & Kerfoot. The results of the study were reported in two papers (2004 and 2007). Results were identical in both publications (same dataset, same analysis and same outcomes). There was no overlap.
The study did not report the outcome measured (i.e., suspension). A case in point is Arter, (2007). The author reported the results of her thesis in a journal. The journal article did not describe the outcome suspension, probably because no effect was found. We included the thesis since it reported all the outcomes measured.

Based on that, those 11 manuscripts were not included in the meta‐analysis. There was no overlap among them, consequently, no dependence in the outcomes was observed.

Secondly, included studies reported multiple time points, for instance, multiple follow‐up measures. The inclusion of multiple follow‐ups would create statistical dependence because the different measures are based on the same subjects (i.e., correlated with each other). We calculated effect sizes separately for those studies reporting short‐term and long‐term follow‐up measures. We also corrected variances estimation (see 3.4.8).

3.4.6 Measures of treatment effect

We use Standardised Means Differences (SMD or Cohen's d) to measure the treatment effects of the school‐based interventions included in the review. The decision to use this specific effect size is based on the fact that most of the included manuscripts report results where measurements are expressed in continuous scales (see section 9.3). The standardised mean effect size for a non‐clustered study is given by

d = \frac{{\bar{X}}_{e} - {\bar{X}}_{c}}{S p},

(3)

where ${\bar{X}}_{e}$ and ${\bar{X}}_{c}$ represent the experimental and control group means, respectively, S_p is the pooled sample standard deviation given by

S_{p} = \sqrt{\frac{(n_{e} - 1) s_{e}^{2} + (n_{c} - 1) s_{c}^{2}}{(n_{e} - 1) + (n_{c} - 1)}}

(4)

where n_e and n_c are the sample size in each group, and $s_{e}^{2}$ and $s_{c}^{2}$ are the experimental and control group standard deviation, respectively.

3.4.7 Issues with the unit of analysis

In the present review, we anticipated the inclusion of primary studies involving individually randomised as well as cluster‐randomised unit (e.g., schools or classrooms). One key issue emerges when meta‐analyses include cluster‐randomised studies: participants nested in the same cluster tend to be more similar to one another (as measured by the intra class correlation – ICC). Furthermore, when units of randomisation are clusters instead of individuals, we need to deal with the fact that the data presents different levels of variation (i.e., within clusters variation, between clusters variation and the total variance). This issue needs to be taken into account when computing effect size estimates. When this correlation is not accounted for, standard errors, confidence intervals and p‐values will tend to be too small. These conditions affect the meta‐analysis in two different ways. Firstly, the primary trial gets a mistakenly large weight. Secondly, the pooled result produces estimated effect sizes with an overly small standard error (Borenstein, Hedges, Higgins, & Rothstein, 2009; Higgins et al., 2011).

For the case of clustered data with dichotomous outcome measures (e.g., odds ratios), we followed the strategy proposed by the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2011), which corrects standard errors of effect sizes. The handbook suggests that the effective sample size in a cluster‐randomised trial can be obtained by dividing the original sample size by the design effect, which is calculated via

1 + (M - 1) \times I C C .

(5)

In this equation, M is the average cluster size (units per cluster) and ICC is the intra‐cluster correlation coefficient. Once we were able to identify the design effect, the squared root of the design effect could be multiplied by the original standard error of the log Odds Ratio. Since ICC is rarely reported in primary studies, we have assumed a value of .05, based on the review of multiple meta‐analyses testing similar populations, produced by Ahn, Myers, & Jin, (2012).

In the case of clustered studies with continuous outcomes (e.g., school level means and standard deviations), we followed the strategy suggested by Hedges (2007) and Spier et al. (2013). Effect sizes were computed using d_T2 assuming equal cluster sample size:

d_{T 2} = [\frac{{\bar{\bar{X}}}_{■}^{E} - {\bar{\bar{X}}}_{■}^{C}}{S_{T}}] \sqrt{1 - \frac{2 (n - 1) ρ}{N - 2}}

(6)

In this equation ${\bar{\bar{X}}}_{■}^{E}$ and ${\bar{\bar{X}}}_{■}^{C}$ represent the overall means of the experimental and control group and S_T is the total sample standard deviation estimated from the pooled sample standard deviation across the experimental and the control group. Rho (is the notation used to represent the intra class correlation. N is the total sample size and the sample size of the clusters is represented by n. Based on the characteristics of our data and following Spier et al., (2013) we assume equal cluster size in our calculations. ⁵ When the clusters have different sizes, we will take a conservative approach, including the smallest cluster size in our calculation.

The variance of the effect size will be calculated by

\begin{array}{c} V {d_{T 2}} = (\frac{N^{E} + N^{C}}{N^{E} N^{C}}) (1 + (n - 1) ρ) + \\ d_{T 2}^{2} (\frac{(N - 2) (1 - ρ) {(1 - ρ)}^{2} + n (N - 2 n) ρ^{2} + 2 (N - 2 n) ρ (1 - ρ)}{2 (N - 2) [(N - 2) - 2 (n - 1) ρ]}) \end{array}

(7)

In this equation, N^E and N^C represent the experimental and control group sample across clusters. As suggested by Higgins & Green, (2011), in the event that the value of ρ is not reported, analysts are advised to assume a reasonable value based in previous studies with similar population (Ahn et al., 2012). As detailed previously, we have assumed a value of (ρ)= .05.

3.4.8 Dealing with missing data

In those reports where key statistical information was missing, we attempted to obtain data from principal investigators. When that was not possible, the study was excluded from calculations of effect sizes. All the studies excluded for this reason have been identified and systematically reported in section 4.1.3.

3.4.9 Time points within a study

In the present review, many of our selected studies involved repeated measures of the outcome exclusion. In fact, manuscripts reported measures of exclusion at baseline and post treatment (e.g., Hawkins, Doueck, & Lishner, 1988) or posttreatment and follow‐up (e.g., Farrell, Meyer, & White, 2001). In these cases, we have calculated a synthesis index or effect size of the difference, representing the change between those different measures (i.e., time points). Consequently, the change in exclusion from the baseline is computed by subtracting the means (X) as follows:

Y_{d i f f} = X_{2} - X_{1}

(8)

One issue arises when pursuing this strategy. Because measures at baseline and posttreatment are positively correlated, the calculation of the variance must be corrected. If we avoid the correction, assuming the two measures to be independent (correlation equals zero), we could be overestimating the variance and underestimating the precision of the difference (Borenstein et al., 2009). For a fair approximation of the value of the variance, we would need to know the correlation between the pre‐ and post‐measures (covariation). However, the covariance is not usually reported in primary research (and this was commonly the case in our set of included studies). Consequently, we proceed to assume a value for that correlation. After checking previous meta‐analysis of similar populations, testing school‐based interventions with estimates for the stability of serious problem behaviours (e.g., Farrington & Ttofi, 2009; Wilson et al., 2011), we concluded that the value of the pre‐post correlation should be assumed to be equal to .75. We then calculated the variance of the difference by using equation 9 below (Borenstein et al., 2009), where V₁ and V₂ represent the variances of the original point estimates and r represents the pre‐post correlation value:

V Y_{d i f f} = V_{1} + V_{2} - 2 r \sqrt{V_{1}} \sqrt{V_{2}} .

(9)

As suggested by Higgins & Green, (2011), we undertook sensitivity analyses to determine whether the overall result of the analysis is robust in the use of imputed correlation coefficient.

3.4.10 Assessment of heterogeneity

We report weighted mean effect sizes, under a random model using 95% confidence intervals and accompanied with graphical representation (i.e., forest plots). For investigating heterogeneity, we use the estimates suggested by Borenstein et al. (2009), specifically; Tau‐squared, Q‐statistic and I² .

Tau‐squared, or the difference between the total variance or variance observed and the within‐studies variance, will be estimated and reported.

The final calculation of the Q‐statistic includes reporting its value, degrees of freedom and p‐values. Significant p‐values provide evidence of heterogeneity in intervention effects.

Bearing in mind that Q can appear distorted when the number of studies meta‐analysed is small (Higgins, Thompson, Deeks, & Altman, 2003), we also report I2. I2 “is the proportion of observed dispersion that is real rather than spurious” (Borenstein et al., 2009). High percentages will be interpreted as an indication of high heterogeneity, meaning that the study‐to‐study dispersion is due to real differences in true effect size and not attributable to random error.

3.4.11 Data synthesis

Since our review has a wide scope, we use the random effect inverse variance weighted models for meta‐analytical calculations. The random effect model is the most appropriate when effect sizes are not homogeneous or consistently coming from a single population (Borenstein et al., 2009). Under a random effects model the variance includes the original (within‐studies) variance plus the between‐studies variance, Tau² .

Following Chandler, Churchill, Higgins, Lasserson, & Tovey, (2013), effect sizes will be coded such that a positive effect will reflect the outcomes favouring the treatment group. To illustrate our analysis, we provide summary forest plots displaying the estimated effect sizes along with their 95% confidence intervals.

3.4.12 Subgroup analysis and investigation of heterogeneity

In the present review, we use moderator analysis involving categorical variables estimating models analogous to ANOVA. Analyses are run under a random‐effect model assuming separate variance components for each group. Meta regression has been run in order to explore heterogeneity.

3.4.13 Outliers

The distribution of SMD effect sizes was examined to determine the presence of outliers. Following Lipsey & Wilson, (2001), outliers were defined as those values which are more than two standard deviations from the overall mean of effect sizes. One outlier was detected (i.e., Collier, 2002) and it was windsorised to the next closest value (Lipsey & Wilson, 2001).

3.4.14 Sensitivity analysis

Since the present meta‐analysis involved a wide range of decisions, we conducted sensitivity analysis to test the robustness of these decisions (Higgins & Green, 2011). Specifically, we ran sensitivity analysis for the pre‐post correlations (i.e., covariance) assumed to be .75. We re‐ran the analysis using a correlation equal .50. As expected there was no change in the effect sizes and no relevant difference in standard errors. We also ran sensitivity analysis testing the impact of the outlier and the impact of the windsorisation (see section 4.7).

As stated in our protocol, we ran sensitivity analysis testing differences between published and unpublished reports.

3.4.15 Duration of follow‐up

Included studies reported multiple time points data, for instance, multiple follow‐up measures. Since the inclusion of multiple follow‐ups would create statistical dependence due to the different measures based on the same subjects (i.e., correlated with each other), we calculate effect sizes separately for those studies reporting short‐term (i.e., post‐treatment) and long‐term (i.e., follow‐up) measures.

4. Results

4.1 DESCRIPTION OF STUDIES

4.1.1 Results of the searches

We attempted to identify and retrieve the body of published and unpublished studies that met our inclusion criteria. Figure 1 shows a PRISMA flow diagram describing the results of our searches.

At the beginning, different combination of terms produced a total of 42,749 references from different electronic databases, of which we kept 1,474 relevant hits.

The 1,474 hits were selected based on screening the title, abstract and key words. We targeted studies broadly defined as evaluations, testing the impact of interventions on school exclusion.

As originally planned, we complemented electronic searches for papers with two extra strategies: i) reviewing lists of references from retrieved manuscripts, and ii) communication with key authors. Based on these steps, an additional 22 manuscripts were added to our sample. The full list of studies and their references were imported into Endnote X7. After removing duplicates, a total of 516 unique manuscripts were saved for further assessment.

Efforts were made to retrieve the full text copies of all 516 selected manuscripts. Notably, a high percentage of them corresponded to unpublished reports (48.6%), mainly PhD theses from American universities and Technical or Governmental reports. In the end, we were able to retrieve almost all full text manuscripts. Only eight studies out of 516 were categorised as “studies awaiting classification” and they are reported in Table 6.

Table 6.

Details of studies awaiting classification

Author	Type of publication	Design	Sample	Outcomes	Intervention
Allen (1981)	Report	Pre‐post design. It is unclear if the study uses random allocation of participants.	12 seventh grade teachers	Disciplinary referrals Corporal punishment School suspensions	Positive approach to discipline (PAD) is a system of classroom management, incorporating counselling, problem‐solving, and time‐out centres.
Forbes (1996)	Thesis	Pre‐post design. It is unclear if the study uses random allocation of participants.	900 students (grades six to eight)	School infractions Out‐of‐school suspension	Social skills training
Foster (2011)	Book	No information	African American boys in elementary classrooms	No information	Social Skills Curriculum
Gaines & Schram (2005)	Conference proceedings	Intervention is given to some students, but not to others. Unclear if it is RCT.	Middle and high school students	School absences Suspension/expulsion Disciplinary actions Attendance Grades	School Probation Officer Programme is aimed at identifying juveniles who may be at‐risk of engaging in delinquent behaviour
Gallegos (1998)	Book	Review of interventions	Unclear	Suspensions Expulsion Dropout school	Not given
Neise (1983)	Thesis	Two treatment groups and one control group. Unclear if they were randomly allocated in conditions	37 middle school students	The Devereaux Adolescent Behaviour Rating Scale (DABRS) The Behaviour Rating Scale (BRS) Seventh hour In‐school suspension Out‐of‐school suspension	Group counselling methods that used interpersonal problem solving strategies versus non‐directive counselling
Norris (2009)	Conference proceedings	Evaluation	Unclear	Suspension Expulsion	Restorative justice
Spillman (1993)	Thesis	Unclear	Ninth grade students	Achievement Motivation Attendance Suspension rate	Interdisciplinary teaming and parent contacts

Open in a new tab

The next round of evaluation was based on reading the information available in abstracts, methods and results sections using the eligibility checklist (see Section 12.1). The checklist included the basic criteria for inclusion in this systematic review. We concluded the second round of evaluation with 471 manuscripts excluded for reasons laid out in section 4.1.3. Thirty‐seven papers presented enough statistical data for inclusion in our meta‐analysis.

4.1.2 Inter‐rater reliability

Two trained researchers (AC & SV) independently assessed the 516 pre‐selected manuscripts for inclusion or exclusion. We calculated Cohen's Kappa for testing inter‐rater reliability (Cohen, 1960; Sim & Wright, 2005). The value of Kappa ranges between 0 and 1, where a value of 1 represents perfect agreement between the two raters and a value of 0 indicates no more agreement than that expected by chance. We obtained a Cohen's Kappa= .76; SD=.81; SE=.036, reflecting a high level of agreement between coders. After calculating the agreement between coders, they went through the papers where they found disagreement. Differences were solved by further analysis and discussion. When needed, a more senior member of the team was consulted.

4.1.3 Excluded studies

Following our published protocol, we excluded a total of 471 manuscripts. Table 4 summarises the reasons for the exclusion of each report. References of the excluded papers are availablein section 7.2.

Table 4.

Synthesis of the reasons for the exclusion of 471 papers

Reason for exclusion	k	%
Outcome measure was absent	52	11.0
Type of intervention	53	11.2
Methodological design	339	72.0
Participants	5	1.1
Time span	5	1.1
Pilot study	1	0.2
Reports based on the same data	11	2.3
Not enough data for meta‐calculations	5	1.1
Total	471	100

Open in a new tab

Outcome. We excluded 52 reports (11%) because they did not present a suitable measure of school exclusion. In some specific cases the primary outcome was not reported (e.g., Gage, Sugai, Lewis, & Brzozowy, 2015; Webster‐Stratton, Rinaldi, & Reid, 2011) or it was reported in a composite measure along with other disciplinary measures which did not involve any type of exclusion from school (e.g., De Blank, 2009; Wright, Offord, John, Duku, & DeWit, 2005). Since it was not possible to isolate our primary outcome we excluded those reports. In other exceptional cases exclusion was measured as a predictor instead of an outcome (e.g., Rosenbaum, 2012).

Type of intervention. Following our protocol, we excluded 53 (11.2%) studies because the tested intervention was not delivered in schools, supported by schools or with at least one component implemented in school settings. Consequently, we excluded community programmes when they had no alliance with a school (.e.g., Henderson & Green, 2014; Wiggins et al., 2009). Alternative schools for high‐risk students (e.g., Rhea, 2010)and intervention in special schools (e.g., Kutash, Duchnowski, Green, & Ferron, 2013) were also excluded. Since our protocol stated that targeted intervention must be an “alternative to school exclusion”, we dismissed studies testing for instance the impact of restorative justice in the context of an in‐school exclusion programme (e.g., Brown‐Kersey, 2011), or conflict resolution in the context of an in‐school exclusion programme (e.g., Devlin, 2006). In both cases, the tested intervention was delivered in addition to exclusion more than being an alternative to it.

Methodological design. 339 (72%) studies were excluded because they did not satisfy the methodological characteristics defined in the protocol. We excluded studies lacking a control group (e.g., McDaniel‐Campbell, 2011; Nocera, Whitbread, & Nocera, 2014) and those studies where the control group was not equivalent in demographics and risk factor variables (e.g., Kilian, Fish, & Maniago, 2006; May, Stokes, Oliver, & McClure, 2015).

During the searches, we kept 28 manuscripts which corresponded to literature reviews (e.g., Fronius, Persson, Guckenburg, Hurley, & Petrosino, 2016; Gonzalez, 2012), systematic reviews (e.g., Wolgemuth, Cobb, & Dugan, 2007) or meta‐analysis (e.g., Noltemeyer & Ward, 2015) related to school exclusion or behavioural problems in schools. These types of manuscripts were initially retained on the understanding that they could be a source for identifying extra primary research reports. All 28 of these studies were excluded in the second round of assessment once we had checked their citation lists.

We identified nine manuscripts evaluating the impact of obligatory use of school uniform on levels of school exclusion (i.e., Draa, 2005; Gentile & Imberman, 2012; Gouge, 2011; Johnson, 2010; Samuels & Bishop, 2003; Shimizu & Peterson, 2000; Stevenson III & Brooks II, 1999; Vaughan, 2001; Washington‐Labat & Ginn, 2003). Even if they represented a particular type of intervention that could be interesting to meta‐analyse and compare, none of these eight interventions presented the research design targeted by our review. For that reason, they were excluded.

Finally, under the method design criteria we excluded a number of qualitative studies (Huston, 1999; Maguire, Macrae, & Milbourne, 2003; Rose, 2008) and case studies (e.g., Navarro, Aguilar, Aguilar, Alcalde, & Marchena, 2007) since none of them contributed with statistical data for meta‐analysis.

Participants. Five reports (1.1%) presented data focused on students with special needs (e.g., Cramer, 1990), or young offenders (e.g., Parkes, 2008). Since these participants were not targeted in our protocol, all these reports were excluded from our analysis.

Time span. Five studies (1.1%) were excluded because they were published before 1980 (i.e., Rogers, 1972)or because they reported the evaluation of an intervention implemented before 1980 (i.e., Feldi, 1980; Fisher, 1980; Herzog, 1980; Safer, Heaton, & Parker, 1981).

Reports based on the same dataset. Finally, we excluded 11 reports (2.3%) because they presented additional results based on the same data reported elsewhere (e.g., Allen & Philliber, 2001; Arter, 2007; Panayiotopoulos & Kerfoot, 2007; Vincent, Sprague, Pavel, Tobin, & Gau, 2015). In these cases, we kept the most complete report and the additional manuscripts assisted in a better understanding of the included research; they were however defined as excluded.

Pilot study. We excluded a single report presenting data from a pilot study (Bonell et al., 2015). In page 12 of the cited report, the author states that the aim of the pilot was “to evaluate feasibility and acceptability of the intervention and trial methods, and not to estimate intervention effects.” For those reasons, even though it was reporting results, the study was excluded. We kept references to the report in the category of ongoing research (see section 4.1.4).

Not enough data for meta‐calculations. Five reports (1.1%) were excluded because they did not present enough data for calculation of effect sizes (e.g., Grinage, 2005).

4.1.4 On‐going studies

We identified the protocol of four ongoing studies whose outcomes had not been published at the end of the searches in December 2015. As observed in Table 5, all of them are cluster randomised control trials testing the impact of school‐based interventions and measuring school exclusion as an outcome.

Table 5.

On‐going studies

Author	Design	Sample	Outcome of interest	Intervention
Acosta (2015)	Cluster‐RCT	Unclear (US)	Suspension or expulsion	The Restorative Practices Intervention (RPI)
Bonell et al. (2014)	Cluster‐RCT	40 schools (UK)	Temporary and permanent school exclusion	INCLUSIVE (combines changes to the school environment, promotion of social and emotional skills and restorative practices)
Philliber (2015)	Cluster‐RCT	6 schools (US)	In‐ and out‐of‐school suspension	School‐wide positive behavioural interventions and supports (SWPBIS)
Eiraldi (2014)	Cluster‐RCT	12 school (US)	School suspension	Teen Outreach Programme in Kansas City (replication)

Open in a new tab

4.1.5 Studies awaiting classification

We were unable to classify eight studies. We selected them based on title, abstract and key words but we have not been able to locate the full text copies. A list of studies that could be potentially included in a future updated version of this review is given in Table 6.

4.1.6 Included studies

Thirty‐seven studies reporting 38 interventions’ effect sizes were included in this meta‐analysis. As we mention before, all of them were randomised controlled trials. In 23 studies the control group received no‐treatment (62.2%); six studies reported controls receiving intervention or business as usual (16.2%), four experiments offered a placebo to the control group (10.8%) and four studies allocated controls in a waiting list (10.8%).

Following sections provide a general overview of 37 included studies. Data is organised by characteristics of the included studies, characteristics of participants and characteristics of the delivered interventions.

Characteristics of the included studies

We included studies presenting interventions (M_date =2003; SD=9.5) carried out and reported between 1980 and December 1, 2015 when we finished our searches. Exceptionally the review involves three manuscripts published in 2016. In the first case, Obsuth et al., (2016), the registered protocol of the study had been identified in our electronic searches and we were waiting for the published version, which was released in March 2016. The second case involves two different papers: Okonofua, Pauneskua, & Walton, (2016) and Sprague, Biglan, Rusby, Gau, & Vincent, (2016). Both studies were sent to us from key authors in the field. As they were recent studies, matching our inclusion criteria, we decided to retain them in our analysis. No other study was sent to us after the end of the searches in December 2015. Figure 10.1 (in appendix) presents the distribution of studies per year. Results show that the number of RCTs has increased over the specific years involved in this report.

As shown in Table 7, we included published and unpublished reports in almost equal proportion (51% published versus 49% of unpublished reports). All of them were written in English, and represent studies implemented in the United States and the United Kingdom only. Although we explored global databases, in particular databases from Latin‐American countries we were not able to find studies conducted in other locations.

Table 7.

Characteristics of studies included in meta‐analysis

Study characteristics
Publication year (range)	1980 ‐2016

Type of publication	n	%
Journal articles (published)	19	51.3
PhD thesis, technical report (unpublished)	18	48.6

Language	n	%
English	37	100

Country of the sample	n	%
United States	33	89.1
United Kingdom	3	8.1
Unclear	1	2.7

Author's main discipline	n	%
Education	13	35.3
Social work	1	2.7
Psychology	12	32.3
Criminal justice	1	2.7
Psychiatry/medicine	6	16.2
Econometrist‐economics	2	5.4
Not reported	2	5.4

Declared conflict of Interest in published studies (N=19)	n	%
Yes	7	36.8
No	12	63.2

Conflict of financial Interest	n	%
Unlikely	18	48.6
Possibly	13	35.1
Likely	2	5.4
Not enough information	4	10.8

Unit of randomisation	n	%
Individuals	26	70.3
School clusters	8	21.6
Classroom clusters	3	8.1

Statistical analysis	n	%
Multilevel modelling	4	10.8
Differences in means (MANOVA, X² , ANOVA, ANCOVA)	22	59.4
Regression	8	21.6
Frequencies	3	8.1

Exclusion measurement	n	%
Self‐report	2	5.4
Teacher‐report	3	8.1
School records (official records)	30	81.1
Unknown	2	5.4

Effective²⁸ sample size	n	%
< 300	24	64.8
Between 300 and 800	7	18.9
> 900	6	16.2

Open in a new tab

We coded data on the main author's discipline. Our data shows that more than 60% of studies testing interventions intended to reduce exclusion have been carried out by researchers in the fields of Education and Psychology.

Interestingly, only seven studies (36.8%) of experimental evaluations published in peer‐reviewed journals disclosed a personal or organisational Conflict of Interest (CoI). This percentage is coherent with the findings of Eisner, Humphreys, Wilson, & Gardner, (2015) who found limited attention to full CoI disclosure in the evaluation of psychosocial interventions. They argue that even if “transparency about CoI in itself does not necessarily improve the quality of research, and researchers with a CoI should not be presumed to conduct less valid scholarship, transparency is needed for readers, to assess the study findings and their particular context” (Eisner et al., 2015, p. 10).

In addition to the presence/absence of CoI statements, we evaluated studies on their potential conflict of financial interest (CoFI) by using a scale developed by Eisner & Humphreys, (2012). It is a trichotomous scale that helps to identify three levels of conflict. The three categories in the scale are defined as follows: i) Unlikely conflict of interest: none of the study authors are programme developers or licence holders; ii) Possible: a study author is a programme developer or collaborator with a programme developer AND the programme is not (yet) commercially available OR the business model is ‘not‐for‐profit’; or iii) Likely: study author, is a programme developer or collaborator with a programme developer AND programme is commercially available AND business model is ‘for‐profit’. For details on the instrument see section 12.4.

We found 18 studies (48.6%) where the CoFI was defined as ‘unlikely’. Essentially, in this set of studies, none of the programme evaluators were involved (i.e., directly or as a collaborator) in the development of the intervention or were licence holders. However, we found 13 studies (35.1%) where we assessed a ‘possible’ CoFI. In those cases, the evaluator was a programme developer/deliverer or a previous collaborator with a programme developer; the programme was not commercially available, or the business model was defined as ‘not‐for‐profit’. Finally, only two studies (5.4%) in our evaluation transparently declared information that allowed us to classify them as ‘likely’ to present a potential financial conflict of interest. In both cases the authors reported that one of the members of the evaluation team was related to the holder of a licensed programme evaluated. The evaluation of each study is reported in section 9.2.

In terms of methodological design, all included studies were randomised controlled trials. While 70.3% of our studies randomised individuals, almost 30% (11 studies) randomised clusters of students, that is, entire schools (i.e., Bradshaw et al., 2012; Cornell, Allen, & Fan, 2012; Lewis et al., 2013; Obsuth et al., 2016; Snyder et al., 2010; Sprague et al., 2016; Ward & Gersten, 2013; Wyman et al., 2010) or classrooms within schools (i.e., Brett, 1993; Farrell, Meyer, & White, 2010; Ialongo, Poduska, Werthamer, & Kellam, 2001). Where necessary, we corrected data in clusters in order to combine it with individual level data. For further details on corrections, see section 9.3.

The most frequent analysis of the data was differences in means such as chi‐squared, ANOVA, ANCOVA or MANOVA (59.4%) followed by regression (21.6%), and with a minority of the studies running Multi‐level analysis (10.8%).

Additionally, measures of disciplinary exclusion were mainly based on official records (81%) provided by schools or other official institutions. Table 7 offers a general description of studies’ sample size. The average sample size was M_size=1,168 (SD=3107.3) participants. But this average should be cautiously interpreted since the included studies range widely from 20 to 13,498 participants. This is an important issue for the statistical power of the calculated impacts.

Characteristics of participants in included studies

As observed in Table 8, in the present meta‐analysis, sampled students most frequently come from the higher school grades (i.e., middle and high schools), with a M_age=12.9; SD= 2.8.

Table 8.

Characteristics of participants included in meta‐analysis

*Population's characteristics*
Study average age	mean 12.9	standard deviation 2.8
Grade	frequency	%
Elementary school	11	29.7
Middle school	16	43.2
High school	8	21.6
Mixture of levels	2	5.4

Ethnicity	mean	standard deviation
% Black or Afro‐American	54.1	37.2
% White	24.6	27.9
% Latino	20.2	25.5

Free school meals	mean	standard deviation
	66.2	23.9

Open in a new tab

Note. Means and standard deviations are calculated at study level.

In addition, data shows that students were nested in schools with a high percentage of Black and Latino students and, subsequently, with a low percentage of White‐Caucasian peers (24.6%). In fact, five of the included studies are based in schools where a 100% of the pupils were Black (i.e., Barnes, Bauza, & Treiber, 2003; Brett, 1993; Collier, 2002; Mack, 2001; Reese, Murphy, & Filipczak, 1981).

As stated by Hobbs & Vignoles, (2010) and Sutherland, Ilie, & Vignoles, (2015), in this report the percentage of students with access to free school meals was understood as an index of the vulnerability of the population. Across our selected studies the pupils receiving meals funded by the school was predominant (M_fsm =66.2; SD=23.9%).

Intervention characteristics: dosage, delivery and targeted change

The present review includes a wide range of school‐based interventions covering, for instance, strategies such as counselling, mentoring, community services, classroom management, after‐school academic support, and school‐wide strategies, among others. Taking a broad view, 27 % of the interventions were focused on a change at the level of the school or teacher, and a 73% anticipating that a change in pupils’ skills/behaviours could affect suspension/exclusion rates.

As shown in Table 9, school staff (32.4%) or school staff assisted by external facilitators (24.3%) delivered most of the interventions. This means that schools interested in reducing exclusion typically put resources into the intervention process. Most of those delivering the interventions were school psychologists/counsellors (32.2%) and, in two particular cases (5.4%), the intervention was delivered by a police or probation officers, which seems to be an increasing trend in the United States (e.g., Gottfredson et al., 2012; Hirschfield, 2008).

Table 9.

Characteristics of interventions included in meta‐analysis

Intervention characteristics	Frequency	%
At school level	10	27.0
At students’ level	27	73.0

Programme delivery
External facilitators	9	24.3
School facilitators	12	32.4
School facilitators plus external facilitators	12	32.4
Missing	4	10.8

Who delivered programme?
Social worker	2	5.4
Psychologist/counsellor	12	32.2
Teacher	9	24.3
Police/probation officer	2	5.4
Trained community agent	7	18.9
Not clearly stated	5	13.5

Role of the evaluator
Deliver the programme	4	10.8
Design the programme	8	21.6
Delivery and design	3	7.8
Independent	18	48.6
Unknown	4	10.8

Is the programme curricular?
Yes	25	67.6
No	12	32.4

Duration in weeks
Less than 12 weeks	14	37.8
Between 13 and 24 weeks	4	10.8
More than 24 weeks	14	37.8
Unknown	5	13.5

Duration of intervention	m	sd
Hours per week (n=25)	1.78	2.09
Number of weeks	20.4	11.5

Open in a new tab

Note. For specific details regarding each included study, see section 9.1 in appendix.

We coded data on the role of the evaluator. We identified four potential approaches taken by evaluators; i) delivering the programme, meaning that the researcher implemented the intervention as well as acting as evaluator (e.g., Harding, 2011); ii) designing the programme, meaning that the evaluator defined or took part in the defining the theoretical base, aims and activities of the intervention (e.g., Cornell et al., 2012), iii) both, designing and delivering the programme (Mack, 2001) and iv) independent evaluation, referring to those researchers or research teams where any member was involved in any stage of the designed or delivery (e.g., Bradshaw et al., 2012; Dynarski et al., 2003). A high percentage of interventions were designed and/or delivered by the same researcher who evaluated the impact of the intervention. Independent evaluators conducted 48.6% of the RCTs included in this review.

A high percentage of the interventions were curricular (67.6%). For instance, social skills trainings or anger management training were based on a pre‐designed curriculum with detailed aims and activities for each session. In the present review, non‐curricular interventions are those targeting school‐wide change or those focused on counselling and individual therapies.

On average, included interventions lasted M=20; SD=11.5 weeks. 37.8% of the interventions lasted less than 12 weeks and an equal percentage were delivered over more than 24 weeks. In general, school‐wide interventions lasted longer in our sample (i.e., 35 weeks or more).

4.1.7 Description of the interventions

Table 10 provides a description of the included school‐based programmes targeting school exclusion as a primary or secondary outcome. In some specific cases this grouping could be restrictive, because some of the interventions involve multiple components, however we have attempted to create an exhaustive list of categories.

Table 10.

Types of intervention programmes

Programme	Number of effect sizes	Number of studies	% of studies
Enhancement of academic skills	2	2	5.4
After‐school programmes	2	2	5.4
Mentoring/monitoring	5	5	13.5
Skills training for students	9	9	24.3
Skills training for teachers	4	3	8.1
School‐wide strategies	6	6	16.2
Violence reduction	3	3	8.1
Counselling, mental health	3	3	8.1
Other	4	4	10.8
Total	38	37	100

Open in a new tab

Enhancement of academic skills. We found two effect sizes targeting the enhancement of academic skills as a strategy in order to improve academic performance, increase motivation and promote more adaptive behaviour. Edmunds et al., (2012) tested an intervention to boost the academic progress of students in order to facilitate their future access to college while in the case of Cook et al., (2014), the intervention involved academic remediation plus social skills training.

After‐school programmes. Two effect sizes come from interventions that offered students after‐school activities. While the intervention tested by Dynarski et al., (2003) was more focused on academic support and recreational activities, Hirsch, Hedges, Stawicki, & Mekinda, (2011) tested an after‐school programme offering students paid apprenticeships.

Mentoring/monitoring programmes. Five effect sizes reported the impact of interventions focused on mentoring/monitoring. These programmes involved structured and supportive relationships between a young person who presents academic, emotional or behavioural difficulties and a non‐parental adult, their mentor. Mentoring entails a volunteer member of the community serving as a role model and providing support to a younger person over an extended period of time (e.g., Brett, 1993; Wyman et al., 2010). In other cases, adults served as tutors for the students, supervising their performance, providing advice or counselling, and assisting the students with academic tasks (Johnson, 1983; Peck, 2006; Reese et al., 1981). Such tutors were normally schoolteachers or school counsellors.

Social skills training for students. We found nine effect sizes representing the impact of social skills training. These programmes were based on social learning and cognitive behavioural theories (e.g., Burcham, 2002; Collier, 2002; Harding, 2011; Hostetler & Vondracek, 1995; Shetgiri, Kataoka, Lin, & Flores, 2011; Smith, 2004) and their goal is to enhance individuals’ socio‐cognitive, socio‐emotional, and behavioural skills in order to regulate maladaptive conducts. Social skills training programmes typically consist of a curriculum with focused training modules. Some more specific programmes target communication skills (e.g., Obsuth et al., 2016) or approaches to reducing stress (e.g., Barnes, Bauza, & Treiber, 2003). Group‐based sessions and occasionally one‐to‐one sessions (e.g., Russell, 2007) offer the opportunity to implement specific techniques (e.g., instruction, modelling, role‐playing, feedback and reinforcement) in a “real‐world environment”.

Skills training for teachers. We found four independent interventions targeting teachers’ skills. The training provides preventive strategies and techniques that help to maintain classroom discipline, create a supportive educational environment, and enhance students’ positive behaviour. These involve training in facilitating mutual respect between teacher and student (e.g., Okonofua, Pauneskua, & Walton, 2016) as well as training to establish clear classroom rules (e.g., Hawkins, Doueck, & Lishner, 1988). Skills for teachers also involve strategies for working in an alliance with parents to promote students’ engagement to the school activities (e.g., Ialongo et al., 2001).

School‐wide interventions. Six effect sizes represent comprehensive interventions targeting systemic changes across the whole school. They involve pupils, teachers, parents, and sometimes also the community where the school is based. These programmes aim to create positive environments, with clear rules that promote good behaviour, learning and safety. School‐wide interventions are capable of addressing the needs of schools as institutions as well as the particular needs of individual school children. We found six studies testing school‐wide strategies: Bradshaw et al., (2012); Cornell et al., (2012); Lewis et al., (2013); Snyder et al., (2010); Sprague et al., (2016); Ward & Gersten, (2013).

Violence reduction. We included three effect sizes measuring the impact of violence reduction programmes. Although these interventions could be classified as skills training, we have isolated them, because they are specifically targeted at increasing self‐control and reducing violence (e.g., Feindler, Marriott, & Iwata, 1984; Mack, 2001). We also included anger management programmes encouraging peaceful responses to conflict (e.g., Farrell et al., 2010).

Counselling and mental health interventions. We included three effect sizes primarily focused on the provision of counselling in schools (e.g., Berlanga, 2004; Tilghman, 1988)and on a more specialised provision from community mental health services (i.e., Panayiotopoulos & Kerfoot, 2004).

Other interventions. Four effect sizes were classified in this general category. They encompass a community services programme (Allen, Philliber, & Herrling, 1997), a multicomponent programme (Arter, 2005), a career awareness intervention (Bragdon, 2010), and a programme focused on character‐building education, promoting civic behaviour and national values (Crowder, 2001).

4.2 RISK OF BIAS IN INCLUDED STUDIES

The methodological quality of each publication included in the review was evaluated using the EPOC risk of bias tool (see section 12.3). The instrument evaluates the internal validity of reported results. Two coders (SV & AS) independently applied the EPOC tool to each study at different locations. The following results represent the agreed rating of both coders. ⁷ Below, we report the results for each of the eight criteria involved in EPOC. Figure 2 shows a summary of the overall result, and Table 9.2 (section 9.2 in appendix), offers a detailed evaluation of each study.

Risk of bias in included studies based on EPOC risk of bias tool

*Note. Each of the eight evaluated criteria have been assigned one of three possible alternatives: high risk= HR; low risk=LR and unclear risk= UR, as expressed in the first column underneath the graph*.

Adequate sequence generation

Adequate sequence generation is intended to produce comparable groups in an experimental evaluation. Accurate methods used to generate the allocation are, for instance, the use of random number tables or a computer random number generator (Higgins & Green, 2011). Regardless of the selected methods, it should be clearly described in enough detail to allow an assessment of whether the sequence generation was adequate.

As shown in Figure 2, only three studies (8%) presented a high risk of bias with respect to allocation sequence. A case in point:Allen, Philliber, Herrling, & Kuperminc, (1997) used three different methods of randomisation (i.e., picking names from a hat, coin toss and using an alphabetical list of names) as well as running randomisation at different levels in the same study (i.e., individuals and classrooms). In our opinion this mixture of methods could lead to a high risk of bias in the final results: in particular, the use of a list of names is known not to be appropriate for allocating cases. In the case of Barnes, Bauza, & Treiber, (2003:2), the author mentions that “each school was alternately assigned to either TM or health education control.” Since the study involves only two schools, allocation would be predictable and at a high risk of bias.

Although we identify a low percentage of high‐risk cases, it is important to note that a large proportion of studies (59%) were categorised as “unclear risk”. This is because many reports presented succinct descriptions of the randomisation process without detailing the methods of sequence generation (e.g., Arter, 2005; Brett, 1993; Feindler, Marriott, & Iwata, 1984; Reese, Murphy, & Filipczak, 1981). 32% of manuscripts were defined as presenting a low risk of bias (e.g., Bradshaw et al., 2012; Hirsch et al., 2011; Lewis et al., 2013).

Allocation concealment

Correct allocation concealment safeguards a rigorous implementation of the randomisation process by not allowing researchers or participants prior knowledge of the results of assignment (i.e., by using sealed envelopes or other procedure that prevent knowledge about the condition that the participant is going to be allocated). To achieve this aim it has been suggested that allocation should be centralised and executed at the beginning of the study (Schulz & Grimes, 2002).

In the present meta‐analysis, only 5% of the studies were evaluated as having a high risk of bias for allocation concealment. Some examples of high‐risk studies are those where reports suggested that schoolteachers instead of researchers performed the random allocation (e.g., Crowder, 2001) or that the randomisation was performed after the participants’ screening (i.e., Mack, 2001). Even if we cannot be sure that the results of the screening biased the randomisation, this conduct could potentially interfere with the chances of participants being placed in the control or treatment group.

Once again, we found a high percentage of studies (54%) reporting minimal details of allocation concealment. In concrete terms, 20 out of 37 included reports were classified as “unclear risk” of bias. The remaining 41% (i.e. 15 studies) were classified as having a low risk of bias.

Baseline equivalence in the outcome measured

A key element of a randomised controlled trial is that it ensures, in theory, that participants (and their associated outcomes and characteristics) are distributed by chance in the control and treatment groups. Pre‐existing baseline differences between groups – particularly of outcomes – could suggest problems in randomisation, hence it is a key focus when assessing risk of bias (Shadish et al., 2002). It is important to mention that we did not focus on whether there were statistically significant differences between groups, as that is a function of sample size; instead we reviewed means and distributions.

We found seven studies (19%) whose description of the baseline equivalence suggested a high‐risk. This was found, for instance, in cases where the control group presented higher levels of exclusion than the treatment group and there was not clear mention of adjustment (e.g., Panayiotopoulos & Kerfoot, 2004). Another example concerns the imbalance of one specific type of exclusion: specifically, expulsion and out‐of‐school exclusion were equivalent in the treatment and the control groups, but display substantial imbalance in the case of in‐school exclusion (e.g., Berlanga, 2004).

Again, it is important to mention that nearly 30% of assessed studies reported limited information, and for that reason they were assigned an unclear risk of bias. The remaining 49% of our included randomised controlled trials were evaluated as low risk of bias.

Baseline equivalence in other participants’ characteristics

Regarding this criterion, the instrument considers the balance in demographics or any other behavioural outcomes, which again should be equivalent in the treatment and the control group if randomisation has been successful. ⁸ We found 11 studies (30%) displaying non‐equivalent results between treatment and control groups. Studies reporting imbalance in gender (e.g., Berlanga, 2004), ethnicity (e.g., Hostetler & Fisher, 1997), initial levels of problem behaviour, self‐control or size of the school (e.g., Harding, 2011)were all classified as a potentially having a high risk of bias.

A further 22% (eight studies) did not report enough data for judgement (i.e., unclear risk) and around half (49%) of the studies were assessed as presenting a low risk of bias.

Addressing incomplete outcome data

The fact that our included studies pursued more than one measure across time makes it likely that attrition or other forms of missingnes affected sample sizes.

We assess a high risk of bias when i) substantial attrition was present in the study and the researchers did not mention a strategy to deal with that issue (e.g., Peck, 2006); ii) when they used list‐wise deletion (e.g., Russell, 2007), and iii) in those cases where the attrition affected the treatment or control group in an unequal proportion of missing cases across arms (e.g. Hostetler & Fisher, 1997). Using these criteria, we judged that eight studies (22%) presented a high risk of bias.

As in previous cases, a high number of studies (30%) did not report enough data to be judged. Seventeen studies (46%) were evaluated as presenting a low risk of bias. Low risk cases involved i) those studies reporting zero attrition; ii) studies where attrition was represented by a low percentage of cases; iii) when missingnes was equivalent in the treatment and the control group, and iv) when the researcher reported attrition, analysed it and used methods to deal with attrition (e.g., multiple imputation, full information maximum likelihood or intention to treat rather than assessing the effect of treatment on the treated).

Blinding of outcome assessment

This criterion covers bias arising from the fact that those collecting outcome data are aware of the condition assigned to each participant (e.g. individuals, classrooms, schools). In our evaluation, 35% or 13 of the coded studies were assessed as having a high risk of bias. We assigned a high risk of bias when those in charge of delivering the intervention were also collecting data (e.g., Harding, 2011) or when teachers rated the students’ behaviour while being aware of the allocated condition (e.g., Tilghman, 1988). Although the outcome was in the majority of studies, based on official records, we did not assume this meant blind assessment. In most cases, teachers or school staff who imposed exclusion were not necessarily blind to the allocated condition of participants. In any case, it must be said that in school‐based experiments blinding is likely infeasible. Most of the studies require at least a minimal participation of school staff and allocated condition is mostly evident for participants (Hutchison & Styles, 2010).

As in previous cases, the number of reports lacking the data to evaluate blindness of the outcome assessment (i.e., unclear risk) was high (54%) and only four studies (11%) were assessed as low risk bias.

Protection against contamination

Protection against contamination refers to the measures taken to avoid a spill‐over of the treatment into the control group. Specifically, contamination is the risk that the control group might accidentally receive the intervention, but also the risk that the two groups could influence each other's outcomes. If the groups cannot remain isolated during the experimental study, contamination would threaten the validity of the results. In our meta‐analysis 76% of the studies presented a high risk of contamination. The main explanation for high risk of bias was the fact that schools in the trials host both the treatment and the control participants (e.g., Cook et al., 2014; Smith, 2004).

One single study (3%) did not report enough data for evaluation and it was categorised as unclear risk of bias. Eight studies (22%) were classified as low risk of bias. Normally low risk studies were cluster‐randomised experiments where control and treatment participants were in different schools (e.g., Bradshaw et al., 2012; Snyder et al., 2010).

Selective outcome reporting

Selective outcome reporting occurs when there is a difference between the proposed outcomes for evaluation and those finally reported. In our evaluation, four studies (11%) presented a high risk of selective outcome reporting (e.g., Panayiotopoulos & Kerfoot, 2004; Russell, 2007). In the case of thesis or trials without published protocols, the assessment was only based on discrepancies between outcomes proposed and reported in those documents ⁹ . Thirty‐two studies (86%) displayed a low risk of bias and only 3% did not report enough data for judgement.

Summary

Overall, our assessment demonstrates that a high number of the included studies lacked enough information to use the EPOC tool to judge risk of bias in all areas. Specifically, 22 studies were not clear on how they allocated units to treatment and control groups. Similarly, blinding to allocation was another area infrequently reported with enough detail to allow assessment (i.e., in the case of 21 studies). These points aside, contamination or spill‐over is probably the main threat to the validity of results among the RCTs included in our review.

Arguably the seven methodological strongest studies were Bradshaw et al., (2012), Cook et al., (2014), Hirsch et al., (2011), Johnson, (1983), Lewis et al., (2013), Obsuth et al., (2016) and Wyman et al., (2010). All of them presented low risk in the randomisation process and most of them are clustered studies. In particular, these studies achieved low risk of bias in six or more of the eight EPOC criteria.

4.3 SYNTHESIS OF RESULTS

4.3.1 Primary outcome: overall impact of school‐based intervention

The present analysis incorporates 38 effect sizes across 37 studies producing enough statistical information for meta‐analysis. These studies represent a total sample of 31,273 students (M_age =12.5; SD=2.85) partaking in completed trials as treatment, control or placebo groups.

On average, school exclusion was significantly reduced in the experimental group compared with the control group, post‐treatment (i.e.,sixmonths on average). Under a random effects model, the standardised mean reduction was SMD=.30; (95% CI .20 to .41; p<.001). Figure 3 shows that results are positive and statistically significant, meaning that those participating in school‐based interventions were less likely to be excluded than those in the control group. Results exhibit significant heterogeneity (Q=301.3; df= 37; p<.001; I² = 87.7; τ² =.078) which was expected in this meta‐analysis bearing in mind that we include different school‐based programmes, administering different “doses of treatment” with participants from different locations and in different school grades.

Forest plot of the effect sizes for the impact of school‐based programmes on school exclusion

As suggested by Lipsey and Wilson (2001, p. 153) the U3 statistic for a SMD= .30 is 62%. This would indicate that 62% of the treatment group is above the median of the control group. Stated differently, if we assume a 50/50 success rate for both groups, the treatment group sees 62% success, versus 38% in the control group.

Long‐term effects

When we isolated studies measuring impact at follow‐up (i.e., 12 or more months after finishing the intervention), benefits of the interventions were less clear. In fact, Figure 4 demonstrate that the effect was reduced by half (SMD=.15; 95%CI ‐.06 to .35) and it was shown to be non‐statistically significant.

To increase precision in our results, we ran a meta‐analysis with a subset of studies reporting both post‐treatment and follow‐up measures. Only seven studies reported short and long‐term effect measures. As observed, the overall effect size at post‐treatment (Figure 5) and follow‐up (Figure 6) are lower than the ones initially reported but they follow the same direction.

Forest plot of the effect sizes for the impact of school‐based programmes on school exclusion: post‐treatment (seven studies only)

Forest plot of the effect sizes for the impact of school‐based programmes on school exclusion: follow‐up (seven studies only)

At post‐treatment, under a random effect model, the standardised mean reduction was SMD=.21 (95%CI .11 to .30). However, when we ran the meta‐analysis including only the subset of seven studies, the average time involved under “post treatment” was 12 months. It implies that although the overall effect is slightly lower than the general measure reported in Figure 3, the impact lasts longer (i.e., 12 instead of 6 months on average). In the case of the effect at follow‐up, the subset of studies produced and overall impact that was almost null (SMD=.054; 95%CI ‐.04 to .15) and non‐significant. As shown in Table 11, heterogeneity was highly reduced when the analysis with the subset of studies was carried out.

Table 11.

Summary of overall effect at post‐treatment and follow‐up (seven studies only)

Impact	SMD	95% CI	n	k	Measure of Heterogeneity
Post‐treatment	.21	(.11; .30)	7	7	Q=6.54;df=6;p>.05;I ²=8.2;τ² =.001
Follow‐up	.054	(‐.04; .15)	7	7	Q=7.80;df=6;p>.05;I² =23;τ² =.004

Open in a new tab

4.3.2 Moderator analysis by type of exclusion

As proposed, we ran an analysis for the different outcome measures: in‐school exclusion, out‐of‐school exclusion, expulsion, and general exclusion. Results of the four independent meta‐analysesare summarised below in Table 12.

Table 12.

Results of four independent meta‐analyses by type of disciplinary exclusion

Type of Exclusion	SMD	95% CI	n	k	Measure of Heterogeneity
In‐school	.35	(.11; .58)	6	6	Q=11.62;df=5;p<.05;I² =57;τ² =.045
Out‐of‐school	.02	(‐.16; .19)	9	9	Q=22.72;df=8;p<.05;I² =65;τ² =.041
Expulsion	.53	(.07; .98)	4	4	Q=13.66; df= 3p<.05; I²=78; τ²=.14
General measure	.32	(.21 .43)	27	28	Q=171.4; df=27 p<.001; I²=84; τ²=.056

Open in a new tab

Note. n= number of studies; k=number of effect sizes.

In‐school exclusion. Only six trials were concerned with the effect of school‐based intervention on reducing in‐school exclusion. Under a random model, the effect of the intervention remained positive and statistically significant. The final effect described in Figure 7 was SMD=.35 (95% CI .11 to .58; p<.005). Once again heterogeneity was tested, finding significant but smaller variability across studies (Q=11.62; df=5; p<.05; I² =57; τ² =.045).

Forest plot of the effect sizes for the impact of school‐based programmes on in‐school exclusion

Out‐of‐school exclusion. Correspondingly, nine studies reported data for the impact of interventions on out‐of‐school exclusion. Figure 8 shows an effect close to zero, non‐statistically significant of SMD=.02 (95% CI ‐.16 to .19; p>.05).

Forest plot of the effect sizes for the impact of school‐based programmes on out‐of‐school

Expulsion. The impact of school‐based interventions on expulsion was significantly higher than any other impact described so far. Figure 9 shows that expulsion was reduced by SMD=.53 (95% CI .07 to .98; p<.05) with significant heterogeneity (Q=13.66; df=3; p<.05; I² =78; τ² =.14) but based on only four reports presenting data for analysis. Therefore, these results must be evaluated cautiously.

Forest plot of the effect sizes for the impact of school‐based programmes on school expulsion

General Suspension. Finally, a number of studies presented data on suspension as a broad and general measure. These studies did not describe operational definitions about the type of disciplinary suspensions involved in the outcome. We have defined this category as General Suspension. In the aim of transparency, we report these results separately, although this measure could involve any of the previous outcomes reported above and, therefore, be a subset of the overall effect size reported at the beginning of this section.

Figure 10 shows 27 studies reporting 28 independent effect sizes concerned with the impact of targeted interventions on general suspension. The effect of school‐based interventions was positive SMD= .32 (95% CI .21 to .43; p<.001), it was statistically significant and similar to the overall effect size reported in Figure 3 above. In addition, heterogeneity remained substantial (Q=171.45; df=27 p<.001; I² =84; τ² =.056).

Forest plot of the effect sizes for the impact of school‐based programmes on general school suspension

4.3.3 Secondary outcome: overall impact of school‐based intervention on internalising behaviours

As stated in our protocol, for any identified study reporting data on school exclusion, we also coded a secondary outcome referring to internalising and externalising behaviours. Data on internalising behaviour was not very prolific in our set of included studies. Only five trials presented statistical results but sometimes the data was insufficient for effect size calculations. For that reason, we are not running meta‐analysis on internalising behaviours, but we summarise the results in Table 13.

Table 13.

Impact of school based interventions on internalising behaviours

Study	Sample	Measure	Statistical measures
Bradshaw 2012	N=12,334	Emotion regulation	Students in SWPBIS school fared better in comparison with control schools. (y=.05; t=2.38; p<.05)
Harding 2011	N=43	Emotional Symptoms (SDQ)	Negative effect, not statistically significant. SMD=‐.30 (95%CI ‐.3 to .91); p>.05
Russell 2007	N=61	Internalising problems	On page 20, Table 4, the author presents means, standard deviations and sample size for treatment and control group. The author asserts that intervention reduced antisocial behaviour. Based on our calculations, that reduction is not significant (SMD=.32; 95%CI ‐.14 to .79; p>.05)
Tilgham 1988	N=100	Anxiety	The definition of the measure suggests that anxiety is part of a composite measure. Impact of treatment on anxiety is unclear (p.49).
Wyman	N=226	Assertive vs. withdrawn	Measure is described as a measure of anxiety (e.g.,“Nervous, frightened”). The programme has a positive effect on the internalizing behaviour ES=.37 (.03 to .71).

Open in a new tab

4.3.4 Secondary outcomes: overall impact of school‐based intervention on externalising behaviours

We found a diverse range of measures referring to externalising behaviours, such as substance misuse, violence, aggression, and problematic behaviour in school. To pool together comparable measures, we ran a meta‐analysis only on behaviours that could be categorised as antisocial such as aggression, physical fights, delinquency, bullying and conduct disorder ¹⁰ .

Fourteen studies reported complete data for a composite measure of antisocial behaviour (See Figure 11). The fourteen studies provided 15 independent effect sizes. Unusually, Wyman et al. (2010) reported a measure of behaviour control (e.g., children accepting imposed limits). We reversed the results as a proxy of antisocial behaviour and included this information in our calculations. The same procedure was followed with Feindler et al. (1984) who reported a measure of increase in self‐control. Once again, we reversed the effects size as a proxy of antisocial behaviour.

Forest plot of the effect sizes for the impact of school‐based programmes on antisocial behaviour

As reported in Figure 11, the overall impact of school‐based interventions on antisocial behaviour, under a random effects model, was not statistically different from zero SMD= ‐.005 (95% CI ‐.097 to .09; p >.05) indicating an overall null effect of these programmes at reducing antisocial behaviour.

It must be highlighted that some of the included studies in this overall measure reported negative effect sizes, meaning that in some specific cases the intervention had iatrogenic impact (Hawkins et al., 1988; Hostetler & Fisher, 1997; Obsuth et al., 2016). For instance, Hawkins et al., (1988) reported the impact of an intervention focused on interactive teaching and co‐operative learning targeting low achievers in mainstream schools. The treatment group showed a reduction in the number of exclusions but an increase in the mean value for serious crime. The study reported by Hostetler & Fisher, (1997)described a similar case. For its part, Obsuth et al., (2016) reported the results of a clustered randomized controlled trial testing the impact of “Engage in Education‐London”. The intervention targeted high risk students and it was aimed at improving communication and broader social skills. Results suggest that the iatrogenic effects were observable not only for exclusion but also for the case of antisocial behavior.

It is important to mention that Feindler et al., 1984; as well as Wyman et al., (2010) were reverted and used as proxies of antisocial behaviour. For this reason, they should not be understood as interventions which report iatrogenic results.

4.4 SUB‐GROUP ANALYSIS

A number of potential effect modifiers that could help to explain the expected heterogeneity in our results were initially identified in the published protocol. Based on previous research we pre‐defined moderators that covered four aspects, namely: i) participants’ demographic characteristics; ii) behavioural problems; iii) the theoretical basis of the interventions, and iv) the quality of the intervention. In this section we present sub‐group analysis results. The calculations have been run under a random effects model assuming a separate variance component. Forest plots with further details are reported in Section 11 (appendix).

4.4.1 Effects moderated by participants’ demographic characteristics

Of the 38 effects reported in this meta‐analysis, 11 were tested in schools whose population was predominantly male (i.e., more than 60% of students were male), and 19 effects were tested in schools presenting a mixed population (i.e., neither gender exceeded 60 per cent).

Post‐intervention effects were different for both groups. For studies targeting predominantly male schools, the standardised mean difference was SMD=.41; (95% CI .10 to .72; p<.05). In studies targeting mixed‐gender schools, the impact was lower (SMD=.17; 95% CI .02 to .32: p<.05). Differences between groups were not statistically significant (Q=1.84; df=1; p>.05).

In terms of age, the best proxy variable was school grade. This information was reported in 34 studies and 35 independent effects. To test the hypothesis that effect sizes vary by age, we ran sub‐group analysis for 12 studies involving elementary school students (SMD=.27; 95%CI .09 to .45; p<.05), 16 targeting middle schools (SMD=.23; 95% CI .04 to .41; p<.05), and eight targeting high schools (SMD=.45; 95%CI .18 to .72; p=.001). The effect was statistically significant in each sub‐group and larger in high school populations. However, the between effect difference was not statistically significant (Q=1.81; df=2; p=.41) meaning that there is no evidence that the effect of school‐based interventions differs by age (i.e., grade at school) in the present meta‐analysis.

Ethnicity was reported as a continuous variable (i.e., percentage of each ethnic group). For that reason, the role of ethnicity in explaining the effect of school‐based programmes will be explored in a meta‐regression analysis.

4.4.2 Effects moderated by participants’ behavioural problems

In our review, only a limited number of studies presented data on behaviour (i.e., internalising or externalising behaviours). In addition, the overall impact of school‐based interventions on antisocial behaviour was not statistically different from zero.

Pupils involved in included studies shared a similar high‐risk condition. They were registered in schools with a high percentage of ethnic minorities and more than 60% of the students in those schools received free school meals, which is an indicator of disadvantaged socio‐economic backgrounds. In that sense, participants do not display high variability.

4.4.3 Effect of different school‐based programmes on school exclusion

One of the aims of the present meta‐analysis was to compare the effect of different interventions on the reduction of school exclusion. Table 14 presents the standardised mean differences, confidence intervals and p‐values as well as measures of heterogeneity for each of the ninetypes of programmes included in the review. The typology is the same as described in section 4.1.7.

Table 14.

Effect size by type of school‐based intervention

Type of Intervention	SMD	95% CI	p‐value	k	Measure of Heterogeneity
Enhancement of academic skills	.43	(.25; .61)	p<.001	2	Q=.09;df=1;p>.05;I² =0;τ² =0
After‐school programme	.05	(‐.08; .17)	p>.05	2	Q=.91; df=1; p>.05; I²>=0; τ² =0
Mentoring/monitoring	.47	(.02; .93)	p<.05	5	Q=20.1;df=4;p<.001;I² =80;τ² =.21
Skills training for students	.31	(‐.05; .67)	p>.05	9	Q=60.4;df=8;p<.001; I² =86;τ² =.23
Skills training for teachers	.31	(.11; .52)	p<.05	4	Q=13.6;df=3;p<.05; I² =78;τ² =.03
School‐wide strategies	.20	(‐.03; .43)	p>.05	6	Q=34.8;df=5;p<.001; I² =86;τ² =.06
Violence reduction	.48	(‐.33; 1.3)	p>.05	3	Q=13.3;df=2;p<.001; I² =85;τ² =.44
Counselling, mental health	.46	(.23; .68)	p<.001	3	Q=2.65;df=2;p>.05;I² =25;τ² =.01
Other	.21	(.03; .39)	p<.05	4	Q=4.11;df=3;p>.05;I² =27;τ² =.01

Open in a new tab

First of all, as observed in Table 14,most of the interventions are represented by a restricted number of effect sizes, and for this reason, these results should be interpreted cautiously.

Secondly, the standardised mean differences of only five types of programmes present positive (small to moderate effect sizes) and statistically significant results in favour of the reduction of school exclusion. Those programmes are: i) Enhancement of academic skills, ii) Mentoring/monitoring, iii) Skills training for teachers, iv) Counselling/mental health services, and v) Other programmes. Since “other programmes” involve a mixture of different interventions we believe they cannot be interpreted in the same way as the remaining four types. Similarly, when it comes to the number of studies included in each sub‐group, it seems that the most stable results are Mentoring/monitoring and Skills training for teachers since they are based on a larger number of studies.

Thirdly, to test the hypothesis that differences were significant among the compared sub‐types, we ran further analysis. The comparison demonstrates that differences are statistically significant (Q=18.4; df= 8; p<. 05), meaning that variation in effect sizes can be explained by the type of intervention implemented.

4.4.4 Theoretical bases

Reported information on the theoretical bases of the interventions was not very comprehensive. Selected studies described components of interventions more frequently than reporting the theory or set of theories framing the praxis. Based on the provided details, 20 interventions were clearly based on a cognitive behavioural frame, while another five were concerned with ecological ideas, targeting a change of the school system as a whole. The remaining effects refer to different theories such as emotional intelligence (i.e., Smith, 2004), empathy (i.e., Okonofua et al., 2016), civic values (i.e., Crowder, 2001) or developmental theories (i.e., Allen et al., 1997). A number of studies did not provide enough information to make a judgement. We tried to run sub‐group analysis on the theoretical bases of the interventions but it demonstrated low power, given that the number of effects was, in some cases, as low as one per sub‐group. In that scenario, we decided that a measure of the level of targeted change could inform more consistent data on the theory framing the interventions. We then divided the studies into those targeting a systemic change versus those targeting a change in pupils.

The standardised mean difference for the 10 evaluations targeting a change at school level provideda significant reduction with a value of SMD=.25 (95%CI .04 to .45), whereas the 28 evaluations targeting a change at the pupil's level reported a significant reduction with a value of SMD=.33; (95%CI .19 to .48). Both independent effects are statistically significant; however, the between‐group comparison reported non‐significant differences (Q=.48; df=1; p>.05) meaning that there is no evidence that the effect differs by level of targeted change.

4.4.5 Moderator analysis: quality of implementation

Previous research demonstrates that well‐implemented programmes – those including training, monitoring and supervision – display larger and more consistent effect sizes (e.g., Durlak et al., 2011; Gottfredson & Wilson, 2003; Lösel & Beelmann, 2006). We tested this hypothesis based on two variables, namely, “training before implementation” and “monitoring during intervention”.

Twenty‐five studies (reporting 26 independent effect sizes) clearly stated the presence of training hours before the intervention was delivered (i.e., training hours for those delivering the intervention). In the remaining 12 studies, authors did not mention any kind of training. We ran sub‐group analysis to test the hypothesis that those that reported training could produce a significantly different effect. In the end, those reporting prior training yielded a result equal to SMD=.29 (95% CI .16 to .43; p<.001) whereas those that did not report training produced an effect equal to SMD=.34 (95% CI .15 to .53; p<.001). Both effects were positive and significant, however the test of the difference between the two sub‐groups of studies yielded a result of Q=.16 with df=1 and p>.05, meaning that there is not enough evidence that effects differ by presence/absence of prior training.

Fifteen studies that reported monitoring the implementation of the programme during the trial yielded a result equal to SMD=.20 (95%CI .05 to .35; p<.05). In parallel, the 23 studies that did not report monitoring produced an SMD=.37 (95%CI .25 to .50; p<.001). Both results were positive and statistically significant. The test for differences between sub‐groups showed non‐significant differences (Q=2.89; df=1; p>.05).

4.4.6 Post‐hoc moderators

Based on descriptive analysis of data, and also based on previous research findings (Beelmann & Lösel, 2006; Farrington, Ttofi, & Lösel, 2016), we selected three post‐hoc moderators, namely: i) reasons for conducting the research; ii) evaluator role, and iii) risk of quality bias. Their results are described below:

Reasons for conducting the research. We coded data related to the reasons for conducting research. Two categories are explored: i) demonstration study, referring to those studies testing the impact of an intervention under highly controlled optimal conditions and ii) routine evaluation, focusing on testing established programmes under circumstances that approach real‐life conditions (Singal, Higgins, & Waljee, 2014).

Sub‐group analysis produced different effects for the 16 studies conducted for demonstration (SMD=.43; 95% CI .26 t0.59; p<.001), and the 18 studies carried out for routine evaluation (SMD= .13; (95% CI .00 to .25; p<.05). Demonstration studies reported the highest effect. The between‐studies comparison results were significant (Q=8.15; df=1; p<.05), meaning that the effect varies depending on the reasons for conducting the research.

Evaluator role. We coded data identifying the role of the evaluator. Independent evaluators were those not taking part in the design or implementation of the evaluated programme. Dependent evaluators were those also contributing to the design and/or the implementation of the programme.

The 18 trials carried out by independent evaluators produced an SMD= .13 (95% CI .00 to .25; p<.05). Unsurprisingly, based on what is known about developer‐led trials, the 16 RCTs conducted by dependent evaluators (i.e., those who also developed and/or designed the intervention) yielded results where SMD=.47 (95% CI .32 to .62; p<.001). The between‐groups comparison demonstrated that effect sizes are lower for studies conducted by independent evaluators. These differences were statistically significant (Q=12.36; df=1; p<.001).

Risk of quality bias. As reported in section 4.2, we ran EPOC tool for assessing risk of quality bias. Results show that studies vary in the amount of information available for judging their level of risk as well as their risk of bias. We speculated that the quality of studies could explain some of the heterogeneity present in our results. In order to use the results reported by EPOC, we transformed the “low risk”, “unclear risk” and “high risk” into continuous 0, 2 and 3 values respectively. The sum of the results of the eight EPOC criteria resulted in an index that, in our case, was used as a representation of the level of risk of each single study. We tested the role of RoB in meta‐regression.

4.5 META‐REGRESSION

In order to explore heterogeneity, we ran meta‐regression using the moderators defined a priori (participants’ characteristics and intervention characteristics). Model I included participants’ characteristics only, to get a sense of the net value of these variables in the results. Students’ gender, grade at school or ethnicity (continuous variable), as reported in Table 15, did not explain heterogeneity in the study results. In model II, we introduced the intervention characteristics (i.e., interventions targeting a change at the individual level versus those targeting a change at the school level). Once again, none of the variables present significant results. This suggests that based on the present data, variability across effect sizes cannot be explained by the a priori defined modifiers.

Table 15.

Meta‐regression results: a priori moderators

Predictors	b	Model I SE	*95%CI*	b	Model II SE	*95%CI*
Intercept Participants’ characteristics	.37	.33	‐.28; 1.06	.37	.34	‐.30;1.04
Gender Male predominant Mixed	.09 ‐.05	.38 .37	‐.64;.82 ‐.77;.67	.11 ‐.06	.39 .38	‐.66; .87 ‐.81; .69
Grade at school (High) Elementary Middle Mixture	‐.11 ‐.22 .17	.21 .20 .40	‐.52;.29 ‐.60;.16 ‐.62;.96	‐.16 ‐.24 .04	.23 .20 .48	‐.61; .29 ‐.64; .16 ‐.90; .98
% of White	‐.00	.004	‐.008; .006	‐.00	.003	‐.00; .00
Intervention characteristics
Individual versus school level change				.11	.21	‐.30; .52

Open in a new tab

^* p<.05

We proceed with a meta‐regression model involving post‐hoc variables that could potentially explain the heterogeneity present in our results. Based on the descriptive results in section 4.1.6, and bearing in mind some previous findings, we hypothesise that the characteristics of the research, the ends of the research (demonstration versus routine), the role of the evaluator, and the risk of quality bias could play a role in explaining the results (see Table 16).

Table 16.

Meta‐regression results: post‐hoc moderators

Post‐hoc predictors	Model
	b	SE	*95%CI*
Intercept Characteristics of the research	.32	.17	‐.01;
The ends of the research (routine vs. demonstration)	.06	.14	‐.20; .33
Role of the evaluator (independent vs. dependent)	‐.36	.14	‐.63; ‐.09 ^*
Risk of quality bias	.008	.01	‐.01; .03

Open in a new tab

* p<.05

The results were significant (p<.05) only for the variable role of the evaluator. The coefficient is negative with the category of reference “dependent”, meaning that the effect is lower when an independent team runs research.

Figure 12 presents the value of R‐squared or percentage of explained variance between‐studies based on study‐level characteristics. To compute the total variance (of all studies about the grand mean), we run the regression with no covariates (a). To compute the variance not explained by the model (of all studies about the regression line), we run the regression with covariates (b). Finally, the difference between these values gives us the variance explained by the model which is R²=.58, meaning that the model explains 58% of the between‐studies variance.

R‐squared graphic. Proportion of the variance explained by the role of the evaluator

4.6 PUBLICATION BIAS ANALYSIS

Publication bias in systematic reviews occurs when the included set of manuscripts fail to systematically represent the whole population of completed studies that should have been included. The whole population of studies can involve a range of results that must be present in a meta‐analysis to make it valid. However, consistent evidence indicates that studies presenting large effects are more likely to be published than those presenting null or modest effects (Rothstein, Sutton, & Borenstein, 2006). This means that publication bias can lead to an overestimation of the impact of a treatment when running meta‐analysis.

In the present study, much effort was spent in finding most complete collection of published and unpublished studies that test the impact of school‐based intervention on exclusion rates. In fact, almost 50% (see Table 7) of our included studies have never been published in books or peer reviewed journals (i.e., PhD thesis, technical or governmental reports).

As originally proposed, we use statistical procedures to quantify potential bias that could affect our analysis. First of all, we produce a funnel plot of standard errors by standardised differences in means, presented in Figure 13. In theory, symmetrical distributions of dots under the funnel represent a normal distribution of studies. As anticipated, our studies mostly fall under the funnel, and they are distributed around the main effect. However, since the evaluation of funnel plots can be subjective, we conducted additional statistical measures of publication bias, specifically, Duval and Tweedie's trim‐and‐fill analysis.

Funnel plot of standard error by standard differences in means

Duval and Tweedie's trim‐and‐fill analysis compares the differences in effect sizes that could potentially be attributed to bias. The technique imputes effect sizes until the error distribution gets close to normality. In this way, the test offers the best estimate of the unbiased effect (Borenstein et al., 2009). Results of Duval and Tweedie's trim‐and‐fill analysis suggest that there were no differences in effect sizes attributable to bias. Under the random effect model, the values were SMD=.30; 95%CI .20 and .41. Based on the parameter of Duval and Tweedie's trim‐and‐fill, it seems that no studies are missing.

4.7 SENSITIVITY ANALYSIS

In the present meta‐analysis, the effect of the treatment was calculated as the difference between post‐treatment and baseline. We corrected the value of the variance by assuming a value of correlation equals to .75. In order to test the robustness of this assumption, we ran sensitivity analysis with a correlation equal to .50. Table 17, panel A shows that overall results remain stable when the correlation is smaller.

Table 17.

Sensitivity analysis

Panel A: with outlier
Covariate	SMD	95% CI	SE	V	n	k	Measure of Heterogeneity
0.50	0.31	(.20; .43)	0.058	0.003	36	37	<Q=259.0;df=36;p<.001;I² =86;τ² =.081
0.75	0.32	(.21; .44)	0.058	0.003	37	38	<Q=338.4;df=37;p<.001;I² =89;τ² =.090
Panel B: without outlier
0.50	0.27	(.16; .38)	0.056	0.003	36	37	<Q=233.3;df=36;p<.001;I² =84.5;τ² =.072
0.75	0.28	(.17; .39)	0.055	0.003	36	37	<Q=288.0;df=36;p<.001; I² =87.5;τ² =.076
Panel C: with outlier Winzorised
0.50	0.30	(.19; .41)	0.057	0.003	37	38	<Q=246.23;df=37;p<.001;I² =84.9;τ² =.076
0.75	0.30	(.19; .41)	0.055	0.003	37	38	<Q=301.3;df=37;p<.001;I² =87.7;τ² =.078

Open in a new tab

Another decision was related to the presence of outliers. We found one study presenting an effect size more than three standard deviations from the mean effect size, which was defined as an outlier (Collier, 2002). We tested the impact of the outlier and also the impact of winsorization. The size of the effects, their direction and significance were not altered. See Table 17 below, panels B and C.

Finally, as stated in the protocol, we ran a sensitivity analysis to test differences between published and unpublished reports. The 20 independent effect sizes reported in 19 peer‐reviewed journalsyield a SMD=.31 (95%CI .17 to .45) whereas the 18 effect sizes extracted from unpublished reports reported a SMD=.29 (95%CI .11 to .47). The between effect difference was not statistically significant (Q=.028; df=1; p>.05).

5. Discussion

5.1 SUMMARY OF MAIN RESULT

In the following paragraphs we present a summary of our results based on the research questions leading this review.

Do school‐based programmes reduce the use of exclusionary sanctions in schools?

The analyses reported in previous chapters suggest that school‐based interventions are capable of producing a small and significant (SMD=.30; 95% CI .20 to .41; p<.001) drop in exclusion rates. It means that those participating in interventions are less likely to be excluded than those allocated to control/placebo groups. These results are based on measures of impact collected on average, six months after treatment. When the impact was tested in the long‐term (i.e., 12 or more months after treatment), the effect of interventions was not sustained. In fact, the impact of school‐based programmes showed a substantial reduction (50%), and was no longer statistically significant.

We ran a further analysis including only the seven studies which reported post‐treatment and follow‐up measures. The overall impact of school‐based interventions at post‐treatment was SMD=.21 (95%CI .11 to .30). Even if the impact was slightly reduced, when we included those seven studies the average time involved under “post treatment” was 12 months on average, meaning that this measure of impact would be more precise and the impact would last longer. Coherent with the original results, the effect produced by combining the seven studies only was null and non‐significant in the long term (i.e., follow‐up measures).

In addition, our results suggest that school‐based interventions present different levels of impact depending on the type of exclusion. After running moderator analysis by type of exclusion, the evidence across studies demonstrated that school‐based interventions are more effective at reducing in‐school than out‐of‐school exclusion. The impact of programmes in the latter case was close to zero with p>.05, meaning that the result could have arisen by chance. Moreover, a larger impact was observed in the sanction expulsion, with a moderate and significant effect size (SMD=.53; 95%CI .07 to .98; p<. 05). However, these results must be cautiously interpreted due to the low number of manuscripts involved in the calculations.

We found 28 studies lacking an operational definition of the dependent variable. The studies did not specify the type of exclusion or exclusion tested, and in that scenario, we decided not to assume this general measure was out‐of‐school exclusion. The impact of school‐based interventions in those “general suspensions” was similar to the overall impact, which probably suggests that these studies are simply a subset of the overall measure.

As originally planned, in this review we ran independent meta‐analysis testing the impact of school‐based interventions ona secondary outcome, that is, internalising and externalising behaviours. It was hypothesised that a reduction in exclusion would be linked with variations in students’ behaviours.

In the case of externalising behaviours, we were not able to calculate a pooled effect size. Results suggest that only five out of the 37 included studies reported a measure for internalising behaviours. However, the narrative description suggests that school‐based intervention had a small effect on the reduction of the above symptoms.

We ran a meta‐analysis based on 15 effect sizes reporting the impact of school based intervention on externalising behaviours. Results show negative and non‐significant impact (SMD=‐ .005; 95%CI ‐.09 to .086; p>.05). It therefore follows that interventions aimed at reducing exclusion do not necessarily reduce antisocial behaviour. This could be in line with evidence suggesting that changes in school policies, rather than changes in behaviour, produce a reduction in disciplinary exclusion (Noltemeyer & Fenning, 2013; Skiba et al., 2015). Two questions arise from these results. First, how do we interpret them, and second, is it still worth implementing programmes that show no impact on antisocial behaviour?

Regarding the first question, one could hypothesise that if race, or more precisely being a racial minority in a given country, is a stronger predictor of school exclusion than other demographic and behavioural characteristics (as shown in Section 1.2), antisocial behaviour would not necessarily explain the rates of exclusion. What is more, it would not be the main cause of this punishment being used. A second possibility is that the behaviour outcome was not measured properly.

The second question considers whether the school‐based interventions tested in this review are still worth implementing if they do not reduce antisocial behaviour. The evidence produced by the present review shows that those programmes presenting positive results do not report negative side effects, in the sense that control groups do not increase the rate of punishment. If that is the case, we suggest that these efforts should be continued. However, based on our data, these ideas cannot yet be regarded as conclusive and more research is needed about the causes that explain the phenomenon of exclusion.

Are some school‐based approaches more effective than others in reducing exclusionary sanctions?

The nine different types of interventions presented in the set of included studies were compared to test the hypothesis that some are more effective than others. There was a considerable variability across the programmes in overall effectiveness. Enhancement of academic skills, mentoring/monitoring, skills training for teachers and counselling/mental health services reported the largest and most significant effect sizes. Based on the number of studies included in each sub‐type, we believe that skills training for teachers and mentoring/monitoring represent the stronger and more reliable findings. The subgroup analysis reported that differences across types of programmes were significant, implying that effects vary depending on the programme implemented.

In line with previous findings, Tolan et al. (2008) examined 39 experimental and quasi‐experimental studies and specifically analysed the effectiveness of mentoring in reducing delinquency, aggression and drug use, and improving academic achievements. The largest positive effects were found for aggression (SMD = .40; 95%; CI = 0.06 to 0.74) and delinquency (SMD = .23; 95% CI .11 to .36). The meta‐analysis conducted by Eby, Allen, Evans, Ng, and Dubois (2008) also found that mentoring programmes had a small and significant positive effect on academic performance.

Overall, mentoring programmes seem to be an effective strategy for reducing violence and conduct problems during adolescence. However, on average the effect sizes reported in relevant studies are small. In this respect, it is important to bear in mind the findings by Tolan et al. (2008). This study suggests that mentoring was more effective when (a) participants had greater pre‐existing behavioural problems or had been exposed to significant levels of environmental risk, (b) they were male, (c) the educational or occupational backgrounds of the mentors fitted the goals of the program, (d) mentors and youths were successfully paired, with similar interests, and (e) programmes were structured to support mentors.

As far as teachers’ skills for managing students’ behaviour is concerned, previous evidence suggests that such programmes canimprove teachers’ general instructional and behavioural management skills in planning, implementing and maintaining effective classroom practices. In fact, a meta‐analysis conducted by Gottfredson, Wilson, and Najaka (2002) found that classroom or instructional management programmes (k=25) demonstrated a small and significant effect in reducing antisocial/aggressive behaviour (SMD= .13, p<.05). The most recent meta‐analysis carried out by Oliver et al. (2011), concluded that classroom management practices (N=12 studies) had a significant, positive effect on reducing problem behaviour. Students taking part in the intervention display less disruptive, inappropriate and aggressive behaviour in the classroom compared to those in control classrooms. The overall effect of the intervention was g= .22; p<.05 (Oliver et al., 2011). Evidence also suggests that programmes with the most positive effects tend to be of a longer duration and tend to combine classroom and instructional management strategies with some other major ingredient (e.g. parent training or social skills instruction) (Gottfredson et al., 2002).

The positive relationship between teachers and students has been found to be a factor in promoting more prosocial and less aggressive behaviours later in life. A recent study (Obsuth et al., 2016; p. 16), using a non‐bipartite propensity score matching technique, found that “teachers who reported having a more positive relationship with a student at age ten observed significantly fewer aggressive and defiant behaviours and more prosocial behaviours in the same student concurrently and one year later, at age 11. This was also associated with more prosocial behaviours two years later, at age 12 and also with less aggressive behaviour at age 13. Similarly, students who perceived a more positive relationship with their teacher at age 11 reported fewer aggressive behaviours and more prosocial behaviours concurrently and fewer aggressive behaviours two and four years later, at ages 13 and 15.” All these results make us believe that investing in teachers’ skills and positive relationships between students and teachers is worthwhile, with schools becoming target locations for preventing crime and promoting positive psychosocial development.

Finally, it is worth mentioning that we grouped interventions by those targeting a change at student level versus those expecting a change at school level. Even if the programmes targeting a change at student level display a larger effect size, the between‐group analyses do not allow us to conclude that the differences between these intervention strategies are statistically significant.

Do participants’ characteristics (e.g., age, gender, ethnicity) affect the impact of school‐based programmes on exclusionary sanctions in schools?

Differences in post‐intervention effects were non‐significant when we compared schools whose population was predominantly masculine versus mixed schools. Thus, based on our findings, we could not confirm the hypothesis that the effect differs by the distribution of gender in schools. The same occurred with ethnicity and age. As both variables were continuous, we included them in our meta‐regression. Based on the available data, we could not confirm the hypothesis that the impact of the interventions varies when the intervention was implemented with younger or older students. Similarly, variation in the percentage of white students did not play a role in explaining the heterogeneity of the effect.

Do characteristics of the interventions, implementation and methodology affect the impact of school‐based programmes on exclusionary sanctions in schools?

Twenty‐five studies (reporting 26 independent effect sizes) stated the presence of training hours before the intervention was delivered. In the remaining 12 studies, the authors did not mention any kind of training. When comparing the effect of both sets of studies, the between‐group analysis was non‐significant, meaning that there is no evidence that effect differs by presence/absence of training before implementation.

We ran an analysis to investigate the differences between those interventions that deployed monitoring during implementation and those which lacked monitoring. Interventions without monitoring produced significantly larger effects than those with it. In other words, programmes lacking monitoring produced larger effect sizes. This result is in line with previous findings indicating that a lack of monitoring, for instance in routine trials, tends to show smaller effects than demonstration evaluations (Farrington, Ttofi, & Lösel, 2016).

On the other hand, we observed that interventions run by independent evaluators found significantly smaller effect sizes when compared to those studies carried out by researchers involved in the design and/or delivery of the programme. Again, this is not surprising. Previous research has called attention to this phenomenon as well as the advantages of running impendent trials for producing more precise and realistic results (e.g., Eisner, 2009; Manuel Eisner & Humphreys, 2012; Lösel & Beelmann, 2006b; Petrosino & Soydan, 2005; Wilson, Lipsey, & Soydan, 2003).

5.2 OVERALL COMPLETENESS AND APPLICABILITY OF EVIDENCE

In the present review we screened a total of 42,749 citations across 27 electronic databases. Pre‐defined searches targeted published and unpublished reports from any country, in any language, as long as the abstract was presented in English. Only one study in a language other than English was identified by our searches. We purposively conducted electronic searches in databases involving manuscripts produced in Latin America and other Spanish and Portuguese‐speaking countries (e.g., SciELO‐Scientific Electronic Library Online). However, we did not find any evaluations whose characteristics make them includable in our review.

When compared to other contemporary reports (e.g., Dymnicki, Weissberg, & Henry, 2011; Mytton et al., 2006), our searches seemto be comprehensive enough to reduce the risk of publications bias. All in all, the extent of our searches and the high percentage of retrieved full reports make us confident that we have identified a substantially complete collection of the available relevant research.

However, this systematic review has allowed us to identify some gaps in the literature. First of all, studies testing the impact of school‐based interventions fail to disentangle the impact on different forms of disciplinary exclusion. In future, the availability of data calculating the impact of interventions across different types of exclusionary punishment could offer more detailed evidence in respect to what works in the field of school discipline.

Secondly, we did not find includable studies testing the impact of restorative justice strategies. One single ongoing study (i.e., Acosta, 2015) was detected by our searches but its results are to be released in 2018. As recently stated by Fronius et al., (2016: 17 and 19), the research evidence to support restorative justice in schools is still in a nascent state. In fact, the scarce evaluations produced so far rest in pre‐post designs lacking control comparison groups, which would likely have a serious impact on the internal validity of the empirical results. The potential for restorative principles to be applied in school settings needs to be explored in more detail.

Thirdly, we believe that the production of independent, high quality evaluations could contribute to more transparent and precise evidence regarding the impact of school‐based programmes. Previous research shows that demonstration programmes or those where designers take part in the evaluation tend to produce larger effect sizes (Beelmann & Lösel, 2006).

Fourth, much of the evidence presented in this review has been produced in the United States where school safety and exclusion is a salient concern for researchers and policy makers alike. As shown in Table 1 of the present report, in a sample of high‐ and middle‐income countries, exclusion is a widespread school punishment. In European countries, exclusion is a more regulated sanction, while in some Latin‐American societies exclusion is under the discretional decision of each particular school (e.g., Chile). We think that research on the use of exclusion in those contexts needs more development.

Fifth, we observe a lack of research testing the long‐term impact of school‐based interventions. The evidence shows that the impact immediately after intervention tends to be larger than that measured several months later (Farrington, Ttofi, & Lösel, 2016). Since in real life policy makers need to invest in effective programmes whose results endure long‐term, we think that an effort should be made to test the impact at least 12 months after intervention ends.

Finally, although we did not include cost‐benefit analysis in our aims, data was coded regarding the presence/absence of interventions’ costs. Two of our included studies reported the economic cost of programme implementation. We believe that cost‐benefit analyses are key for alerting decision makers about the advantages of early intervention. The prevention of exclusion and expulsion from school can reduce future social exclusion, violence and other negative correlates described in the research (Noltemeyer, Ward, & Mcloughlin, 2015).

5.3 QUALITY OF THE EVIDENCE

The present review involves only randomised controlled trials. They are considered the best methodological design for isolating confounding factors and producing an accurate measure of intervention effects. However, as stated earlier in the results section, our included reports are lacking a considerable amount of information for judging the quality of the procedures carried out.

More than 50% of the reports fail to provide enough data to judge the precision of randomisation, that is, sequence generation and allocation concealment. Likewise, 30% of the studies did not report data on the equivalence of the groups after randomisation. As recently stated by Roberts & Ker, (2016), missing data on those details can drive analysts to identify “false positive” RCTs. We cannot claim that was the case among our studies, but clearly the absence of detail imposes serious limitations on the assessment of quality in regard to potential threats to internal validity.

A similar lack of data affected the evaluation of blind outcome measures. Even if most of the exclusion measures in our report are based on official records, we cannot ignore the fact that teachers or school staff are in charge of imposing sanctions and could potentially be aware of a student's participation in the experiment. More than 50% of the studies present few details for judging the level of risk involved in this criterion.

Randomised controlled trials of educational programmes are receptive to contamination (Hutchison & Styles, 2010; Simmons et al., 2015). As participants in the control arm may be surrounded by those receiving the treatment, because they share the same school, the likelihood of contamination is a matter of concern. Some of the effects of contamination have been identified by previous research (Giraudeau & Ravaud, 2009; Torgerson, 2001).

Contamination may reduce the effect of the intervention, leading to a type II error – that is, incorrectly retaining a false null hypothesis (a “false negative”) or, described in a different way, a “rejection of an effective intervention as ineffective because the observed effect size was neither statistically nor clinically significant” (Torgerson, 2001). In the present meta‐analysis, 76% of the studies suggest a high risk of contamination bias. It could be the case that our estimation of the impact is underestimated.

In future research, it seems that the use of cluster randomised control trials would help ameliorate these biases. Even if clustered RCTs involve complex designs and demand huge efforts, the quality of the much‐needed evidence can be enhanced by using this methodological design (Hutchison & Styles, 2010). However, the randomisation of clusters implies challenges for researchers. “The main issue is that observations from the same cluster are more similar than observations from two different clusters. This situation requires the use of both an inflated sample size and adapted statistical analysis to take into account this concern” (Giraudeau & Ravaud, 2009, p. 1).

5.4 LIMITATIONS AND POTENTIAL BIASES IN THE REVIEW PROCESS

There are several limitations that could affect the results of the present review. It is important to acknowledge that even though we focused on randomised controlled trials, which are supposed to be the best evidence for measuring evaluation impact, included studies present limited information for judging quality bias.

Some 35% of the included studies reported results based on samples with less than 100 participants. The small size of the samples involved in some of the primary research could impose clear limitations on the ability to estimate the effects of interventions.

As reported in the moderator analysis, the independence of the evaluator explained the heterogeneity of effect sizes. Even if independent teams reported on a good number of our studies, close to 50% of them did not use independent evaluators. This fact could add some bias in to our results.

However, it is also important to elucidate some of the advantages of the present study. First of all, this systematic review and meta‐analysis is the first attempt to collect and statistically summarise interventions pursuing a reduction in school exclusion. As such, we believe that our report offers an overview of the amount, characteristics, limitations, and quality of the available evidence, as well as a measure of the size of the effect achieved by different types of intervention. The reader can find in this report an updated review of the evidence produced in the United States and the United Kingdom.

Secondly, we have endeavoured to use an exhaustive coding process to provide careful descriptions of the studies as well as a meticulous analysis of the statistical data available. Due to the inclusion of cluster data, corrections were introduced to make the information comparable at an individual level. Such corrections were carried out using the most recent strategies suggested by Hedges, (2007) and Pigott, (n.d.). We therefore believe that our calculations are at a low risk of underestimating the size of standard errors.

5.5 AGREEMENTS AND DISAGREEMENTS WITH OTHER STUDIES OR REVIEWS

The present review is the first meta‐analytical effort to identify the impact of school‐based intervention at reducing school exclusion. In that sense, we have no similar evidence with which to contrast our findings.

However, the results provided by our review seem coherent with other studies testing the impact of school‐based intervention on behavioural outcomes. For instance, Wilson et al., (2011) conducted a meta‐analysis on school‐based programmes looking for a reduction in drop‐outs. Our results more or less target similar programmes and the detected effect size follows a similar trend. In the same vein, Mytton et al., (2006) conducted another review testing the impact of school‐based interventions in preventing violence. They found similar problems with quality assessment and reported an overall effect size slightly better than the one produced by the present review. However, in that last case, the long‐term effects are statistically stronger and last longer. The reduction in aggression reported by Mytton et al., (2006) is not clear in our results.

In a more general examination, a recent review of reviews conducted by Farrington, Ttofi, & Lösel, (2016) found a mean effect ¹¹ of school‐based programmes equal to SMD=.184 (95%CI .16 to .20). Across the studies, effect sizes ranged between SMD=.091 to SMD=.631. Our findings are, therefore, more or less coherent with the impact of interventions in school settings evaluated by previous meta‐analysis.

6. Authors’ conclusions

The empirical evidence produced by this report suggests that non‐punitive school‐based programmes can reduce the use of exclusion. Even if the effects are not sustained for the long term, data shows that in the short term (i.e., six months on average) schools can opt for different and more effective approaches to managing discipline, rather than zero‐tolerance policies. This review, aimed at testing the effectiveness of school‐based programmes, offers a broad overview not only of the effectiveness of different interventions, but also uncovering findings that can guide public policy and future research.

6.1 IMPLICATIONS FOR PRACTICE AND POLICY

Research consistently reveals that school exclusion is disproportionately used as a punishment for ethnic minorities, males and those coming from low socio‐economic backgrounds. Exclusion as a disciplinary school measure seems to reduce school attendance, increase drop‐out rates and restrict future possibilities for inclusion in the labour market. Aside from that, research has also been consistent in finding a correlation between permanent exclusion and antisocial behaviour.

Though the causal link between exclusion and the above outcomes is still not clearly established, research based on observational data suggests that the consequences of exclusion affect not only students and their families but extend to the whole society. Indeed, as stated by Rumberger and Losen (2016), exclusion may involve a high economic cost to the taxpayer. In their most recent report, the authors provide a conservative estimate of the economic impact of exclusion. They assert that, in the United States, tenth grade exclusions alone account for more than 67,000 school drop‐outs, at a cost of $35 billion. ¹²

Although our results must be cautiously considered, the evidence produced by this review suggests that school administrators and policymakers do have alternatives to exclusion when dealing with disciplinary problems. In our findings, prevention strategies have a small but encouraging impact on exclusion rates, at least in the short term. When comparing different types of programmes, it appears that prevention programmes targeting teachers’ skills as well as those introducing mentoring/monitoring schemes can have a positive impact in reducing exclusion. These results are in line with previous research and also with previous narrative reviews which emphasize the importance of teachers’ skills and mentoring programmes in promoting prosocial behaviours and values (Freiberg & Lapointe, 2006; Gottfredson et al., 2002; Oliver et al., 2011; Tolan et al., 2008).

It is important to clarify that this is most applicable to the United States, where the majority of the assessed evidence was collected. As expressed in the previous paragraphs, some flaws affect the contemporary primary evaluations present in this review. However, these results should encourage researchers to produce better quality evidence rather than abandoning their efforts to find strategies to replace exclusionary punishments.

6.2 IMPLICATIONS FOR RESEARCH

Clearly, more primary research evaluating the impact of prevention programmes targeting school discipline is needed. Ideally this research should be conducted under high methodological standards, accounting for mediating mechanisms that lead to a reduction in exclusion rates.

In particular, the results of this first review on the topic identify some implications for future research, laid out below:

Addressing the racial gap. Most of the literature reviewed in the present study indicates that racial or ethnic identity plays a central role in predicting school exclusion, even after controlling for demographic and behavioural variables. More research needs to be developed for testing the mechanisms that produce different treatment for some racial minorities, such as African‐American and Latino students in the United States or Black Caribbean and Gypsy/Romany students in the United Kingdom. It is necessary to understand the individual and social processes that lead to the overrepresentation of minorities in school exclusion. As suggested by some scholars, the divergence between students and teachers’ cultural expectations could potentially contribute to misinterpretation of each other's behaviour, fears and conflicts (Gregory et al., 2010). Understanding the mechanisms that make race a predictor of exclusion could have implications for future policy and practice. For instance, as stated in previous research, greater diversity among staff could be helpful in promoting understanding of cultural differences and reducing bias (Irvin, 2002).
Causal effects of exclusion. The present research has addressed the lack of certainty about the genuine causal effects of exclusion. A review of previous research suggests that exclusion could be a risk factor for a set of negative outcomes already described in the above paragraphs. At the same time, an overview of previous criminological theories suggests that punishment could present different effects on different individuals. The truth is that the causal effect of punishment on students’ behaviour is still a long way from being fully understood. If school exclusion is simply a marker of an underlying antisocial syndrome, it could be beneficial to invest in prevention programmes targeting, for example, deviant behaviours (e.g., violence, drug use, crime, abuse, and neglect) and personality features (e.g., aggressiveness, lack of empathy, lack of remorse) associated with antisocial syndrome (Farrington, 2003). However, if school exclusion is proved to be the cause of detrimental outcomes later in life, it will be worth investing in more programmes focusing specifically on the reduction of school exclusion.
Mediating mechanisms that explain reductions/increases in exclusion rates. Future evaluations of school‐based interventions aimed at reducing exclusion need to explore the presumed causal mechanisms that lead to that reduction. More theoretically informed trials that examine causal mechanisms are important to design better interventions. In fact, this information would be crucial in planning future prevention programmes since causal mechanisms can shed light on what works for whom and under what conditions.
Recognise the key role of statistical power. Future research could benefit from prospective power analysis. That means that, during the early stages of the evaluation's design, there must be consideration given to the sample size required to detect an effect. This early effort can protect studies from “underpowered or wastefully overpowered” samples which can affect outcomes by Type I and Type II errors (Ellis, 2010). Future meta‐analysis should also probably give more attention to this issue and, as suggested by Farrington et al., (2016), it seems advantageous to set a minimum sample size for inclusion in reviews.
Tackle the challenges of running randomised controlled trials in school settings. Research conducted within schools often struggles to isolate experimental and control groups, with the subsequent risk of contamination. As stated earlier in this report, the use of cluster randomised controlled trials can present an opportunity for tackling this when conducting research in educational settings.
Attempt blind assessment in school‐based randomised controlled trials. The characteristics of schools make it hard to blind all those involved in trials. Teachers, students and school counsellors are likely to be involved in the experiment, making it difficult to control the social desirability effect of those participating in the study (Hutchison & Styles, 2010). This challenge needs to be addressed by future studies by at least blinding those who collect data. It also seems necessary, at the level of meta‐analytical studies, that the tools used to measure quality bias in school contexts be adapted. In our experience, some of the available “risk of bias tools” seemed more suited for medical trials than for school‐based experiments.
Risk of quality bias. Meta‐analysis integrates the quantitative evidence from different but related studies to summarise a whole body of knowledge on a research question. As in the present review, meta‐analysis can answer the question about the effectiveness of a given intervention. By the fact that meta‐analysis combines results from different primary research reports, quality bias involved in primary research can jeopardise the validity of the “meta” results. In brief, quality bias refers to systematic error, “meaning that multiple replications of the same study will produce different effect estimates because of sampling variation even if they would give the right answer on average” (Higgins & Green, 2011, p.188).

As stated by Higgins & Green (2011, p. 189) “more rigorous studies are more likely to yield results that are closer to the truth”. Differences in the quality of performed studies can result in false positive conclusions when less rigorous studies are biased toward overestimating an intervention's effect. They can also arrive to false negative conclusions in those cases where less rigorous studies are inclined towards underestimating an effect.

To evaluate risk of quality bias, the present study involves an evaluation of each of the included reports. Three categories were used to judge a study report; i) low risk; ii) unclear risk and iii) high risk of bias. When it came to randomisation, we found a small percentage of high‐risk cases. However, it is important to note that a large proportion of studies (59%) were categorised as “unclear risk”. This is because many reports presented succinct descriptions of the randomisation process without detailing the methods of sequence generation for instance. When we tested the quality of allocation procedures, once again, we found a high percentage of studies (54%) reporting minimal details of allocation concealment. Similar findings were observed for baseline equivalence (22% of assessed studies reported limited information), attrition (30% unclear risk) and blindness of the outcome assessment (54%). These findings do not necessarily mean that studies present quality bias, they just inform us that the authors omitted data which is relevant to interpret the results. Future research would benefit from following CONSORT standards. Registration of trials, even if they are PhD theses, would represent a huge benefit not only for the scientific community, but also for those interested in evidence‐based decision making.
Cross‐cultural research. Based on our findings, the evidence so far is largely coming from the United States. More research needs to be done in other countries where school exclusion is an issue. We know that evidence suggesting effective approaches in some countries/cultures will not necessarily have the same effectiveness when translated to different populations. Those making decisions about how to reduce exclusion in their own country need to have access to detailed information addressing their particular needs.
Innovative strategies. More research needs to be conducted on innovative strategies, for example those involving empathy‐based philosophies. This was the basis for the intervention tested by Okonofua et al. (2016, p.5521) included in this review. The intervention is focused on encouraging teachers to adopt an empathetic attitude towards discipline; it is low cost and demonstrates long‐term effects (rates from 9.6% to 4.8%). As stated by the authors, “teachers’ mind‐sets about discipline directly affect the quality of teacher‐student relationships and student exclusions and, moreover, can be changed through scalable intervention”.
Restorative justice programmes. In addition to the above, the debate about strategies for reducing exclusion in schools have recently been enriched by some research suggesting that restorative justice programmes could have a promising impact (Anyon, Gregory, Stone, Farrar, & Downing, 2016; Fronius et al., 2016). Scholars have highlighted the advantages of restorative justice over exclusion by showing that these programmes focus on building peaceful and empathetic relationships, ask for the involvement of all parties in achieving conflict resolution, look at the harm done to those affected by misbehaviour, and promote reintegration rather than exclusion (Drewery, 2004; Hostetler, 2014; Varnham, 2005). However, to date, there is no clear evidence about its effectiveness (Fronius et al., 2016). Empirical research needs to be conducted testing the impact of restorative justice in school settings. Two challenges need to be addressed here. First, restorative programmes need to be implemented following the guidelines and characteristics of the restorative processes. This is not a simple task for schools used to dealing with conflict in a punitive way. Secondly, randomised controlled trials or well‐controlled studies need to test the impact of restorative justice practices in dealing with conflict inside schools, such as bullying or other forms of violence (Sherman & Strang, 2007). A potential impact in the reduction of conflict could lead to a reduction in the use of exclusion or other exclusionary strategies.

8. Information about this review

8.1 REVIEW AUTHORS

Lead review author:

The lead author is the person who develops and co‐ordinates the review team, discusses and assigns roles for individual members of the review team, liaises with the editorial base and takes responsibility for the on‐going updates of the review.

Name: Sara Valdebenito

Title: Doctor

Affiliation: Institute of Criminology, University of Cambridge

Address: Institute of Criminology, Sidgwick Avenue.

City, State, Province or County: Cambridge

Postal Code: CB3 9DA

Country: United Kingdom

Phone: +44 1223 767373 

Email: sv331@cam.ac.uk; sara.valdebenito@gmail.com

Co‐authors:

Name: Manuel Eisner

Title: Professor

Affiliation: Institute of Criminology, University of Cambridge

Address: Sidgwick Avenue, Cambridge CB3 9DA 

City, State, Province or County: Cambridge

Postal Code: CB3 9DA

Country: United Kingdom

Phone: +44 1223 335374 

Email: mpe23@cam.ac.uk

Name: David P. Farrington

Title: Doctor

Affiliation: Institute of Criminology, University of Cambridge

Address: Sidgwick Avenue, Cambridge CB3 9DA 

City, State, Province or County: Cambridge

Postal Code: CB3 9DA

Country: United Kingdom

Phone:+44 1223 767186 

Email: dpf1@cam.ac.uk

Name: Maria M. Ttofi

Title: Doctor

Affiliation: Institute of Criminology, University of Cambridge

Address: Sidgwick Avenue, Cambridge CB3 9DA 

City, State, Province or County: Cambridge

Postal Code: CB3 9DA

Country: United Kingdom

Phone: +44 1223 767186 

Email: mt394@cam.ac.uk

Name: Alex Sutherland

Title: Doctor

Affiliation: RAND

Address: Westbrook Centre, Milton Road

City, State, Province or County: Cambridge

Postal Code: CB4 1YG

Country: United Kingdom

Phone: +44 1223 273 884

Email: asutherl@rand.org

Open in a new tab

8.2 ROLES AND RESPONSIBILITIES

Please give brief description of content and methodological expertise within the review team. The recommended optimal review team composition includes at least one person on the review team who has content expertise, at least one person who has methodological expertise and at least one person who has statistical expertise. It is also recommended to have one person with information retrieval expertise.

Who is responsible for the below areas? Please list their names:

Information retrieval and coding: Mr Aiden Cope and Dr Sara Valdebenito M.
Risk of Bias assessment: Dr Alex Sutherland and Dr Sara Valdebenito M.
Advise in statistical methods and contents: Professor Manuel Eisner, Professor David P. Farrington, Dr Alex Sutherland and Dr Maria M. Ttofi
Statistical analysis and report writing: Dr Sara Valdebenito M.

8.3 ACKNOWLEDGEMENT

We are very grateful of Aiden Cope for his assistance with information retrieval and double coding.

8.4 SOURCES OF SUPPORT

Professor Manuel Eisner and Dr Sara Valdebenito have been awarded a grant by the Nuffield Foundation for conducting the proposed systematic review. Terms and conditions agreed with the sponsor involve the submission of results during 2016.

8.5 DECLARATIONS OF INTEREST

None of the researchers involved in the team present financial interest in this review. None of them have been involved in the development of interventions or systematic reviews on the scope of the present one. Three authors (Dr Sara Valdebenito M, Professor Manuel Eisner and Dr Alex Sutherland) were involved in the London Education and Inclusion Project cluster‐randomised controlled trial (ISRCTN 23244695). The study was designed as an independent evaluation and the authors have no financial or other links to the evaluated programme.

8.6 PLANS FOR UPDATING THE REVIEW

Dr Sara Valdebenito will be responsible for updating the present review every three years.

8.7 AUTHOR DECLARATION

Authors’ responsibilities

By completing this form, you accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group. If an update is not submitted according to agreed plans, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to propose the update to alternative authors.

Publication in the Campbell Library

The Campbell Collaboration places no restrictions on publication of the findings of a Campbell systematic review in a more abbreviated form as a journal article either before or after the publication of the monograph version in Campbell Systematic Reviews. Some journals, however, have restrictions that preclude publication of findings that have been, or will be, reported elsewhere, and authors considering publication in such a journal should be aware of possible conflict with publication of the monograph version in Campbell Systematic Reviews. Publication in a journal after publication or in press status in Campbell Systematic Reviews should acknowledge the Campbell version and include a citation to it. Note that systematic reviews published in Campbell Systematic Reviews and co‐registered with the Cochrane Collaboration may have additional requirements or restrictions for co‐publication. Review authors accept responsibility for meeting any co‐publication requirements.

I understand the commitment required to update a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:

Form completed by:

Dr Sara Valdebenito

Date:

25 September 2017

9. Tables

9.1 CHARACTERISTICS OF INCLUDED STUDIES

The following tables offer a succinct characterisation of each included study. We selected 11 characteristics from our coding which will allow the reader an overview of the study.

1.
Methodological design, clarifying the type of randomised controlled trial (e.g., cluster randomised, matched pairs randomisation).
2.
Characteristics of the participants, involving the size of the sample during randomisation, school grade of participants, percentage of males and predominant ethnicity.
3.
Location of the study, detailing the city and country where the study was implemented.
4.
Brief description of the intervention.
5.
Programme deliverers, those in charge of implementing the programme.
6.
Evaluator role. When the evaluator is only in charge of conducting the trial, we define that as “an independent evaluator.” Ifin addition the researcher delivers or designs the programme, we report that additional role.
7.
Outcomes measured in the study, not only the outcomes we are interested in.
8.
Length of the intervention.
9.
Assessment. This data refers to the measures carried out during the evaluation; for instance, baseline and post treatment, or baseline, post treatment and 12 month follow up.
10.
Attrition. This data provides an estimate of the percentage of reported attrition.
11.
Conflict of Interest Statement (COI). Offers information on the absence/presence of a formal and explicit statement declaring conflict of personal or institutional interest.
12.
Potential Conflict of Financial Interest (CoFI)

1) Allen, Philliber, Herrling, & Kuperminc (1997)

Methods	Randomised controlled trial
Participants	695 ninth through twelfth grade school students Treatment: 14% Male, 68% Black Control: 17% Male, 67% Black
Location	United States (nationwide)
Interventions	Teen Outreach Program. It is a volunteer service programme, designed to prevent teen pregnancy and academic failure by enhancing normative processes of social development. The intervention involves supervised volunteer community service, classroom‐based discussion of service experience and classroom‐based discussions related to social development tasks of adolescence.
Programme deliverers	Teachers
Evaluator role	Unclear
Outcomes measured	School suspension Teen pregnancy Course failure
Length of intervention	One academic year (around 35 weeks)
Assessment	Baseline and post treatment
Attrition	Treatment group=5.3% Control group= 8.4%
COI statement	Not presented
CoFI	Unclear

Open in a new tab

2) Arter (2005)

Methods	Randomised controlled trial
Participants	52 sixth grade school students 100% Male, 72.5% African‐American
Location	Maryland, US
Interventions	Positive Alternative Learning Support (PALS). The programme aims to make the student more competent in the school environment by providing integrated behavioural support, academic support, group counselling and mentoring.
Programme deliverers	School administrator, school psychologist, counsellors, teachers, special education teachers, one volunteer parent and the researcher
Evaluator Role	Delivered the programme
Outcomes measured	Attendance Academic achievement Office referrals Suspension from school
Length of the intervention	18 weeks
Assessment	Baseline and post treatment (immediately after intervention)
Attrition	24%
COI statement	Not presented
CoFI	Possible. The author delivered the intervention although the programme does not appear to be commercially available (p. 39)

Open in a new tab

3) Barnes, Bauza, & Treiber (2003)

Methods	Randomised controlled trial
Participants	45 high school students 71% Male, 100% African American
Location	Richmond County, United States
Interventions	Stress reduction via Transcendental Meditation.
Programme deliverers	Certified instructor
Evaluator role	Independent
Outcomes	Tardy periods Absentee periods Grades  Rule infractions Days suspended
Length of intervention	12 weeks
Assessment	Baseline and during treatment
Attrition	9%
COI statement	Declared
CoFI	Unlikely, independent evaluator.

Open in a new tab

4) Berlanga (2004)

Methods	Randomised controlled trial
Participants	80 eighth grade school students Gender is not clearly reported (mixed). 80% Hispanic
Location	South Texas, United States
Interventions	Grades, Attendance and Behaviour (GAB). In this programme, a cognitive‐behavioural classroom guidance curriculum is combined with supportive and individualized solution‐focused counselling sessions (six sessions).
Programme deliverer	School counsellor
Evaluator role	Independent
Outcomes	Self‐esteem Self‐conception Academic achievements Attendance Office referrals Suspensions
Length	12 weeks
Assessment	Baseline and post treatment
Attrition	21%
COI statement	Not presented
CoFI	Unlikely, independent evaluator (p. 135‐36)

Open in a new tab

5) Bradshaw, Waasdorp, & Leaf (2012)

Methods	Clustered randomised controlled trial
Participants	12,334 students nested in 37 elementary schools 52% Male, 46.1% White (largest ethnic group)
Location	Maryland, United States
Interventions	School‐Wide Positive Behavioural Interventions and Supports (SWPBIS): a programme that targets a systemic change process in a whole school or in a district. SWPBIS aims to reduce students’ behaviour problems by changing staff behaviours and developing systems and supports to meet children's behavioural needs.
Programme deliverers	School staff, teachers and administrators
Evaluator role	Independent
Outcomes	Aggressive and disruptive behaviour Concentration problems Pro‐social behaviour Emotion regulation Office disciplinary referrals Out‐of‐school suspension
Length	Four years of implementation
Assessment	Five times over the course of four years.
Attrition	“Participation rate was consistently high, […] no evidence that missing data was problematic” p. e1140
COI statement	Declared
CoFI	Unlikely, independent evaluator (p. e1136)

Open in a new tab

6) Bragdon (2010)

Methods	Randomised controlled trial
Participants	68 eighth‐grade students 61% Male; 53% African‐American
Location	United States
Interventions	Teach team project. The intervention is a multi‐component drop‐out prevention involving career exploration, a career awareness course and daily check‐in/check‐out monitoring by a school counsellor or a mentor teacher.
Programme deliverers	Counsellor and mentor teachers
Evaluator role	Independent
Outcomes	Academic performance in Mathematics and Language Attendance Disciplinary referrals Suspensions
Length	Nine weeks
Assessment	Baseline, post treatment and follow‐up
Attrition	Results suggest no attrition
COI statement	Not presented
CoFI	Possible. Manuscript suggests that the researcher adapted an intervention based on previous evidence (p. 7‐15).

Open in a new tab

7) Brett (1993)

Methods	Randomised controlled trial (randomisation of classrooms N=6)
Participants	126 seventh grade students. Mixed gender (percentages not given), 100% African‐American
Location	District of Columbia, United States
Interventions	Efficacy, DC is a mentoring programme aimed at motivating and assisting students to excel academically and socially.
Programme deliverers	Eight adult volunteer mentors
Evaluator role	Independent
Outcomes	Absenteeism School suspension Self‐esteem Academic achievements Attitudes towards learning
Duration	12 weeks
Assessment	Baseline and post‐treatment (immediately after the intervention)
Attrition	23%
COI statement	Not presented
CoFI	Unlikely. The study author is not the programme developer, collaborator with the programme developer or licence holder (p.110)

Open in a new tab

8) Burcham (2002)

Methods	Randomised controlled trial
Participants	71 seventh and eighth grade pupils in two schools 56% Male, ethnicity not reported
Location	Kentucky, United States
Interventions	Social Problem Solving Skills Training. Framed in a cognitive behavioural model, the intervention “taught children problem solving skills using modelling, guided practice or applied practice”. It is a manualised intervention of 27 lessons.
Programme deliverer	Two school psychologists, one of them the evaluator
Evaluator role	Delivered the programme
Outcomes	Social and academic competency Self‐concept In‐school and out‐of‐school suspension Disciplinary referrals School tardies
Length of intervention	30 weeks
Assessment	Baseline, post treatment (immediately after intervention) and follow‐up (18 months later)
Attrition	One students missed at post test 43% at follow up
COI statement	Not presented
CoFI	Possible. Researcher seems to be involved in programme development (p. 42) and also delivered part of the intervention (p.53‐4)

Open in a new tab

9) Collier (2002)

Methods	Randomised controlled trial
Participants	60 elementary school students 100% Male, 100% African‐American
Location	Washington, US
Interventions	Pro‐social skills training. Skills involve anger management, the development of interpersonal skills and problem solving techniques.
Programme deliverer	External psychologist
Evaluator role	Delivered the programme
Outcomes	Oppositional behaviour Cognitive problems Hyperactivity ADHD School suspension
Length	Eight weeks
Assessment	Baseline and post treatment
Attrition	15% (51 remain)
COI statement	Not presented
CoFI	Possible. Author delivered and tested the intervention (p.69). Programme seems not to be commercially available.

Open in a new tab

10) Cook et al. (2014)

Methods	Randomised controlled trial
Participants	106 ninth to tenth grade school students 100% Male, 95% Black
Location	Chicago, US
Interventions	Two interventions to be tested. Becoming a Man (BAM) is a non‐academic intervention, which exposes youth to pro‐social adults and provides social‐cognitive skill training. The Match Model is an academic intervention, providing intensive individualised instruction by tutors.
Programme deliverer	College educated individuals
Evaluator role	Unclear. “Intervention was delivered by staff hired by our own research team” (p.11)
Outcomes	Maths achievements Reading achievements Discipline incidents Days absent Out‐of‐school suspension Participation in extra‐curricular activities
Length	BAM: 27 weeks during one academic year Match‐Model: daily, one academic year
Assessment	Baseline and post treatment
Attrition	Author recognise attrition. Use of Multiple Imputation and ITT analysis
COI statement	Not presented
CoFI	Unclear

Open in a new tab

11) Cornell, Allen, & Fan (2012)

Methods	Randomised controlled trial
Participants	201 school students (K‐12) 73% boys, 73% African‐American
Location	Virginia, US
Intervention	Virginia Student Threat Assessment Guidelines is a problem‐solving approach to violence prevention aimed at resolving conflict and working out a solution that allows the student to continue in school.
Programme deliverer	School administrator, a law enforcement officer or school resource officer, and one or more mental health professionals
Evaluator role	Designed the programme
Outcomes	Long‐term school suspension Access to counselling Alternative school placement Parent conference Victim's parents notified
Length	Not detailed
Assessment	Baseline and post treatment
Attrition	Results suggest no attrition
COI statement	Declared. Main author designed the Virginia student threat assessment guidelines.
CoFI	Possible. See p.100, footnote.

Open in a new tab

12) Crowder (2001)

Methods	Randomised controlled trial
Participants	109 seventh grade school students 45% Male, 99% Black
Location	United States
Interventions	Gang resistance, education and training (GREAT). Intervention involves leadership skills development and a character education curriculum. The curriculum involved drug resistance, gang resistance, and non‐violent confrontational skills among others.
Programme deliverer	Police Officer, assisted by two teachers
Evaluator role	Independent evaluator
Outcomes	Out‐of‐school suspension In‐school suspension Office referrals Tardies Unexcused absences Students achieving honour roll
Length	Nine weeks
Assessment	Baseline and post treatment
Attrition	Results suggest no attrition
COI statement	Not presented
CoFI	Unlikely. Researcher was a PhD student who asked permission to run an evaluation implemented by the Metropolitan Police Service (see appendix B).

Open in a new tab

13) Dynarski et al. (2003)

Methods	Randomised controlled trial
Participants	968 elementary school students 46.4% Male, 67% Black
Location	United States
Interventions	21^st Century Community Learning is an after‐school programme delivered in a public school building providing academic support and recreational/cultural activities.
Programme deliverer	Teachers
Evaluator role	Independent
Outcomes	Achievements Levels of effort School suspension Absenteeism Tardies Parental supervision After school activities
Length	One academic year
Assessment	Baseline, post treatment and one‐year follow‐up
Attrition	11%
COI statement	Not declared
CoFI	Unlikely. See Dynarski et al., 2004, p. XV

Open in a new tab

14) Edmunds et al. (2012)

Methods	Randomised controlled trial
Participants	1607 ninth grade school students 41.4% Male, 60.2% White
Location	North Carolina, US
Interventions	Early College High School targets students underrepresented in college (i.e., low income, the first in their family to go to college or members of a minority group). It is a personalised programme designed to boost the academic progress of students, minimising barriers between high school and college.
Program deliverers	Teachers, school staff.
Evaluator role	Independent
Outcomes	Course taking patterns and success Suspension Absences Planning to attend four‐year college Algebra I College prep. maths courses English
Length	Not clearly stated
Assessment	Baseline and post treatment. Unclear how many months or years after intervention.
Attrition	4.8%
COI statement	Not presented
CoFI	Unlikely.

Open in a new tab

15) Farrell, Meyer, & White (2001)

Methods	Clustered randomised controlled trial
Participants	626 sixth grade school students, nested in 27 classrooms 50.2% Male, 96% Black
Location	Richmond, Virginia, US
Interventions	Responding in Peaceful and Positive Ways (RIPP) is a universal violence prevention program. “The goal of RIPP is to increase adolescents’ capacity and motivation to respond to developmental challenges in ways that facilitate social skill acquisition and acceptance of personal responsibility” (p.452).
Programme deliverers	Prevention specialists
Evaluator role	Designed the programme
Outcomes	Disciplinary violations for violence In‐school suspensions Out‐of‐school suspensions Violent behaviour frequency Drug use frequency RIPP knowledge test Problem situation inventory Attitudes toward nonviolent behaviour Attitudes toward violent behaviour
Length	25 weeks
Assessment	Baseline, post‐treatment, six‐month follow up and 12‐months follow‐up.
Attrition	Yes. Attrition affected both groups. ITT analysis.
COI statement	Not declared
CoFI	Possible. See p. 452

Open in a new tab

16) Feindler, Marriott, & Iwata (1984)

Methods	Randomised controlled trial
Participants	36 junior high school students Gender and ethnicity not reported
Location	Not reported
Interventions	Anger Control Training is based on a cognitive behavioural model. Training sessions focus on the components of the provocation cycle, self‐monitoring skills, self‐control, problem solving, time‐out responses, relaxation techniques, and appropriate verbal and non‐verbal assertive responses.
Programme deliverers	Trained therapist and programme aide
Evaluator role	Unclear
Outcomes	Problem solving Self‐control School expulsion
Length	Seven weeks
Assessment	Baseline, during treatment and five‐week follow‐up
Attrition	Results suggest no attrition
COI statement	Not declared
CoFI	Unclear

Open in a new tab

17) Harding (2011)

Methods	Randomised controlled trial
Participants	48 eighth grade school students 60.4% Male, 97.9% White
Location	East England, United Kingdom.
Interventions	Over to You is a programme based in a cognitive behavioural approach. It encourages behavioural change as well as offering training in social skills enhancement (e.g., self‐awareness, decision‐making, empathy, conflict resolution).
Programme deliverer	Educational psychologist, trained member of the school staff
Evaluator role	Delivered the programme
Outcomes	Emotional literacy Behaviour (SDQ) School exclusion
Length	Six weeks
Assessment	Six‐months follow‐up
Attrition	8%
COI statement	Not declared
CoFI	Possible. The researcher implemented the intervention (p. 97‐107)

Open in a new tab

18) Hawkins, Doueck, & Lishner (1988)

Methods	Randomised controlled trial
Participants	160 seventh grade school students (low achievers) 47% Male, ethnicity not reported.
Location	Seattle, US.
Interventions	Proactive Classroom management can be defined as a package of instructional methods (interactive teaching and co‐operative learning) in mainstream schools. Improved instruction is aimed at benefitting low achievers by improving behaviour, attitudes and academic results.
Programme deliverer	Teachers
Evaluators role	Independent
Outcomes	Students social bonding to school Students achievements Expectations and aspirations for education Antisocial behaviour Suspension and expulsion from school
Length	One academic year
Assessment	Baseline and post treatment
Attrition	11%
COI Statement	Not declared
CoFI	Unlikely

Open in a new tab

19) Hirsch, Hedges, Stawicki, & Mekinda (2011)

Methods	Randomised controlled trial
Participants	535 High school students 41% Male, 76% African‐American
Location	Chicago, United States.
Interventions	After School Matters (ASM) is a programme offering paid apprenticeship‐type experiences. Instructors provide information, guidance and feedback, and introduce students to the standards, language and culture of that line of work. The experience helps students begin to appreciate and adapt to the culture of the workplace and improve the “soft skills” increasingly demanded by employers. Intervention takes place in the host high school.
Programme deliverer	Instructor and school members
Evaluator role	Independent
Outcomes	Positive youth development  Marketable job skills  Academic outcomes  Problem behaviour School suspension
Length	20 weeks
Assessment	Baseline and post treatment
Attrition	18%
COI statement	Not declared
CoFI	Unlikely. See acknowledgments section.

Open in a new tab

20) Hostetler & Fisher (1997)

Methods	Randomised controlled trial
Participants	317 fourth grade school students and their families. 57.4% Male, 33.8% Caucasian
Location	Lancaster, US.
Interventions	Project CARE, a substance abuse prevention program, aimed at improving students’ skills (problem solving, peer resistance) and parents’ skills.
Programme deliverer	Psychologist and prevention specialist
Evaluator role	Independent
Outcomes	Negative behaviours Intent to use substances Alternative activities Communication with parents Substance use School absences, school suspensions, school grades
Length	One school year plus a summer, one session per week
Assessment	Baseline, post treatment, 12‐months follow‐up, 21‐months follow‐up
Attrition	40% (12 months follow‐up) 75% (21‐months follow‐up)
COI statement	Not declared
CoFI	Unlikely

Open in a new tab

21) Ialongo, Poduska, Werthamer, & Kellam (2001)

Methods	Clustered randomised controlled trial
Participants	678 elementary school students, nested in 27 classrooms (nine schools) 53% Male, 86.8% African‐American
Location	Baltimore, US.
Interventions	1) Classroom centred intervention (CC) is aimed at improving classroom management. It involves three main components: curriculum enhancement, enhanced behaviour management practices and back‐up strategies for children not performing adequately. 2) Family School Partnership (FSP) is anintervention targeting improvement in parent‐teacher communication and parents’ child behaviour management strategies.
Programme deliverers	CC is delivered by teachers FSP is delivered by teachers plus school psychologist or school social worker
Evaluator role	Designed the programme
Outcomes	Conduct problems in school School suspension Academic achievements Mental health needs
Assessment	Baseline and five‐year follow‐up. There was a post treatment measure but school suspension was not measured.
Attrition	24%
COI statement	Not declared
CoFI	Possible. Researcher seems to be the programme designer (p.602)

Open in a new tab

22) Johnson (1983)

Methods	Randomised controlled trial
Participants	60 seventh and eighth grade school students Gender and ethnicity not reported
Location	Washington, US
Interventions	Project A.T.T.E.N.D. (Alternatives To Trouble Encouraging New Directions). An education programme targeting self‐responsibility for maintaining school discipline. An alternative to punitive disciplinary methods that combines supervision of attendance, behaviour and classwork as well as counselling.
Program deliverer	School counsellor
Evaluator role	Design and delivery
Outcomes	Attendance (absences and tardies) Suspension Number of misbehaviour referrals Grades
Length	Nine weeks
Assessment	Baseline and post treatment
Attrition	No attrition reported
COI statement	Not declared
CoFI	Possible. Researcher is programme developer and deliverer (p.7)

Open in a new tab

23) Lewis et al. (2013)

Methods	A matched‐pair, clustered randomised controlled trial
Participants	624 elementary school students, nested in 14 schools 47% Male, 48% African‐American
Location	Chicago, United States
Interventions	Positive Action programme includes a K‐12 classroom curriculum involving six components: self‐concept, social and emotional positive actions for managing oneself responsibly, positive actions directed toward physical and mental health, honesty, getting along with others, and continually improving oneself. The programme also includes teacher, counsellor, family, and community training as well as activities directed toward school‐wide climate development.
Programme deliverer	School facilitators
Evaluator role	Design the programme
Outcomes	Support of aggression Bullying Disruptive behaviours Violence Disciplinary referrals Suspension
Length	One year
Assessment	Baseline (2004), post treatment (2005) plus multiple follow‐ups.
Attrition	5% post test
COI statement	Declared
CoFI	Likely. “The research described herein was done using the program, the training, and technical support of Positive Action, Inc., in which Dr. Flay's spouse holds a substantial financial interest. Issues regarding conflict of interest were reported to the relevant institutions and appropriately managed following the institutional guidelines.” (p.629)

Open in a new tab

24) Mack (2001)

Methods	Randomised controlled trial
Participants	20 fourth to six grade school students 50% Male, 100% African‐American
Location	Alabama, US
Interventions	ICAN Kids! Control Anger Now and Skills for Living curriculum. Abehavioural group counselling intervention programme involving techniques such as behaviour contracts, relaxation, role‐play and modelling.
Programme deliverer	Counsellor
Evaluator role	Design and delivery
Outcomes	Disciplinary referrals School Suspension
Length	Six weeks
Assessment	Baseline, during treatment, post treatment and three weeks follow‐up.
Attrition	No attrition reported
COI statement	Not presented
CoFI	Possible. See p. 110

Open in a new tab

25) Obsuth et al. (2016)

Methods	Clustered randomised controlled trial (minimisation)
Participants	738 secondary school students, nested in 36 schools 71% Male, 40.3% Black‐Caribbean/Black‐African
Location	London, United Kingdom
Interventions	Engage in Education – London (EiE‐L). The intervention targeted their social communication and broader social skills with the aim of reducing school exclusions and problem behaviours.
Programme deliverer	Core worker trained to deliver the intervention
Evaluator role	Independent
Outcomes	School exclusion Interpersonal communication Pro‐social skills Student‐teacher relationship Antisocial behaviour Bullying perpetration Delinquency Arrests
Length	12 weeks
Assessment	Baseline and post treatment
Attrition	12% (based on official records)
COI statement	Declared
CoFI	Unlikely. Independent evaluation.

Open in a new tab

26) Okonofua, Pauneskua, & Walton (2016)

Methods	Clustered randomised controlled trial
Participants	1682 middle school students, 31 teachers 48% Male, 54% Latino
Location	California, United States
Interventions	Empathic Discipline is a brief intervention (i.e., online training, low cost) aimed at encouraging teachers to adopt an empathic mind‐set about school discipline. The intervention encourages teachers to understand the values and perspectives of students which can cause misbehaviour. Teachers are empowered to handle difficult interactions, in a context of mutual understanding and trust. Empathic discipline is seen as an alternative to punitive, zero‐tolerance practices.
Programme deliverer	Teachers
Evaluator role	Unclear
Outcomes	School suspension Respect to teachers
Length	One year
Assessment	Baseline and post treatment
Attrition	13%
COI statement	Declared
CoFI	Unclear

Open in a new tab

27) Panayiotopoulos & Kerfoot (2004)

Methods	Randomised controlled trial
Participants	124 secondary school students 90% Male. Most of the students were White British. No percentage reported.
Location	Manchester, United Kingdom
Interventions	Home and School Support Project (HASSP)is delivered by a multidisciplinary team offering rapid assessment and a treatment plan for the child, family and school staff. Treatment involves family therapy, individual CBT therapy and school support.
Programme deliverer	Social worker, psychologist, community psychiatric nurse, play therapist
Evaluator role	Independent
Outcomes	School exclusion Disruptive antisocial behaviour and emotional symptoms
Length	Unclear
Assessment	Baseline, three months post‐treatment
Attrition	6%
COI statement	Not declared
CoFI	Unlikely. Intervention was delivered by community services (p.110)

Open in a new tab

28) Peck (2006)

Methods	Randomised controlled trial
Participants	1050 fifth through eighth grade school students 59% Male, 62.5% Hispanic
Location	United States
Interventions	Student Targeted with Opportunities for Prevention (STOP program). It is a crime prevention programme offering a tutorial component, family and individual counselling, gang education as well as a drug and alcohol counselling intervention. The aim of the programme is that students will enter high school having what it takes to be a successful student and community member.
Programme deliverers	Probation Officer, community agencies
Evaluator role	Independent
Outcomes	School grades School attendance School suspension School expulsion Alcohol and drug use Contacts with the Juvenile Justice System
Length	One year
Assessment	Follow‐up
Attrition	39%
COI statement	Not presented
CoFI	Unlikely. Programme was implemented by the probation service in collaboration with public agencies (p. 41‐3)

Open in a new tab

29) Reese, Murphy, & Filipczak (1981)

Methods	Randomised controlled trial
Participants	98 seventh through ninth grade school students Gender is not reported, 100% Black
Location	United States
Interventions	Preparation through Responsive Education Programs (PREP). The intervention is designed to improve academic performance and social skills. Intervention involves reinforcement, and teaching adaptive behaviour as well as skills for self‐control.
Program deliverer	Teachers
Evaluator role	Unclear
Outcomes	School grades School attendance School suspension School behaviour
Length	27 weeks
Assessment	Baseline, during treatment
Attrition	Unknown
COI statement	Not presented
CoFI	Unlikely

Open in a new tab

30) Russell (2007)

Methods	Randomised controlled trial
Participants	61 sixth grade school students 63.5% Male, 83% Caucasian
Location	Oregon, US
Interventions	Abbreviated version of Coping Power (CP)attempts to improve a child's social competence, self‐regulation, self‐control and social bonds with peers, teachers and caregivers. CP incorporates individual counselling sessions, weekly group meetings and monthly parent meetings.
Programme deliverer	Psychologists, master level clinicians
Evaluator role	Independent evaluator
Outcomes	Adaptive functioning School referrals Detentions Suspensions
Length	24 weeks
Assessment	Post treatment
Attrition	14%
COI statement	Not presented
CoFI	Unlikely. Researcher is not programme developer, there is no evidence that the programme is commercialised.

Open in a new tab

31) Shetgiri, Kataoka, Lin, & Flores (2011)

Methods	Randomised controlled trial
Participants	108 ninth grade school students Experimental group: 51% Male, 81% Latino Control group: 33% Male, 75% Latino
Location	California, United States
Interventions	School‐based programme to reduce violence and substance use. Originally designed for White and African‐American youths, the intervention is group based and aimed at increasing resilience through skills enhancement sessions (e.g., anger management, conflict resolution) and counselling sessions.
Programme deliverer	School clinical social worker
Evaluator role	Independent
Outcomes	Smoking Alcohol and drug use Fighting Grades Suspension Expulsion
Length	28 weeks
Assessment	Baseline and eight month follow‐up
Attrition	20%
COI statement	Not presented
CoFI	Unlikely

Open in a new tab

32) Smith (2004)

Methods	Randomised controlled trial
Participants	40 eleventh to twelfth grade school students (Hispanic‐serving schools) 60% Male, 65% Hispanic‐Latino
Location	Texas, United States
Interventions	The Personal Responsibility Group consists of instructional activity sessions on Emotional Intelligence skills (e.g., assertiveness, comfort and rapport, empathy, time‐management, anxiety reduction, and motivation). The aim of the intervention is to assist students in identifying EI strength and areas of growth and positive change.
Programme deliverer	Unknown
Evaluator role	Delivered the programme
Outcomes	Personal responsibility Constructive thinking Expectative Grades School behaviour In‐school and out‐of‐school suspension
Length	10 weeks
Assessment	Baseline and post treatment
Attrition	No attrition
COI statement	Not presented
CoFI	Possible. Researcher delivered the intervention (p. 285)

Open in a new tab

33) Snyder et al. (2010)

Methods	Cluster randomised controlled trial
Participants	544 students nested in 20 elementary schools Gender is not reported, 30% Part Hawai'i an
Location	Three Hawaiian Islands, US.
Interventions	Positive Action programme includes a K‐12 classroom curriculum involving six components: self‐concept, social and emotional positive actions for managing oneself responsibly, positive actions directed toward physical and mental health, honesty, getting along with others, and continually improving oneself. The programme also includes teacher, counsellor, family, and community training as well as activities directed toward school‐wide climate development.
Programme Deliverer	Teachers, school administrators, school staff, counsellors
Evaluator role	Designed the programme
Outcomes	Absenteeism Suspension Retention in grades School achievements
Length	35 weeks
Assessment	Baseline, post treatment, one year follow‐up
Attrition	Unclear
COI statement	Declared
CoFI	Likely. “The research described herein was done using the program and the training and technical support of Positive Action, Inc. Dr. Flay's spouse holds a significant financial interest in Positive Action, Inc.” (p.50)

Open in a new tab

34) Sprague, Biglan, Rusby, Gau, & Vincent (2016)

Methods	Cluster randomised controlled trial
Participants	13,498 nested in 35 middle schools No data on gender; ≈70% White
Location	Oregon, US.
Interventions	School‐Wide Positive Behavioural Interventions and Supports (SWPBIS) refers to a programme that targets a systemic change process in a whole school or in a district. SWPBIS aims to reduce students’ behaviour problems by changing staff behaviours and developing systems and supports to meet children's behavioural needs.
Programme Deliverer	School staff, teachers and administrators
Evaluator role	Independent
Outcomes	In‐school suspension, out‐of‐school suspension and expulsion School level achievements Antisocial behaviour and aggression Safety Substance use Relationship with teacher Attachment to school rules
Length	One year
Assessment	Baseline, post treatment and 12‐month follow‐up
Attrition	Unclear
COI statement	Not declared
CoFI	Unlikely

Open in a new tab

35) Tilghman, (1988)

Methods	Randomised controlled trial
Participants	100 sixth to eighth grade school students Gender and ethnicity are not reported
Location	United States
Interventions	Counsellor Peers are “students who have been properly trained through a certified peer counselling programme coordinator to work with and listen their peers. Rather than being an “advice‐giver” or “problem‐solver”, peer counsellors are sensitive listeners trained in communication skills to help their peers through the process of decision making and self‐exploration.” (p.10)
Programme deliverer	Trained students
Evaluator role	Evaluator trained the counsellor peers
Outcomes	Attitude towards school, themselves and peers Suspension
Length	Nine weeks
Assessment	Unclear
Attrition	9%
COI statement	Not presented
CoFI	Possible. Researcher was involved in the selection and training of counsellors. It is regarded as a limitation in the study (p. 9). No evidence that the programme is commercialised.

Open in a new tab

36) Ward & Gersten, (2013)

Methods	Cluster randomised controlled trial
Participants	32 elementary schools; 7500 students Gender is not reported. Ethnic minorities represent ≈80%
Location	United States
Interventions	Safe and Civil Schools (SCS) is a school‐wide intervention promoting positive behaviour and support. “Program deliverers receive training on how to implement improvements related to safety, behaviour and discipline”. (p.320)
Programme deliverer	School administrator, three general education teachers, one special education teacher and one or two other staff.
Evaluator role	Independent
Outcomes	Suspension Achievements test scores Resiliency, protective factors, and risk behaviours Opinion on the level of implementation of the school‐wide intervention
Length	One year
Assessment	Post treatment and one‐year follow‐up
Attrition	Unclear
COI statement	Not presented
CoFI	Unlikely. Programme was developed by Sprick et al.1992 for profit but evaluator seems to be independent.

Open in a new tab

37) Wyman et al. (2010)

Methods	Cluster randomised controlled trial (randomised within classrooms)
Participants	226 kindergarten to third grade school students 54% Male, more than 60% Black
Location	Rochester City, US
Interventions	Rochester Resilience Program: “In 14 lessons with school‐based mentors, children were taught a hierarchical set of skills: monitoring of emotions; self‐control/reducing escalation of emotions; and maintaining control and regaining equilibrium. Mentors provided classroom reinforcement of skill use”. (p. 707)
Program deliverer	Resilience Mentors
Evaluator role	Design the programme
Outcomes	Classroom behaviour Social‐emotional functioning Office disciplinary referrals Out‐of‐school suspension
Length	14 weeks
Assessment	Baseline and post treatment
Attrition	13%
COI statement	Not presented
CoFI	Possible. Main author developed the programme (p.709). No information about commercialisation of the programme.

Open in a new tab

9.2 RISK OF BIAS IN INCLUDED STUDIES BASED ON EPOC RISK OF BIAS TOOL (PER STUDY)

Study	Sequence generation	Concealment	Equivalence (outcome)	Equivalence (Characteristics)	Incomplete data	Blinding	Contamination	Selective outcome reporting
Allen et al. 1997	HR	UR	HR	HR	LR	HR	HR	LR
Arter, 2005	UR	UR	UR	HR	UR	UR	HR	LR
Barnes et al. 2003	HR	UR	HR	LR	HR	UR	LR	LR
Berlanga, 2004	UR	UR	HR	HR	UR	HR	HR	LR
Bradshaw et al. 2012	LR	LR	LR	LR	LR	UR	LR	LR
Bragdon, 2010	UR	UR	HR	HR	UR	UR	HR	LR
Brett, 1993	UR	LR	UR	HR	HR	UR	HR	LR
Burcham, 2002	LR	LR	LR	UR	HR	UR	HR	LR
Cook et al. 2014	LR	LR	LR	LR	LR	UR	HR	LR
Collier, 2002	UR	UR	LR	LR	LR	HR	HR	LR
Cornell et al. 2012	UR	UR	LR	LR	LR	UR	LR	HR
Crowder, 2001	UR	HR	UR	UR	LR	UR	HR	LR
Dynarski et al. 2003	UR	LR	LR	LR	UR	UR	HR	LR
Edmunds et al. 2012	HR	UR	UR	LR	UR	UR	HR	LR
Farrell et al. 2001	LR	UR	HR	LR	LR	LR	HR	LR
Feindler et al. 1984	UR	UR	LR	HR	LR	LR	HR	LR
Harding, 2011	UR	UR	UR	HR	HR	HR	HR	LR
Hawkins et al. 1988	UR	UR	LR	LR	UR	HR	HR	LR
Hirsch et al. 2011	LR	LR	LR	LR	LR	UR	HR	LR
Hostetler & Fisher 1997	UR	UR	UR	HR	HR	HR	HR	LR
Ialongo et al. 2001	UR	UR	LR	LR	UR	HR	HR	LR
Johnson, 1983	LR	LR	LR	LR	LR	HR	HR	LR
Lewis et al. 2013	LR	LR	UR	LR	LR	HR	LR	LR

Open in a new tab

9.6 EXTRACTED DATA FOR EFFECT SIZE CALCULATIONS: PRIMARYOUTCOME

Author	Type of Study	Sample	Mean ³⁵ Age (SD)	School‐based Programme	Universal/ Indicated ³⁶	% FSM	Cluster ³⁷	Extracted data for effect size calculations	Effect size calculation	Measure of exclusion
1) Allen et al. (1997)	Journal article	695	15.8 (1.13)	Teen outreach	Unclear	Unknown	No	Suspension Baseline T1: 58 cases (17%); N=342 C1: 81 cases (23.8%); N=353 Post treatment (immediately after treatment) T2: 42 cases (13%); N=324 C2: 93 cases (28.7%); N=323	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75. See methods section for further details	Presence/absence
2) Arter, (2005)	Journal article	52	Secondary school	Positive Alternative Learning Support (PALS)	Indicated	40% FSM	No	Suspensions No baseline reported Post Treatment (presumably after treatment) T2: M=.675; SD=.194; N=23 C2: M= 675; SD=.227; N=17	SMD was calculated using equations 3 and 4 in the methods section.	Nº of days
3) Barnes et al. (2003)	Journal article	45	16 (1.3)	Stress reduction	Universal	Unknown	No	Suspension Baseline T1: M=0.8 days; SD=1.8; N=23 C1: M=0.0 days; SD=.0; N=18 During intervention T2(during): M= 0.5 days; SD=1.2; N=23 C2(during): M=1.2 days; SD=3.0 N=18	The principal investigator provided N size for T2. SMD was calculated as the difference between time 1 and time 2, accounting for the covariation between pre‐ and post measures (equations 8 and 9 in the methods section).	Nº of days
4) Berlanga (2004)	PhD Thesis	80	Eighth grade	Grades, Attendance and Behaviour (GAB)	Indicated	Unknown	No	In School Suspension Baseline T1: M=1.15; SD=1.29; N=32 C1: M=.61; SD=1.14; N=31 Post Intervention (immediately after treatment) T2: M=1.12; SD=1.21; N=32 C2: M=1.03; SD=1.88; N=31 Suspension Baseline T1: M=.31; SD=.53; N=32 C1: M=.35; SD=.83; N=31 Post Intervention(immediately after treatment) T2: M=.34; SD=.90; N=32 C2: M=.58; SD=1.11; N=31 Removal/Expulsion Baseline T1: M=.06; SD=.24; N=32 C1: M=.06; SD=.24; N=31 Post Intervention (immediately after treatment) T2: M=.03; SD=.17; N=32 C2: M=.16; SD=.45; N=31	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal. 75. See methods section for further details	Nº of events
5) Bradshaw et al. (2012)	Journal article	12,334	Elementary school	School‐Wide Positive Behavioural Interventions and Support (SWPBIS)	Universal	49% FSM	Yes	Out‐of‐school suspension Follow‐up (four years) Student level Control: 21 schools (N=5124) Treatment: 16 schools (N=6614)OR= .73; 95% CI .59 and .91	The study reports results using multi‐level analysis. In this case, we have not applied any correction of standard errors. We assume that MLM accounted for clusters and subsequently corrected the bias (see p. e1140). See methods section for further details	Nº of events
6) Bragdon (2010)	PhD Thesis	68		Teach Team Project	Indicated	49% FSM	No	Suspension Baseline T1:M=.06; SD=.34; N= 34 C1:M=.96; SD=2.42; N=34 Post treatment (during) T2:M=.03; SD=.17; N= 34 C2:M=1.10; SD=2.61; N=34 Follow‐up (three months later) T3:M=.09; SD=.51; N=34 C3:M=1.07; SD=3.19; N=34	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal. 75. See methods section for further details.	Nº of days
7) Brett (1993)	PhD Thesis	126	12‐14 years	Efficacy, DC	Universal	Unknown	Yes	School Suspension Clusters: 3 control (n=66) 3 Experimental (n=60) Post‐treatment (1 Month) T2: M=.53; SD= 1.02; N=40 C2: M=.63; SD=1.21; N=57;	Based on Hedges (2007) and Spier et al. (2013), effect sizes were computed using d_T2 , assuming equal cluster sample size, ρ=.05. See methods section for further details	Nº of events
8) Burcham (2002)	PhD Thesis	71	Middle school	Social problem solving skills training	Indicated	38% FSM	No	In‐School Suspension‐Baseline T1:M=8.62; SD=6.44; N=37 C1:M=7.88; SD=4.47; N=32 p=.58 ‐Immediately after treatment T2: f=.18; p= 0.67; N=69 ‐18 months after treatment T3: f=.04; p=0.84; N=38 Out‐of‐school Suspension Baseline T1: M=3.22; SD=3.71; N=37 C1: M=2.56; SD=3.40; N=32 p=.45 Immediately after treatment T2: f=1.09; p=.30; N=69 18 months after treatment T3: f= 1.83; p=.18; N=38	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal. 75. See methods section for further details	Nº of events
9) Collier (2002)	PhD Thesis	60	5‐14 years	Pro‐social skills training	Indicated	Unknown	No	School Suspension (+) Baseline T1: M=1.93; SD=.4498; N=26 C1: M=1.86; SD=.5074; N=25 Post treatment (presumably after treatment) T2:M=1.15; SD=.6748; N=26 C2:M=2.16; SD=.3742; N=25	Study was identified as an outlier value. It was winsorised as suggested by Wilson & Lipsey 2001. Effect size was calculated as the difference between baseline and post treatment. We corrected final calculation by adding the value of pre/post correlation, assumed to be equal .75.	Nº of days
10) Cook et al. (2014)	Technical report	106		BAM (skills‐training) and MATCH (tutoring)	Indicated	26% FSM	No	Baseline data is reported but incomplete Out‐of‐school suspension (ITT) b=‐.642; SE=.501; (unclear the number of months/weeks of post treatment measured)	Data was entered into CMA by using the option Log OR and its SE. No further corrections. Even if the evaluation was testing two different interventions, data was reported in a composite measure. The original author took that option because they recognised contamination between groups (spill‐over).	Nº of events
11) Cornell et al. (2012)	Journal article	201		Threat assessment	Indicated	Unknown	Yes	Long‐term suspension Post treatment (presumably after treatment) T2: 25 (25%); N=100 C2: 49 (49%); N=101	No corrections	Nº of events
12) Crowder (2001)	PhD Thesis	109		Gang Resistance, Education and Training (GREAT)	Unclear	Unknown	No	Out of School Suspension No baseline measure reported Post intervention (presumably after treatment) T2: M=.1329; SD=.4629; N=53 C2: M=.1429; SD=.1610; N=56 In School Suspension No baseline measure reported Post intervention (presumably immediately after treatment) T2: M=.3584; SD=.7464; N=53 C2: M=.4464; SD=.8464; N=56	SMD was calculated using equations 3 and 4 in the methods section. No further corrections.	Nº of events
13) Dynarski et al. (2003:2004)	Technical report	968	Elementary school	21^st Century Community Learning	Unclear	Unknown	No	Post treatment 12 months (2003) T2: 7.1% (38) N=537 C2: 5.2% (16) N=317 Follow up 24 months T3: 60 (6.2%) N=537 C3: 43 (4.4%) N=317	Data was entered into CMA by using a 2x2 table. No further corrections.	Nº students
14) Edmunds et al. (2012)	Journal article	1607	15.3	Early College High School Academic skills enhancing	Unclear	50.6%	No?	% Suspended at least once T2: 6.4%; (57) N=885 C2: 13.3%; (86) N=644	The principal investigator provided measures for effect size calculation (via mail communication).	Nº of events
15) Farrell et al. (2001)	Journal article	626	11.7 (0.6)	Responding in Peaceful and Positive Ways (RIPP)	Universal	Unknown	Yes	In‐school suspension Post intervention (immediately after treatment) OR=5.0 (95%CI 1.5; 17.1) 6 months OR=1.4 (95% CI .7; 2.8) 12 months OR=1.4 (95%CI .6; 3.0) Out‐of‐School Suspension Post‐intervention (immediately after treatment) OR=0.9 (95%CI .5; 1.8) 6 months OR=1.1 (95% CI .6; 2.0 12 months OR=0.9 (95% CI .6; 1.4)	Although the study is based on clustered data, we have not applied any correction of standard errors. The author mentions the use of GEE to calculate robust estimates of standard errors (see Farrell et al., 2001, p. 455). See methods section for further details.	Nº of events
16) Feindler et al. (1984)	Journal article	36	13.8 (.68)	Anger control training	Indicated	Unknown	No	School Expulsion Baseline T1: M=1.45; SD=.71; N=18 C1: M=1.40; SD=.44; N=18 Five‐weeks follow‐up T2: M=.77; SD=.29; N=18 C2: M=.1.2; SD=.46; N=18	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculation by adding the value of pre/post correlation assumed to be equal .75. See methods section for further details.	Nº of events
17) Harding (2011)	PhD Thesis	48	Eighth grade	Over to you	Indicated	Unknown	No	Six‐months follow‐up Fixed term exclusion T2: 5 N=20 C2: 5 N=23 Permanent exclusion T2: 1 N=20 C2: 1 N=23	Data was entered into CMA by using a 2x2 table. No further corrections.	Nº of events
18) Hawkins et al. (1988)	Journal article	160	Seventh grade	Proactive Classroom Management	Indicated	Unknown	No	Times Suspended Post‐treatment (presumably after treatment) T2: M=.48; SD=1.3; N=67 C2: M=.89; SD=2.1; N=75	SMD was calculated using equations 3 and 4 in the methods section. No further corrections.	Nº of events
19) Hirsch et al, (2011)	Technical report	535	15.9	After School Matters	Indicated	86%	No	School Suspension Baseline T1: M=1.27; SD=.63; N=259 C1: M=1.29; SD=.64; N=178 Post treatment (immediately after treatment) T2: M=1.36; SD=.72; N=259 C2: M=1.40; SD=.75; N=178	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75. See methods section for further details.	Nº of events
20) Hostetler & Fisher (1997)	Journal article	317	Third grade	Project CARE (Skill for parents and children)	Indicated	Unknown	No	Suspension Baseline T1: M=0.13; SD=0.56; N=151 C1: M=0.07; SD=0.35; N=140 Post treatment (a few months after treatment, no clear specification) T2: M=0.20; SD=0.53; N=155 C2: M=0.25; SD=0.89 N=141 One year follow‐up T3: M=0.26; SD=0.80; N=90 C3: M=0.15; SD=0.66, N=86 Two year follow‐up T4: M=0.27; SD=0.74; N=30 C4: M=0.09; SD=0.29; N=34	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75. See methods section for further details.	Nº of events
21) Ialongo et al. (2001)	Journal article	678	6.20 (.34)	Two interventions i) Classroom‐centred (CC) ii) Family‐school partnership (FSP)	Universal	62.3% FSM	Yes	Suspension Five year follow‐up Classroom‐centred vs. control OR=.73; (95%CI=.56; .95) treatment group less likely to be suspended Family‐school partnership OR=.59 (95%CI .35; .97) Boys: OR=1.13 (95%CI .61; 2.09) Girls: OR=.38 (95%CI .17; .86) treatment group less likely to be suspended	Since data was dichotomous and nested in clusters, we corrected standard errors of effect sizes. The design effect was corrected by using the formula suggested by Higgins & Green (2011) expressed by the equation [1+(M‐1) x1]. See methods section for further details.	Presence/absence
22) Johnson (1983)	PhD Thesis	60	Seventh and eighth grade	ATTEND (Counselling and monitoring)	Indicated	Unknown	No	Suspension Baseline T1: M=.76; SD=.85; N=30 C1: M=.83; SD=.87; N=30 Post intervention (after treatment) T2: M=.36; SD=.55; N=30 C2: M=1.5; SD=1.25 N=30	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75.	Nº of events
23) Lewis et al. (2013)	Journal article	624	Elementary school	Positive Action	Universal	Grade 3 84% FSM	Yes	Suspension Baseline (2001) T1: M=40.95; SD=48.13; N=3648 C1: M=65.25; SD=56.15; N=3800 Post treatment (2004) T2: M=55.17; SD=64.84; N=3407 C2: M=77.63; SD=66.8; N=3687 Follow‐up (2005) T3: M=68.08; SD=80.02; N=3367 C3: M=88.96; SD=76.56; N=3539	The principal investigator provided data for calculations. Based on Hedges (2007) and Spier et al. (2013), effect sizes were computed using d_T2 , assuming equal cluster sample size, ρ=.05.. See methods section for further details.	Nº of events
24) Mack (2001)	PhD Thesis	20	Fourth to sixth grade	ICAN Kids! Behavioural group counselling	Indicated	95% (for school, no stated for sample)	No	Out‐of‐school Suspension Baseline T1: M=1.5000; SD=.9718; N=10 C1: M=1.9000; SD=.8756; N=10 3 weeks (during) Tduring: M=.8000; SD=.6325; N=10 Cduring: M=.9000; SD=.7379; N=10 Post‐treatment T2: M=.3000; SD=.4830; N=10 C2: M=.4000; SD=.5164; N=10 + 3 weeks T3: M=.0000; SD=.0000; N=10 C3: M=1.0000; SD=.6667; N=10	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation assumed to be equal .75. See methods section for further details.	Nº of events
25) Obsuth et al. (2016)	Journal article	738	13.9	Engage in Education (Skills training)	Indicated	32%	Yes	Exclusion Official Records Baseline T1: OR=2.784; SE=.300; p=.001 Post‐treatment (1 month) T2: OR=1.444; SE=.389; p=.344	The study reports results using multi‐level analysis. In this case, we have not applied any correction of standard errors. We assume that MLM accounted for clusters and subsequently corrected bias (see p. 11). The study offered measures of impact based on self‐reporting, teachers’ reports and official records. We extracted from OR based on official records (most of our studies report official records of suspension).	Presence/absence
26) Okonofua et al. (2016)	Journal article	1682	Middle school	Empathic Discipline	Universal	Unknown	Yes	Suspension Post treatment (unclear number of months/weeks) T2: OR=.42; z= ‐3.33; p=.001; N=1449 31 clusters	Since data was dichotomous and nested in clusters, we corrected standard errors of effect sizes. The design effect was corrected by using the formula suggested by Higgins & Green (2011) expressed by the equation [1+(M‐1) x1]. See methods section for further details. Published data did not provide confidence intervals or SE. We tried to contact authors but it was not possible. We calculate an approximate SE=.013.	Presence/absence
27) Panayiotopoulos & Kerfoot (2004)	Journal article	124	10	Home and School Support Project (HASSP)	Indicated	Unknown	No	Exclusion T1: M=9.50; SD=14.81; N=62 C1: M=5.11; SD=7.56; N= 62 Post treatment (After three months) T2: M=4.95; SD=13.11; N=61 C2: M=5.51; SD=11.94; N=62	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75.	Nº days
28) Peck (2006)	PhD Thesis	1050	Fifth to eighth grade	Student Targeted with Opportunities for Prevention (STOP)	Unclear	Unknown	No	Suspension Post‐treatment (unclear number of weeks/months after treatment) T2: 22; N=315 C2: 22; N=321	Data was entered into CMA by using a 2x2 table. No further corrections.	Nº of events
29) Reese et al. (1981)	Journal article	98	Seventh to ninth grade	Preparation through Responsive Education Programs (PREP)	Indicated	Unknown	Matched peers	Suspension During school year T2 vs C2: X² (1)= 6.58, p<.02	Data was entered into CMA by using X² originally reported. No further corrections.	Nº of days
30) Russell (2007)	PhD Thesis	61	11.5 (.46)	Coping Power (Skills training for reducing aggression)	Indicated?	Unknown	No	Suspension Post treatment T2(School A): M=.15; SD=.38; N=13; C2(School B): M=.31; SD=.60; N=16; T2(School B: M=.29; SD=.61; N=14 C2(School B):M=.00; SD=.00;N=10	SMD was calculated using equations 3 and 4 in the methods section. No further corrections.	Nº of events
31) Shetguiri et al. (2011)	Journal article	108	14	Violence and drug use reduction	Indicated	100%	No	Suspended or Expelled Baseline T1: 8 (21%) N=40 C1: 10 (22%) N=46 Eight months follow‐up T2: 6 (14%) N=40 C2: 4 (8%) N=46	Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75.	Presence/absence
32) Smith (2004)	PhD Thesis	40		The Personal Responsibility Group (Emotional Intelligence skills)	Indicated	Unknown	No	In‐school Suspension Post‐treatment T2: f=11.085; p greater than or equal to .002; Out‐of‐school Suspension Post treatment T2: f= 10.088; p greater than or equal to .003	SMD was calculated based on f‐test. No further corrections.	Nº of events
33) Snyder et al. (2010)	Journal article	544	Elementary school children	Positive Action	Universal	55%	Yes	Suspension (% of students suspended) 2002 T1: M=1.12; SD=1.10; N=5000 C1: M=.98; SD=1.11; N=5000 2006 T2: M=.67; SD=.64; N=5000 C2: M=1.72; SD=1.55; N=5000 2007 T3: M=.84; SD=.61; N=5000 C3: M=2.53; SD=2.80; N=5000	The principal investigator provided sample sizes for calculations. Based on Hedges (2007) and Spier et al. (2013), effect sizes were computed using d_T2 , assuming equal cluster sample size. Since the report presented the value of ρ, we used this value in calculations.	Nº of students
34) Sprague et al. (2016)	Unpublished paper	13,498	Middle school children	School‐Wide Positive Behavioural Interventions and Support (SWPBIS)	Universal	Unknown	Yes	Expulsion Baseline T1: M=.002; SD=.004; N=6492 C1: M=.003;SD=.004; N=7006 Post treatment T2: M=.002; SD=.004; N=6492 C2: M=.003; SD=.005; N=7006 Follow‐up (1 year later) T3: M=.003; SD=.006; N=6492 C3: M=.003; SD=.004; N=7006 In School Suspension Baseline T1: M=.071; SD=.094; N=6492 C1: M=.135;SD=.189; N=7006 Post treatment T2: M=.064; SD=.087; N=6492 C2: M=.097; SD=.133; N=7006 Follow‐up (1 year later) T3: M=.058; SD=.060; N=6492 C3: M=.095; SD=.145; N=7006 Out‐of‐School Suspension Baseline T1: M=.082; SD=.063; N=6492 C1: M=.078; SD=.065; N=7006 Post treatment T2: M=.076; SD=.077; N=6492 C2: M=.061; SD=.042; N=7006 Follow‐up (1 year later) T3: M=.073; SD=.064; N=6492 C3: M=.075; SD=.051; N=7006	Principal investigator provided data for calculations. Based on Hedges (2007) and Spier et al. (2013), effect sizes were computed using d_T2 , assuming equal cluster sample size, ρ=.05. See methods section for further details.	Nº of events
35) Tilghman (1988)	PhD Thesis	100	12.5	Counsellor Peers	Indicated?	Unknown	No	Suspension T2: 11 (N=46) C2: 26 (N=45) Significance test	Data was entered into CMA by using a 2x2 table. No further corrections.	Nº of students
36) Ward & Gersten (2013)	Journal article	33 schools ≈ 25,000 students record	Elementary school children	Safe and Civil Schools	Universal	90%	Yes	Post treatment (end of intervention) OR=.83; SE=.05 Follow‐up (1 year later) (cumulative impact) OR=.77; SE=.04	Since data was nested in clusters, we corrected standard errors of effect sizes. The design effect was corrected by using the formula suggested by Higgins & Green (2011) expressed by the equation [1+(M‐1)x1].	Nº of days
37) Wyman et al. (2010)	Journal article	226	K ‐ 3rd	Rochester Resilience Programme	Indicated	90%	Yes	Suspension events Post‐treatment (immediately after intervention) TC2: Exp b=−0.57; SE=0.23; z=−2.48; p=0.013 Controlling for suspension T1 59 classrooms, 4 students per classroom	Since data was nested in clusters, we corrected standard errors of effect sizes. The design effect was corrected by using the formula suggested by Higgins & Green (2011), expressed by the equation [1+(M‐1) x1]. See methods section for further details	Nº of events

Open in a new tab

Abbreviations : T1(treatment group baseline measure); C1 (control group baseline measure); T2 (treatment group post treatment measure); C2 (control group post treatment measure); T3(treatment group follow up measure); C3(control group follow up measure); FSM (free school meals); M (mean); SD (standard deviation); N (sample size); OR (Odds ratio); 95% CI (95% confidence interval); SE (standard error); f (ANCOVA coefficient); p (p‐value); b (beta coefficient); X² (chi‐squared).

³⁵

When the mean age was not available in the original study, their grade in school has been reported. Their school grade gives the reader a general idea of the age of the students.

³⁶

Universal intervention strategies are those oriented to reach the entire population of students, without regard to individual risk factors. Indicated programmes were defined as those targeting students displaying behavioural problems, punished at school or presenting a specific risk to their educational development.

³⁷

Due to the nature of the settings (schools) some studies reported clustered data. We corrected SE errors when it was needed. See methods section for further details.

Figures

10.1 PERCENTAGE OF REPORTS WITHIN YEAR PERIOD

graphic file with name CL2-14--g019.jpg

11. Data and analyses

11.1 SUB‐GROUP ANALYSES

Forest plot sub‐group analysis: Predominant gender in school

graphic file with name CL2-14--g020.jpg

Forest plot sub‐group analysis: Grade at school

graphic file with name CL2-14--g021.jpg

Forest plot sub‐group analysis: Type of intervention

graphic file with name CL2-14--g022.jpg

Forest plot sub‐group analysis: Theoretical bases of interventions

graphic file with name CL2-14--g023.jpg

Forest plot sub‐group analysis: Training before intervention

graphic file with name CL2-14--g001.jpg

Forest plot sub‐group analysis: Monitoring the implementation of the intervention

graphic file with name CL2-14--g002.jpg

Forest plot sub‐group analysis: Reasons for conducting the research

graphic file with name CL2-14--g003.jpg

Forest plot sub‐group analysis: Evaluator role

graphic file with name CL2-14--g004.jpg

Forest plot sub‐group analysis: Type of exclusion

graphic file with name CL2-14--g005.jpg

12. Data collection instruments

12.1 ELIGIBILITY CHECK LIST

Criteria	Evaluation
1. Does this paper measure school exclusion as an outcome?	⑥ Yes	⑥ No
2. Does the intervention is school based? (or at least one component in the school)	⑥ Yes	⑥ No
3. Are the target individuals school students?	⑥ Yes	⑥ No
4. The report is based in an experimental, quasi‐experimental design?	⑥ Yes	⑥ No
5. Is this report informing statistical results able to be transformed in effect sizes?	⑥ Yes	⑥ No
6. Is this report included?	⑥ Yes	⑥ No
Reasons for exclusion:

Open in a new tab

12.2 DATA COLLECTION INSTRUMENT

DATA‐CODING INSTRUMENT

School‐based interventions for reducing disciplinary school exclusion. A systematic review

[Variable names in brackets]

Contents

Section A. Codification

Section B. Bibliographical information

Section C. Ethics

Section D. Research design

Section E. Sample

Section F. Primary outcome coding

Section G. Secondary outcomes coding

Section H. Base‐line measurements

Section I. Programme delivered

Section J. Follow‐up measurement

Section K. Effect sizes

Section A. Codification

Instruction: use one data‐coding instrument for each manuscript. When more than one manuscript reports the same research project, select one of them as the principal (e.g., the older) and give it an ID number. The following manuscripts should use the same ID but it must be registered in the Crossref field.

[STUDYID] Study ID number:

[CROSSREF1] Cross reference document identifier:

[CROSSREF2] Cross reference document identifier:

[CROSSREF3] Cross reference document identifier:

[DATESCR] Date of screening:

[CODER] Coder Initials:

Section B. Bibliographical information

Before completing this section, please be sure that the manuscript is correctly uploaded in the reference manager programme.

[AUTHOR] Name of the main author(s):

[AFFIL] Main author affiliation:

[DATEPUB] Year of publication:

[DATEFIEDW] Year of fieldwork (usually reported in a range):

[COISTATEMENT] Has the paper included a conflict of interest statement?

⑥ 1. Yes
⑥ 0. No

[LANGPUB] Language of the publication:

⑥ 1. English
⑥ 2. German
⑥ 3. Italian
⑥ 4. Spanish
⑥ 5. Portuguese
⑥ 999. Other:________

[COUNTPUB] Country of publication:

⑥ 1. UK
⑥ 2. USA
⑥ 3. Canada
⑥ 4. Australia
⑥ 999. Other:_________
⑥ 99. Unknown

[TYPUB] Type of publication:

⑥ 1. Journal
⑥ 2. Book/book chapter
⑥ 3. Masters thesis
⑥ 4. PhD/doctoral thesis
⑥ 5. Technical/governmental report
⑥ 6. Conference proceedings
⑥ 999. Other:________

[AUTDIS] Main author discipline:

⑥ 1. Education
⑥ 2. Social Work
⑥ 3. Psychology
⑥ 4. Criminal Justice
⑥ 5. Sociology
⑥ 6. Psychiatry/Medicine
⑥ 999. Other:________
⑥ 99. Unknown

[LOCAT] How was the study/report located?

⑥ 1. Electronic database
⑥ 2. Web search
⑥ 3. Reference in a book/paper. Please specify:
⑥ 4. Hand search in specialised journal
⑥ 5. Peer/expert suggestion
⑥ 999. Other. Specify:____________

Section C. Ethics

[CONSENT] Did the study declare the use of “consent agreement forms”?

⑥ 1. Yes
⑥ 0. No
⑥ 999. Other:________
⑥ 99. Unknown

[SIGNCONS] Who signed the consent?

⑥ 1. Students
⑥ 2. Parents
⑥ 3. Teachers
⑥ 4. Schools
⑥ 5. Parents and student
⑥ 999. Other. Specify:______________
⑥ 99. Unknown

Section D. Design

The present systematic review includes randomised control trials as well as quasi‐experimental reports (before/after measure plus a control or comparison group). If the control/comparison group is randomly allocated, non‐randomly allocated or matched and no intervention expected to produce impact is provided to it, you will be able to code that group as CONTROL. Subsequently, the TREATMENT group could be understood as the group that receives the intervention, no matter if that condition has been randomly allocated or not.

Please select always the data that is related with the sample effectively analysed.

[DESTYPE] What kind of design is this paper based on?

⑥ 1. Randomised controlled trial (true experiment)
⑥ 2. Before‐and‐after with control/comparison group/s
⑥ 3. Instrumental variable
⑥ 4. Propensity score matching
⑥ 5. Interrupted time series
⑥ 6. Pre/post measures with unmatched control/comparison group
⑥ 7. Inverse probability weighting
⑥ 999. Other. Specify:____________

[RANDUNIT] Units of randomization

⑥ 1. Individuals
⑥ 2. Clusters/groups (classroom, schools)
⑥ 999. Other. Specify:____________
⑥ 99. Unknown

[ANALUNIT] Unit of analysis

⑥ 1. Students
⑥ 2. Clusters/groups (classroom, schools)
⑥ 999. Other. Specify:____________
⑥ 99. Unknown

[COMPVAR] Variables measured to create comparability? (e.g., variables used to match the control and treatment groups)

__________________________________________________________________________________________

[MAINSTAT] What is the main statistical analysis used to produce the final results?

⑥ 1. Multilevel modelling
⑥ 2. Differences of means
⑥ 3. MANOVA
⑥ 4. Chi‐squared
⑥ 5. Propensity Score Matching
⑥ 999. Other. Specify:____________

Section E. Sample

[SAMPSELECT] How was the sample selected?

⑥ 1. Randomly
⑥ 2. Assessment
⑥ 3. Self‐selection
⑥ 999. Other. Specify:___________

[INSAMP] Initial sample size (i.e., individuals/schools):

[NUMBFOLL] Nº of follow‐up:

[FOLLSAMP1] Follow‐up 1 sample size:

[FOLLSAMP2] Follow‐up 2 sample size:

[FOLLSAMP3] Follow‐up 3 sample size:

[NSCHOOL] Initial number of schools:

[NSFOLL1] Follow‐up 1 sample size:

[NSFOLL2] Follow‐up 2 sample size:

[NSFOLL3] Follow‐up 3 sample size:

[NCLASS] Initial number of classes:

[NCFOLL1] Follow‐up 1 sample size:

[NCFOLL2] Follow‐up 2 sample size:

[NCFOLL3] Follow‐up 3 sample size:

Please code here the information on attrition described in the manuscript:

Total number

of students at

Baseline

Total number of students at Follow‐up

Treatment

[NTREBAS]

[NTREFOLL]

Control

[NCONTBA]

[NCONTFOL]

Open in a new tab

[MEANAGE] Mean age and standard deviation of overall sample at beginning of intervention:

[GENDER] Gender

⑥ % of males
⑥ % of females
⑥ 99. Unknown

[LOCAT] Location of programme

⑥ Urban area
⑥ Suburban area
⑥ Rural area
⑥ Mixture of areas
⑥ 99. Not enough information to determine

[GRADEX] Grade level of students

⑥ % of students in Elementary school or equivalent
⑥ % of students in Secondary school or equivalent
⑥ % of students in High school or equivalent
⑥ 4. Other:
⑥ 99. Unknown

[ETHNI] Predominant ethnicity ¹³

⑥ 1. % of Caucasian:
⑥ 2. % of Black:
⑥ 3. % of Hispanic:
⑥ 4. % of Asian:
⑥ 5. % of other mixed background:
⑥ 99. Unknown

[COUNTRY] Please state the name of the country where schools and sample of students were located when tested.

______________________ (99 if unknown)

[LUNEX] Socio‐economic status

% of students receiving free/reduced school lunch:

99. Unknown

[SENEX] Special Educational Needs

% of students declaring SEN:

99. Unknown

Section F. Primary Outcome (School Exclusion)

[EXCLUSION] Is the manuscript reporting outcomes for school exclusion?

⑥ 1. Yes
⑥ 0. No

[TYPEXC] Type of exclusion measured

⑥ 1. In‐school exclusion
⑥ 2. Out‐of‐school exclusion
⑥ 99. Unknown

[CHEKTIP] Duration of school exclusion measured

⑥ 1. Days of Fixed‐term exclusion
(Expressed in number or days, frequencies, percentages)
⑥ 2. Days of Permanent exclusion
(Expressed in number or days, frequencies, percentages)
⑥ 99. Unknown

[ICCEXCLU] If the statistical analysis include cluster in MLM, please register the ICC for Exclusion:

Section G. Secondary outcomes

[BEHAVMES] Did the study include measures on behaviour domains?

⑥ 1. Yes
⑥ 0. No
⑥ 99. Unknown

What types of the following behaviours are measured?

[PROSO] Pro‐social behaviour (e.g., helping, empathy). Specify:____________
- ⑥ 1. Yes
- ⑥ 0. No

[MEPROSO] Measure(s) used to test the behaviour (name):

[ALPHAPROSO]

⑥ Reliability test. Specify alpha value:_____________
⑥ Non reported

Groups	Effect size before	Effect size after
Control or comparison	[PROBC]	[PROAC]
Treatment	[PROBT]	[PROAT]

Open in a new tab

[PAGEPROSO] Number of the page from where you extract statistical data:

[ICCPROSO] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes:

[INTERNAL] Internalising problem behaviour
(e.g., anxiety, depression, attention‐deficit and hyperactivity disorder (ADHD), attention deficit, hyperactivity). Specify:____________
- ⑥ 1. Yes
- ⑥ 0. No

[MINTERNAL] Measure (s) used to test the behaviour (name):

[ALPHAINTERNAL]

⑥ Reliability test. Specify alpha value:_____________
⑥ Non reported

Groups	Effect size before	Effect size after
Control or comparison	[PROBC]	[PROAC]
Treatment	[PROBT]	[PROAT]

Open in a new tab

[PAGEINTERNAL] Number of the page from where you extract statistical data:

[ICCINTERNAL] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes:

[NAEXTERNAL] Non‐aggressive externalising problem behaviour
(e.g., stealing, lying, graffiti, illegal drugs). Specify: _____________
- ⑥ 1. Yes
- ⑥ 0. No

[MNAEXTERNAL] Measure used to test the behaviour (name):

[ALPHANAEXTER]

⑥ Reliability test. Specify alpha value:_____________
⑥ Non reported

Groups	Effect size before	Effect size after
Control or comparison	[PROBC]	[PROAC]
Treatment	[PROBT]	[PROAT]

Open in a new tab

[PAGENAXTERN] Number of the page from where you extract statistical data:

[ICCNAEXT] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes:

[AAGRESEXT] Aggressive externalising problem behaviour
(e.g., Opposition/defiance, physical aggression, indirect aggression, instrumental aggressions/dominance, reactive aggression, school bullying). Specify:_____________
- ⑥ 1. Yes
- ⑥ 0. No

[MAGRESSEXT] Measure used to test the behaviour (name):

[ALPHAAEXT]

⑥ Reliability test. Specify alpha value:_____________
⑥ Non reported

Groups	Effect size before	Effect size after
Control or comparison	[PROBC]	[PROAC]
Treatment	[PROBT]	[PROAT]

Open in a new tab

[AGRESPAGE] Number of the page from where you extract statistical data:

[ICCAGREEX] If the statistical analysis include cluster in MLM, please register the ICC for Behavioural outcomes:

Section H. Base‐line measurements

[DATABAS] Date of baseline assessment:

What measures were used?

[SRMES] Self‐report

⑥ 1. Yes
⑥ 0. No
⑥ 99. Unknown

[TRMES] Teachers’ report

⑥ 1. Yes
⑥ 0. No
⑥ 99. Unknown

[SCHRMES] School records

⑥ 1. Yes
⑥ 0. No
⑥ 99. Unknown

[PAREP] Parents

⑥ 1. Yes
⑥ 0. No
⑥ 99. Unknown

[OMES] Other:_________________

[EXCBL] Frequency of exclusion at baseline (register any measure given by the study)

Section I. Programme delivered

This section aims to codify data on the delivery process. Be aware that sometimes final reports do not describe all the data related to delivery. In those cases it would be helpful to search for registered protocols or earlier publications reporting more data on this.

[PRONAME] Name of the programme:

[PROCURRI] Was the programme curricular?

⑥ 1. Yes
⑥ 0. No
⑥ 99. Unknown
⑥ 999. Other. Specify:____________

[PROEND] The programme was conducted for:

⑥ 1. Research ends
⑥ 2. Demonstration ends
⑥ 3. Routine
⑥ 99. Unknown
⑥ 999. Other. Specify:_____________

[PROSIT] Primary programme site:

⑥ 1. Public school
⑥ 2. Private school
⑥ 3. Other, (specify):_________
⑥ 99. Unknown

[PROSCH] Was at least one of the components of the intervention was settled at school?

⑥ 1. Yes
⑥ 0. No
⑥ 99. Unknown

[PRODEL] Who delivered the programme?

⑥ 1. External facilitators
⑥ 2. School facilitators
⑥ 3. Both
⑥ 99. Unknown

[PDBACK] Deliverer's background 1

⑥ 1. Social worker
⑥ 2. Psychologist
⑥ 3. Teacher
⑥ 4. Police officers
⑥ 5. Peers
⑥ 999. Other. Specify:_____________
⑥ 99. Unknown

[PDBACK] Deliverer's background 2

⑥ 1. Social worker
⑥ 2. Psychologist
⑥ 3. Teacher
⑥ 4. Police officers
⑥ 5. Peers
⑥ 999. Other. Specify:_____________
⑥ 99. Unknown

[TRAINBEF] Did the deliverer receive training BEFORE implementing the programme?

⑥ 1. Yes.
⑥ 0. No.
⑥ 99. Unknown

[THOURS] How long was the training in hours?:____________

[TRAINDUR] Did the deliverer receive training DURING the implementation?

⑥ 1. Yes.
⑥ 0. No.
⑥ 99. Unknown

[THOURS2] How long was the training in hours?:

What type of intervention was delivered? If the manuscript indicates a mixture of interventions you can select more than one using TYPEPRO 1, 2 and 3.

	[TYPEPRO1]	[TYPEPRO2]	[TYPEPRO3]
1. Mentoring programme
2. Restorative programme
3. Skills training programme
4. School‐wide systemic intervention
5. Classroom management
6. Counselling/therapy
999. Other

Open in a new tab

Theoretical background of the intervention. If the manuscript indicates a mixture of theories, you can select more than one using THEORY 1, 2 and 3.

	[THEORY1]	[THEORY2]	[THEORY3]
1. Cognitive behavioural
2. Learning theory
3. Restorative theories
4. Organisational theories or principles
99. Unknown
999. Other (Specify)

Open in a new tab

[PROCONT] What happened to the control group?

⑥ 1. No intervention
⑥ 2. Wait‐list control
⑥ 3. Minimal contact
⑥ 4. Treatment as usual
⑥ 5. Alternative treatment
⑥ 5. Placebo
⑥ 999. Other. Specify:____________

[PROFORM] Delivery format:

⑥ 1. Manualised programme
⑥ 2. Unstructured programme
⑥ 3. Mixed
⑥ 99. Unknown
⑥ 999. Other. Specify:____________

What was the programme dosage?

[PRODOSW] AVERAGE Duration in weeks:

[PRODOSH] AVERAGE Hours per week:

[PROFREQ] What was the frequency of the programme counted?

⑥ 1. Less than a week
⑥ 2. Once a week
⑥ 3. Twice a week
⑥ 4. 3‐4 times a week
⑥ 5. Daily
⑥ 99. Unknown

[EVROLE] What was the “evaluator” role?

⑥ 1. Deliver the programme
⑥ 2. Designed the programme
⑥ 3. Both design and delivery
⑥ 4. Independent evaluator
⑥ 99. Unknown

[MONITOR] Was the programme implementation monitored?

⑥ 1. Yes
⑥ 0. No
⑥ 99. Unknown. Not enough information

[IMPROB] Does the report provide information about implementation problems?

⑥ 1. Yes, there were clear problems which are reported
⑥ 0. No, non‐reported problems, reasonably well implemented
⑥ 2. Possible problems based on the description of the intervention
⑥ 99. Unknown. Not enough information

[PROCOST] Is the cost of the intervention mentioned?

⑥ 1. Yes
⑥ 0. No

[AMOUNT] Cost:

[UNITCURR] Currency:

Section J. Follow‐up measurement

[DATEFALL] Date of follow up:

Multiple follow‐ups

[MONTHFO1] Nº of months from baseline to 1^st follow‐up:

[MONTHFO2] Nº of months from baseline to 2^nd follow‐up:

[MONTHFO3] Nº of months from baseline to 3^rd follow‐up:

[MONTHFO4] Nº of months from baseline to 4^th follow‐up:

What measures were used?

[POSTSR] Children/adolescent self‐report

⑥ 1. Yes
⑥ 0. No

[POSTTR] Teachers’ report

⑥ 1. Yes
⑥ 0. No

[POSTSR] School records

⑥ 1. Yes
⑥ 0. No

[POSTPR] Parents report

⑥ 1. Yes
⑥ 0. No

[POSTO] Other:__________

[FREQEXFOLL] Frequency of exclusion at follow‐up (register any measure given by the study)

Section K. Effect sizes of intervention on school exclusion

Effect size: outcomes expressed in continuous data.

[CSSEX] Sample size for the ES (Treatment group)

[CSSCON] Sample size for the ES (Control group)

[MEANEX] Mean (Treatment group)

[MEANCON] Mean (Control group)

[MEANADJ] Are the Means adjusted?

⑥ 1. Yes.
⑥ 0. No

[ADJBY] Adjusted by (describe):______________

[SDEX] Standard deviation (Treatment group)

[SDCON] Standard deviation (Control group)

[SEEX] Standard error (Treatment group)

[SECON] Standard error (Control group)

[CORREX] Correlation coefficient + p value (Treatment group)

[CORRCON] Correlation coefficient + p value (Control group)

[SMDTREAT] Standardised mean difference + confidence intervals

Effect size: outcomes expressed in dichotomous data.

[DSSTRE] Sample size for the ES (Treatment group)

[DSSCONT] Sample size for the ES (Control group)

[NUMTRE] Treatment group; number of successful cases:

[NUMCON] Control group; number of successful cases:

[PROPTRE] Treatment group; proportion of successful cases:

[PROPCON] Control group; proportion of successful cases:

[ORTRE] Treatment group; odds ratios:

Confidence Intervals:
p‐value:

[ORCON] Control group; odds ratios:

Confidence Intervals:
p‐value:

[ORADJ] Are the odds ratios adjusted?

⑥ 1. Yes.
⑥ 0. No

Adjusted by (explain):_________

[CHISC] X² value with df:

[PAGEEFFECT] Number of the page from where you extract statistical data:

Effect sizes at follow‐up

[ESFOLLOW1] Calculated effect at follows up 1:______

[ESFOLL1] Number of months after intervention for follow‐up 1:______

[ESFOLLOW2] Calculated effect at follows up 2:______

[ESFOLL2] Number of months after intervention for follow‐up 2:______

[ESFOLLOW3] Calculated effect at follows up 3:______

[ESFOLL3] Number of months after intervention for follow‐up 3:______

[ESFOLLOW4] Calculated effect at follows up 4:______

[ESFOLL4] Number of months after intervention for follow‐up 4:______

12.3 EPOC ‘RISK OF BIAS’ TOOL

Study:		Date:	Coder:
Item	Criteria	Evaluation	Justification
1	Was the allocation sequence adequately generated? Score “Low risk” if a random component in the sequence generation process is described (eg., Referring to a random number table). Score “High risk” when a non‐random method is used (eg performed by date of admission). NRCTs and CBA studies should be scored “High risk”. Score “Unclear risk” if not specified in the paper.
2	Was the allocation adequately concealed? Score “Low risk” if the unit of allocation was by institution, team or professional and allocation was performed on all units at the start of the study; or if the unit of allocation was by patient or episode of care and there was some form of centralised randomisation scheme, an on‐site computer system or sealed opaque envelopes were used. CBA studies should be scored “High risk”. Score “Unclear risk” if not specified in the paper.
3	Were baseline outcome measurements similar?^1,2 Score “Low risk” if performance or patient outcomes were measured prior to the intervention, and no important differences were present across study groups. In RCTs, score “Low risk” if imbalanced but appropriate adjusted analysis was performed (e.g. Analysis of covariance). Score “High risk” if important differences were present and not adjusted for in analysis. If RCTs have no baseline measure of outcome, score “Unclear risk”
4	Were baseline characteristics similar? Score “Low risk” if baseline characteristics of the study and control providers are reported and similar. Score “Unclear risk” if it is not clear in the paper (e.g. characteristics are mentioned in text but no data were presented). Score “High risk” if there is no report of characteristics in text or tables or if there are differences between control and intervention providers. Note that in some cases imbalance in patient characteristics may be due to recruitment bias whereby the provider was responsible for recruiting patients into the trial.
5	Were incomplete outcome data adequately addressed? ¹ Score “Low risk” if missing outcome measures were unlikely to bias the results (e.g. the proportion of missing data was similar in the intervention and control groups or the proportion of missing data was less than the effect size i.e. unlikely to overturn the study result). Score “High risk” if missing outcome data was likely to bias the results. Score “Unclear risk” if not specified in the paper (Do not assume 100% follow up unless stated explicitly).
6	Was knowledge of the allocated interventions adequately prevented during the study? ¹ Score “Low risk” if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcomes are objective, e.g. length of hospital stay. Primary outcomes are those variables that correspond to the primary hypothesis or question as defined by the authors. Score “High risk” if the outcomes were not assessed blindly. Score “Unclear risk” if not specified in the paper.
7	Was the study adequately protected against contamination? Score “Low risk” if allocation was by community, institution or practice and it is unlikely that the control group received the intervention. Score “High risk” if it is likely that the control group received the intervention (e.g. if patients rather than professionals were randomised). Score “Unclear risk” if professionals were allocated within a clinic or practice and it is possible that communication between intervention and control professionals could have occurred (e.g. physicians within practices were allocated to intervention or control)
8	Was the study free from selective outcome reporting? Score “Low risk” if there is no evidence that outcomes were selectively reported (e.g. all relevant outcomes in the methods section are reported in the results section). Score “High risk” if some important outcomes are subsequently omitted from the results. Score “Unclear risk” if not specified in the paper.

Open in a new tab

12.4 TRICHOTOMOUS SCALE FOR POTENTIAL FINANCIAL CONFLICT OF INTEREST

Study:		Date:
Financial COI	Coding Rule	Justification
Unlikely	None of the study authors is programme developer, collaborator of programme developer or license holder.
Possible	(Programme developer or collaborator of programme developer is study author) AND ((programme is not (yet) commercially available) OR (business model is ‘not‐for‐profit’))
Likely	(Programme developer or collaborator of programme developer is study author) AND (programme is commercially available) AND (business model is ‘for profit’)

Open in a new tab

^*This instrument has been developed by Eisner & Humphreys (2012)

13. Electronic searches

Australian Education Index

Date: 05/10/ 2015

Output of the searches: 148

Saved hits: 1

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

British Education Index

Date: 05/10/ 2015

Output of the searches: 202

Saved hits: 5

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school AND stand down

BMJ Controlled Trials

Date: 06/10/ 2015

Output of the searches: 550

Saved hits: 0

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

CBCA Education (Canada)

Date: 14/10/2015

Output of the searches: 5652

Saved hits: 58

Final searches string:

1)
school exclusion
2)
school suspension
3)
school expelled
4)
school expulsion
5)
school stand down

ClinicalTrials.gov

Date: 07/10/ 2015

Output of the searches: 285

Saved hits: 3

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

Criminal Justice Abstract

Date: 06/10/ 2015

Output of the searches: 369

Saved hits: 19

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

Cochrane Central Register of Controlled Trials (CENTRAL)

Date: 16/10/2015

Output of the searches: 154

Saved hits: 12

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

Educational Resources Information Center ‐ ERIC

Date: 14/10/2015

Output of the searches: 1491

Saved hits: 48

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

Ethos Beta

Date: 08/10/ 2015

Output of the searches: 482

Saved hits: 5

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

EMBASE

Date: From 08/10/ 2015 to 12/10/2015

Output of the searches: 1569

Saved hits: 12

Final searches string:

1)
“school exclusion” AND
2)
“school exclusion” AND evaluation
3)
“school exclusion” AND random*
4)
“school exclusion” AND intervention,
5)
“school exclusion” AND effective*,
6)
“school exclusion” AND efficacy,
7)
“school exclusion” AND quasi,
8)
“school exclusion” AND impact,
9)
“school exclusion” AND RCT,
10)
“school exclusion” AND school management
11)
“school exclusion” AND classroom management
12)
“school exclusion” AND school support project*
13)
“school exclusion” AND skills training
14)
“school exclusion” AND disciplinary methods
15)
“school exclusion” AND token economy
16)
“school exclusion” AND program*
17)
“school exclusion” AND *intervention*
18)
“school exclusion” AND strateg*
19)
“school exclusion” AND schoolchildren
20)
“school exclusion” AND *children*
21)
“school exclusion” AND school‐age*
22)
“school exclusion” AND adolescent*
23)
“school exclusion” AND pupil*
24)
“school exclusion” AND student*
25)
“school suspension” AND experiment*
26)
“school suspension” AND evaluation
27)
“school suspension” AND random*
28)
“school suspension” AND intervention,
29)
“school suspension” AND effective*,
30)
“school suspension” AND efficacy,
31)
“school suspension” AND quasi,
32)
“school suspension” AND impact,
33)
“school suspension” AND RCT,
34)
“school suspension” AND school management
35)
“school suspension” AND classroom management
36)
“school suspension” AND school support project*
37)
“school suspension” AND skills training
38)
“school suspension” AND disciplinary methods
39)
“school suspension” AND token economy
40)
“school suspension” AND program*
41)
“school suspension” AND *intervention*
42)
“school suspension” AND strateg*
43)
“school suspension” AND schoolchildren
44)
“school suspension” AND *children*
45)
“school suspension” AND school‐age*
46)
“school suspension” AND adolescent*
47)
“school suspension” AND pupil*
48)
“school suspension” AND student*
49)
“school expulsion” AND experiment*
50)
“school expulsion” AND evaluation
51)
“school expulsion” AND random*
52)
“school expulsion” AND intervention,
53)
“school expulsion” AND effective*,
54)
“school expulsion” AND efficacy,
55)
“school expulsion” AND quasi,
56)
“school expulsion” AND impact,
57)
“school expulsion” AND RCT,
58)
“school expulsion” AND school management
59)
“school expulsion” AND classroom management
60)
“school expulsion” AND school support project*
61)
“school expulsion” AND skills training
62)
“school expulsion” AND disciplinary methods
63)
“school expulsion” AND token economy
64)
“school expulsion” AND program*
65)
“school expulsion” AND *intervention*
66)
“school expulsion” AND strateg*
67)
“school expulsion” AND schoolchildren
68)
“school expulsion” AND *children*
69)
“school expulsion” AND school‐age*
70)
“school expulsion” AND adolescent*
71)
“school expulsion” AND pupil*
72)
“school expulsion” AND student*

Google Scholar

Date: From 27/10/ 2015 to 04/11/2015

Output of the searches: 13525

Saved hits: 165

Final searches string:

1)
“school exclusion” AND experiment OR evaluation experiment OR evaluation OR quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random*
2)
“school exclusion” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy OR program* OR *intervention* OR strateg*
3)
“school suspen*” AND experiment* OR evaluation OR quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random*
4)
“school suspen*” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy OR program* OR *intervention* OR strateg*
5)
“school expulsion” AND experiment OR evaluation experiment OR evaluation OR quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random*
6)
“school expulsion” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy OR program* OR *intervention* OR strateg*
7)
“stand‐down” AND experiment* OR evaluation OR quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random*
8)
“stand‐down” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy OR program* OR *intervention* OR strateg*

[In order to manage the searches we break down the searches by year. For instance 1980‐1990, 1991‐2005 and 2006 to date]

Google

Date: From 27/10/ 2015 to 04/11/2015

Output of the searches: 4092

Saved hits: 22

Final searches string:

1)
“school exclusion” AND experiment* OR evaluation OR quasi OR RCT OR impact OR efficacy OR intervention
2)
“school exclusion” AND random*
3)
“school exclusion” AND effective*
4)
“school exclusion” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy
5)
“school exclusion” AND *intervention*
6)
“school exclusion” AND strateg*
7)
“school exclusion” AND program*
8)
“school suspension” AND experiment* OR evaluation OR quasi OR RCT OR impact OR efficacy OR intervention
9)
“school suspension” AND random*
10)
“school suspension” AND effective*
11)
“school suspension” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy
12)
“school suspension” AND *intervention*
13)
“school suspension” AND strateg*
14)
“school suspension” AND program*
15)
school expulsion AND…intervention* OR strategy* OR program*
16)
school expulsion AND…random* OR effective*
17)
school expulsion AND…school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy
18)
school expulsion AND…random* OR effective
19)
school expulsion AND…experiment* OR evaluation OR quasi OR RCT OR impact OR efficacy OR intervention

Institute of Education Sciences – What Works Clearinghouse

Date: 30/09/ 2015

Output of the searches: 2

Saved hits: 0

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

ISI Web of Knowledge

Date: from 07/09/ 2015 to 17/09/2015

Output of the searches: 4391

Saved hits: 270

Final searches string:

1)
TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)

AND

TOPIC: (school) AND TOPIC: (*exclusion)
2)
TOPIC: (school) AND TOPIC: (*suspension*)

AND

TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)
3)
TOPIC: (school) AND TOPIC: (suspended)

AND

TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)

Refined by: RESEARCH AREAS: (PSYCHOLOGY OR FAMILY STUDIES OR EDUCATION EDUCATIONAL RESEARCH OR BEHAVIORAL SCIENCES OR PSYCHIATRY OR CRIMINOLOGY PENOLOGY OR SOCIOLOGY OR ETHNIC STUDIES OR SOCIAL WORK OR URBAN STUDIES OR SOCIAL SCIENCES OTHER TOPICS OR SOCIAL ISSUES)

Timespan=1980‐2015
4)
TOPIC: (school) AND TOPIC: (expelled)

AND

TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)
5)
TOPIC: (school) AND TOPIC: (expulsion)

AND

TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)
6)
TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)

AND

TOPIC: (school) AND TOPIC: (*exclusion)
7)
TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)

AND

TOPIC: (school) AND TOPIC: (*exclusion)
8)
TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)

AND

TOPIC: (school) AND TOPIC: (*suspension*)
9)
TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)

AND

TOPIC: (school) AND TOPIC: (suspended)
10)
TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)

AND

TOPIC: (school) AND TOPIC: (expelled)
11)
TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)

AND

TOPIC: (school) AND TOPIC: (expulsion)
12)
TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school‐age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student)

AND

TOPIC: (school) AND TOPIC: (*exclusion)
13)
TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school‐age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student)

AND

TOPIC: (school) AND TOPIC: (*suspension*)
14)
TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school‐age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student)

AND

TOPIC: (school) AND TOPIC: (suspended)
15)
TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school‐age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student)

AND

TOPIC: (school) AND TOPIC: (expelled)
16)
TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)

AND

TOPIC: (school) AND TOPIC: (expulsion)

All searches in ISI Web of Knowledge were Refined by: RESEARCH AREAS: (PSYCHOLOGY OR FAMILY STUDIES OR EDUCATION EDUCATIONAL RESEARCH OR BEHAVIORAL SCIENCES OR PSYCHIATRY OR CRIMINOLOGY PENOLOGY OR SOCIOLOGY OR ETHNIC STUDIES OR SOCIAL WORK OR URBAN STUDIES OR SOCIAL SCIENCES OTHER TOPICS OR SOCIAL ISSUES)

Timespan=1980‐2015

MEDLINE

Date: 05/10/ 2015

Output of the searches: 142

Saved hits: 0

Final searches string:

1)
“school exclusion” AND experiment*
2)
“school exclusion” AND evaluation
3)
“school exclusion” AND random*
4)
“school exclusion” AND intervention,
5)
“school exclusion” AND effective*,
6)
“school exclusion” AND efficacy,
7)
“school exclusion” AND quasi,
8)
“school exclusion” AND impact,
9)
“school exclusion” AND RCT,
10)
“school exclusion” AND school management
11)
“school exclusion” AND classroom management
12)
“school exclusion” AND school support project*
13)
“school exclusion” AND skills training
14)
“school exclusion” AND disciplinary methods
15)
“school exclusion” AND token economy
16)
“school exclusion” AND program*
17)
“school exclusion” AND *intervention*
18)
“school exclusion” AND strateg*
19)
“school exclusion” AND schoolchildren
20)
“school exclusion” AND *children*
21)
“school exclusion” AND school‐age*
22)
“school exclusion” AND adolescent*
23)
“school exclusion” AND pupil*
24)
“school exclusion” AND student*
25)
“school suspension” AND experiment*
26)
“school suspension” AND evaluation
27)
“school suspension” AND random*
28)
“school suspension” AND intervention,
29)
“school suspension” AND effective*,
30)
“school suspension” AND efficacy,
31)
“school suspension” AND quasi,
32)
“school suspension” AND impact,
33)
“school suspension” AND RCT,
34)
“school suspension” AND school management
35)
“school suspension” AND classroom management
36)
“school suspension” AND school support project*
37)
“school suspension” AND skills training
38)
“school suspension” AND disciplinary methods
39)
“school suspension” AND token economy
40)
“school suspension” AND program*
41)
“school suspension” AND *intervention*
42)
“school suspension” AND strateg*
43)
“school suspension” AND schoolchildren
44)
“school suspension” AND *children*
45)
“school suspension” AND school‐age*
46)
“school suspension” AND adolescent*
47)
“school suspension” AND pupil*
48)
“school suspension” AND student*
49)
“school expulsion” AND experiment*
50)
“school expulsion” AND evaluation
51)
“school expulsion” AND random*
52)
“school expulsion” AND intervention,
53)
“school expulsion” AND effective*,
54)
“school expulsion” AND efficacy,
55)
“school expulsion” AND quasi,
56)
“school expulsion” AND impact,
57)
“school expulsion” AND RCT,
58)
“school expulsion” AND school management
59)
“school expulsion” AND classroom management
60)
“school expulsion” AND school support project*
61)
“school expulsion” AND skills training
62)
“school expulsion” AND disciplinary methods
63)
“school expulsion” AND token economy
64)
“school expulsion” AND program*
65)
“school expulsion” AND *intervention*
66)
“school expulsion” AND strateg*
67)
“school expulsion” AND schoolchildren
68)
“school expulsion” AND *children*
69)
“school expulsion” AND school‐age*
70)
“school expulsion” AND adolescent*
71)
“school expulsion” AND pupil*
72)
“school expulsion” AND student*

The National dropout prevention centre network

Date: 16/10/2015

Output of the searches: 26

Saved hits: 0

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

The Netherlands Trial Register ‐ NTR

Date: 07/10/ 2015

Output of the searches: 0

Saved hits: 0

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

Open Grey

Date: 07/10/ 2015

Output of the searches: 169

Saved hits: 3

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

ProQuest Dissertations & Theses A&I: Social Sciences

Including: Dissertation Abstracts & Index to Theses Database

Date: 19/10/2015

Output of the searches: 5280

Saved hits: 344

Final searches string:

1)
“school exclusion” AND experiment
2)
“school exclusion” AND evaluation
3)
“school exclusion” AND random*
4)
“school exclusion” AND intervention
5)
“school exclusion” AND effective*
6)
“school exclusion” AND efficacy
7)
“school exclusion” AND quasi
8)
“school exclusion” AND impact
9)
“school exclusion” AND RCT
10)
“school exclusion” AND school management
11)
“school exclusion” AND classroom management
12)
“school exclusion” AND school support project*
13)
“school exclusion” AND skills training
14)
“school exclusion” AND disciplinary methods
15)
“school exclusion” AND token economy
16)
“school exclusion” AND program*
17)
“school exclusion” AND strateg*
18)
“school suspension” AND experiment
19)
“school suspension” AND evaluation
20)
“school suspension” AND random*
21)
“school suspension” AND intervention
22)
“school suspension” AND effective*
23)
“school suspension” AND efficacy
24)
“school suspension” AND quasi
25)
“school suspension” AND impact
26)
“school suspension” AND RCT
27)
“school suspension” AND school management
28)
“school suspension” AND classroom management
29)
“school suspension” AND school support project*
30)
“school suspension” AND skills training
31)
“school suspension” AND disciplinary methods
32)
“school suspension” AND token economy
33)
“school suspension” AND program*
34)
“school suspension” AND strateg*
35)
“school stand‐down” AND experiment
36)
“school stand‐down” AND evaluation
37)
“school stand‐down” AND random*
38)
“school stand‐down” AND intervention
39)
“school stand‐down” AND effective*
40)
“school stand‐down” AND efficacy
41)
“school stand‐down” AND quasi
42)
“school stand‐down” AND impact
43)
“school stand‐down” AND RCT
44)
“school stand‐down” AND school management
45)
“school stand‐down” AND classroom management
46)
“school stand‐down” AND school support project*
47)
“school stand‐down” AND skills training
48)
“school stand‐down” AND disciplinary methods
49)
“school stand‐down” AND token economy
50)
“school stand‐down” AND program*
51)
“school stand‐down” AND strateg*
52)
“school expelled” AND experiment
53)
“school expelled” AND evaluation
54)
“school expelled” AND random*
55)
“school expelled” AND intervention
56)
“school expelled” AND effective*
57)
“school expelled” AND efficacy
58)
“school expelled” AND quasi
59)
“school expelled” AND impact
60)
“school expelled” AND RCT
61)
“school expelled” AND school management
62)
“school expelled” AND classroom management
63)
“school expelled” AND school support project*
64)
“school expelled” AND skills training
65)
“school expelled” AND disciplinary methods
66)
“school expelled” AND token economy
67)
“school expelled” AND program*
68)
“school expelled” AND strateg*
69)
“school expulsion” AND experiment
70)
“school expulsion” AND evaluation
71)
“school expulsion” AND random*
72)
“school expulsion” AND intervention
73)
“school expulsion” AND effective*
74)
“school expulsion” AND efficacy
75)
“school expulsion” AND quasi
76)
“school expulsion” AND impact
77)
“school expulsion” AND RCT
78)
“school expulsion” AND school management
79)
“school expulsion” AND classroom management
80)
“school expulsion” AND school support project*
81)
“school expulsion” AND skills training
82)
“school expulsion” AND disciplinary methods
83)
“school expulsion” AND token economy
84)
“school expulsion” AND program*
85)
“school expulsion” AND strateg*
86)
“school suspended” AND experiment
87)
“school suspended” AND evaluation
88)
“school suspended” AND random*
89)
“school suspended” AND intervention
90)
“school suspended” AND effective*
91)
“school suspended” AND efficacy
92)
“school suspended” AND quasi
93)
“school suspended” AND impact
94)
“school suspended” AND RCT
95)
“school suspended” AND school management
96)
“school suspended” AND classroom management
97)
“school suspended” AND school support project*
98)
“school suspended” AND skills training
99)
“school suspended” AND disciplinary methods
100)
“school suspended” AND token economy
101)
“school suspended” AND program*
102)
“school suspended” AND strateg*

PsychINFO

Date: From 12/10/ 2015 to 14/10/2015

Output of the searches: 1538

Saved hits: 86

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

Sociological Abstract (ProQuest)

Date: From 21/09/ 2015 to 23/09/2015

Output of the searches: 2440

Saved hits: 355

Final searches string:

1)
“school exclusion” AND experiment*
2)
“school exclusion” AND evaluation
3)
“school exclusion” AND random*
4)
“school exclusion” AND intervention,
5)
“school exclusion” AND effective*,
6)
“school exclusion” AND efficacy,
7)
“school exclusion” AND quasi,
8)
“school exclusion” AND impact,
9)
“school exclusion” AND RCT,
10)
“school exclusion” AND school management
11)
“school exclusion” AND classroom management
12)
“school exclusion” AND school support project*
13)
“school exclusion” AND skills training
14)
“school exclusion” AND disciplinary methods
15)
“school exclusion” AND token economy
16)
“school exclusion” AND program*
17)
“school exclusion” AND *intervention*
18)
“school exclusion” AND strateg*
19)
“school exclusion” AND schoolchildren
20)
“school exclusion” AND *children*
21)
“school exclusion” AND school‐age*
22)
“school exclusion” AND adolescent*
23)
“school exclusion” AND pupil*
24)
“school exclusion” AND student*
25)
“school suspension” AND experiment*
26)
“school suspension” AND evaluation
27)
“school suspension” AND random*
28)
“school suspension” AND intervention,
29)
“school suspension” AND effective*,
30)
“school suspension” AND efficacy,
31)
“school suspension” AND quasi,
32)
“school suspension” AND impact,
33)
“school suspension” AND RCT,
34)
“school suspension” AND school management
35)
“school suspension” AND classroom management
36)
“school suspension” AND school support project*
37)
“school suspension” AND skills training
38)
“school suspension” AND disciplinary methods
39)
“school suspension” AND token economy
40)
“school suspension” AND program*
41)
“school suspension” AND *intervention*
42)
“school suspension” AND strateg*
43)
“school suspension” AND schoolchildren
44)
“school suspension” AND *children*
45)
“school suspension” AND school‐age*
46)
“school suspension” AND adolescent*
47)
“school suspension” AND pupil*
48)
“school suspension” AND student*
49)
“school expulsion” AND experiment*
50)
“school expulsion” AND evaluation
51)
“school expulsion” AND random*
52)
“school expulsion” AND intervention,
53)
“school expulsion” AND effective*,
54)
“school expulsion” AND efficacy,
55)
“school expulsion” AND quasi,
56)
“school expulsion” AND impact,
57)
“school expulsion” AND RCT,
58)
“school expulsion” AND school management
59)
“school expulsion” AND classroom management
60)
“school expulsion” AND school support project*
61)
“school expulsion” AND skills training
62)
“school expulsion” AND disciplinary methods
63)
“school expulsion” AND token economy
64)
“school expulsion” AND program*
65)
“school expulsion” AND *intervention*
66)
“school expulsion” AND strateg*
67)
“school expulsion” AND schoolchildren
68)
“school expulsion” AND *children*
69)
“school expulsion” AND school‐age*
70)
“school expulsion” AND adolescent*
71)
“school expulsion” AND pupil*
72)
“school expulsion” AND student*

SciElo

Date: 06/10/ 2015

Output of the searches: 32

Saved hits: 0

Final searches string:

1)
“school exclusion evaluation”
2)
“school exclusion experiment”
3)
“school suspension evaluation”
4)
“school suspension experiment”
5)
“school expulsion evaluation”
6)
“school expulsion experiment”
7)
suspension escolar
8)
suspension escolar evaluación
9)
suspension escolar experiment
10)
expulsion escolar
11)
expulsion escolar evaluación
12)
expulsion escolar experiment
13)
expulsion escuela

Sciencegov

Date: From 05/10/ 2015 to 06/10/2015

Output of the searches: 319

Saved hits: 61

Final searches string:

1)
“school suspension” AND evaluation
2)
“school suspension” AND experiment*
3)
“school exclusion” AND evaluation
4)
“school exclusion” AND experiment*
5)
“school expulsion” AND evaluation
6)
“school expulsion” AND experiment*

The Campbell Collaboration Social, Psychological, Educational and Criminological Trials Register (C2‐SPECTR)

Date: 16/10/2015

Output of the searches: 1

Saved hits: 1

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

Trials Journal

Date: 06/10/ 2015

Output of the searches: 56

Saved hits: 1

Final searches string:

1)
school exclusion
2)
school suspension
3)
school suspended
4)
school expelled
5)
school expulsion
6)
school stand down

WHO‐ International Clinical Trials Registry Platform (ICTRP)

Date: 30/09/2015

Output of the searches: 3

Saved hits: 3

Final searches string:

1)
“school exclusion” (other outcome terms searched with no results)

Valdebenito S, Eisner M, Farrington DP, Ttofi M, Sutherland School‐based interventions for reducing disciplinary school exclusion: a systematic review. Campbell Systematic Reviews 2018:1 DOI: 10.4073/csr.2018.1

Linked article:

PROTOCOL.

Plain language summary on the Campbell website

Footnotes

In this document school exclusion and school suspension are used synonymously.

Odds ratios are presented without confidence intervals or p‐values as in the source.

The strong link between school exclusion and subsequent school drop‐out/entry into the juvenile justice system has been termed the school‐to‐prison pipeline (Snyder & Sickmund, 2006; Fenning et al., 2012;Chin et al., 2012 Christle et al., 2005; Nicholson‐ Crotty et al., 2009). It describes the escalating punitive consequences of harsh discipline in school and its exclusionary practices.

⁴

Studies reporting a large difference between the treatment and control group at pre‐test were said to be excluded as they would not help in distinguishing intervention effects from other effects. The protocol also stated that quasi‐experimental studies based on one group pre‐test and post‐test designs or one group post‐test‐only designs should be excluded from the review.

⁵

The assumption of equal sample size seems to be a good approximation for the calculation of effect sizes. Hedges, (2007) asserts that effect size calculation based on equal and unequal cluster sizes are not substantially different.

⁶

The presented flow chart has been adapted from Moher, Liberati, Tetzlaff, & Altman, (2009)

⁷

At the beginning of the evaluation we selected two studies and coded them as a pilot assessment to ensure we understood the criteria consistently. This exercise helped to test the instrument and also to adjust some criteria. Following the complete coding of all studies, the coders met in late June 2016 to discuss and moderate their ratings.

⁸

How many variables would we expect to be out of balance by chance in an RCT? This figure is usually obtained by the following calculation (k*0.05) where k is the number of variables, so 20*0.05 = 1.

⁹

Many of our included studies were non‐published PhD thesis (i.e., 18 out of 37). As a result, it was impossible to find pre‐registration publications for these studies. We observed that most of the theses reported small or null effects (i.e., Arter, 2005; Bragdon, 2010; Brett, 1993; Burcham, 2002; Crowder, 2001; Harding, 2011; Peck, 2006). Since null and small effects were reported, we assumed low risk of bias regarding selective outcome reporting.

¹⁰

Substance misuse was not included in the calculation.

¹¹

The original effect sizes are presented in Fig. 2.1, page 63. They are reported in OR; we have converted those values to make it more comparable with the metric we use in the present review.

¹²

To calculate the impact of exclusion on dropout rates, the study used longitudinal data collected via surveys and standardised tests to observe the percentage of students who graduated on time, two years after the expected date of graduation, and four years after. The independent variable was exclusion and the study involved independent controls such as demographic variables, family variables and school performance variables. All in all, the study showed that in the U.S., only 71 percent of tenth graders who received a suspension graduated from high school, compared to 94 percent of tenth graders who did not receive a suspension. In other words, being suspended [was] associated with a 23 percentage‐point decrease in the likelihood of graduating (Rumberger & Losen, 2016; p.12).

¹³

Based on Lipsey & Wilson (2001)

7. References

7.1 REFERENCES TO INCLUDED STUDIES

Allen, J. P. , Philliber, S. , Herrling, S. , & Kuperminc, G. P. (1997). Preventing teen pregnancy and academic failure: Experimental evaluation of a developmentally based approach. Child Development, 64(4), 729–742. doi:10.2307/1132122 [Google Scholar]
Arter, P. S. (2005). The effects of the positive alternative learning supports program on the attendance, academic performance, and behavior of students with chronic behavior problems. (Doctoral Dissertation). The Johns Hopkins University, Maryland. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Barnes, V. A. , Bauza, L. B. , & Treiber, F. A. (2003). Impact of stress reduction on negative school behavior in adolescents. Health and quality of life outcomes, 1(10), 1–7. doi:10.1186/1477‐7525‐1‐10 [DOI] [PMC free article] [PubMed] [Google Scholar]
Berlanga, D. T. (2004). The efficacy of an intensive guidance intervention program to increase academic success of at‐risk middle school students. (Doctoral Dissertation). Texas A&M University ‐ Corpus Christi. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Bradshaw, C. P. , Waasdorp, T. E. , & Leaf, P. J. (2012). Effects of School‐Wide Positive Behavioral Interventions and Supports on child behavior problems. Pediatrics, 130(5), E1136–E1145. doi:10.1542/peds.2012‐0243 [DOI] [PMC free article] [PubMed] [Google Scholar]
Bragdon, D. (2010). The effects of a career awareness project on the academic performance of at‐risk eighth grade students. (Doctoral Dissertation). The Johns Hopkins University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Brett, P. F. (1993). An examination of the impact of a mentoring program on inner city, seventh‐grade, African American students' attitudes toward learning and other selected variables. (Doctoral Dissertation). The George Washington University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Burcham, B. G. (2002). Impact of school‐based social problem‐solving training on middle school students with disruptive behavior. (Doctoral Dissertation). University of Kentucky. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Collier, C. E. (2002). The effect of pro‐social skills training on the problem behavior of selected African American children in the District of Columbia public schools: Recommendations for change. (Doctoral Dissertation). The George Washington University, US. Retrieved from EBSCOhost psyh database. [Google Scholar]
Cook, P. J. , Dodge, K. , Farkas, G. , Fryer Jr, R. G. , Guryan, J. , Ludwig, J. , … Steinberg, L. (2014). The (surprising) efficacy of academic and behavioral intervention with disadvantaged youth from a randomized experiment in Chicago. Institute for Policy Research Northwestern University. WP‐14‐03. doi:10.3386/w19862 [Google Scholar]
Cornell, D. G. , Allen, K. , & Fan, X. (2012). A randomized controlled study of the Virginia Student Threat Assessment Guidelines in kindergarten through grade 12. School Psychology Review, 41(1), 100–115. Retrieved from <Go to ISI>://WOS:000302757000006 [Google Scholar]
Crowder, D. M. (2001). The effect of the gang resistance education and training program on character education and the development of leadership skills in children at an inner‐city middle school. (Doctoral Dissertation). Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Dynarski, M. , Moore, M. , Mullens, J. , Gleason, P. , James‐Burdumy, S. , Rosenberg, L. , … Levy, D. (2003). When schools stay open late: The national evaluation of the 21st‐Century Community Learning Centers program. First year findings. Retrieved from Mathematic Policy Research, Inc. Decision Information Resources, Inc.: U.S. Department of Education; [Google Scholar]
Edmunds, J. A. , Bernstein, L. , Unlu, F. , Glennie, E. , Willse, J. , Smith, A. , & Arshavsky, N. (2012). Expanding the start of the college pipeline: Ninth‐grade findings from an experimental study of the impact of the early college high school model. Journal of Research on Educational Effectiveness, 5(2), 136–159. doi:10.1080/19345747.2012.656182 [Google Scholar]
Farrell, A. D. , Meyer, A. L. , & White, K. S. (2001). Evaluation of Responding in Peaceful and Positive Ways (RIPP): A school‐based prevention program for reducing violence among urban adolescents. Journal of Clinical Child Psychology, 30(4), 451–463. doi:10.1207/s15374424jccp3004_02 [DOI] [PubMed] [Google Scholar]
Feindler, E. , Marriott, S. , & Iwata, M. (1984). Group anger control training for junior high school delinquents. Cognitive Therapy and Research, 8(3), 299–311. doi:10.1007/BF01173000 [Google Scholar]
Harding, H. (2011). An evaluation of a targeted group intervention delivered to year 8 pupils and broadly based on cognitive‐behavioural approaches. (Doctoral Dissertation). University of Nottingham. Retrieved from http://eprints.nottingham.ac.uk/11843/ [Google Scholar]
Hawkins, J. D. , Doueck, H. J. , & Lishner, D. M. (1988). Changing teaching practices in mainstream classrooms to improve bonding and behavior of low achievers. American Educational Research Journal, 25(1), 31–50. doi: 10.3102/00028312025001031 [DOI] [Google Scholar]
Hirsch, B. J. , Hedges, L. V. , Stawicki, J. , & Mekinda, M. A. (2011). After‐school programs for high school students. An evaluation of After School Matters. Retrieved from http://www.wallacefoundation.org/knowledge‐center/Pages/After‐School‐Programs‐for‐High‐School‐Students‐An‐Evaluation‐of‐After‐School‐Matters.aspx [Google Scholar]
Hostetler, M. , & Fisher, K. (1997). Project CARE substance abuse prevention program for high‐risk youth: A longitudinal evaluation of program effectiveness. Journal of Community Psychology, 25(5), 397–419. doi:10.1002/(sici)1520‐6629(199709)25:5<397::aid‐jcop3>3.0.co;2‐q [Google Scholar]
Ialongo, N. , Poduska, J. , Werthamer, L. , & Kellam, S. (2001). The distal impact of two first‐grade preventive interventions on conduct problems and disorder in early adolescence. Journal of Emotional and Behavioral Disorders, 9(3), 146–160. doi:10.1177/106342660100900301 [Google Scholar]
Johnson. (1983). The effects of affective education on the discipline of selected seventh and eighth grade students in the district of Columbia public schools. (Doctoral Dissertation). The George Washington University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Lewis, K. M. , Schure, M. B. , Bavarian, N. , DuBois, D. L. , Day, J. , Ji, P. , … Flay, B. R. (2013). Problem behavior and urban, low‐income youth a randomized controlled trial of Positive Action in Chicago. American Journal of Preventive Medicine, 44(6), 622–630. doi:10.1016/j.amepre.2013.01.030 [DOI] [PMC free article] [PubMed] [Google Scholar]
Mack, M. A. P. (2001). The effects of a behavioral intervention on discipline referrals and school suspension. (Doctoral Dissertation). Auburn University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Obsuth, I. , Sutherland, A. , Cope, A. , Pilbeam, L. , Murray, A. L. , & Eisner, M. (2016). London Education and Inclusion Project (leip): Results from a cluster‐randomized controlled trial of an intervention to reduce school exclusion and antisocial behavior. Journal of Youth & Adolescence. doi:10.1007/s10964‐016‐0468‐4 [DOI] [PMC free article] [PubMed] [Google Scholar]
Okonofua, J. A. , Pauneskua, D. , & Walton, G. M. (2016). Brief intervention to encourage empathic discipline cuts suspension rates in half among adolescents. PNAS, 113(19), 5221–5226. doi:10.1073/pnas.1523698113 [DOI] [PMC free article] [PubMed] [Google Scholar]
Panayiotopoulos, C. , & Kerfoot, M. (2004). A home and school support project for children excluded from primary and first year secondary school. Child and Adolescent Mental Health, 9(3), 109–114. [DOI] [PubMed] [Google Scholar]
Peck, M. (2006). The effects of group involvement in a prevention program on at‐risk youth's grades, attendance, substance use, and contacts with juvenile probation: An archival study of the STOP program. (Doctoral Dissertation). Alliant International University, Fresno, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Reese, S. C. , Murphy, R. J. , & Filipczak, J. (1981). Assessment of multiple behavioral procedures on academic and social classroom behavior. Psychology in the Schools, 18(3), 349–355. doi:10.1002/1520‐6807(198107)18:3<349::AID‐PITS2310180317>3.0.CO;2‐Q [Google Scholar]
Russell, A. (2007). Influencing adaptive functioning in school‐age children: implementation and program evaluation of the Coping Power Program. (Doctoral Dissertation). George Fox University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Shetgiri, R. , Kataoka, S. , Lin, H. , & Flores, G. (2011). A randomized, controlled trial of a school‐based intervention to reduce violence and substance use in predominantly Latino high school students. Journal of the National Medical Association, 103(9), 932–940. [DOI] [PubMed] [Google Scholar]
Smith, M. E. (2004). A mixed paradigm: Study of a transformational learning program for at‐risk high school students. (Doctoral Dissertation). Texas A&M University ‐ Corpus Christi. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Snyder, F. , Flay, B. , Vuchinich, S. , Acock, A. , Washburn, I. , Beets, M. , & Li, K.‐K. (2010). Impact of a social‐emotional and character development program on school‐level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched‐pair, cluster‐randomized, controlled trial. Journal of Research on Educational Effectiveness, 3(1), 26. [DOI] [PMC free article] [PubMed] [Google Scholar]
Sprague, J. R. , Biglan, A. , Rusby, J. , Gau, J. M. , & Vincent, C. G. (2016). Implementing School‐Wide PBIS in middle schools: Results of a randomized trial (unpublished paper). The University of Oregon. [Google Scholar]
Tilghman, W. S. (1988). The effect of peer counseling on lowering the rate of middle school suspensions. (Doctoral Dissertation). Florida Atlantic University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Ward, B. , & Gersten, R. (2013). A randomized evaluation of the safe and civil schools model for positive behavioral interventions and supports at elementary schools in a large urban school district. School Psychology Review, 42(3), 317–333. [Google Scholar]
Wyman, P. A. , Cross, W. , Brown, C. H. , Yu, Q. , Tu, X. , & Eberly, S. (2010). Intervention to strengthen emotional self‐regulation in children with emerging mental health problems: Proximal impact on school behavior. Journal of Abnormal Child Psychology, 38(5), 707–720. doi:10.1007/s10802‐010‐9398‐x [DOI] [PMC free article] [PubMed] [Google Scholar]

7.2 REFERENCES TO EXCLUDED STUDIES: 7.2.1 PhD and master thesis

Ballard, K. L. (2009). School‐related and social‐emotional outcomes of providing mental health services in schools. (Doctoral Dissertation). The University of Wisconsin; ‐ Madison, US. Retrieved from EBSCOhost psyh database. [DOI] [PubMed] [Google Scholar]
Barber, M. S. (2002). The influence of the Alabama Reading Initiative on the culture of three inner‐city elementary schools in the state of Alabama.(Doctoral Dissertation). The University of Alabama Birmingham, Alabama. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Barnes, L. B. (2010). An evaluation of the right choices program to determine effectiveness in delivering constructive interventions and providing an early support program in order to modify behavior of first‐time student offenders who commit drug and violent acts. (Doctoral Dissertation). Gardner‐Webb University, United States. [Google Scholar]
Barton, C. L. (2013). Effectiveness of 1‐2‐3 Magic for Teachers to improve classroom management and discipline. (Doctoral Dissertation). Capella University. Retrieved from EBSCOhost ERIC database. [Google Scholar]
Beard, M. L. (2014). Positive behavior intervention and support: An evaluation of its impact in elementary schools. (Doctoral Dissertation). Wingate University, New York. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Blackwell, S. D. (2008). The impact of a transition program on ninth grade students' performance. (Doctoral Dissertation). Virginia Polytechnic Institute and State University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Blevins, L. D. (2007). The use of school‐wide positive behavior support at a rural high school to decrease disruptive behavior for both typical students and students identified with special needs. (Doctoral Dissertation). East Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Blocker, J. B. I. (2013). The effects of the positive behavior intervention and support program on the out‐of‐school suspension rates and english achievement of African american males in grade nine. (Doctoral Dissertation). Union University, United States. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Bogle, K. E. (2007). Evaluation of a brief group parent training intervention in the context of an after‐school program for middle‐school students. (Doctoral Dissertation). University of South Carolina. [Google Scholar]
Bond, E. W. (2010). Stuarts draft high school: Quantitative study evaluating the ninth‐grade transition program. (Doctoral Dissertation). Virginia Polytechnic Institute and State University. [Google Scholar]
Booth, B. A. (1995). A Cognitively Based Anger Control Training Program with Aggressive Adolescents in School Settings. (Doctoral Dissertation). University at Albany, New York. [Google Scholar]
Bradley, O. W. (1985). The effects of the teacher‐advisor system on the students of omaha technical high school, Omaha, Nebraska, in the areas of attitude, attendance, behavior, and achievement. (Doctoral Dissertation). The University of Nebraska ‐ Lincoln. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Bridges, L. (2012). Positive behavior intervention and support: An alternative approach. (Doctoral Dissertation). Loyola University Chicago, United States. Retrieved from EBSCOhost psyh database. [Google Scholar]
Brinson, J. P. I. (2012). Assessing the effectiveness of a behavior intervention model on fifth grade in‐school suspension rates. (Doctoral Dissertation). Cambridge College, United States. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Brown‐Kersey, N. (2011). The effects of implementing restorative justice in an in‐school suspension program. (Doctoral Dissertation). Western Michigan University. [Google Scholar]
Buettner, D. A. (2013). The effect of PBIS on the school environment and test scores. (Doctoral Dissertation). Lindenwood University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Burnham, S. V. (2009). A case study of reducing a junior high school disciplinary infractions and increasing a positive school climate: After‐school detentions vs. evidence‐based computer program. (Doctoral Dissertation). Widener University, US. Retrieved from EBSCOhost psyh database. [Google Scholar]
Campbell‐Peralta, P. J. (1995). Reducing Suspensions of Minority Males through a Group Guidance/Mentoring Intervention Program. (Master Thesis). Nova Southeastern University. [Google Scholar]
Chartas, N. D. (2001). An evaluation of the short‐term influence of Lock Haven University's Upward Bound program. (Doctoral Dissertation). The Pennsylvania State University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Chatzinikolaou, R. (2015). Strength‐based interventions in secondary school: How can they be most helpful for pupils at risk and not‐at‐risk of exclusion?(Doctoral Dissertation). University of East London. Retrieved from http://roar.uel.ac.uk/4432/ [Google Scholar]
Clark, W. H. (1980). The effectiveness of supervised discipline centers as an alternative to suspensions in two public secondary schools in Prince George's County, Maryland. (Doctoral Dissertation). University of Southern California, US. Retrieved from EBSCOhost psyh database. [Google Scholar]
Clarke, P. N. (2008). Program evaluation: A study to evaluate the effectiveness of an alternative Education program in a rural county of a mid‐Atlantic U.S. State. (Doctoral Dissertation). Capella University, US. [Google Scholar]
Conyers, R. E. D. (2005). The effects of group counseling compared with individual counseling in the reduction of at‐risk behaviors using grade point average, discipline referrals, and attendance among sixth grade black males. (Doctoral Dissertation). South Carolina State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Corwin, E. P. (2014). An evaluation of the effectiveness of the Check‐in/Check‐out intervention for students engaging in relational aggression and their classmates. (Doctoral Dissertation). Louisiana State University. [Google Scholar]
Cramer, J. A. (1990). Orientation to continuation high school: A controlled experiment. (Doctoral Dissertation). University of the Pacific. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Creamean, S. L. , & Horvath, R. J. (2000). The effectiveness of block scheduling. (Master Thesis), Saint Xavier University, Chicago. Illinois. Retrieved from http://search.proquest.com/docview/62347371?accountid=9851 ProQuest Social Sciences Premium Collection database. [Google Scholar]
Dalessio, E. S. (2012). The effects of online education on the at‐risk student in an alternative school setting. (Doctoral Dissertation). College of Saint Elizabeth, Morristown, New Jersey. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=2&id=ED549663 ERIC Educational Resources Information Center database. [Google Scholar]
Davis, K. S. (2011). Program evaluation of universal Positive Behavioral Interventions and Supports in Kentucky. (Doctoral Dissertation). Western Kentucky University, Bowling Green, Kentucky. Retrieved from http://digitalcommons.wku.edu/diss/6 [Google Scholar]
Davis, M. V. C. (1998). A comparison based on student outcomes of an individualized student improvement program with an in‐school suspension program. (Doctoral Dissertation). Georgia State University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
De Blank, G. (2009). Effects of quality and quantity of the mentoring relationship on outcomes for at‐risk early adolescent girls. (Doctoral Dissertation). University of Virginia. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Denmark, V. M. (1995). A program evaluation of the peer mediation programs in the junior high schools of a large urban school system. (Doctoral Dissertation). Georgia State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Devlin, N. F. (2006). Effects of in‐school suspension with conflict resolution training on suspensions, referrals, and normative beliefs about aggression. (Doctoral Dissertation). Walden University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Deyoe, M. M. (2015). The relationship of disability status on attendance, behavior, and achievement indicators for students receiving intensive school‐based mental health counseling. (Doctoral Dissertation). University at Albany, State University of New York. [Google Scholar]
Draa, V. A. B. (2005). School uniforms in urban public high schools. (Doctoral Dissertation). Youngstown State University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED497409&site=ehost‐live&scope=site Available from EBSCOhost eric database. [Google Scholar]
Dunlap, J. A. (2010). Evaluation of the union county alternative to suspension program. (Doctoral Dissertation). Gardner‐Webb University, US. Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Social Sciences Premium Collection database. [Google Scholar]
Dykes, D. A. (2015). School‐Wide Positive Behavior Interventions and Supports: A case study measuring implementation fidelity and implementation impact. (Doctoral Dissertation). Georgia State University. Retrieved from http://scholarworks.gsu.edu/eps_diss/121 [Google Scholar]
Ebbe, A. R. (2011). Implementing successful middle school and elementary school tier one positive behavior intervention support strategies in a high school. (Doctoral Dissertation). Edgewood College. Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Social Sciences Premium Collection database. [Google Scholar]
Eineder, D. V. (1996). The effects of block scheduling in a high school. (Doctoral Dissertation). The University of Alabama, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Feldi, K. (1980). Behavioral treatment of alienated adolescents. (Doctoral Dissertation). Columbia University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Fisher, C. L. (1980). Effects of a middle school counseling program on attendance, academic achievement, and suspensions with a study of attitudes of students, principals, counselors and teachers toward the program. (Doctoral Dissertation). University of South Carolina, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1982‐50083‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Fisher, P. H. (1994). The effects of an urban middle school dropout prevention program on the academic achievement, attendance, attitudes, and conflict resolution skills of at‐risk student. (Doctoral Dissertation). Old Dominion University, Ann Arbor. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Ford, M. L. (2004). Alternative schooling: The influence of a positive peer culture model in a large urban public school setting. (Doctoral Dissertation). Morgan State University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Frost, A. M. (2012). Bully prevention, peer mediation and conflict resolution: Impact of prevention programs on reducing school suspensions. (Doctoral Dissertation). Kansas State University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2013‐99230‐405&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Garcia, S. H. (2015). An analysis of the impact of emotional literacy instruction on at‐risk students. (Doctoral Dissertation). Brandman University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Gouge, W. E. (2011). A comparison research study on the use of school uniforms and graduation, attendance, and suspension rates in East Tennessee. (Doctoral Dissertation). Liberty University, Lynchburg, VA. Retrieved from http://digitalcommons.liberty.edu/cgi/viewcontent.cgi?article=1471&context=doctoral [Google Scholar]
Grady, P. L. (2006). Increasing academic, behavioral and psychological engagement among at risk junior high students. (Doctoral Dissertation). California Polytechnic State University. University of Calidornia. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Gretzinger, B. Q. (1988). System‐wide implementation of an assertive discipline‐based behavior management plan: A program evaluation. (Doctoral Dissertation). The University of Southern Mississippi. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Grinage, A. L. (2005). Character education programs and student suspension rates from school: Do character education programs decrease student suspensions from regular instructional public elementary schools in Texas?. (Doctoral Dissertation). University of North Texas. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Guardino, D. (2013). The disproportionate use of discipline: An investigation of the potential impact of school‐wide positive behavioral interventions and supports. (Doctoral Dissertation). University of Oregon. [Google Scholar]
Haithcock, F. O. (1996). The effects of a research based high school dropout prevention program on multiple student outcomes. (Doctoral Dissertation). University of Miami. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Hardin, S. A. (2003). An evaluation of the Technical Preparation Program in the Tennessee public school system. (Doctoral Dissertation). Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Harless, M. D. R. (2011). Disciplinary referrals in response to School‐Wide Positive Behavior Plan in a rural middle school setting. (Doctoral Dissertation). East Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Harrison, C. D. (2013). An evaluation of the effects of the academics and behavior Check‐in/Check‐out intervention. (Doctoral Dissertation). University of Oregon. Retrieved from https://scholarsbank.uoregon.edu/xmlui/bitstream/handle/1794/13327/Harrison_oregon_0171A_10763.pdf?sequence=1 [Google Scholar]
Hasson, R. C., Jr. (2011). Positive Behavior Supports: A study of implementation at the intermediate school level. (Doctoral Dissertation). The University of Houston‐Clear Lake. Retrieved from EBSCOhost eric database. [Google Scholar]
Henderson, D. X. (2012). A mixed methods examination of resilience and social connectedness among youth in an alternative to suspension program. (Doctoral Dissertation). North Carolina State University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2014‐99021‐027&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Herzog, M. R. (1980). An analysis of data and student opinion regarding two alternatives for reducing aberrant student behavior at Niles East High School. (Doctoral Dissertation). George Peabody College for Teachers, Vanderbilt University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1982‐50931‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Hickman, A. L. (2014). Evaluating a school‐based day treatment program for students with challenging behaviors. (Doctoral Dissertation). The College of William and Mary in Virginia. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&id=ED556953 ERIC Educational Resources Information Center database. [Google Scholar]
Hill, C. (2014). An examination of the effectiveness of the louisiana gear up program in promoting self‐efficacy, improving academic achievement and increasing teachers' aspirations for their students. (Doctoral Dissertation). Louisiana Tech University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Hill, N. O. (2011). The effects of positive behavioral interventions and supports on students behavior in grade five. (Doctoral Dissertation). Capella University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Hochman, J. S. (1986). The effectiveness of in‐school suspension as a disciplinary technique in an urban high school with and without intervention strategies. (Dissertation). Virginia Polytechnic Institute & State University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1987‐53042‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Hostetler, M. L. (1995). Project C.A.R.E. substance abuse prevention program: An evaluation of program effectiveness. (Doctoral Dissertation). The Pennsylvania State University, Ann Arbor. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Howard, G. P. (2008). Effect of group counseling on at‐risk African‐American female students. (Doctoral Dissertation). Roosevelt University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Huston, K. (1999). A Comparison study of the San Francisco community board whole schools conflict resolution project. (Master Thesis), University of Nevada, Las Vegas. Retrieved from http://digitalscholarship.unlv.edu/cgi/viewcontent.cgi?article=1289&context=thesesdissertations [Google Scholar]
Jackson‐Chapman, D. (2006). After‐school intervention programs' impact upon select variables among 10th grade students in the Education Longitudinal Study (ELS) 2002 dataset. (Doctoral Dissertation). The University of Southern Mississippi. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Jacoby, R. L. (2008). The effectiveness of a proactive school‐wide approach to discipline at the middle school level. (Doctoral Dissertation). University Of Missouri‐Kansas City, US. Retrieved from EBSCOhost psyh database. [Google Scholar]
Johnson, D., Jr. (1980). The impact of developmental counseling in relation to selected variables associated with disruptive behavior of public secondary school students. (Doctoral Dissertation). Union Institute and University, Ann Arbor. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Johnson, W. (2010). Analyses of the impact of school uniforms on violence in North Carolina public high schools. (Doctoral Dissertation). East Carolina University. Retrieved from Available from EBSCOhost eric database. [Google Scholar]
Jordan, A. D. (2008). Measuring the efficacy of a ninth grade academy on students with disabilities. (Doctoral Dissertation). Capella University. Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Social Sciences Premium Collection database. [Google Scholar]
Kershes, M. B. (1984). An evaluation of an alternative discipline method: The saturday suspension program. (Doctoral Dissertation). George Peabody College for Teachers of Vanderbilt University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Kost, C. P. I. (2013). Assessing an induction program designed to improve achievement of ninth grade students receiving federal lunch benefits. (Doctoral Dissertation). Grand Canyon University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Kula, K. (2012). Social and emotional learning in the schools: The effects of Why Try as an alternative to suspension program. (Doctoral Dissertation). Loyola University Chicago, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Lafrance, J. A. (2009). Examination of the fidelity of School‐Wide Positive Behavior Support implementation and its relationship to academic and behavioral outcomes in Florida. (Doctoral Dissertation). University of Central Florida, US. Retrieved from EBSCOhost psyh database. [Google Scholar]
Lassen, S. R. (2007). Impact of school‐wide PBS on indicators of social development and academic performance in an inner‐city middle school. (Doctoral Dissertation). University of Kansas, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2007‐99010‐234&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Lavender, D. R. (2009). The effect of an in‐class behavioral intervention plus differentiated instruction program on the achievement and behavior outcomes of verbally disruptive 8th‐grade students with and without co‐occurring reading delimitations. (Doctoral Dissertation). University of Nebraska at Omaha, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Lee, D. H. (2003). The effects of the responsible discipline and IMPACT transition models on the behaviors and attitudes of ninth‐grade students in a large urban high school. (Doctoral Dissertation). The University of North Carolina at Charlotte, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Long, L. (2009). Effectiveness of animal‐assisted therapy interventions in altering childhood aggressive behaviors. (Doctoral Dissertation). Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Long, P. K. (2015). Diminishing the discipline gap: Restorative justice as a promising alternative in one urban school. (Doctoral Dissertation). University of Dayton. Retrieved from https://etd.ohiolink.edu/!etd.send_file?accession=dayton1436815423&disposition=inline [Google Scholar]
Martinez, S. (2012). The impact of a literacy intervention program on student achievement and behavior at two urban high schools. (Doctoral Dissertation). California State University. Retrieved from ProQuest Social Sciences Premium Collection database. [Google Scholar]
Matarazzo, T. M. (1999). Block scheduling: A comparison of block scheduling and traditional scheduling and its relationship to student achievement, student satisfaction and teacher methodology in a junior‐senior high school in New Jersey. (Doctoral Dissertation). Seton Hall University, College of Education and Human Services. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Matthews, A. L. G. (1997). A longitudinal study of two at‐risk student programs in Henry County. (Doctoral Dissertation). University of Georgia. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
McCallum, L. C. (1989). An evaluation of the assertive discipline program at Charles D. Owen high school. (Doctoral Dissertation). The University of North Carolina at Greensboro, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1990‐57395‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
McCauley, A. E. (1991). An evaluation of the High Point, North Carolina Cities in Schools Program. (Doctoral Dissertation). The University of North Carolina at Greensboro. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
McDaniel‐Campbell, C. (2011). The impact of an alternative education intervention (Student transition and recovery) on middle schools' attendance, academic performance, and discipline. (Doctoral Dissertation). Georgia Southern University US. [Google Scholar]
McGreevy, S. A. (2007). Girl power: The impact of an academic and social support program on the educational outcomes of urban seventh and eighth grade Latinas. (Doctoral Dissertation). State University of New York at Buffalo. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
McKinney, M. L. (2004). The effectiveness of a school‐based drug and alcohol treatment program. (Doctoral Dissertation). Alliant International University, San Francisco Bay. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
McTague, J. R. (2006). The effect of a master schedule on student discipline and classroom attendance. (Doctoral Dissertation). Loyola University Chicago, Ann Arbor. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Melton, J. M. (2002). An analysis of the relationship between the Fight Free Program and school climate. (Doctoral Dissertation). Saint Louis University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Mitchell, S. A. (2007). Examining the effectiveness of a school‐based mental health center's services. (Doctoral Dissertation). University of Rhode Island and Rhode Island College. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Muehlrath, K. L. (1996). The effect of conflict management training programs in preventing discipline problems of students in grades nine through twelve in a suburban school district. (Doctoral Dissertation). Saint Louis University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Mushok, M. A. (1989). An exploration of the effects of a behavior modification program and a school/business partnership upon adolescents' self‐esteem and school performance. (Doctoral Dissertation). University of Massachusetts Amherst. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Norris, A. N. (2008). An exploratory study on the effect of restorative justice on school success and disciplinary incidents. (Master Thesis), Michigan State University. Retrieved from ProQuest Social Sciences Premium Collection database. [Google Scholar]
O'Leary, J. L. (2012). The impact of the ‘Social And Emotional Aspects Of Learning’ framework on primary aged pupils' learning, attendance and behaviour. (Doctoral Dissertation). University of Manchester. Retrieved from https://www.escholar.manchester.ac.uk/uk‐ac‐man‐scw:161327 [Google Scholar]
Ohlson, M. (2009). A study of school culture, leadership, teacher quality and student outcomes via a performance framework in elementary schools participating in a school reform initiative. (Doctoral Dissertation). University of Florida, US. Retrieved from EBSCOhost ERIC database. [Google Scholar]
Okunade, A. O. (2015). Survey of students' perceptions of assertive discipline and restorative discipline in American Christian Academy, Ibadan, Nigeria. (Master Thesis), Dordt College, Sioux Center, Iowa. Retrieved from http://digitalcollections.dordt.edu/med_theses/89/ [Google Scholar]
Perreault, A. (2013). An examination of the leadership practices that support and sustain school based mental health programs. (Doctoral Dissertation). San Diego State University. Retrieved from https://sdsu‐dspace.calstate.edu/bitstream/handle/10211.3/115548/Perreault_sdsu_0220D_10167.pdf?sequence=1 [Google Scholar]
Phipps, J. D. (2012). The effects of character education in South Carolina's secondary schools. (Doctoral Dissertation). Capella University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Pilot, G. C. (2011). The impact of a school‐based enterprise program on the achievement and behavior of special education students attending high schools in South Carolina. (Doctoral Dissertation). South Carolina State University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Pollitt, K. (2010). Exploring the value of engagement mentoring as a preventative strategy with at‐risk youth. (Doctoral Dissertation). University of Birmingham. Retrieved from http://etheses.bham.ac.uk/1001/1/Pollitt10ApEdPsyD1_A1a.pdf [Google Scholar]
Porter, S. M. (2007). The impact of a School‐Wide Token economy on behavior, attendance, and academics at Morgan High School. (Master Thesis), Marietta College. Retrieved from https://etd.ohiolink.edu/!etd.send_file%3Faccession%3Dmarietta1174947309%26disposition%3Dinline [Google Scholar]
Power, J. (2012). The effect of implementing School‐Wide Positive Behavior Interventions and Supports on student misbehavior in a large urban high school. (Doctoral Dissertation). Liberty University, US. Retrieved from EBSCOhost eric database. [Google Scholar]
Pratt, L. (2009). Supporting pupils at risk of exclusion: an evaluation of an intensive, out‐of‐school, emotional literacy programme for key stage 3 pupils. Institute of Education, University of London. [Google Scholar]
Ranelli, F. (2007). The effectiveness of different disciplinary programs in two New Jersey high schools. (Doctoral Dissertation). Wilmington University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2008‐99090‐565&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Reed, J. G. (2004). An examination of treatment integrity practices and behavioral outcomes when utilizing the Second Step curriculum. (Doctoral Dissertation). University of Maryland. Retrieved from http://drum.lib.umd.edu/bitstream/handle/1903/1559/umi‐umd‐1465.pdf?sequence=1&isAllowed=y [Google Scholar]
Rentschler, D. M. (1996). A longitudinal study to determine the long‐term effects of prevention and intervention substance abuse strategies on at‐risk fourth grade students and their families. (Doctoral Dissertation). Widener University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Riese, S. A. (2004). The efficacy of a violence prevention intervention with elementary school children. (Doctoral Dissertation). Walden University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Rizzi, L. (2011). Program evaluation: A study to evaluate the effectiveness of academic and behavioral interventions in an alternative to out of school suspension program. (Doctoral Dissertation). Missouri Baptist University. Retrieved from EBSCOhost eric database. [Google Scholar]
Robinson, R. L. (1985). Investigation of three specific intervention strategies to reduce the attrition rate of the potential high school dropout. (Doctoral Dissertation). University of Maryland College Park. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Romero, A. (1998). The academic effectiveness of school‐site drug intervention programs. (Master Thesis), California State University, Long Beach. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Roseberry, L. L. (1997). An applied experimental evaluation of conflict resolution curriculum and social skills development. (Doctoral Dissertation). Loyola University Chicago, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1997‐95017‐218&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Samuels, K. S. (2003). The relationship of school uniforms to students' achievement, attendance, discipline referrals and perceptions: An analysis of one urban school district. (Doctoral Dissertation). The University of Alabama at Birmingham, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Schellenberg, R. C. (2005). School violence: Evaluation of an elementary school peer mediation program. (Doctoral Dissertation). Regent University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Schmidt, P. A. M. (1995). Helping teen mothers make positive choices thereby reducing suspensions through a curriculum for conflict resolution/peer mediation. (Dissertation). Nova Southeastern University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED386642&site=ehost‐live&scope=site Available from EBSCOhost eric database. [Google Scholar]
Schultz, E. K. (2007). State and local level implementation of Schoolwide Positive Behavior Support: An examination of the Texas Behavior Support initiative (TBSI). (Doctoral Dissertation). University of North Texas. [Google Scholar]
Shimizu, B. (2000). A study to determine what impact, if any, the implementation of a mandatory uniform dress policy has had on improving student behavior among ninth‐grade students in three california high schools. (Doctoral Dissertation). University of La Verne, California. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Simington, K. W. (1996). The use of brief family counseling with middle school students experiencing discipline problems. (Doctoral Dissertation). The University of North Carolina at Greensboro. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Simmons, C. L. (2015). Impacts of prosocial intervention on delinquent and defiant behaviors in youth grades 7‐12: A program evaluation. (Doctoral Dissertation). Alliant International University, Fresno. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Simpson, M. T. (2010). The impact of Schoolwide Positive Behavior Support on the discipline gap in the middle school setting. (Doctoral Dissertation). Walden University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2011‐99110‐058&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Skoczek, S. J. (2011). The effects of the web transition program on middle school student gpa, attendance, and suspensions. (Doctoral Dissertation). Baker University, US. Retrieved from https://www.bakeru.edu/images/pdf/SOE/EdD_Theses/Skoczek_Steve.pdf [Google Scholar]
Smith, S.‐R. S. (2009). Positive behavior support in schools universal intervention: Suspension rates and school climate in implementing middle schools. (Doctoral Dissertation). Fairleigh Dickinson University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Speirs, R. (1994). Decreasing suspensions in grades nine through twelve through the implementation of a peace curriculum. (Doctoral Dissertation). Nova University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED378090&site=ehost‐live&scope=site Available from EBSCOhost eric database. [Google Scholar]
Stevenson, H. L., III. (1999). An analysis of requiring school uniforms and its impact on student behavior: Implications for school reform. (Doctoral Dissertation). Texas Southern University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Swantic, F. M. (1986). An investigation of the effectiveness of bibliotherapy on middle grade students who repeatedly display inappropriate behavior in the school setting. (Doctoral Dissertation). Georgia State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Thornsberry, J. L. (2010). Freshman transition and its effectiveness on student success as measured by improved attendance, improved grades, decreased discipline referrals, and decreased dropout rate. (Doctoral Dissertation). Lindenwood University. [Google Scholar]
Thornton, K. M. (2008). A quantitative study comparing traditional high schools and high schools implementing freshman academies in the state of Tennessee. (Doctoral Dissertation). East Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Thunell, T. E. (1990). An evaluation of the effectiveness of PASS, a junior high school dropout prevention program. (Doctoral Dissertation). Colorado State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Tomlin, V. E. (1994). A mentor program for improving the academic attainment of black adolescent males. (Doctoral Dissertation). University of Denver. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Toms, O. M. (2012). The effects of Check‐in Check‐out on the social and academic planning and outcomes of African‐American males in an urban secondary setting. (Doctoral Dissertation). The University of North Carolina at Charlotte, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Tyre, A. D. (2003). State‐level implementation of schoolwide positive behavior support: An evaluation of the Arizona Behavioral Initiative. (Doctoral Dissertation). Northern Arizona University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2003‐95015‐114&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
VanMetre, J. P. (2009). Examining the effectiveness of a ninth‐grade transition program. (Doctoral Dissertation). Lamar University, US. Retrieved from http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&id=ED526690 ERIC Educational Resources Information Center database. [Google Scholar]
Vaughan, A. C. (2001). The implementation of uniform dress policies: Student performance outcomes and perceptions of student impact at five public elementary schools in Hampton roads, Virginia. (Doctoral Dissertation). The George Washington University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2001‐95017‐007&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Veliz, P. (2012). The role of interscholastic sport in public high schools: A zero‐sum game or a bridge to success. (Doctoral Dissertation). University at Buffalo, State University of New York. Retrieved from ProQuest Social Sciences Premium Collection database. [Google Scholar]
Warren, R. A. (2011). The positive alternative credit experience (PACE) program a quantitative comparative study. (Doctoral Dissertation). Lindenwood University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED534547&site=ehost‐live&scope=site; http://gateway.proquest.com/openurl?url_ver=Z39.88‐2004&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&res_dat=xri:pqdiss&rft_dat=xri:pqdiss:3481165 Available from EBSCOhost eric database. [Google Scholar]
Washington‐Labat, C. A. (2003). The effects of school uniforms on attendance and discipline among schools with and without mandatory school uniform policies. (Doctoral Dissertation). The University of Southern Mississippi. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Weissflug, K. G. (2009). Effects of Positive Behavior Support Programs on Student Behaviors. (Doctoral Dissertation). Lindenwood University. [Google Scholar]
Welch, C. Y. (2008). A mixed‐method study utilizing a token economy to shape behavior and increase academic success in urban students. (Doctoral Dissertation). Walden University. [Google Scholar]
Whitaker, F. (2015). The impact of behavior intervention programs on student suspensions. (Doctoral Dissertation). East Carolina University. [Google Scholar]
White, C. (2013). Reducing school stand‐downs: A case study of one New Zealand secondary school. (Master Thesis), Massey University, New Zealand. [Google Scholar]
Wickham, I. (2010). A comparative evaluation to determine the effectiveness of the behaviour support classrooms and other positive behaviour management interventions in designated disadvantaged schools. (Doctoral Dissertation). Dublin City University. [Google Scholar]
Wilczynski, J. D. (2004). An evaluation of a high school alternative to suspension program for students with gang‐related disciplinary violations. (Doctoral Dissertation). Loyola University of Chicago, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2004‐99017‐053&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]
Wilkerson‐Arbisi, E. L. (2015). The influence of participation in School‐Wide Positive Behavior Support on elementary student behavior. (Doctoral Dissertation). Baker University. Retrieved from http://bakeru.bakeruniversity.netdna‐cdn.com/images/pdf/SOE/EdD_Theses/Wilkerson‐Arbisi_Elizabeth.pdf [Google Scholar]
Wilson, L. J. (1998). Conflict resolution programs and the Houston independent school district: Peer mediation vs. non‐peer mediation approaches for resolving student conflict in elementary and middle schools. (Master Thesis), University of Houston‐Clear Lake. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Wolgemuth, J. , Cobb, R. , & Dugan, J. (2007). The effects of self‐management interventions on academic outcomes for youth with disabilities. Ft. Collins, CO: Colorado State University, School of Education. [Google Scholar]
Woodfork, J. D. (1987). The effects of an assertive discipline approach on disciplnary problems in selected high schools. (Education Especialist Degree), Atlanta University. [Google Scholar]
Wooten, S. D. (2006). The impact of a ninth and tenth grade academy program on student attendance, discipline, and achievement. (Doctoral Dissertation). Clemson University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
York, M. T. (2012). The impact of problem‐based learning on ninth‐grade african american male disengagement in high school. (Doctoral Dissertation). Gardner‐Webb University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Youmans, C. W. (2001). Group counseling compared with individual counseling in the reduction of at‐risk behaviors in black male students. (Doctoral Dissertation). South Carolina State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]
Zucca‐Brown, S. (1997). An elementary school mediation program: Its effect on student mediators and school violence. (Doctoral Dissertation). Temple University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

7.2.2 Journal articles

Achilles, G. M. , McLaughlin, M. J. , & Croninger, R. G. (2007). Sociocultural correlates of disciplinary exclusion among students with emotional, behavioral, and learning disabilities in the SEELS national dataset. Journal of Emotional and Behavioral Disorders, 15(1), 33–45. doi:10.1177/10634266070150010401 [Google Scholar]
Alexander, R. , & Curtis, C. M. (1995). A critical review of strategies to reduce school violence. Children & Schools, 17(2), 73–82. [Google Scholar]
Allen, J. P. , Kuperminc, G. , Philliber, S. , & Herre, K. (1994). Programmatic prevention of adolescent problem behaviors: The role of autonomy, relatedness, and volunteer service in the Teen Outreach Program. American Journal of Community Psychology, 22(5), 617–638. doi:10.1007/bf02506896 [DOI] [PubMed] [Google Scholar]
Allen, J. P. , & Philliber, S. (2001). Who benefits most from a broadly targeted prevention program? Differential efficacy across populations in the Teen Outreach Program. Journal of Community Psychology, 29(6), 637–655. [DOI] [PMC free article] [PubMed] [Google Scholar]
Allen‐Meares, P. , Montgomery, K. L. , & Kim, J. S. (2013). School‐based social work interventions: A cross‐national systematic review. Social Work, 58(3), 253–262. doi:10.1093/sw/swt022 [DOI] [PubMed] [Google Scholar]
Andrews, S. P. , Taylor, P. B. , Martin, E. P. , & Slate, J. R. (1998). Evaluation of an alternative discipline program. The High School Journal, 81(4), 209–217. [Google Scholar]
Anyon, Y. , Jenson, J. M. , Altschul, I. , Farrar, J. , McQueen, J. , Greer, E. , … Simmons, J. (2014). The persistent effect of race and the promise of alternatives to suspension in school discipline outcomes. Children and Youth Services Review, 44, 379–386. doi:10.1016/j.childyouth.2014.06.025 [Google Scholar]
Arter, P. S. (2007). The positive alternative learning supports program: collaborating to improve student success. Teaching Exceptional Children, 40(2), 38–46. [Google Scholar]
Atkins, M. S. , McKay, M. M. , Frazier, S. L. , Jakobsons, L. J. , Arvanitis, P. , Cunningham, T. , … Lambrecht, L. (2002). Suspensions and detentions in an urban, low‐income school: Punishment or reward? Journal of Abnormal Child Psychology, 30(4), 361–371. doi:10.1023/a:1015765924135 [DOI] [PubMed] [Google Scholar]
Aughinbaugh, A. (2001). Does Head Start yield long‐term benefits? The Journal of Human Resources, 36(4), 641–665. [Google Scholar]
Bagley, C. , & Mallick, K. (2000). Spiralling up and spiralling down: Implications of a long‐term study of temperament and conduct disorder for social work with children. Child & Family Social Work, 5(4), 291–301. [Google Scholar]
Bagley, C. , & Pritchard, C. (1998). The reduction of problem behaviours and school exclusion in at‐risk youth: An experimental study of school social work with cost‐benefit analyses. Child & Family Social Work, 3(4), 219–226. doi:10.1046/j.1365‐2206.1998.00101.x [Google Scholar]
Ballard, K. L. , Sander, M. A. , & Klimes‐Dougan, B. (2014). School‐related and social‐emotional outcomes of providing mental health services in schools. Community Mental Health Journal, 50(2), 145–149. doi:10.1007/s10597‐013‐9670‐y [DOI] [PubMed] [Google Scholar]
Barboza, G. E. (2015). The association between school exclusion, delinquency and subtypes of cyber‐ and F2F‐victimizations: Identifying and predicting risk profiles and subtypes using latent class analysis. Child Abuse & Neglect, 39, 109–122. doi: 10.1016/j.chiabu.2014.08.007 [DOI] [PubMed] [Google Scholar]
Bartik, T. J. , & Lachowska, M. (2012). The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes. Upjohn Institute Working Paper No. 12‐186. W. E. Upjohn Institute for Employment Research. [Google Scholar]
Bartik, T. J. , & Lachowska, M. (2014). The Kalamazoo Promise Scholarship. Education Next, 14(2), 72–78. [Google Scholar]
Bauer, N. S. , Lozano, P. , & Rivara, F. P. (2007). The effectiveness of the Olweus Bullying Prevention Program in Public Middle Schools: A Controlled Trial. Journal of Adolescent Health, 40(3), 266–274. doi:10.1016/j.jadohealth.2006.10.005 [DOI] [PubMed] [Google Scholar]
Beets, M. , Flay, B. , Vuchinich, S. , Snyder, F. , Acock, A. , Li, K.‐K. , … Durlak, J. (2009). Use of a social and character development program to prevent substance use, violent behaviors, and sexual activity among elementary‐school students in Hawaii. Research and Practice, 99(8), 1438–1445. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bell, S. K. , Coleman, J. K. , Anderson, A. , Whelan, J. P. , & Wilder, C. (2000). The effectiveness of peer mediation in a low‐SES rural elementary school. Psychology in the Schools, 37(6), 505–516. doi:10.1002/1520‐6807(200011)37:6<505::aid‐pits3>3.0.co;2‐5 [Google Scholar]
Bevington, T. J. (2015). Appreciative evaluation of restorative approaches in schools. Pastoral Care in Education, 33(2), 105–115. doi:10.1080/02643944.2015.1046475 [Google Scholar]
Bonell, Fletcher, A. , Fitzgerald‐Yau, N. , Hale, D. , Allen, E. , Elbourne, D. , … Viner, R. (2015). Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): A pilot randomised controlled trial. Health Technology Assessment, 19(53), i‐109. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bouhours, T. , Bryer, F. , & Fleming, S. (2003). Exclusion of primary school students: Archival analysis of school records over a 30‐year period. Reimagining Practice: Researching Change, 1, 102–114. [Google Scholar]
Bowey, L. , & McGlaughlin, A. (2006). The Youth Crime Reduction Video Project: An evaluation of a pilot intervention targeting young people at risk of crime and school exclusion. The Howard Journal, 45(3), 268–283. [Google Scholar]
Bowllan, N. M. (2011). Implementation and evaluation of a comprehensive, school‐wide bullying prevention program in an urban/suburban middle school. Journal of School Health, 81(4), 167–173. doi:10.1111/j.1746‐1561.2010.00576.x [DOI] [PubMed] [Google Scholar]
Bradley, R. H. , & Gilkey, B. (2002). The impact of the home instructional program for preschool youngsters (HIPPY) on school performance in 3rd and 6th grades. Early Education and Development, 13(3), 301–312. [Google Scholar]
Bradshaw, C. P. , Mitchell, M. M. , & Leaf, P. J. (2010). Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes. Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12(3), 133–148. doi:10.1177/1098300709334798 [Google Scholar]
Brady, K. P. , Balmer, S. , & Phenix, D. (2007). School‐police partnership effectiveness in urban schools ‐ An analysis of New York City's Impact Schools Initiative. Education and Urban Society, 39(4), 455–478. doi:10.1177/0013124507302396 [Google Scholar]
Breunlin, D. C. , Cimmarusti, R. A. , Bryant‐Edwards, T. L. , & Hetherington, J. S. (2002). Conflict resolution training as an alternative to suspension for violent behavior. Journal of Educational Research, 95(6), 349–357. doi: 10.1080/00220670209596609 [DOI] [Google Scholar]
Breunlin, D. C. , Cimmarusti, R. A. , Hetherington, J. S. , & Kinsman, J. (2006). Making the Smart Choice: A systemic response to school‐based violence. Journal of Family Therapy, 28(3), 246–266. doi:10.1111/j.1467‐6427.2006.00350.x [Google Scholar]
Bruns, E. J. , Moore, E. , Stephan, S. H. , Pruitt, D. , & Weist, M. D. (2005). The impact of school mental health services on out‐of‐school suspension rates. Journal of Youth and Adolescence, 34(1), 23–30. doi:10.1007/s10964‐005‐1333‐z [Google Scholar]
Burke, R. V. , Oats, R. G. , Ringle, J. L. , Fichtner, L. O. N. , & DelGaudio, M. B. (2011). Implementation of a classroom management program with urban elementary schools in low‐income neighborhoods: Does program fidelity affect student behavior and academic outcomes? Journal of Education for Students Placed at Risk (JESPAR), 16(3), 201–218. doi: 10.1080/10824669.2011.585944 [DOI] [Google Scholar]
Burton, S. (2006). ‘Over to You’: Group work to help pupils avoid school exclusion. Educational Psychology in Practice, 22(3), 215–236. doi: 10.1080/02667360600845778 [DOI] [Google Scholar]
Calbreath, W. , & Crews, M. E. (2011). Peer mediation and middle school students. Perspectives in Peer Programs, 23(1), 3–8. [Google Scholar]
Campbell‐Whatley, G. D. , Obiakor, F. , & Algozzine, B. (1997). Effects of innovative programming on youngsters at risk. Journal of At‐Risk Issues, 4(1), 3–9. [Google Scholar]
Carlson, C. E. (1990). HIPP: A comprehensive school‐based substance abuse program with cooperative community involvement. Journal of Primary Prevention, 10(4), 289–302. doi:10.1007/bf01326623 [DOI] [PubMed] [Google Scholar]
Carney, M. M. , & Buttell, F. (2003). Reducing juvenile recidivism: Evaluating the wraparound services model. Research on Social Work Practice, 13(5), 551–568. doi:DOI: 10.1177/1049731503253364 [Google Scholar]
Case, S. , & Haines, K. (2004). Promoting prevention: Evaluating a multi‐agency initiative of Youth Consultation and Crime Prevention in Swansea. Children & Society, 18, 355–370. doi:10.1002/CHI.814 [Google Scholar]
Casey, R. E. (1996). Delinquency prevention through vocational entrepreneurship: The new Smyrna Beach Employability skill training model for youthful offenders. Preventing School Failure: Alternative Education for Children and Youth, 40(2), 60–62. doi:10.1080/1045988X.1996.9944654 [Google Scholar]
Catalano, R. F. , Berglund, M. L. , Ryan, J. A. , Lonczak, H. S. , & Hawkins, J. D. (2004). Positive youth development in the United States: Research findings on evaluations of Positive Youth Development programs. The Annals of the American Academy, 591(January), 98–124. doi:DOI: 10.1177/0002716203260102 Catch22. Engage in Education Overview. [Google Scholar]
Chafouleas, S. M. , & Whitcomb, M. (2004). Integrating home, school, and community resources: Evaluation of a district‐wide prevention program. Reclaiming Children & Youth, 12(4), 203–209. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=cja&AN=12328455&site=ehost‐live&scope=site [Google Scholar]
Charlton, T. , Panting, C. , & Willis, H. (2004). Targeting exclusion, disaffection and truancy in secondary schools: An evaluation of an alternative curriculum for older pupils. Emotional and Behavioural Difficulties, 9(4), 261–275. doi:DOI:10.1177/1363275204050372 [Google Scholar]
Chobot, R. B. , & Garibaldi, A. (1982). In‐school alternatives to suspension: A description of Ten School District programs. The Urban Review, 14(4), 317–336. doi:10.1007/BF02171976 [Google Scholar]
Cooper, P. , & Lovey, J. (1999). Early intervention in emotional and behavioural difficulties: The role of Nurture Groups. European Journal of Special Needs Education, 14(2), 122–131. doi:10.1080/0885625990140202 [Google Scholar]
Cornell, D. , Sheras, P. , Gregory, A. , & Fan, X. (2009). A retrospective study of school safety conditions in high schools using the Virginia Threat Assessment Guidelines versus alternative approaches. School Psychology Quarterly, 24(2), 119–129. doi: 10.1037/a0016182 [Google Scholar]
Cornell, D. G. , Gregory, A. , & Fan, X. (2011). Reductions in long‐term suspensions following adoption of the Virginia Student Threat Assessment Guidelines. NASSP Bulletin, 95(3), 175–194. doi:10.1177/0192636511415255 [Google Scholar]
Cornell, D. G. , Sheras, P. L. , Kaplan, S. , McConville, D. , Douglas, J. , Elkton, A. , … Cole, J. (2004). Guidelines for student threat assessment: Field‐test findings. School Psychology Review, 33(4), 527–546. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=cja&AN=CJA0380020000830s&site=ehost‐live&scope=site [Google Scholar]
Cunningham, C. E. , Vaillancourt, T. , Cunningham, L. J. , Chen, Y. , & Ratcliffe, J. (2011). Modeling the bullying prevention program design recommendations of students from grades five to eight: A discrete choice conjoint experiment. Aggressive Behavior, 37(6), 521–537. doi:10.1002/ab.20408 [DOI] [PubMed] [Google Scholar]
Cunningham, P. B. , & Henggeler, S. W. (2001). Implementation of an empirically based drug and violence prevention and intervention program in public school settings. Journal of Clinical Child & Adolescent Psychology, 30(2), 221–232. doi:10.1207/s15374424jccp3002_9 [DOI] [PubMed] [Google Scholar]
Curtis, R. , Van Horne, J. W. , Robertson, P. , & Karvonen, M. (2010). Outcomes of a School‐Wide Positive Behavioral Support Program. Professional School Counseling, 13(3), 159–164. [Google Scholar]
Daly, B. P. , Sander, M. A. , Nicholls, E. G. , Medhanie, A. , Vanden Berk, E. , & Johnson, J. (2014). Three‐year longitudinal study of school behavior and academic outcomes: Results from a comprehensive expanded school mental health program. Advances in School Mental Health Promotion, 7(1), 24–41. doi:10.1080/1754730X.2013.867712 [Google Scholar]
Davis, F. (2014). Discipline with dignity: Oakland classrooms try healing Instead of punishment. Reclaiming Children and Youth, 23(1), 38–41. [Google Scholar]
Dawson, C. A. (2003). A Study on the effectiveness of life space crisis Intervention for students identified with emotional disturbances. Reclaiming Children and Youth, 11(4), 223–230. [Google Scholar]
Dawson, C. A. (2015). The multifaceted conflict cycle. A New York City perspective. Reclaiming Children and Youth, 24(1), 25–32. [Google Scholar]
Day, C. , Kowalenko, S. , Ellis, M. , Dawe, S. , Harnett, P. , & Scott, S. (2011). The Helping Families Programme: A new parenting intervention for children with severe and persistent conduct problems. Child and Adolescent Mental Health, 16(3), 167–171. doi:10.1111/j.1475‐3588.2010.00588.x [DOI] [PubMed] [Google Scholar]
Day‐Vines, N. L. , & Terriquez, V. (2008). A strengths‐based approach to promoting prosocial behavior among African American and Latino students. Professional School Counseling, 12(2), 170–175. [Google Scholar]
DeAngelo, A. J. (2005). The evolution of juvenile justice: Community‐based partnerships through balanced and restorative justice. Corrections Today, 67(5), 104–106. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=cja&AN=17800473&site=ehost‐live&scope=site [Google Scholar]
Dupper, D. R. (1998). An alternative to suspension for middle school youths with behavior problems: Findings from a “school survival” group. Research on Social Work Practice, 8(3), 354–366. doi:10.1177/104973159800800307 [Google Scholar]
Dupper, D. R. , & Krishef, C. H. (1993). School‐based social‐cognitive skills training for middle school students with school behavior problems. Children and Youth Services Review, 15(2), 131–142. doi:10.1016/0190‐7409(93)90040‐g [Google Scholar]
Englander‐Golden, P. , Elconin, J. , & Miller, K. J. (1985). Aay it straight: Adolescent substance abuse prevention training. Academic Psychology Bulletin, 7, 65–79. [Google Scholar]
Englander‐Golden, P. , Elconin, J. , Miller, K. J. , & Schwarzkopf, A. B. (1986). Brief SAY IT STRAIGHT training and follow‐up in adolescent substance abuse prevention. Journal of Primary Prevention, 6(4), 219–230. [DOI] [PubMed] [Google Scholar]
Englander‐Golden, P. , Jackson, J. E. , Crane, K. , Schwarzkopf, A. B. , & Lyle, P. S. (1989). Communication‐skills and self‐esteem in prevention of destructive behaviors. Adolescence, 24(94), 481–502. Retrieved from <Go to ISI>://WOS:A1989AF24100023 [PubMed] [Google Scholar]
Ethier, K. , & St Lawrence, J. S. (2002). The role of early, multilevel youth development programs in preventing health risk behavior in adolescents and young adults. Archives of Pediatrics & Adolescent Medicine, 156(5), 429–430. doi:PMID: 11980546 [DOI] [PubMed] [Google Scholar]
Etscheidt, S. (2002). Discipline provisions of IDEA: Misguided policy or tacit reform initiatives? Behavioral Disorders, 27(4), 408. [Google Scholar]
Evangelou, M. , Brooks, G. , & Smith, S. (2007). The birth to school study: Evidence on the effectiveness of PEEP, an early intervention for children at risk of educational under‐achievement. Oxford Review of Education, 33(5), 581–609. doi:10.1080/03054980701476477 [Google Scholar]
Fernandez, E. (2008). Unravelling emotional, behavioural and educational outcomes in a longitudinal study of children in foster‐care. British Journal of Social Work, 38(7), 1283–1301. doi: 10.1093/bjsw/bcm028 [DOI] [Google Scholar]
Fernandez‐Batanero, J. M. (2011). In search of differentiating features to enhance the performance and expectations of students at risk of educational exclusion. Revista De Educacion, 355(May), 309–330. doi:10.4438/1988‐592x‐re‐2011‐355‐026 [Google Scholar]
Flay, B. R. , & Allred, C. G. (2003). Long‐term effects of the Positive Action® program. American Journal of Health Behavior, 27(Supplement 1), S6–S21. doi:10.5993/AJHB.27.1.s1.2 [PubMed] [Google Scholar]
Fletcher, A. , Harden, A. , Brunton, G. , Oakley, A. , & Bonell, C. (2007). Interventions addressing the social determinants of teenage pregnancy. Health Education, 108(1), 29–39. doi: 10.1108/09654280810842111 [DOI] [Google Scholar]
Freiberg, H. J. , Huzinec, C. A. , & Templeton, S. M. (2009). Classroom management—a pathway to student achievement: A study of fourteen inner‐city elementary schools. The Elementary School Journal, 110(1), 63–80. doi:10.1086/598843 [Google Scholar]
Frigon, A. P. , & Krank, M. D. (2009). Self‐coded indirect memory associations in a brief school‐based intervention for substance use suspensions. Psychology of Addictive Behaviors, 23(4), 736–742. doi:10.1037/a0017125 [DOI] [PubMed] [Google Scholar]
Frith, G. H. , Lindsey, J. D. , & Sasser, J. L. (1980). An alternative approach to school suspension: The Dothan Model CHECK UL. Phi Delta Kappan, 61(9), 637–638. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ221601&site=ehost‐live&scope=site [Google Scholar]
Gage, N. A. , Sugai, G. , Lewis, T. J. , & Brzozowy, S. (2015). Academic achievement and School‐Wide Positive Behavior Supports. Journal of Disability Policy Studies, 25(4), 199–209. doi:10.1177/1044207313505647 [Google Scholar]
Garibaldi, A. , Blanchard, L. , & Brooks, S. (1996). Conflict resolution training, teacher effectiveness, and student suspension: The impact of a health and safety initiative in the New Orleans public schools. Journal of Negro Education, 65(4), 408–413. doi:10.2307/2967143 [Google Scholar]
Gentile, E. , & Imberman, S. A. (2012). Dressed for success? The effect of school uniforms on student achievement and behavior. Journal of Urban Economics, 71(1), 1–17. doi:10.1016/j.jue.2011.10.002 [Google Scholar]
Gonzalez, T. (2012). Keeping kids in school: Restorative justice, punitive discipline, and the school to prison pipeline. Journal of Law & Education, 41(2), 281–335. [Google Scholar]
Gottfredson, D. C. (1986). An empirical test of school‐based environmental and individual interventions to reduce the risk of delinquent behavior. Criminology, 24(4), 705–731. doi:10.1111/j.1745‐9125.1986.tb01508.x [Google Scholar]
Gottfredson, D. C. , Gottfredson, G. D. , & Hybl, L. G. (1993). Managing adolescent behavior. A multiyear, multischool study. American Educational Research Journal, 30(1), 179–215. doi:10.3102/00028312030001179 [Google Scholar]
Gottfredson, G. D. , Jones, E. M. , & Gore, T. W. (2002). Implementation and evaluation of a cognitive‐behavioral intervention to prevent problem behavior in a disorganized school. Prevention Science, 3(1), 43–56. [DOI] [PubMed] [Google Scholar]
Gray, P. , & Seddon, T. (2005). Prevention work with children disaffected from school: Findings from the evaluation of two innovative community‐based projects. Health Education, 105(1), 62–72. [Google Scholar]
Green, D. G. (2006). Welcome to the House System. Educational Leadership, 63(7), 64–67. [Google Scholar]
Green, J. M. , Wood, A. J. , Kerfoot, M. J. , Trainor, G. , Roberts, C. , Rothwell, J. , … Harrington, R. (2011). Group therapy for adolescents with repeated self harm: Randomised controlled trial with economic evaluation. British Medical Journal, 342(d682). doi:10.1136/bmj.d682 [DOI] [PMC free article] [PubMed] [Google Scholar]
Greflund, S. , McIntosh, K. , Mercer, S. H. , & May, S. L. (2014). Examining disproportionality in school discipline for Aboriginal students in schools implementing PBIS. Canadian Journal of School Psychology, 29(3), 213–235. doi:10.1177/0829573514542214 [Google Scholar]
Gresham, F. M. , Hunter, K. K. , Corwin, E. P. , & Fischer, A. J. (2013). Screening, assessment, treatment, and outcome evaluation of behavioral difficulties in an RTI model. Exceptionality, 21(1), 19–33. doi:10.1080/09362835.2013.750115 [Google Scholar]
Grizenko, N. , Zappitelli, M. , Langevin, J.‐P. , Hrychko, S. , El‐Messidi, A. , Kaminester, D. , … Stepanian, M. T. (2000). Effectiveness of a social skills training program using self/other perspective‐taking: A nine‐month follow‐up. American Journal of Orthopsychiatry, 70(4), 501–509. doi:10.1037/h0087662 [DOI] [PubMed] [Google Scholar]
Güneri, O. Y. , & Çoban, R. (2004). The effect of conflict resolution training on Turkish elementary school students: A quasi‐experimental investigation. International Journal for the Advancement of Counselling, 26(2), 109–124. doi:10.1023/B:ADCO.0000027425.24225.87 [Google Scholar]
Haines, K. , & Case, S. (2005). Promoting prevention: Targeting family‐based risk and protective factors for drug use and youth offending in Swansea. British Journal of Social Work, 35(2), 169–187. doi:10.1093/bjsw/bch177 [Google Scholar]
Hallam, S. (2007). Evaluation of behavioural management in schools: A review of the behaviour improvement programme and the role of behaviour and education support teams. Child and Adolescent Mental Health, 12(3), 106–112. doi:10.1111/j.1475‐3588.2007.00442.x [DOI] [PubMed] [Google Scholar]
Hallam, S. , & Castle, F. (2001). Exclusion from school: What can help prevent it? Educational Review, 53(2), 169–179. doi:10.1080/00131910120055598 [Google Scholar]
Harris, I. M. (1998). The coalition for nonviolence in schools. A case study in Milwaukee, Wisconsin, USA. Peace Research, 30(2), 62. [Google Scholar]
Hartnell, N. (2010). Multi‐disciplinary approaches to pupil behaviour in school. The role of evaluation in service delivery. Educational Psychology in Practice, 26(2), 187–203. doi:10.1080/02667361003768534 [Google Scholar]
Hausman, A. , Pierce, G. , & Briggs, L. (1996). Evaluation of comprehensive violence prevention education: Effects on student behavior. Journal of Adolescent Health, 19(2), 104–110. doi:10.1016/1054‐139x(96)00128‐0 [DOI] [PubMed] [Google Scholar]
Hawkins, J. D. , Catalano, R. F. , Kosterman, R. , Abbott, R. , & Hill, K. G. (1999). Preventing adolescent health‐risk behaviors by strengthening protection during childhood. Archives of Pediatrics & Adolescent Medicine, 153(3), 226–234. Retrieved from <Go to ISI>://WOS:000079019600003 [DOI] [PubMed] [Google Scholar]
Hayden, C. (2009). Family group conferences. Are they an effective and viable way of working with attendance and behaviour problems in schools? British Educational Research Journal, 35(2), 205–220. doi:10.1080/01411920802041939 [Google Scholar]
Hayden, C. , & Martin, T. (1998). ‘Safer Cities’ and exclusion from school. Journal of Youth Studies, 1(3), 315–331. doi:10.1080/13676261.1998.10593016 [Google Scholar]
Head, G. , Kane, J. , & Cogan, N. (2003). Behaviour support in secondary schools: what works for schools? Emotional and Behavioural Difficulties, 8(1), 33–42. doi:10.1080/13632750300507004 [Google Scholar]
Heilbrun, A. , Cornell, D. , & Lovegrove, P. (2015). Principal attitudes regarding zero tolerance and racial disparities in school suspensions. Psychology in the Schools, 52(5), 489–499. doi:10.1002/pits.21838 [Google Scholar]
Hemphill, S. A. , Kotevski, A. , Herrenkohl, T. I. , Smith, R. , Toumbourou, J. W. , & Catalano, R. F. (2013). Does school suspension affect subsequent youth non‐violent antisocial behaviour? A longitudinal study of students in Victoria, Australia and Washington State, United States. Australian Journal of Psychology, 65(4), 236–249. doi:10.1111/ajpy.12026 [DOI] [PMC free article] [PubMed] [Google Scholar]
Hemphill, S. A. , McMorris, B. J. , Toumbourou, J. W. , Herrenkohl, T. I. , Catalano, R.F. , & Mathers, M. (2007). Rates of student‐reported antisocial behavior, school suspensions, and arrests in Victoria, Australia and Washington state, United States*. The Journal of School Health, 77(6), 303–311. doi:10.1111/j.1746‐1561.2007.00211.x [DOI] [PubMed] [Google Scholar]
Hemphill, S. A. , Plenty, S. M. , Herrenkohl, T. L. , Toumbourou, J. W. , & Catalano, R. F. (2014). Student and school factors associated with school suspension: A multilevel analysis of students in Victoria, Australia and Washington State, United States. Children and Youth Services Review, 36, 187–194. doi:10.1016/j.childyouth.2013.11.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
Hemphill, S. A. , Toumbourou, J. W. , Smith, R. , Kendall, G. E. , Rowland, B. , Freiberg, K. , & Williams, J. W. (2010). Are rates of school suspension higher in socially disadvantaged neighbourhoods? An Australian study. Health Promotion Journal of Australia, 21(1), 12–18. Retrieved from <Go to ISI>://WOS:000289221300004 [DOI] [PubMed] [Google Scholar]
Henderson, D. X. , & Green, J. (2014). Using mixed methods to explore resilience, social connectedness, and re‐suspension among youth in a community‐based alternative‐to‐suspension Program. International Journal of Child, Youth and Family Studies, 5(3), 423–446. [Google Scholar]
Herrenkohl, T. I. , Hemphill, S. A. , Mason, W. A. , Toumbourou, J. W. , & Catalano, R. F. (2012). Predictors and responses to the growth in physical violence during adolescence: A comparison of students in Washington State and Victoria, Australia. American Journal of Orthopsychiatry, 82(1), 41–49. doi:10.1111/j.1939‐0025.2011.01139.x [DOI] [PMC free article] [PubMed] [Google Scholar]
Hilberth, M. , & Slate, J. R. (2014). Middle school black and white student assignment to disciplinary consequences. A clear lack of equity. Education and Urban Society, 46(3), 312–328. doi:10.1177/0013124512446218 [Google Scholar]
Hill, D. , & Brown, D. (2013). Supporting inclusion of at risk students in secondary school through positive behaviour support. International Journal of Inclusive Education, 17(8), 868–881. doi:10.1080/13603116.2011.602525 [Google Scholar]
Hockman, S. , & Worner, W. (1987). In‐school suspension and group counseling: Helping the at‐risk student. NASSP Bulletin, 71(501), 93–96. doi:10.1177/019263658707150124 [Google Scholar]
Holmlund, H. , & Silva, O. (2009). Targeting non‐cognitive skills to improve cognitive outcomes: Evidence from a remedial education intervention. IZA Discussion Paper, No. 4476. doi:10.1086/676460
Horigian, V. E. , Robbins, M. S. , Dominguez, R. , Ucha, J. , & Rosa, C. L. (2010). Principles for defining adverse events in behavioral intervention research: Lessons from a family‐focused adolescent drug abuse trial. Clinical Trials, 7(1), 58–68. doi:10.1177/1740774509356575 [DOI] [PMC free article] [PubMed] [Google Scholar]
Hudley, C. , & Friday, J. (1996). Attributional bias and reactive aggression. American Journal of Preventive Medicine, 12(5), 75–81. Retrieved from <Go to ISI>://WOS:A1996VR49100011 [PubMed] [Google Scholar]
Huff, J. A. (1988). Personalized behavior modification: An in‐school suspension program that teaches students how to change. The School Counselor, 35(3), 210–214. doi:http://www.jstor.org/stable/23900901 [Google Scholar]
Humphrey, N. , & Brooks, A. G. (2006). An evaluation of a short cognitive‐behavioural anger management intervention for pupils at risk of exclusion. Emotional & Behavioural Difficulties, 11(1), 5–23. doi:10.1080/13632750500392856 [Google Scholar]
James‐Burdumy, S. , Dynarski, M. , & Deke, J. (2008). After‐school program effects on behavior: Results from the 21st century community learning centers program national evaluation. Economic Inquiry, 46(1), 13–18. doi:10.1111/j.1465‐7295.2007.00074.x [Google Scholar]
Jennings, W. G. , Gover, A. R. , & Hitchcock, D. M. (2008). Localizing restorative justice: An in‐depth look at a Denver public school program. Restorative Justice: From Theory to Practice, 11, 167–187. doi:10.1016/s1521‐6136(08)00407‐7 [Google Scholar]
Johnson, D. W. , & Johnson, R. T. (1996). Conflict resolution and peer mediation programs in elementary and secondary schools: A review of the research. Review of Educational Research, 66(4), 459–506. doi: 10.3102/00346543066004459 [DOI] [Google Scholar]
Johnston, J. (2001). Evaluating national initiatives: The case of ‘On Track’. Children & Society, 15(1), 33–36. doi:10.1111/j.1099‐0860.2001.tb00201.x [Google Scholar]
Jolivette, K. , Patterson, D. P. , Swoszowski, N. C. , McDaniel, S. C. , Kennedy, C. , & Ennis, R. P. (2014). School‐wide positive behavioral interventions and supports in a residential school for students with emotional and behavioral disorders: First years of implementation and maintenance follow‐up focus groups. Residential Treatment for Children & Youth, 31(1), 63–79. doi:10.1080/0886571X.2014.878584 [Google Scholar]
Jones, J. , & Smith, C. (2004). Reducing exclusions whilst developing effective intervention and inclusion for pupils with behaviour difficulties. Emotional and Behavioural Difficulties, 9(2), 115–129. doi:10.1177/1363275204045732 [Google Scholar]
Joo, M. (2010). Long‐term effects of Head Start on academic and school outcomes of children in persistent poverty: Girls vs. boys. Children and Youth Services Review, 32(6), 807–814. doi:10.1016/j.childyouth.2010.01.018 [Google Scholar]
Kang‐Yi, C. D. , Mandell, D. S. , & Hadley, T. (2013). School‐based mental health program evaluation: Children's school outcomes and acute mental health service use. Journal of School Health, 83(7), 463–472. doi:10.1111/josh.12053 [DOI] [PubMed] [Google Scholar]
Kanu, M. , Hepler, N. , & Labi, H. (2015). Assessing effectiveness of students taking a right stand (STARS) Nashville Student Assistance Programs. Journal of Child & Adolescent Substance Abuse, 24(6), 323–330. doi:10.1080/1067828X.2013.829015 [Google Scholar]
Kearney, C. A. , & Graczyk, P. (2014). A response to intervention model to promote school attendance and decrease school absenteeism. Child & Youth Care Forum, 43(1), 1–25. doi:10.1007/s10566‐013‐9222‐1 [Google Scholar]
Kennedy, C. , & Jolivette, K. (2008). The effects of positive verbal reinforcement on the time spent outside the classroom for students with emotional and behavioral disorders in a residential setting. Behavioral Disorders, 33(4), 211–221. [Google Scholar]
Kilian, J. M. , Fish, M. C. , & Maniago, E. B. (2006). Making schools safe: A system‐wide school intervention to increase student prosocial behaviors and enhance school climate. Journal of Applied School Psychology, 23(1), 1–30. doi:10.1300/J370v23n01_01 [Google Scholar]
Koffman, S. , Ray, A. , Berg, S. , Covington, L. , Albarran, N. M. , & Vasquez, M. (2009). Impact of a comprehensive whole child intervention and prevention program among youths at risk of gang involvement and other forms of delinquency. Children & Schools, 31(4), 239–245. doi:10.1093/cs/31.4.239 [Google Scholar]
Kremer, K. P. , Maynard, B. R. , Polanin, J. R. , Vaughn, M. G. , & Sarteschi, C. M. (2015). Effects of after‐school programs with at‐risk youth on attendance and externalizing behaviors: A systematic review and meta‐analysis. Journal of Youth & Adolescence, 44, 616–636. doi:10.1007/s10964‐014‐0226‐4 [DOI] [PMC free article] [PubMed] [Google Scholar]
Kutash, K. , Duchnowski, A. J. , Green, A. L. , & Ferron, J. (2013). Effectiveness of the parent connectors program: Results from a randomized controlled trial. School Mental Health, 5(4), 192–208. doi:10.1007/s12310‐013‐9106‐4 [Google Scholar]
Kutash, K. , Duchnowski, A. J. , & Lynn, N. (2009). The use of evidence‐based instructional strategies in special education settings in secondary schools: Development, implementation and outcomes. Teaching and Teacher Education, 25(6), 917–923. doi:10.1016/j.tate.2009.02.016 [Google Scholar]
Lassen, S. R. , Steele, M. M. , & Sailor, W. (2006). The relationship of school‐wide positive behavior support to academic achievement in an urban middle school. Psychology in the Schools, 43(6), 701–712. doi:10.1002/pits.20177 [Google Scholar]
Law, J. , & Sivyer, S. (2003). Promoting the communication skills of primary school children excluded from school or at risk of exclusion: An intervention study. Child Language Teaching and Therapy, 19(1), 1–25. doi: 10.1191/0265659003ct241oa [DOI] [Google Scholar]
Leff, S. S. (2015. On going). Determining the efficacy of a relational aggression intervention for urban African American girls. Retrieved from https://clinicaltrials.gov/ct2/show/NCT00510094?term=school+suspension&age=0&rank=23
Leventhal, T. , & Brooks‐Gunn, J. (2004). A randomized study of neighborhood effects on low‐income children's educational outcomes. Developmental Psychology, 40(4), 488–507. doi:10.1037/0012‐1649.40.4.488 [DOI] [PubMed] [Google Scholar]
Lovey, J. , & Cooper, P. (1997). Positive alternatives to school exclusion: An empirical investigation. Emotional and Behavioural Difficulties, 2(3), 12–22. doi:10.1080/1363275970020303 [Google Scholar]
Luiselli, J. K. , Putnam, R. F. , Handler, M. W. , & Feinberg, A. B. (2005). Whole‐school positive behaviour support: Effects on student discipline problems and academic performance. Educational Psychology, 25(2/3), 183–198. [Google Scholar]
Lupton, C. , & Sheppard, C. (1999). Lost lessons?: The experience of a time‐limited Home‐School support project. Children & Society, 13(1), 20–31. doi:10.1111/j.1099‐0860.1999.tb00098.x [Google Scholar]
Maag, J. W. (2012). School‐wide discipline and the intransigency of exclusion. Children and Youth Services Review, 34(10), 2094–2100. doi:10.1016/j.childyouth.2012.07.005 [Google Scholar]
Macpherson, R. (2013). Evaluating three school‐based integrated health centres established by a partnership in cornwall to inform future provision and practice. International Journal of Educational Management, 27(5), 1–41. Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+evaluation&pg=3&id=EJ1014226 [Google Scholar]
Maguire, M. , Macrae, S. , & Milbourne, L. (2003). Early interventions: Preventing school exclusions in the primary setting. Westminster studies in education, 26(1), 45–62. doi:10.1080/0140672030260105 [Google Scholar]
Maimon, D. , Antonaccio, O. , & French, M. T. (2012). Severe sanctions, easy choice? Investigating the role of school sanctions in preventing adolescent violent offending. Criminology, 50(2), 495–524. doi: 10.1111/j.1745-9125.2011.00268.x [DOI] [Google Scholar]
Maras, P. , Bradshaw, V. , & Croft, C. (2008). Psychology and interventions for pupils at risk of underachievement and school exclusion: The year 10 effect. International Journal of Psychology, 43(3‐4), 379–379. Retrieved from <Go to ISI>://WOS:000259264304490 [Google Scholar]
Martin, D. , Martin, M. , Gibson, S. S. , & Wilkins, J. (2007). Increasing prosocial behavior and academic achievement among adolescent African American males. Adolescence, 42(168), 689–698. Retrieved from <Go to ISI>://WOS:000251753900005 [PubMed] [Google Scholar]
Martinez Martinez, K. I. , Salazar Garza, M. L. , Pedroza Cabrera, F. J. , Ruiz Torres, G. M. , & Ayala Velazquez, H. E. (2008). Preliminary study of a brief intervention program for adolescents who initiate alcohol and other drugs consumption. Salud Mental, 31(2), 119–127. Retrieved from <Go to ISI>://WOS:000256273500006 [Google Scholar]
Massey, O. T. , Boroughs, M. , & Armstrong, K. H. (2007). School violence interventions in the safe schools/healthy students initiative: Evaluation of two early intervention programs. Journal of School Violence, 6(2), 57–74. doi:10.1300/J202v06n02_04 [Google Scholar]
Matthews, A. , & Swan, W. W. (1999). TOPS: An alternative approach for the at‐risk middle school student. Journal of At‐Risk Issues, 6(1), 24–31. [Google Scholar]
Mattison, R. E. , & Schneider, J. (2009). First‐year effectiveness on school functioning of a self‐contained ED middle school. Behavioral Disorders, 34(2), 60–71. Retrieved from <Go to ISI>://WOS:000280896400001 [Google Scholar]
Mavroveli, S. , Petrides, K. V. , Shove, C. , & Whitehead, A. (2008). Investigation of the construct of trait emotional intelligence in children. European Child & Adolescent Psychiatry, 17(8), 516–526. doi:10.1007/s00787‐008‐0696‐6 [DOI] [PubMed] [Google Scholar]
May, D. C. , Stokes, E. , Oliver, A. , & McClure, T. (2015). Exploring alternatives to out‐of‐school suspension in Kentucky: A quasi‐experimental study examining the effectiveness of community service work. PRISM: A Journal of Regional Engagement, 4(1), 15–32. [Google Scholar]
McCord, M. T. , Klein, J. D. , Foy, J. M. , & Fothergill, K. (1993). School‐based clinic use and school performance. Journal of Adolescent Health, 14(2), 91–98. doi:10.1016/1054‐139x(93)90091‐3 [DOI] [PubMed] [Google Scholar]
McCrary, D. , Lechtenberger, D. , & Wang, E. (2012). The effect of schoolwide positive behavioral supports on children in impoverished rural community schools. Preventing School Failure, 56(1), 1–7. doi:10.1080/1045988X.2010.548417 [Google Scholar]
McDaniel, S. C. , & Bruhn, A. L. (2015). Using a changing‐criterion design to evaluate the effects of Check‐in/Check‐out with goal modification. Journal of Positive Behavior Interventions, 1–12. [Google Scholar]
McGarvey, E. L. , Leon‐Verdin, M. , Bloomfield, K. , Wood, S. , Winters, E. , & Smith, J. (2014). Effectiveness of A‐CRA/ACC in treating adolescents with cannabis‐use disorders. Community Mental Health Journal, 50(2), 150–157. doi:10.1007/s10597‐012‐9566‐2 [DOI] [PubMed] [Google Scholar]
Mears, C. J. , Lawler, E. N. , Sanders, L. D. , & Katz, B. Z. (2009). Efficacy of LAIV‐T on absentee rates in a school‐based health center sample. Journal of Adolescent Health, 45(1), 91–94. doi:10.1016/j.jadohealth.2008.12.010 [DOI] [PubMed] [Google Scholar]
Menendez, A. L. , Payne, L. D. , & Mayton, M. R. (2008). The Implementation of Positive Behavioral Support in an elementary school: Processes, procedures, and outcomes. The Alberta Journal of Educational Research, 54(4), 448–462. [Google Scholar]
Mitchell, R. R. , Tingstrom, D. H. , Dufrene, B. A. , Ford, W. B. , & Sterling, H. E. (2015). The effects of the Good Behavior Game with general‐education high school students. School Psychology Review, 44(2), 191–207. doi: 10.17105/spr-14-0063.1 [DOI] [Google Scholar]
Moore, A. J. (2014). Policy in practice: Enabling and inhibiting factors for the success of suspension centres. Australian Journal of Teacher Education, 39(11), 107–132. doi: 10.14221/ajte.2014v39n11.7 [DOI] [Google Scholar]
Moore, C. W. , & Allen, J. A. (1996). The effects of volunteering on the young volunteer. The Journal of Primary Prevention, 17(2), 231–258. doi:10.1007/BF02248794 [DOI] [PubMed] [Google Scholar]
Moraru, A. , Stoica, M. , Tomuletiu, E.‐A. , & Filpisan, M. (2011). Evaluation of a program for developing socio‐emotional competencies in preschool children. Procedia ‐ Social and Behavioral Sciences, 30, 2161–2164. doi:10.1016/j.sbspro.2011.10.419 [Google Scholar]
Munoz, M. A. , Fischetti, J. C. , & Prather, J. R. (2014). An early college initiative in an urban, high‐poverty high school: First‐year effects on student achievement and non‐academic indicators. Journal of Education for Students Placed at Risk (JESPAR), 19(1), 36–52. doi: 10.1080/10824669.2014.927746 [DOI] [Google Scholar]
Muschkin, C. G. , Glennie, E. , & Beck, A. N. (2014). Peer contexts: Do old for grade and retained peers influence student behavior in middle school? Teachers College Record, 116(4). [Google Scholar]
Muscott, H. S. , Mann, E. L. , & LeBrun, M. R. (2008). Positive Behavioral Interventions and Supports in New Hampshire Effects of Large‐Scale Implementation of Schoolwide Positive Behavior Support on Student Discipline and Academic Achievement. Journal of Positive Behavior Interventions, 10(3), 190–205. doi:10.1177/1098300708316258 [Google Scholar]
Muscott, H. S. , Szczesiul, S. , Berk, B. , Staub, K. , Hoover, J. , & Perry‐Chisholm, P. (2008). Creating home‐school partnerships by engaging families in schoolwide positive behavior supports. Teaching Exceptional Children, 40(6), 7–14. [Google Scholar]
Mytton, J. , DiGuiseppi, C. , Gough, D. , Taylor, R. , & Logan, S. (2006). School‐based secondary prevention programmes for preventing violence. Cochrane Database Syst Rev (3), Art. No.CD004606. doi:10.1002/14651858.CD004606.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
Navarro, J. I. , Aguilar, M. , Aguilar, C. , Alcalde, C. , & Marchena, E. (2007). Positive behavioral intervention in children who were wards of the court attending a mainstream school. Psychological Reports, 101(3), 1067–1078. doi:10.2466/pr0.101.4.1067‐1078 [DOI] [PubMed] [Google Scholar]
Nelson, J. R. , Hurley, K. D. , Synhorst, L. , Epstein, M. H. , Stage, S. , & Buckley, J. (2009). The child outcomes of a behavior model. Exceptional Children, 76(1), 7–30. Retrieved from http://digitalcommons.unl.edu/specedfacpub/57 [Google Scholar]
Nocera, E. J. , Whitbread, K. M. , & Nocera, G. P. (2014). Impact of School‐Wide Positive Behavior Supports on student behavior in the middle grades. RMLE Online: Research in Middle Level Education, 37(8), 1–14. [Google Scholar]
Noltemeyer, A. L. , Ward, R. M. , & McLoughlin, C. (2015). Relationship between school suspension and student outcomes: A meta‐analysis. School Psychology Review, 44(2), 224–240. doi:10.17105/spr‐14‐0008.1 [Google Scholar]
Obsuth, I. , Sutherland, A. , Pilbeam, L. , Scott, S. , Valdebenito, S. , Carr, R. , & Eisner, M. (2014). London Education and Inclusion Project (LEIP): A cluster‐randomised controlled trial protocol of an intervention to reduce antisocial behaviour and improve educational/occupational attainment for pupils at risk of school exclusion. BMC Psychology, 2(1), 24. Retrieved from http://www.biomedcentral.com/content/pdf/s40359‐014‐0024‐5.pdf [Google Scholar]
Olds, D. , Henderson, C. R. , Cole, R. , Eckenrode, J. , Kitzman, H. , Luckey, D. , … Powers, J. (1998). Long‐term effects of nurse home visitation on children's criminal and antisocial behavior. 15‐year follow‐up of a randomized controlled trial. Journal of the American Medical Association, 280(14), 1238–1244. doi:10.1001/jama.280.14.1238 [DOI] [PubMed] [Google Scholar]
Olweus, D. , & Limber, S. P. (2010). Bullying in school: Evaluation and dissemination of the Olweus Bullying Prevention program. American Journal of Orthopsychiatry, 80(1), 124–134. doi:10.1111/j.1939‐0025.2010.01015.x [DOI] [PubMed] [Google Scholar]
Orchard, L. (2007). Evaluating parenting classes held at a secondary school. Research in Post‐Compulsory Education, 12(1), 91–105. doi:10.1080/13596740601155454 [Google Scholar]
Pack, C. , White, A. , Raczynski, K. , & Wang, A. (2011). Evaluation of the safe school ambassadors program: A student‐led approach to reducing mistreatment and bullying in schools. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 84(4), 127–133. doi:10.1080/00098655.2011.564974 [Google Scholar]
Panayiotopoulos, C. (2004). A follow‐up of a home and school support project for children with emotional and behavioural difficulties. Emotional and Behavioural Difficulties, 9(2), 85–98. doi:10.1177/1363275204045730 [Google Scholar]
Panayiotopoulos, C. , & Kerfoot, M. (2007). Early intervention and prevention for children excluded from primary schools. International Journal of Inclusive Education, 11(1), 59–80. doi:10.1080/13603110500392882 [Google Scholar]
Park, H. , Lin, C.‐H. , Liu, C. , & Tabb, K. M. (2015). The relationships between after‐school programs, academic outcomes, and behavioral developmental outcomes of Latino children from immigrant families: Findings from the 2005 National Household Education Surveys Program. Children and Youth Services Review, 53, 77–83. doi:10.1016/j.childyouth.2015.03.019 [Google Scholar]
Parkes, R. (2008). Persistent young offenders: An evaluation of two projects. British Journal of Social Work, 38(5), 1031–1033. doi: 10.1093/bjsw/bcn091 [DOI] [Google Scholar]
Pas, E. T. , Bradshaw, C. P. , Hershfeldt, P. A. , & Leaf, P. J. (2010). A multilevel exploration of the influence of teacher efficacy and burnout on response to student problem behavior and school‐based service use. School Psychology Quarterly, 25(1), 13–27. doi:10.1037/a0018576 [Google Scholar]
Pelham, W. E., Jr. , Massetti, G. M. , Wilson, T. , Kipp, H. , Myers, D. , Standley, B. B. N. , … Waschbusch, D. A. (2005). Implementation of a comprehensive schoolwide behavioral intervention: The ABC program. Journal of Attention Disorders, 9(1), 248–260. doi:10.1177/1087054705281596 [DOI] [PubMed] [Google Scholar]
Potter, L. , & Bulach, C. (1996). An alternative program to improve student behavior: The focus program. Thresholds in Education, November, 14–16. [Google Scholar]
Powell, L. , Gilchrist, M. , & Stapley, J. (2008). A journey of self‐discovery: An intervention involving massage, yoga and relaxation for children with emotional and behavioural difficulties attending primary schools. European Journal of Special Needs Education, 23(4), 403–412. doi:10.1080/08856250802387398 [Google Scholar]
Pritchard, C. , Cotton, A. , Bowen, D. , & Williams, R. (1998). A consumer view study of young people's views on their educational social worker: Engagement as a measure of an effective relationship. British Journal of Social Work, 28, 915–938. [Google Scholar]
Pritchard, C. , & Williams, R. (2001). A three‐year comparative longitudinal study of a school‐based social work family service to reduce truancy, delinquency and school exclusions. Journal of Social Welfare & Family Law, 23(1), 23–43. doi:10.1080/09649060010012255 [Google Scholar]
Pritchard, C. , & Williams, R. (2009). Does social work make a difference?. A controlled study of former ‘looked‐after‐children’ and ‘excluded‐from‐school’ adolescents now men aged 16‐24 subsequent offences, being victims of crime and suicide. Journal of Social Work, 93(3), 285–307. [Google Scholar]
Putnam, R. F. , Handler, M. W. , Ramirez‐Platt, C. M. , & Luiselli, J. K. (2003). Improving student bus‐riding behavior through a whole‐school intervention. Journal of Applied Behavior Analysis, 36(4), 583–590. doi:10.1901/jaba.2003.36‐583 [DOI] [PMC free article] [PubMed] [Google Scholar]
Rapp‐Paglicci, L. , Stewart, C. , & Rowe, W. (2011). Can a self‐regulation skills and cultural arts program promote positive outcomes in mental health symptoms and academic achievement for at‐risk youth? Journal of Social Service Research, 37(3), 309–319. doi:10.1080/01488376.2011.564067 [Google Scholar]
Rauch, S. P. , Brack, C. J. , & Orr, D. P. (1987). School‐based, short‐term group treatment for behaviorally disturbed young adolescent males. A pilot intervention. Journal of School Health, 57(1), 19–22. doi:10.1111/j.1746‐1561.1987.tb00515.x [DOI] [PubMed] [Google Scholar]
Reglin, G. , Akpo‐Sanni, J. , & Losike‐Sedimo, N. (2012). The effect of a professional development classroom management model on at‐risk elementary students' misbehaviors. Education, 133(1), 3–18. [Google Scholar]
Reid, K. (2003). A strategic approach to tackling school absenteeism and truancy: The PSCC scheme. Educational Studies, 29(4), 351–371. doi:10.1080/0305569032000159660 [Google Scholar]
Reid, K. (2003). Strategic approaches to tackling school absenteeism and truancy: The traffic lights (TL) scheme. Educational Review, 55(3), 305–321. doi:10.1080/0013191032000118956 [Google Scholar]
Reynolds, C. R. , Skiba, R. J. , Graham, S. , Sheras, P. , Conoley, J. C. , & Garcia‐Vazquez, E. (2008). Are zero tolerance policies effective in the schools? An evidentiary review and recommendations. American Psychologist, 63(9), 852–862. doi:10.1037/0003‐066x.63.9.852 [DOI] [PubMed] [Google Scholar]
Rollin, S. A. , Kaiser‐Ulrey, C. , Potts, I. , & Creason, A. H. (2003). A school‐based violence prevention model for at‐risk eighth grade youth. Psychology in the Schools, 40(4), 403–416. doi:10.1002/pits.10111 [Google Scholar]
Rose, R. (2008). Encouraging engagement: an emerging role for school based family workers in english secondary schools. International Journal on School Disaffection, 6(1), 5–11. Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+evaluation&pg=4&id=EJ871300;http://files.eric.ed.gov/fulltext/EJ871300.pdf [Google Scholar]
Rosenbaum, J. (2012). Does school suspension increase long‐term substance use?: Quantifying the impact of school suspension on young adult substance use 12 years after school suspension using the Rubin causal inference model. Journal of Adolescent Health, 50(2), S75–S75. [Google Scholar]
Ross, S. W. , & Horner, R. H. (2009). Bully prevention in positive behavior support. Journal of Applied Behavior Analysis, 42(4), 747–759. doi:10.1901/jaba.2009.42‐747 [DOI] [PMC free article] [PubMed] [Google Scholar]
Royer, E. , Bitaudeau, I. , & Verville, H. (1996). L'exclusion scolaire au secondaire: évaluation d'un programme alternatif à la suspension scolaire. Suspension from secondary school: Evaluation of an alternative program. Revue Canadienne de Psycho‐Education, 25(1), 25–40. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1996‐93831‐001&site=ehost‐live&scope=site [Google Scholar]
Rundle‐Thiele, S. , Schuster, L. , Dietrich, T. , Russell‐Bennett, R. , Drennan, J. , Leo, C. , & Connor, J. P. (2015). Maintaining or changing a drinking behavior? Goka's short‐term outcomes. Journal of Business Research, 68(10), 2155–2163. doi: 10.1016/j.jbusres.2015.03.015 [DOI] [Google Scholar]
Safer, D. J. , Heaton, R. C. , & Parker, F. C. (1981). A behavioral‐program for disruptive junior‐high school students: Results and follow‐up. Journal of Abnormal Child Psychology, 9(4), 483–494. doi:10.1007/bf00917797 [DOI] [PubMed] [Google Scholar]
Sandler, I. , Wolchik, S. A. , Cruden, G. , Mahrer, N. E. , Ahn, S. , Brincks, A. , & Brown, C. H. (2014). Overview of meta‐analyses of the prevention of mental health, substance use and conduct problems. Annual Review of Clinical Psychology, 10, 243. doi:10.1146/annurev‐clinpsy‐050212‐185524 [DOI] [PMC free article] [PubMed] [Google Scholar]
Schellenberg, R. C. , Parks‐Savage, A. , & Rehfuss, M. (2007). Reducing levels of elementary school violence with peer mediation. Professional School Counseling, 10(5), 475–481. doi: 10.5330/prsc.10.5.q7866077l3v5q044 [DOI] [Google Scholar]
Schmidt, J. J. , & Biles, J. W. (1985). Puppetry as a group counseling technique with middle school students. Elementary School Guidance and Counseling, 20(1), 67–73. doi: http://www.jstor.org/stable/42868705 [Google Scholar]
Schnelling, K. , & Dew‐Hughes, D. (2002). A solution to exclusion: A total involvement model to support pupils and reduce exclusions. Emotional & Behavioural Difficulties, 7(4), 229–240. doi:10.1177/1363275202007004005 [Google Scholar]
Schwartz, S. E. O. , Rhodes, J. E. , Spencer, R. , & Grossman, J. B. (2013). Youth initiated mentoring: Investigating a new approach to working with vulnerable adolescents. American Journal of Community Psychology, 52(1‐2), 155–169. doi:10.1007/s10464‐013‐9585‐3 [DOI] [PubMed] [Google Scholar]
Scott, T. M. , & Barrett, S. B. (2004). Using staff and student time engaged in disciplinary procedures to evaluate the impact of school‐wide PBS. Journal of Positive Behavior Interventions, 6(1), 21–27. doi:10.1177/10983007040060010401 [Google Scholar]
Shapiro, J. P. , Burgoon, J. D. , Welker, C. J. , & Clough, J. B. (2002). Evaluation of the Peacemakers Program: School‐based violence prevention for students in grades four through eight. Psychology in the Schools, 39(1), 87–100. doi:10.1002/pits.10040 [Google Scholar]
Sharp, S. , & Davids, E. (2003). Early intervention in behaviour: A study of the FAST‐Track programme. Emotional and Behavioural Difficulties, 8(3), 173–188. doi:10.1080/13632750300507017 [Google Scholar]
Sheldon, S. B. , & Epstein, J. L. (2002). Improving student behavior and school discipline with family and community involvement. Education and Urban Society, 35(1), 4–26. doi:10.1177/001312402237212 [Google Scholar]
Simonsen, B. , Eber, L. , Black, A. C. , Sugai, G. , Lewandowski, H. , Sims, B. , & Myers, D. (2012). Illinois statewide Positive Behavioral Interventions and Supports: Evolution and impact on student outcomes across years. Journal of Positive Behavior Interventions, 14(1), 5–16. doi:10.1177/1098300711412601 [Google Scholar]
Skaggs, G. , & Bodenhorn, N. (2006). Relationships between implementing character education, student behavior, and student achievement. Journal of Advanced Academics, 18(1), 82–114. doi:10.4219/jaa‐2006‐345 [Google Scholar]
Skiba, R. , & Peterson, R. (2003). Teaching the social curriculum: School discipline as instruction. Preventing School Failure, 47(2), 66–73. doi:10.1080/10459880309604432 [Google Scholar]
Skiba, R. , Rausch, M. K. , & Ritter, S. (2004). “Discipline is always teaching”: Effective alternatives to zero tolerance in Indiana's schools. Education Policy Briefs., 2(3), 1–11. Retrieved from http://files.eric.ed.gov/fulltext/ED488903.pdf [Google Scholar]
Skiba, R. , Ritter, S. , Simmons, A. , Peterson, R. , & Miller, C. (2005). The safe and responsive schools project: A school reform model for implementing best practices in violence prevention. Handbook of school violence and school safety: From research to practice, 631–650.
Skiba, R. J. , Horner, R. H. , Chung, C.‐G. , Rausch, M. K. , May, S. L. , & Tobin, T. (2011). Race is not neutral: A national investigation of African American and Latino disproportionality in school discipline. School Psychology Review, 40(1), 85–107. [Google Scholar]
Smith, C. , Bicard, S. C. , Bicard, D. F. , & Casey, L. B. (2012). Decreasing in‐school suspensions with function‐based interventions. Kappa Delta Pi Record, 48(4), 174–177. doi:10.1080/00228958.2012.734013 [Google Scholar]
Smith, J. , Twemlow, S. W. , & Hoover, D. W. (1999). Bullies, victims and bystanders: A method of in‐school intervention and possible parental contributions. Child Psychiatry & Human Development, 30(1), 29–37. doi:10.1023/a:1018681819748 [DOI] [PubMed] [Google Scholar]
Spaulding, S. A. , Irvin, L. K. , Horner, R. H. , May, S. L. , Emeldi, M. , Tobin, T. J. , & Sugai, G. (2010). Schoolwide social‐behavioral climate, student problem behavior, and related administrative decisions. Journal of Positive Behavior Interventions, 12(2), 69–85. doi: 10.1177/1098300708329011 [DOI] [Google Scholar]
Spielman, D. A. , & Staub, E. (2000). Reducing boys' aggression: Learning to fulfill basic needs constructively. Journal of Applied Developmental Psychology, 21(2), 165–181. doi:10.1016/s0193‐3973(99)00034‐9 [Google Scholar]
Stader, D. L. (2000). Reducing violence in the high school: The power of peers. Journal of At‐Risk Issues, 7(1), 6–11. [Google Scholar]
Stage, S. A. (1997). A preliminary investigation of the relationship between in‐school suspension and the disruptive classroom behavior of students with behavioral disorders. Behavioral Disorders, 23(1), 57–76. Retrieved from http://www.jstor.org/stable/23888789 [Google Scholar]
Statham, J. (2004). Effective services to support children in special circumstances. Child Care, Health and Development, 30(6), 589–598. doi:10.1111/j.1365‐2214.2004.00472.x/pdf [DOI] [PubMed] [Google Scholar]
Sterrett, E. , Jones, D. J. , Zalot, A. , & Shook, S. (2010). A pilot study of a brief motivational intervention to enhance parental engagement: A brief report. Journal of Child and Family Studies, 19(6), 697–701. doi:10.1007/s10826‐010‐9356‐9 [Google Scholar]
Stewart, C. , Rapp, L. , & Rowe, W. (2015). Determining the efficacy of a community‐based early intervention program adapted for middle school. J Child Adolesc Behav, 3(4), 1–6. doi:10.4172/2375‐4494.1000228 [Google Scholar]
Strompolis, M. , Vishnevsky, T. , Reeve, C. L. , Munsell, E. P. , Cook, J. R. , & Kilmer, R. P. (2012). Educational outcomes in a system of care for children with emotional disturbance. American Journal of Orthopsychiatry, 82(1), 129–136. doi:10.1111/j.1939‐0025.2011.01126.x [DOI] [PubMed] [Google Scholar]
Sullivan, A. L. , Klingbeil, D. A. , & Van Norman, E. R. (2013). Beyond behavior: Multilevel analysis of the influence of sociodemographics and school characteristics on students' risk of suspension. School Psychology Review, 42(1), 99–114. [Google Scholar]
Sweeney‐Rader, J. , Snyder, G. L. , Goldstein, H. , & Rosenwald, P. (1980). School suspensions: An in‐house prevention model. Children today, 9(2), 19–21. [PubMed] [Google Scholar]
Tanner‐Smith, E. E. , & Wilson, S. J. (2013). A meta‐analysis of the effects of dropout prevention programs on school absenteeism. Prevention Science, 14, 468–478. doi:10.1007/s11121‐012‐0330‐1 [DOI] [PubMed] [Google Scholar]
Teasley, M. L. , & Miller, C. R. (2011). School social workers' perceived efficacy at tasks related to curbing suspension and undesirable behaviors. Children & Schools, 33(3), 136. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&id=EJ937659 [Google Scholar]
Teske, S. C. (2011). A study of zero tolerance policies in schools: A multi‐integrated systems approach to improve outcomes for adolescents. Journal of Child and Adolescent Psychiatric Nursing, 24(2), 88–97. doi:10.1111/j.1744‐6171.2011.00273.x [DOI] [PubMed] [Google Scholar]
Theriot, M. T. (2009). School resource officers and the criminalization of student behavior. Journal of Criminal Justice, 37(3), 280–287. doi:10.1016/j.jcrimjus.2009.04.008 [Google Scholar]
Theriot, M. T. , Craun, S. W. , & Dupper, D. R. (2010). Multilevel evaluation of factors predicting school exclusion among middle and high school students. Children and Youth Services Review, 32(1), 13–19. doi:10.1016/j.childyouth.2009.06.009 [Google Scholar]
Thompson, S. J. , Pollio, D. E. , Constantine, J. , Reid, D. , & Nebbitt, V. (2002). Short‐term outcomes for youth receiving runaway and homeless shelter services. Research on Social Work Practice, 12(5), 589–603. doi:10.1177/1049731502012005001 [Google Scholar]
Thompson, S. M. (1996). Peer mediation: A peaceful solution. The School Counselor, 44(2), 151–154. Retrieved from http://www.jstor.org/stable/23903453 [Google Scholar]
Timmins, P. , Shepherd, D. , & Kelly, T. (2003). The research and development in organisations approach and the evaluation of a mainstream behaviour support initiative. Educational Psychology in Practice, 19(3), 229–242. Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+evaluation&pg=6&id=EJ770788 [Google Scholar]
Tischler, V. , Vostanis, P. , Bellerby, T. , & Cumella, S. (2002). Evaluation of a mental health outreach service for homeless families. Archives of Disease in Childhood, 86(3), 158–163. doi:https://doi.org/10.1136%2Fadc.86.3.158 [DOI] [PMC free article] [PubMed] [Google Scholar]
Trulson, C. , Triplett, R. , & Snell, C. (2001). Social control in a school setting: evaluating a school‐based boot camp. Crime & Delinquency, 47(4), 573–609. doi:10.1177/0011128701047004005 [Google Scholar]
Turner, E. , & Waterhouse, S. (2003). Towards inclusive schools. Sustaining normal in‐school careers: an alternative to pupil exclusions. Emotional and Behavioural Difficulties, 8(1), 19. doi:10.1080/13632750300507003 [Google Scholar]
Twemlow, S. W. , Biggs, B. K. , Nelson, T. D. , Vernberg, E. M. , Fonagy, P. , & Twemlow, S. W. (2008). Effects of participation in a martial arts‐based antibullying program in elementary schools. Psychology in the Schools, 45(10), 947–959. doi:10.1002/pits.20344 [Google Scholar]
Vandever, T. R. (1983). Comparison of suspensions of EMR and non retarded students. American Journal of Mental Deficiency, 87(4), 430–434. [PubMed] [Google Scholar]
Veliz, P. , & Shakib, S. (2012). Interscholastic sports participation and school based delinquency: Does participation in sport foster a positive high school environment? Sociological Spectrum, 32(6), 558–580. doi:10.1080/02732173.2012.700837 [Google Scholar]
Vincent, C. G. , & Tobin, T. J. (2011). The relationship between implementation of School‐Wide Positive Behavior Support (SWPBS) and disciplinary exclusion of students from various ethnic backgrounds with and without disabilities. Journal of Emotional and Behavioral Disorders, 19(4), 217–232. doi:10.1177/1063426610377329 [Google Scholar]
Vincus, A. A. , Ringwalt, C. , Harris, M. S. , & Shamblen, S. R. (2010). A short‐term, quasi‐experimental evaluation of D.A.R.E's revised elementary school curriculum. Journal of Drug Education, 40(1), 37–49. doi:10.2190/DE.40.1.c [DOI] [PubMed] [Google Scholar]
Vulliamy, G. , & Webb, R. (2003). Reducing school exclusions: An evaluation of a multi‐site development project. Oxford Review of Education, 29(1), 33–49. Retrieved from http://www.jstor.org/stable/1050728 [Google Scholar]
Waters, T. (2014). Story links: Working with parents of pupils at risk of exclusion. Support for Learning, 29(4), 298–318. doi:10.1111/1467‐9604.12072 [Google Scholar]
Webster‐Stratton, C. , Rinaldi, J. , & Reid, J. M. (2011). Long‐term outcomes of incredible years parenting program: Predictors of adolescent adjustment. Child and Adolescent Mental Health, 16(1), 38–46. doi:10.1111/j.1475‐3588.2010.00576.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wheatley, A. , Murrihy, R. , van Kessel, J. , Wuthrich, V. , Remond, L. , Tuqiri, R. , … Kidman, A. (2009). Aggression management training for youth in behaviour schools: A quasi‐experimental study. Youth Studies Australia, 28(1), 29–36. Retrieved from http://search.informit.com.au/documentSummary;dn=639719523230811;res=IELAPA [Google Scholar]
Wiggins, M. , Bonell, C. , Sawtell, M. , Austerberry, H. , Burchett, H. , Allen, E. , & Strange, V. (2009). Health outcomes of youth development programme in england: Prospective matched comparison study. British Medical Journal, 339, 1–8. doi:10.1136/bmj.b2534 [DOI] [PMC free article] [PubMed] [Google Scholar]
Williams, E. , Johnson, J. L. , & Bott, C. A. (2008). Evaluation of a program for reduction of childhood aggression. Psychological Reports, 103(2), 347–357. doi:10.2466/pr0.103.2.347‐357 [DOI] [PubMed] [Google Scholar]
Williams, R. , & Pritchard, C. (2016). An analysis of the psychosocial backgrounds of youths (13‐18) who‐pose‐sexual‐ risk to children. Journal of Social Work, 0(0), 1–19. doi:10.1177/1468017316651992 [Google Scholar]
Winborne, C. R. (1980). In‐school suspension programs: The King William county model. Educational Leadership, 37(6), 466–469. Retrieved from <Go to ISI>://WOS:A1980JJ88900004 [Google Scholar]
Wright, McMahon, P. M. , Daly, C. , & Haney, J. P. (2012). Getting the law involved a quasi‐experiment in early intervention involving collaboration between schools and the district attorney's office. Criminology & Public Policy, 11(2), 227–249. doi:10.1111/j.1745‐9133.2012.00803.x [Google Scholar]
Wright, R. , Offord, D. , John, L. , & Duku, E. (2005). Secondary schools demonstration project: program effects of school‐based interventions on antisocial behaviour. Exceptionality Education Canada, 15(2), 27–50. [Google Scholar]
Ya‐yu, L. , Loe, S. A. , & Cartledge, G. (2002). The effects of social skills instruction on the social behaviors of students at risk for emotional or behavioral disorders. Behavioral Disorders, 27(4), 371–385. [Google Scholar]
Yokley, J. (2010). Social responsibility therapy for harmful, abusive behavior. Journal of Contemporary Psychotherapy, 40(2), 105–113. doi:10.1007/s10879‐009‐9131‐7 [Google Scholar]
Zivin, G. , Hassan, N. R. , DePaula, G. F. , Monti, D. A. , Harlan, C. , Hossain, K. D. , & Patterson, K. (2001). An effective approach to violence prevention: Traditional martial arts in middle school. Adolescence, 36(143), 443–459. Retrieved from <Go to ISI>://WOS:000173402600004 [PubMed] [Google Scholar]

7.2.3 Reports

Ackerman, C. M. , Cooksy, L. J. , Murphy, A. , Rubright, J. , Bear, G. , & Fifield, S. (2010). Positive behavior support in Delaware schools: Developing perspectives on implementation and outcomes. (Technical Report No. T2010.3). Newark, DE. University of Delaware: Education Research & Development Center. [Google Scholar]
Allen, J. P. , & Philliber, S. (1991). Evaluating why and how the teen outreach program works: years 3‐5 of the teen outreach national replication (1986/87‐1988/89). New York: Association of Junior Leagues. [Google Scholar]
Armour, M. (2014). Ed White middle school restorative discipline evaluation: Implementation and impact, 2013/2014. Sixth & seventh grade. Retrieved from www.utexas.edu [Google Scholar]
Baker, M. L. (2009). DPS restorative justice project: Year Three. Retrieved from Outcomes, Inc: Denver. [Google Scholar]
Burke, A. (2015). Suspension, expulsion, and achievement of english learner students in six Oregon districts. (REL 2015‐094). Regional Educational Laboratory Northwest. US: Department of Education. [Google Scholar]
Clarke, A. , Wydall, S. , & Williams, K. (2008). An evaluation of the introduction of a youth inclusion programme in rural Wales (Final Report). Wales: Department of Law and Criminology Aberystwyth University. [Google Scholar]
Commission of the European Communities, B. B. D.‐G. f. E. a. C . (2001). Second chance schools: The results of a European pilot project (Report). Brussels: Directorate‐General for Education & Culture. [Google Scholar]
Cullen, M. , Johnstone, M. , Lloyd, G. , & Munn, P. (1996). Exclusion from school and alternatives. Report on phase 3: The case study. [Google Scholar]
Davis, A. J. , & Pfeifer, H. (2014). How has the Baltimore county public school system addressed disproportionate minority suspensions?. Schaefer Center for Public Policy University of Baltimore: College of Public Affairs. [Google Scholar]
Dynarski, M. , James‐Burdumy, S. , Moore, M. , Rosenberg, L. , Deke, J. , Mansfield, W. , & Warner, E. (2004). When schools stay open late: the national evaluation of the 21st century community learning centers program. New findings. Retrieved from U.S. Department of Education, National Center for Education Evaluation and Regional Assistance. Washington, DC: U.S. Government Printing Office, 2004. [Google Scholar]
Durbin, K. (1994). Project success: A school‐based alternative to expulsion. Lancaster County Schol District. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED379542&site=ehost‐live&scope=site [Google Scholar]
Elliot, W. L. (1991). RE‐FOCUS program. redefine efforts: Focus on change under supervision. (a support program for students and teachers). Kansas City, MO. Hickman Mills Consolidated School District. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED337830&site=ehost‐live&scope=site [Google Scholar]
Eric Clearinghouse on Educational Management . (1986). Preventing dropouts. The best of ERIC on educational management (No.83). Office of Educational Research and Improvement. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+experiment&pg=2&id=ED268666;http://files.eric.ed.gov/fulltext/ED268666.pdf [Google Scholar]
Flay, B. , Acock, A. , Vuchinich, S. , & Beets, M. (2006). Progress report of the randomized triall of Positive Action in Hawaii's end of third year of intervention (spring 2005). Oregon: Oregon State University. [Google Scholar]
Foley, E. M. , Klinge, A. , & Reisner, E. R. (2007). Evaluation of new century high schools: Profile of an initiative to create and sustain small, successful high schools (final report). Policy Studies Associates, Inc. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=4&id=ED498781;http://files.eric.ed.gov/fulltext/ED498781.pdf [Google Scholar]
Fronius, T. , Persson, H. , Guckenburg, S. , Hurley, N. , & Petrosino, A. (2016). Restorative justice in us schools: A research review. Retrieved from West Ed. Justice & Prevention Research Center.: http://jprc.wested.org/wp‐content/uploads/2016/02/RJ_Literature‐Review_20160217.pdf [Google Scholar]
Gao, J. , Hallar, B. , & Hartmann, T. A. (2014). A snapshot of ost programs in philadelphia: An evaluation of eleven 21st century community learning center grantees. Prepared by Research For Action. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=3&id=ED553131;http://files.eric.ed.gov/fulltext/ED553131.pdf [Google Scholar]
Grice, M. (1986). Positive Alternatives to School Suspension (P.A.S.S.), 1985‐86. Portland Public Schools: Evaluation Report. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED276794&site=ehost‐live&scope=site [Google Scholar]
Hallam, S. , Castle, F. , & Rogers, L. (2005). Research and evaluation of the behaviour improvement programme. Retrieved from http://oro.open.ac.uk/4565/
Harlow, K. , & Baenen, N. (2002). NovaNET, 2000‐2001: Analyses of Student Outcomes Relative to a Comparison Group. Eye on Evaluation. E&R Report. Retrieved from Wake County Public School System, Raleigh, NC: http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED466488&site=ehost‐live&scope=site [Google Scholar]
Hartmann, T. A. , Gao, J. , Kumar, A. , & Edmunds, K. (2013). A snapshot of OST programming in Philadelphia: An evaluation of six 21st Century Community Learning Center grantees. Research For Action. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=3&id=ED553147;http://files.eric.ed.gov/fulltext/ED553147.pdf [Google Scholar]
Heller, S. , Pollack, H. A. , Ander, R. , & Ludwig, J. (2013). Preventing youth violence and dropout: A randomized field experiment. Massachusetts: National Bureau of Economic Research. [Google Scholar]
Horner, R. , & Sugai, G. (2007). Is school‐wide positive behavior support an evidence‐based practice. Retrieved from www.pbis.org/common/cms/files/pbisresources/EvidenceBaseSWPBS08_04_08.doc
Houston Independent School District . (1997). Project SAFE alternative school, 1996‐97. Research report on an educational program. Texas:Department of Research and Accountability. Retrieved from http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=5&id=ED428166;http://files.eric.ed.gov/fulltext/ED428166.pdf [Google Scholar]
James‐Burdumy, S. , Dynarski, M. , Moore, M. , Deke, J. , & Mansfield, W. (2005). When schools stay open late: The national evaluation of the 21st Century Community Learning Centers program. Final report. Retrieved from U.S. Department of Education; [Google Scholar]
Kendall, S. , Kinder, K. , Halsey, K. , Fletcher‐Morgan, C. , White, R. , & Brown, C. (2003). An evaluation of alternative education initiatives (Research Report RR403). National Foundation for Educational Research. Retrieved from http://www.opengrey.eu/item/display/10068/421869 [Google Scholar]
Laliberté, D. , Rosario, G. , Léonard, L. , Smith‐Moncrieffe, D. , & Warner, A. (2014). Results of crime prevention programs for 12 to 17 year olds (Research Report: 2015‐R006). Ottawa, Ontario Canada: National Crime Prevention Centre (NCPC) Public Safety Canada; Retrieved from https://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/rslts‐crm‐prvntn‐12‐17/index‐en.aspx [Google Scholar]
Leonard, C. M. (1984). Evaluation Report on “PASS.” Positive Alternatives to Student Suspensions. Portland Public Schools: Research & Evaluation Department. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED256772&site=ehost‐live&scope=site [Google Scholar]
Linker, J. , & Marion, B. (1995). A study of the effects of options, an alternative educational program, on the personal success of at‐risk students: A five year study. Alleghany County School District NC. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED398338&site=ehost‐live&scope=site [Google Scholar]
McAleavy, G. , & O'Hagan, C. (n/d) An evaluation of the impact of the xL intervention in schools in northern ireland. Northern Ireland: The Prince's Trust. Retrieved from https://www.researchgate.net/profile/Celia_OHagan/publication/237343072_An_Evaluation_of_The_Impact_of_the_xl_Intervention_in_Schools_in_Northern_Ireland/links/0046352f7d3fb49433000000.pdf [Google Scholar]
Mearns, C. (1999). Brief family counseling intervention: Cibola cluster, 1997‐98, reserach, development and accountability. Albuquerque Public Schools. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED448374&site=ehost‐live&scope=site [Google Scholar]
Munn, P. , Cullen, M. , Johnstone, M. , & Lloyd, G. (1997). Exclusions and in‐school alternatives. The Scottish Office Education and Industry Department: Research and Intelligence Unit. Retrieved from http://files.eric.ed.gov/fulltext/ED414616.pdf [Google Scholar]
Norton, M. , & Piccinino, K. (2014). Project LIFT, year two report. Philadelphia: Research for Action. Retrieved from http://8rri53pm0cs22jk3vvqna1ub‐wpengine.netdna‐ssl.com/wp‐content/uploads/2015/10/RFA_Project_LIFT_Year_Two_Report_October_2014.pdf [Google Scholar]
Opuni, K. A. , Tullis, R. , Sanchez, K. , & Gonzalez, J. (1991). Student assignment centers: An in‐school suspension program, 1990‐91. Houston TX: Department of Researh and Education. Retrieved from http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=6&id=ED339137;http://files.eric.ed.gov/fulltext/ED339137.pdf [Google Scholar]
Peterson, P. E. , Myers, D. , & Howell, W. G. (1998). An evaluation of the new york city school choice scholarships program: The first year. New York: Mathematica Policy Research, Washington, DC. Retrieved from http://hks.harvard.edu/pepg/PDF/Papers/ny1ex.pdf [Google Scholar]
Pritchard, C. (2001). A family‐teacher‐social work alliance to reduce truancy and delinquency: The Dorset healthy alliance project (RDS occasional paper no. 78). England:Home Office, Communication Development Unit, Research, Development and Statistics Directorate. Retrieved from http://webarchive.nationalarchives.gov.uk/20110218135832/http:/rds.homeoffice.gov.uk/rds/pdfs/occ78‐dha.pdf [Google Scholar]
Puma, M. , Bell, S. , Shapiro, G. , Broene, P. , Cook, R. , Friedman, J. , & Heid, C. (2001). Building futures: the head start impact study. Research design plan. Washington DC: Administration for Children, Youth, and Families; Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+experiment&id=ED478789;http://files.eric.ed.gov/fulltext/ED478789.pdf [Google Scholar]
Rhea, A. (2010). An evaluation of the Wake county public school system alternative educational options (E&R Report No. 10.15). Wake County, Public School System. Retrieved from http://www.wcpss.net/evaluation‐research [Google Scholar]
Risner, M. , Abati, G. , Vanderhaar, J. , & Muñoz, M. A. (2005). Program evaluation. Positive Outreach Program (POP). Jefferson County Public Schools: Shaping the Future. Retrieved from http://www.jefferson.k12.ky.us/departments/Planning/ProgramEvaluation/FullReports_3_09/POPpdf.pdf [Google Scholar]
Rogers, J. W. (1972). A report of the team‐teaching, team‐governance program at rockland junior high. Distributed by ERIC Clearinghouse. Retrieved from http://eric.ed.gov/?id=ED083123 [Google Scholar]
Schoeneberger, J. (2008). The effect of Positive Behavioral Intervention Support (PBIS). Charlotte‐ Mecklenburg Schools: Center for Research & Evaluation. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.563.5527&rep=rep1&type=pdf [Google Scholar]
Smith, L. , Jackson, L. , & Comber, N. (2013). Therapeutic early interventions to prevent school exclusion: Evaluation of three contemporaneous projects. Office for Public Management ‐ OPM. Retrieved from http://www.opm.co.uk/wp‐content/uploads/2014/01/Therapeutic‐early‐interventions‐to‐prevent‐school‐exclusion‐and‐truancy‐Final‐Report.pdf [Google Scholar]
St James‐Roberts, I. , & Singh, C. S. (2001). Can mentors help primary school children with behaviour problems? (Study 233). England: Home Office. Retrieved from: https://core.ac.uk/download/files/161/4157929.pdf [Google Scholar]
Tarling, R. , Davison, T. , & Clarke, A. (2004). Mentoring projects. The national evaluation of the Youth Justice Board's Mentoring Projects. London: Youth Justice Board. Retrieved from http://yjbpublications.justice.gov.uk/Resources/Downloads/mentoringfull.pdf [Google Scholar]
U.S. Department of Education, I. o. E. S., What Works Clearinghouse . (2014). The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes. Princeton: What Works Clearinghouse Single Study Review. Retrieved from http://search.proquest.com/docview/1651831519?accountid=9851 [Google Scholar]
Wasilewski, Y. , Gifford, B. , Bonneau, K. , Duke University, C. F. C. , & Family, P. (2008). Evaluation of the School‐Wide Positive Behavioral Support Program in eight North Carolina elementary schools. Duke University: Center for Child and Family Policy. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED503191&site=ehost‐live&scope=site [Google Scholar]
What Works, C. (2011). Check & Connect. What Works Clearinghouse Intervention Report. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=3&id=ED525366;http://files.eric.ed.gov/fulltext/ED525366.pdf
What Works, C. (2014). WWC review of the report “The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes”. What Works Clearinghouse Single Study Review. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=2&id=ED546479;http://files.eric.ed.gov/fulltext/ED546479.pdf [Google Scholar]
Youth Justice Board England & Wales (2004). National evaluation of the restorative justice in schools programme. Retrieved from http://www.creducation.org/resources/National_Eval_RJ_in_Schools_Full.pdf

7.2.4 Conference papers, conference proceedings

Armstrong, K. , Boroughs, M. , Massey, O. T. , Uzzell, D. , Sansosti, F. , & Perry, A. (2002). Symposium‐The safe schools/healthy students initiative: Methodologies and results in program‐based evaluation. Paper presented at the A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida. [Google Scholar]
Bickmore, K. (2001). Good training is not enough: Research on peer mediation program implementation. Paper presented at the Association for Conflict Resolution Annual Conference, Toronto, Canada. [Google Scholar]
Boulay, B. , Gamse, B. , Checkoway, A. , Maree, K. , & Fox, L. (2011). Evaluation of Massachusetts Expanded Learning Time (ELT) initiative: Implementation and outcomes after four years. Paper presented at the Society for Research on Educational Effectiveness.
Buhi, E. R. (2015). Effects of an evidence‐based positive youth development program on academic outcomes among adolescents living in non‐metropolitan communities. Paper presented at the 143rd APHA Annual Meeting and Exposition (October 31‐November 4, 2015).
Collins, K. M. , & Onwuegbuzie, A. J. (2001). Effect of an After‐School Tutorial program on academic performance of middle school students at‐risk. Paper presented at the Anual Meting of the Mid‐South Educational Research Asociation, Little Rock, AR. [Google Scholar]
Eber, L. , Lewis‐Palmer, T. , & Pacchiano, D. (2002). School‐Wide Positive Behavior Systems: Improving school environments for all students including those with EBD. Paper presented at the 14th Annual Conference Proceedings. A Symposium of Care for Children's Mental Health: Expanding the Research Base, University of South Florida, Tampa. [Google Scholar]
Eber, L. , Rolf, K. , & Sullivan, M. P. (1998). School‐based systems of care: Early intervention and day treatment examples from Illinois. Paper presented at the 1th Anual Research Conference. A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, FL. [Google Scholar]
Hammond, R. W. , Kadis, P. , & Betty, Y. (1990). Positive Adolescents Choices Training (PACT): Preliminary findings of the effects of a school‐based violence prevention program for African American Adolescents. Paper presented at the 98th Annual Convention of the American Psychological Asociation, Boston, MA. http://search.proquest.com/docview/62955491?accountid=9851 [Google Scholar]
Head, G. , Cogan, N. , & Kane, J. (2001). Evaluating behaviour support in mainstream secondary schools in one education authority. Paper presented at the British Educational Research Association Annual Conference, University of Leeds. http://www.leeds.ac.uk/educol/documents/00001916.htm [Google Scholar]
Heger, H. K. (1992). Retaining Hispanic youth in school: An evaluation of a counseling‐based alternative school program. Paper presented at the Anual Conference of the Rocky Mountain Educational Research Asociation, Stilwater, OK. http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=4&id=ED355302;http://files.eric.ed.gov/fulltext/ED355302.pdf [Google Scholar]
Howell, W. G. , & Peterson, P. E. (2000). School choice in Dayton, Ohio: An evaluation after one year. Paper presented at the Conference on Vouchers, Charters and Public Education (Harvard University, Cambridge, MA: ). [Google Scholar]
Kane, J. , Cogan, N. , & Head, G. (2002). Lessons to learn? Models of behaviour support in secondary schools in one education authority in Scotland. Paper presented at the European Conference on Educational Research, University of Lisbon. http://www.leeds.ac.uk/educol/documents/00002306.htm [Google Scholar]
O'Sullivan, R. G. (1988). Evaluation of a decision‐making program to improve school success with at‐risk middle school students. Paper presented at the The Anual Meting of the American Educational Research Asociation (New Orleans: ). http://files.eric.ed.gov/fulltext/ED296054.pdf [Google Scholar]
Smith, J. D. , Pare, D. , & Gravelle, F. (2002). Alternatives to school suspension: An intervention for at‐risk students. Paper presented at the Anual Meting of the American Educational Research Asociation, New Orleans, LA. [Google Scholar]
Winborn, J. D. (1992). A study of the effectiveness of a saturday school in reducing suspension, expulsion, and corporal punishment. Paper presented at the The Anual Meting of the Mid‐South Educational Research Asociation, Knoxvile, TN. http://search.proquest.com/docview/62851284?accountid=9851 [Google Scholar]

7.2.5 Books and book's chapters

Billig, S. H. (2009). Does quality really matter? Testing the new k‐12 service‐learning standards for quality practice. In Moely B. E. B., Holland, B. A. S. H. (Ed.), Creating our identities in service‐learning and community engagement (pp. 131–157). Greenwich, CT: Information Age. [Google Scholar]
McIntosh, K. , Fisher, E. S. , Kennedy, K. S. , Craft, C. B. , & Morrison, G. M. (2011). Using office discipline referrals and school exclusion data to assess school discipline. In Jimerson S. R. N., Amanda B.; Mayer, Matthew J. ; Furlong, Michael J. (Ed.), Handbook of school violence and school safety: International research and practice, 2nd Edition (pp. 305–315). New York: Taylor & Francis. [Google Scholar]
Sexton, T. L. , Fisher, A. R. , Graham, C. , & Elnahrawy, N. A. (2015). Evidence‐based practice in the prevention and treatment of adolescent behavior problems. In Gullotta T. P., Plant, Robert W. , Evans, Melanie (Ed.), Handbook of adolescent behavioral problems (pp. 85–104). United States: Springer. [Google Scholar]
Trotter, J. C. , & Jones, L. T. (1998). Create peace now. The peace project: An in‐school suspension program for middle and high school students with violent behaviors. Atlanta: Wholistic Stress Control Institute. [Google Scholar]
Vincent, C. G. , Sprague, J. R. , Pavel, M. , Tobin, T. , & Gau, J. M. (2014). Effectiveness of Schoolwide Positive Behavior Interventions and Supports in educing racially inequitable disciplinary exclusion. In Losen D. J. (Ed.), Closing the School Discipline Gap: Equitable Remedies for Excessive Exclusion (pp. 207). New York & London: Teachers College Columbia University. [Google Scholar]

7.3 REFERENCES TO STUDIES AWAITING CLASSIFICATION

Allen, S. (1981). A study to determine the effectiveness of a positive approach to discipline system for classroom management. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED203490&site=ehost‐live&scope=site
Forbes, R. D. (1996). The implementation of a positive discipline program to increase the social skills of middle grade students. (Master Thesis). Nova Southeastern University. Retrieved from http://search.proquest.com/docview/62672929?accountid=9851 ProQuest Social Sciences Premium Collection database. [Google Scholar]
Foster, E. L. (2011). Implementation of a social skills curriculum to reduce behavioral problems of african american boys in elementary classroom settings. United States of America: Xlibris Corporation. [Google Scholar]
Gaines, L. , & Schram, P. (2005). Addressing the needs of at‐risk youth: An evaluation of the San Bernardino County, California School Probation Officer Program. Paper presented at the American Society of Criminology. 2005 Annual Meeting.
Gallegos, A. E. (1998). School expulsions, suspensions, and dropouts: Understanding the issues. Hot topics series. Bloomington: Phi Delta Kappa International. [Google Scholar]
Neise, M. R. (1983). The effectiveness of problem solving training on the performance of high frequency discipline students' classroom and school. (Doctoral Dissertation). University of Missouri ‐ Columbia. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]
Norris, A. (2009). Gender and race effects of a Restorative Justice intervention on school success. Paper presented at the American Society of Criminology. Article retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=cja&AN=54432927&site=ehost‐live&scope=site
Spillman, K. E. C. (1993). The effects of interdisciplinary teaming and parent contacts on the academic achievement, motivation, attendance, and suspension rate of students identified to be at‐risk of repeating the ninth‐grade. (Doctoral Dissertation). ProQuest Information & Learning, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1995‐73295‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

7.4 REFERENCES TO ONGOING STUDIES

Acosta, J. (2015). RCT of the restorative practices intervention (RPI). Clinical trials.gov, RAND. Retrieved from https://clinicaltrials.gov/ct2/show/NCT02155296?term=RCT+of+the+Restorative+Practices+Intervention+%28RPI%29&rank=1 [Google Scholar]
Bonell, C. , Allen, E. , Christie, D. , Elbourne, D. , Fletcher, A. , Grieve, R. , … Viner, R. M. (2014). Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): Study protocol for a cluster randomised controlled trial. Trials, 15(381), 2–14. doi:10.1186/1745‐6215‐15‐381 [DOI] [PMC free article] [PubMed] [Google Scholar]
Philliber, S. (2015). Replication of the Teen Outreach program in Kansas City. Retrieved from https://clinicaltrials.gov/ct2/show/NCT02514759?term=school+suspension&age=0&rank=1
Eiraldi, R. , McCurdy, B. , Khanna, M. , Mautone, J. , Jawad, A. F. , Power, T. , … Sugai, G. (2014). A cluster randomized trial to evaluate external support for the implementation of Positive Behavioral Interventions and Supports by school personnel (Study protocol). Implementation Science, 9(1), 12. doi:10.1186/1748‐5908‐9‐12 [DOI] [PMC free article] [PubMed] [Google Scholar]

7.5 ADDITIONAL REFERENCES

Achenbach, T. M. (1978). The child behavior profile: I. Boys aged 6‐11. Journal of Consulting and Clinical Psychology, 46(3), 478–488. [DOI] [PubMed] [Google Scholar]
Achenbach, T. M. , & Edelbrock, C. S. (1979). The child behavior profile: II Boys aged 12‐16 and girls 6‐11 and 12‐16. Journal of Consulting and Clinical Psychology, 47(2), 223–233. 10.1037/0022-006X.52.2.207 [DOI] [PubMed] [Google Scholar]
Acosta, J. (2015). Reducing problem behaviors through PYD: An RCT of restorative school practices. Clinical Trials.gov, RAND. Retrieved from https://clinicaltrials.gov/ct2/show/NCT02155296?term=RCT+of+the+Restorative+Practices+Intervention+%28RPI%29&rank=1 [Google Scholar]
Ahn, S. , Myers, N. D. , & Jin, Y. (2012). Use of the estimated intraclass correlation for correcting differences in effect size by level. Behavior Research Methods, 44(2), 490–502. 10.3758/s13428-011-0153-1 [DOI] [PubMed] [Google Scholar]
APA ZTTF . (2008). Are zero tolerance policies effective in the schools? An evidentiary review and recommendations. The American Psychologist; (Vol. 63). Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/19086747 [DOI] [PubMed] [Google Scholar]
Arcia, E. (2006). Achievement and enrollment status of suspended students: Outcomes in a large, multicultural school district. Education and Urban Society, 38(3), 359–369. 10.1177/0013124506286947 [DOI] [Google Scholar]
Averdijk, M. , Eisner, M. , Luciano, E. , Valdebenito, S. , & Obsuth, I. (2014). Wirksame gewaltprävention. Eine übersicht zum internationalen wissensstand. Retrieved from http://www.jugendundgewalt.ch/fileadmin/user_upload_jug/3_Good_Practice/Handbuch/Übersicht_wirksame_Gewaltprävention.pdf [Google Scholar]
Bear, G. G. (2012). Both suspension and alternatives work, depending on one's aim. Journal of School Violence, 11(2), 174–186. 10.1080/15388220.2012.652914 [DOI] [Google Scholar]
Beelmann, A. , & Lösel, F. (2006). Child social skills training in developmental crime prevention: effects on antisocial behavior and social competence. Psicothema, 18, 603–610. [PubMed] [Google Scholar]
Borenstein, M. , Hedges, L. , Higgins, J. P. T. , & Rothstein, H. R. (2009). Introduction to meta‐analysis. Chichester, UK: John Wiley & Sons, Ltd. [Google Scholar]
Bowman‐Perrott, L. , Benz, M. R. , Hsu, H.‐Y. , Kwok, O.‐M. , Eisterhold, L. A. , & Zhang, D. (2013). Patterns and predictors of disciplinary exclusion over time: An analysis of the SEELS national dataset. Journal of Emotional and Behavioral Disorders, 21(2), 83–96. 10.1177/1063426611407501 [DOI] [Google Scholar]
Brookes, M. , Goodall, E. , & Heady, L. (2007). Misspent youth. The cost of truancy and exclusion. New Philanthropy Capital, (June). Retrieved from http://socialwelfare.bl.uk/subject‐areas/services‐activity/education‐skills/newphilanthropycapital/misspent07.aspx [Google Scholar]
Brown, T. M. (2007). Lost and turned out: Academic, social, and emotional experiences of students excluded from school. Urban Education, 42(5), 432–455. 10.1177/0042085907304947 [DOI] [Google Scholar]
Burrell, N. A. , Zirbel, C. S. , & Allen, M. (2003). Evaluating peer mediation outcomes in educational settings: A meta‐analytic review. Conflict Resolution Quarterly, 21(1), 7–26. 10.1002/crq.46 [DOI] [Google Scholar]
Challen, M. , & Walton, T. (2004). Juveniles in custody: A unique insight into the perceptions of young people held in prison service custody in England and Wales. Her Majesty's Inspectorate of Prisons, 36. [Google Scholar]
Chandler, J. , Churchill, R. , Higgins, J. , Lasserson, T. , & Tovey, D. (2013). Methodological standards for the conduct of new Cochrane Intervention Reviews. Version2.3.
Chin, J. K. , Dowdy, E. , Jimerson, S. R. , & Rime, W. J. (2012). Alternatives to suspensions: Rationale and recommendations. Journal of School Violence, 11(2), 156–173. 10.1080/15388220.2012.652912 [DOI] [Google Scholar]
Cohen, J. (1960). A Coefficient for Agreement for Nominal Scales. Educational and Psychological Measurement, 20(1), 37–46. 10.1177/001316446002000104 [DOI] [Google Scholar]
Costenbader, V. , & Markson, S. (1998). School suspension: A study with secondary school students. Journal of School Psychology, 36(1), 59–82. Retrieved from http://libsta28.lib.cam.ac.uk:2070/science/article/pii/S0022440597000502?np=y [Google Scholar]
Cratty, D. (2012). Potential for significant reductions in dropout rates: Analysis of an entire 3rd grade state cohort. Economics of Education Review, 31(5), 644–662. 10.1016/j.econedurev.2012.04.001 [DOI] [Google Scholar]
DfE . (2012). A profile of pupil exclusions in England. Research Report DFE‐RR190. London: Department of Education. Retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/183498/DFE‐RR190.pdf [Google Scholar]
DfE . (2013). Statistical first release: Permanent and fixed period exclusion from schools and exclusion appeals in England, 2011/12 (SFR 29/2013, 12) London: Department of Education. Retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/224893/SFR29‐2013.pdf [Google Scholar]
Durlak, J. A. , Weissberg, R. P. , Dymnicki, A. B. , Taylor, R. D. , & Schellinger, K. B. (2011). The impact of enhancing students' social and emotional learning: A meta‐analysis of school‐based universal interventions. Child Development, 82(1), 405–432. 10.1111/j.1467-8624.2010.01564.x [DOI] [PubMed] [Google Scholar]
Dymnicki, A. B. , Weissberg, R. P. , & Henry, D. B. (2011). Understanding how programs work to prevent overt aggressive behaviors: A meta‐analysis of mediators of elementary school‐based programs. Journal of School Violence, 10(4), 315–337. 10.1080/15388220.2011.602599 [DOI] [Google Scholar]
Eby, L. T. , Allen, T. D. , Evans, S. C. , Ng, T. , & Dubois, D. L. (2008). Does Mentoring Matter? A Multidisciplinary Meta‐Analysis Comparing Mentored and Non‐Mentored Individuals. Journal of Vocational Behavior, 72(2), 254–267. 10.1016/j.jvb.2007.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
Eisner, M. (2009). No effects in independent prevention trials: Can we reject the cynical view? Journal of Experimental Criminology, 5(2), 163–183. 10.1007/s11292-009-9071-y [DOI] [Google Scholar]
Eisner, M. , & Humphreys, D. K. (2012). Measuring conflict of interest in prevention and intervention research. A feasibility study. In Bliesener T., Beelmann A., & Stemmler M. (Eds.), Antisocial behavior and crime: Contributions of developmental and evaluation research to prevention and intervention. Göttingen, London: Hogrefe Publishing. [Google Scholar]
Eisner, M. , Humphreys, D. K. , Wilson, P. , & Gardner, F. (2015). Disclosure of financial conflicts of interests in interventions to improve child psychosocial health: A cross‐sectional study. PLoS ONE, 10(11), e0142803. Retrieved from https://doi.org/10.1371%2Fjournal.pone.0142803 [DOI] [PMC free article] [PubMed] [Google Scholar]
Ellis, P. D. (2010). The essential guide to effect sizes. Statistical power, meta‐analysis, and the interpretation of research results. Cambridge, UK: Cambridge University Press. [Google Scholar]
Fabelo, T. , Thompson, M. D. , Plotkin, M. , Carmichael, D. , Marchbanks III, M. P. , & Booth, E. A. (2011). Breaking schools' rules: A statewide study of how school discipline relates to students' success and juvenile justice involvement. New York: Council of State Governments Justice Center. [Google Scholar]
Farrington, D. P. (1989). Early predictors of adolescent aggression and adult violence. Violence and Victims, (4), 79–100. [PubMed] [Google Scholar]
Farrington, D. P. (1997). Human development and criminal careers. In Maguire M., Morgan R., & Reiner R. (Eds.) The Oxford handbook of criminology (Second edition). Oxford University Press. [Google Scholar]
Farrington, D. P. (2003). Advancing knowledge about the early prevention of adult antisocial behaviour. In Farrington D. P. & Coid J. W. (Eds.), Early Prevention of Adult Antisocial Behavior (pp. 1–20). Cambridge, UK: Cambridge University Press. [Google Scholar]
Farrington, D. P. , Ttofi, M. M. , & Lösel, F. (2016). Developmental and social prevention. In Weisburd D., Farrington D. P., & Gill C. (Eds.), What works in crime prevention and rehabilitation. Lessons from systematic reviews (pp. 15–75). New York: Springer Series on Evidence‐Based Crime Policy. [Google Scholar]
Farrington, D. , & Ttofi, M. M. (2009). School‐based programs to reduce bullying and victimization. Campbell Systematic Reviews, 6. 10.4073/csr.2009.6 [DOI] [PMC free article] [PubMed] [Google Scholar]
Fenning, P. A. , Pulaski, S. , Gomez, M. , Morello, M. , Maciel, L. , Maroney, E. , … Maltese, R. (2012). Call to action: A critical need for designing alternatives to suspension and expulsion. Journal of School Violence, 11(2), 105–117. 10.1080/15388220.2011.646643 [DOI] [Google Scholar]
Freiberg, H. J. , & Lapointe, J. M. (2006). Research‐based programs for preventing and solving discipline problems. In Evertson C. M. & Weinstein C. S. (Eds.), Handbook of classroom management research, practice, and contemporary issues (pp. 735–786). London: Routledge. 10.4324/9780203874783.ch28 [DOI] [Google Scholar]
Fronius, T. , Persson, H. , Guckenburg, S. , Hurley, N. , & Petrosino, A. (2016). Restorative justice in us schools: A research review. Retrieved from West Ed. Justice & Prevention Research Center.: http://jprc.wested.org/wp‐content/uploads/2016/02/RJ_Literature‐Review_20160217.pdf [Google Scholar]
Giraudeau, B. , & Ravaud, P. (2009). Preventing bias in cluster randomised trials. PLoS Medicine, 6(5), 1–6. 10.1371/journal.pmed.1000065 [DOI] [PMC free article] [PubMed] [Google Scholar]
Gottfredson, D. C. , & Wilson, D. B. (2003). Characteristics of effective school‐based substance abuse prevention. Prevention Science, 4(1), 27–38. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12611417 [DOI] [PubMed] [Google Scholar]
Gottfredson, D. C. , Cook, P. J. , & Na, C. (2012). Schools and prevention. In Welsh B. C. & Farrington D. P. (Eds.), The Oxford handbook of crime prevention (pp. 269–287). Oxford, United Kingdom: Oxford University Press. [Google Scholar]
Gottfredson, D. , Wilson, D. B. , & Najaka, S. (2002). School‐based crime prevention. In Sherman L. W., Farrington D. P., Welsh B. C., & MacKenzie D. (Eds.), Evidence‐Based Crime Prevention (pp. 56–164). London: Routledge. [Google Scholar]
Gregory, A. , & Weinstein, R. (2008). The discipline gap and African Americans: Defiance or cooperation in the high school classroom. Journal of School Psychology, 46(4), 455–75. 10.1016/j.jsp.2007.09.001 [DOI] [PubMed] [Google Scholar]
Gregory, A. , Skiba, R. J. , & Noguera, P. A. (2010). The achievement gap and the discipline gap: Two sides of the same coin? Educational Researcher, 39(1), 59–68. 10.3102/0013189X09357621 [DOI] [Google Scholar]
Hedges, L. V. (2007). Effect sizes in cluster‐randomized designs. Journal of Educational and Behavioral Statistics, 32(4), 341–370. 10.3102/1076998606298043 [DOI] [Google Scholar]
Hemphill, S. A. , Toumbourou, J. W. , Herrenkohl, T. I. , McMorris, B. J. , & Catalano, R. F. (2006). The effect of school suspensions and arrests on subsequent adolescent antisocial behavior in Australia and the United States. The Journal of Adolescent Health, 39(5), 736–744. 10.1016/j.jadohealth.2006.05.010 [DOI] [PubMed] [Google Scholar]
Higgins, J. P. T. , & Green, S. (Eds). (2011). Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, V 5.1.0. [Google Scholar]
Higgins, J. P. T. , Thompson, S. G. , Deeks, J. J. , & Altman, D. G. (2003). Measuring inconsistency in meta‐analyses. BMJ, 327, 557–60. 10.1136/bmj.327.7414.557 [DOI] [PMC free article] [PubMed] [Google Scholar]
Higgins, J. , Altman, D. , Gøtzsche, P. , Jüni, P. , Moher, D. , & Oxman, A. (2011). The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 343, 1–9. 10.1136/bmj.d5928 [DOI] [PMC free article] [PubMed] [Google Scholar]
Hirschfield, P. J. (2008). Preparing for prison?: The criminalization of school discipline in the USA. Theoretical Criminology, 12(1), 79–101. 10.1177/1362480607085795 [DOI] [Google Scholar]
Hobbs, G. , & Vignoles, A. (2010). Is children's free school meal “eligibility” a good proxy for family income?? British Educational Research Journal, 36(4), 673–690. 10.1080/01411920903083111 [DOI] [Google Scholar]
Hutchison, D. , & Styles, B. (2010). A guide to running randomised controlled trials for educational researchers. Slough, UK: NFER. [Google Scholar]
Irvin, J. (2002). In search of wholeness. African American teachers and their culturally specific classroom practices. New York: Palgrave Macmillan. 10.1057/9780230107182 [DOI] [Google Scholar]
Johnson, K. (2012). Building better schools not prisons: A review of the literature surrounding school suspension and expulsion programs and the implications of such programs on the lives of racial and ethnic minority students. (Master Thesis). University of Toronto. [Google Scholar]
Kaplan, A. , Gheen, M. , & Midgley, C. (2002). Classroom goal structure and student disruptive behaviour. The British Journal of Educational Psychology, 72(2), 191–211. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12028608 [DOI] [PubMed] [Google Scholar]
Krohn, M. D. , Lopes, G. , & Ward, J. T. (2014). Effects of official intervention on later offending in the Rochester youth development study. In Farrington D. P. & Murray J. (Eds.), Labelling theory. Empirical tests. Advances in criminological theory. Volume 18 (pp. 179–207). New Brunswick: Transaction Publishers. [Google Scholar]
Lewis, R. , Romi, S. , & Roache, J. (2012). Excluding students from classroom: Teacher techniques that promote student responsibility. Teaching and Teacher Education, 28(6), 870–878. 10.1016/j.tate.2012.03.009 [DOI] [Google Scholar]
Lipsey, M. W. , & Wilson, D. B. (2001). Practical Meta‐Analysis. Thousand Oaks, California: Sage Publications. [Google Scholar]
Liu, J. C. (2013). The suspension spike: Changing the discipline culture in NYC's middle schools. New York City Comptroller's Office, 1–65. [Google Scholar]
Lösel, F. , & Beelmann, A. (2006). Child social skills training. In Welsh B. C. & Farrington D. P. (Eds.), Preventing crime. What works for children, offenders, victims and places. (pp. 32–54). Dordrecht, The Netherlands: Springer. [Google Scholar]
Massey, A. (2011). Best behaviour: School discipline, intervention and exclusion. Policy Exchange, 75. Retrieved from http://www.policyexchange.org.uk/publications/category/item/best‐behaviour‐school‐discipline‐intervention‐and‐exclusion [Google Scholar]
McLoughlin, C. S. , & Noltemeyer, A. L. (2010). Research into factors contributing to discipline use and disproportionality in major urban schools. Current Issues in Education, 13(2), 1–21. [Google Scholar]
Michail, S. (2011). Understanding school responses to students' challenging behaviour: A review of literature. Improving Schools, 14(2), 156–171. 10.1177/1365480211407764 [DOI] [Google Scholar]
Moher, D. , Liberati, A. , Tetzlaff, J. , & Altman, D. (2009). Preferred reporting items for systematic reviews and meta‐analyses: The PRISMA statement. PLoS Med, 6(7), e1000097. 10.1371/journal.pmed1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
Monroe, C. R. (2005). Why are “bad boys” always black? Causes of disproportionality in school discipline and recommendation for changes. The Clearing House. Classroom Management for Middle and Secondary Schools, 79(1), 45–50. [Google Scholar]
Munn, P. , Cullen, M. A. , Johnstone, M. , & Lloyd, G. (2001). Exclusion from school: A view from Scotland of policy and practice. Research Papers in Education, 16(1), 23–42. 10.1080/02671520010011851 [DOI] [Google Scholar]
Mytton, J. , DiGuiseppi, C. , Gough, D. , Taylor, R. , & Logan, S. (2006). School‐based secondary prevention programmes for preventing violence. Cochrane Database Syst Rev, (3), Art. No.CD004606. 10.1002/14651858.CD004606.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
Nichols, J. D. (2004). An exploration of discipline and suspension data. The Journal of Negro Education, 73(4), 408–423. [Google Scholar]
Nickerson, A. B. , & Spears, W. H. (2007). Influences on authoritarian and educational/therapeutic approaches to school violence prevention. Journal of School Violence, 6(4), 3–31. [Google Scholar]
Noltemeyer, A. L. , & Fenning, P. (2013). Systemic school discipline: Issues of equity from a social justice perspective. In Shriberg D., Song S. Y., Miranda A. H., & Radliff K. M. (Eds.), School psychology and social justice: Conceptual foundations and tools for practice. (pp. 91–117). New York, NY: Routledge. [Google Scholar]
Noltemeyer, A. L. , & McLoughlin, C. S. (2010). Changes in exclusionary discipline rates and disciplinary disproportionality over time. International Journal of Special Education, 25(1), 59–70. [Google Scholar]
Obsuth, I. , Louise, A. , Tina, M. , Philippe, M. , Ribeaud, D. , & Eisner, M. (2016). A non‐bipartite propensity score analysis of the effects of teacher ‐ student relationships on adolescent problem and prosocial behavior. Journal of Youth & Adolescence, 1–27. 10.1007/s10964-016-0534-y [DOI] [PMC free article] [PubMed] [Google Scholar]
OECD . (2009). Creating effective teaching and learning environments: First results from TALIS. OECD Publishing, 305. [Google Scholar]
OECD . (2010). PISA 2009 results: What makes a school successful? Resources, policies and practices. OECD Publishing, Vol. 4, 306. [Google Scholar]
OECD . (2013). PISA 2012 results: What makes schools successful? Resources, policies and practices (Volume IV). PISA, OECD Publishing. 10.1787/9789264201156-en [DOI] [Google Scholar]
Oliver, R. M. , Wehby, J. H. , & Reschly, D. J. (2011). Teacher Classroom Management Practices: Effects on Disruptive or Aggressive Student Behavior. Campbell Systematic Reviews, (4), 1–55. 10.4073/csr.2011.4 [DOI] [Google Scholar]
Pane, D. M. , Rocco, T. S. , Miller, L. D. , & Salmon, A. K. (2013). How teachers use power in the classroom to avoid or support exclusionary school discipline practices. Urban Education, XX(X), 1–32. 10.1177/0042085913478620 [DOI] [Google Scholar]
Petrosino, A. , & Soydan, H. (2005). The impact of program developers as evaluators on criminal recidivism: Results from meta‐analyses of experimental and quasi‐experimental research. Journal of Experimental Criminology, 1, 435–450. [Google Scholar]
Pigott, T. (n.d.). Computing effects sizes for clustered randomized trials. The Campbell Collaboration. Retrieved from http://www.campbellcollaboration.org/artman2/uploads/1/2_Pigott_Cluster_Randomized_Trials.pdf [Google Scholar]
Raffaele‐Mendez, L. M. (2003). Predictors of suspension and negative school outcomes: A longitudinal investigation. New directions for youth development. Special issue: Deconstructing the school‐to‐prison pipeline, 2003(99), 17–33. 10.1002/yd.52 [DOI] [PubMed] [Google Scholar]
Raush, M. K. , & Skiba, R. (2004). Unplanned outcomes: Suspensions and expulsions in Indiana. Education Policy Briefs, 2(2), 1–8. [Google Scholar]
Reinke, W. M. , Herman, K. C. , Petras, H. , & Ialongo, N. S. (2008). Empirically derived subtypes of child academic and behavior problems: Co‐occurrence and distal outcomes. Journal of Abnormal Child Psychology, 36, 759–770. 10.1007/s10802-007-9208-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
Roberts, I. , & Ker, K. (2016). Cochrane: The unfinished symphony of research synthesis. BioMed Central, 5, 1–5. 10.1186/s13643-016-0290-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
Rocque, M. , & Paternoster, R. (2011). Understanding the antecedents of the “School‐to‐Jail” link: The relationship between race and school discipline. Journal of Criminal Law and Criminology, 101(2), 633–666. [Google Scholar]
Rumberger, R. W. , & Losen, D. J. (2016). The high cost of harsh discipline and its disparate impact. The Centre for Civil Rights Remedies, June. Retrieved fromhttps://www.civilrightsproject.ucla.edu/resources/projects/center‐for‐civil‐rights‐remedies/school‐to‐prison‐folder/federal‐reports/the‐high‐cost‐of‐harsh‐discipline‐and‐its‐disparate‐impact/UCLA_HighCost_6‐2_948.pdf [Google Scholar]
Schindler, H. S. , & Yoshikawa, H. (2012). Preventing crime through intervention in the preschool years. In Welsh B. C. & Farrington D. P. (Eds.), The Oxford Handbook of Crime Prevention (pp. 71–88). Oxford, United Kingdom: Open University Press. [Google Scholar]
Schulz, K. F. , & Grimes, D. A. (2002). Allocation concealment in randomised trials: Defending against deciphering. The Lancet, 359(9306), 614–618. [DOI] [PubMed] [Google Scholar]
Schwartz, S. E. , Rhodes, J. E. , Spencer, R. , & Grossman, J. B. (2013). Youth initiated mentoring: Investigating a new approach to working with vulnerable adolescents. American Journal of Community Psychology, 52(1‐2), 155–169. 10.1007/s10464-013-9585-3 [DOI] [PubMed] [Google Scholar]
Shadish, W. , Cook, T. , & Campbell, D. (2002). Experimental and Quasi‐Experimental Designs for Generalized Causal Inference. Boston New York: Houghton Mifflin Company. [Google Scholar]
Shapiro, J. P. , Burgoon, J. D. , Welker, C. J. , & Clough, J. B. (2002). Evaluation of the Peacemakers Program: School‐based violence prevention for students in grades four through eight. Psychology in the Schools, 39(1), 87–100. 10.1002/pits.10040 [DOI] [Google Scholar]
Sharkey, J. D. , & Fenning, P. A. (2012). Rationale for designing school contexts in support of proactive discipline. Journal of School Violence, 11(2), 95–104. 10.1080/15388220.2012.646641 [DOI] [Google Scholar]
Sherman, L. W. (1993). Defiance, deterrence, and irrelevance: A theory of the criminal sanction. Journal of Research in Crime and Delinquency, 30(4), 445–473. 10.1177/0022427893030004006 [DOI] [Google Scholar]
Sherman, L. W. , Farrington, D. , Welsh, B. C. , & Mackenzie, D. L. (2002). Evidence‐based crime prevention. Revised edition. New York NY: Routledge. [Google Scholar]
Sherman, L. W. , Strang, H. , Barnes, G. , Bennett, S. , Angel, C. M. , Newbury‐Birch, D. , … & Gill, C. E. (2007). Restorative justice: The evidence. Retrieved from http://tees.openrepository.com/tees/handle/10149/600940
Singal, A. G. , Higgins, P. D. R. , & Waljee, A. K. (2014). Clinical/narrative review. A primer on effectiveness and efficacy trials. Clinical and Translational Gastroenterology, 5(e45), 1–4. 10.1038/ctg.2013.13 [DOI] [PMC free article] [PubMed] [Google Scholar]
Simmons, N. , Donnell, D. , Ou, S. , Celentano, D. D. , Aramrattana, A. , Davis‐vogel, A. , … Latkin, C. (2015). Assessment of contamination and misclassification biases in a randomized controlled trial of a social network peer education intervention to reduce HIV risk behaviors among drug users and risk partners in Philadelphia, PA and Chiang Mai, Thailand. AIDS Behaviour, 19(10), 1818–1827. 10.1007/s10461-015-1073-3.Assessment [DOI] [PMC free article] [PubMed] [Google Scholar]
Skiba, R. , & Peterson, R. (1999). The dark side of zero tolerance: Can punishment lead to safe schools? The Phi Delta Kappan, 80(5), 372–382. [Google Scholar]
Skiba, R. , Trachok, M. , Chung, C. , Baker, T. , & Hughes, R. (2012). Parsing disciplinary disproportionality: Contributions of behavior, student, and school characteristics to suspension and expulsion. Paper Presented at the Annual Meeting of the American Educational Research Association, Vancouver, British Columbia, Canada. 1–32. [Google Scholar]
Skiba, R. J. (2014). The failure of zero tolerance. Reclaiming Children and Youth, 22(4), 27–33. [Google Scholar]
Skiba, R. J. , Chung, C.‐G. , Trachok, M. , Baker, T. , Sheya, A. , & Hughes, R. (2015). Where should we intervene? In Losen D. J. (Ed.), Closing the school discipline gap: Equitable remedies for excessive exclusion (pp. 132–146). New York, NY: Teachers College, Columbia University. [Google Scholar]
Skiba, R. J. , Michael, R. S. , Nardo, A. C. , & Peterson, R. L. (2002). The colour of discipline: Sources of racial and gender disproportionality in school punishment. The Urban Review, 34(4), 317–342. [Google Scholar]
Solomon, B. G. , Klein, S. A. , Hintze, J. M. , Cressey, J. M. , & Peller, S. L. (2012). A meta‐analysis of School‐Wide Positive Behavior Support: An exploratory study using single‐case synthesis. Psychology in the Schools, 49(2), 105–121. 10.1002/pits [DOI] [Google Scholar]
Spielhofer, T. , Benton, T. , Evans, K. , Featherstone, G. , Golden, S. , Nelson, J. , & Smith, P. (2009). Increasing participation: Understanding young people who do not participate in education or training at 16 and 17. National Foundation, DCSF‐RR072, i‐121. [Google Scholar]
Spier, E. , Britto, P. , Pigott, T. , Kidron, Y. , Roehlkepartain, E. , Scales, P. , … Glover, J. (2013). Parental, Familial, and Community Support Interventions to Improve Children's Literacy in Developing Countries?: A Systematic Review, (July), 1–11. [Google Scholar]
Spink, L. A. (2011). Disciplinary exclusion: The impact of intervention and influence of school ethos. School of Education, Communication and Language Sciences. Doctorate Thesis, (August), i‐80. [Google Scholar]
Sprick, R. S. , Sprick, M. , & Garrison, M. (1992). Foundations: Establishing positive discipline policies. Longmont, CO: Sopris West. [Google Scholar]
Stern, J. A. , Higgins, J. P. T. , & Reeves, B. C. (2014). A Cochrane Risk Of Bias Assessment Tool: for Non‐ Randomized Studies of Interventions (ACROBAT‐NRSI), Version 1.0.0. Retrieved from https://53e94a67‐a‐62cb3a1a‐s‐sites.googlegroups.com/site/riskofbiastool/ACROBAT‐NRSIVersion1_0_0.pdf?attachauth=ANoY7creuVJvG2gOPNF1P_n5VfzHzV6dH8b2xsik6Ybbzpch56S5KvqB2olZtd7BfsRmKwezAExgJggqhy2MfRLdIb3HaelzIAz_QGSMEXxmjo8REilzGq3XjJjx_z9yYh_46Sulg99n
Sullivan, A. L. , Van Norman, E. R. , & Klingbeil, D. A. (2014). Exclusionary discipline of students with disabilities: Student and school characteristics predicting suspension. Remedial and Special Education, 35(4), 199–210. 10.1177/0741932513519825 [DOI] [Google Scholar]
Sutherland, A. , Ilie, S. , & Vignoles, A. (2015). Factors associated with achievement: key stage 2. RAND Europe & University of Cambrdge. [Google Scholar]
Torche, F. , & Mizala, A. (2012). School discipline in Chile. In Arum R. & Velez M. (Eds.), Improving learning environments. School discipline and student achievement in comparative perspective (pp. 74–103). Stanford, California: Stanford University Press. [Google Scholar]
Torgerson, D. J. (2001). Contamination in trials: Is cluster randomisation the answer? BMJ, 322, 355–357. [DOI] [PMC free article] [PubMed] [Google Scholar]
U.S. Department of Education . (2012). School climate and discipline: Know the data. Retrieved from http://www2.ed.gov/policy/gen/guid/school‐discipline/data.html#civil‐rights‐data
Webb, R. , & Vulliamy, G. (2004). A multi‐agency approach to reducing disaffection and exclusions from school. Nottingham: DfES Publications. [Google Scholar]
Wilson, S. J. , Lipsey, M. W. , & Soydan, H. (2003). Are mainstream programs for juvenile delinquency less effective with minority youth than majority youth? A meta‐analysis of outcomes research. Research on Social Work Practice, 13(1), 3–26. 10.1177/1049731502238754 [DOI] [Google Scholar]
Wilson, S. J. , Tanner‐smith, E. E. , Lipsey, M. W. , Steinka‐Fry, K. , & Morrison, J. (2011). Dropout prevention and intervention programs: Effects on school completion and dropout among school‐aged children and youth. Campbell Systematic Reviews, (8), 1–62. 10.4073/csr.2011.8 [DOI] [Google Scholar]
Yudof, M. G. (1975). Suspension and expulsion of black students from the public schools: Academic capital punishment and the constitution. Law and Contemporary Problems, 39(2), 374–411. [Google Scholar]

[cl2014001034-bib-0001] Allen, J. P. , Philliber, S. , Herrling, S. , & Kuperminc, G. P. (1997). Preventing teen pregnancy and academic failure: Experimental evaluation of a developmentally based approach. Child Development, 64(4), 729–742. doi:10.2307/1132122 [Google Scholar]

[cl2014001034-bib-0002] Arter, P. S. (2005). The effects of the positive alternative learning supports program on the attendance, academic performance, and behavior of students with chronic behavior problems. (Doctoral Dissertation). The Johns Hopkins University, Maryland. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-0003] Barnes, V. A. , Bauza, L. B. , & Treiber, F. A. (2003). Impact of stress reduction on negative school behavior in adolescents. Health and quality of life outcomes, 1(10), 1–7. doi:10.1186/1477‐7525‐1‐10 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-0004] Berlanga, D. T. (2004). The efficacy of an intensive guidance intervention program to increase academic success of at‐risk middle school students. (Doctoral Dissertation). Texas A&M University ‐ Corpus Christi. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-0005] Bradshaw, C. P. , Waasdorp, T. E. , & Leaf, P. J. (2012). Effects of School‐Wide Positive Behavioral Interventions and Supports on child behavior problems. Pediatrics, 130(5), E1136–E1145. doi:10.1542/peds.2012‐0243 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-0006] Bragdon, D. (2010). The effects of a career awareness project on the academic performance of at‐risk eighth grade students. (Doctoral Dissertation). The Johns Hopkins University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-0007] Brett, P. F. (1993). An examination of the impact of a mentoring program on inner city, seventh‐grade, African American students' attitudes toward learning and other selected variables. (Doctoral Dissertation). The George Washington University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-0008] Burcham, B. G. (2002). Impact of school‐based social problem‐solving training on middle school students with disruptive behavior. (Doctoral Dissertation). University of Kentucky. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-0009] Collier, C. E. (2002). The effect of pro‐social skills training on the problem behavior of selected African American children in the District of Columbia public schools: Recommendations for change. (Doctoral Dissertation). The George Washington University, US. Retrieved from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00010] Cook, P. J. , Dodge, K. , Farkas, G. , Fryer Jr, R. G. , Guryan, J. , Ludwig, J. , … Steinberg, L. (2014). The (surprising) efficacy of academic and behavioral intervention with disadvantaged youth from a randomized experiment in Chicago. Institute for Policy Research Northwestern University. WP‐14‐03. doi:10.3386/w19862 [Google Scholar]

[cl2014001034-bib-00011] Cornell, D. G. , Allen, K. , & Fan, X. (2012). A randomized controlled study of the Virginia Student Threat Assessment Guidelines in kindergarten through grade 12. School Psychology Review, 41(1), 100–115. Retrieved from <Go to ISI>://WOS:000302757000006 [Google Scholar]

[cl2014001034-bib-00012] Crowder, D. M. (2001). The effect of the gang resistance education and training program on character education and the development of leadership skills in children at an inner‐city middle school. (Doctoral Dissertation). Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00013] Dynarski, M. , Moore, M. , Mullens, J. , Gleason, P. , James‐Burdumy, S. , Rosenberg, L. , … Levy, D. (2003). When schools stay open late: The national evaluation of the 21st‐Century Community Learning Centers program. First year findings. Retrieved from Mathematic Policy Research, Inc. Decision Information Resources, Inc.: U.S. Department of Education; [Google Scholar]

[cl2014001034-bib-00014] Edmunds, J. A. , Bernstein, L. , Unlu, F. , Glennie, E. , Willse, J. , Smith, A. , & Arshavsky, N. (2012). Expanding the start of the college pipeline: Ninth‐grade findings from an experimental study of the impact of the early college high school model. Journal of Research on Educational Effectiveness, 5(2), 136–159. doi:10.1080/19345747.2012.656182 [Google Scholar]

[cl2014001034-bib-00015] Farrell, A. D. , Meyer, A. L. , & White, K. S. (2001). Evaluation of Responding in Peaceful and Positive Ways (RIPP): A school‐based prevention program for reducing violence among urban adolescents. Journal of Clinical Child Psychology, 30(4), 451–463. doi:10.1207/s15374424jccp3004_02 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-00016] Feindler, E. , Marriott, S. , & Iwata, M. (1984). Group anger control training for junior high school delinquents. Cognitive Therapy and Research, 8(3), 299–311. doi:10.1007/BF01173000 [Google Scholar]

[cl2014001034-bib-00017] Harding, H. (2011). An evaluation of a targeted group intervention delivered to year 8 pupils and broadly based on cognitive‐behavioural approaches. (Doctoral Dissertation). University of Nottingham. Retrieved from http://eprints.nottingham.ac.uk/11843/ [Google Scholar]

[cl2014001034-bib-00018] Hawkins, J. D. , Doueck, H. J. , & Lishner, D. M. (1988). Changing teaching practices in mainstream classrooms to improve bonding and behavior of low achievers. American Educational Research Journal, 25(1), 31–50. doi: 10.3102/00028312025001031 [DOI] [Google Scholar]

[cl2014001034-bib-00019] Hirsch, B. J. , Hedges, L. V. , Stawicki, J. , & Mekinda, M. A. (2011). After‐school programs for high school students. An evaluation of After School Matters. Retrieved from http://www.wallacefoundation.org/knowledge‐center/Pages/After‐School‐Programs‐for‐High‐School‐Students‐An‐Evaluation‐of‐After‐School‐Matters.aspx [Google Scholar]

[cl2014001034-bib-00020] Hostetler, M. , & Fisher, K. (1997). Project CARE substance abuse prevention program for high‐risk youth: A longitudinal evaluation of program effectiveness. Journal of Community Psychology, 25(5), 397–419. doi:10.1002/(sici)1520‐6629(199709)25:5<397::aid‐jcop3>3.0.co;2‐q [Google Scholar]

[cl2014001034-bib-00021] Ialongo, N. , Poduska, J. , Werthamer, L. , & Kellam, S. (2001). The distal impact of two first‐grade preventive interventions on conduct problems and disorder in early adolescence. Journal of Emotional and Behavioral Disorders, 9(3), 146–160. doi:10.1177/106342660100900301 [Google Scholar]

[cl2014001034-bib-00022] Johnson. (1983). The effects of affective education on the discipline of selected seventh and eighth grade students in the district of Columbia public schools. (Doctoral Dissertation). The George Washington University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00023] Lewis, K. M. , Schure, M. B. , Bavarian, N. , DuBois, D. L. , Day, J. , Ji, P. , … Flay, B. R. (2013). Problem behavior and urban, low‐income youth a randomized controlled trial of Positive Action in Chicago. American Journal of Preventive Medicine, 44(6), 622–630. doi:10.1016/j.amepre.2013.01.030 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-00024] Mack, M. A. P. (2001). The effects of a behavioral intervention on discipline referrals and school suspension. (Doctoral Dissertation). Auburn University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00025] Obsuth, I. , Sutherland, A. , Cope, A. , Pilbeam, L. , Murray, A. L. , & Eisner, M. (2016). London Education and Inclusion Project (leip): Results from a cluster‐randomized controlled trial of an intervention to reduce school exclusion and antisocial behavior. Journal of Youth & Adolescence. doi:10.1007/s10964‐016‐0468‐4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-00026] Okonofua, J. A. , Pauneskua, D. , & Walton, G. M. (2016). Brief intervention to encourage empathic discipline cuts suspension rates in half among adolescents. PNAS, 113(19), 5221–5226. doi:10.1073/pnas.1523698113 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-00027] Panayiotopoulos, C. , & Kerfoot, M. (2004). A home and school support project for children excluded from primary and first year secondary school. Child and Adolescent Mental Health, 9(3), 109–114. [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-00028] Peck, M. (2006). The effects of group involvement in a prevention program on at‐risk youth's grades, attendance, substance use, and contacts with juvenile probation: An archival study of the STOP program. (Doctoral Dissertation). Alliant International University, Fresno, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00029] Reese, S. C. , Murphy, R. J. , & Filipczak, J. (1981). Assessment of multiple behavioral procedures on academic and social classroom behavior. Psychology in the Schools, 18(3), 349–355. doi:10.1002/1520‐6807(198107)18:3<349::AID‐PITS2310180317>3.0.CO;2‐Q [Google Scholar]

[cl2014001034-bib-00030] Russell, A. (2007). Influencing adaptive functioning in school‐age children: implementation and program evaluation of the Coping Power Program. (Doctoral Dissertation). George Fox University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00031] Shetgiri, R. , Kataoka, S. , Lin, H. , & Flores, G. (2011). A randomized, controlled trial of a school‐based intervention to reduce violence and substance use in predominantly Latino high school students. Journal of the National Medical Association, 103(9), 932–940. [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-00032] Smith, M. E. (2004). A mixed paradigm: Study of a transformational learning program for at‐risk high school students. (Doctoral Dissertation). Texas A&M University ‐ Corpus Christi. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00033] Snyder, F. , Flay, B. , Vuchinich, S. , Acock, A. , Washburn, I. , Beets, M. , & Li, K.‐K. (2010). Impact of a social‐emotional and character development program on school‐level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched‐pair, cluster‐randomized, controlled trial. Journal of Research on Educational Effectiveness, 3(1), 26. [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-00034] Sprague, J. R. , Biglan, A. , Rusby, J. , Gau, J. M. , & Vincent, C. G. (2016). Implementing School‐Wide PBIS in middle schools: Results of a randomized trial (unpublished paper). The University of Oregon. [Google Scholar]

[cl2014001034-bib-00035] Tilghman, W. S. (1988). The effect of peer counseling on lowering the rate of middle school suspensions. (Doctoral Dissertation). Florida Atlantic University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00036] Ward, B. , & Gersten, R. (2013). A randomized evaluation of the safe and civil schools model for positive behavioral interventions and supports at elementary schools in a large urban school district. School Psychology Review, 42(3), 317–333. [Google Scholar]

[cl2014001034-bib-00037] Wyman, P. A. , Cross, W. , Brown, C. H. , Yu, Q. , Tu, X. , & Eberly, S. (2010). Intervention to strengthen emotional self‐regulation in children with emerging mental health problems: Proximal impact on school behavior. Journal of Abnormal Child Psychology, 38(5), 707–720. doi:10.1007/s10802‐010‐9398‐x [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-00038] Ballard, K. L. (2009). School‐related and social‐emotional outcomes of providing mental health services in schools. (Doctoral Dissertation). The University of Wisconsin; ‐ Madison, US. Retrieved from EBSCOhost psyh database. [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-00039] Barber, M. S. (2002). The influence of the Alabama Reading Initiative on the culture of three inner‐city elementary schools in the state of Alabama.(Doctoral Dissertation). The University of Alabama Birmingham, Alabama. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00040] Barnes, L. B. (2010). An evaluation of the right choices program to determine effectiveness in delivering constructive interventions and providing an early support program in order to modify behavior of first‐time student offenders who commit drug and violent acts. (Doctoral Dissertation). Gardner‐Webb University, United States. [Google Scholar]

[cl2014001034-bib-00041] Barton, C. L. (2013). Effectiveness of 1‐2‐3 Magic for Teachers to improve classroom management and discipline. (Doctoral Dissertation). Capella University. Retrieved from EBSCOhost ERIC database. [Google Scholar]

[cl2014001034-bib-00042] Beard, M. L. (2014). Positive behavior intervention and support: An evaluation of its impact in elementary schools. (Doctoral Dissertation). Wingate University, New York. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00043] Blackwell, S. D. (2008). The impact of a transition program on ninth grade students' performance. (Doctoral Dissertation). Virginia Polytechnic Institute and State University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00044] Blevins, L. D. (2007). The use of school‐wide positive behavior support at a rural high school to decrease disruptive behavior for both typical students and students identified with special needs. (Doctoral Dissertation). East Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00045] Blocker, J. B. I. (2013). The effects of the positive behavior intervention and support program on the out‐of‐school suspension rates and english achievement of African american males in grade nine. (Doctoral Dissertation). Union University, United States. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00046] Bogle, K. E. (2007). Evaluation of a brief group parent training intervention in the context of an after‐school program for middle‐school students. (Doctoral Dissertation). University of South Carolina. [Google Scholar]

[cl2014001034-bib-00047] Bond, E. W. (2010). Stuarts draft high school: Quantitative study evaluating the ninth‐grade transition program. (Doctoral Dissertation). Virginia Polytechnic Institute and State University. [Google Scholar]

[cl2014001034-bib-00048] Booth, B. A. (1995). A Cognitively Based Anger Control Training Program with Aggressive Adolescents in School Settings. (Doctoral Dissertation). University at Albany, New York. [Google Scholar]

[cl2014001034-bib-00049] Bradley, O. W. (1985). The effects of the teacher‐advisor system on the students of omaha technical high school, Omaha, Nebraska, in the areas of attitude, attendance, behavior, and achievement. (Doctoral Dissertation). The University of Nebraska ‐ Lincoln. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00050] Bridges, L. (2012). Positive behavior intervention and support: An alternative approach. (Doctoral Dissertation). Loyola University Chicago, United States. Retrieved from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00051] Brinson, J. P. I. (2012). Assessing the effectiveness of a behavior intervention model on fifth grade in‐school suspension rates. (Doctoral Dissertation). Cambridge College, United States. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00052] Brown‐Kersey, N. (2011). The effects of implementing restorative justice in an in‐school suspension program. (Doctoral Dissertation). Western Michigan University. [Google Scholar]

[cl2014001034-bib-00053] Buettner, D. A. (2013). The effect of PBIS on the school environment and test scores. (Doctoral Dissertation). Lindenwood University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00054] Burnham, S. V. (2009). A case study of reducing a junior high school disciplinary infractions and increasing a positive school climate: After‐school detentions vs. evidence‐based computer program. (Doctoral Dissertation). Widener University, US. Retrieved from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00055] Campbell‐Peralta, P. J. (1995). Reducing Suspensions of Minority Males through a Group Guidance/Mentoring Intervention Program. (Master Thesis). Nova Southeastern University. [Google Scholar]

[cl2014001034-bib-00056] Chartas, N. D. (2001). An evaluation of the short‐term influence of Lock Haven University's Upward Bound program. (Doctoral Dissertation). The Pennsylvania State University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00057] Chatzinikolaou, R. (2015). Strength‐based interventions in secondary school: How can they be most helpful for pupils at risk and not‐at‐risk of exclusion?(Doctoral Dissertation). University of East London. Retrieved from http://roar.uel.ac.uk/4432/ [Google Scholar]

[cl2014001034-bib-00058] Clark, W. H. (1980). The effectiveness of supervised discipline centers as an alternative to suspensions in two public secondary schools in Prince George's County, Maryland. (Doctoral Dissertation). University of Southern California, US. Retrieved from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00059] Clarke, P. N. (2008). Program evaluation: A study to evaluate the effectiveness of an alternative Education program in a rural county of a mid‐Atlantic U.S. State. (Doctoral Dissertation). Capella University, US. [Google Scholar]

[cl2014001034-bib-00060] Conyers, R. E. D. (2005). The effects of group counseling compared with individual counseling in the reduction of at‐risk behaviors using grade point average, discipline referrals, and attendance among sixth grade black males. (Doctoral Dissertation). South Carolina State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00061] Corwin, E. P. (2014). An evaluation of the effectiveness of the Check‐in/Check‐out intervention for students engaging in relational aggression and their classmates. (Doctoral Dissertation). Louisiana State University. [Google Scholar]

[cl2014001034-bib-00062] Cramer, J. A. (1990). Orientation to continuation high school: A controlled experiment. (Doctoral Dissertation). University of the Pacific. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00063] Creamean, S. L. , & Horvath, R. J. (2000). The effectiveness of block scheduling. (Master Thesis), Saint Xavier University, Chicago. Illinois. Retrieved from http://search.proquest.com/docview/62347371?accountid=9851 ProQuest Social Sciences Premium Collection database. [Google Scholar]

[cl2014001034-bib-00064] Dalessio, E. S. (2012). The effects of online education on the at‐risk student in an alternative school setting. (Doctoral Dissertation). College of Saint Elizabeth, Morristown, New Jersey. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=2&id=ED549663 ERIC Educational Resources Information Center database. [Google Scholar]

[cl2014001034-bib-00065] Davis, K. S. (2011). Program evaluation of universal Positive Behavioral Interventions and Supports in Kentucky. (Doctoral Dissertation). Western Kentucky University, Bowling Green, Kentucky. Retrieved from http://digitalcommons.wku.edu/diss/6 [Google Scholar]

[cl2014001034-bib-00066] Davis, M. V. C. (1998). A comparison based on student outcomes of an individualized student improvement program with an in‐school suspension program. (Doctoral Dissertation). Georgia State University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00067] De Blank, G. (2009). Effects of quality and quantity of the mentoring relationship on outcomes for at‐risk early adolescent girls. (Doctoral Dissertation). University of Virginia. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00068] Denmark, V. M. (1995). A program evaluation of the peer mediation programs in the junior high schools of a large urban school system. (Doctoral Dissertation). Georgia State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00069] Devlin, N. F. (2006). Effects of in‐school suspension with conflict resolution training on suspensions, referrals, and normative beliefs about aggression. (Doctoral Dissertation). Walden University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00070] Deyoe, M. M. (2015). The relationship of disability status on attendance, behavior, and achievement indicators for students receiving intensive school‐based mental health counseling. (Doctoral Dissertation). University at Albany, State University of New York. [Google Scholar]

[cl2014001034-bib-00071] Draa, V. A. B. (2005). School uniforms in urban public high schools. (Doctoral Dissertation). Youngstown State University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED497409&site=ehost‐live&scope=site Available from EBSCOhost eric database. [Google Scholar]

[cl2014001034-bib-00072] Dunlap, J. A. (2010). Evaluation of the union county alternative to suspension program. (Doctoral Dissertation). Gardner‐Webb University, US. Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Social Sciences Premium Collection database. [Google Scholar]

[cl2014001034-bib-00073] Dykes, D. A. (2015). School‐Wide Positive Behavior Interventions and Supports: A case study measuring implementation fidelity and implementation impact. (Doctoral Dissertation). Georgia State University. Retrieved from http://scholarworks.gsu.edu/eps_diss/121 [Google Scholar]

[cl2014001034-bib-00074] Ebbe, A. R. (2011). Implementing successful middle school and elementary school tier one positive behavior intervention support strategies in a high school. (Doctoral Dissertation). Edgewood College. Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Social Sciences Premium Collection database. [Google Scholar]

[cl2014001034-bib-00075] Eineder, D. V. (1996). The effects of block scheduling in a high school. (Doctoral Dissertation). The University of Alabama, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00076] Feldi, K. (1980). Behavioral treatment of alienated adolescents. (Doctoral Dissertation). Columbia University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00077] Fisher, C. L. (1980). Effects of a middle school counseling program on attendance, academic achievement, and suspensions with a study of attitudes of students, principals, counselors and teachers toward the program. (Doctoral Dissertation). University of South Carolina, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1982‐50083‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00078] Fisher, P. H. (1994). The effects of an urban middle school dropout prevention program on the academic achievement, attendance, attitudes, and conflict resolution skills of at‐risk student. (Doctoral Dissertation). Old Dominion University, Ann Arbor. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00079] Ford, M. L. (2004). Alternative schooling: The influence of a positive peer culture model in a large urban public school setting. (Doctoral Dissertation). Morgan State University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00080] Frost, A. M. (2012). Bully prevention, peer mediation and conflict resolution: Impact of prevention programs on reducing school suspensions. (Doctoral Dissertation). Kansas State University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2013‐99230‐405&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00081] Garcia, S. H. (2015). An analysis of the impact of emotional literacy instruction on at‐risk students. (Doctoral Dissertation). Brandman University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00082] Gouge, W. E. (2011). A comparison research study on the use of school uniforms and graduation, attendance, and suspension rates in East Tennessee. (Doctoral Dissertation). Liberty University, Lynchburg, VA. Retrieved from http://digitalcommons.liberty.edu/cgi/viewcontent.cgi?article=1471&context=doctoral [Google Scholar]

[cl2014001034-bib-00083] Grady, P. L. (2006). Increasing academic, behavioral and psychological engagement among at risk junior high students. (Doctoral Dissertation). California Polytechnic State University. University of Calidornia. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00084] Gretzinger, B. Q. (1988). System‐wide implementation of an assertive discipline‐based behavior management plan: A program evaluation. (Doctoral Dissertation). The University of Southern Mississippi. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00085] Grinage, A. L. (2005). Character education programs and student suspension rates from school: Do character education programs decrease student suspensions from regular instructional public elementary schools in Texas?. (Doctoral Dissertation). University of North Texas. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00086] Guardino, D. (2013). The disproportionate use of discipline: An investigation of the potential impact of school‐wide positive behavioral interventions and supports. (Doctoral Dissertation). University of Oregon. [Google Scholar]

[cl2014001034-bib-00087] Haithcock, F. O. (1996). The effects of a research based high school dropout prevention program on multiple student outcomes. (Doctoral Dissertation). University of Miami. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00088] Hardin, S. A. (2003). An evaluation of the Technical Preparation Program in the Tennessee public school system. (Doctoral Dissertation). Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00089] Harless, M. D. R. (2011). Disciplinary referrals in response to School‐Wide Positive Behavior Plan in a rural middle school setting. (Doctoral Dissertation). East Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00090] Harrison, C. D. (2013). An evaluation of the effects of the academics and behavior Check‐in/Check‐out intervention. (Doctoral Dissertation). University of Oregon. Retrieved from https://scholarsbank.uoregon.edu/xmlui/bitstream/handle/1794/13327/Harrison_oregon_0171A_10763.pdf?sequence=1 [Google Scholar]

[cl2014001034-bib-00091] Hasson, R. C., Jr. (2011). Positive Behavior Supports: A study of implementation at the intermediate school level. (Doctoral Dissertation). The University of Houston‐Clear Lake. Retrieved from EBSCOhost eric database. [Google Scholar]

[cl2014001034-bib-00092] Henderson, D. X. (2012). A mixed methods examination of resilience and social connectedness among youth in an alternative to suspension program. (Doctoral Dissertation). North Carolina State University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2014‐99021‐027&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00093] Herzog, M. R. (1980). An analysis of data and student opinion regarding two alternatives for reducing aberrant student behavior at Niles East High School. (Doctoral Dissertation). George Peabody College for Teachers, Vanderbilt University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1982‐50931‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00094] Hickman, A. L. (2014). Evaluating a school‐based day treatment program for students with challenging behaviors. (Doctoral Dissertation). The College of William and Mary in Virginia. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&id=ED556953 ERIC Educational Resources Information Center database. [Google Scholar]

[cl2014001034-bib-00095] Hill, C. (2014). An examination of the effectiveness of the louisiana gear up program in promoting self‐efficacy, improving academic achievement and increasing teachers' aspirations for their students. (Doctoral Dissertation). Louisiana Tech University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00096] Hill, N. O. (2011). The effects of positive behavioral interventions and supports on students behavior in grade five. (Doctoral Dissertation). Capella University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-00097] Hochman, J. S. (1986). The effectiveness of in‐school suspension as a disciplinary technique in an urban high school with and without intervention strategies. (Dissertation). Virginia Polytechnic Institute & State University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1987‐53042‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-00098] Hostetler, M. L. (1995). Project C.A.R.E. substance abuse prevention program: An evaluation of program effectiveness. (Doctoral Dissertation). The Pennsylvania State University, Ann Arbor. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-00099] Howard, G. P. (2008). Effect of group counseling on at‐risk African‐American female students. (Doctoral Dissertation). Roosevelt University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000100] Huston, K. (1999). A Comparison study of the San Francisco community board whole schools conflict resolution project. (Master Thesis), University of Nevada, Las Vegas. Retrieved from http://digitalscholarship.unlv.edu/cgi/viewcontent.cgi?article=1289&context=thesesdissertations [Google Scholar]

[cl2014001034-bib-000101] Jackson‐Chapman, D. (2006). After‐school intervention programs' impact upon select variables among 10th grade students in the Education Longitudinal Study (ELS) 2002 dataset. (Doctoral Dissertation). The University of Southern Mississippi. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000102] Jacoby, R. L. (2008). The effectiveness of a proactive school‐wide approach to discipline at the middle school level. (Doctoral Dissertation). University Of Missouri‐Kansas City, US. Retrieved from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000103] Johnson, D., Jr. (1980). The impact of developmental counseling in relation to selected variables associated with disruptive behavior of public secondary school students. (Doctoral Dissertation). Union Institute and University, Ann Arbor. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000104] Johnson, W. (2010). Analyses of the impact of school uniforms on violence in North Carolina public high schools. (Doctoral Dissertation). East Carolina University. Retrieved from Available from EBSCOhost eric database. [Google Scholar]

[cl2014001034-bib-000105] Jordan, A. D. (2008). Measuring the efficacy of a ninth grade academy on students with disabilities. (Doctoral Dissertation). Capella University. Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Social Sciences Premium Collection database. [Google Scholar]

[cl2014001034-bib-000106] Kershes, M. B. (1984). An evaluation of an alternative discipline method: The saturday suspension program. (Doctoral Dissertation). George Peabody College for Teachers of Vanderbilt University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000107] Kost, C. P. I. (2013). Assessing an induction program designed to improve achievement of ninth grade students receiving federal lunch benefits. (Doctoral Dissertation). Grand Canyon University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000108] Kula, K. (2012). Social and emotional learning in the schools: The effects of Why Try as an alternative to suspension program. (Doctoral Dissertation). Loyola University Chicago, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000109] Lafrance, J. A. (2009). Examination of the fidelity of School‐Wide Positive Behavior Support implementation and its relationship to academic and behavioral outcomes in Florida. (Doctoral Dissertation). University of Central Florida, US. Retrieved from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000110] Lassen, S. R. (2007). Impact of school‐wide PBS on indicators of social development and academic performance in an inner‐city middle school. (Doctoral Dissertation). University of Kansas, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2007‐99010‐234&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000111] Lavender, D. R. (2009). The effect of an in‐class behavioral intervention plus differentiated instruction program on the achievement and behavior outcomes of verbally disruptive 8th‐grade students with and without co‐occurring reading delimitations. (Doctoral Dissertation). University of Nebraska at Omaha, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000112] Lee, D. H. (2003). The effects of the responsible discipline and IMPACT transition models on the behaviors and attitudes of ninth‐grade students in a large urban high school. (Doctoral Dissertation). The University of North Carolina at Charlotte, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000113] Long, L. (2009). Effectiveness of animal‐assisted therapy interventions in altering childhood aggressive behaviors. (Doctoral Dissertation). Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000114] Long, P. K. (2015). Diminishing the discipline gap: Restorative justice as a promising alternative in one urban school. (Doctoral Dissertation). University of Dayton. Retrieved from https://etd.ohiolink.edu/!etd.send_file?accession=dayton1436815423&disposition=inline [Google Scholar]

[cl2014001034-bib-000115] Martinez, S. (2012). The impact of a literacy intervention program on student achievement and behavior at two urban high schools. (Doctoral Dissertation). California State University. Retrieved from ProQuest Social Sciences Premium Collection database. [Google Scholar]

[cl2014001034-bib-000116] Matarazzo, T. M. (1999). Block scheduling: A comparison of block scheduling and traditional scheduling and its relationship to student achievement, student satisfaction and teacher methodology in a junior‐senior high school in New Jersey. (Doctoral Dissertation). Seton Hall University, College of Education and Human Services. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000117] Matthews, A. L. G. (1997). A longitudinal study of two at‐risk student programs in Henry County. (Doctoral Dissertation). University of Georgia. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000118] McCallum, L. C. (1989). An evaluation of the assertive discipline program at Charles D. Owen high school. (Doctoral Dissertation). The University of North Carolina at Greensboro, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1990‐57395‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000119] McCauley, A. E. (1991). An evaluation of the High Point, North Carolina Cities in Schools Program. (Doctoral Dissertation). The University of North Carolina at Greensboro. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000120] McDaniel‐Campbell, C. (2011). The impact of an alternative education intervention (Student transition and recovery) on middle schools' attendance, academic performance, and discipline. (Doctoral Dissertation). Georgia Southern University US. [Google Scholar]

[cl2014001034-bib-000121] McGreevy, S. A. (2007). Girl power: The impact of an academic and social support program on the educational outcomes of urban seventh and eighth grade Latinas. (Doctoral Dissertation). State University of New York at Buffalo. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000122] McKinney, M. L. (2004). The effectiveness of a school‐based drug and alcohol treatment program. (Doctoral Dissertation). Alliant International University, San Francisco Bay. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000123] McTague, J. R. (2006). The effect of a master schedule on student discipline and classroom attendance. (Doctoral Dissertation). Loyola University Chicago, Ann Arbor. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000124] Melton, J. M. (2002). An analysis of the relationship between the Fight Free Program and school climate. (Doctoral Dissertation). Saint Louis University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000125] Mitchell, S. A. (2007). Examining the effectiveness of a school‐based mental health center's services. (Doctoral Dissertation). University of Rhode Island and Rhode Island College. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000126] Muehlrath, K. L. (1996). The effect of conflict management training programs in preventing discipline problems of students in grades nine through twelve in a suburban school district. (Doctoral Dissertation). Saint Louis University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000127] Mushok, M. A. (1989). An exploration of the effects of a behavior modification program and a school/business partnership upon adolescents' self‐esteem and school performance. (Doctoral Dissertation). University of Massachusetts Amherst. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000128] Norris, A. N. (2008). An exploratory study on the effect of restorative justice on school success and disciplinary incidents. (Master Thesis), Michigan State University. Retrieved from ProQuest Social Sciences Premium Collection database. [Google Scholar]

[cl2014001034-bib-000129] O'Leary, J. L. (2012). The impact of the ‘Social And Emotional Aspects Of Learning’ framework on primary aged pupils' learning, attendance and behaviour. (Doctoral Dissertation). University of Manchester. Retrieved from https://www.escholar.manchester.ac.uk/uk‐ac‐man‐scw:161327 [Google Scholar]

[cl2014001034-bib-000130] Ohlson, M. (2009). A study of school culture, leadership, teacher quality and student outcomes via a performance framework in elementary schools participating in a school reform initiative. (Doctoral Dissertation). University of Florida, US. Retrieved from EBSCOhost ERIC database. [Google Scholar]

[cl2014001034-bib-000131] Okunade, A. O. (2015). Survey of students' perceptions of assertive discipline and restorative discipline in American Christian Academy, Ibadan, Nigeria. (Master Thesis), Dordt College, Sioux Center, Iowa. Retrieved from http://digitalcollections.dordt.edu/med_theses/89/ [Google Scholar]

[cl2014001034-bib-000132] Perreault, A. (2013). An examination of the leadership practices that support and sustain school based mental health programs. (Doctoral Dissertation). San Diego State University. Retrieved from https://sdsu‐dspace.calstate.edu/bitstream/handle/10211.3/115548/Perreault_sdsu_0220D_10167.pdf?sequence=1 [Google Scholar]

[cl2014001034-bib-000133] Phipps, J. D. (2012). The effects of character education in South Carolina's secondary schools. (Doctoral Dissertation). Capella University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000134] Pilot, G. C. (2011). The impact of a school‐based enterprise program on the achievement and behavior of special education students attending high schools in South Carolina. (Doctoral Dissertation). South Carolina State University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000135] Pollitt, K. (2010). Exploring the value of engagement mentoring as a preventative strategy with at‐risk youth. (Doctoral Dissertation). University of Birmingham. Retrieved from http://etheses.bham.ac.uk/1001/1/Pollitt10ApEdPsyD1_A1a.pdf [Google Scholar]

[cl2014001034-bib-000136] Porter, S. M. (2007). The impact of a School‐Wide Token economy on behavior, attendance, and academics at Morgan High School. (Master Thesis), Marietta College. Retrieved from https://etd.ohiolink.edu/!etd.send_file%3Faccession%3Dmarietta1174947309%26disposition%3Dinline [Google Scholar]

[cl2014001034-bib-000137] Power, J. (2012). The effect of implementing School‐Wide Positive Behavior Interventions and Supports on student misbehavior in a large urban high school. (Doctoral Dissertation). Liberty University, US. Retrieved from EBSCOhost eric database. [Google Scholar]

[cl2014001034-bib-000138] Pratt, L. (2009). Supporting pupils at risk of exclusion: an evaluation of an intensive, out‐of‐school, emotional literacy programme for key stage 3 pupils. Institute of Education, University of London. [Google Scholar]

[cl2014001034-bib-000139] Ranelli, F. (2007). The effectiveness of different disciplinary programs in two New Jersey high schools. (Doctoral Dissertation). Wilmington University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2008‐99090‐565&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000140] Reed, J. G. (2004). An examination of treatment integrity practices and behavioral outcomes when utilizing the Second Step curriculum. (Doctoral Dissertation). University of Maryland. Retrieved from http://drum.lib.umd.edu/bitstream/handle/1903/1559/umi‐umd‐1465.pdf?sequence=1&isAllowed=y [Google Scholar]

[cl2014001034-bib-000141] Rentschler, D. M. (1996). A longitudinal study to determine the long‐term effects of prevention and intervention substance abuse strategies on at‐risk fourth grade students and their families. (Doctoral Dissertation). Widener University, US. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000142] Riese, S. A. (2004). The efficacy of a violence prevention intervention with elementary school children. (Doctoral Dissertation). Walden University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000143] Rizzi, L. (2011). Program evaluation: A study to evaluate the effectiveness of academic and behavioral interventions in an alternative to out of school suspension program. (Doctoral Dissertation). Missouri Baptist University. Retrieved from EBSCOhost eric database. [Google Scholar]

[cl2014001034-bib-000144] Robinson, R. L. (1985). Investigation of three specific intervention strategies to reduce the attrition rate of the potential high school dropout. (Doctoral Dissertation). University of Maryland College Park. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000145] Romero, A. (1998). The academic effectiveness of school‐site drug intervention programs. (Master Thesis), California State University, Long Beach. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000146] Roseberry, L. L. (1997). An applied experimental evaluation of conflict resolution curriculum and social skills development. (Doctoral Dissertation). Loyola University Chicago, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1997‐95017‐218&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000147] Samuels, K. S. (2003). The relationship of school uniforms to students' achievement, attendance, discipline referrals and perceptions: An analysis of one urban school district. (Doctoral Dissertation). The University of Alabama at Birmingham, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000148] Schellenberg, R. C. (2005). School violence: Evaluation of an elementary school peer mediation program. (Doctoral Dissertation). Regent University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000149] Schmidt, P. A. M. (1995). Helping teen mothers make positive choices thereby reducing suspensions through a curriculum for conflict resolution/peer mediation. (Dissertation). Nova Southeastern University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED386642&site=ehost‐live&scope=site Available from EBSCOhost eric database. [Google Scholar]

[cl2014001034-bib-000150] Schultz, E. K. (2007). State and local level implementation of Schoolwide Positive Behavior Support: An examination of the Texas Behavior Support initiative (TBSI). (Doctoral Dissertation). University of North Texas. [Google Scholar]

[cl2014001034-bib-000151] Shimizu, B. (2000). A study to determine what impact, if any, the implementation of a mandatory uniform dress policy has had on improving student behavior among ninth‐grade students in three california high schools. (Doctoral Dissertation). University of La Verne, California. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000152] Simington, K. W. (1996). The use of brief family counseling with middle school students experiencing discipline problems. (Doctoral Dissertation). The University of North Carolina at Greensboro. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000153] Simmons, C. L. (2015). Impacts of prosocial intervention on delinquent and defiant behaviors in youth grades 7‐12: A program evaluation. (Doctoral Dissertation). Alliant International University, Fresno. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000154] Simpson, M. T. (2010). The impact of Schoolwide Positive Behavior Support on the discipline gap in the middle school setting. (Doctoral Dissertation). Walden University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2011‐99110‐058&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000155] Skoczek, S. J. (2011). The effects of the web transition program on middle school student gpa, attendance, and suspensions. (Doctoral Dissertation). Baker University, US. Retrieved from https://www.bakeru.edu/images/pdf/SOE/EdD_Theses/Skoczek_Steve.pdf [Google Scholar]

[cl2014001034-bib-000156] Smith, S.‐R. S. (2009). Positive behavior support in schools universal intervention: Suspension rates and school climate in implementing middle schools. (Doctoral Dissertation). Fairleigh Dickinson University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000157] Speirs, R. (1994). Decreasing suspensions in grades nine through twelve through the implementation of a peace curriculum. (Doctoral Dissertation). Nova University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED378090&site=ehost‐live&scope=site Available from EBSCOhost eric database. [Google Scholar]

[cl2014001034-bib-000158] Stevenson, H. L., III. (1999). An analysis of requiring school uniforms and its impact on student behavior: Implications for school reform. (Doctoral Dissertation). Texas Southern University, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000159] Swantic, F. M. (1986). An investigation of the effectiveness of bibliotherapy on middle grade students who repeatedly display inappropriate behavior in the school setting. (Doctoral Dissertation). Georgia State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000160] Thornsberry, J. L. (2010). Freshman transition and its effectiveness on student success as measured by improved attendance, improved grades, decreased discipline referrals, and decreased dropout rate. (Doctoral Dissertation). Lindenwood University. [Google Scholar]

[cl2014001034-bib-000161] Thornton, K. M. (2008). A quantitative study comparing traditional high schools and high schools implementing freshman academies in the state of Tennessee. (Doctoral Dissertation). East Tennessee State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000162] Thunell, T. E. (1990). An evaluation of the effectiveness of PASS, a junior high school dropout prevention program. (Doctoral Dissertation). Colorado State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000163] Tomlin, V. E. (1994). A mentor program for improving the academic attainment of black adolescent males. (Doctoral Dissertation). University of Denver. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000164] Toms, O. M. (2012). The effects of Check‐in Check‐out on the social and academic planning and outcomes of African‐American males in an urban secondary setting. (Doctoral Dissertation). The University of North Carolina at Charlotte, US. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000165] Tyre, A. D. (2003). State‐level implementation of schoolwide positive behavior support: An evaluation of the Arizona Behavioral Initiative. (Doctoral Dissertation). Northern Arizona University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2003‐95015‐114&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000166] VanMetre, J. P. (2009). Examining the effectiveness of a ninth‐grade transition program. (Doctoral Dissertation). Lamar University, US. Retrieved from http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&id=ED526690 ERIC Educational Resources Information Center database. [Google Scholar]

[cl2014001034-bib-000167] Vaughan, A. C. (2001). The implementation of uniform dress policies: Student performance outcomes and perceptions of student impact at five public elementary schools in Hampton roads, Virginia. (Doctoral Dissertation). The George Washington University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2001‐95017‐007&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000168] Veliz, P. (2012). The role of interscholastic sport in public high schools: A zero‐sum game or a bridge to success. (Doctoral Dissertation). University at Buffalo, State University of New York. Retrieved from ProQuest Social Sciences Premium Collection database. [Google Scholar]

[cl2014001034-bib-000169] Warren, R. A. (2011). The positive alternative credit experience (PACE) program a quantitative comparative study. (Doctoral Dissertation). Lindenwood University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED534547&site=ehost‐live&scope=site; http://gateway.proquest.com/openurl?url_ver=Z39.88‐2004&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&res_dat=xri:pqdiss&rft_dat=xri:pqdiss:3481165 Available from EBSCOhost eric database. [Google Scholar]

[cl2014001034-bib-000170] Washington‐Labat, C. A. (2003). The effects of school uniforms on attendance and discipline among schools with and without mandatory school uniform policies. (Doctoral Dissertation). The University of Southern Mississippi. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000171] Weissflug, K. G. (2009). Effects of Positive Behavior Support Programs on Student Behaviors. (Doctoral Dissertation). Lindenwood University. [Google Scholar]

[cl2014001034-bib-000172] Welch, C. Y. (2008). A mixed‐method study utilizing a token economy to shape behavior and increase academic success in urban students. (Doctoral Dissertation). Walden University. [Google Scholar]

[cl2014001034-bib-000173] Whitaker, F. (2015). The impact of behavior intervention programs on student suspensions. (Doctoral Dissertation). East Carolina University. [Google Scholar]

[cl2014001034-bib-000174] White, C. (2013). Reducing school stand‐downs: A case study of one New Zealand secondary school. (Master Thesis), Massey University, New Zealand. [Google Scholar]

[cl2014001034-bib-000175] Wickham, I. (2010). A comparative evaluation to determine the effectiveness of the behaviour support classrooms and other positive behaviour management interventions in designated disadvantaged schools. (Doctoral Dissertation). Dublin City University. [Google Scholar]

[cl2014001034-bib-000176] Wilczynski, J. D. (2004). An evaluation of a high school alternative to suspension program for students with gang‐related disciplinary violations. (Doctoral Dissertation). Loyola University of Chicago, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2004‐99017‐053&site=ehost‐live&scope=site Available from EBSCOhost psyh database. [Google Scholar]

[cl2014001034-bib-000177] Wilkerson‐Arbisi, E. L. (2015). The influence of participation in School‐Wide Positive Behavior Support on elementary student behavior. (Doctoral Dissertation). Baker University. Retrieved from http://bakeru.bakeruniversity.netdna‐cdn.com/images/pdf/SOE/EdD_Theses/Wilkerson‐Arbisi_Elizabeth.pdf [Google Scholar]

[cl2014001034-bib-000178] Wilson, L. J. (1998). Conflict resolution programs and the Houston independent school district: Peer mediation vs. non‐peer mediation approaches for resolving student conflict in elementary and middle schools. (Master Thesis), University of Houston‐Clear Lake. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000179] Wolgemuth, J. , Cobb, R. , & Dugan, J. (2007). The effects of self‐management interventions on academic outcomes for youth with disabilities. Ft. Collins, CO: Colorado State University, School of Education. [Google Scholar]

[cl2014001034-bib-000180] Woodfork, J. D. (1987). The effects of an assertive discipline approach on disciplnary problems in selected high schools. (Education Especialist Degree), Atlanta University. [Google Scholar]

[cl2014001034-bib-000181] Wooten, S. D. (2006). The impact of a ninth and tenth grade academy program on student attendance, discipline, and achievement. (Doctoral Dissertation). Clemson University. Retrieved from ProQuest Dissertations & Theses A&I database. [Google Scholar]

[cl2014001034-bib-000182] York, M. T. (2012). The impact of problem‐based learning on ninth‐grade african american male disengagement in high school. (Doctoral Dissertation). Gardner‐Webb University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000183] Youmans, C. W. (2001). Group counseling compared with individual counseling in the reduction of at‐risk behaviors in black male students. (Doctoral Dissertation). South Carolina State University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000184] Zucca‐Brown, S. (1997). An elementary school mediation program: Its effect on student mediators and school violence. (Doctoral Dissertation). Temple University. Retrieved from ProQuest Dissertations & Theses A&I: Social Sciences database. [Google Scholar]

[cl2014001034-bib-000185] Achilles, G. M. , McLaughlin, M. J. , & Croninger, R. G. (2007). Sociocultural correlates of disciplinary exclusion among students with emotional, behavioral, and learning disabilities in the SEELS national dataset. Journal of Emotional and Behavioral Disorders, 15(1), 33–45. doi:10.1177/10634266070150010401 [Google Scholar]

[cl2014001034-bib-000186] Alexander, R. , & Curtis, C. M. (1995). A critical review of strategies to reduce school violence. Children & Schools, 17(2), 73–82. [Google Scholar]

[cl2014001034-bib-000187] Allen, J. P. , Kuperminc, G. , Philliber, S. , & Herre, K. (1994). Programmatic prevention of adolescent problem behaviors: The role of autonomy, relatedness, and volunteer service in the Teen Outreach Program. American Journal of Community Psychology, 22(5), 617–638. doi:10.1007/bf02506896 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000188] Allen, J. P. , & Philliber, S. (2001). Who benefits most from a broadly targeted prevention program? Differential efficacy across populations in the Teen Outreach Program. Journal of Community Psychology, 29(6), 637–655. [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000189] Allen‐Meares, P. , Montgomery, K. L. , & Kim, J. S. (2013). School‐based social work interventions: A cross‐national systematic review. Social Work, 58(3), 253–262. doi:10.1093/sw/swt022 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000190] Andrews, S. P. , Taylor, P. B. , Martin, E. P. , & Slate, J. R. (1998). Evaluation of an alternative discipline program. The High School Journal, 81(4), 209–217. [Google Scholar]

[cl2014001034-bib-000191] Anyon, Y. , Jenson, J. M. , Altschul, I. , Farrar, J. , McQueen, J. , Greer, E. , … Simmons, J. (2014). The persistent effect of race and the promise of alternatives to suspension in school discipline outcomes. Children and Youth Services Review, 44, 379–386. doi:10.1016/j.childyouth.2014.06.025 [Google Scholar]

[cl2014001034-bib-000192] Arter, P. S. (2007). The positive alternative learning supports program: collaborating to improve student success. Teaching Exceptional Children, 40(2), 38–46. [Google Scholar]

[cl2014001034-bib-000193] Atkins, M. S. , McKay, M. M. , Frazier, S. L. , Jakobsons, L. J. , Arvanitis, P. , Cunningham, T. , … Lambrecht, L. (2002). Suspensions and detentions in an urban, low‐income school: Punishment or reward? Journal of Abnormal Child Psychology, 30(4), 361–371. doi:10.1023/a:1015765924135 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000194] Aughinbaugh, A. (2001). Does Head Start yield long‐term benefits? The Journal of Human Resources, 36(4), 641–665. [Google Scholar]

[cl2014001034-bib-000195] Bagley, C. , & Mallick, K. (2000). Spiralling up and spiralling down: Implications of a long‐term study of temperament and conduct disorder for social work with children. Child & Family Social Work, 5(4), 291–301. [Google Scholar]

[cl2014001034-bib-000196] Bagley, C. , & Pritchard, C. (1998). The reduction of problem behaviours and school exclusion in at‐risk youth: An experimental study of school social work with cost‐benefit analyses. Child & Family Social Work, 3(4), 219–226. doi:10.1046/j.1365‐2206.1998.00101.x [Google Scholar]

[cl2014001034-bib-000197] Ballard, K. L. , Sander, M. A. , & Klimes‐Dougan, B. (2014). School‐related and social‐emotional outcomes of providing mental health services in schools. Community Mental Health Journal, 50(2), 145–149. doi:10.1007/s10597‐013‐9670‐y [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000198] Barboza, G. E. (2015). The association between school exclusion, delinquency and subtypes of cyber‐ and F2F‐victimizations: Identifying and predicting risk profiles and subtypes using latent class analysis. Child Abuse & Neglect, 39, 109–122. doi: 10.1016/j.chiabu.2014.08.007 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000199] Bartik, T. J. , & Lachowska, M. (2012). The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes. Upjohn Institute Working Paper No. 12‐186. W. E. Upjohn Institute for Employment Research. [Google Scholar]

[cl2014001034-bib-000200] Bartik, T. J. , & Lachowska, M. (2014). The Kalamazoo Promise Scholarship. Education Next, 14(2), 72–78. [Google Scholar]

[cl2014001034-bib-000201] Bauer, N. S. , Lozano, P. , & Rivara, F. P. (2007). The effectiveness of the Olweus Bullying Prevention Program in Public Middle Schools: A Controlled Trial. Journal of Adolescent Health, 40(3), 266–274. doi:10.1016/j.jadohealth.2006.10.005 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000202] Beets, M. , Flay, B. , Vuchinich, S. , Snyder, F. , Acock, A. , Li, K.‐K. , … Durlak, J. (2009). Use of a social and character development program to prevent substance use, violent behaviors, and sexual activity among elementary‐school students in Hawaii. Research and Practice, 99(8), 1438–1445. [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000203] Bell, S. K. , Coleman, J. K. , Anderson, A. , Whelan, J. P. , & Wilder, C. (2000). The effectiveness of peer mediation in a low‐SES rural elementary school. Psychology in the Schools, 37(6), 505–516. doi:10.1002/1520‐6807(200011)37:6<505::aid‐pits3>3.0.co;2‐5 [Google Scholar]

[cl2014001034-bib-000204] Bevington, T. J. (2015). Appreciative evaluation of restorative approaches in schools. Pastoral Care in Education, 33(2), 105–115. doi:10.1080/02643944.2015.1046475 [Google Scholar]

[cl2014001034-bib-000205] Bonell, Fletcher, A. , Fitzgerald‐Yau, N. , Hale, D. , Allen, E. , Elbourne, D. , … Viner, R. (2015). Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): A pilot randomised controlled trial. Health Technology Assessment, 19(53), i‐109. [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000206] Bouhours, T. , Bryer, F. , & Fleming, S. (2003). Exclusion of primary school students: Archival analysis of school records over a 30‐year period. Reimagining Practice: Researching Change, 1, 102–114. [Google Scholar]

[cl2014001034-bib-000207] Bowey, L. , & McGlaughlin, A. (2006). The Youth Crime Reduction Video Project: An evaluation of a pilot intervention targeting young people at risk of crime and school exclusion. The Howard Journal, 45(3), 268–283. [Google Scholar]

[cl2014001034-bib-000208] Bowllan, N. M. (2011). Implementation and evaluation of a comprehensive, school‐wide bullying prevention program in an urban/suburban middle school. Journal of School Health, 81(4), 167–173. doi:10.1111/j.1746‐1561.2010.00576.x [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000209] Bradley, R. H. , & Gilkey, B. (2002). The impact of the home instructional program for preschool youngsters (HIPPY) on school performance in 3rd and 6th grades. Early Education and Development, 13(3), 301–312. [Google Scholar]

[cl2014001034-bib-000210] Bradshaw, C. P. , Mitchell, M. M. , & Leaf, P. J. (2010). Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes. Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12(3), 133–148. doi:10.1177/1098300709334798 [Google Scholar]

[cl2014001034-bib-000211] Brady, K. P. , Balmer, S. , & Phenix, D. (2007). School‐police partnership effectiveness in urban schools ‐ An analysis of New York City's Impact Schools Initiative. Education and Urban Society, 39(4), 455–478. doi:10.1177/0013124507302396 [Google Scholar]

[cl2014001034-bib-000212] Breunlin, D. C. , Cimmarusti, R. A. , Bryant‐Edwards, T. L. , & Hetherington, J. S. (2002). Conflict resolution training as an alternative to suspension for violent behavior. Journal of Educational Research, 95(6), 349–357. doi: 10.1080/00220670209596609 [DOI] [Google Scholar]

[cl2014001034-bib-000213] Breunlin, D. C. , Cimmarusti, R. A. , Hetherington, J. S. , & Kinsman, J. (2006). Making the Smart Choice: A systemic response to school‐based violence. Journal of Family Therapy, 28(3), 246–266. doi:10.1111/j.1467‐6427.2006.00350.x [Google Scholar]

[cl2014001034-bib-000214] Bruns, E. J. , Moore, E. , Stephan, S. H. , Pruitt, D. , & Weist, M. D. (2005). The impact of school mental health services on out‐of‐school suspension rates. Journal of Youth and Adolescence, 34(1), 23–30. doi:10.1007/s10964‐005‐1333‐z [Google Scholar]

[cl2014001034-bib-000215] Burke, R. V. , Oats, R. G. , Ringle, J. L. , Fichtner, L. O. N. , & DelGaudio, M. B. (2011). Implementation of a classroom management program with urban elementary schools in low‐income neighborhoods: Does program fidelity affect student behavior and academic outcomes? Journal of Education for Students Placed at Risk (JESPAR), 16(3), 201–218. doi: 10.1080/10824669.2011.585944 [DOI] [Google Scholar]

[cl2014001034-bib-000216] Burton, S. (2006). ‘Over to You’: Group work to help pupils avoid school exclusion. Educational Psychology in Practice, 22(3), 215–236. doi: 10.1080/02667360600845778 [DOI] [Google Scholar]

[cl2014001034-bib-000217] Calbreath, W. , & Crews, M. E. (2011). Peer mediation and middle school students. Perspectives in Peer Programs, 23(1), 3–8. [Google Scholar]

[cl2014001034-bib-000218] Campbell‐Whatley, G. D. , Obiakor, F. , & Algozzine, B. (1997). Effects of innovative programming on youngsters at risk. Journal of At‐Risk Issues, 4(1), 3–9. [Google Scholar]

[cl2014001034-bib-000219] Carlson, C. E. (1990). HIPP: A comprehensive school‐based substance abuse program with cooperative community involvement. Journal of Primary Prevention, 10(4), 289–302. doi:10.1007/bf01326623 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000220] Carney, M. M. , & Buttell, F. (2003). Reducing juvenile recidivism: Evaluating the wraparound services model. Research on Social Work Practice, 13(5), 551–568. doi:DOI: 10.1177/1049731503253364 [Google Scholar]

[cl2014001034-bib-000221] Case, S. , & Haines, K. (2004). Promoting prevention: Evaluating a multi‐agency initiative of Youth Consultation and Crime Prevention in Swansea. Children & Society, 18, 355–370. doi:10.1002/CHI.814 [Google Scholar]

[cl2014001034-bib-000222] Casey, R. E. (1996). Delinquency prevention through vocational entrepreneurship: The new Smyrna Beach Employability skill training model for youthful offenders. Preventing School Failure: Alternative Education for Children and Youth, 40(2), 60–62. doi:10.1080/1045988X.1996.9944654 [Google Scholar]

[cl2014001034-bib-000223] Catalano, R. F. , Berglund, M. L. , Ryan, J. A. , Lonczak, H. S. , & Hawkins, J. D. (2004). Positive youth development in the United States: Research findings on evaluations of Positive Youth Development programs. The Annals of the American Academy, 591(January), 98–124. doi:DOI: 10.1177/0002716203260102 Catch22. Engage in Education Overview. [Google Scholar]

[cl2014001034-bib-000224] Chafouleas, S. M. , & Whitcomb, M. (2004). Integrating home, school, and community resources: Evaluation of a district‐wide prevention program. Reclaiming Children & Youth, 12(4), 203–209. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=cja&AN=12328455&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000225] Charlton, T. , Panting, C. , & Willis, H. (2004). Targeting exclusion, disaffection and truancy in secondary schools: An evaluation of an alternative curriculum for older pupils. Emotional and Behavioural Difficulties, 9(4), 261–275. doi:DOI:10.1177/1363275204050372 [Google Scholar]

[cl2014001034-bib-000226] Chobot, R. B. , & Garibaldi, A. (1982). In‐school alternatives to suspension: A description of Ten School District programs. The Urban Review, 14(4), 317–336. doi:10.1007/BF02171976 [Google Scholar]

[cl2014001034-bib-000227] Cooper, P. , & Lovey, J. (1999). Early intervention in emotional and behavioural difficulties: The role of Nurture Groups. European Journal of Special Needs Education, 14(2), 122–131. doi:10.1080/0885625990140202 [Google Scholar]

[cl2014001034-bib-000228] Cornell, D. , Sheras, P. , Gregory, A. , & Fan, X. (2009). A retrospective study of school safety conditions in high schools using the Virginia Threat Assessment Guidelines versus alternative approaches. School Psychology Quarterly, 24(2), 119–129. doi: 10.1037/a0016182 [Google Scholar]

[cl2014001034-bib-000229] Cornell, D. G. , Gregory, A. , & Fan, X. (2011). Reductions in long‐term suspensions following adoption of the Virginia Student Threat Assessment Guidelines. NASSP Bulletin, 95(3), 175–194. doi:10.1177/0192636511415255 [Google Scholar]

[cl2014001034-bib-000230] Cornell, D. G. , Sheras, P. L. , Kaplan, S. , McConville, D. , Douglas, J. , Elkton, A. , … Cole, J. (2004). Guidelines for student threat assessment: Field‐test findings. School Psychology Review, 33(4), 527–546. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=cja&AN=CJA0380020000830s&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000231] Cunningham, C. E. , Vaillancourt, T. , Cunningham, L. J. , Chen, Y. , & Ratcliffe, J. (2011). Modeling the bullying prevention program design recommendations of students from grades five to eight: A discrete choice conjoint experiment. Aggressive Behavior, 37(6), 521–537. doi:10.1002/ab.20408 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000232] Cunningham, P. B. , & Henggeler, S. W. (2001). Implementation of an empirically based drug and violence prevention and intervention program in public school settings. Journal of Clinical Child & Adolescent Psychology, 30(2), 221–232. doi:10.1207/s15374424jccp3002_9 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000233] Curtis, R. , Van Horne, J. W. , Robertson, P. , & Karvonen, M. (2010). Outcomes of a School‐Wide Positive Behavioral Support Program. Professional School Counseling, 13(3), 159–164. [Google Scholar]

[cl2014001034-bib-000234] Daly, B. P. , Sander, M. A. , Nicholls, E. G. , Medhanie, A. , Vanden Berk, E. , & Johnson, J. (2014). Three‐year longitudinal study of school behavior and academic outcomes: Results from a comprehensive expanded school mental health program. Advances in School Mental Health Promotion, 7(1), 24–41. doi:10.1080/1754730X.2013.867712 [Google Scholar]

[cl2014001034-bib-000235] Davis, F. (2014). Discipline with dignity: Oakland classrooms try healing Instead of punishment. Reclaiming Children and Youth, 23(1), 38–41. [Google Scholar]

[cl2014001034-bib-000236] Dawson, C. A. (2003). A Study on the effectiveness of life space crisis Intervention for students identified with emotional disturbances. Reclaiming Children and Youth, 11(4), 223–230. [Google Scholar]

[cl2014001034-bib-000237] Dawson, C. A. (2015). The multifaceted conflict cycle. A New York City perspective. Reclaiming Children and Youth, 24(1), 25–32. [Google Scholar]

[cl2014001034-bib-000238] Day, C. , Kowalenko, S. , Ellis, M. , Dawe, S. , Harnett, P. , & Scott, S. (2011). The Helping Families Programme: A new parenting intervention for children with severe and persistent conduct problems. Child and Adolescent Mental Health, 16(3), 167–171. doi:10.1111/j.1475‐3588.2010.00588.x [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000239] Day‐Vines, N. L. , & Terriquez, V. (2008). A strengths‐based approach to promoting prosocial behavior among African American and Latino students. Professional School Counseling, 12(2), 170–175. [Google Scholar]

[cl2014001034-bib-000240] DeAngelo, A. J. (2005). The evolution of juvenile justice: Community‐based partnerships through balanced and restorative justice. Corrections Today, 67(5), 104–106. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=cja&AN=17800473&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000241] Dupper, D. R. (1998). An alternative to suspension for middle school youths with behavior problems: Findings from a “school survival” group. Research on Social Work Practice, 8(3), 354–366. doi:10.1177/104973159800800307 [Google Scholar]

[cl2014001034-bib-000242] Dupper, D. R. , & Krishef, C. H. (1993). School‐based social‐cognitive skills training for middle school students with school behavior problems. Children and Youth Services Review, 15(2), 131–142. doi:10.1016/0190‐7409(93)90040‐g [Google Scholar]

[cl2014001034-bib-000243] Englander‐Golden, P. , Elconin, J. , & Miller, K. J. (1985). Aay it straight: Adolescent substance abuse prevention training. Academic Psychology Bulletin, 7, 65–79. [Google Scholar]

[cl2014001034-bib-000244] Englander‐Golden, P. , Elconin, J. , Miller, K. J. , & Schwarzkopf, A. B. (1986). Brief SAY IT STRAIGHT training and follow‐up in adolescent substance abuse prevention. Journal of Primary Prevention, 6(4), 219–230. [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000245] Englander‐Golden, P. , Jackson, J. E. , Crane, K. , Schwarzkopf, A. B. , & Lyle, P. S. (1989). Communication‐skills and self‐esteem in prevention of destructive behaviors. Adolescence, 24(94), 481–502. Retrieved from <Go to ISI>://WOS:A1989AF24100023 [PubMed] [Google Scholar]

[cl2014001034-bib-000246] Ethier, K. , & St Lawrence, J. S. (2002). The role of early, multilevel youth development programs in preventing health risk behavior in adolescents and young adults. Archives of Pediatrics & Adolescent Medicine, 156(5), 429–430. doi:PMID: 11980546 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000247] Etscheidt, S. (2002). Discipline provisions of IDEA: Misguided policy or tacit reform initiatives? Behavioral Disorders, 27(4), 408. [Google Scholar]

[cl2014001034-bib-000248] Evangelou, M. , Brooks, G. , & Smith, S. (2007). The birth to school study: Evidence on the effectiveness of PEEP, an early intervention for children at risk of educational under‐achievement. Oxford Review of Education, 33(5), 581–609. doi:10.1080/03054980701476477 [Google Scholar]

[cl2014001034-bib-000249] Fernandez, E. (2008). Unravelling emotional, behavioural and educational outcomes in a longitudinal study of children in foster‐care. British Journal of Social Work, 38(7), 1283–1301. doi: 10.1093/bjsw/bcm028 [DOI] [Google Scholar]

[cl2014001034-bib-000250] Fernandez‐Batanero, J. M. (2011). In search of differentiating features to enhance the performance and expectations of students at risk of educational exclusion. Revista De Educacion, 355(May), 309–330. doi:10.4438/1988‐592x‐re‐2011‐355‐026 [Google Scholar]

[cl2014001034-bib-000251] Flay, B. R. , & Allred, C. G. (2003). Long‐term effects of the Positive Action® program. American Journal of Health Behavior, 27(Supplement 1), S6–S21. doi:10.5993/AJHB.27.1.s1.2 [PubMed] [Google Scholar]

[cl2014001034-bib-000252] Fletcher, A. , Harden, A. , Brunton, G. , Oakley, A. , & Bonell, C. (2007). Interventions addressing the social determinants of teenage pregnancy. Health Education, 108(1), 29–39. doi: 10.1108/09654280810842111 [DOI] [Google Scholar]

[cl2014001034-bib-000253] Freiberg, H. J. , Huzinec, C. A. , & Templeton, S. M. (2009). Classroom management—a pathway to student achievement: A study of fourteen inner‐city elementary schools. The Elementary School Journal, 110(1), 63–80. doi:10.1086/598843 [Google Scholar]

[cl2014001034-bib-000254] Frigon, A. P. , & Krank, M. D. (2009). Self‐coded indirect memory associations in a brief school‐based intervention for substance use suspensions. Psychology of Addictive Behaviors, 23(4), 736–742. doi:10.1037/a0017125 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000255] Frith, G. H. , Lindsey, J. D. , & Sasser, J. L. (1980). An alternative approach to school suspension: The Dothan Model CHECK UL. Phi Delta Kappan, 61(9), 637–638. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ221601&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000256] Gage, N. A. , Sugai, G. , Lewis, T. J. , & Brzozowy, S. (2015). Academic achievement and School‐Wide Positive Behavior Supports. Journal of Disability Policy Studies, 25(4), 199–209. doi:10.1177/1044207313505647 [Google Scholar]

[cl2014001034-bib-000257] Garibaldi, A. , Blanchard, L. , & Brooks, S. (1996). Conflict resolution training, teacher effectiveness, and student suspension: The impact of a health and safety initiative in the New Orleans public schools. Journal of Negro Education, 65(4), 408–413. doi:10.2307/2967143 [Google Scholar]

[cl2014001034-bib-000258] Gentile, E. , & Imberman, S. A. (2012). Dressed for success? The effect of school uniforms on student achievement and behavior. Journal of Urban Economics, 71(1), 1–17. doi:10.1016/j.jue.2011.10.002 [Google Scholar]

[cl2014001034-bib-000259] Gonzalez, T. (2012). Keeping kids in school: Restorative justice, punitive discipline, and the school to prison pipeline. Journal of Law & Education, 41(2), 281–335. [Google Scholar]

[cl2014001034-bib-000260] Gottfredson, D. C. (1986). An empirical test of school‐based environmental and individual interventions to reduce the risk of delinquent behavior. Criminology, 24(4), 705–731. doi:10.1111/j.1745‐9125.1986.tb01508.x [Google Scholar]

[cl2014001034-bib-000261] Gottfredson, D. C. , Gottfredson, G. D. , & Hybl, L. G. (1993). Managing adolescent behavior. A multiyear, multischool study. American Educational Research Journal, 30(1), 179–215. doi:10.3102/00028312030001179 [Google Scholar]

[cl2014001034-bib-000262] Gottfredson, G. D. , Jones, E. M. , & Gore, T. W. (2002). Implementation and evaluation of a cognitive‐behavioral intervention to prevent problem behavior in a disorganized school. Prevention Science, 3(1), 43–56. [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000263] Gray, P. , & Seddon, T. (2005). Prevention work with children disaffected from school: Findings from the evaluation of two innovative community‐based projects. Health Education, 105(1), 62–72. [Google Scholar]

[cl2014001034-bib-000264] Green, D. G. (2006). Welcome to the House System. Educational Leadership, 63(7), 64–67. [Google Scholar]

[cl2014001034-bib-000265] Green, J. M. , Wood, A. J. , Kerfoot, M. J. , Trainor, G. , Roberts, C. , Rothwell, J. , … Harrington, R. (2011). Group therapy for adolescents with repeated self harm: Randomised controlled trial with economic evaluation. British Medical Journal, 342(d682). doi:10.1136/bmj.d682 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000266] Greflund, S. , McIntosh, K. , Mercer, S. H. , & May, S. L. (2014). Examining disproportionality in school discipline for Aboriginal students in schools implementing PBIS. Canadian Journal of School Psychology, 29(3), 213–235. doi:10.1177/0829573514542214 [Google Scholar]

[cl2014001034-bib-000267] Gresham, F. M. , Hunter, K. K. , Corwin, E. P. , & Fischer, A. J. (2013). Screening, assessment, treatment, and outcome evaluation of behavioral difficulties in an RTI model. Exceptionality, 21(1), 19–33. doi:10.1080/09362835.2013.750115 [Google Scholar]

[cl2014001034-bib-000268] Grizenko, N. , Zappitelli, M. , Langevin, J.‐P. , Hrychko, S. , El‐Messidi, A. , Kaminester, D. , … Stepanian, M. T. (2000). Effectiveness of a social skills training program using self/other perspective‐taking: A nine‐month follow‐up. American Journal of Orthopsychiatry, 70(4), 501–509. doi:10.1037/h0087662 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000269] Güneri, O. Y. , & Çoban, R. (2004). The effect of conflict resolution training on Turkish elementary school students: A quasi‐experimental investigation. International Journal for the Advancement of Counselling, 26(2), 109–124. doi:10.1023/B:ADCO.0000027425.24225.87 [Google Scholar]

[cl2014001034-bib-000270] Haines, K. , & Case, S. (2005). Promoting prevention: Targeting family‐based risk and protective factors for drug use and youth offending in Swansea. British Journal of Social Work, 35(2), 169–187. doi:10.1093/bjsw/bch177 [Google Scholar]

[cl2014001034-bib-000271] Hallam, S. (2007). Evaluation of behavioural management in schools: A review of the behaviour improvement programme and the role of behaviour and education support teams. Child and Adolescent Mental Health, 12(3), 106–112. doi:10.1111/j.1475‐3588.2007.00442.x [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000272] Hallam, S. , & Castle, F. (2001). Exclusion from school: What can help prevent it? Educational Review, 53(2), 169–179. doi:10.1080/00131910120055598 [Google Scholar]

[cl2014001034-bib-000273] Harris, I. M. (1998). The coalition for nonviolence in schools. A case study in Milwaukee, Wisconsin, USA. Peace Research, 30(2), 62. [Google Scholar]

[cl2014001034-bib-000274] Hartnell, N. (2010). Multi‐disciplinary approaches to pupil behaviour in school. The role of evaluation in service delivery. Educational Psychology in Practice, 26(2), 187–203. doi:10.1080/02667361003768534 [Google Scholar]

[cl2014001034-bib-000275] Hausman, A. , Pierce, G. , & Briggs, L. (1996). Evaluation of comprehensive violence prevention education: Effects on student behavior. Journal of Adolescent Health, 19(2), 104–110. doi:10.1016/1054‐139x(96)00128‐0 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000276] Hawkins, J. D. , Catalano, R. F. , Kosterman, R. , Abbott, R. , & Hill, K. G. (1999). Preventing adolescent health‐risk behaviors by strengthening protection during childhood. Archives of Pediatrics & Adolescent Medicine, 153(3), 226–234. Retrieved from <Go to ISI>://WOS:000079019600003 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000277] Hayden, C. (2009). Family group conferences. Are they an effective and viable way of working with attendance and behaviour problems in schools? British Educational Research Journal, 35(2), 205–220. doi:10.1080/01411920802041939 [Google Scholar]

[cl2014001034-bib-000278] Hayden, C. , & Martin, T. (1998). ‘Safer Cities’ and exclusion from school. Journal of Youth Studies, 1(3), 315–331. doi:10.1080/13676261.1998.10593016 [Google Scholar]

[cl2014001034-bib-000279] Head, G. , Kane, J. , & Cogan, N. (2003). Behaviour support in secondary schools: what works for schools? Emotional and Behavioural Difficulties, 8(1), 33–42. doi:10.1080/13632750300507004 [Google Scholar]

[cl2014001034-bib-000280] Heilbrun, A. , Cornell, D. , & Lovegrove, P. (2015). Principal attitudes regarding zero tolerance and racial disparities in school suspensions. Psychology in the Schools, 52(5), 489–499. doi:10.1002/pits.21838 [Google Scholar]

[cl2014001034-bib-000281] Hemphill, S. A. , Kotevski, A. , Herrenkohl, T. I. , Smith, R. , Toumbourou, J. W. , & Catalano, R. F. (2013). Does school suspension affect subsequent youth non‐violent antisocial behaviour? A longitudinal study of students in Victoria, Australia and Washington State, United States. Australian Journal of Psychology, 65(4), 236–249. doi:10.1111/ajpy.12026 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000282] Hemphill, S. A. , McMorris, B. J. , Toumbourou, J. W. , Herrenkohl, T. I. , Catalano, R.F. , & Mathers, M. (2007). Rates of student‐reported antisocial behavior, school suspensions, and arrests in Victoria, Australia and Washington state, United States*. The Journal of School Health, 77(6), 303–311. doi:10.1111/j.1746‐1561.2007.00211.x [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000283] Hemphill, S. A. , Plenty, S. M. , Herrenkohl, T. L. , Toumbourou, J. W. , & Catalano, R. F. (2014). Student and school factors associated with school suspension: A multilevel analysis of students in Victoria, Australia and Washington State, United States. Children and Youth Services Review, 36, 187–194. doi:10.1016/j.childyouth.2013.11.022 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000284] Hemphill, S. A. , Toumbourou, J. W. , Smith, R. , Kendall, G. E. , Rowland, B. , Freiberg, K. , & Williams, J. W. (2010). Are rates of school suspension higher in socially disadvantaged neighbourhoods? An Australian study. Health Promotion Journal of Australia, 21(1), 12–18. Retrieved from <Go to ISI>://WOS:000289221300004 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000285] Henderson, D. X. , & Green, J. (2014). Using mixed methods to explore resilience, social connectedness, and re‐suspension among youth in a community‐based alternative‐to‐suspension Program. International Journal of Child, Youth and Family Studies, 5(3), 423–446. [Google Scholar]

[cl2014001034-bib-000286] Herrenkohl, T. I. , Hemphill, S. A. , Mason, W. A. , Toumbourou, J. W. , & Catalano, R. F. (2012). Predictors and responses to the growth in physical violence during adolescence: A comparison of students in Washington State and Victoria, Australia. American Journal of Orthopsychiatry, 82(1), 41–49. doi:10.1111/j.1939‐0025.2011.01139.x [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000287] Hilberth, M. , & Slate, J. R. (2014). Middle school black and white student assignment to disciplinary consequences. A clear lack of equity. Education and Urban Society, 46(3), 312–328. doi:10.1177/0013124512446218 [Google Scholar]

[cl2014001034-bib-000288] Hill, D. , & Brown, D. (2013). Supporting inclusion of at risk students in secondary school through positive behaviour support. International Journal of Inclusive Education, 17(8), 868–881. doi:10.1080/13603116.2011.602525 [Google Scholar]

[cl2014001034-bib-000289] Hockman, S. , & Worner, W. (1987). In‐school suspension and group counseling: Helping the at‐risk student. NASSP Bulletin, 71(501), 93–96. doi:10.1177/019263658707150124 [Google Scholar]

[cl2014001034-bib-000290] Holmlund, H. , & Silva, O. (2009). Targeting non‐cognitive skills to improve cognitive outcomes: Evidence from a remedial education intervention. IZA Discussion Paper, No. 4476. doi:10.1086/676460

[cl2014001034-bib-000291] Horigian, V. E. , Robbins, M. S. , Dominguez, R. , Ucha, J. , & Rosa, C. L. (2010). Principles for defining adverse events in behavioral intervention research: Lessons from a family‐focused adolescent drug abuse trial. Clinical Trials, 7(1), 58–68. doi:10.1177/1740774509356575 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000292] Hudley, C. , & Friday, J. (1996). Attributional bias and reactive aggression. American Journal of Preventive Medicine, 12(5), 75–81. Retrieved from <Go to ISI>://WOS:A1996VR49100011 [PubMed] [Google Scholar]

[cl2014001034-bib-000293] Huff, J. A. (1988). Personalized behavior modification: An in‐school suspension program that teaches students how to change. The School Counselor, 35(3), 210–214. doi:http://www.jstor.org/stable/23900901 [Google Scholar]

[cl2014001034-bib-000294] Humphrey, N. , & Brooks, A. G. (2006). An evaluation of a short cognitive‐behavioural anger management intervention for pupils at risk of exclusion. Emotional & Behavioural Difficulties, 11(1), 5–23. doi:10.1080/13632750500392856 [Google Scholar]

[cl2014001034-bib-000295] James‐Burdumy, S. , Dynarski, M. , & Deke, J. (2008). After‐school program effects on behavior: Results from the 21st century community learning centers program national evaluation. Economic Inquiry, 46(1), 13–18. doi:10.1111/j.1465‐7295.2007.00074.x [Google Scholar]

[cl2014001034-bib-000296] Jennings, W. G. , Gover, A. R. , & Hitchcock, D. M. (2008). Localizing restorative justice: An in‐depth look at a Denver public school program. Restorative Justice: From Theory to Practice, 11, 167–187. doi:10.1016/s1521‐6136(08)00407‐7 [Google Scholar]

[cl2014001034-bib-000297] Johnson, D. W. , & Johnson, R. T. (1996). Conflict resolution and peer mediation programs in elementary and secondary schools: A review of the research. Review of Educational Research, 66(4), 459–506. doi: 10.3102/00346543066004459 [DOI] [Google Scholar]

[cl2014001034-bib-000298] Johnston, J. (2001). Evaluating national initiatives: The case of ‘On Track’. Children & Society, 15(1), 33–36. doi:10.1111/j.1099‐0860.2001.tb00201.x [Google Scholar]

[cl2014001034-bib-000299] Jolivette, K. , Patterson, D. P. , Swoszowski, N. C. , McDaniel, S. C. , Kennedy, C. , & Ennis, R. P. (2014). School‐wide positive behavioral interventions and supports in a residential school for students with emotional and behavioral disorders: First years of implementation and maintenance follow‐up focus groups. Residential Treatment for Children & Youth, 31(1), 63–79. doi:10.1080/0886571X.2014.878584 [Google Scholar]

[cl2014001034-bib-000300] Jones, J. , & Smith, C. (2004). Reducing exclusions whilst developing effective intervention and inclusion for pupils with behaviour difficulties. Emotional and Behavioural Difficulties, 9(2), 115–129. doi:10.1177/1363275204045732 [Google Scholar]

[cl2014001034-bib-000301] Joo, M. (2010). Long‐term effects of Head Start on academic and school outcomes of children in persistent poverty: Girls vs. boys. Children and Youth Services Review, 32(6), 807–814. doi:10.1016/j.childyouth.2010.01.018 [Google Scholar]

[cl2014001034-bib-000302] Kang‐Yi, C. D. , Mandell, D. S. , & Hadley, T. (2013). School‐based mental health program evaluation: Children's school outcomes and acute mental health service use. Journal of School Health, 83(7), 463–472. doi:10.1111/josh.12053 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000303] Kanu, M. , Hepler, N. , & Labi, H. (2015). Assessing effectiveness of students taking a right stand (STARS) Nashville Student Assistance Programs. Journal of Child & Adolescent Substance Abuse, 24(6), 323–330. doi:10.1080/1067828X.2013.829015 [Google Scholar]

[cl2014001034-bib-000304] Kearney, C. A. , & Graczyk, P. (2014). A response to intervention model to promote school attendance and decrease school absenteeism. Child & Youth Care Forum, 43(1), 1–25. doi:10.1007/s10566‐013‐9222‐1 [Google Scholar]

[cl2014001034-bib-000305] Kennedy, C. , & Jolivette, K. (2008). The effects of positive verbal reinforcement on the time spent outside the classroom for students with emotional and behavioral disorders in a residential setting. Behavioral Disorders, 33(4), 211–221. [Google Scholar]

[cl2014001034-bib-000306] Kilian, J. M. , Fish, M. C. , & Maniago, E. B. (2006). Making schools safe: A system‐wide school intervention to increase student prosocial behaviors and enhance school climate. Journal of Applied School Psychology, 23(1), 1–30. doi:10.1300/J370v23n01_01 [Google Scholar]

[cl2014001034-bib-000307] Koffman, S. , Ray, A. , Berg, S. , Covington, L. , Albarran, N. M. , & Vasquez, M. (2009). Impact of a comprehensive whole child intervention and prevention program among youths at risk of gang involvement and other forms of delinquency. Children & Schools, 31(4), 239–245. doi:10.1093/cs/31.4.239 [Google Scholar]

[cl2014001034-bib-000308] Kremer, K. P. , Maynard, B. R. , Polanin, J. R. , Vaughn, M. G. , & Sarteschi, C. M. (2015). Effects of after‐school programs with at‐risk youth on attendance and externalizing behaviors: A systematic review and meta‐analysis. Journal of Youth & Adolescence, 44, 616–636. doi:10.1007/s10964‐014‐0226‐4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000309] Kutash, K. , Duchnowski, A. J. , Green, A. L. , & Ferron, J. (2013). Effectiveness of the parent connectors program: Results from a randomized controlled trial. School Mental Health, 5(4), 192–208. doi:10.1007/s12310‐013‐9106‐4 [Google Scholar]

[cl2014001034-bib-000310] Kutash, K. , Duchnowski, A. J. , & Lynn, N. (2009). The use of evidence‐based instructional strategies in special education settings in secondary schools: Development, implementation and outcomes. Teaching and Teacher Education, 25(6), 917–923. doi:10.1016/j.tate.2009.02.016 [Google Scholar]

[cl2014001034-bib-000311] Lassen, S. R. , Steele, M. M. , & Sailor, W. (2006). The relationship of school‐wide positive behavior support to academic achievement in an urban middle school. Psychology in the Schools, 43(6), 701–712. doi:10.1002/pits.20177 [Google Scholar]

[cl2014001034-bib-000312] Law, J. , & Sivyer, S. (2003). Promoting the communication skills of primary school children excluded from school or at risk of exclusion: An intervention study. Child Language Teaching and Therapy, 19(1), 1–25. doi: 10.1191/0265659003ct241oa [DOI] [Google Scholar]

[cl2014001034-bib-000313] Leff, S. S. (2015. On going). Determining the efficacy of a relational aggression intervention for urban African American girls. Retrieved from https://clinicaltrials.gov/ct2/show/NCT00510094?term=school+suspension&age=0&rank=23

[cl2014001034-bib-000314] Leventhal, T. , & Brooks‐Gunn, J. (2004). A randomized study of neighborhood effects on low‐income children's educational outcomes. Developmental Psychology, 40(4), 488–507. doi:10.1037/0012‐1649.40.4.488 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000315] Lovey, J. , & Cooper, P. (1997). Positive alternatives to school exclusion: An empirical investigation. Emotional and Behavioural Difficulties, 2(3), 12–22. doi:10.1080/1363275970020303 [Google Scholar]

[cl2014001034-bib-000316] Luiselli, J. K. , Putnam, R. F. , Handler, M. W. , & Feinberg, A. B. (2005). Whole‐school positive behaviour support: Effects on student discipline problems and academic performance. Educational Psychology, 25(2/3), 183–198. [Google Scholar]

[cl2014001034-bib-000317] Lupton, C. , & Sheppard, C. (1999). Lost lessons?: The experience of a time‐limited Home‐School support project. Children & Society, 13(1), 20–31. doi:10.1111/j.1099‐0860.1999.tb00098.x [Google Scholar]

[cl2014001034-bib-000318] Maag, J. W. (2012). School‐wide discipline and the intransigency of exclusion. Children and Youth Services Review, 34(10), 2094–2100. doi:10.1016/j.childyouth.2012.07.005 [Google Scholar]

[cl2014001034-bib-000319] Macpherson, R. (2013). Evaluating three school‐based integrated health centres established by a partnership in cornwall to inform future provision and practice. International Journal of Educational Management, 27(5), 1–41. Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+evaluation&pg=3&id=EJ1014226 [Google Scholar]

[cl2014001034-bib-000320] Maguire, M. , Macrae, S. , & Milbourne, L. (2003). Early interventions: Preventing school exclusions in the primary setting. Westminster studies in education, 26(1), 45–62. doi:10.1080/0140672030260105 [Google Scholar]

[cl2014001034-bib-000321] Maimon, D. , Antonaccio, O. , & French, M. T. (2012). Severe sanctions, easy choice? Investigating the role of school sanctions in preventing adolescent violent offending. Criminology, 50(2), 495–524. doi: 10.1111/j.1745-9125.2011.00268.x [DOI] [Google Scholar]

[cl2014001034-bib-000322] Maras, P. , Bradshaw, V. , & Croft, C. (2008). Psychology and interventions for pupils at risk of underachievement and school exclusion: The year 10 effect. International Journal of Psychology, 43(3‐4), 379–379. Retrieved from <Go to ISI>://WOS:000259264304490 [Google Scholar]

[cl2014001034-bib-000323] Martin, D. , Martin, M. , Gibson, S. S. , & Wilkins, J. (2007). Increasing prosocial behavior and academic achievement among adolescent African American males. Adolescence, 42(168), 689–698. Retrieved from <Go to ISI>://WOS:000251753900005 [PubMed] [Google Scholar]

[cl2014001034-bib-000324] Martinez Martinez, K. I. , Salazar Garza, M. L. , Pedroza Cabrera, F. J. , Ruiz Torres, G. M. , & Ayala Velazquez, H. E. (2008). Preliminary study of a brief intervention program for adolescents who initiate alcohol and other drugs consumption. Salud Mental, 31(2), 119–127. Retrieved from <Go to ISI>://WOS:000256273500006 [Google Scholar]

[cl2014001034-bib-000325] Massey, O. T. , Boroughs, M. , & Armstrong, K. H. (2007). School violence interventions in the safe schools/healthy students initiative: Evaluation of two early intervention programs. Journal of School Violence, 6(2), 57–74. doi:10.1300/J202v06n02_04 [Google Scholar]

[cl2014001034-bib-000326] Matthews, A. , & Swan, W. W. (1999). TOPS: An alternative approach for the at‐risk middle school student. Journal of At‐Risk Issues, 6(1), 24–31. [Google Scholar]

[cl2014001034-bib-000327] Mattison, R. E. , & Schneider, J. (2009). First‐year effectiveness on school functioning of a self‐contained ED middle school. Behavioral Disorders, 34(2), 60–71. Retrieved from <Go to ISI>://WOS:000280896400001 [Google Scholar]

[cl2014001034-bib-000328] Mavroveli, S. , Petrides, K. V. , Shove, C. , & Whitehead, A. (2008). Investigation of the construct of trait emotional intelligence in children. European Child & Adolescent Psychiatry, 17(8), 516–526. doi:10.1007/s00787‐008‐0696‐6 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000329] May, D. C. , Stokes, E. , Oliver, A. , & McClure, T. (2015). Exploring alternatives to out‐of‐school suspension in Kentucky: A quasi‐experimental study examining the effectiveness of community service work. PRISM: A Journal of Regional Engagement, 4(1), 15–32. [Google Scholar]

[cl2014001034-bib-000330] McCord, M. T. , Klein, J. D. , Foy, J. M. , & Fothergill, K. (1993). School‐based clinic use and school performance. Journal of Adolescent Health, 14(2), 91–98. doi:10.1016/1054‐139x(93)90091‐3 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000331] McCrary, D. , Lechtenberger, D. , & Wang, E. (2012). The effect of schoolwide positive behavioral supports on children in impoverished rural community schools. Preventing School Failure, 56(1), 1–7. doi:10.1080/1045988X.2010.548417 [Google Scholar]

[cl2014001034-bib-000332] McDaniel, S. C. , & Bruhn, A. L. (2015). Using a changing‐criterion design to evaluate the effects of Check‐in/Check‐out with goal modification. Journal of Positive Behavior Interventions, 1–12. [Google Scholar]

[cl2014001034-bib-000333] McGarvey, E. L. , Leon‐Verdin, M. , Bloomfield, K. , Wood, S. , Winters, E. , & Smith, J. (2014). Effectiveness of A‐CRA/ACC in treating adolescents with cannabis‐use disorders. Community Mental Health Journal, 50(2), 150–157. doi:10.1007/s10597‐012‐9566‐2 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000334] Mears, C. J. , Lawler, E. N. , Sanders, L. D. , & Katz, B. Z. (2009). Efficacy of LAIV‐T on absentee rates in a school‐based health center sample. Journal of Adolescent Health, 45(1), 91–94. doi:10.1016/j.jadohealth.2008.12.010 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000335] Menendez, A. L. , Payne, L. D. , & Mayton, M. R. (2008). The Implementation of Positive Behavioral Support in an elementary school: Processes, procedures, and outcomes. The Alberta Journal of Educational Research, 54(4), 448–462. [Google Scholar]

[cl2014001034-bib-000336] Mitchell, R. R. , Tingstrom, D. H. , Dufrene, B. A. , Ford, W. B. , & Sterling, H. E. (2015). The effects of the Good Behavior Game with general‐education high school students. School Psychology Review, 44(2), 191–207. doi: 10.17105/spr-14-0063.1 [DOI] [Google Scholar]

[cl2014001034-bib-000337] Moore, A. J. (2014). Policy in practice: Enabling and inhibiting factors for the success of suspension centres. Australian Journal of Teacher Education, 39(11), 107–132. doi: 10.14221/ajte.2014v39n11.7 [DOI] [Google Scholar]

[cl2014001034-bib-000338] Moore, C. W. , & Allen, J. A. (1996). The effects of volunteering on the young volunteer. The Journal of Primary Prevention, 17(2), 231–258. doi:10.1007/BF02248794 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000339] Moraru, A. , Stoica, M. , Tomuletiu, E.‐A. , & Filpisan, M. (2011). Evaluation of a program for developing socio‐emotional competencies in preschool children. Procedia ‐ Social and Behavioral Sciences, 30, 2161–2164. doi:10.1016/j.sbspro.2011.10.419 [Google Scholar]

[cl2014001034-bib-000340] Munoz, M. A. , Fischetti, J. C. , & Prather, J. R. (2014). An early college initiative in an urban, high‐poverty high school: First‐year effects on student achievement and non‐academic indicators. Journal of Education for Students Placed at Risk (JESPAR), 19(1), 36–52. doi: 10.1080/10824669.2014.927746 [DOI] [Google Scholar]

[cl2014001034-bib-000341] Muschkin, C. G. , Glennie, E. , & Beck, A. N. (2014). Peer contexts: Do old for grade and retained peers influence student behavior in middle school? Teachers College Record, 116(4). [Google Scholar]

[cl2014001034-bib-000342] Muscott, H. S. , Mann, E. L. , & LeBrun, M. R. (2008). Positive Behavioral Interventions and Supports in New Hampshire Effects of Large‐Scale Implementation of Schoolwide Positive Behavior Support on Student Discipline and Academic Achievement. Journal of Positive Behavior Interventions, 10(3), 190–205. doi:10.1177/1098300708316258 [Google Scholar]

[cl2014001034-bib-000343] Muscott, H. S. , Szczesiul, S. , Berk, B. , Staub, K. , Hoover, J. , & Perry‐Chisholm, P. (2008). Creating home‐school partnerships by engaging families in schoolwide positive behavior supports. Teaching Exceptional Children, 40(6), 7–14. [Google Scholar]

[cl2014001034-bib-000344] Mytton, J. , DiGuiseppi, C. , Gough, D. , Taylor, R. , & Logan, S. (2006). School‐based secondary prevention programmes for preventing violence. Cochrane Database Syst Rev (3), Art. No.CD004606. doi:10.1002/14651858.CD004606.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000345] Navarro, J. I. , Aguilar, M. , Aguilar, C. , Alcalde, C. , & Marchena, E. (2007). Positive behavioral intervention in children who were wards of the court attending a mainstream school. Psychological Reports, 101(3), 1067–1078. doi:10.2466/pr0.101.4.1067‐1078 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000346] Nelson, J. R. , Hurley, K. D. , Synhorst, L. , Epstein, M. H. , Stage, S. , & Buckley, J. (2009). The child outcomes of a behavior model. Exceptional Children, 76(1), 7–30. Retrieved from http://digitalcommons.unl.edu/specedfacpub/57 [Google Scholar]

[cl2014001034-bib-000347] Nocera, E. J. , Whitbread, K. M. , & Nocera, G. P. (2014). Impact of School‐Wide Positive Behavior Supports on student behavior in the middle grades. RMLE Online: Research in Middle Level Education, 37(8), 1–14. [Google Scholar]

[cl2014001034-bib-000348] Noltemeyer, A. L. , Ward, R. M. , & McLoughlin, C. (2015). Relationship between school suspension and student outcomes: A meta‐analysis. School Psychology Review, 44(2), 224–240. doi:10.17105/spr‐14‐0008.1 [Google Scholar]

[cl2014001034-bib-000349] Obsuth, I. , Sutherland, A. , Pilbeam, L. , Scott, S. , Valdebenito, S. , Carr, R. , & Eisner, M. (2014). London Education and Inclusion Project (LEIP): A cluster‐randomised controlled trial protocol of an intervention to reduce antisocial behaviour and improve educational/occupational attainment for pupils at risk of school exclusion. BMC Psychology, 2(1), 24. Retrieved from http://www.biomedcentral.com/content/pdf/s40359‐014‐0024‐5.pdf [Google Scholar]

[cl2014001034-bib-000350] Olds, D. , Henderson, C. R. , Cole, R. , Eckenrode, J. , Kitzman, H. , Luckey, D. , … Powers, J. (1998). Long‐term effects of nurse home visitation on children's criminal and antisocial behavior. 15‐year follow‐up of a randomized controlled trial. Journal of the American Medical Association, 280(14), 1238–1244. doi:10.1001/jama.280.14.1238 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000351] Olweus, D. , & Limber, S. P. (2010). Bullying in school: Evaluation and dissemination of the Olweus Bullying Prevention program. American Journal of Orthopsychiatry, 80(1), 124–134. doi:10.1111/j.1939‐0025.2010.01015.x [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000352] Orchard, L. (2007). Evaluating parenting classes held at a secondary school. Research in Post‐Compulsory Education, 12(1), 91–105. doi:10.1080/13596740601155454 [Google Scholar]

[cl2014001034-bib-000353] Pack, C. , White, A. , Raczynski, K. , & Wang, A. (2011). Evaluation of the safe school ambassadors program: A student‐led approach to reducing mistreatment and bullying in schools. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 84(4), 127–133. doi:10.1080/00098655.2011.564974 [Google Scholar]

[cl2014001034-bib-000354] Panayiotopoulos, C. (2004). A follow‐up of a home and school support project for children with emotional and behavioural difficulties. Emotional and Behavioural Difficulties, 9(2), 85–98. doi:10.1177/1363275204045730 [Google Scholar]

[cl2014001034-bib-000355] Panayiotopoulos, C. , & Kerfoot, M. (2007). Early intervention and prevention for children excluded from primary schools. International Journal of Inclusive Education, 11(1), 59–80. doi:10.1080/13603110500392882 [Google Scholar]

[cl2014001034-bib-000356] Park, H. , Lin, C.‐H. , Liu, C. , & Tabb, K. M. (2015). The relationships between after‐school programs, academic outcomes, and behavioral developmental outcomes of Latino children from immigrant families: Findings from the 2005 National Household Education Surveys Program. Children and Youth Services Review, 53, 77–83. doi:10.1016/j.childyouth.2015.03.019 [Google Scholar]

[cl2014001034-bib-000357] Parkes, R. (2008). Persistent young offenders: An evaluation of two projects. British Journal of Social Work, 38(5), 1031–1033. doi: 10.1093/bjsw/bcn091 [DOI] [Google Scholar]

[cl2014001034-bib-000358] Pas, E. T. , Bradshaw, C. P. , Hershfeldt, P. A. , & Leaf, P. J. (2010). A multilevel exploration of the influence of teacher efficacy and burnout on response to student problem behavior and school‐based service use. School Psychology Quarterly, 25(1), 13–27. doi:10.1037/a0018576 [Google Scholar]

[cl2014001034-bib-000359] Pelham, W. E., Jr. , Massetti, G. M. , Wilson, T. , Kipp, H. , Myers, D. , Standley, B. B. N. , … Waschbusch, D. A. (2005). Implementation of a comprehensive schoolwide behavioral intervention: The ABC program. Journal of Attention Disorders, 9(1), 248–260. doi:10.1177/1087054705281596 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000360] Potter, L. , & Bulach, C. (1996). An alternative program to improve student behavior: The focus program. Thresholds in Education, November, 14–16. [Google Scholar]

[cl2014001034-bib-000361] Powell, L. , Gilchrist, M. , & Stapley, J. (2008). A journey of self‐discovery: An intervention involving massage, yoga and relaxation for children with emotional and behavioural difficulties attending primary schools. European Journal of Special Needs Education, 23(4), 403–412. doi:10.1080/08856250802387398 [Google Scholar]

[cl2014001034-bib-000362] Pritchard, C. , Cotton, A. , Bowen, D. , & Williams, R. (1998). A consumer view study of young people's views on their educational social worker: Engagement as a measure of an effective relationship. British Journal of Social Work, 28, 915–938. [Google Scholar]

[cl2014001034-bib-000363] Pritchard, C. , & Williams, R. (2001). A three‐year comparative longitudinal study of a school‐based social work family service to reduce truancy, delinquency and school exclusions. Journal of Social Welfare & Family Law, 23(1), 23–43. doi:10.1080/09649060010012255 [Google Scholar]

[cl2014001034-bib-000364] Pritchard, C. , & Williams, R. (2009). Does social work make a difference?. A controlled study of former ‘looked‐after‐children’ and ‘excluded‐from‐school’ adolescents now men aged 16‐24 subsequent offences, being victims of crime and suicide. Journal of Social Work, 93(3), 285–307. [Google Scholar]

[cl2014001034-bib-000365] Putnam, R. F. , Handler, M. W. , Ramirez‐Platt, C. M. , & Luiselli, J. K. (2003). Improving student bus‐riding behavior through a whole‐school intervention. Journal of Applied Behavior Analysis, 36(4), 583–590. doi:10.1901/jaba.2003.36‐583 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000366] Rapp‐Paglicci, L. , Stewart, C. , & Rowe, W. (2011). Can a self‐regulation skills and cultural arts program promote positive outcomes in mental health symptoms and academic achievement for at‐risk youth? Journal of Social Service Research, 37(3), 309–319. doi:10.1080/01488376.2011.564067 [Google Scholar]

[cl2014001034-bib-000367] Rauch, S. P. , Brack, C. J. , & Orr, D. P. (1987). School‐based, short‐term group treatment for behaviorally disturbed young adolescent males. A pilot intervention. Journal of School Health, 57(1), 19–22. doi:10.1111/j.1746‐1561.1987.tb00515.x [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000368] Reglin, G. , Akpo‐Sanni, J. , & Losike‐Sedimo, N. (2012). The effect of a professional development classroom management model on at‐risk elementary students' misbehaviors. Education, 133(1), 3–18. [Google Scholar]

[cl2014001034-bib-000369] Reid, K. (2003). A strategic approach to tackling school absenteeism and truancy: The PSCC scheme. Educational Studies, 29(4), 351–371. doi:10.1080/0305569032000159660 [Google Scholar]

[cl2014001034-bib-000370] Reid, K. (2003). Strategic approaches to tackling school absenteeism and truancy: The traffic lights (TL) scheme. Educational Review, 55(3), 305–321. doi:10.1080/0013191032000118956 [Google Scholar]

[cl2014001034-bib-000371] Reynolds, C. R. , Skiba, R. J. , Graham, S. , Sheras, P. , Conoley, J. C. , & Garcia‐Vazquez, E. (2008). Are zero tolerance policies effective in the schools? An evidentiary review and recommendations. American Psychologist, 63(9), 852–862. doi:10.1037/0003‐066x.63.9.852 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000372] Rollin, S. A. , Kaiser‐Ulrey, C. , Potts, I. , & Creason, A. H. (2003). A school‐based violence prevention model for at‐risk eighth grade youth. Psychology in the Schools, 40(4), 403–416. doi:10.1002/pits.10111 [Google Scholar]

[cl2014001034-bib-000373] Rose, R. (2008). Encouraging engagement: an emerging role for school based family workers in english secondary schools. International Journal on School Disaffection, 6(1), 5–11. Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+evaluation&pg=4&id=EJ871300;http://files.eric.ed.gov/fulltext/EJ871300.pdf [Google Scholar]

[cl2014001034-bib-000374] Rosenbaum, J. (2012). Does school suspension increase long‐term substance use?: Quantifying the impact of school suspension on young adult substance use 12 years after school suspension using the Rubin causal inference model. Journal of Adolescent Health, 50(2), S75–S75. [Google Scholar]

[cl2014001034-bib-000375] Ross, S. W. , & Horner, R. H. (2009). Bully prevention in positive behavior support. Journal of Applied Behavior Analysis, 42(4), 747–759. doi:10.1901/jaba.2009.42‐747 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000376] Royer, E. , Bitaudeau, I. , & Verville, H. (1996). L'exclusion scolaire au secondaire: évaluation d'un programme alternatif à la suspension scolaire. Suspension from secondary school: Evaluation of an alternative program. Revue Canadienne de Psycho‐Education, 25(1), 25–40. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1996‐93831‐001&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000377] Rundle‐Thiele, S. , Schuster, L. , Dietrich, T. , Russell‐Bennett, R. , Drennan, J. , Leo, C. , & Connor, J. P. (2015). Maintaining or changing a drinking behavior? Goka's short‐term outcomes. Journal of Business Research, 68(10), 2155–2163. doi: 10.1016/j.jbusres.2015.03.015 [DOI] [Google Scholar]

[cl2014001034-bib-000378] Safer, D. J. , Heaton, R. C. , & Parker, F. C. (1981). A behavioral‐program for disruptive junior‐high school students: Results and follow‐up. Journal of Abnormal Child Psychology, 9(4), 483–494. doi:10.1007/bf00917797 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000379] Sandler, I. , Wolchik, S. A. , Cruden, G. , Mahrer, N. E. , Ahn, S. , Brincks, A. , & Brown, C. H. (2014). Overview of meta‐analyses of the prevention of mental health, substance use and conduct problems. Annual Review of Clinical Psychology, 10, 243. doi:10.1146/annurev‐clinpsy‐050212‐185524 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000380] Schellenberg, R. C. , Parks‐Savage, A. , & Rehfuss, M. (2007). Reducing levels of elementary school violence with peer mediation. Professional School Counseling, 10(5), 475–481. doi: 10.5330/prsc.10.5.q7866077l3v5q044 [DOI] [Google Scholar]

[cl2014001034-bib-000381] Schmidt, J. J. , & Biles, J. W. (1985). Puppetry as a group counseling technique with middle school students. Elementary School Guidance and Counseling, 20(1), 67–73. doi: http://www.jstor.org/stable/42868705 [Google Scholar]

[cl2014001034-bib-000382] Schnelling, K. , & Dew‐Hughes, D. (2002). A solution to exclusion: A total involvement model to support pupils and reduce exclusions. Emotional & Behavioural Difficulties, 7(4), 229–240. doi:10.1177/1363275202007004005 [Google Scholar]

[cl2014001034-bib-000383] Schwartz, S. E. O. , Rhodes, J. E. , Spencer, R. , & Grossman, J. B. (2013). Youth initiated mentoring: Investigating a new approach to working with vulnerable adolescents. American Journal of Community Psychology, 52(1‐2), 155–169. doi:10.1007/s10464‐013‐9585‐3 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000384] Scott, T. M. , & Barrett, S. B. (2004). Using staff and student time engaged in disciplinary procedures to evaluate the impact of school‐wide PBS. Journal of Positive Behavior Interventions, 6(1), 21–27. doi:10.1177/10983007040060010401 [Google Scholar]

[cl2014001034-bib-000385] Shapiro, J. P. , Burgoon, J. D. , Welker, C. J. , & Clough, J. B. (2002). Evaluation of the Peacemakers Program: School‐based violence prevention for students in grades four through eight. Psychology in the Schools, 39(1), 87–100. doi:10.1002/pits.10040 [Google Scholar]

[cl2014001034-bib-000386] Sharp, S. , & Davids, E. (2003). Early intervention in behaviour: A study of the FAST‐Track programme. Emotional and Behavioural Difficulties, 8(3), 173–188. doi:10.1080/13632750300507017 [Google Scholar]

[cl2014001034-bib-000387] Sheldon, S. B. , & Epstein, J. L. (2002). Improving student behavior and school discipline with family and community involvement. Education and Urban Society, 35(1), 4–26. doi:10.1177/001312402237212 [Google Scholar]

[cl2014001034-bib-000388] Simonsen, B. , Eber, L. , Black, A. C. , Sugai, G. , Lewandowski, H. , Sims, B. , & Myers, D. (2012). Illinois statewide Positive Behavioral Interventions and Supports: Evolution and impact on student outcomes across years. Journal of Positive Behavior Interventions, 14(1), 5–16. doi:10.1177/1098300711412601 [Google Scholar]

[cl2014001034-bib-000389] Skaggs, G. , & Bodenhorn, N. (2006). Relationships between implementing character education, student behavior, and student achievement. Journal of Advanced Academics, 18(1), 82–114. doi:10.4219/jaa‐2006‐345 [Google Scholar]

[cl2014001034-bib-000390] Skiba, R. , & Peterson, R. (2003). Teaching the social curriculum: School discipline as instruction. Preventing School Failure, 47(2), 66–73. doi:10.1080/10459880309604432 [Google Scholar]

[cl2014001034-bib-000391] Skiba, R. , Rausch, M. K. , & Ritter, S. (2004). “Discipline is always teaching”: Effective alternatives to zero tolerance in Indiana's schools. Education Policy Briefs., 2(3), 1–11. Retrieved from http://files.eric.ed.gov/fulltext/ED488903.pdf [Google Scholar]

[cl2014001034-bib-000392] Skiba, R. , Ritter, S. , Simmons, A. , Peterson, R. , & Miller, C. (2005). The safe and responsive schools project: A school reform model for implementing best practices in violence prevention. Handbook of school violence and school safety: From research to practice, 631–650.

[cl2014001034-bib-000393] Skiba, R. J. , Horner, R. H. , Chung, C.‐G. , Rausch, M. K. , May, S. L. , & Tobin, T. (2011). Race is not neutral: A national investigation of African American and Latino disproportionality in school discipline. School Psychology Review, 40(1), 85–107. [Google Scholar]

[cl2014001034-bib-000394] Smith, C. , Bicard, S. C. , Bicard, D. F. , & Casey, L. B. (2012). Decreasing in‐school suspensions with function‐based interventions. Kappa Delta Pi Record, 48(4), 174–177. doi:10.1080/00228958.2012.734013 [Google Scholar]

[cl2014001034-bib-000395] Smith, J. , Twemlow, S. W. , & Hoover, D. W. (1999). Bullies, victims and bystanders: A method of in‐school intervention and possible parental contributions. Child Psychiatry & Human Development, 30(1), 29–37. doi:10.1023/a:1018681819748 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000396] Spaulding, S. A. , Irvin, L. K. , Horner, R. H. , May, S. L. , Emeldi, M. , Tobin, T. J. , & Sugai, G. (2010). Schoolwide social‐behavioral climate, student problem behavior, and related administrative decisions. Journal of Positive Behavior Interventions, 12(2), 69–85. doi: 10.1177/1098300708329011 [DOI] [Google Scholar]

[cl2014001034-bib-000397] Spielman, D. A. , & Staub, E. (2000). Reducing boys' aggression: Learning to fulfill basic needs constructively. Journal of Applied Developmental Psychology, 21(2), 165–181. doi:10.1016/s0193‐3973(99)00034‐9 [Google Scholar]

[cl2014001034-bib-000398] Stader, D. L. (2000). Reducing violence in the high school: The power of peers. Journal of At‐Risk Issues, 7(1), 6–11. [Google Scholar]

[cl2014001034-bib-000399] Stage, S. A. (1997). A preliminary investigation of the relationship between in‐school suspension and the disruptive classroom behavior of students with behavioral disorders. Behavioral Disorders, 23(1), 57–76. Retrieved from http://www.jstor.org/stable/23888789 [Google Scholar]

[cl2014001034-bib-000400] Statham, J. (2004). Effective services to support children in special circumstances. Child Care, Health and Development, 30(6), 589–598. doi:10.1111/j.1365‐2214.2004.00472.x/pdf [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000401] Sterrett, E. , Jones, D. J. , Zalot, A. , & Shook, S. (2010). A pilot study of a brief motivational intervention to enhance parental engagement: A brief report. Journal of Child and Family Studies, 19(6), 697–701. doi:10.1007/s10826‐010‐9356‐9 [Google Scholar]

[cl2014001034-bib-000402] Stewart, C. , Rapp, L. , & Rowe, W. (2015). Determining the efficacy of a community‐based early intervention program adapted for middle school. J Child Adolesc Behav, 3(4), 1–6. doi:10.4172/2375‐4494.1000228 [Google Scholar]

[cl2014001034-bib-000403] Strompolis, M. , Vishnevsky, T. , Reeve, C. L. , Munsell, E. P. , Cook, J. R. , & Kilmer, R. P. (2012). Educational outcomes in a system of care for children with emotional disturbance. American Journal of Orthopsychiatry, 82(1), 129–136. doi:10.1111/j.1939‐0025.2011.01126.x [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000404] Sullivan, A. L. , Klingbeil, D. A. , & Van Norman, E. R. (2013). Beyond behavior: Multilevel analysis of the influence of sociodemographics and school characteristics on students' risk of suspension. School Psychology Review, 42(1), 99–114. [Google Scholar]

[cl2014001034-bib-000405] Sweeney‐Rader, J. , Snyder, G. L. , Goldstein, H. , & Rosenwald, P. (1980). School suspensions: An in‐house prevention model. Children today, 9(2), 19–21. [PubMed] [Google Scholar]

[cl2014001034-bib-000406] Tanner‐Smith, E. E. , & Wilson, S. J. (2013). A meta‐analysis of the effects of dropout prevention programs on school absenteeism. Prevention Science, 14, 468–478. doi:10.1007/s11121‐012‐0330‐1 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000407] Teasley, M. L. , & Miller, C. R. (2011). School social workers' perceived efficacy at tasks related to curbing suspension and undesirable behaviors. Children & Schools, 33(3), 136. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&id=EJ937659 [Google Scholar]

[cl2014001034-bib-000408] Teske, S. C. (2011). A study of zero tolerance policies in schools: A multi‐integrated systems approach to improve outcomes for adolescents. Journal of Child and Adolescent Psychiatric Nursing, 24(2), 88–97. doi:10.1111/j.1744‐6171.2011.00273.x [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000409] Theriot, M. T. (2009). School resource officers and the criminalization of student behavior. Journal of Criminal Justice, 37(3), 280–287. doi:10.1016/j.jcrimjus.2009.04.008 [Google Scholar]

[cl2014001034-bib-000410] Theriot, M. T. , Craun, S. W. , & Dupper, D. R. (2010). Multilevel evaluation of factors predicting school exclusion among middle and high school students. Children and Youth Services Review, 32(1), 13–19. doi:10.1016/j.childyouth.2009.06.009 [Google Scholar]

[cl2014001034-bib-000411] Thompson, S. J. , Pollio, D. E. , Constantine, J. , Reid, D. , & Nebbitt, V. (2002). Short‐term outcomes for youth receiving runaway and homeless shelter services. Research on Social Work Practice, 12(5), 589–603. doi:10.1177/1049731502012005001 [Google Scholar]

[cl2014001034-bib-000412] Thompson, S. M. (1996). Peer mediation: A peaceful solution. The School Counselor, 44(2), 151–154. Retrieved from http://www.jstor.org/stable/23903453 [Google Scholar]

[cl2014001034-bib-000413] Timmins, P. , Shepherd, D. , & Kelly, T. (2003). The research and development in organisations approach and the evaluation of a mainstream behaviour support initiative. Educational Psychology in Practice, 19(3), 229–242. Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+evaluation&pg=6&id=EJ770788 [Google Scholar]

[cl2014001034-bib-000414] Tischler, V. , Vostanis, P. , Bellerby, T. , & Cumella, S. (2002). Evaluation of a mental health outreach service for homeless families. Archives of Disease in Childhood, 86(3), 158–163. doi:https://doi.org/10.1136%2Fadc.86.3.158 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000415] Trulson, C. , Triplett, R. , & Snell, C. (2001). Social control in a school setting: evaluating a school‐based boot camp. Crime & Delinquency, 47(4), 573–609. doi:10.1177/0011128701047004005 [Google Scholar]

[cl2014001034-bib-000416] Turner, E. , & Waterhouse, S. (2003). Towards inclusive schools. Sustaining normal in‐school careers: an alternative to pupil exclusions. Emotional and Behavioural Difficulties, 8(1), 19. doi:10.1080/13632750300507003 [Google Scholar]

[cl2014001034-bib-000417] Twemlow, S. W. , Biggs, B. K. , Nelson, T. D. , Vernberg, E. M. , Fonagy, P. , & Twemlow, S. W. (2008). Effects of participation in a martial arts‐based antibullying program in elementary schools. Psychology in the Schools, 45(10), 947–959. doi:10.1002/pits.20344 [Google Scholar]

[cl2014001034-bib-000418] Vandever, T. R. (1983). Comparison of suspensions of EMR and non retarded students. American Journal of Mental Deficiency, 87(4), 430–434. [PubMed] [Google Scholar]

[cl2014001034-bib-000419] Veliz, P. , & Shakib, S. (2012). Interscholastic sports participation and school based delinquency: Does participation in sport foster a positive high school environment? Sociological Spectrum, 32(6), 558–580. doi:10.1080/02732173.2012.700837 [Google Scholar]

[cl2014001034-bib-000420] Vincent, C. G. , & Tobin, T. J. (2011). The relationship between implementation of School‐Wide Positive Behavior Support (SWPBS) and disciplinary exclusion of students from various ethnic backgrounds with and without disabilities. Journal of Emotional and Behavioral Disorders, 19(4), 217–232. doi:10.1177/1063426610377329 [Google Scholar]

[cl2014001034-bib-000421] Vincus, A. A. , Ringwalt, C. , Harris, M. S. , & Shamblen, S. R. (2010). A short‐term, quasi‐experimental evaluation of D.A.R.E's revised elementary school curriculum. Journal of Drug Education, 40(1), 37–49. doi:10.2190/DE.40.1.c [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000422] Vulliamy, G. , & Webb, R. (2003). Reducing school exclusions: An evaluation of a multi‐site development project. Oxford Review of Education, 29(1), 33–49. Retrieved from http://www.jstor.org/stable/1050728 [Google Scholar]

[cl2014001034-bib-000423] Waters, T. (2014). Story links: Working with parents of pupils at risk of exclusion. Support for Learning, 29(4), 298–318. doi:10.1111/1467‐9604.12072 [Google Scholar]

[cl2014001034-bib-000424] Webster‐Stratton, C. , Rinaldi, J. , & Reid, J. M. (2011). Long‐term outcomes of incredible years parenting program: Predictors of adolescent adjustment. Child and Adolescent Mental Health, 16(1), 38–46. doi:10.1111/j.1475‐3588.2010.00576.x. [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000425] Wheatley, A. , Murrihy, R. , van Kessel, J. , Wuthrich, V. , Remond, L. , Tuqiri, R. , … Kidman, A. (2009). Aggression management training for youth in behaviour schools: A quasi‐experimental study. Youth Studies Australia, 28(1), 29–36. Retrieved from http://search.informit.com.au/documentSummary;dn=639719523230811;res=IELAPA [Google Scholar]

[cl2014001034-bib-000426] Wiggins, M. , Bonell, C. , Sawtell, M. , Austerberry, H. , Burchett, H. , Allen, E. , & Strange, V. (2009). Health outcomes of youth development programme in england: Prospective matched comparison study. British Medical Journal, 339, 1–8. doi:10.1136/bmj.b2534 [DOI] [PMC free article] [PubMed] [Google Scholar]

[cl2014001034-bib-000427] Williams, E. , Johnson, J. L. , & Bott, C. A. (2008). Evaluation of a program for reduction of childhood aggression. Psychological Reports, 103(2), 347–357. doi:10.2466/pr0.103.2.347‐357 [DOI] [PubMed] [Google Scholar]

[cl2014001034-bib-000428] Williams, R. , & Pritchard, C. (2016). An analysis of the psychosocial backgrounds of youths (13‐18) who‐pose‐sexual‐ risk to children. Journal of Social Work, 0(0), 1–19. doi:10.1177/1468017316651992 [Google Scholar]

[cl2014001034-bib-000429] Winborne, C. R. (1980). In‐school suspension programs: The King William county model. Educational Leadership, 37(6), 466–469. Retrieved from <Go to ISI>://WOS:A1980JJ88900004 [Google Scholar]

[cl2014001034-bib-000430] Wright, McMahon, P. M. , Daly, C. , & Haney, J. P. (2012). Getting the law involved a quasi‐experiment in early intervention involving collaboration between schools and the district attorney's office. Criminology & Public Policy, 11(2), 227–249. doi:10.1111/j.1745‐9133.2012.00803.x [Google Scholar]

[cl2014001034-bib-000431] Wright, R. , Offord, D. , John, L. , & Duku, E. (2005). Secondary schools demonstration project: program effects of school‐based interventions on antisocial behaviour. Exceptionality Education Canada, 15(2), 27–50. [Google Scholar]

[cl2014001034-bib-000432] Ya‐yu, L. , Loe, S. A. , & Cartledge, G. (2002). The effects of social skills instruction on the social behaviors of students at risk for emotional or behavioral disorders. Behavioral Disorders, 27(4), 371–385. [Google Scholar]

[cl2014001034-bib-000433] Yokley, J. (2010). Social responsibility therapy for harmful, abusive behavior. Journal of Contemporary Psychotherapy, 40(2), 105–113. doi:10.1007/s10879‐009‐9131‐7 [Google Scholar]

[cl2014001034-bib-000434] Zivin, G. , Hassan, N. R. , DePaula, G. F. , Monti, D. A. , Harlan, C. , Hossain, K. D. , & Patterson, K. (2001). An effective approach to violence prevention: Traditional martial arts in middle school. Adolescence, 36(143), 443–459. Retrieved from <Go to ISI>://WOS:000173402600004 [PubMed] [Google Scholar]

[cl2014001034-bib-000435] Ackerman, C. M. , Cooksy, L. J. , Murphy, A. , Rubright, J. , Bear, G. , & Fifield, S. (2010). Positive behavior support in Delaware schools: Developing perspectives on implementation and outcomes. (Technical Report No. T2010.3). Newark, DE. University of Delaware: Education Research & Development Center. [Google Scholar]

[cl2014001034-bib-000436] Allen, J. P. , & Philliber, S. (1991). Evaluating why and how the teen outreach program works: years 3‐5 of the teen outreach national replication (1986/87‐1988/89). New York: Association of Junior Leagues. [Google Scholar]

[cl2014001034-bib-000437] Armour, M. (2014). Ed White middle school restorative discipline evaluation: Implementation and impact, 2013/2014. Sixth & seventh grade. Retrieved from www.utexas.edu [Google Scholar]

[cl2014001034-bib-000438] Baker, M. L. (2009). DPS restorative justice project: Year Three. Retrieved from Outcomes, Inc: Denver. [Google Scholar]

[cl2014001034-bib-000439] Burke, A. (2015). Suspension, expulsion, and achievement of english learner students in six Oregon districts. (REL 2015‐094). Regional Educational Laboratory Northwest. US: Department of Education. [Google Scholar]

[cl2014001034-bib-000440] Clarke, A. , Wydall, S. , & Williams, K. (2008). An evaluation of the introduction of a youth inclusion programme in rural Wales (Final Report). Wales: Department of Law and Criminology Aberystwyth University. [Google Scholar]

[cl2014001034-bib-000441] Commission of the European Communities, B. B. D.‐G. f. E. a. C . (2001). Second chance schools: The results of a European pilot project (Report). Brussels: Directorate‐General for Education & Culture. [Google Scholar]

[cl2014001034-bib-000442] Cullen, M. , Johnstone, M. , Lloyd, G. , & Munn, P. (1996). Exclusion from school and alternatives. Report on phase 3: The case study. [Google Scholar]

[cl2014001034-bib-000443] Davis, A. J. , & Pfeifer, H. (2014). How has the Baltimore county public school system addressed disproportionate minority suspensions?. Schaefer Center for Public Policy University of Baltimore: College of Public Affairs. [Google Scholar]

[cl2014001034-bib-000444] Dynarski, M. , James‐Burdumy, S. , Moore, M. , Rosenberg, L. , Deke, J. , Mansfield, W. , & Warner, E. (2004). When schools stay open late: the national evaluation of the 21st century community learning centers program. New findings. Retrieved from U.S. Department of Education, National Center for Education Evaluation and Regional Assistance. Washington, DC: U.S. Government Printing Office, 2004. [Google Scholar]

[cl2014001034-bib-000445] Durbin, K. (1994). Project success: A school‐based alternative to expulsion. Lancaster County Schol District. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED379542&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000446] Elliot, W. L. (1991). RE‐FOCUS program. redefine efforts: Focus on change under supervision. (a support program for students and teachers). Kansas City, MO. Hickman Mills Consolidated School District. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED337830&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000447] Eric Clearinghouse on Educational Management . (1986). Preventing dropouts. The best of ERIC on educational management (No.83). Office of Educational Research and Improvement. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+experiment&pg=2&id=ED268666;http://files.eric.ed.gov/fulltext/ED268666.pdf [Google Scholar]

[cl2014001034-bib-000448] Flay, B. , Acock, A. , Vuchinich, S. , & Beets, M. (2006). Progress report of the randomized triall of Positive Action in Hawaii's end of third year of intervention (spring 2005). Oregon: Oregon State University. [Google Scholar]

[cl2014001034-bib-000449] Foley, E. M. , Klinge, A. , & Reisner, E. R. (2007). Evaluation of new century high schools: Profile of an initiative to create and sustain small, successful high schools (final report). Policy Studies Associates, Inc. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=4&id=ED498781;http://files.eric.ed.gov/fulltext/ED498781.pdf [Google Scholar]

[cl2014001034-bib-000450] Fronius, T. , Persson, H. , Guckenburg, S. , Hurley, N. , & Petrosino, A. (2016). Restorative justice in us schools: A research review. Retrieved from West Ed. Justice & Prevention Research Center.: http://jprc.wested.org/wp‐content/uploads/2016/02/RJ_Literature‐Review_20160217.pdf [Google Scholar]

[cl2014001034-bib-000451] Gao, J. , Hallar, B. , & Hartmann, T. A. (2014). A snapshot of ost programs in philadelphia: An evaluation of eleven 21st century community learning center grantees. Prepared by Research For Action. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=3&id=ED553131;http://files.eric.ed.gov/fulltext/ED553131.pdf [Google Scholar]

[cl2014001034-bib-000452] Grice, M. (1986). Positive Alternatives to School Suspension (P.A.S.S.), 1985‐86. Portland Public Schools: Evaluation Report. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED276794&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000453] Hallam, S. , Castle, F. , & Rogers, L. (2005). Research and evaluation of the behaviour improvement programme. Retrieved from http://oro.open.ac.uk/4565/

[cl2014001034-bib-000454] Harlow, K. , & Baenen, N. (2002). NovaNET, 2000‐2001: Analyses of Student Outcomes Relative to a Comparison Group. Eye on Evaluation. E&R Report. Retrieved from Wake County Public School System, Raleigh, NC: http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED466488&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000455] Hartmann, T. A. , Gao, J. , Kumar, A. , & Edmunds, K. (2013). A snapshot of OST programming in Philadelphia: An evaluation of six 21st Century Community Learning Center grantees. Research For Action. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=3&id=ED553147;http://files.eric.ed.gov/fulltext/ED553147.pdf [Google Scholar]

[cl2014001034-bib-000456] Heller, S. , Pollack, H. A. , Ander, R. , & Ludwig, J. (2013). Preventing youth violence and dropout: A randomized field experiment. Massachusetts: National Bureau of Economic Research. [Google Scholar]

[cl2014001034-bib-000457] Horner, R. , & Sugai, G. (2007). Is school‐wide positive behavior support an evidence‐based practice. Retrieved from www.pbis.org/common/cms/files/pbisresources/EvidenceBaseSWPBS08_04_08.doc

[cl2014001034-bib-000458] Houston Independent School District . (1997). Project SAFE alternative school, 1996‐97. Research report on an educational program. Texas:Department of Research and Accountability. Retrieved from http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=5&id=ED428166;http://files.eric.ed.gov/fulltext/ED428166.pdf [Google Scholar]

[cl2014001034-bib-000459] James‐Burdumy, S. , Dynarski, M. , Moore, M. , Deke, J. , & Mansfield, W. (2005). When schools stay open late: The national evaluation of the 21st Century Community Learning Centers program. Final report. Retrieved from U.S. Department of Education; [Google Scholar]

[cl2014001034-bib-000460] Kendall, S. , Kinder, K. , Halsey, K. , Fletcher‐Morgan, C. , White, R. , & Brown, C. (2003). An evaluation of alternative education initiatives (Research Report RR403). National Foundation for Educational Research. Retrieved from http://www.opengrey.eu/item/display/10068/421869 [Google Scholar]

[cl2014001034-bib-000461] Laliberté, D. , Rosario, G. , Léonard, L. , Smith‐Moncrieffe, D. , & Warner, A. (2014). Results of crime prevention programs for 12 to 17 year olds (Research Report: 2015‐R006). Ottawa, Ontario Canada: National Crime Prevention Centre (NCPC) Public Safety Canada; Retrieved from https://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/rslts‐crm‐prvntn‐12‐17/index‐en.aspx [Google Scholar]

[cl2014001034-bib-000462] Leonard, C. M. (1984). Evaluation Report on “PASS.” Positive Alternatives to Student Suspensions. Portland Public Schools: Research & Evaluation Department. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED256772&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000463] Linker, J. , & Marion, B. (1995). A study of the effects of options, an alternative educational program, on the personal success of at‐risk students: A five year study. Alleghany County School District NC. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED398338&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000464] McAleavy, G. , & O'Hagan, C. (n/d) An evaluation of the impact of the xL intervention in schools in northern ireland. Northern Ireland: The Prince's Trust. Retrieved from https://www.researchgate.net/profile/Celia_OHagan/publication/237343072_An_Evaluation_of_The_Impact_of_the_xl_Intervention_in_Schools_in_Northern_Ireland/links/0046352f7d3fb49433000000.pdf [Google Scholar]

[cl2014001034-bib-000465] Mearns, C. (1999). Brief family counseling intervention: Cibola cluster, 1997‐98, reserach, development and accountability. Albuquerque Public Schools. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED448374&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000466] Munn, P. , Cullen, M. , Johnstone, M. , & Lloyd, G. (1997). Exclusions and in‐school alternatives. The Scottish Office Education and Industry Department: Research and Intelligence Unit. Retrieved from http://files.eric.ed.gov/fulltext/ED414616.pdf [Google Scholar]

[cl2014001034-bib-000467] Norton, M. , & Piccinino, K. (2014). Project LIFT, year two report. Philadelphia: Research for Action. Retrieved from http://8rri53pm0cs22jk3vvqna1ub‐wpengine.netdna‐ssl.com/wp‐content/uploads/2015/10/RFA_Project_LIFT_Year_Two_Report_October_2014.pdf [Google Scholar]

[cl2014001034-bib-000468] Opuni, K. A. , Tullis, R. , Sanchez, K. , & Gonzalez, J. (1991). Student assignment centers: An in‐school suspension program, 1990‐91. Houston TX: Department of Researh and Education. Retrieved from http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=6&id=ED339137;http://files.eric.ed.gov/fulltext/ED339137.pdf [Google Scholar]

[cl2014001034-bib-000469] Peterson, P. E. , Myers, D. , & Howell, W. G. (1998). An evaluation of the new york city school choice scholarships program: The first year. New York: Mathematica Policy Research, Washington, DC. Retrieved from http://hks.harvard.edu/pepg/PDF/Papers/ny1ex.pdf [Google Scholar]

[cl2014001034-bib-000470] Pritchard, C. (2001). A family‐teacher‐social work alliance to reduce truancy and delinquency: The Dorset healthy alliance project (RDS occasional paper no. 78). England:Home Office, Communication Development Unit, Research, Development and Statistics Directorate. Retrieved from http://webarchive.nationalarchives.gov.uk/20110218135832/http:/rds.homeoffice.gov.uk/rds/pdfs/occ78‐dha.pdf [Google Scholar]

[cl2014001034-bib-000471] Puma, M. , Bell, S. , Shapiro, G. , Broene, P. , Cook, R. , Friedman, J. , & Heid, C. (2001). Building futures: the head start impact study. Research design plan. Washington DC: Administration for Children, Youth, and Families; Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+experiment&id=ED478789;http://files.eric.ed.gov/fulltext/ED478789.pdf [Google Scholar]

[cl2014001034-bib-000472] Rhea, A. (2010). An evaluation of the Wake county public school system alternative educational options (E&R Report No. 10.15). Wake County, Public School System. Retrieved from http://www.wcpss.net/evaluation‐research [Google Scholar]

[cl2014001034-bib-000473] Risner, M. , Abati, G. , Vanderhaar, J. , & Muñoz, M. A. (2005). Program evaluation. Positive Outreach Program (POP). Jefferson County Public Schools: Shaping the Future. Retrieved from http://www.jefferson.k12.ky.us/departments/Planning/ProgramEvaluation/FullReports_3_09/POPpdf.pdf [Google Scholar]

[cl2014001034-bib-000474] Rogers, J. W. (1972). A report of the team‐teaching, team‐governance program at rockland junior high. Distributed by ERIC Clearinghouse. Retrieved from http://eric.ed.gov/?id=ED083123 [Google Scholar]

[cl2014001034-bib-000475] Schoeneberger, J. (2008). The effect of Positive Behavioral Intervention Support (PBIS). Charlotte‐ Mecklenburg Schools: Center for Research & Evaluation. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.563.5527&rep=rep1&type=pdf [Google Scholar]

[cl2014001034-bib-000476] Smith, L. , Jackson, L. , & Comber, N. (2013). Therapeutic early interventions to prevent school exclusion: Evaluation of three contemporaneous projects. Office for Public Management ‐ OPM. Retrieved from http://www.opm.co.uk/wp‐content/uploads/2014/01/Therapeutic‐early‐interventions‐to‐prevent‐school‐exclusion‐and‐truancy‐Final‐Report.pdf [Google Scholar]

[cl2014001034-bib-000477] St James‐Roberts, I. , & Singh, C. S. (2001). Can mentors help primary school children with behaviour problems? (Study 233). England: Home Office. Retrieved from: https://core.ac.uk/download/files/161/4157929.pdf [Google Scholar]

[cl2014001034-bib-000478] Tarling, R. , Davison, T. , & Clarke, A. (2004). Mentoring projects. The national evaluation of the Youth Justice Board's Mentoring Projects. London: Youth Justice Board. Retrieved from http://yjbpublications.justice.gov.uk/Resources/Downloads/mentoringfull.pdf [Google Scholar]

[cl2014001034-bib-000479] U.S. Department of Education, I. o. E. S., What Works Clearinghouse . (2014). The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes. Princeton: What Works Clearinghouse Single Study Review. Retrieved from http://search.proquest.com/docview/1651831519?accountid=9851 [Google Scholar]

[cl2014001034-bib-000480] Wasilewski, Y. , Gifford, B. , Bonneau, K. , Duke University, C. F. C. , & Family, P. (2008). Evaluation of the School‐Wide Positive Behavioral Support Program in eight North Carolina elementary schools. Duke University: Center for Child and Family Policy. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED503191&site=ehost‐live&scope=site [Google Scholar]

[cl2014001034-bib-000481] What Works, C. (2011). Check & Connect. What Works Clearinghouse Intervention Report. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=3&id=ED525366;http://files.eric.ed.gov/fulltext/ED525366.pdf

[cl2014001034-bib-000482] What Works, C. (2014). WWC review of the report “The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes”. What Works Clearinghouse Single Study Review. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=2&id=ED546479;http://files.eric.ed.gov/fulltext/ED546479.pdf [Google Scholar]

[cl2014001034-bib-000483] Youth Justice Board England & Wales (2004). National evaluation of the restorative justice in schools programme. Retrieved from http://www.creducation.org/resources/National_Eval_RJ_in_Schools_Full.pdf

[cl2014001034-bib-000484] Armstrong, K. , Boroughs, M. , Massey, O. T. , Uzzell, D. , Sansosti, F. , & Perry, A. (2002). Symposium‐The safe schools/healthy students initiative: Methodologies and results in program‐based evaluation. Paper presented at the A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, Florida. [Google Scholar]

[cl2014001034-bib-000485] Bickmore, K. (2001). Good training is not enough: Research on peer mediation program implementation. Paper presented at the Association for Conflict Resolution Annual Conference, Toronto, Canada. [Google Scholar]

[cl2014001034-bib-000486] Boulay, B. , Gamse, B. , Checkoway, A. , Maree, K. , & Fox, L. (2011). Evaluation of Massachusetts Expanded Learning Time (ELT) initiative: Implementation and outcomes after four years. Paper presented at the Society for Research on Educational Effectiveness.

[cl2014001034-bib-000487] Buhi, E. R. (2015). Effects of an evidence‐based positive youth development program on academic outcomes among adolescents living in non‐metropolitan communities. Paper presented at the 143rd APHA Annual Meeting and Exposition (October 31‐November 4, 2015).

[cl2014001034-bib-000488] Collins, K. M. , & Onwuegbuzie, A. J. (2001). Effect of an After‐School Tutorial program on academic performance of middle school students at‐risk. Paper presented at the Anual Meting of the Mid‐South Educational Research Asociation, Little Rock, AR. [Google Scholar]

[cl2014001034-bib-000489] Eber, L. , Lewis‐Palmer, T. , & Pacchiano, D. (2002). School‐Wide Positive Behavior Systems: Improving school environments for all students including those with EBD. Paper presented at the 14th Annual Conference Proceedings. A Symposium of Care for Children's Mental Health: Expanding the Research Base, University of South Florida, Tampa. [Google Scholar]

[cl2014001034-bib-000490] Eber, L. , Rolf, K. , & Sullivan, M. P. (1998). School‐based systems of care: Early intervention and day treatment examples from Illinois. Paper presented at the 1th Anual Research Conference. A System of Care for Children's Mental Health: Expanding the Research Base, Tampa, FL. [Google Scholar]

[cl2014001034-bib-000491] Hammond, R. W. , Kadis, P. , & Betty, Y. (1990). Positive Adolescents Choices Training (PACT): Preliminary findings of the effects of a school‐based violence prevention program for African American Adolescents. Paper presented at the 98th Annual Convention of the American Psychological Asociation, Boston, MA. http://search.proquest.com/docview/62955491?accountid=9851 [Google Scholar]

[cl2014001034-bib-000492] Head, G. , Cogan, N. , & Kane, J. (2001). Evaluating behaviour support in mainstream secondary schools in one education authority. Paper presented at the British Educational Research Association Annual Conference, University of Leeds. http://www.leeds.ac.uk/educol/documents/00001916.htm [Google Scholar]

[cl2014001034-bib-000493] Heger, H. K. (1992). Retaining Hispanic youth in school: An evaluation of a counseling‐based alternative school program. Paper presented at the Anual Conference of the Rocky Mountain Educational Research Asociation, Stilwater, OK. http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=4&id=ED355302;http://files.eric.ed.gov/fulltext/ED355302.pdf [Google Scholar]

[cl2014001034-bib-000494] Howell, W. G. , & Peterson, P. E. (2000). School choice in Dayton, Ohio: An evaluation after one year. Paper presented at the Conference on Vouchers, Charters and Public Education (Harvard University, Cambridge, MA: ). [Google Scholar]

[cl2014001034-bib-000495] Kane, J. , Cogan, N. , & Head, G. (2002). Lessons to learn? Models of behaviour support in secondary schools in one education authority in Scotland. Paper presented at the European Conference on Educational Research, University of Lisbon. http://www.leeds.ac.uk/educol/documents/00002306.htm [Google Scholar]

[cl2014001034-bib-000496] O'Sullivan, R. G. (1988). Evaluation of a decision‐making program to improve school success with at‐risk middle school students. Paper presented at the The Anual Meting of the American Educational Research Asociation (New Orleans: ). http://files.eric.ed.gov/fulltext/ED296054.pdf [Google Scholar]

[cl2014001034-bib-000497] Smith, J. D. , Pare, D. , & Gravelle, F. (2002). Alternatives to school suspension: An intervention for at‐risk students. Paper presented at the Anual Meting of the American Educational Research Asociation, New Orleans, LA. [Google Scholar]

[cl2014001034-bib-000498] Winborn, J. D. (1992). A study of the effectiveness of a saturday school in reducing suspension, expulsion, and corporal punishment. Paper presented at the The Anual Meting of the Mid‐South Educational Research Asociation, Knoxvile, TN. http://search.proquest.com/docview/62851284?accountid=9851 [Google Scholar]

[cl2014001034-bib-000499] Billig, S. H. (2009). Does quality really matter? Testing the new k‐12 service‐learning standards for quality practice. In Moely B. E. B., Holland, B. A. S. H. (Ed.), Creating our identities in service‐learning and community engagement (pp. 131–157). Greenwich, CT: Information Age. [Google Scholar]

[cl2014001034-bib-000500] McIntosh, K. , Fisher, E. S. , Kennedy, K. S. , Craft, C. B. , & Morrison, G. M. (2011). Using office discipline referrals and school exclusion data to assess school discipline. In Jimerson S. R. N., Amanda B.; Mayer, Matthew J. ; Furlong, Michael J. (Ed.), Handbook of school violence and school safety: International research and practice, 2nd Edition (pp. 305–315). New York: Taylor & Francis. [Google Scholar]

[cl2014001034-bib-000501] Sexton, T. L. , Fisher, A. R. , Graham, C. , & Elnahrawy, N. A. (2015). Evidence‐based practice in the prevention and treatment of adolescent behavior problems. In Gullotta T. P., Plant, Robert W. , Evans, Melanie (Ed.), Handbook of adolescent behavioral problems (pp. 85–104). United States: Springer. [Google Scholar]

PERMALINK

School‐based interventions for reducing disciplinary school exclusion: a systematic review

Sara Valdebenito

Manuel Eisner

David P Farrington

Maria M Ttofi

Alex Sutherland

Abstract

Plain language summary

Interventions can reduce school exclusion but the effect is temporary

The review in brief

What is the aim of this review?

What is this review about?

What are the main findings of this review?

What do the findings of this review mean?

How up‐to‐date is this review?

Executive Summary/Abstract

BACKGROUND

OBJECTIVES

SEARCH METHODS

SELECTION CRITERIA

DATA COLLECTION AND ANALYSIS

RESULTS

AUTHORS’ CONCLUSIONS

1. Background

1.1 THE PROBLEM, CONDITION OR ISSUE

1.1.1 School discipline

Table 1.

1.2 PREDICTORS AND OUTCOMES

1.2.1 Predictors of school exclusion

Gender as a predictor of exclusion

Ethnicity

Age as a predictor of exclusion

Socio‐economic status (SES)

Special educational needs (SEN)

1.2.2 Negative outcomes linked to school exclusion

Behaviour

Academic achievements

Future social inclusion

1.3 THE INTERVENTION

1.3.1 School‐based programmes

Previous reviews

1.4 WHY IT IS IMPORTANT TO DO THE REVIEW

2. Objectives

3. Methods

3.1 TITLE REGISTRATION AND REVIEW PROTOCOL

3.2 CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW

3.2.1 Research design

3.2.2 Types of participants

3.2.3 Included interventions

3.2.4 Excluded interventions

3.2.5 Types of outcome measures

Primary outcomes

Secondary outcomes

3.2.6 Included literature

3.3 SEARCH METHODS FOR IDENTIFICATION OF STUDIES

3.3.1 Electronic searches

Table 2.

Table 3.

3.3.2 Other resources searched

3.4 DATA COLLECTION AND ANALYSIS

3.4.1 Selection of studies

3.4.2 Data extraction and management

3.4.3 Strategy to test inter‐rate reliability

3.4.4 Assessment of risk of bias in included studies

3.4.5 Criteria for determination of independent findings

3.4.6 Measures of treatment effect

3.4.7 Issues with the unit of analysis

3.4.8 Dealing with missing data

3.4.9 Time points within a study

3.4.10 Assessment of heterogeneity

3.4.11 Data synthesis

3.4.12 Subgroup analysis and investigation of heterogeneity

3.4.13 Outliers

3.4.14 Sensitivity analysis

3.4.15 Duration of follow‐up

4. Results

4.1 DESCRIPTION OF STUDIES

4.1.1 Results of the searches

Figure 1.