Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 May 1.
Published in final edited form as: J Youth Adolesc. 2019 Mar 21;48(5):876–890. doi: 10.1007/s10964-019-01013-2

Evaluation of a Whole-School Change Intervention: Findings from a Two-Year Cluster-Randomized Trial of the Restorative Practices Intervention

Joie Acosta a, Matthew Chinman a, Patricia Ebener a, Patrick S Malone b, Andrea Phillips a, Asa Wilks a
PMCID: PMC6508973  NIHMSID: NIHMS1525084  PMID: 30900083

Abstract

This study fills a gap in research on multi-level school-based approaches to promoting positive youth development and reducing bullying, in particular cyberbullying, among middle school youth. The study evaluates the Restorative Practices Intervention, a novel whole-school intervention designed to build a supportive environment through the use of 11 restorative practices (e.g., communication approaches that aim to build stronger bonds among leadership, staff, and students such as using “I” statements, encouraging students to express their feelings) that had only quasi-experimental evidence prior to this study. Studying multilevel (e.g., individual, peer group, school) approaches like the Restorative Practices Intervention is important because they are hypothesized to address a more complex interaction of risk factors than single level efforts, which are more common. Baseline and two-year post survey data was collected from 2,771 students at 13 middle schools evenly split between grades 6 (48 percent) and 7 (52 percent), and primarily ages 11 (38 percent) or 12 (41 percent). Gender was evenly split (51 percent male), and 92 percent of students were white. The intervention did not yield significant changes in the treatment schools. However, student self-reported experience with restorative practices significantly predicted improved school climate and connectedness, peer attachment, and social skills, and reduced cyberbullying victimization. While more work is needed on how interventions can reliably produce restorative experiences, this study suggests that the restorative model can be useful in promoting positive behaviors and addressing bullying.

Introduction

Some schools are safe and supportive environments for staff and students; others are steeped in bullying and poor relationships among teachers and students. Both school climates affect students’ development but in opposite ways. This underscores the need to involve the whole school—all administrators, staff, and students—in efforts to improve school environments to promote positive youth development and eliminate bullying and other negative outcomes. While various approaches that both promote positive youth development and attempt to impact the whole school climate have been developed, they have not been systematically evaluated. This article discusses the results of a randomized controlled trial evaluation of the Restorative Practices Intervention, a whole-school intervention designed to build supportive school environments through stronger bonds among leadership, staff, and students.

Changing the Whole-School Environment

A supportive school environment, which comes from changes across multiple levels (individual and their environment), has significant impacts on youth behavior and may be more effective and lasting than an effort focused on any single level (Sallis, Owen & Fisher, 2015). For example, a lack of school connectedness is one of the strongest predictors of alcohol use among adolescents (Weatherson, O'Neill, Lau, Qian, Leatherdale, & Faulkner, 2018), while the presence of connectedness has been shown to reduce the prevalence and impacts of violence, bullying, and victimization (Arango, Cole-Lewis, Lindsay, Yeguez, Clark & King, 2018; Duggins, Kuperminc, Henrich, Smalls-Glover, & Perilla, 2016; O'Brennan & Furlong 2010). As another example, negative teacher-student relationships may increase the risk that certain adolescents will engage in problem behavior such as substance abuse, risk-taking (e.g., stealing, riding in a car without a seatbelt), and aggression (De Laet, Colpin, Vervoort, Doumen, Van Leeuwen, Goossens, & Verschueren, 2015; Rudasill, Reio Jr et al. 2010). Similarly, peer relationships (when negative) can also be associated with bullying (Salmivalli, 2010). These findings underscore the need to involve all staff and peer groups to effectively promote youth development and mitigate behavioral problems (Thomberg, Wanstrom, Pozzoli, & Gini, 2018). The Restorative Practices Intervention, the intervention being evaluated, involves all peers and school staff in an effort to improve peer relationships and the school environment.

Although there is no widely agreed upon definition of school climate, research has identified several environmental characteristics that might indicate a positive school climate, including teacher and peer support, engagement of students, and safety as established through consistent and clear rules (Thapa, Cohen etal., 2013). When school climate leans toward the negative, bullying behaviors are more prevalent among children and can affect them on a daily basis over an extended time (Konishi, Miyazaki, Hymel, & Waterhouse, 2017). When school climate is positive—especially when it involves high levels of teacher and peer support, engagement of students, and safety—it supports students developing higher levels of assertiveness, empathy, and other key social skills (Weare, 2015).

Addressing school climate has been recognized as a way to prevent bullying (Espelage, 2018), which has spurred new policies and research on the link between positive school climate and improved student development (Thapa, Cohen et al., 2013). For example, the Every Student Succeeds Act (ESSA, 2015) emphasizes the importance of school climate in preparing students for college and career. The Every Student Succeeds Act also recommends that states use school climate as an indicator of school performance (alongside academic data) to differentiate school quality and student success in their accountability systems (Astor, Benbenishty et al., 2009). Many bullying programs (e.g., Gregory et al., 2010) are based on the improvement of the school climate, and some evaluations of those programs have shown significant decreases in bullying (e.g., Low & Van Ryzin, 2014). The implication is that school climate can either promote or minimize bullying behavior and is associated with the development of social skills.

Approaches to Support Youth Development through Positive School Environments

Adolescents’ problem behaviors often develop from a complex interaction of their risk factors, developmental changes, and the environment in which they live. As a result of this complexity, addressing problem behaviors may require a more comprehensive approach than programs that narrowly address certain risks. For example, effective drug and alcohol prevention programs (e.g., Project ALERT; Ringwalt, Clark, Hanley, Shamblen, & Flewelling, 2009) may address certain deficits (e.g., poor skills in decision making, communication) but not the myriad risk factors that youth experience at school (e.g., negative peer influences). Other deficit-oriented approaches like strong discipline (e.g., zero tolerance) fail to help youth develop the relationships they need to thrive and can lead to unintended negative consequences such as posttraumatic stress disorder, anxiety, depression, academic failure, and dropping out, especially among minorities (Ashworth et al., 2008).

Research has evaluated whole-school (students, staff, their relationships and climate), approaches in various areas. Olweus’ Bullying Prevention Program—one of the most widely disseminated antibullying prevention programs in the United States—has consistently demonstrated positive impacts on bullying perpetration and victimization. However, studies on Olweus have yielded only short-term behavior changes and have not specified the program components responsible for behavioral change (Farrington & Ttofi, 2009). An evaluation of the School Wide Positive Behavioral Interventions and Supports program, which is effective for addressing behavioral issues among elementary school students, found it was moderately effective at improving school climate in middle school settings, but not school safety (Young, Shatzer etal., 2009). Overall however, research involving middle schools is limited and has not examined program impacts on bullying (Durlak, Weissberg, Dymnicki, Taylor & Schellinger, 2011).

Another promising approach is comprehensive positive youth development programs, which leverage youths’ innate potential for positive growth to improve their health and quality of life through support, opportunities, and positive challenges. The strength of comprehensive positive youth development programs is their specific focus on youths’ developmental milestones that recognizes the diverse and interactive nature of individual and environmental influences, both positive and negative. By building competencies (e.g., moral, social), increasing healthy bonding with peers and adults, and intervening for at least nine months or longer (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2002), comprehensive positive youth development programs have been shown to impact school outcomes, including school readiness, academic achievement, life satisfaction (Benner, Graham, & Mistry, 2008) and problem behaviors like drug use (Taylor, Oberle, Durlak & Weissberg, 2017).

A recent study reviewed 82 universal school-based positive youth development programs to identify their effects on developmental outcomes, and found 29 programs that improved social competencies, and 34 programs that reduced problem behaviors—specifically drug and alcohol use, school misbehavior and aggressive behavior and violence (Taylor et al., 2017). A seminal study in positive youth development found that programs that influenced positive youth development and problem behaviors used the core set of strategies mentioned above: building competencies, increasing healthy bonding with peers and adults; and intervening for nine months or longer (Catalano, Berglund et al., 2002).

The above comprehensive positive youth development studies have established a strong foundation on which to build, but much of this research was quasi-experimental or used outcome measures with limited validity and reliability (Taylor et al., 2017). Therefore, more rigorous research (i.e., randomized controlled trials) is needed (Catalano, Gavin, & Markham, 2010). The first randomized controlled trial of its kind, this study examines the degree to which the Restorative Practices Intervention can improve a school environment to positively influence peer relationships, positive developmental outcomes, and bullying victimization. The Restorative Practices Intervention has the potential to be more efficient at preventing bullying than narrower programs because it targets the middle school environment broadly and social competencies.

Restorative Practices Intervention

The International Institute for Restorative Practices developed the Restorative Practices Intervention in 1999. The Restorative Practices Intervention involves training all school staff on how to enact 11 “Essential Elements” (see Table 1), a continuum of practices that range from informal (e.g., using “affective” statements that communicate feelings) to formal (e.g., hosting a restorative “circle” or “conference”, in which participants are encouraged to express emotions and form emotional bonds after problematic or disruptive behavior). Significant components of the Restorative Practices Intervention, circles and conferences are group meetings between school staff and students.

Table 1.

11 Essential Restorative Practices and Sample Indicators of Proficiency in Each Practice

Essential practices Sample indicators of proficiency in practice
1. Affective statements Use “I” statements; make students aware of the positive or negative impact of their behavior; focus on behavior; encourage students to express their feelings
2. Restorative questions Reflect standard restorative questions (What harm has been done? How has it impacted you? What needs to happen to make things right?); require a response, written or verbal
3. Small impromptu conferences Use to resolve low-level incidents between 2 people; take place as soon as possible after the incident has occurred; use the standard set of restorative questions; use affective statements; ask students to conduct a specific activity to repair harm from the incident
4. Proactive circles (comprise at least 80% of circles conducted at a school) Use to set behavioral expectations (e.g., for academic goal setting or planning, to establish ground rules for student projects, to monitor or build understanding of academic content); use standard set of restorative questions; use affective statements; run by students, after being facilitated 5 times
5. Responsive circles (comprise no more than 20% of circles conducted at a school) Use in response to behavior or tensions affecting a group of students or entire class; Require all people involved to play a role; Use standard set of restorative questions; Use affective statements
6. Restorative conferences Use in response to serious incidents or a cumulative pattern of repeated less serious incidents; use scripted approach and trained facilitator; use standard set of restorative questions and affective statements
7. Fair process Allow students to provide input into decisions affecting them; explain the reasoning behind decisions to the students affected; clarify expectations so students understand implications of the decision, specific expectations for carrying out the decision, and consequences for not meeting expectations
8. Reintegrative management of shame Avoid labels that stigmatize wrong-doers; discourage dwelling on shame; acknowledge person’s worth while rejecting unacceptable behavior (i.e., separate the deed from the doer)
9. Restorative staff community Use restorative practices to resolve staff conflicts and proactive circles to build sense of community among staff
10. Restorative approach with families Use restorative practices during interactions with family members, including proactive circles that focus on intentional communication of positive student behavior and academic achievement
11. Fundamental hypothesis Maintain high expectations for behavior; do not ignore inappropriate behavior; use the appropriate mix of control/pressure and support; minimize the role of staff facilitators
Open in a new tab

There are several types of circles and conferences (Table 1). Circles can be initiated by students or staff to establish ground rules (proactive circle) or as a planned way to respond to inappropriate behavior affecting a group of students or an entire class (restorative circle). Conferences can be an immediate response to low-level conflicts between two people (impromptu conference) or a planned response to serious or repeated patterns of behavior (restorative conference). Written guidelines govern how schools should use each type of circle or conference (and other restorative practices) and provide corresponding indicators of proficiency for each type (Table 1). For example, in schools proficient in the Restorative Practices Intervention, proactive circles are expected to make up about 80 percent of all circles implemented.

Some of the 11 practices are meant to be used in other intervention practices, so all school staff are trained in them. For example, all school staff who interact with youth (e.g., paraprofessionals, educational technician, maintenance staff, bus drivers, cafeteria workers) are first trained in the use of affective or “I” statements and restorative questions (e.g., What needs to happen to make things right?). This question/answer approach serves as the core communication practice of the Restorative Practices Intervention model. Other key cross-cutting practices are used primarily by administrators, teachers, and other instructional staff, who are trained how to facilitate a fair and transparent process with students and recognize and manage shame among students. School staff are encouraged to use the restorative practices to build relationships and resolve staff issues (restorative staff community), and to interact with parents (restorative approach with families). All restorative practices encourage acting “with” youth and setting high expectations.

Theory of the Restorative Practices Intervention.

The Restorative Practices Intervention integrates ecological systems theory (Broffenbrenner, 2005) and psychology of affect theory (Figure 1; Ttofi & Farrington, 2008a; Ttofi & Farington, 2008b) into a single model. First, the Restorative Practices Intervention operationalizes and expands psychology of affect theory, which is an alternative to the punitive approaches often used by schools. The psychology of affect (Higgins, 1987) explains how the Restorative Practices Intervention achieves improved behavior and increased connectedness through three psychological mechanisms: 1) the Restorative Practices Intervention maximizes positive affect through proactive practices such as restorative circles (Table 1), which are aimed at developing closer bonds and relationships among youth; 2) the Restorative Practices Intervention minimizes negative affect by providing responsive practices that ensure that offenders can take public responsibility for their behavior and reintegrate into normal community life; and 3) the Restorative Practices Intervention encourages free expression of emotion through training in practices such as affective statements and questions.

Figure 1.

Figure 1.

Open in a new tab

RP’s Theoretical Model

While the psychology of affect theory explains how the Restorative Practices Intervention changes interactions at the individual level, the ecological systems theory explains how these individual level changes perpetuate improvements in the peer and overall school environments. Ecological systems theory argues that individual behavior is determined by multiple causes and is sensitive to multiple influences from microsystems (e.g., peers) all the way up to macrosystems (e.g., social conditions; Bronfenbrenner, 2005; Overton, 2006). The Restorative Practices Intervention primarily focuses on the peer and school systems at the micro- and meso-system levels, respectively (Figure 2). At the student level, restorative practices (e.g., circles, conferences) help to proactively build relationships with their peer and teachers and to have a more active voice in responding to specific school incidents. When students have positive interactions with their peers, this improves their peer relationships and can lead to more active participation in school, which in turn promotes the positive environment of their school. These changes in the peer and school environments can have a reciprocal positive influence on student development and that, in turn, can promote the three individual psychological mechanisms listed above. The results are continually reinforcing and reciprocal positive individual, peer and school cultures that effectively regulate and consistently build and repair social bonds, connectedness, and relationships—key mechanisms to prevent high-risk and harmful behaviors.

Figure 2.

Figure 2.

Open in a new tab

Study Conceptual Model

The Restorative Practices Intervention includes three core components of a comprehensive positive youth development approach (Lerner, Lerner & Benson, 2011): 1) sustained relationships with adults because the Restorative Practices Intervention creates positive and sustained adult-youth relationships through teacher-student dialogue that occurs in circles (#3-5 in Table 1); (2) skills building because the Restorative Practices Intervention uses teachers and other school staff to coach students on 7 of the 11 essential practices (#1-5, 7 and 8); and (3) application of skills building because, as students grow proficient in the 7 essential practices that they are coached to perform, school staff transfers responsibility for running the circles to students (though teachers continue to facilitate restorative conferences for serious and or chronic behavior problems).

Intended implementation of the Restorative Practices Intervention.

Training, monthly consultation, and ongoing participatory learning groups are used to support and monitor the Restorative Practices Intervention’s implementation, and International Institute of Restorative Practices coaches visit campuses twice per year to troubleshoot on-site. Staff receive typically four days of training over two school years to learn how to use restorative practices and participate in interactive exercises focused on building the skills needed to run effective circles and conferences. Other optional trainings promote leadership development for school administrators and “train-the-trainer” instruction (i.e., a small group of school staff are trained so they can train new staff on the intervention) to sustain the Restorative Practices Intervention after the support from the International Institute of Restorative Practices coaches concludes. In addition, International Institute of Restorative Practices facilitators consult with the school monthly by phone for 60-90 minutes to discuss implementation progress, solve issues, and answer questions that may arise. Finally, groups of 8–12 staff members convene regularly (typically monthly, but more frequently in some schools) for participatory learning groups to review educational resources and discuss their proficiency in the 11 Essential Elements. The International Institute of Restorative Practices facilitator will meet with staff to introduce the process and online tools to assist these groups, but staff are responsible for facilitating the ongoing meetings. After the first year of this study at the request of schools, the research team also developed several implementation tools, including sample plans, and identified implementation targets for schools (e.g., using a proactive circle daily). While the research team did not have any direct contact with schools or assist with training, they did advise the implementation team from the International Institute of Restorative Practices on best practices in implementation science and assist in developing these implementation tools.

Prior research on the Restorative Practices Intervention.

Quasi-experimental studies have shown that schools implementing the Restorative Practices Intervention achieve significant impacts on environment (e.g.,McCold, 2008). The results also suggest that the Restorative Practices Intervention’s effects diminish only modestly over time. Specifically, McCold & Wachtel (2002) found that significant improvements in both attitude (social competencies and self-esteem of participants) and behavioral measures (delinquency, disciplinary referrals, academic achievement, graduation rates) are positively related to how long students participate in RPI. These relationships between improvement and participation remained statistically significant after accounting for relevant risk factors. Quasi-experimental evaluations of restorative interventions have shown they can reduce conflict and develop positive relationships, both of which aid positive youth development (e.g., Hamilton, 2008). To build on these studies, a randomized controlled trial is needed to examine whether the Restorative Practices Intervention influences peer relationships, school climate, and ultimately, youth developmental outcomes.

Study Hypotheses

The measures selected for the study (Figure 2) are guided by ecological systems theory emphasizing how a comprehensive positive youth development approach like the Restorative Practices Intervention aims to influence the school environment, peer relationships, and subsequently youth developmental outcomes and bullying victimization (Bronfenbrenner, 2005). As shown in Figure 2, ecological systems theory nests the individual youth within multiple levels of interaction. Youth interact with peers, and they influence each other. Similarly, youths interact with school environment, which influences both the individual youth and the youth’s peers (reflected by double-headed arrows). These interactions mediate youth developmental outcomes and bullying victimization. When implemented with fidelity, comprehensive positive youth development approaches like the Restorative Practices Intervention are specifically focused on improving peer-to-peer communication and relationships as well as the school environment.

Impacts of the Restorative Practices Intervention were assessed using a web-based, self-reported student survey across multiple domains: 1) School (youth perceptions of school climate), 2) Peer (youths’ perceptions of attachment to and influence of peers); 3) Student (youth developmental outcomes including such social competencies as cooperation, assertiveness, responsibility, empathy, and self-control, and incidences of physical, verbal and cyberbullying as either victim or perpetrator). Based on ecological systems theory, the research team hypothesized that if the environment is positive, adolescents will have positive developmental outcomes and experience or perpetrate fewer incidences of bullying.

Taking an intent-to-treat approach, the research team hypothesized that students in schools randomly assigned to receive the Restorative Practices Intervention would report more school connectedness, better school climate, more positive peer relationships and developmental outcomes, and less bullying victimization compared to students in control schools (Hypothesis 1). Given the challenges past evaluations have found with implementing whole school approaches, the research team also conducted analyses to test a second hypothesis that students in the intervention schools who reported experiencing more restorative practices (either through the Restorative Practices Intervention or because they had specific teachers who naturally acted in a restorative manner outside of the Restorative Practices Intervention) would report more positive outcomes (Hypothesis 2). Testing this hypothesis allowed the research team to examine whether restorative practices was associated with more positive outcomes, regardless of whether the whole school implementation of the Restorative Practices Intervention was successful.

Methods

Study Design

This randomized controlled trial of the Restorative Practices Intervention (Acosta, Chinman et al., 2016) was conducted in 14 middle schools throughout Maine. Intervention and control schools were matched based on demographic, academic, and disciplinary data, and then randomized them so that seven schools received the Restorative Practices Intervention and seven did not. The intervention was implemented for two years, and the evaluation assessed the extent of implementation and changes in school connectedness, positive developmental outcomes, and bullying victimization from baseline to post two years later. During the course of the intervention, one intervention middle school dropped out of the study because of data privacy concerns, leaving seven intervention and six control schools. Middle school students were selected because research has shown that it is critical to intervene during early adolescence before the transition to high school, when risky behaviors become more prevalent (Merrell, Gueldner et al., 2008).

Data Collection

The survey was completed via computer in classrooms under the supervision of teachers. Most survey questions were asked about the general school environment and teacher behavior at the time of the survey, except bullying victimization questions, which asked about student experience over the past 30 days. All procedures were reviewed and approved by RAND’s Human Subjects Protection Committee. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Participants

Across the 13 middle schools, 2,824 participating students (response rate of 83 percent) were evenly split between grades 6 (48 percent) and 7 (52 percent) and were primarily ages 11 (38 percent) or 12 (41 percent). Gender was evenly split (51 percent male), and 92 percent of students were white (Table 2). To reduce heterogeneity, the analyses only included at baseline students who were no older than 12 for sixth grade or 13 for seventh grade. As a result, six sixth-graders, nine seventh-graders, and 38 students who did not indicate their grade were omitted, for a final sample of 2,771 students. Table 2 and all of the following statistics are based on this sample. Multilevel logistic analyses predicted student demographics at Year 2, student attrition at Year 2 from experimental condition, and the Year 1 values of key study variables. After false discovery rate (FDR) correction, none of these predicted attrition in individual models, adjusted ps > .05.

Table 2.

Student Characteristics (n=2,834)

Characteristic Percent
RPI+
Schools
(n = 977)		 Control
Schools
(n = 1,794)
Gender
Female 48 50
Male 52 50
Grade
6th 49 48
7th 51 52
Race/Ethnicity*
Hispanic or Latino 4 3
American Indian or Alaska Native 9 7*
Asian 1 3*
Black or African American 2 2
Native Hawaiian or Other Pacific Islander 1 1
White 88 87
Other 9 7
Open in a new tab

Note: Race/ethnicity does not add up to 100% because students could select more than one race/ethnicity.

*

Indicates p<0.05

+

Restorative Practices Intervention

The 13 schools housed grades 6–8 in the same building; five schools included grades K–5 on campus, and nine did not. Similar to the national average, 48 percent of students across the 13 schools received free and reduced lunches (ranging from 24 to 68 percent at the individual campuses; U.S. Department of Education, 2012). Schools collectively had a 94 percent retention rate (similar to the national average of 91 percent) for not only a single school year (rates for individual schools ranged from 85 to 99 percent of all students) but also from year to year (rates ranged from 90 to 99 percent for individual schools). Average enrollment was 430 students (compared to the national middle school average of 595), and schools ranged in size from 91 to 921 students (U.S. Department of Education, 2012). Attendance rates at schools were slightly higher than the national average (95 percent versus 91 percent nationally) and ranged from 93 to 99 percent on individual campuses. The average suspension rate was less than 5 percent (U.S. Department of Education, 2011). Out-of-school and in-school suspensions ranged from less than 1 percent on the low end to highs of 9 percent, and 21 percent, respectively.

Measures

School Climate.

Four select scales from the Inventory of School Climate (Brand, Felner et al., 2003): Consistency and Clarity of Rules and Expectations (e.g., Students are given clear instructions about how to do their work in classes), Teacher Support (e.g., Teachers go out of their way to help students), Positive Peer Interactions (e.g., Students get to know each other well in classes), and Student Input Into Decision Making (e.g., Students in this school have a say in how things work) were used to assess school climate. In past studies, the scales demonstrated good reliability (one-year test re-test ranged from 0.69 to 0.81; internal consistency ranged from an alpha of 0.70 to 0.76) and explained significant between-school variance in measures of academic, behavioral, and socio-emotional adjustment in prior studies (suggesting the scales’ validity; Brand, Felner et al., 2003). In the present study, internal consistency for the perceived school climate measures was assessed using McDonald’s (1999) coefficient omega. Omega is a measure of internal consistency on the same metric as coefficient alpha. It has advantages over alpha in that it requires more realistic assumptions, has fewer problems with inflation due to number of items, and allows confidence intervals to be generated for a more accurate evaluation of the reliability of a scale (Dunn, Baguley et al., 2014).

In the study, all perceived climate scaled had adequate internal consistency: Consistency and Clarity of Rules and Expectations, coefficient omega = 0.70, 95% confidence interval (CI) [0.68, 0.72]; Teacher Support, omega = 0.77, 95% CI [0.75, 0.78]; Positive Peer Interactions, omega = 0.76, 95% CI [0.74, 0.77]; and Student Input into Decision Making, omega = 0.75 95% CI [0.73, 0.77].

School Connectedness.

A five-item scale (1=strongly disagree to 5=strongly agree) from the National Adolescent Health Study was used to measure students’ perceptions of closeness to people at their school (e.g., peers, teachers), happiness at their school, belonging at school, and safety at school (e.g., I feel like I am part of this school; McNeely, Nonnemaker et al., 2002). The scale has shown good internal consistency in past studies (alpha = 0.78; Anderman, 2002). In the study data, coefficient omega was 0.81, 95% CI [0.79, 0.82]. The scale also has indicators that suggest its validity: It has been associated with measures of emotional well-being, which is consistent with other research (e.g., Frydenberg, Care, Chan, & Freeman, 2009).

Peer Attachment.

A three-item scale developed by Acosta (2003) in areas such as receiving encouragement from peers to do well in school, confiding in peers, emulating peers, and considering peers’ reactions before acting (e.g., I confide in my friends; 1=Never to 6=Always) was used to assess peer attachment. Past research has suggested the scale is reliable (alpha = 0.71) and valid: It is associated with variations in peer group activity, with more attached peers reporting more peer interaction (Acosta, 2003). Omega could not be calculated because the scale is only three items.

Social Skills.

The Social Skills Improvement System-Rating Scale (SSIS-RS; Gresham, Elliott et al., 2010) was used to assess students’ perceptions of prosocial behavior in assertiveness (e.g., I ask for help when I need it) and empathy (e.g., I try to think about how others feel). Students self-rate their behavior on a four-point scale (0 = Never, 1 = Seldom, 2 = Often, and 3 = Almost Always). For ages 13–18, the SSIS-RS has alpha coefficients above 0.70 for all scales, and test-retest indices range from 0.77 to 0.92 (Vaz, Parsons et al. 2013). Prior research has established the criterion-related validity of the self-report form through correlations with associated measures (e.g., Youth Self-Report Form, Piers-Harris Children’s Self-Concept Scale; Gresham & Elliot, 1990). In this study’s measurement modeling, items did not consistently coalesce in the scales they were intended to represent. Accordingly, exploratory structural equation modeling (ESEM) was followed by confirmatory factor analysis (CFA) to improve the fit of the modeled conceptual scales to the data. ESEM is a procedure that can estimate an SEM model with no measurement constraints on a block of indicators within the model. By varying the number of factors indicated by that block, the ESEM simultaneously maximizes the fit of the structural model including the factors and the exploratory portion of the measurement model. The ESEM modeled the full set of SSIS-RS items as mediators between school climate variables and bullying outcomes, with varying numbers of factors reflected in the mediating block. After selecting the two-factor exploratory model for the SSIS-RS, post-hoc judgments were applied to use two of the original four assertiveness items and move a third assertiveness item (“I stand up for others when they are not treated well”) to the empathy scale, where it showed greater inter-item correlations and reasonable face validity. All four empathy items were used. In the data, coefficient omega was 0.74, 95% CI [0.72, 0.75] for the revised assertiveness scale and 0.87, 95% [0.86, 0.88] for the revised empathy scale.

Bullying Victimization.

Three items from the Communities That Care Survey (Arthur, Briney et al., 2007) were used to assess prevalence and frequency (not at all, somewhat, a whole lot) of verbal bullying (how often have you been taunted, teased, experienced name-calling, or been excluded or ignored by others in a mean way), physical bullying (how often has someone hit, kicked, or shoved you, or taken your money or belongings), and cyber bullying (how often has someone sent mean emails, text messages, or instant messages, or posted hurtful information about you on the Internet) in the past 30 days. Communities That Care was a ten-year longitudinal study of prevention that surveyed thousands of middle school youth about risk and protective behaviors, delinquency, bullying (Arthur, Hawkins et al., 2002), and other problem behaviors. In addition to their face validity, these questions have been found to be positively correlated with alcohol use among middle school students (Peleg-Oren, Cardenas et al., 2012). This in an indicator of validity because of the documented relationship between bullying victimization and high-risk behaviors like alcohol use (Tharp-Taylor, Haviland, D’Amico, 2009). Bolstering the case for validity is the fact that similar questions used in a study of rural schools found similar rates of bullying (Dulmus, Theriot, Sowers, & Blackburn, 2004). Because of sparse use of the “a whole lot” response in the sample, the three items were dichotomized to “not at all” vs. “somewhat or a whole lot.

Student Report of Restorative Practices.

In both the baseline and post-test surveys, students were asked 17 questions about their experience of restorative practices at school, covering the use of affective statements (e.g., My teacher encourages students to express their feelings), restorative questions (e.g., When someone misbehaves, my teacher responds to negative behaviors by asking students questions), fair process (e.g., My teacher takes the thoughts and ideas of students into account when making decisions), and reintegrative management of shame (e.g., My teacher focuses on behavior and not whether students are “good” or “bad” people). Students in the intervention schools were asked ten additional questions at post-test covering the specific use of circles, both proactively (e.g., My teacher uses circles to help students get to know each other and build relationships) and in response to a behavioral issue (e.g., My teacher uses circles to respond to behavior problems and repair harm caused by misbehavior). Youth responded using a Likert scale from one (not at all) to five (always). Neither confirmatory nor exploratory factor analysis yielded a clear solution for subscales within these items. Instead the first principal component was extracted from the 17 items at pre-test for all students and the 27 items at post-test for intervention students. The first component accounted for 48 percent and 51 percent, respectively, of the original variability across items.

Statistical Analyses

All analyses were conducted in the most recent version of SAS software (v9.4, SAS Institute, 2016), Mplus (v8.0, L. K. Muthén & Muthén, 2017), and R (v3.4.1, R Core Team, 2017). All models accounted for the nested sample of students within schools by incorporating a school-level random intercept for each outcome. Data linking students to specific teachers or other staff members were unavailable to the project, so just two levels were modeled. Initial confirmatory measurement modeling was conducted for existing scales. Where noted, exploratory measurement modeling, in which the a priori model was not reflected in the data, was used.

Hypothesis 1 (Intent-to-Treat; ITT) was tested as a series of two-level (school and student) linear regression models predicting student outcomes (school climate, school connectedness, peer attachment, and social skills) and two-level logistic regression models for students’ binary reports of the three categories of bullying with experimental condition as the predictor. Experimental condition was modeled as a level 2 (school) predictor. Pre-test scores on all student measures were included as covariates at level 1, as well as select student demographics (age [truncated to “13 or higher”], gender, and a binary coding of white vs. any other race or ethnicity). The between-within method was used for calculating degrees of freedom Due to the modest level 2 sample size, no covariates associated with whole schools were included. The Benjamini-Hochberg (1995) correction was used to control the FDR to 0.05 across the multiple outcomes.

Hypothesis 2 focused on assessing the experience of students in the Restorative Practices Intervention—assigned schools with restorative practices by school staff. The restorative practice items in the first principal component described above was the operationalization of restorative practice experience at the student level. Relations between the principal component and the student outcomes from Hypothesis 1 were assessed. The predictor and outcomes were both at level 1 (student). Random school-level intercepts were modeled in the outcomes and student-level covariates (which also included the baseline measure of the principal component). The Benjamini-Hochberg correction was applied across the set of student outcomes.

Measurement modeling was conducted in Mplus using the “MLR” estimator. MLR is a maximum likelihood estimator that produces a χ2 statistic for model fit asymptotically equivalent to the Yuan-Bentler T2* statistic (Yuan & Bentler, 2000) and generates standard errors of model parameters with a sandwich estimator. MLR uses all cases to maximize statistical power and is unbiased when item-level data are missing at random (MAR; Little & Rubin, 1989). Coefficient omega and its confidence interval for each scale were estimated using ci. reliability in the MBESS package for R (v.4.3.0, Kelley, 2017), using the accelerated bootstrap with 3,000 draws for the confidence intervals. The ci. reliability also assumes MAR for item-level missing data.

Hypotheses 1 used listwise deletion, including only students who responded to the survey at Year 2 (n = 1,685). Listwise deletion requires the stricter assumption of missing completely at random (MCAR; Little & Rubin, 1989). Two-level models in Mplus that would have been valid for Hypothesis 1 with the less restrictive MAR assumption resulted in computational difficulties, most likely due to the small level 2 sample size. For Hypothesis 2, because there were no level 2 predictors, only the adjustments for clustering could be used and included all respondents (n = 955), with the assumption that responses missing at post-test were MAR.

Results

Hypothesis 1: Did students in the Restorative Practices Intervention schools report more school connectedness, positive developmental outcomes, and less bullying victimization than students in control schools?

ITT results were not significant for any of the 11 outcomes, ps > 0.14. Detailed results are reported in Table 3. The table includes estimates of effect sizes with 95 percent confidence intervals; the median effect size estimate for the continuous outcomes (Cohen’s d) was 0.57. Odds ratio point estimates for bullying were 1.18 (physical), 1.06 (emotional), and 0.89 (cyber).

Table 3.

Student Outcomes Predicted by Intervention Condition

Student Outcome 95% CI
Lower
Bound		 b 95% CI
Upper
Bound		 t (11) 95% CI
Lower
Bound		 d 95% CI
Upper
Bound
Clarity/ Consistency −0.083 0.005 0.093 0.13 −1.02 0.07 1.16
Teacher Support −0.071 0.099 0.269 1.28 −0.43 0.71 1.83
Positive Peer Interactions −0.142 0.015 0.171 0.21 −0.98 0.12 1.21
Student Input −0.056 0.139 0.334 1.57 −0.29 0.87 2.00
School Connectedness −0.090 0.090 0.288 1.15 −0.50 0.64 1.75
Peer Attachment −0.090 0.099 0.288 1.15 −0.50 0.64 1.75
Social Skill: Assertiveness −0.190 0.031 0.253 0.31 –0.92 0.17 1.26
Social Skill: Empathy −0.051 0.036 0.123 0.91 −0.62 0.51 1.61
Logistic
b		 Standard
Error		 t (11) 95% CI
Lower
Bound		 Odds
Ratio		 95% CI
Upper
Bound
Physical Bullying 0.167 0.224 0.74 0.72 1.18 1.93
Emotional Bullying 0.063 0.160 0.39 0.75 1.06 1.51
Cyber Bullying −0.113 0.262 −0.43 0.50 0.89 1.59
Open in a new tab

Note: N = 1,685; k (schools) = 13. None of the coefficients by intervention condition were statistically significant, p < 0.05, after FDR correction. Covariates not shown.

Figure 3 shows the distribution of students’ self-reported experience of restorative practices using the standardized principal component scores. Only a minority of students in intervention schools experienced restorative practices to a great extent, while students in control schools experienced much more restorative practices than would be expected. The similarity across treatment and control schools could explain the lack of significant difference between treatment and control groups in the ITT analyses.

Figure 3.

Figure 3.

Figure 3.

Open in a new tab

Student Self-reported Experiences with Restorative Practices

Hypothesis 2: Did the student reports of restorative practices experience predict more school connectedness, positive developmental outcomes, and less bullying victimization?

Models predicting student outcomes from the self-reported restorative practices experience at follow-up showed significant and positive relations with all school climate, school connectedness, peer attachment, and social skills outcomes, as well as significant negative relations with reports of physical and cyber bullying (negative relations being beneficial because higher restorative practices experience is associated with less reported bullying) after the FDR correction.

These models included baseline values of the 11 outcomes, the baseline principal component, and student-level demographics. Relations with emotional bullying were not significant. Details are shown in Table 4, including effect size estimates and their 95 percent confidence intervals. For the continuous outcomes, the median semi-partial standardized regression coefficient (β) was 0.42. Odds ratio point estimates for a standard-deviation change in the RP predictor for bullying were 0.74 (physical), 0.82 (emotional), and 0.69 (cyber).

Table 4.

Student Outcomes Predicted by Standardized Experience of Restorative Practices

Student Outcome  b SE z 95% CI
Lower
Bound		    β 95% CI
Upper
Bound
Clarity/Consistency 0.343 0.040 8.47 0.44 0.51 0.58
Teacher Support 0.424 0.027 15.44 0.50 0.56 0.62
Positive Peer Relations 0.239 0.021 11.27 0.31 0.35 0.39
Student Input 0.402 0.024 16.89 0.44 0.51 0.57
School Connectedness 0.421 0.037 11.32 0.41 0.48 0.56
Peer Attachment 0.318 0.045 7.14 0.25 0.33 0.40
Social Skill: Assertiveness 0.364 0.077 4.76 0.21 0.33 0.44
Social Skill: Empathy 0.256 0.033 7.76 0.28 0.36 0.44
 b SE z 95% CI
Lower
Bound		 Standardized
Odds Ratio		 95% CI
Upper
Bound
Physical Bullying −0.284 0.095 −3.00 0.63 0.74 0.88
Emotional Bullying −0.208 0.204 −1.02 0.55 0.82 1.24
Cyber Bullying −0.371 0.103 −3.59 0.57 0.69 0.84
Open in a new tab

Note: N = 955; k = 7 schools. All coefficients for experience statistically significant, p < 0.05, after FDR correction. Covariates not shown.

Sensitivity Analyses

Two a priori changes to the outcome measures had the potential to alter results: Re-arranging items from the Assertiveness and Empathy subscales of the SSIS-RS and constraining the three-option bullying responses to binary “none” vs. “any.” Sensitivity analyses involved repeating the Hypotheses 1 and 2 models for these outcomes using the original measures. There were no substantive differences in the results for Hypothesis 1. For Hypothesis 2, three findings changed significance in the sensitivity analyses: The effects of restorative practice experiences on cyberbullying, assertiveness, and empathy, significant in the primary analyses, were not significant in these analyses. The difference for SSIS-RS Empathy was likely a degree of fluctuation, with a new p-value of .079 (and no change in sign). The new tests for cyberbullying and assertiveness were not significant, p’s > .20.

Discussion

Addressing adolescent’s problem behaviors that arise from a complex interaction of risk factors, developmental changes, and environmental factors may require a more comprehensive approach than programs that narrowly address certain risks. This study represents the first randomized controlled trial of the Restorative Practices Intervention and advances the prior quasi-experimental research on positive youth development and environmental approaches to build positive school environments for youth to thrive (Catalano, Gavin, & Markham, 2010). These types of whole-school, environment-focused and comprehensive positive youth development approaches are critical to study because a supportive school environment, which impacts multiple levels (individual and their environment), may be more effective and lasting than an effort focused on any single level (Sallis, Owen & Fisher, 2015). The evaluation found that while the Restorative Practices Intervention did not have significant effects on students in the treatment schools, a restorative school environment is associated with more positive youth development and reductions in in bullying among middle school students.

Specifically, the evaluation found that middle-school students who received the Restorative Practices Intervention did not report more school connectedness, better school climate, more positive peer relationships and developmental outcomes, or less victimization than students in control schools did. Further, the Restorative Practices Intervention in this study delivered only a modest amount of restorative practices experiences—an amount not much different than control schools received. Based on this evaluation, it is unclear whether the Restorative Practices Intervention as designed can impact the whole school as hypothesized.

While the research team did not find evidence that the Restorative Practices Intervention was an effective comprehensive positive youth development program, they did find further evidence to support the importance of a restorative school environment for positive youth development. Specifically, students who reported having the greatest restorative practices experiences because of their teachers’ actions (regardless of whether the teacher had received the Restorative Practices Intervention) reported more positive outcomes (higher school connectedness, better school climate, more positive peer relationships and developmental outcomes) and less victimization from physical and cyber bullying. These findings suggest that while the intervention itself did not create a whole-school change, restorative practices—if used consistently enough—hold promise for reducing bullying victimization by building a supportive environment through stronger bonds among leadership, staff, and students. This adds to a growing body of literature suggesting that building a more supportive school environment could reduce bullying victimization (Low & Van Ryzin, 2014), filling a gap in research on middle school youth (Bradshaw, Reinke, Brown, Bevans, & Leaf, 2008). Given the significant negative developmental impacts of bullying, the findings that school practices both support positive youth development and reduce bullying victimization are a significant advancement of prior research. It is particularly notable that these results were found for cyberbullying victimization, which is especially difficult to address because it occurs outside of formal and controlled settings (such as school). Future research is needed to identify and develop comprehensive positive youth development approaches that can reliably build a restorative school environment.

Similar to past evaluations of whole-school approaches, implementing the Restorative Practices Intervention in the middle schools was challenging. Consistent whole-school implementation of the Restorative Practices Intervention did not happen over the two years which could have explained the lack of significant findings from the intent-to-treat analysis. This could be due to the fact that not all teachers were actively participating in the Restorative Practices Intervention or were not implementing the Restorative Practices Intervention at the recommended levels (i.e., were using circles sporadically, rather than regularly as intended). In addition, whole-school approaches involve many different actors (administration, staff, students), and turnover among these actors means there is a strong need for implementation support, including tools and coaching. Coaching was provided monthly, which may not be sufficient for such an intensive intervention with such a broad scope. The fact that other implementation tools, such as sample plans and identified implementation targets, were not available to intervention schools at the start of the study could also have weakened implementation. Similarly, while the International Institute of Restorative Practices identified implementation targets (e.g., using a proactive circle daily) during the course of the study, their late introduction could have led to confusion among teachers and leadership about what constitutes sufficient implementation and how to incorporate it into the daily school schedule. The International Institute of Restorative Practices offers tailored leadership training for administrators and staff championing the Restorative Practices Intervention at schools to teach them how to be engaging, collaborative, and effective at using restorative practices in their organizational and implementation strategies. This training was optional for intervention schools and available only after the Restorative Practices Intervention’s first year had concluded, but making this training a requirement may be beneficial, given the implementation challenges. In addition to sufficient implementation support, the International Institute of Restorative Practices could experiment with strategies to anticipate and overcome structural barriers to implementation such as leadership engagement, staffing levels and staff turnover, and school schedules and other competing demands on administrative and teaching staff. More research is needed to identify implementation best practices for these whole-school and comprehensive positive youth development approaches. The Comprehensive Framework for Implementation Research provides a structured list of constructs associated with effective intervention of health innovations, and could be useful in systematically assessing potential barriers and facilitators in future comprehensive positive youth development research and by schools that are preparing for implementing a comprehensive positive youth development approach like the Restorative Practices Intervention.

There were three main limitations to this study. First, this study relied on self-reported data from students. To address this limitation, self-report data was collected used reliable and valid measures where available. Second, this study collected limited measures of intervention. Since many of the mechanisms for the intervention were impromptu (e.g., impromptu conferences, responsive circles) arising out of need, it was challenging to collect observational data on these impromptu components of the intervention. However, implementation data was collected directly from students. Since students were the targets of the intervention, assessing their experiences of restorative practices provided an important perspective on whether the intervention was implemented to scale. Finally, bullying victimization was measured by three single items. The study assessed whether students had been bullied in the past 30 days, not whether they witnessed bullying or perpetrated bullying. While limited, these three items have been used in one of the largest and longest running research study on adolescents (i.e., the Communities That Care Study; (Arthur, Briney et al., 2007). Future research could benefit from additional observational measures of both implementation and outcomes. Measures should take into account the interactional nature of the intervention and capture implementation at the classroom level, which would allow for a more nuanced understanding of how the intervention was diffused across the school.

An additional caution is warranted as a result of the sensitivity analyses: Two findings that were significant, adjusted p < .05, in the original analyses showed new p-values > .20 when using the unmodified measures (a third resulted in an un-adjusted p of .079). Among the bullying measures, it is unsurprising that cyber-bullying was the most influenced by the restoration of the three-level response as it had the lowest frequency of endorsement of the highest response, meaning the logistic threshold would be less stably estimated. For the modified SSIS-RS Assertiveness subscale, the explanation is less obvious. Both versions of the subscale have comparable (and high) internal consistency, so there is no empirical reason in the current analyses to place greater weight on one finding than the other. That said, the changes made to the SSIS-RS were based on measurement modeling independent of the current paper and the modified version was our a priori choice for the current analyses.

Additional research should also assess whether whole-school approaches are practical or whether an alternative model is better suited to diffusing restorative practices into schools (e.g., integrating restorative practices into teacher training and education so all teachers are skilled in these practices). Research should also determine the length of time it takes for these interventions to become integrated into the school environment (e.g., was two years long enough for the Restorative Practices Intervention to be fully implemented and to see student-level changes based on an improved whole-school environment). Given the interactional nature of the intervention, future studies should also consider more creative ways of capturing dose (e.g., monitoring teacher-student interactions). Since teachers in the control schools were already using restorative practices without a specific intervention framework, it will be important for the International Institute of Restorative Practices to clarify what makes the Restorative Practices Intervention unique and a value added above and beyond what teachers are already doing.

Conclusion

Adolescent problem behavior is complex and likely requires comprehensive, multi-level approaches that address a wide range of youth development factors. However, these approaches have not been studied in rigorous trials. This study is the first randomized controlled trial of a comprehensive, multi-level intervention—the Restorative Practices Intervention—and advances the prior quasi-experimental research in this area, as well as filling a gap in research on positive youth development approaches for middle schoolers. The two-year, whole-school intervention studied here did not yield significant changes in the schools participating in the Restorative Practices Intervention. The Restorative Practices Intervention implementation was modest, and the results were not noticeably different from those of control schools. However, students’ restorative practices experience significantly predicted improvement across a range of student, peer, and school outcomes. These findings have implications for the understanding of adolescence, namely that a restorative environment can enhance youth development and reduce bullying. While the specific Restorative Practices Intervention did not yield differential outcomes, the findings suggest that interventions which are able to successfully create a restorative environment maybe most effective because they address the complex interaction of risk factors which underlie adolescence.

Acknowledgments

Funding

This research was supported by a grant from the National Institute of Child Health and Human Development (R01# 1HD072235).

Biography

Joie Acosta is a Senior Behavioral Scientist at the RAND Corporation. She specializes in community-based research focused on the implementation and evaluation of prevention and health promotion programs in the areas of public health, positive youth development, substance abuse, and mental health.

Matthew Chinman is a Senior Behavioral Scientist at the RAND Corporation. His major research interests include developing and testing capacity-building approaches to improve the implementation of evidence-based programs in community-based settings.

Patricia Ebener is a Survey Director and a Senior Policy Analyst at the RAND Corporation. Her major research interests include innovative survey methodologies and supporting and evaluating implementation of behavioral health prevention and treatment services.

Patrick S. Malone is Partner and Senior Consultant with Malone Quantitative in Durham, North Carolina. His major research area is quantitative research methods, focusing on latent variable and longitudinal modeling as applied to child development and health risk behaviors.

Andrea Phillips is a Senior Policy Analyst at the RAND Corporation. Her major research interests include developing and testing approaches to improve implementation of programs and strategies in school settings.

Asa Wilks is a Statistical Programmer at the RAND Corporation. His major interests include large-scale data applications for health care quality and outcomes research.

Footnotes

Data Sharing Declaration

The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors report no conflict of interests.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

REFERENCES

  1. Acosta J (2003). The effects of cultural differences on peer group relationships. Journal of Prevention & Intervention in the Community, 25(2), 13–26. [Google Scholar]
  2. Acosta JD, Chinman M, Ebener P, Phillips A, Xenakis L, & Malone PS (2016). A Cluster-Randomized Trial of Restorative Practices: An Illustration to Spur High-Quality Research and Evaluation. Journal of Educational and Psychological Consultation, 26(4), 413–430. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Act, E. S. S. (2015). of 2015, Pub. L, 114–95.
  4. Anderman EM (2002). School effects on psychological outcomes during adolescence. Journal of Educational Psycholgoy, 94,795–809. [Google Scholar]
  5. Arango A, Cole-Lewis Y, Lindsay R, Yeguez CE, Clark M, & King C (2018). The Protective Role of Connectedness on Depression and Suicidal Ideation Among Bully Victimized Youth. Journal of Clinical Child & Adolescent Psychology, 1–12. [DOI] [PubMed] [Google Scholar]
  6. Arthur MW, Briney JS, Hawkins JD, Abbott RD, Brooke-Weiss BL, & Catalano RF (2007). Measuring risk and protection in communities using the Communities That Care Youth Survey. Evaluation and program planning, 30(2), 197–211. [DOI] [PubMed] [Google Scholar]
  7. Arthur MW, Hawkins JD, Pollard JA, Catalano RF, & Baglioni AJ Jr (2002). Measuring risk and protective factors for use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Survey. Evaluation review, 26(6), 575–601. [DOI] [PubMed] [Google Scholar]
  8. Ashworth J, Van Bockern S, Ailts J, Donnelly J, Erickson K, & Woltermann J (2008). The restorative justice center: An alternative to school detention. Reclaiming Children and Youth, 17(3), 22. [Google Scholar]
  9. Astor RA, Benbenishty R, & Estrada JN (2009). School violence and theoretically atypical schools: The principal’s centrality in orchestrating safe schools. American Educational Research Journal, 46(2), 423–461. [Google Scholar]
  10. Benjamini Y, & Hochberg Y (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57,289–300. [Google Scholar]
  11. Benner AD, Graham S, & Mistry RS (2008). Discerning direct and mediated effects of ecological structures and processes on adolescents' educational outcomes. Developmental Psychology, 44(3), 840. [DOI] [PubMed] [Google Scholar]
  12. Bradshaw CP, Reinke WM, Brown LD, Bevans KB, & Leaf PJ (2008). Implementation of school-wide positive behavioral interventions and supports (PBIS) in elementary schools: Observations from a randomized trial. Education and Treatment of Children, 31 (1), 1–26. [Google Scholar]
  13. Brand S, Felner R, Shim M, Seitsinger A, & Dumas T (2003). Middle school improvement and reform: Development and validation of a school-level assessment of climate, cultural pluralism, and school safety. Journal of Educational Psychology, 95(3), 570–588. [Google Scholar]
  14. Bronfenbrenner U (2005). Ecological systems theory. In U. Bronfenbrenner (Ed.), Making human beings human: Bioecological perspectives on human development (pp. 106–173). Thousand Oaks, CA: Sage Publications Ltd. [Google Scholar]
  15. Catalano RF, Berglund ML, Ryan JA, Lonczak HS, & Hawkins JD (2002). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Prevention & Treatment, 5(1), 15a. [Google Scholar]
  16. Catalano RF, Gavin LE, & Markham CM (2010). Future directions for positive youth development as a strategy to promote adolescent sexual and reproductive health. Journal of Adolescent Health, 46(3),S92–S96. [DOI] [PubMed] [Google Scholar]
  17. De Laet S, Colpin H, Vervoort E, Doumen S, Van Leeuwen K, Goossens L, & Verschueren K (2015). Developmental trajectories of children’s behavioral engagement in late elementary school: Both teachers and peers matter. Developmental psychology, 51(9), 1292. [DOI] [PubMed] [Google Scholar]
  18. Duggins SD, Kuperminc GP, Henrich CC, Smalls-Glover C, & Perilla JL (2016). Aggression among adolescent victims of school bullying: Protective roles of family and school connectedness. Psychology of violence, 6(2), 205. [Google Scholar]
  19. Dulmus CN, Theriot MT, Sowers KM, & Blackburn JA (2004). Student reports of peer bullying victimization in a rural school. Stress, Trauma, and Crisis, 7(1), 1–16. [Google Scholar]
  20. Dunn TJ, Baguley T, & Brunsden V (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. [DOI] [PubMed] [Google Scholar]
  21. Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD,&Schellinger KB (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child development, 82(1), 405–432. [DOI] [PubMed] [Google Scholar]
  22. Espelage DL (2018). Section 3 Introduction: Bullying, Sexual Violence, and Suicide in Education The Wiley Handbook on Violence in Education: Forms, Factors, and Preventions ,321–326. [Google Scholar]
  23. Farrington DP, & Ttofi MM (2009). School-based programs to reduce bullying and victimization. The Campbell Collaboration, 6,1–149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Frydenberg E, Care E, Chan E, & Freeman E (2009). Interrelationships between coping, school connectedness and wellbeing Erica Frydenberg. Australian Journal of Education, 53(3), 261–276. [Google Scholar]
  25. Gregory A, Cornell D, Fan X, Sheras P, Shih TH, & Huang F (2010). Authoritative school discipline: High school practices associated with lower bullying and victimization. Journal of Educational Psychology, 102(2), 483. [Google Scholar]
  26. Gresham FM, & Elliot SN (1990). Manual for the social skills rating system. Circle Pines, MN: American Guidance Service. [Google Scholar]
  27. Gresham FM, Elliott SN, & Kettler RJ (2010). Base rates of social skills acquisition/performance deficits, strengths, and problem behaviors: An analysis of the Social Skills Improvement System—Rating Scales. Psychological assessment, 22(4), 809. [DOI] [PubMed] [Google Scholar]
  28. Hamilton MV (2008). Restorative justice: Reconceptualizing school disciplinary theory and practice. University of the Pacific. [Google Scholar]
  29. Higgins ET (1987). Self-discrepancy: a theory relating self and affect. Psychological Review, 94(3), 319–340. [PubMed] [Google Scholar]
  30. Kelley K (2017). The MBESS R package. Retrieved from https://cran.r-project.org/web/packages/MBESS/MBESS.pdf
  31. Konishi C, Miyazaki Y, Hymel S, & Waterhouse T (2017). Investigating associations between school climate and bullying in secondary schools: Multilevel contextual effects modeling. School psychology international, 38(3), 240–263. [Google Scholar]
  32. Lerner M, Lerner JV, & Benson JB (Eds.). (2011). Advances in child development and behavior (Vol. 41, pp. 195–228). London: Elsevier. [DOI] [PubMed] [Google Scholar]
  33. Little RJ, & Rubin DB (1989). The analysis of social science data with missing values. Sociological Methods & Research, 18,292–326. [Google Scholar]
  34. Low S, & Van Ryzin M (2014). The moderating effects of school climate on bullying prevention efforts. School Psychology Quarterly, 29(3), 306. [DOI] [PubMed] [Google Scholar]
  35. McCold P (2008). Evaluation of a restorative milieu: Restorative practices in context In Restorative justice: From theory to practice (pp. 99–137). Bingley, UK: Emerald Group Publishing Limited. [Google Scholar]
  36. McCold P, & Wachtel T (2002). Restorative justice theory validation In Restorative justice: Theoretical foundations (pp. 110–142). Devon, UK: Willan Publishing. [Google Scholar]
  37. McDonald RP (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum [Google Scholar]
  38. McNeely CA, Nonnemaker JM, & Blum RW (2002). Promoting school connectedness: Evidence from the national longitudinal study of adolescent health. Journal of school health, 72(4), 138–146. [DOI] [PubMed] [Google Scholar]
  39. Merrell KW, Gueldner BA, Ross SW, & Isava DM (2008). How effective are school bullying intervention programs? A meta-analysis of intervention research. School psychology quarterly, 23(1), 26–42. [Google Scholar]
  40. Muthén LK, & Muthén BO (2017). Mplus user’s guide. Los Angeles: Muthén & Muthén. [Google Scholar]
  41. O Brennan LM and Furlong MJ (2010). Relations Between Students' Perceptions of School Connectedness and Peer Victimization. Journal of School Violence, 9(4), 375–391. [Google Scholar]
  42. Overton WF (2006). Developmental psychology: Philosophy, concepts, methodology In Damon W and Lerner RM (Eds), Handbook of Child Psychology: Theoretical models of human development (pp. 18–88). Hoboken, NJ: John Wiley. [Google Scholar]
  43. Peleg-Oren N, Cardenas GA, Comerford M, & Galea S (2012). An association between bullying behaviors and alcohol use among middle school students. The Journal of Early Adolescence, 32(6), 761–775. [Google Scholar]
  44. R Core Team (2017). R Retrieved from https://cran.r-project.org/
  45. Ringwalt CL, Clark HK, Hanley S, Shamblen SR, & Flewelling RL (2009). Project ALERT: A cluster randomized trial. Archives of pediatrics & adolescent medicine, 163(7), 625–632. [DOI] [PubMed] [Google Scholar]
  46. Rudasill KM, Reio TG, Stipanovic N, & Taylor JE (2010). A longitudinal study of student–teacher relationship quality, difficult temperament, and risky behavior from childhood to early adolescence. Journal of school psychology, 48(5), 389–412. [DOI] [PubMed] [Google Scholar]
  47. Sallis JF, Owen N, & Fisher E (2015). Ecological models of health behavior. Health behavior: Theory, research, and practice, 5,43–64. [Google Scholar]
  48. Salmivalli C (2010). Bullying and the peer group: A review. Aggression and Violent Behavior, 15(2), 112–120. [Google Scholar]
  49. SAS Institute, Inc. (2016). SAS software. Cary, NC: SAS Institute:. [Google Scholar]
  50. Taylor RD, Oberle E, Durlak JA, & Weissberg RP (2017). Promoting positive youth development through school-based social and emotional learning interventions: A meta-analysis of follow-up effects. Child development, 88(4), 1156–1171. [DOI] [PubMed] [Google Scholar]
  51. Thapa A, Cohen J, Guffey S, & Higgins-D’ Alessandro A (2013). A review of school climate research. Review of educational research, 83(3), 357–385. [Google Scholar]
  52. Tharp-Taylor S, Haviland A, & D'Amico EJ (2009). Victimization from mental and physical bullying and substance use in early adolescence. Addictive behaviors, 34(6-7), 561–567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Thornberg R, Wänström L, Pozzoli T, & Gini G (2018). Victim prevalence in bullying and its association with teacher–student and student–student relationships and class moral disengagement: a class-level path analysis. Research Papers in Education, 33(3), 320–335. [Google Scholar]
  54. Ttofi MM & Farrington DP (2008a). Bullying: Short-term and long-term effects, and the importance of defiance theory in explanation and prevention. Victims & Offenders, 3(2), 289–312. [Google Scholar]
  55. Ttofi MM & Farrington DP (2008b). Reintegrative shaming theory, moral emotions and bullying. Aggressive Behavior, 34(4). 352–368. [DOI] [PubMed] [Google Scholar]
  56. U.S. Department of Education, Office of Safe and Drug-Free Schools. (2007). Mobilizing for evidence-based character education. Washington, DC: Government printing office. [Google Scholar]
  57. U. S. Department of Education. (2011). Average daily attendance (ADA) as a percentage of total enrollment, school day length, and school year length in public schools, by school level and state: 2003-04 and 2007-08, Digest of Education Statistics, Schools and Staffing Survey, Table 43. Washington, DC: Government printing office. [Google Scholar]
  58. U.S. Department of Education. (2012). Digest of Education Statistics, National Center for Education Statistics, Common Core Data Set, Table 46. Washington, DC: Government printing office. [Google Scholar]
  59. Vaz S, Parsons R, Passmore AE, Andreou P, & Falkmer T (2013). Internal consistency, test–retest reliability and measurement error of the self-report version of the Social Skills Rating System in a sample of Australian adolescents. PloS one, 8(9), e73924. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Weare K (2015). What works in promoting social and emotional well-being and responding to mental health problems in schools. London: National Children’s Bureau. [Google Scholar]
  61. Weatherson KA, O'Neill M, Lau EY, Qian W, Leatherdale ST, & Faulkner GE (2018). The protective effects of school connectedness on substance use and physical activity. Journal of Adolescent Health, 63(6), 724–731. [DOI] [PubMed] [Google Scholar]
  62. Young KR, et al. (2009). The Effects of School-Wide Positive Behavior Support on School Climate: A Middle School Logitudinal Study Presented on July 29 at the Office of Special Education Programs teacher education and professional development conference. Washington DC: US Department of Education. [Google Scholar]
  63. Yuan KH, & Bentler PM (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological methodology, 30, 165–200. [Google Scholar]

RESOURCES