A Cluster-Randomized Trial of Restorative Practices: An Illustration to Spur High-Quality Research and Evaluation

Abstract

Restorative Practices in schools lack rigorous evaluation studies. As an example of rigorous school-based research, this paper describes the first randomized control trial of restorative practices to date, the Study of Restorative Practices. It is a 5-year, cluster-randomized controlled trial (RCT) of the Restorative Practices Intervention (RPI) in 14 middle schools in Maine to assess whether RPI impacts both positive developmental outcomes and problem behaviors and whether the effects persist during the transition from middle to high school. The two-year RPI intervention began in the 2014–2015 school year. The study’s rationale and theoretical concerns are discussed along with methodological concerns including teacher professional development. The theoretical rationale and description of the methods from this study may be useful to others conducting rigorous research and evaluation in this area.

The restorative practices in schools literature is in need of rigorous evaluation studies. The purpose of this paper is to provide an example of such a study by describing the first randomized trial evaluating the impact of RPI on middle school youth. While the traditional rationale for restorative practices has been discussed by other authors in this special issue, restorative practices are important, also, because they contribute to the youth development literature. Unfortunately, many programs that are delivered in schools to address poor outcomes among adolescents focus only on specific negative behaviors and deficits, like poor communication and decision making skills, and label adolescents as problems in need of fixing without leveraging the natural strengths and resiliency of youth (Farrington & Ttofi, 2009). In contrast, this study was conceptualized with a youth development framework, which examined both traditional outcomes expected of restorative practices (e.g., decreases in disciplinary referrals) and outcomes related to positive youth development (e.g., improvements in social skills).

Theoretical Concerns

Using restorative practices in schools may be a more comprehensive prevention approach to address the complexity of adolescent development than more narrow prevention programs. Below we describe the complexity of adolescent development, the Restorative Practices Intervention, and the theory underlying it.

Need for Comprehensive Positive Youth Development Approach to Prevention in Schools

Adolescents’ problem behaviors often develop from the complex interaction of their risk factors, developmental changes, and the context in which they live, and may require a more comprehensive approach than programs that narrowly address certain risks. For example, effective drug and alcohol prevention programs (e.g., Project ALERT; Ellickson, Bell, & Harrison, 1993; Ellickson, McCaffrey, Ghosh-Dastidar, & Longshore, 2003) may address certain deficits (e.g., poor skills in decision making, communication), but are not always efficient for the significant investment made because they do not address the myriad of risk factors youth experience within the school environment. Further, other deficit-oriented approaches like strong discipline (e.g., zero tolerance) fail to help youth develop the relationships they need to thrive and can lead to unintended negative consequences such as posttraumatic stress disorder, anxiety, depression, academic failure, and school drop-out (Cameron & Sheppard, 2006), especially among minorities (Ashworth et al., 2008).

In contrast, there are approaches that target the “whole school” (students, staff, their relationships and climate), but research has revealed certain challenges in these approaches. Olweus’ Anti-bullying Program, one of the most widely disseminated anti-bullying prevention programs in the U.S., has yielded only short-term behavior changes and has not specified the program components responsible for behavior change (Farrington & Ttofi, 2009). The Positive Behavioral Interventions and Supports (PBIS) program, which is effective for addressing behavioral issues among elementary school students, has not yet shown effectiveness in middle school settings (Bradshaw, Reinke, Brown, Bevans, & Leaf, 2008).

Another promising approach, comprehensive positive youth development (C-PYD) programs, leverage youth’s innate potential for positive growth to improve their health and quality of life through support, opportunities, and positive challenges. The strength of C-PYD programs is that they specifically focus on the developmental milestones of youth, recognizing the heterogeneity and interactive nature of both positive and negative individual and environmental influences. By building competencies (e.g., moral, social), increasing healthy bonding with peers and adults, and intervening for at least nine months or longer (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2002), C-PYD programs have been shown to impact school outcomes, including school readiness, academic achievement, life satisfaction (Benner, Graham, & Mistry, 2008; Wentzel, Filisetti, & Looney, 2007) as well as problem behaviors including drug use (Tebes et al., 2007). However, much of this research was quasi-experimental, did not assess sustainability of initial results, and did not specify the mechanisms of impact (Catalano, Berglund, Ryan, Lonczak & Hawkins, 2004). Therefore, more research is needed that is rigorous (i.e., RCTs), examines whether the positive impacts of C-PYD persist, and specifies the mechanisms through which PYD influences problem behaviors (Catalano, Gavin, & Markham, 2010).

One C-PYD, Restorative Practices Intervention or RPI, shows particular promise because it is a two-year whole school intervention grounded in a strong theoretical basis (psychology of affect and ecological systems theories) that encourages the practices to be integrated into the routine school environment. It has strong quasi-experimental evidence supporting its effectiveness in reducing conflict, disciplinary referrals, and disruptive behavior, while improving positive relationships, academic achievement, graduation rates and the overall school environment (Hamilton, 2008; McCold, 2008). The nature of the RPI training and consultation (staff are eventually trained-to-train new staff in RPI) suggests RPI can be sustained over time.

Despite its promise, like other C-PYD approaches, there has not been rigorous scientific study of RPI’s impacts and underlying mechanisms. One hypothesis is that RPI improves a school’s environment and peer relationships (both of which mediate risk), which subsequently improves positive developmental outcomes and reduces problem behaviors. However, more research is needed to understand whether RPI improves school connectedness and peer relationships; prevents and reduces problem behaviors like bullying; improves positive developmental outcomes; and whether any of these impacts persist over time.

The Restorative Practices Intervention

The International Institute for Restorative Practices (IIRP) developed RPI in 1999. RPI involves training all school staff on how to enact 11 “Essential Elements” (see Table 1), a continuum of practices that range from informal (e.g., using “affective” statements that communicate feelings) to formal (e.g., hosting a restorative “circle” or “conference” where participants are encouraged to express emotions and form emotional bonds). A significant component of RPI, circles and conferences, are group meetings designed to take place between school staff and students. There are several types of circles and conferences (Table 1). Circles can be initiated by students or staff to establish ground rules (proactive circle) or as a planned way to respond to inappropriate behavior affecting a group of students or entire class (restorative circle). Conferences can be an immediate response to low-level disagreements or conflicts between two people (impromptu conference) or a planned response to serious or repeated patterns of behavior (restorative conference). There are written guidelines that govern how schools are to use each type of circle or conference (and other RPI practices) and corresponding indicators of proficiency with each (Table 1). For example, in RPI-proficient schools, proactive circles are expected to make up about 80% of all circles implemented.

Table 1.

11 Essential Restorative Practices and Sample Indicators of Proficiency in Each Practice

Essential practices	Sample indicators of proficiency in practice
1. Affective statements	Use “I” statements; make students aware of positive or negative impact of their behavior; focus on behavior; encourage students to express their feelings
2. Restorative questions	Reflect standard restorative questions (What was the harm? How has it impacted you? What needs to happen to make things right?); Require a response
3. Small impromptu conferences	Use to resolve low-level incidents between 2 people; Take place as soon as the incident has occurred; Use standard restorative questions; Use affective statements; Ask students to conduct a specific activity to repair harm from the incident
4. Proactive circles	Comprise at least 80% of circles conducted; Use to set behavioral expectations (e.g., for academic goal setting or planning, to establish ground rules for student projects, to monitor or build understanding of academic content); Use standard restorative questions; Use affective statements; Run by students, after being facilitated 5 times
5. Responsive circles	Comprise no more than 20% of circles at the school; Use in response to behavior or tensions affecting a group of students or entire class; Require all people involved to play a role; Use standard set of restorative questions; Use affective statements
6. Restorative conferences	Use in response to serious incidents or a pattern of repeated less serious incidents; Use standard restorative questions, affective statements, and a trained facilitator
7. Fair process	Allow students to provide input into decisions; Explain the reasoning behind decisions to the students affected; Clarify expectations so students understand implications of decision, specific expectations for carrying out the decision, and consequences for not meeting the expectations
8. Reintegrative management of shame	Avoid stigmatizing wrong doers; Discourage dwelling on shame; Acknowledge worth of person while rejecting unacceptable behavior (i.e., separate deed from the doer)
9. Restorative staff community	Use restorative practices to resolve conflicts and proactive circles to build sense of community
10. Restorative approach with families	Use restorative practices during interactions with family members, including proactive circles that focus on intentional communication of positive student behavior and academic achievement
11. Fundamental hypothesis	Have high expectations for behavior; do not ignore inappropriate behavior; use the appropriate mix of control/pressure and support; minimize use of staff facilitators

A unique feature of RPI is that practices specified in the 11 Essential Elements are meant to be integrated into all relevant aspects of school life. Also, all school staff are trained in these elements because of their cross-cutting nature. For example, all school staff that interact with youth (e.g., maintenance staff, bus drivers, cafeteria workers) are first trained in the use of affective or “I” statements (Element #1: e.g., I feel upset with the class now because the rules about talking we set up in the beginning of the year are not being respected) and restorative questions (e.g., What needs to happen to make things right?). This restorative question/answer approach serves as the core communication practices of the RPI model and thus is meant to be used in other restorative practices (i.e., circles/conferences) as well as all relevant general discourse between teachers and students.

Other examples of key cross-cutting practices are used primarily by administrators, teachers, and other instructional staff who are trained to facilitate a fair and transparent process with students and recognize and manage shame among students. School staff are encouraged to use the restorative practices to build relationships and resolve staff issues (restorative staff community), as well as when interacting with parents (restorative approach with families). All restorative practices encourage acting “with” (not “for”) youth and setting high expectations. A Restorative Practices School is one that is proficient in all 11 Essential Elements, as determined through implementation tracking used by IIRP and in this study.

Theory of RPI

RPI combines ecological systems (Bronfenbrenner, 2000; Cicchetti & Lynch, 1993) and psychology of affect theories in one model (Figure 1) (Lerner & Steinberg, 2004; Ttofi & Farrington, 2008). First, RPI operationalizes and expands psychology of affect theory, which is an alternative to punitive approaches often used by schools. The psychology of affect explains how RPI achieves improved behavior and increased connectedness through three psychological mechanisms: 1) RPI maximizes positive affect through proactive practices such as “restorative circles” (Table 1), which are aimed at developing closer bonds and relationships among youth. 2) RPI minimizes negative affect by training teachers to help students engage in practices that ensure offenders can take public responsibility for their behavior and reintegrate into normal community life. 3) RPI encourages staff and students to freely express emotions through training in such practices as affective statements and questions. These improvements then feed into ecological systems theory, which posits that behavior is determined by multiple causes and is sensitive to environmental influences (Overton & Ennis, 2006; R. Lerner, Almerigi, Theokas, & J. Lerner, 2005). RPI’s practices proactively build relationships and respond to specific school incidents, which modify the school environment to encourage youth to more actively participate. Youth participation promotes further advances in positive development and the positive environment of their school. That, in turn, can promote the three mechanisms stated above. The result is a highly participatory school culture that effectively regulates affect, consistently builds and repairs social-bonds, and promotes connectedness and relationships–key mechanisms to prevent high-risk and harmful behaviors.

Figure 1.

Restorative Practices Intervention Theoretical Model. This figure illustrates theoretical model that guided the design of the Restorative Practices Intervention explicitly showing the linkages between the psychology of affect and the ecological systems students reside in.

Also, RPI has the three core components of an optimal C-PYD (Lerner & Steinberg, 2004): (1) sustained relationships with adults—created via teacher-student dialogue in circles and conferences (#3–5 in Table 1); (2) skills building—RPI uses teachers and other staff to coach students on seven of the 11 essential practices (#1–5, 7 and 8); and (3) application of skills building—e.g., as students develop proficiency in the seven essential practices they are coached to perform, school staff transfer responsibility for running the circles and some conferences over to students. Teachers continue to facilitate restorative conferences for serious and or chronic behavior problems.

Methodological Concerns

This study is the first to use a randomized design to assess the impact of RPI on key outcomes at the student, peer, and school levels. Below we describe the study design, sites, intervention delivery and professional development, measures, planned analyses, and anticipated challenges to the study.

Study Design

This study is a five-year, cluster-randomized controlled trial assessing the fidelity of implementation of RPI, as well as the effects of RPI on school environment, developmental outcomes and problem behaviors in 14 middle schools in Maine. We matched schools on demographic and academic and disciplinary data, and then randomized them so that seven schools are receiving RPI and seven are not. RPI is implemented over two school years. After two years, the external support for RPI implementation will be moved to the remaining seven schools, making it possible to test the sustainability of any gains in the initial implementation schools. We chose Maine because we have developed a strong relationship with both the PYD practitioners and schools there. Locating all the sites in Maine has other benefits including controlling for state-to-state variations that might influence outcomes (e.g., structure of school administration).

Figure 2 shows a conceptual model for the study, emphasizing how RPI practices are hypothesized to influence the school environment, school staff, peer relationships, and subsequently student developmental outcomes and problem behaviors. The study and the measures selected are guided by ecological systems theory, which states that Students are nested within multiple levels of interaction (Figure 2) (Bronfenbrenner, 2000; Cicchetti & Lynch, 1993). Students interact with Peers, which influence each other. Similarly, Students interact with the School environment and staff which influence both the individual Student and their Peers (as shown by double headed arrows). These interactions mediate youth developmental outcomes and problem behaviors. When well implemented, RPI is focused on improving both peer-to-peer communication and relationships and the school environment.

Figure 2.

Study Conceptual Model. This figure illustrates the conceptual model used to guide the study design and describes the data collection for each aspect of the model (e.g., student level, school level, etc.).

Study Sites

The 14 middle schools have about 3500 students, with each averaging 250 11–13 year-olds (from 38 to 506) enrolled. Although there is some variability in school size and configuration, each has co-located sixth through eighth grades. The ethnic/racial make-up is similar across schools: about 95% White, 2% Black, and 1% mixed race or other. The 14 schools are spread across the state of Maine in rural and suburban areas. Urban schools do present complex challenges. However, rural and suburban schools serve 35% and 23% of all U.S. students respectively, as compared with 29% of students in urban centers (Hoffman, 2009). Further, Maine youth experience relevant risk factors to the same extent, or more, as youth across the U.S. (see Table 2). Therefore, our findings should be relevant to most U.S. schools.

Table 2.

Risk factors for Maine youth are similar to risk factors for youth across the country

	% experiencing
Risk factors	Maine	Average U.S.
Dropout rate (2008)	9.12	3.51
Percent of children whose parents lack full-time work (2008)2	31	31
Percent of children in poverty (2010)2	17	20
Percent of children in single parent families (2010)2	33	34
	% experiencing among 9th–12th graders
	Maine	Average U.S.
Attempted suicide3	6.8	6.3
Bullied on school property3	22.4	19.9
Did not go to school because they felt unsafe at school or on their way to or from school on at least 1 day3	5.5	5.1
Threatened or injured with a weapon on school property one or more times3	7.7	7.7
Hit, slapped, or physically hurt on purpose by their boyfriend or girlfriend3	15.4	9.8

¹Note: Adapted from: http://nces.ed.gov/pubs2011/dropout08/findings1.asp. No copyright.

²Note: Adapted from: http://www.maine.gov/education/gradrates/gradrate0910.html. No copyright.

³Note: From the Youth Risk Behavioral Surveillance – 2010, Department of Health and Human Services, Centers for Disease Control and Prevention, Surveillance Summaries.

Study recruitment

At faculty meetings in the beginning of the school year, study staff spoke with administrators, teachers, educational technicians, and others that directly support students in sixth and seventh grades to inform them of the study, explain the requirements, and request their participation in both observational and survey measures. All sixth and seventh graders receive RPI and were recruited to participate in the student survey via letters sent home to parents/guardians who could opt their child out. Consistent with other research on RPI, active parent refusal was very low (3%). We also obtain youth assent at the time of the online student survey. All study protocols were reviewed and approved by the RAND Human Subjects Protection Committee.

RPI Delivery and Professional Development

After initial engagement with schools in May 2014, the active RPI intervention began in Fall 2014 with training of intervention school staff (including teachers, educational technicians, specials teachers, school counselors, administrators, and all other staff contractually obligated to attend professional development days) by IIRP certified instructors assisted by Maine-based staff supported by the project. As is typical for RPI, training was approximately 2.5 days (delivered via professional development) of onsite skills training for all school staff across the 11 essential practices listed in Table 1, with an emphasis on the first six. As prescribed by RPI, each school then formed multiple Project Leadership Groups (PLGs) consisting of naturally occurring clusters of teachers (e.g., English department). All staff within a school were expected to join one PLG. PLGs meet monthly and read and complete exercises from The Restorative Practices Handbook (Costello, J. Wachtel, & T. Wachtel, 2009), that reinforce the 11 Essential Elements.

Consistent with IIRP’s process to facilitate whole school change, the participating schools have been receiving a series of supports for implementation. First, school leaders as well as other key members or champions of RPI implementation participate in monthly calls with Maine-based and IIRP staff. These champions, identified by school leaders, are staff that are particularly interested in and believe in the promise of restorative practices. These calls address successes, challenges with implementation, enablers of implementation, and tailor support to the specific needs of the school or staff. Second, a Maine-based staff person visits each school, on average, one time per month to observe implementation and provide support to administrators and teachers. Third, staff from IIRP has been conducting two school visits in the first year: one at the close of the first semester and the other at the close of the school year. During these meetings, IIRP and school staff use tools to develop specific work plans, set benchmarks for proficiency and monitor progress (e.g., implement proactive circles daily)¹. After one year, typically schools are expected to gain proficiency in the first six Essential Elements.

At the start of the second year of active RPI implementation support, schools are scheduled to receive two additional days of training during professional development focused on Essential Elements 7–11. For example, schools receive training to form a special multi-disciplinary team of teachers, counselors, social workers and administrators to facilitate Restorative Conferences, the one essential practice that addresses more serious conduct problems in the school. The training also addresses sustainability planning—i.e., training select school staff to provide ongoing training and professional development in RPI to other staff, so the intervention can continue.

Measures

We are assessing RPI fidelity and its impact at the school (and staff), peer, and student levels (in italics in Figure 2).

RPI fidelity

RPI at the seven intervention schools will be assessed along four dimensions of fidelity—dosage, adherence, quality of delivery, and participant response (Mihalic, 2004; Mihalic & Irwin, 2003)—with the same measures used extensively by the IIRP and supplemented with tools used in prior implementation research studies (i.e., dosage assessment; Chinman et al., 2008) Dosage. Based on measures used extensively by IIRP, every Fall and Spring during the two year intervention period, and also during a third year following the removal of the active intervention support, all staff will be asked to answer an online implementation survey of ten questions about how many circles and conferences they ran per month or week and to what extent they used other restorative practices (1 = Not at all to Always = 5). During the Spring administrations of the implementation survey, staff will receive these ten questions plus an expanded set of questions about use of restorative practices. A reworded version of these expanded questions will also be on the student survey in the intervention schools.

Adherence, quality of delivery, participant response

In-person observations will be conducted at the seven schools receiving RPI on a monthly basis during the two year intervention period to assess circles and conferences, using the RP-Observe (Gregory et al., 2014), and during regular class time, using the RP-Observe Supplement (Chinman, Acosta, Ebener, & Phillips, 2015). Each month, trained observers will conduct one Observe and one Supplement at each intervention school, assessing a different teacher each month chosen at random. The RP-Observe is designed to assess the quality with which Proactive and Responsive Circles and Restorative Conferences are implemented by students and staff, across 10 dimensions that underlie the RPI intervention (on a 1–7 scale): circle rules, positive respect and responsiveness between students and between adults and students, autonomy, relevancy, risk-taking, and problem-solving. In psychometric testing at two large high schools, eight scales of the RP Observe were reliable (intraclass correlations range from .56–.78) and two had low reliability (automony, .42 and student responsiveness, .15). RP-Observe demonstrated construct validity as all the scales correlated in expected directions with student and teacher surveys used to assess RPI practices (Gregory et al., 2014). The RP-Observe Supplement was developed specifically for this study. Borrowing from, and using the same scale as the RP-Observe, it focuses on certain Restorative Practices that would be present during regular instructional time in a school facile with the approach, including: the greeting of students by teachers, teacher-student rapport, student input, setting behavioral expectations, consistency of discipline, response to rule breaking, use of affective statements, and feedback to students about behavior.

School climate

School environment at all schools will be assessed three ways—via observation and a survey of both staff and students. The observational measure is the Youth Program Quality Assessment (YPQA), in which study staff will observe at least five separate class periods during a school day at all 14 schools and make ratings on several scales in four domains: safe environment, supportive environment, interaction, and engagement—domains found to be critical to PYD interventions like RPI. The ratings will be made at baseline, at the end of the second school year (end of RPI), and one year after the end of RPI. Validation studies found that the YPQA had appropriate predictive validity needed to model process-outcome relationships where quality scores explain sizable amounts of variance in youth-level data (Blazevski & Smith, 2007; Yohalem & Wilson-Ahlstrom, 2010).

Select scales from the Inventory of School Climate, modified for use with school staff (Brand, Felner, Shim, Seitsinger, & Dumas, 2003) will be administered to the RPI-assigned schools each Fall and Spring during the two year active intervention period as well as during the subsequent third year in which the active RPI support will be moved to the other seven schools. Control schools will only complete the school climate measures at the start of each year (i.e., Fall) within that same three year period. The scales—Consistency and Clarity of Rules and Expectations, Safety Problems, Teacher Support, Positive Peer Interactions, and Student Input Into Decision Making—were chosen because they are theorized to be impacted by RPI and in a series of field tests in a large sample across multiple schools, demonstrated good reliability (one year test re-test ranged from .69 to .81; internal consistency ranged from .70 to .76) and validity (explained significant between-school variance in measures of academic, behavioral, and socio-emotional adjustment; Brand et al., 2003).

Student outcomes

Students will also complete the same items of school climate and RPI implementation as staff, along with six specific sets of youth outcomes, some of which replicate measures used in previous studies of RPI: (1) School Connectedness – A five-item scale (1=strong disagree to 5=strongly agree) from the National Adolescent Health Study will measure adolescents’ perceptions of closeness to peers, happiness at their school, belonging at school, fair treatment by teachers, and safety at school (McNeely, Nonnemaker, & Blum, 2002). The scale has shown strong internal consistency (alpha = .78; Anderman, 2002). (2) Peer Relationships – A nine-item scale developed by Acosta (2003) will measure peer attachment and influence (1=Never to 6=Always) in areas such as receiving encouragement from peers to do well in school, confiding in peers, emulating peers, and considering peers’ reactions before acting. Both attachment and influence subscales are reliable (alpha = .71 for both) and sensitive to detecting changes over time (Acosta, 2003). (3) Social Competency – The Social Skills Improvement System-Rating Scale (SSIS-RS) is being used to assess students’ perceptions of prosocial behavior in five domains: cooperation, assertion, responsibility, empathy, and self-control. Students self-rate their behavior on a 4-point scale (0 = Never, 1 = Seldom, 2 = Often, and 3 = Almost Always). The SSIS-RS is a revision of the SSRS with updated national norms. For ages 13–18, the SSIS-RS has alpha coefficients above .70 for all scales, test-retest indices range from .77 to .92, and the SSIS-RS is positively correlated with the SSRS (.36; Klaussen & Rasmussen, 2013). (4) Bullying – The Communities That Care Survey questions (Arthur et al., 2007; Arthur, Hawkins, Pollard, Catalano, & Baglioni, 2002) are being used to assess prevalence and frequency (not at all, somewhat, a whole lot) of verbal, physical, and cyber bullying behaviors in the past 30 days (perpetrator and victim). These questions were recently used in a study that found associations between bullying behaviors and alcohol use among middle school students (Peleg-Oren, Cardenas, Comerford, & Galea, 2010). (5) Academic achievement, and (6) Disciplinary referrals –Grades and incidences of detention, suspension, or expulsion will be used as indicators of academic achievement and disciplinary referrals, respectively. Standardized test scores were considered, but Maine has used a different test each cycle over the last few years, precluding longitudinal analysis of that data. Participating schools do not grade on a distribution, thus, student grades could improve school-wide as an outcome of RPI.

The student survey was administered at baseline (in Fall before RPI started), and will be repeated in the Spring after two years (end of RPI), three years (one year post active intervention), and four years (sixth graders only, to assess transition to high school). Collected through Maine’s standardized school administrative data system, participating schools will provide academic and disciplinary data for five years (year before baseline plus study years 1–4).

Hypotheses and Data Analysis

It is hypothesized that the school environment of RPI schools will improve significantly more on YPQA ratings than in schools without RPI. Changes in YPQA scores will be compared between groups and fidelity scores will be correlated with percent changes in the YPQA between baseline and Year 1, and baseline and Year 2 of the intervention to determine whether fidelity is associated with YPQA scores. It is also hypothesized that students in RPI schools will show significantly greater improvements on developmental outcomes and fewer problem behaviors than students at schools without RPI, both during their middle school years and after their transition from middle to high school. We will use a hierarchical regression model, structured in a growth curve fashion, to compare students from RPI and control schools, incorporating YPQA climate ratings (or fidelity scores in alternative models) as a second-level predictor. Finally, we hypothesize that students and staff in RPI schools will report a greater improvement in school climate than those at schools without RPI. We will use a similar hierarchical regression models to test these hypotheses. Additional analyses (e.g., correlating school climate with students’ school connectedness) will be conducted to determine the exact mechanism of action for RPI.

Design Challenges

Randomizing by school creates “clusters”—sites nested within study condition—which can reduce the effective sample size. However, the analyses will use appropriate statistical techniques to account for clustering. If a major community-wide change occurs that might affect a school’s outcome measures, sensitivity analyses will be conducted with and without an affected site. A matched design was chosen since it yields more powerful analyses, given that there are only seven schools per condition (Martin & Huebner, 2007). School randomization also reduces potential contamination from when individuals are assigned to specific classroom level interventions (Daunic, Smith, Brank, & Penfield, 2006). Further, randomizing by site is a more “real world” test because models like RPI are either adopted at the school level or not. Given most schools are not systematically addressing school climate and youth outcomes in a manner like RPI, efficacy should first be determined compared to this norm. Adding an intervention arm of a non-PYD whole school intervention was considered. However, there is not a comparable intervention because RPI is integrated into the school day and does not require additional time from students as other whole school interventions do. Therefore, we determined the most appropriate design was to compare RPI to usual school conditions.

Trial Status and Conclusion

The two-year RPI intervention started with the 2014–15 academic year, which is almost finished. RPI training is typically completed at the start of each school year. However, this training was somewhat delayed by limited professional development days being available from the participating school districts. For example, in five schools, the first day of training occurred before the start of the school year, the second day occurred in October, and the final half day occurred by the November break. In two schools, the first full day of training occurred in October, the second by the December break, and the half-day in the second semester of the school year. However, plans were made to integrate these delayed trainings into other meetings during the school day earlier in the year. In addition, a severe winter in Maine that resulted in numerous school day cancelations caused further delays in implementation and data collection. Despite the delays, all schools have formed PLGs and the PLGs and consultation meetings are occurring on a monthly basis. Regarding data collection, the baseline YPQA (observational measure of school climate) was completed at all 14 schools. To date, 68% of staff agreed to participate in the survey and in the RP classroom observations. Across all 14 schools, about 92% of consented staff completed the baseline survey (includes questions on school climate and restorative practice). The second staff survey is currently underway. Maine-based staff have conducted 53 RP-Observes since November 2014 and 29 RP-Supplements since January 2015, surpassing the study goal of one per month, per school for both instruments. Across all 14 schools, about 81% of students completed the baseline survey. Efforts are currently underway to create feedback reports for each RPI-assigned school using the YPQA, RP-Observe and Supplement, and two rounds of the staff survey to inform second year implementation planning.

This study has the potential to document, for the first time utilizing rigorous scientific methods, whether a C-PYD like the Restorative Practices Intervention impacts both positive developmental outcomes and problem behaviors and whether the effects persist during the transition from middle to high school. This information is critical as states and particularly the federal government, are allocating a great deal of funds to implement programs targeting specific risk factors. Findings also have the potential to advance the theory of positive youth development by empirically validating for the first time that restorative justice principles can be applied universally as an environmental approach to promote positive youth development.