A randomized controlled trial of a trauma-informed school prevention program for urban youth: Rationale, design, and methods

Abstract

Introduction:

Youth in disadvantaged urban areas are frequently exposed to chronic stress and trauma, including housing instability, neighborhood violence, and other poverty-related adversities. These exposures increase risk for emotional, behavioral, and academic problems and ultimately, school dropout. Schools are a promising setting in which to address these issues; however, there are few universal, trauma-informed school-based interventions for urban youth.

Methods Design:

Project POWER (Promoting Options for Wellness and Emotion Regulation) is a randomized controlled trial testing the impact of RAP Club, a trauma-informed intervention for eighth graders that includes mindfulness as a core component. Students in 32 urban public schools (n = 800) are randomly assigned to either RAP Club or a health education active control group. We assess student emotional, behavioral, and academic outcomes using self-report surveys and teacher ratings at baseline, post-intervention, and 4-month follow up. Focus groups and interviews with students, teachers, and principals address program feasibility, acceptability, and fidelity, as well as perceived program impacts. Students complete an additional self-report survey in ninth grade. Schools provide students’ academic and disciplinary data for their seventh, eighth, and ninth grade years. In addition, data on program costs are collected to conduct an economic analysis of the intervention and active control programs.

Discussion:

Notable study features include program co-leadership by young adults from the community and building capacity of school personnel for continued program delivery. In addition to testing program impact, we will identify factors related to successful program implementation to inform future program use and dissemination.

1. Introduction

Inner-city youth are at high risk for exposure to community violence, family disruption, and other traumatic stressors linked to poverty (Breslau, Wilcox, Storr, Lucia, & Anthony, 2004; Buka, Stichick, Birdthistle, & Earls, 2001; Evans, 2004). Trauma exposure impairs stress management (Compas, 2006), increasing risk for emotional, behavioral, and academic problems (Romeo, 2010) (Hinshaw, 1992; Needham, Crosnoe, & Muller, 2004; Reid, Gonzalez, Nordness, Trout, & Epstein, 2004). The National Task Force on Children Exposed to Violence concluded that costs of children’s trauma exposure are “staggering” due to the burden on public systems of care and victims’ lifetime loss of productivity (Listenbee et al., 2012).

Schools are positioned to mitigate these effects by delivering services to improve the mental health of trauma-affected youth (Listenbee et al., 2012; Atkins et al., 2010). Specifically, middle schools offering universal trauma-informed interventions may provide valuable skills and supports to students prior to high school entry, thereby increasing student academic success and reducing high school dropout. The few universal mental health programs for upper middle school students generally do not target depression, anxiety, and trauma symptoms (Greenberg, Domitrovich, & Bumbarger, 2001). Most mental health interventions are not universal and are delivered only to students screening positive for symptoms or trauma exposure (e.g., Cognitive Behavioral Intervention for Trauma in Schools (Jaycox, 2004)). Although targeted mental health approaches have merit, barriers to in-school screening include stigma, resource constraints, and difficulty identifying all students who would benefit from intervention. In addition, targeted approaches that pull at-risk students from the classroom may not embed interventions sufficiently within general school practices to facilitate skills generalization (Atkins et al., 2010). Universal delivery, on the other hand, has potential to benefit most students and reduce burden on schools’ limited resources for addressing mental health.

To address these limitations, our research team adapted Structured Psychotherapy for Adolescents Responding to Chronic Stress (SPARCS; DeRosa et al., 2006; DeRosa & Pelcovitz, 2009), a group trauma treatment for urban adolescents, for use as a universal prevention program for eighth graders (Mendelson, Tandon, O'Brennan, Leaf, & Ialongo, 2015). The adapted intervention, called RAP (Relax, be Aware, do a Personal rating) Club, is a 12-session group prevention program delivered during school over a period of six weeks in the beginning of the school year. In a pilot randomized study, RAP Club improved teacher-rated student social, emotional, behavioral, and academic outcomes (Mendelson et al., 2015). The current study is the first randomized controlled trial to evaluate RAP Club’s efficacy.

The intervention and study design include several notable features: (1)RAP Club includes mindfulness as a core component, an approach that has rarely been evaluated in a trauma-informed school intervention context; (2) RAP Club is compared with an active control condition (Healthy Topics, a health education program); (3) programs are co-facilitated by young adults from the community to increase student buy in and relatability; and (4) training and support is provided for school personnel to deliver programming following the study.

2. Methods/Design

2.1. Study design and objectives

Project POWER (Promoting Options for Wellness and Emotion Regulation) is a school-based randomized controlled efficacy trial of the RAP Club program. Eighth grade public school students are randomized within schools to receive either the 12-session RAP Club program or the 12-session Healthy Topics program. As displayed in Fig. 1, this study will test the hypothesis that compared to the active control program (i.e., Healthy Topics), students randomized to RAP Club will have improved self-regulatory capacities, such as emotion regulation and decision making (underlying processes), emotional and behavioral functioning (emotional and behavioral outcomes), and academic performance (education outcomes). Additional aims include exploring potential moderators (e.g., baseline trauma exposure) and mediators (e.g., changes in self-regulation and emotional symptoms) of RAP Club’s effects and evaluating factors related to program implementation. Program implementation factors include the cost of delivering RAP Club versus Healthy Topics and perceptions of key stakeholders (principals, teachers, students) regarding intervention acceptability and sustainability.

Figure 1.

Intervention theory of change.

2.2. Participants and setting

The study sample consists of eighth graders attending public schools in Baltimore City, MD, that serve disadvantaged communities experiencing high rates of crime, poverty, and homelessness. In the current academic year (2018-19), Baltimore City Public School data indicate 79% of students are African American, 8% White, and 11% Hispanic/Latino. The majority of students (53%) are low-income based on their eligibility for programs including Temporary Assistance for Needy Families and Supplemental Nutrition Assistance (retrieved from https://www.baltimorecityschools.org/district-overview). In total, we aim to recruit 32 schools and 800 participants, 400 in each intervention arm. Eighth graders in general education classrooms at participating schools are eligible to take part in the study; students in self-contained special education classrooms are not eligible for participation due to their different emotional and behavioral needs.

2.2. Recruitment and consent/assent procedures

We aim to recruit 8 schools per year over the first four project years, for a total of 32 schools. School recruitment is conducted via emails to Baltimore City Public School principals, followed by informational meetings with principals who express interest in project participation. Schools are enrolled as study partners if they meet the following criteria: the administration is interested in participating, the necessary program and evaluation time can be provided during the school day, appropriate school personnel are available to be trained in RAP Club and Healthy Topics, and no other programming is currently being offered to the eighth grade that overlaps substantially with RAP Club or Healthy Topics.

Based on our prior research, we estimate enrolling a final average of 30 students per school for an average sample of 25 students per school in the 4-month follow up assessment (83% retention) to account for school transfers and other attrition. All participating students must provide signed parental permission and signed assent. Obtaining parental permission is challenging in our target population because financial pressures result in frequent changes of address and inactivated phone numbers. We follow a multi-pronged recruitment strategy successful in our prior work: we mail study descriptions and permission forms home, visit schools and send forms home with students, and follow up as needed by phone, obtaining parent phone numbers from students if school records are not current. Students complete the assent form at school with a trained research assistant and have an opportunity to ask questions. The Institutional Review Boards at the Johns Hopkins Bloomberg School of Public Health and Baltimore City Public Schools approved all study procedures and materials.

2.3. Randomization

2.3.1. Sequence generation and allocation concealment.

After recruitment and baseline surveys are completed, participants at a given school are randomly allocated to the intervention or control arm using a random number generator. Randomization is not stratified by sex, as the percentage of boys and girls is approximately equal in this school district, and it is assumed that sex distribution will be similar to the district population within study arms. Post-randomization analysis will explore whether this assumption was met, and analytic tools such as inverse probability weighting can be used if needed. The project biostatistician oversees the randomization process, and the Project Coordinator communicates students’ assignments to the intervention facilitators at each school. Other investigators do not have access to the list of randomized students or allocation system.

2.3.2. Blinding

Except for the biostatistician, investigators only have access to anonymized baseline data at the end of recruitment; study groups are categorized as A and B without identification of intervention or control status. Data will be blinded until analyses are finalized and approved by all investigators. Students and school personnel are not informed of study hypotheses. Students are not informed of study hypotheses.

2.4. Data collection and measures

2.4.1. Data collection.

Quantitative data includes student surveys, teacher rating scales, and data from academic records (see 2.5.2). At each school, students complete pre- and post-intervention surveys in the fall semester, a follow-up survey in the spring semester, and another follow-up survey in the spring semester of ninth grade. Two eighth grade teachers who are not being trained to deliver the RAP Club or Healthy Topics program complete rating scales at baseline, post-intervention, and spring semester follow-up to evaluate each participant’s classroom behavior, academic performance, and emotional functioning. As these teachers are not trained in the study curricula and do not observe program sessions, they are less likely to have preconceived notions about how each program should affect student functioning. Teachers are compensated $4 for each survey they complete at each assessment point. We also receive academic performance and disciplinary data from records maintained by the school system, including standardized test scores, grades, attendance, and number of suspensions and office visits. These data are collected for each participant at three time points: the year prior to participation (end of seventh grade), year of participation (end of eighth grade), and year after participation (end of ninth grade).

Following the intervention, we conduct focus groups with students, principals, and teachers to solicit qualitative feedback about experiences in the RAP Club and Healthy Topics programs (see 2.5.3). We also collect data on fidelity of implementation for program trainings and program delivery (see 2.5.4) and data on program costs (see 2.5.5).

2.4.2. Quantitative measures for study outcomes, moderators, and mediators.

Key measures for each domain from our intervention theory of change are shown in Table 1. We selected psychometrically sound student-reported and teacher-reported measures that have been developed or used with diverse adolescent samples. Our primary outcomes are education outcomes, including teacher-rated performance and school record data on grades, standardized test scores, attendance, and disciplinary sanctions. Secondary outcomes include emotional and behavioral outcomes (e.g., externalizing symptoms) and underlying processes (e.g., self-regulation); we will also test these more proximal outcomes as potential mediators.

Table 1.

Measures of underlying processes, intermediate outcomes, education outcomes, and covariates and moderators.

Domain	Construct	Measure	Informant
Underlying processes	Rumination	Children’s Response Style Questionnaire (Abela, Rochon, & Vanderbilt, 2000)	Student
	Coping/Decision making	Children’s Coping Strategies Checklist (Ayers, Sandler, West, & Roosa, 1996)	Student
	Self-efficacy	NIH Toolbox Self-Efficacy Scale (Kupst et al., 2015)	Student
Emotional and behavioral outcomes	Social-emotional competence	SCS Social-Emotional Competence (Kam, Greenberg, & Walls, 2003)	Teacher
	Emotion/behavior problems	Strengths & Difficulties Questionnaire (Goodman, 1997)	Teacher
	Internalizing and externalizing	Strengths and Difficulties Questionnaire (Goodman, 1997)	Teacher
	Depressive symptoms	Children’s Depression Inventory (Kovacs, 1992)	Student
	Dysregulated Behavior	SCS Dysregulated Behaviors (Kam, & Greenberg, 1998)	Teacher
	PTSD symptoms	Child PTSD Symptom Scale-Revised (Foa, Johnson, Feeny, &Treadwell, 2001)	Student
	Behavioral dysfunction Emotional distress	Youth Outcomes Questionnaire (Wells, Burlingame, & Rose, 2003)	Student
	Anxiety symptoms	PROMIS Anxiety Symptoms (Pilkonis et al., 2011)	Student
	Flourishing	NSCH Flourishing Scale (CAHMI, 2012)	Student
	Hope	Hope Scale (Lippman et al., 2014)	Student
Student education outcomes	Academic performance	Academic Competence Evaluation (DiPerna & Elliott, 1999)	Teacher
	Grades	End-of-year grades	School
	Test scores	Maryland State Assessment	School
	Attendance/disciplinary actions	Attendance, probations, suspensions	Teacher School
Potential covariates and moderators	Support and safety	Health Literacy Study (Starfield et al., 1995).	Student
	Adverse life experiences	NSCH ACES’s (CAHMI, 2012)	Student
	Sociodemographic factors	Demographic form	Student Teacher

The student survey includes a measure of depressive symptoms, the Child Depression Inventory Short Form (Kovacs, 1992). If a participant’s CDI score exceeds the recommended clinical cutoff for moderate depression, parents are notified and given contact information for the appropriate school or community mental health resources should they want to pursue further assessment for the child and potential referral for services. We document mental health services participants receive (if any) and will include mental health service use in our analytic models.

2.4.3. Qualitative measures.

Focus groups and interviews with key stakeholders will be used both to contribute to fidelity of implementation and (see discussion of responsiveness in 2.5.4 below) and to provide insight into factors associated with effective program implementation and potential for sustainability. Student focus group guides address several domains, including intervention concepts students recall; program aspects students did and did not enjoy; changes students may have observed in themselves as a result of participation; and how students envision themselves using skills in the future, if at all. Principals at participating schools, as well as eighth grade teachers not directly involved in program delivery, are interviewed individually to obtain their feedback on program implementation and effects. Principal and teacher interviews explore their perceptions of the RAP Club program, changes they may have observed in student participants, barriers and facilitators to using the program in schools, and considerations regarding sustainability and scale up. Principals and teachers each receive a $20 gift card for participating in an interview.

2.4.4. Fidelity of implementation (FOI) measures.

FOI is assessed for RAP Club facilitator trainings and program delivery and for Healthy Topics facilitator trainings and program delivery. FOI is assessed for four domains: adherence (degree to which trainings/programs are delivered per the manuals), quality (degree to which trainings/programs are delivered as intended), dosage (amount of training/program received), and responsiveness (level of participant engagement) (Dusenbury, Brannigan, Falco, & Hansen, 2003).

Adherence and quality are assessed using facilitator ratings and observational coding. At the summer training sessions, training facilitators rate their own adherence to the training protocol. Similarly, following each program session, group facilitators rate their own fidelity to the curriculum and their quality of delivery. RAP Club and Healthy Topics trainings and program sessions are also videotaped. Two coders blind to study hypotheses are trained to achieve ≥0.80 agreement in rating trainings for each program. Inter-rater reliability is calculated for a random sample of 25% of trainings. At least two coders are trained to achieve ≥0.80 agreement on ratings of program delivery adherence and quality for RAP Club and for Healthy Topics. The coders rate FOI in a random sample of 25% of videotaped sessions for each program using observational checklists developed for each program.

Dosage is assessed using attendance records at each training session and each program session, as well as monitoring of homework completion at each program session. Responsiveness is assessed using instructor ratings and qualitative data collection with participants. During trainings, a study team member rates trainee responsiveness, and trainees complete a brief checklist on their own engagement. In each program session, group facilitators rate student engagement and disruption. Focus groups and interviews with principals, teachers, and students (see 2.95) provide additional data on participant responsiveness to the programs.

2.4.5. Cost monitoring.

Estimates of training and program delivery costs for both programs will be calculated. Training cost will include labor (time) costs for trainers and trainees, space costs (e.g., classrooms), hourly costs for equipment (e.g., projectors), and estimated per person costs for transportation. Program delivery cost will include labor (time) costs for school personnel and community member facilitators (e.g., hourly labor costs, travel cost based on time and mileage) and supervisors, as well as opportunity costs (e.g., use of school space). Hours of time inputs and personnel costs will be obtained for all study and school personnel. Total costs per student will be calculated for each program by summing all training and coaching and other resource cost elements and dividing them by the number of students in each intervention group.

2.5. Study programs

2.5.1. Intervention: RAP Club.

RAP Club is based on SPARCS (DeRosa et al., 2006; DeRosa & Pelcovitz, 2009), a group treatment for urban adolescents exposed to chronic stress and adversity. SPARCS uses psycho-education, cognitive behavioral therapy (CBT), and mindfulness strategies to promote self-regulation among young people. SPARCS is one of the top three interventions disseminated via the National Child Traumatic Stress Network and was included (Jennings, 2004) in the Evidence-based Practice Center Systematic Review Protocol compiled by the Department of Health and Human Services (2012) (AHRQ, 2012).

The 12-session RAP Club program is delivered twice per week over six weeks. Table 2 summarizes RAP Club core components, skills, and strategies. Mindfulness strategies encourage present-focused awareness, through experiential practices like observing the breath (Brown & Ryan, 2003), which improves regulation of thoughts and emotions, reducing stress, anxiety, and depression (Chiesa & Serretti, 2009; Hofmann, Sawyer, Witt, & Oh, 2010; Lutz, Slagter, Dunne, & Davidson, 2008; Tang et al., 2009). CBT strategies modify thoughts and behavior to promote healthy emotional functioning and are empirically supported for treatment of depression, generalized anxiety disorder, and PTSD (Butler, Chapman, Forman, & Beck, 2006). Psychoeducation is provided about stress and trauma, such as witnessing community violence, how stress affects the body, and how stress can make it harder to manage difficult emotions and make effective decisions.

Table 2.

RAP Club components, skills, and strategies.

Core Intervention Component	Skill	Session #	Evidence-Based Strategy
Identifying stress	Stress and the Body	2	Psychoeducation
Awareness of emotional states	States of Mind	3	Mindfulness
Using a mindful approach	Path to Wise Mind	4, 5	Mindfulness
Communication skills	MAKE A LINK	6, 7	CBT
Problem solving skills	LET GO	8, 9	CBT
	Distract	10	Mindfulness/CBT
Distress tolerance skills	Self-soothe	11	Mindfulness

Note: The first and last intervention sessions, not listed above, are an introduction and review/graduation.

2.5.2. Active control condition: Healthy Topics.

The 12-session health education program Healthy Topics, adapted from the Glencoe Health Curriculum (McGraw Hill), serves as our active control group. Our previous RCTs have shown that Healthy Topics is an effective control condition for school-based interventions to promote emotional and behavioral change (Sibinga et al., 2013; Sibinga, Perry-Parrish, Thorpe, Mika, & Ellen, 2014) because, like RAP Club, it provides information on wellbeing, includes didactic instruction and group discussion, and is likely to appeal to youth interested in their own health. Healthy Topics is consistent with RAP Club in session frequency and length, group size, and time offered during the school day. Healthy Topics also controls for effects of meeting with a positive adult instructor, positive peer group experience, learning new material, attention, and time. The Healthy Topics curriculum covers a range of health-related topics, including physical activity, nutrition, skin care, substance use, and health literacy.

2.6. Procedures: Program delivery.

RAP Club and Healthy Topics are each delivered during the school day at a time that does not conflict with core academic courses, such as language arts or math. The 12-session programs are delivered twice per week over a six-week period in the fall. Program sessions last approximately 45 minutes each.

2.6.1. Group leaders and co-facilitators.

RAP Club and Healthy Topics are each delivered by a group facilitator from the study team (e.g., a trained graduate student or clinician) in partnership with a co-facilitator, who is a young person from the community. The rationale for their inclusion is that our pilot work suggested the presence of a co- facilitator closer in age to the students and/or familiar with the students’ cultural context promoted student buy-in and trust. Co-facilitators are encouraged to provide examples of using skills in real-world situations relevant for students. To enhance comparability of the intervention and control groups, community members co-facilitate Healthy Topics as well as RAP Club. Community member co-facilitators are recruited from local colleges or other organizations, such as the Youth Opportunity (YO) Center, which provides GED and job training each year to 700-800 young people who are disconnected from school and the workforce.

2.6.2. School personnel in training.

Each principal selects school personnel to be trained in RAP Club or Healthy Topics. We recommend that schools select mental health personnel (e.g., social worker, psychologist, counselor) for training in RAP Club and a health or gym teacher for training in Healthy Topics. These school personnel each attend a summer training session for their assigned program and then attend and participate in group supervision calls and in the 12 program sessions offered at the school. The rationale for training school personnel is to build their capacity to offer RAP Club and Healthy Topics so that schools can continue to offer one or both of the programs following their participation in the project.

2.6.3. Facilitator trainings.

Trainings for both RAP Club and Healthy Topics occur shortly before the start of the school year in each year of the project. The 2-day group RAP Club training is delivered by two developers of SPARCS, with whom we collaborated in adapting RAP Club. The 1-day group Healthy Topics training is delivered by the study PI and a study co-investigator. The RAP Club training is attended by study team members serving as RAP Club facilitators, community members serving as RAP Club co-facilitators, and school personnel from the eight partner schools being trained in RAP Club for the coming year. Similarly, the Healthy Topics training is attended by study team members serving as Healthy Topics facilitators, community members serving as Healthy Topics co-facilitators, and school personnel from the eight schools being trained in Healthy Topics for the coming year. Facilitators and co-facilitators are only trained in either RAP Club or Healthy Topics to minimize potential for contamination. If all individuals requiring training for a given program cannot be present on the same day, a second training date is offered.

2.6.4. Supervision for RAP Club and Healthy Topics.

Over the six weeks of program delivery each project year, the principal investigator and trained study staff conduct weekly group supervision calls for RAP Club and, separately, for Healthy Topics. Group facilitators, co-facilitators, and school personnel in training attend these calls. The goal of the supervision calls is to review challenges and successes associated with the past week’s group sessions and to plan for sessions the following week. The group format of the calls allows participants to learn from one another. Group instructors complete a fidelity rating form following each session to evaluate the extent to which they covered the curriculum material as specified; supervisors review these forms prior to the supervision call and use the forms to inform the discussion.

2.7. Power calculation.

Power calculations were estimated using PowerUp! (Dong & Maynard, 2013) and assuming power=80%, a two-tailed test with alpha=0.05, and average number of students per school=25. With respect to emotional and behavioral outcomes, intraclass correlation coefficients (ICCs) of 0.02 - 0.033 have been reported for depressive symptoms (Joyce & Early, 2014), and experience of study investigators suggests ICCs for student behavior problems are much smaller (<0.01). With 32 schools and ICC=0.01, there is 80% power to detect an effect of 0.161; for ICC=0.03, the effect size is 0.19. In our pilot study, we found an effect size of d=0.69 for improvements in classroom behavior and d=0.85 for emotion regulation. For academic outcomes, the literature suggests that in low SES or low achievement U.S. schools, ICCs for eighth grade math achievement scores range from 0.056 to 0.116, and ICC for eighth grade reading achievement scores is approximately 0.057, controlling pretest scores and other covariates (Briere, Pascal, Dupere, & Janosz, 2013; Hedges & Hedberg 2007). With 32 schools and ICC=0.10, there is 80% power to detect an effect size of 0.267; for ICC=0.05, the effect size is 0.215. The meaning of effect size varies by context, but an effect size of 0.8 is generally interpreted as large, 0.5 moderate and 0.2 small (Cohen, 1988). Thus, we will have power to detect small or small-to-moderate effects on achievement test scores, consistent with effects found in the literature (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011).

2.8. Statistical analyses.

2.8.1. Primary analyses.

We will first perform randomization checks to ensure the two study groups are equivalent on key demographic and baseline variables. We will use analytic methods appropriate to the design and hierarchical structure of the data in this trial, (Donner & Klar, 2000; Murray, 1998) namely, random effects multilevel models (Bryk & Raudenbush, 1992; Goldstein, 1995; Hedeker & Gibbons, 1994, 1996a, 1996b; Raudenbush & Willms, 1995). Specifically, generalized mixed effects models will be used to estimate baseline and multivariate adjusted between-group differences on primary and secondary outcomes, adjusted odds ratios, and 95% confidence intervals using student-level data but considering student clustering by school. We will use an intent-to-ffeat approach, estimating effects for all enrolled in the study. Longitudinal modeling with these data will explore the change in outcome measures over time and will identify the role RAP Club may play in the trajectories modeled. Continuous outcomes will be analyzed using models that assume normally distributed outcomes; appropriate data transformations will be made to improve normality. Binary outcomes will be tested using logistic regression-based multilevel models. Program efficacy will be tested by examining the coefficient of the treatment versus control indicator in the multilevel model, controlling covariates. We will include random intercepts for schools and will assess whether a random slope for schools is appropriate for the intervention effect. Likelihood ratio tests (p<.05) will be used to simplify models following a backward selection strategy.

2.8.2. Secondary analyses.

We will evaluate mediators and moderators of intervention impact through extensions of the multilevel models described above. Baseline levels of stress/trauma exposure, psychological distress, and emotion regulation difficulties will be tested as potential moderators, as data suggest students with more issues in these areas may experience more pronounced intervention benefits (Diamond, 2014). Moderation by fidelity of training and program implementation will also be explored. Tests of moderation will be conducted by including interactions between the hypothesized moderators and the indicator of study group (intervention versus control) and assessing the significance of the interaction terms. Underlying process variables and emotional and behavioral outcomes will be evaluated as potential mediators. We will conduct two types of mediation analyses. First, we will use the mediation analysis approach proposed by Baron and Kenny (Baron & Kenny, 1986), in which the total effect of treatment assignment is comprised of the total of the direct and indirect effect. Since interpreting these results as causal relies on extreme assumptions (Jo, 2008), we will also use methods based on principal stratification for multilevel settings, as discussed in Jo (2008). This approach assumes the effect of treatment assignment on the outcome is mediated through intended mediators and therefore there is no direct effect of treatment assignment. These mediation and moderation analyses may be somewhat underpowered but will provide key information to guide future work.

2.8.3. Missing data.

If <10% of data are missing, and good predictors of missing values exist, we will use simple imputation based on prediction using observed values (Graham, 2009; Schlomer, Bauman, & Card, 2010). Under an assumption that data are missing at random (Rubin, 1976) and depending on the analysis and statistical software used, we will employ an appropriate missing data technique (Enders, 2010). If data are determined not to be missing at random, we will apply relevant approaches (Little & Raghunathan, 1999; Paik, 1997). We will compare analyses with imputed data to those with observed data to evaluate effects of missing data.

2.8.4. Cost analyses.

Analyses for RAP Club and Healthy Topics will be calculated separately. For both programs, cost analyses will be estimated for training and program delivery components. A range of cost figures will be developed by varying the number of trainees and venue location (e.g., in school vs. other location). Total costs per student will be calculated for each program by summing all training and coaching and other resource cost elements and dividing them by the number of students in each program group.

2.8.5. Qualitative analyses.

Focus groups and interviews will be audiotaped, transcribed, and uploaded into a qualitative data management software program (Atlas.ti) for coding and data analysis. The data will be coded by tagging and then regrouping responses by themes observed when reading the transcripts and when indexing words and phrases electronically. Once the data are coded, matrices of the key codes will be developed to look for patterns across the focus groups and interviews, particularly for themes regarding how the program may influence student stress and behavior. Causal network diagrams may also be developed to illustrate relations emerging from the data (Miles & Huberman, 1994). Research members will meet to discuss themes and interpretations. Identified themes will inform and extend our interpretations of the quantitative data.

3. Discussion

This study is the first randomized controlled trial to evaluate the efficacy of RAP Club, a universal intervention to improve emotional, behavioral, and academic outcomes in urban eighth graders. Adapted from a treatment for urban adolescents exposed to chronic stress, RAP Club is a trauma-informed approach to enhance core regulatory skills that are linked with adaptive coping and stress management. These regulatory skills, in turn, have potential to reduce emotional problems, such as depressive, anxiety, and trauma symptoms, and improve key functional outcomes, including classroom behavior and academic performance. Our pilot research supported the feasibility and acceptability of offering RAP Club as a universal school-based program and suggested the program may be beneficial for urban upper middle school students (Mendelson et al., 2015). This trial will test program impact in comparison with an active control condition. The study will also evaluate potential moderators and mediators of program effects to further inform understanding of how the program works and for whom it is most effective.

The current study has several strengths from a research perspective. First, inclusion of an active control condition provides a rigorous test of intervention efficacy. Healthy Topics, a group health education program, is anticipated to benefit students by teaching them valuable information relevant to physical health and wellness. The program controls for key non-specific factors, such as attention from new adults who are not school personnel, positive group activities, and time spent in a new activity. If RAP Club is found to benefit students’ academic, behavioral, or emotional functioning to a greater extent than Healthy Topics, the finding will suggest that intervention impacts resulted from core components of RAP Club (e.g., mindfulness, CBT), rather than from non-specific factors. Second, collection of data from multiple sources (students, teachers, academic records) enhances completeness and validity of study findings. Third, the study incorporates follow up assessments to test program impacts over time, including teacher and student-reported data at 4-month follow up, student-reported data in ninth grade (12-16 months post-intervention), and academic data for students’ grade year.

Fourth, inclusion of moderation analyses enables exploration of whether students with certain demographic (e.g., gender) or psychosocial characteristics (e.g., higher baseline trauma symptoms) are more likely to benefit from the intervention. Fifth, measurement and analyses of potential mediators, such as improved emotion regulation, will clarify mechanisms of intervention action. Finally, the trial includes collection of data on factors relevant for understanding intervention implementation and potential for sustainability, including data on fidelity of implementation, program costs, and stakeholder perspectives on intervention barriers and facilitators. Use of mixed methods in this context enriches our ability to understand stakeholder experiences with the intervention, informing future implementation and dissemination efforts.

The study intervention, RAP Club, has several innovative features. The program includes not only well-established CBT strategies, such as problem solving and communication skills, but also mindfulness-based principles and practices. Mindfulness-based interventions have been found to reduce depression and anxiety and improve stress management in adults (e.g., Chiesa & Serretti, 2009; Hofmann, Sawyer, Witt, & Oh, 2010; Lutz, Slagter, Dunne, & Davidson, 2008; Tang et al., 2009). An emerging body of literature suggests mindfulness programs may have similar benefits for youth (Chi, Bo, Liu, Zhang, & Chi, 2018; Zoogman et al., 2015), but the evidence base is this area is still limited, particularly with respect to the impact of universal school-based mindfulness interventions on academic, emotional, and behavioral functioning in urban youth. This study has potential to expand the evidence base for mindfulness in education in productive ways.

Program delivery by trained leaders paired with young adult community members is another notable program feature. Co-facilitation by young adult community members is a novel strategy for utilizing community resources to enhance cultural relevance of the intervention. Our pilot research suggested that these co-facilitators increased student buy-in for the program, providing real-world examples of skills use presented by individuals closer in age to students and more familiar with their cultural context. This approach not only promotes behavioral change (Cuijpers, 2002a, 2002b) but also offers training and workforce development for the young adults. Co-facilitators gain experience with group facilitation in a school context, enhancing opportunities for future employment. Our model has promise for dissemination in urban schools across the U.S., as most urban settings have employment training programs, or other similar organizations, that can serve as a source of young adult community member group facilitators. Co-facilitation by high school students may be another viable model for future program delivery.

The program model used in this trial also included capacity building for school personnel to support continued program use by schools. RAP Club training and supervision are provided to relevant school personnel, such as mental health counselors, and these individuals attend and participate in program sessions. Thus, school personnel are not expected to deliver the program in the initial year on their own but are rather provided with supports to promote their readiness to do so the following year. In addition, they receive payment for the time they spend attending trainings, supervision calls, and program sessions. (The same supports are provided for building school capacity in delivery of Healthy Topics.) Schools wishing to continue delivering RAP Club or Healthy Topics following their year of involvement in the trial are encouraged to send staff to attend subsequent summer trainings free of charge (staff already trained the prior year can attend as a “booster” and/or new staff can attend) and to have staff participate in supervision calls; the principal investigator is also available to consult with schools to support their continued use of programming. It is hoped that this approach will encourage schools’ continued program use. Data collected on intervention barriers and facilitators will also inform the study team’s approach to supporting intervention sustainability.

Use of school resources, such as counselors, has been advocated for prevention programming as it supports sustainability and embeds intervention principles within school practices (Atkins et al., 2010). School mental health personnel are employed widely: a nationally representative survey by SAMHSA in 2002-3 of approximately 83,000 public schools in the United States reported that almost all schools (96%) employed at least one staff member responsible for providing mental health services, with many schools employing counselors (77%), psychologists (69%), and/or social workers (44%) (Foster et al., 2005). Using school mental health counselors to deliver universal preventive interventions also has potential to reduce stigma associated with mental health issues and foster familiarity of students with school mental health resources.

This trial has several limitations. Randomization within school creates risk that students in the control condition may be exposed to RAP Club skills. To minimize this risk, RAP Club facilitators and students are instructed not to discuss RAP Club skills with students in Healthy Topics, and group facilitators receive training for only one program. As repeated practice of skills with guidance from an instructor is hypothesized to create change, we do not anticipate that hearing about RAP Club skills without the sustained focus and guidance of the 12-session program will significantly affect outcomes for control students. We will assess contamination by asking control students to report their knowledge of RAP Club components, as well as how they obtained this knowledge, through a brief questionnaire, and will evaluate whether knowledge of RAP Club among control students is associated with improved education outcomes. Gains in statistical power from randomizing within school are of significant benefit in the proposed study. We also chose not to obtain parent-reported data on student functioning given the challenges of reaching parents and guardians in our target population. Collection of data from students, teachers, and academic records provides multiple informants, offsetting concerns about lack of parent-reported data. The current trial does not include collection of biological or physiological data (e.g., heart rate variability, cortisol), which would be valuable for better understanding mechanisms of intervention effects. We anticipate there will be opportunities to collect such data among a sub sample of study participants or in future studies.

The pervasiveness of academic problems and trauma exposure among inner-city adolescents highlights the need for universal trauma-informed interventions to provide students with tools and supports to improve school performance and facilitate a positive transition to high school. Unlike most school-based mental health-focused interventions, RAP Club addresses internalizing problems as well as behavioral issues, it is tailored for upper middle school students, and it can be delivered to all students in the grade without initial screening for trauma or distress symptoms. This trial will provide a rigorous evaluation of RAP Club’s effects on student education, behavioral, and emotional outcomes and will lay the groundwork for a future effectiveness trial. Findings will inform refinements to our proposed theory of change, with implications for the field’s understanding of how student self-regulatory capacities and emotional and behavioral functioning predict educational outcomes.