Problem Behavior and Urban, Low-Income Youth

Abstract

Background

Youth problem behaviors remain a public health issue. Youth in low-income, urban areas are particularly at risk for engaging in aggressive, violent, and disruptive behaviors.

Purpose

To evaluate the effects of a school-based social–emotional learning and health promotion program on problem behaviors and related attitudes among low-income, urban youth.

Design

A matched-pair, cluster RCT.

Setting/participants

Participants were drawn from 14 Chicago Public Schools over a 6-year period of program delivery with outcomes assessed for a cohort of youth followed from Grades 3 to 8. Data were collected from Fall 2004 to Spring 2010, and analyzed in Spring 2012.

Intervention

The Positive Action program includes a scoped and sequenced K–12 classroom curriculum with six components: self-concept, social and emotional positive actions for managing oneself responsibly, and positive actions directed toward physical and mental health, honesty, getting along with others, and continually improving oneself. The program also includes teacher, counselor, family, and community training as well as activities directed toward schoolwide climate development.

Main outcome measures

Youth reported on their normative beliefs in support of aggression and on their bullying, disruptive and violent behaviors; parents rated youths’ bullying behaviors and conduct problems; schoolwide data on disciplinary referrals and suspensions were obtained from school records.

Results

Multilevel growth-curve modeling analyses conducted on completion of the trial indicated that Positive Action mitigated increases over time in (1) youth reports of normative beliefs supporting aggressive behaviors and of engaging in disruptive behavior and bullying (girls only); and (2) parent reports of youth bullying behaviors (boys only). At study end-point, students in Positive Action schools also reported a lower rate of violence-related behavior than students in control schools. Schoolwide findings indicated positive program effects on both disciplinary referrals and suspensions. Program effect sizes ranged from −0.26 to −0.68.

Conclusions

These results extend evidence of the effectiveness of the Positive Action program to low-income, minority, urban school settings and to middle school–aged youth.

Trial registration

This study is registered at ClinicalTrials.gov NCT01025674.

Introduction

Youth violence remains a major public health issue.¹ The U.S. has witnessed high rates of problem behaviors among youth in the past 3 decades, most notably those involving bullying and violence.^2–4 Youth who initiate such behaviors early in development are at greater risk for psychopathology, substance use, and criminal behaviors during adolescence and adulthood.^5–10 Interventions implemented in schools show promise for reducing rates of violent behavior as well as related disruptive behaviors (e.g., bullying¹¹) among youth.^12–14 Yet, the magnitude of beneficial program effects typically has been moderate, thus suggesting a need for ongoing refinement and improvement of programs.^14–16 Findings among schools serving low-income, minority, urban youth have been mixed,¹⁷ highlighting a need for further investigation of program effectiveness in these settings in particular.

Among school-based violence prevention interventions, those focused on social–emotional learning and health promotion have gained increasing popularity in recent years.¹⁸ Such programs show evidence of positive effects on attitudes toward self and others, social behaviors, emotional outcomes, and certain problem behaviors.^18–20 Positive Action²¹ is one example of a promising school-based program of this type. In order to study the effects of Positive Action in an urban, low-income, minority setting, the trial was conducted in Chicago.

The Positive Action program is grounded in theories of self-concept,^22,23 particularly Self-esteem Enhancement Theory,²⁴ and it is also consistent with integrative and social–ecologic theories of health behaviors.^25,26 The theory assumes that individuals will use a wide range of cognitive, affective, and behavioral strategies to help acquire and sustain feelings of worth. Positive outcomes follow to the extent that people are adequately prepared and supported in satisfying their motivation for self-esteem through adaptive beliefs, values, and actions.

In line with this theory, Positive Action includes a classroom-based curriculum in which the desire to feel good about oneself is highlighted, while teaching students the skills needed to act appropriately on this motivation such as self-control, positive interactions with peers, and self-honesty. Ecologic supports (e.g., schoolwide systems for recognizing positive behaviors, family classes) provide social reinforcement for positive behaviors in both school and nonschool settings. More information regarding Positive Action is available on the program website: www.postiveaction.net.

A cluster RCT of the Positive Action program in both rural and suburban schools in Hawaii found beneficial effects on violence-related behavior and substance abuse²⁷ and on disciplinary referrals up to Grade 5.²⁸ Additionally, earlier reported findings²⁹ from the current trial also indicated program effects on violence and substance use up to Grade 5. The current trial builds on previous research by further examining Positive Action’s effectiveness in disadvantaged, inner-city schools and in doing so extending follow-up to the middle school grades, the peak period of onset for many problem behaviors.³⁰ It was hypothesized that findings would indicate favorable effects of Positive Action on: (1) student reported levels of normative beliefs supporting aggression and rates of violence-related, bullying, and other disruptive behaviors; (2) parent-reported levels of youth conduct problems and aggressive behaviors; and (3) school records of disciplinary referrals and suspensions.

Methods

Intervention

The Positive Action program²¹ includes a scoped and sequenced K–12 classroom curriculum. Each grade-level includes 140 lessons (15–20 minutes each; Grades K–6) or 70 lessons (20 minutes each; Grades 7 and higher). The program also includes teacher, counselor, family, and community training as well as activities directed toward schoolwide climate development. Resources available with the grant were not sufficient to support the use of the community component. The core curriculum consists of six components: self-concept, social and emotional positive actions for managing oneself responsibly, and positive actions directed toward a healthy body and mind, being honest with oneself and others, getting along with others, and continually improving oneself. During the current trial, schools received training and technical assistance to help ensure a high level of implementation.

Design and Sample

The evaluation design was longitudinal (i.e., 6 years and eight waves; maximum allowed by funding) at the school level and used a place-focused intent-to-treat design with a dynamic grade cohort (continuously changing because of student mobility).³¹ Participating schools were drawn from the 483 Grades K–6 and Grades K–8 Chicago Public Schools (CPS) with 68 schools meeting eligibility criteria (Figure 1).³² Of the 68 schools, 18 agreed to participate and the seven best-matched pairs (the N that funding would support) were selected for participation.^32,33

Figure 1.

Flow diagram of schools

Funding sufficient for only seven pairs

The following variables from the 2003–2004 CPS data were used as matching variables: percentage of white, African-American, Hispanic, Asian students; percentage of students who met/exceeded criteria on state achievement tests; attendance rate; truancy rate; percentage of students receiving free/reduced-price lunch; percentage of students transferring in or out during the school year (mobility); number of students per grade; percentage of parents involved with school activities; and percentage of teachers who met minimal teaching standards. Information about the crime rate in each school’s neighborhood was also utilized.³⁴ Using these variables and a requirement that each school in a pair be located in the same region of the city, the 18 schools were successfully matched into nine pairs using an SAS program designed for matched pairs (Mathematica Policy Research). . A random-number generation function (Microsoft Excel) was used to randomize the schools within each pair to the control and treatment conditions.

Funding allowed for 14 schools; thus, the seven best-matched pairs were invited to participate. Because of the nature of the intervention, schools, students, those delivering the intervention and the outcome assessors could not be blinded to condition. A series of t-tests revealed that the 14 participating schools did not differ from the remainder of the 68 eligible schools. In addition, the Positive Action and control schools did not differ from each other on any of the matching variables (Table 1).^32,33 All schools were retained for the 6 years of the study.

Table 1.

Baseline and follow-up equivalence on school demographics

	Year 2001					Year 2004					Year 2007					Year 2010
	Control Schools		Positive Action Schools		df=12	Control Schools		Positive Action Schools		df=12	Control Schools		Positive Action Schools		df=12	Control Schools		Positive Action Schools		df=12
	n=7		n=7			n=7		n=7			n=7		n=7			n=7		n=7
%	M	SD	M	SD	t	M	SD	M	SD	t	M	SD	M	SD	t	M	SD	M	SD	t
Male Students	52.70	1.98	52.00	0.98	0.84	52.64	2.89	52.47	2.11	0.13	52.03	2.35	50.90	2.17	0.93	52.09	2.42	52.10	2.22	0.00
White Students	10.51	16.75	9.58	13.07	0.12	9.38	14.80	9.07	12.68	0.04	7.93	13.18	7.00	10.62	0.15	8.57	13.81	7.49	11.53	0.16
African- American Students	55.21	43.91	52.07	48.01	0.13	56.49	43.35	53.64	47.35	0.12	56.73	42.67	55.31	47.10	0.06	56.20	41.64	55.57	47.79	0.03
Hispanic Students	27.27	32.10	32.23	34.23	−0.28	31	35.16	32.79	36.28	−0.09	31.04	33.76	31.33	35.71	−0.02	31.8	34.07	32.63	38.94	−0.40
Asian American Students	4.03	6.17	4.69	7.05	−0.19	2.91	4.30	4.21	6.57	−0.44	2.74	4.45	3.91	7.00	−0.37	3.36	5.94	4.17	7.61	−0.22
“Other” Students	3.91	4.60	4.64	4.92	−0.29	4.46	5.03	4.72	5.21	−0.10	4.63	5.05	4.45	5.19	.063	0.04	0.03	0.02	0.05	−0.40
Students with LEP	14.27	16.13	17.50	16.94	−0.37	11.41	14.10	17.04	17.20	−0.67	12.50	13.82	14.61	16.32	−0.26	10.87	11.92	12.87	14.78	−0.28
Students with an IEP	13.00	6.37	8.70	2.31	1.67	12.84	5.33	9.46	2.36	1.53	14.08	6.46	9.86	3.23	1.55	14.76	6.01	12.06	3.73	1.01
Students Receiving a Free Lunch	83.59	3.39	83.09	6.85	0.17	81.46	3.81	85.51	4.56	−1.81	85.86	7.86	84.97	7.36	0.22	94.60	3.92	92.70	6.30	0.68
School Attendance Rate	93.01	1.17	93.38	1.28	−0.57	93.54	1.09	93.74	1.79	−0.25	92.87	2.09	93.71	1.78	−0.84	93.27	1.87	95.03	1.52	−2.01

Note: 2001 data were used for the matching before randomization, 2004 was the first year of the trial, 2007 was when cohort students ended Grade 5, and 2010 was the last year of the study. Other= percentages for those with race classified as: Native American, multirace, Mexican, Puerto Rican, Cuban, other Hispanic, and multi-Hispanic. At 2010, only information for % Native Americans was available. None of the above t-test were significant at the p<0.05 level.

LEP= Limited English Proficiency, IEP= Individualized Education Plan. For School Attendance Rate, df=13. Adapted from Lewis KM, Bavarian N, Snyder FJ, Acock A, Day J, DuBois DL, et al. Direct and mediated effects of a socialemotional and character development program on adolescent substance use. Int J of Emotional Educ. 2012;4:56-78; reprinted with permission

In each school, all students in Grade 3 at the start of the study were recruited for participation. The study began at Grade 3, both because it was required by the grant funder and because the surveys were designed for students in Grade 3 or higher. Parental consent for study participation was obtained for 79% of students at baseline. Cohort students who entered the schools during later waves consented to be in the study at the time of school entry. The consent rate for these latter students ranged from 60% to 78% across waves. The expiration of the initial study consent form at the end of the first phase of funding required that all students provide consent again at Wave 6 (beginning at Grade 7); the consent rate was 58%. The percentages of parents who provided reports on their children, which were requested at Waves 1, 2, 4, 5, and 8, were 72.3%, 58.9%, 52.2%, 50.5%, and 72.9%, respectively. Throughout the study, data were collected only from students who provided consent. Appendix A, available online at www.ajpmonline.org, provides a diagram of student participation.

Surveys were administered to students beginning in Grade 3 (Fall 2004) and at seven additional times (waves) over the 6 years of the study: spring and fall 2005, spring 2006, spring 2007, fall 2008, spring 2009, and spring 2010 (end of Grade 8). The total number of students that enrolled over the 6 years of the study was 1170, of whom approximately 53% were female and approximately 48% were African-American, 27% Hispanic and 19% other (i.e., white or other minority). The original cohort (Wave 1) included 624 students; by Wave 8, only 363 students (including 131, or 21%, of the original cohort students) remained in the study, reflecting changes in school sizes, consent rates, and the high mobility rate of this population. All study procedures were approved by the IRBs of the University of Illinois at Chicago and Oregon State University.

Youth Report Measures

Except where noted, all youth report outcomes were assessed at each of the eight waves of the study.

Normative beliefs supporting aggression

Students answered questions adapted from The Normative Beliefs About Aggression Scale, with established reliability and validity for school-aged children.³⁵ Items asked about beliefs toward aggression (eight items; e.g., is it ok or wrong to hit, shove, yell, fight other people) and were rated on a 4-point scale (Really wrong to Perfectly ok). An average of these items created a composite score, with higher scores reflecting the belief that aggression is more acceptable (α range=0.81–0.93). Given a skewed distribution of responses, the scale score was split for analysis using a median split across all waves.

Bullying

Students answered a modified set of questions from Orpinas and Frankowski’s³⁶ Aggression Scale, previously demonstrated to be reliable among early adolescents. Scale items asked about the frequency of performing bullying behaviors (six items; e.g., teased, shoved, excluding others, making up things about others) in the past 2 weeks. Response options ranged from 0 to 3 (Never to Many times; α range=0.83–0.90). Responses to scale items were first converted to a dichotomous variable (0=Never and 1=Ever) and converted to a count of the number of items to which a student responded ever.

Disruptive behaviors

An abbreviated set of modified questions from child problem-behavior scales that have been previously examined for validity with youth³⁷ was used to assess the frequency of various disruptive behaviors (six items; e.g., made disruptive noises, took something without permission, performed bad behavior) in the past 2 weeks. Items were rated on a 4-point scale (“Never” to “Many times”; α range=0.77–0.81). A count was created to indicate the number of items to which a student responded ever.

Violence

Students answered questions, adapted from the Risk Behavior Survey,³⁸ about their involvement in violence-related behaviors (six items; e.g., carried a knife, threatened someone, stabbed someone). Students were asked if they had ever engaged in each behavior, with response options on a scale of 1 to 4 (“Never” to “Yes, more than 5 times”; α range=0.74–0.82). Each item was dichotomized (0=“Never” and 1=any other response), with scores then summed to create a count of the number of the behaviors the student had ever performed. This outcome was assessed at Waves 5–8.

Parent-Report Measures

Parents were asked to report on their children’s bullying behaviors and conduct problems observed in any context using items from modified versions of the Aggression and Conduct Problem Subscales of the Behavior and Assessment System for Children (BASC).³⁹ Parents responded to six items (α range=0.73–0.83) regarding bullying (e.g., hits others, teases, threatens to hurt others) and seven items (α range=0.74–0.81) regarding conduct problems (e.g., truancy, cheating, stealing), rated on a 4-point scale (“Never” to “Almost always”) to indicate the frequency of the observed behaviors in the past 30 days. Both bullying and conduct problem items were first dichotomized and then converted to a count (0=“Never” and 1=any other response). These outcomes were assessed at Waves 1–5 and Wave 8.

School-Level Archival Data on Disciplinary Actions

School-level aggregated data on disciplinary referrals and suspensions reported on the school district’s website were accessed for school years 2002/2003 to 2009/2010. Disciplinary referrals and suspensions were based on a range of disruptive, bullying, and illegal student behaviors, the latter of which included (but were not limited to) vandalism, assault, theft, and possession of drugs or dangerous weapons. An average was used of the 2002/2003 and 2003/2004 data (the school years prior to program implementation) to create a reliable baseline measure. Data were not standardized; all schools did, however, follow the same district guidelines for identifying and responding to disciplinary concerns. Analyses on school-level data were adjusted for school size by including it as an exposure variable in the model.

Data Analysis

Data analyses for this paper occurred during Spring 2012. All analyses were conducted using multilevel models in Stata version 12.1. Student-report outcomes were analyzed as three-level models: waves of measurement within students within schools. School-level outcomes were analyzed as two-level models: waves of measurement within schools. Primary analyses consisted of random-intercept models for each of the outcome variables, including condition (i.e., Positive Action or control); time (years since study baseline for survey measures and academic years for school-level measures); condition × time; and quadratic terms for time and the condition × time² interaction. Terms involving the time² term lacking significance were dropped from the final model.

Random-coefficient models, allowing time slope to vary across students, were also estimated, and a likelihood-ratio test was performed to determine whether the random-coefficient model was a better fit for the data. For outcomes with significant quadratic interaction terms, results were graphed (not shown) to facilitate interpretation of growth trajectories. For the binary measure, multilevel logistic regression growth curves were estimated with Stata’s “xtmelogit” command. For count outcomes, data were estimated with Poisson regression models with the “xtmepoisson” command.

Data on violence were collected starting at Grade 5 (Wave 5). Grade 5 results were presented in a previous paper²⁹; the present study focuses on endpoint effects. The baseline covariate of problem behaviors (a combination of the 12 items that make up bullying and disruptive behaviors) was utilized, similar to the previous study²⁹. Missing data were handled using full maximum likelihood estimation.⁴⁰

Tests were also conducted for evidence of differential attrition by comparing scores on outcome measures at Wave 1 baseline across students in program and control schools who had missing data (dropouts) at each subsequent time of assessment (excluding violence, which was not assessed at baseline). No evidence of differential attrition was found in these analyses. A similar analysis was conducted to test for program–control differences for students who were new to the study at each wave subsequent to baseline (joiners) using their scores on outcome measures at the wave of study entry.

These analyses showed that youth in Positive Action schools joining at Waves 5 and 8 had lower normative beliefs supporting aggression that did control school students joining at these waves. Such differences are expected given that these assessments occurred in the spring of the school year when youth who were new to the schools with the program (most likely since the fall) would already have had substantial exposure to the intervention and the school environment that it fostered. Parents of students in the program joining at Wave 4 reported lower bullying and conduct problems than did parents of control students. Effect sizes (ES) in the metric of standardized mean differences were calculated for count outcomes using predicted (model) means and observed SDs⁴¹; for the binary outcome, the Cox transformation was used.⁴²

Sensitivity analyses included re-estimating all models with matched-pairs introduced as an additional, highest level in each model. To provide a more conservative test of program effects, the test statistics provided by Stata, which assume a large sample size and thus were referenced to a standard normal distribution in primary analyses, were also compared to the critical value of 2.18 for a t-distribution with 12 df⁴³ (based on the number of schools). Additional analyses used multiple imputation for missing data.⁴⁴ Results of sensitivity analyses are reported only where they differ from those of primary analyses.

Finally, possible moderating effects of gender and student mobility pattern were tested for all student- and parent-reported outcomes. Moderating effects of mobility patterns were examined using results from a latent class analysis in which a five-class solution was found to be the most appropriate fit for the data: (1) those who stayed in the program (average study duration of 5.72 years, n=158); (2) temporary participants (1.30 years, only in Grades 4 or 5; n=196); (3) late joiners (1.38 years; n=308); (4) early leavers (0.94 years; n=263); and (5) late leavers (3.23 years; n=287; KM Lewis, affiliation, unpublished obervations, year). These analyses tested whether the inclusion of all possible interactions of condition × time × class added significantly to model fit. Results of moderator analyses are reported only where significant.

Data also were collected from teachers on bullying and conduct problems. No evidence of program effects were found on these outcomes (results available from the first author on request). A possible reason for the absence of an effect is a lack of precision introduced by the necessity of having multiple teachers provide ratings of each student over the course of the study.

Data on implementation are currently being examined for a separate paper. In general, there was wide variability between schools in implementation indices (e.g., teacher description of amount and quality of program activities in the classroom, perceived effectiveness of the activities, student reports of exposure to and attitudes toward the program), especially in early years, with improvements over time. By the end of Year 6, one school was implementing at only a moderate level, three at a moderate to high level, and three at high levels. Students in Positive Action schools also reported at each wave on their overall satisfaction with the program (NO! No, Yes, YES!). The mean rating ranged from 2.88 to 3.56 (minimum=1, maximum=4) across the waves of the trial.

Results

For student-report outcomes analyzed as counts, the variation in incidence rate across schools, as indexed by the median incidence risk ratio (MIRR), ranged from 1.21 to 2.54. For the binary outcome (norms about aggression), the median OR⁴⁵ was 3.81, suggesting wide variation among schools in norms about aggression.

Student Self-Report Outcomes

A program (condition × time) effect for the program intervention was found for students’ normative beliefs supporting aggression (OR=0.83, 95% CI=0.72, 0.95, effect size [ES]= −0.68; Table 2). Students in intervention schools also were less likely to report engaging in bullying (condition × time Incidence Risk Ratio [IRR]=0.85, 95% CI=0.76, 0.95; condition × time² IRR=1.02, 95% CI=1.00, 1.04; ES= −0.39); disruptive and violence-related behaviors (IRR=0.92, 95% CI=0.87, 0.96, ES= −0.50, and IRR=0.38, 95% CI=0.18, 0.81, ES= −0.54, respectively) compared to students in control schools. Table 3 shows model probabilities and means at baseline and endpoint and the ESs for all outcomes. There was moderation of the program effect on bullying by gender (condition × time × gender IRR=1.28, 95% CI=1.04, 1.60; condition × time² × gender IRR=0.96, CI=0.93, 1.00), such that there was evidence of a larger program effect for girls (ES= −0.51) than for boys (ES= −0.23).

Table 2.

Multilevel growth curve model estimates for problem behaviors (N=1170 students; 14 schools)

	Intercept	Condition (1= Positive Action)	Time	Condition × Time	Time2	Condition × Time2
	OR (95% CI)
Student Self-Report
Normative Beliefs in Support of Aggressiona	0.26 (0.17, 0.42)	1.09 (0.59, 2.01)	2.21*** (1.78, 2.74)	0.83 ** (0.72, 0.95)	0.93*** (0.90, 0.97)	--
	IRR (95% CI)
Bullying Behaviors	0.92 (0.77, 1.12)	1.18 (0.92, 1.53)	1.86*** (1.72, 2.02)	0.85 ** (0.76, 0.95)	0.93*** (0.92, 0.95)	1.02 * (1.00, 1.04)
Frequency of Disruptive Behaviorsa	0.84 (0.44, 0.66)	1.26 (0.96, 1.95)	1.50*** (1.39, 1.63)	0.92 *** (0.87, 0.96)	0.97*** (0.96, 0.99)	--
Violence-Related Behaviors b,c	1.15 (0.67, 1.96)	0.38 * (0.18, 81)	--	--	--	--
Parent Report of Youth Behaviors
Bullying	1.48 (1.18,1.68)	1.07 (0.78, 1.46)	1.01 (0.98, 1.05)	0.93 * (0.89, 0.97)	--	--
Conduct Problems	2.08 (1.77, 2.44)	1.01 (0.81, 1.25)	0.97 (0.91, 1.04)	0.97 (0.94, 1.01)	1.01** (1.00, 1.02)	--
School-Level Archival Data d
Disciplinary Referrals	0.09 (0.04, 0.23)	0.58 (0.16, 2.09)	1.23*** (1.17, 1.29)	1.14 ** (1.05, 1.24)	0.98*** (0.97,0.99)	0.98*** (0.96, 0.99)
Suspensions	0.08*** (0.03, 0.20)	0.58 (0.15, 2.26)	1.21*** (1.15, 1.28)	1.16** (1.06, 1.27)	0.98 *** (0.97, 0.99)	0.98 ** (0.96, 0.99)

Note: Boldface indicates significance. Model type is random intercept, except as noted.

^aRandom coefficient model

^bIntercept set at Wave 8 to allow for a test of the program effect at study end-point.

^cThe condition effect in the random intercept model was significant (IRR= 0.58, p<0.05). The likelihood ratio test, however, indicated that the random coefficient model was a better fit for the data. For this model, p=0.051.

^dFor school-level measures, the time variable represented academic year rather than time since beginning implementation of the Positive Action intervention. These condition × time effects model differences in the time slope at baseline only, due to the inclusion of a quadratic effect.

^*p<0.05;

^**p<0.01;

^***p<0.001

Table 3.

Predicted Probabilities, Predicted Counts, and Effect Sizes for Impact of the Positive Action Program

Outcome	Wave 1		Wave 8		Effect Size
	Positive Action	Control	Positive Action	Control
Student-Report
Normative Beliefs Supporting Aggression a	0.31	0.27	0.48	0.60	−0.68
Bullying b	1.56	1.33	3.14	3.59	−0.39
Disruptive Behaviors b	0.99	0.78	2.09	2.44	−0.50
Violence b	—	—	0.78	1.34	−0.54
Parent-Report
Bullying b	1.92	2.03	1.41	1.78	−0.31
Conduct b	2.30	2.36	2.76	2.61	−0.26
School-Level Archival Data
Disciplinary Referrals c	42.57	73.05	73.22	137.44	−0.58
Suspensions c	40.95	65.25	65.81	105.48	−0.27

Note. Effect sizes were calculated using the Cox transformation for binary outcomes and predicted means for count outcomes

^aMeasure is dichotomous; predicted probabilities shown.

^bMeasure is a count of the number of behaviors endorsed.

^cMeasure is a count of disciplinary actions adjusted for school size.

Parent-Report Outcomes

Findings also indicated positive effects of Positive Action on both parent-report outcomes. For each year in the study, parents of students in schools with the program reported fewer bullying behaviors by their children (IRR=0.93, 95% CI=0.89, 0.97; ES= −0.31) and marginally fewer conduct problems (IRR=0.97, 95% CI=0.94, 1.01, ES= −0.25) as compared to parents from control schools. These effects were significant for bullying, and marginal (p=0.096) for conduct problems in the pair-level models, and only bullying was still significant with the adjusted df test. Similar to the student reports, there was a difference in gender for parent reports of bullying condition × time × gender IRR=0.89, 95% CI=0.80, 0.98); such that there was evidence of a larger program effect for boys (ES= −0.63) than for girls (ES= −0.42).

Disciplinary Referrals and Suspensions

The school-level models of disciplinary referrals and suspensions each had linear and quadratic interactions of condition × time. The net result of these trends was evidence of favorable program effects on both disciplinary referrals (ES= −0.58) and suspensions (ES= − 0.27) by the end of the study (Table 3).

Discussion

This cluster RCT extends evidence of the effectiveness of the Positive Action program by examining program effects on outcomes relating to violence prevention among students attending low-income, mostly minority urban schools and by extending follow-up into middle-school grades. Students in schools that implemented the program were less likely to report holding normative beliefs supporting aggressive behaviors and engaging in violence-related, bullying, and other disruptive behaviors. This evidence of the program’s beneficial effects on reducing student problem behaviors was largely corroborated by ratings from parents and school-level disciplinary referrals and suspensions and is consistent with findings of prior evaluations of the program with other populations and age groups.^27–29

Present findings suggest program effects on student-reported bullying behavior only for girls, but on parent-reported bullying behavior only for boys. It may be that parents are more likely to be aware of and thus able to report accurately on the relatively more overt (rather than relational) forms of bullying behaviors that boys engage in⁴⁶ due to the greater likelihood of such behaviors resulting in disciplinary and other consequences. Likewise, other findings from the trial suggest that girls have higher levels of self-honesty compared to boys,⁴⁷ which could serve to increase the sensitivity of student self-report data to program effects for bullying behavior.

Limitations

Several study limitations should be considered in interpreting the current findings. First, self-reports of behavior have potential reporting bias.⁴⁸ Further, high mobility rates among students resulted in a high turnover of students in the study, a common phenomenon among low-income, urban schools.⁴⁹ However, the use of analytic methods that utilized all available data from all cohort students present at any time during the study is consistent with recent recommendations for analysis of data in clustered trials in the context of high mobility.^31,50

Sensitivity analyses using multiple imputation produced the same pattern of results as that reported here; differences across program and control schools for those leaving or joining the study were limited, and there was no evidence of differential program effects across student mobility groupings. With respect to construct validity, blinding was not possible with the trial design, which could have influenced student or parent reports of behavior. Finally, with regard to external validity, study findings can be generalized only to similar schools in low-income, inner-city neighborhoods that would self-select to participate in a study of this nature. The small number of eligible schools that agreed to participate in the trial (18 of 68) may have been more likely to have school-level factors in place (e.g., climate) to facilitate engagement in a study of this nature.

Conclusion

In addition to future research to address the preceding concerns, additional investigation is needed to clarify mediators and moderators of the effects of the Positive Action program (and other school-based prevention programs) on youth problem behaviors. Such data will be essential for further refining program content and delivery and thus effectiveness for reducing rates of violent and other disruptive behaviors among youth in the U.S.