Effects of the Positive Action Program on Indicators of Positive Youth Development Among Urban Youth

Abstract

This study evaluated effects of Positive Action, a school-based social-emotional and character development (SECD) intervention, on indicators of positive youth development (PYD) among a sample of low-income, ethnic minority youth attending 14 urban schools. The study used a matched-pair, cluster-randomized controlled design at the school level. A multiple-measure self-report protocol assessed four key strengths and resources for PYD: self-concept, peer affiliations, ethics, and social skills. Students (n=1170) were assessed from grades 3 to 8, the duration of the intervention, with drop-outs and late entrants included in analyses. Growth curve analyses revealed evidence of favorable program effects on each of the four types of resources. The study contributes to PYD research by providing evidence for school-based interventions in low-income, urban contexts for ethnic minority youth.

Recent decades have seen a growing interest in strengths- and promotion-oriented interventions for youth, thus reflecting a move away from approaches focused primarily on prevention of specific problems or remediation of deficits (Snyder & Flay, 2012). This shift has been supported by a convergence of research findings from diverse areas (R. M. Lerner, Almerigi, Theokas, & Lerner, 2005), including life-span development (Baltes, Lindenberger, & Staudinger, 1998), ecological developmental psychology (Bronfrenbrenner, 2005), positive psychology (Seligman, 2000), and social-emotional and character development (SECD; Elias, 2009). Much of this emerging work can be understood using the concept of positive youth development (PYD; Hamilton, 1999). PYD refers to a focus on the psychological and social strengths of developing young persons, such as self-concept, morality, social skills, and to the ways in which these strengths may be both manifested in, and shaped by, experiences in different contexts, including interventions designed for this purpose (Snyder & Flay, 2012). In addition to being a perspective on youth development, PYD is also promoted as an outcome that should lead to the prevention of multiple negative health behaviors (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2004; Flay, 2002; Flay & Allred, 2010; R. M. Lerner, Almerigi, et al., 2005; Snyder & Flay, 2012). Several studies have found evidence that youth interventions can promote the development of PYD indicators (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011), such as sociability, problem solving, interpersonal skills, self-control, and empathy. The importance of doing so is supported by studies that have shown improvements in these areas to be associated with reductions in negative outcomes, including substance use (R. M. Lerner & Lerner, 2011; R. M. Lerner, Lerner, & colleagues, 2013; Schwartz et al., 2010), sexual risk taking (Gavin, Catalano, David-Ferdon, Gloppen, & Markham, 2010; Schwartz et al., 2010), conduct problems and emotional distress (Durlak et al., 2011) and aggression (Beets et al., 2009; Catalano et al., 2004).

One area of research on PYD has found support for a framework in which PYD developmental resources are subsumed into the “five Cs” of character, competence, confidence, connection, and caring; a sixth C, contribution, emerges in the presence of the first five Cs in youth (R. M. Lerner, Almerigi, et al., 2005). The developmental resources addressed in the present study are related to some of these Cs but are not assessed with the same measures as used by Lerner and colleagues. In this study, developmental strengths and resources for PYD in four domains were assessed over six years: self-concept, peer affiliation, ethics (morality), and social skills. Although these resources do not represent an exhaustive list of developmental strengths that are crucial to youth development (e.g., Benson, 2007), they do represent some of the key resources that assist youth in the development of other positive outcomes and with the prevention of problem behaviors.

Consistent with the role of school as a major socializing influence on youth and the setting where youth spend a large portion of their time under supervision of adults, available research indicates that schools are a promising setting for providing youth with programmatic experiences and opportunities that promote PYD (Catalano et al., 2004; Flay, 2002; J. V. Lerner, Phelps, Forman, & Bowers, 2009; R. M. Lerner, Dowling, & Anderson, 2002; R. M. Lerner, Lerner, et al., 2005; Snyder & Flay, 2012). There are, however, some important limitations to this work. First, although the 5 Cs are understood to begin their development in early childhood while remaining influential during adolescence, whether a school-based intervention can influence the trajectory of PYD strengths and resources across the transition from childhood into adolescence remains an open question. Clarifying the longer-term effects of interventions on PYD resources during the period of transition into adolescence is important given that this is the period of onset for most negative health behaviors (e.g., smoking, violence, substance use, risky sex). Second, in view of elevated rates of many adverse outcomes for youth in urban, low-income contexts (O'Connell, Boat, & Warner, 2009), where the preponderance of youth typically are members of racial/ethnic minority groups, there is a need for evaluations of school-based programs pertaining to PYD that add to the knowledge base of the effectiveness of such approaches for these types of settings and populations specifically. The importance of this question is underscored both by indications of more limited effectiveness for other types of school-based interventions in low-income, minority-serving schools (Farahmand, Grant, Polo, Duffy, & DuBois, 2011) and by the reality that characteristics of such schools (e.g., lack of resources, larger class sizes) and students (e.g., greater exposure to risks outside of school that are not supportive of targeted outcomes) may present formidable challenges to intervention effectiveness. A third limitation is that most of the interventions based on PYD principles are of limited duration, typically less than one year. Given the complicated nature of many developmental strengths and resources, and the challenge of improving characteristics that can have biological as well as social bases, it is not surprising that the effect sizes of PYD interventions are often small (e.g., Durlak et al., 2011). The present study addresses this limitation by considering effects of a school-based intervention 6 years in duration.

The Positive Action Program

Positive Action (PA) is a comprehensive, school-based social-emotional and character development (SECD) program (Flay & Allred, 2010). In line with the PYD perspective, the program is designed to apply integrative and social-ecological theories of health and well-being such as the Theory of Triadic Influence (TTI; Flay & Petraitis, 1994; Flay, Snyder, & Petraitis, 2009; Snyder, in press). The TTI provides a unique and comprehensive perspective on behavior and its causes. The TTI consists of three streams of influence: intrapersonal, social, and cultural environment. These streams of influence are consistent with the ecological rings of socio-ecological models. Within these streams exist multiple levels of causation ranging from proximal to distal. Additionally, the TTI considers both cognitive and affective processes within each stream of influence. More specifically, the program, which includes a PreK-12 classroom-based curriculum, emphasizes building students' skills to make responsible decisions, solve problems effectively, recognize and manage their emotions, appreciate the perspectives of others, handle interpersonal situations effectively, be honest with themselves and others, and establish positive goals (Durlak et al., 2011; Snyder & Flay, 2012).

The PA program's sequenced classroom curriculum consists of about 140 15-minute, age-appropriate lessons per grade taught 4 days per week for grades PreK-6, and 70 lessons per grade taught 2 days per week for grades 7 and 8. The program also includes additional classroom components as well as components focused on teacher training, counselor and family training, and school-wide climate development. See Flay and Allred (2010); Flay, Allred, and Ordway (2001) for more detail about program philosophy and content.

Quasi-experimental and experimental evaluations, including the evaluation from which data for the present study are taken, have indicated favorable effects of PA on a range of student academic, behavioral, and emotional outcomes, including academic achievement (Bavarian et al., 2013; Flay et al., 2001; Snyder et al., 2010), motivation and attitudes toward learning (Bavarian et al., 2013), absenteeism (Bavarian et al., 2013; Flay & Allred, 2010; Snyder et al., 2010), disciplinary referrals and suspensions (Flay et al., 2001; Lewis, Schure, et al., 2013; Snyder et al., 2010), bullying, disruptive, and other problem behaviors (Beets et al., 2009; Flay et al., 2001; Lewis, Schure, et al., 2013; Li et al., 2011), and emotional health (Lewis, DuBois, et al., 2013). Findings also have supported favorable effects of PA on school environment and school quality (Snyder, Vuchinich, Acock, Washburn, & Flay, 2012). The TTI theory, and its application in the Positive Action intervention, proposes that the mechanism driving these beneficial effects is due to the intervention's impact on PYD developmental strengths and resources.

Present Study

The present study examines the effectiveness of PA for strengthening indicators of PYD within a matched-pairs cluster-randomized controlled trial of the program in low-performing Chicago public schools. The study thus addresses the need for understanding how well school-based SECD programs such as PA can promote PYD, including specifically when such programs are implemented in urban, low resource schools. Additionally, the study extends previous research on PYD outcomes to include younger children who are followed over time into the early stages of adolescence. We hypothesized that students in the PA intervention would have more growth in indicators of PYD than those in the control group. In addition we predicted that students in the PA intervention would have less growth in a feature of social development, deviant peer affiliations, that interferes with positive youth development.

Method

Design and Sample

The 14 schools participating in the study were drawn from 483 K-6 and K-8 Chicago Public Schools. Sixty-eight schools met eligibility criteria, of which 18 agreed to participate, and the seven best-matched pairs were selected for participation (Ji, DuBois, Flay, & Brechling, 2008; Schochet & Novak, 2003). The following variables from the 2003-2004 CPS data were used as matching variables: percentage of White, African American, Hispanic, and Asian students; percentage of students who met or exceeded standards on the state achievement tests; attendance rate; truancy rate; percentage of students who received a free or reduced-price lunch; percentage of students who enrolled or left school during the school year (mobility); number of students per grade; percentage of parents who were involved with school activities; and percentage of teachers employed by the school who met minimal teaching standards. We also used information about the crime rate in each school's neighborhood (Chicago Police Department, n.d.). Using these variables and an additional requirement that each school in a pair be located in the same region of the city, the 18 schools were successfully matched into nine pairs. Schools were matched using a SAS program provided by Mathematica Policy Research (MPR). The matching variables listed above were entered into the SAS matching program, as well as the school's region code so that schools in similar regions could be matched. The SAS program matched schools into pairs based on a “distance matrix method”, and the best fit-squared distance matrix was utilized. The request for application (RFA; #NCER-03-06) requested five pairs for the trial, and final funding allowed for seven pairs. The seven best-matched pairs, which also happened to provide the most accurate representation of the ethnic composition of CPS schools, were recruited for participation. A random-number generation function in the Microsoft Excel program was used by the principal investigator (PI) to randomize the schools within each pair to control and treatment conditions. Allocation to condition was blinded until assigned by the PI. A series of t-tests revealed that the PA and control schools were not significantly different from each other on any of the matching variables at baseline or at several other points during the study (Lewis et al., 2012) and that these schools as a group did not significantly differ from the remainder of the 68 schools eligible for the study (Flay, 2012; Ji et al., 2008).

The trial was longitudinal with a place-focused, intent-to-treat design (Vuchinich, Flay, Aber, & Bickman, 2012). Specifically, a grade cohort of students in the seven matched pairs of schools was followed, beginning in grade 3 (fall 2004), and at seven additional times (waves) over six years: spring 2005, fall 2005, spring 2006, spring 2007, fall 2008, spring 2009, and spring 2010 (end of grade 8). Parental consent and student assent were obtained before students, parents, or teachers completed surveys when students were in grade 3. Seventy-nine percent of parents provided consent at baseline and consent rates ranged from 65% to 78% for Waves 2-5. Students joining the study at later waves were consented at that time. In accordance with the IRB-approved protocol for the study, once students were consented and assented at their entry into the study, they were not consented or assented again until the second phase of funding at Wave 6 (beginning of grade 7). As is the norm (Ji, Pokorny, & Jason, 2004; Thompson, 1984), consent rates were lower at grades 7 and 8 (58 to 64%) than at grades 3-5.

The total number of students enrolled in the study across all eight waves was 1,170, of whom 53% were female; 48% were African American, 27% Hispanic, 7% White and 12% other (e.g., Asian, and Native American, and “Other”). The average number of waves per student was 3.1. With respect to maintenance of the baseline sample size, 363 students were present at Wave 8 (i.e., 58% of the Wave 1 sample size); the decrease in N over time is consistent with the lower consent rates in grades 7 and 8 and a trend among Chicago Public Schools to decrease in size during the study period. Of these 363 students, 121 were present for the entire study (grade 3 through grade 8). The place-focused intent-to-treat design allows for this kind of mobility as late entrants to the trial are included in the study (Vuchinich et al., 2012).

Implementation

Fidelity of implementation was assessed using several sources of data, including teacher reports of amount of program activities delivered in the classroom, as well as reports of both the school staff person overseeing implementation and the member of the research team who provided technical assistance to treatment schools regarding implementation. In general, there was variability between schools in implementation, especially in initial years, with improvements over the course of the trial (Bickman et al., 2009). By the end of Year 6, implementation fidelity was assessed as moderate for one school, moderate to high for 3 schools, and high at the remaining schools. Students also reported at each wave on their overall satisfaction with the program. The mean quality of delivery rating ranged from 2.88 to 3.56 (minimum=1, maximum=4) across the waves of the trial.

Data Collection Procedures

Student surveys were administered by research staff during class time. During Waves 1 through 5, research staff read survey instructions and items aloud; beginning at Wave 6, students read the individual survey items themselves, although complicated items (e.g., items with double negatives) were explained.

Measures

All measures used in the present study were student self-report and were collected at all eight waves. Measures of some of the constructs included items assessing not only positive, but also negative facets of the construct (e.g., prosocial as well as deviant peer affiliations). This aspect of the measures allowed for findings to inform understanding of effects the PA program may have on both the presence of various indicators of PYD and on attitudes, skills, or behaviors that are likely to interfere with, or serve as countervailing sources of influence on, positive development in those areas. For each measure, a composite score (average of items) was created, with higher scores reflecting more positive scores on the construct being assessed for positively oriented scales, and higher scores reflecting more negative scores on the negatively oriented scale. Unless noted, scores on measures were normally distributed. Table 1 presents measurement information.

Table 1.

Measures Information.

Scale	Source	# of Items	a range across waves	Example Item	Response Options
Outcome
Self
Self-development	Social-Emotional and Character Development Scale (SECDS); Ji, DuBois, & Flay, 2013	4	.64-.83	“I try to be my best”	“None of the time” (1) to “All of the time” (4)
Self-control	SECDS; Ji et al., 2013	4	.64-.82	“I follow the rules even when nobody is watching”	“None of the time” (1) to “All of the time” (4)
Self-concept	Positive Action Inc., 2007	4	.73-.90	“I feel good about what I'm doing”	“None of the time” (1) to “All of the time” (4)
Peer Affiliations
Prosocial Peer Affiliation	Elliott et al., 1996	3	.59-.72	“How many of your friends are interested in school?”	“NONE of my friends” (1) to “ALL of my friends!” (4)
Deviant Peer Affiliation	Elliott et al., 1996	4	.81-.87	“How many of your friends get into fights at school?”	“NONE of my friends“ (1) to “ALL of my friends!” (4)
Ethics
Positive Morality	Arthur, Hawkins, Pollard, Catalano, & Baglioni, 2002	6	.70-.83	“It is important to play by the rules, even if no one is watching”	“NO!” (1) to “YES!” (4)
Negative Morality	Arthur et al., 2002	5	.61-.79	“It is OK to hit someone if they hit you first”	“NO!” (1) to “YES!” (4)
Altruism	Solomon, Battistich, Watson, Schaps, & Lewis, 2000	8	.81-.87	“I helped someone who was hurt”	“Never” (0) to (3) “Many times”
Empathy	Funk, Fox, Chan, & Curtiss, 2008	16	.79-.89	“I understand how other kids feel”	“No” (1), “Sometimes” (2), and “Yes” (3)
Social Skills
Respect for Parents	SECDS; Ji et al., 2013	4	.61-.89	“I listen (without interrupting) to my parents”	“None of the time” (1) to “All of the time” (4)
Respect for Teachers	SECDS; Ji et al., 2013	5	.71-.90	“I speak politely to my teacher”	“None of the time” (1) to “All of the time” (4)
Prosocial Interactions	SECDS; Ji et al., 2013	6	.77-.85	“I play nicely with others”	“None of the time” (1) to “All of the time” (4)
Honesty	SECDS; Ji et al., 2013	5	.71-.82	“I tell the truth when I have done something wrong”	“None of the time” (1) to “All of the time” (4)
Aggressive Problem Solving	Aber, Brown, Jones, & Samples, 1995	5	.65-.84	“Push a child back that pushes you”	--
Competent Problem Solving	Aber et al., 1995	5	.57-.71	“Ask a child who is making fun of you to stop”	--

Aggressive and Competent Problem Solving scores were created from five social scenarios in which youth chose one response and the response indicated an aggressive response, competent response, or neither.

All measures were assessed for measurement invariance across time following guidelines from Cheung and Rensvold (2002), including the recommendation to consider a Comparative Fit Index change of greater than 0.02 when comparing models with varying restrictions on invariance to be a significant change. We tested for weak (i.e., loading) and strong (i.e., intercept) invariance. As indicated below, five of the measures were found to have weak invariance across the 8 waves of assessment, and four had strong invariance; invariance results for each measure are given below. Six scales did not have invariance.

Self

Indicators of the self were assessed using measures of self-development, self-control, and self-concept. Self-development and self-control were measured using the respective subscales from the 28-item Social-Emotional and Character Development Scale (SECDS; DuBois, Ji, Flay, Day, & Silverthorn, 2010; Ji, DuBois, & Flay, 2013). In validation research using data from the larger trial, these subscales of the SECDS showed evidence of criterion validity based on significant correlations in expected directions with indices of outcomes such as substance use, violence, altruism, and life satisfaction (Ji et al., 2013). Scores on both subscales were negatively skewed. Self-concept (Positive Action Inc., 2007) was also assessed. Scores on this measure were negatively skewed as well. In the present trial, this scale exhibited associations in expected directions with indices of a range of other outcomes, including school attachment, substance use, and global self-esteem. Self-development and self-control had weak invariance, whereas self-concept had strong invariance.

Peer Affiliations

Indicators of peer affiliation were assessed using items created for the present trial; these items were adapted from the Conventional Friends scale (Elliott et al., 1996). Scores for the prosocial peer affiliation subscale of this measure were negatively skewed, whereas those for deviant peer affiliation were positively skewed. These scales exhibited expected associations with a range of other outcomes assessed in the trial, including measures of substance use, life satisfaction, and depression. Both measures had strong invariance.

Ethics

Students' ethical development was assessed with a reduced and slightly reworded version of the Belief in Moral Order scale (Arthur, Hawkins, Pollard, Catalano, & Baglioni, 2002). The distribution of scores on positive morality was negatively skewed at each wave whereas for negative morality distributions were positively skewed. In support of their validity, in prior research these subscales have exhibited significant correlations in expected directions with theoretically related outcomes, such as substance use (Arthur et al., 2002).

Additionally, ethics was also measured using eight items assessing altruism (Solomon, Battistich, Watson, Schaps, & Lewis, 2000); one item was dropped for controversial content (I stopped someone from hurting an animal). Finally, ethics was also assessed using the Children's Empathic Attitudes Questionnaire (Funk, Fox, Chan, & Curtiss, 2008). In support of their validity, both measures have been found to be correlated with theoretically related outcomes such as other measures of child empathy and prosocial behaviors (empathy) and other positive social behaviors (altruism), in the expected direction (Funk et al., 2008; Kaminski, Battistich, & David-Ferdon, 2009). Morality had strong invariance, and altruism had weak invariance. Empathy did not have invariance across waves.

Social Skills

Social development was assessed using measures of prosocial interactions with peers and respect, honesty, and problem solving skills. Prosocial interactions, respect for parents and teachers, and honesty were measured using subscales from the previously noted SECDS. These subscale showed evidence of validity similar to that noted above for other subscales on the SECDS (Ji et al., 2013). Prosocial interactions and respect for parents had weak invariance. The remaining scales did not have invariance across waves.

Student's ability to solve interpersonal problems was assessed using five of the eight scenarios from The Social Skills Problem Solving Measure (Aber, Brown, Jones, & Samples, 1995). Each scenario depicts an interpersonal problem, with students choosing which of five possible responses they feel would help solve the problem. Following Aber et al. (1995), we computed scores to represent students’ selection of aggressive and competent social problem solving responses, respectively. Because of their markedly skewed distributions, the scores on each measure were then dichotomized to indicate any use of aggressive or competent problem solving (coded as 1) versus no use of aggressive or competent problem solving (coded as 0). These measures showed evidence of validity in prior research with measures of behavioral adjustment to school (Valente, 1995). Neither scale showed invariance across waves.

Analysis

Primary analyses consisted of estimating two-level growth curve models (waves of observations nested in students) that tested whether the PA intervention had a beneficial effect on PYD-related measures from 3^rd to 8^th grade. This approach allows for an analysis of the multiple waves of available data and takes into account individual variation across students as well as the trajectory of change over time. An important feature of including a random intercept for child is that it provides a statistical control for individual differences that may be due to personality, family characteristics, or pubertal status. Alternative analyses were done for these outcomes in which the random intercept represented variation due to school rather than child, and results were similar (available from first author upon request); therefore only one set of results (variation due to child) are presented. The varying distributions of the measures were taken into account in the specification of the models. Stata's version 12 “xttobit” command was used for most of the outcomes because the scores had a generally normal distribution but excess frequencies of the highest or lowest scores, i.e., ceiling or floor effects. This typically indicates that a normal distribution would have been possible had more extreme item options been used, but the higher scores were censored due to the response options. The censored regression tobit model in “xttobit” provides the most accurate statistical specification for such outcomes, but allows only for two-level model estimation. Stata's “xtmixed” command was used for normally distributed outcomes and “xtmelogit” for binary outcomes. These latter two commands allowed for three-level model estimation (waves of observation within students within schools).

For the growth curve analysis of each measure, we first estimated a random-intercept model including condition (i.e., PA or control school), time (measured as study duration in years), condition by time (condition × time), and quadratic terms for time and the interaction of condition by time (time² and condition × time², respectively). Quadratic terms were included initially to test for nonlinear change and higher order terms found to lack statistical significance were then dropped from the model. For the three-level model this analysis was followed by adding tests for whether the trajectory of change differed across students (i.e., a random coefficient model). A likelihood-ratio test was performed to determine whether the random coefficient model provided a better fit than the random intercept model (Rabe-Hesketh & Skrondal, 2008). Model estimates for the better fitting models are reported in the appropriate tables. Censored (“xttobit”) models did not allow for random coefficient models, and are all random intercept models. For all analyses, missing values were handled using full information maximum likelihood estimation (Enders & Bandalos, 2001). Effect sizes for continuous measures in the form of standardized mean differences were computed using estimated means at baseline and study end-point from fitted models and observed standard deviations (Lipsey & Wilson, 2001), and using the Cox transformation in the case of binary outcomes(Sanchez-Meca, Marin-Martinez, & Chacon-Moscoso, 2003).

Statistical power for detecting intervention effects was limited by the small number of schools (Social and Character Development Research Consortium, 2010). The minimum detectable effect size (MDES; Bloom, 1995) was calculated using Optimal Design (Spybrook et al., 2011); specifically, using the “Cluster Randomized Trials with person-level outcomes” and “Repeated measures” options within the program. The MDES for power of .80, for intraclass correlations (ICC) for observations within the 14 schools of .01, .05, .10, and .15 and accounting for multiple waves of data (by utilizing the repeated measures option) and for clustering are, .28, .42, .54, and .64, respectively. Two measures were binary; therefore and ICC was not appropriate and we instead report the Median Odds Ratio (MOR; Merlo et al., 2006). The average measure within child ICC across waves for outcomes used in the present study ranged from .24 to .42, and MORs were 4.15 and 8.08. ICCs for within school were available for three-level models only (empathy, altruism, and the problem solving measures). The average ICC within school across waves was 0 to 0.19; the average MOR was 1.00 to 2.32. Given the large ICCs and MORs within child, two-level analyses controlled for clustering within child rather than within school. Our use of a matched-pairs design, multiple waves of measures, and appropriate modeling of outcomes based on the shape of the distribution all served to help improve the level of power (Raudenbush, Martinez, & Spybrook, 2007; Shadish, Cook, & Campbell, 2002).

Sensitivity analyses assessed the robustness of results from the primary analyses. One approach involved including a “pairs” variable as an additional level in each of the best-fitting models as random assignment was done at the pair level. Second, to provide a more conservative test of program effects for each outcome, the test z statistic provided by Stata in the primary analyses (N=14 schools) was compared to the critical value for a two-tailed t-distribution with 12 degrees of freedom (based on the number of schools) at a 95% confidence level (Raudenbush & Bryk, 2002). Additionally, the Benjamini Hochberg false discovery rate (Benjamini & Hochberg, 1995) was used to adjust for testing of program effects on multiple measures; results are presented in Table 2.

Finally, possible moderating effects of gender and student mobility were examined. The latter analyses utilized student mobility groups obtained from a latent class analysis (Flay, 2012) that identified the following mobility groups: 1) stayers (average study duration of 5.72 years, 13%), (2) temporary participants (present for grade 4 and/or 5 only; average study duration of 1.30 years; 16%), 3) late joiners (average study duration of 1.38 years; 25%); 4) early leavers (average study duration of .94 years; 22%), and 5) late leavers (average study duration of 3.23 years; 24%); stayers served as the reference group.

Results

Findings from the analyses of PA program effects on PYD developmental resources are presented in Table 2. The negative time coefficients indicate a decline in both intervention and control groups in these developmental resources from childhood to adolescence. However, the statistically significant time-by-condition interaction effects show that students in the PA intervention had significantly less decline than those in the control group. Table 3 reports the baseline and endpoint predicted means from these analyses by condition, together with effect sizes. These predicted means take into account the quadratic change pattern in the growth curve. Although a significant quadratic trend was evident for some measures, it can be seen from the predicted means in Table 3 that there was still a clear net decline in scores on these measures from Wave 1 to 8 for youth in both PA and control schools. In general, the quadratic trends were in the direction of somewhat accelerating change over time for positively-valenced measures and deceleration for those that were negatively-valenced.

Table 2.

Estimates from Multi-level Growth Model Analyses (N=1,170 students)

Scale	Model Type	Intercept	Condition	Time	Condition × Time	Time2
Outcome				B (SE)
Self
Self-Developmenta	RI	3.83 (.04)	−.07 (.06)	−.11 (.01)***	.03 (.02)†	--
Self-Controla	RI	3.55 (.04)	−.06 (.05)	−.41 (.03)***	.06 (.02)***	.04 (.005)***
Self-Concepta	RI	3.57(.03)	−.03 (.04)	−.10 (.01)***	.03 (.01) *	--
Peer Affiliations
Prosocial Peer Affilationa	RI	3.49 (.04)	−.09 (.06)	−.31 (.03)***	.06 (.02)***	.03 (.005)***
Deviant Peer Affiliationa	RI	1.73 (.05)	.15 (.07)*	.15 (.03)***	−.08 (.02)***	−.01 (.006)*
Ethics
Positive Moralitya	RI	3.63 (.04)	−.07 (.05)	−.02 (.03)	.01 (.02)	−.01 (.005)*
Negative Moralitya	RI	1.09 (.04)	−.02 (.05)	.30 (.03)***	−.05 (.02)**	−.02 (.005)
Altruismb	RC	1.64 (.05)	−.07 (.06)	−.26 (.02)***	.03 (.01)†	.03 (.004)***
Empathyb	RC	2.42 (.03)	.04 (.05)	−.20 (.01)***	.02 (.01)†	.02 (.002)***
Social Skills
Respect for Parentsa	RI	4.10 (.04)	−.11 (.06)†	−.20 (.01)***	.06 (.02)***	--
Respect for Teachersa	RI	4.05 (.06)	−.09 (.06)	−.31 (.03)***	.07 (.02)***	.02 (.005)***
Prosocial Interactionsa	RI	3.64 (.04)	−.04 (.05)	−.30 (.02)***	.05 (.01)***	.03 (.004)***
Honestya	RI	3.70 (.04)	−.08 (.05)†	−.35 (.03)***	.05 (.01)**	.03 (.004)***
OR (95% CI)
Aggressive Problem Solvingb	RI	.15 (.08, .31)	.73 (.28,1.86)	4.33 (3.41, 5.51)***	.75 (.66, .86)***	.88 (.85,.92)***
Competent Problem Solvingb	RI	37.57 (18.87, 74.81)	1.27 (.55, 2.91)	.55 (.40, .76)***	1.06 (.87, 1.28)	1.12 (1.06, 1.19)***

Note:

^†p< .10

^*p< .05

^**p< .01

^***p < .001.

RI = Random intercept, fixed slope model. RC = random coefficient (random intercept and random slope) model; LR tests used to determine preferable models. All model significance remained the same after Benjamini Hochberg False Discovery Rate adjustment (Benjamini & Hochberg, 1995) was calculated. Non-significant quadratic terms were removed from model in the interest of parsimony. Three measures showed baseline differences, all favoring control.

^aTwo-level model.

^bThree-level model.

Table 3.

Wave 1 and Wave 8 predicted (model) means and probabilities by condition (N=1,170 students).

Variable	Wave 1		Wave 8
	PA Mean	Control Mean	PA Mean	Control Mean	Effect Size
Self
Self-development	3.76	3.83	3.30	3.21	.29
Self-control	3.49	3.55	2.89	2.61	.50
Self-concept	3.54	3.57	3.14	3.01	.24
Peer Affiliations
Prosocial Peer Affiliation	3.40	3.49	3.15	2.88	.48
Deviant Peer Affiliation	1.88	1.73	1.87	2.16	−.46
Ethics
Positive Morality	3.56	3.63	3.11	3.11	.10
Negative Morality	1.07	1.09	1.87	2.17	−.40
Altruism	1.57	1.64	1.32	1.22	.21
Empathy	2.46	2.42	2.10	1.96	.26
Social Skills
Respect for Parents	3.99	4.10	3.21	2.98	.68
Respect for Teachers	3.95	4.04	3.13	2.84	.78
Prosocial Interactions	3.60	3.64	3.08	2.84	.50
Honesty	3.62	3.70	2.88	2.68	.57
Aggressive Problem Solvinga	.21	.21	.62	.79	−.76
Competent Problem Solvinga	.95	.94	.96	.97	.05

Note: Predicted means for censored outcomes may fall outside of the range as an effect of the estimator.

^aIndicates predicted probabilities; effect size is Cox's d (Sanchez-Meca et al., 2003).

Self

For self-control, there was a significant effect of PA, as evidenced in tests of the condition × time interaction, in the direction of mitigating an overall decline for this measure over time. Additionally, there were marginal program effects for self-development and self-concept. As with self-control, students receiving PA showed less of a decline in these favorably-valenced outcomes than did students in the control group. The preceding effects remained at least marginally significant in sensitivity analyses for self-control and self-development, but not for self-concept (results not shown).

Peer affiliations

For both measures of peer affiliation, there was a significant effect of PA. For prosocial peer affiliation, these results indicated that PA mitigated a general decline in scores on this measure over time. Students in PA schools also exhibited less increase over time on the measure of deviant peer affiliation than did students in control schools. In sensitivity analyses, program effects on these measures remained evident (significant or marginally significant).

Ethics

There was a significant program effect for altruism and a marginal effect for empathy. In both instances, students in PA schools showed less of a decline over time than did control school students. There was no program effect for positive morality. For negative morality, there was a marginal program effect with students in PA schools showing less increase over time compared to control school students. Program effects remained evident in the pair-level sensitivity analysis for altruism and empathy, and in both sensitivity analyses for negative morality.

Social skills

For respect for parents and prosocial interactions, there was a significant effect of PA, revealing that PA mitigated the decline in these measures. Students in PA schools also exhibited less increase over time on the measure of aggressive problem solving than did students in control schools. There were no program effects for competent problem solving. In sensitivity analyses, all program effects remained evident (significant or marginally significant).

No differences in program effects were found by mobility status or gender for any outcome.

Discussion

The present study's finding of favorable effects of the PA program on indicators of PYD is consistent with prior research in which school-based interventions have been found to be beneficial in similar areas (e.g., Durlak et al., 2011). Our findings indicating beneficial program effects on PYD resources are an especially noteworthy contribution given that prior research suggests less effectiveness for school-based interventions in low-income urban contexts for ethnic minority youth (Farahmand et al., 2011). It could be that PA is especially well designed to impact indicators of PYD, at least those that we assessed. Furthermore, the universal orientation in which PA is delivered to all students in the school is consistent with findings of greater effectiveness of school-based programs for improving mental health and behavioral outcomes among low-income urban ethnic minority youth (Farahmand et al., 2011).

This study shows that the development of beneficial PYD-related characteristics can be enhanced in school-based interventions in an urban environment across the elementary and middle school years. This extends the age range for developmental studies of PYD-related outcomes well prior to adolescence and demonstrates the value of early intervention. The PA intervention has been previously shown to prevent several negative adolescent outcomes in other settings such as Hawai'i (Beets et al., 2009) as well as this setting (Lewis et al., 2012; Lewis, Schure, et al., 2013). In addition, the PA intervention has been shown to have effects on academics (Bavarian et al., 2013) and emotional health (Lewis, DuBois, et al., 2013). Considered together with these findings, those of the current study thus raise the possibility that preventive benefits of PA may be attributable at least in part to enhancement of PYD-related strengths among students. Brandstädter (2006), for example, theorized that the bidirectional and influential relationship between a person and their context can contribute to greater functioning. There is other empirical evidence consistent with this perspective in adolescents; for example, Lewin-Bizan, Bowers, and Lerner (2010) show parenting to be related to self-regulation behaviors which, in turn, influence PYD, and PYD is in turn related to greater contribution to society. Thus, the influence of PA on youth development reported in this paper, as well as the influence of PA on school climate reported in Snyder et al. (2010), may lead youth to have better functioning, e.g., fewer problem behaviors, better emotional health, and greater academic achievement. Improvements in functioning as a result of PA may in turn then help to shape the contexts in which such improvements are developing (e.g., school, family).

The present findings of a normative decline in PYD resources are consistent with previous developmental studies (Carlo, Crockett, Randall, & Roesch, 2007; Kokko, Tremblay, Lacourse, Nagin, & Vitaro, 2006; R. M. Lerner, Lerner, & Phelps, 2008; Nantel-Vivier et al., 2009). There has been little discussion of reasons for this decline in the literature as some developmental theory would expect capabilities in prosocial development to improve during this age range (Eisenberg & Morris, 2004). In general, adolescents engage in more risky behaviors and less positive behavior; additionally, youth who develop more positive skills engage in less negative behavior (e.g., Lowe, Acevedo, Griffin, & Botvin, 2013). Kokko et al. (2006) and Nantel-Vivier et al. (2009) suggested that more work is needed to understand the decline, while Carlo et al. (2007) and Washburn et al. (2011) suggested that environmental factors such as exposure to high-risk conditions and lack of access to protective factors (positive role models, opportunities for constructive interactions, and emotional support) could influence these negative trajectories, particularly for students from low-income homes in urban environments.

This study had some limitations. We did not directly assess the 5 Cs, a dominant framework of PYD. The set of measures we did use are strengths, however, in that they assess core features of adolescent development. Assessing these characteristics eight times from age 8 (grade 3 through to age 14, end of grade 8) is also a major strength. Student outcomes were assessed through student self-report, potentially leading to a method bias (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003). Self-reports are also susceptible to social desirability such that students may overstate their positive thoughts, feelings, and behaviors or understate their negative thoughts, feelings, and behaviors to feel as if they “fit in” with expectations of their peers and society. With particular relevance to this investigation, the possibility exists that students in PA schools could have felt greater pressure to report favorably on measures in line with the messages and goals of the program, thus leading to some inflation of estimated program effects. Several measures did not have invariance over time. Measurement invariance may not be a reasonable expectation in a study such as this when we expect there to be developmental change as well as change due to intervention effects for the treatment group. It is important to note, however, that we have no a priori reason to expect that key aspects of invariance (e.g., differences over time in how youth interpreted items on scales) would not be similar for PA and control youth, thus supporting the meaningfulness of our treatment effect estimates (Geldhof et al., 2014). Additionally, given the comprehensive nature of PA, measures were used that provided relatively global assessments of constructs (e.g., overall self-concept rather than domain-specific). Future research should also explore how more specific indicators of positive youth development in these and other areas may be influenced by programs.

With respect to external validity, given the sampling design, the findings have some generalizability to low income schools in Chicago, and perhaps to similar environments: low-income, inner-city schools that would self-select to participate in a trial of this nature. With respect to statistical conclusion validity, the small number of pairs (i.e., seven) and schools (i.e., 14) limited statistical power; however, that evidence of significant program effects was found in spite of this limitation suggests that our findings may be conservative. Additionally, as has been seen in other studies within low income, urban school settings (Tobler & Komro, 2011), there was a relatively high turnover of students across the years. However, this study used latent class analysis (Beunckens, Molenberghs, Verbeke, & Mallinckrodt, 2008; Marsh, Lüdtke, Trautwein, & Morin, 2009) to show that the mobility pattern of the students did not moderate the effects of the intervention.

In addition to addressing the foregoing limitations, future research should test whether the distal effects of programs such as PA on youth outcomes such as problem behavior involvement, mental health, and academic achievement are attributable to (i.e., mediated by) program effects on PYD resources individually and/or in combination. Expanding research to examine whether interventions can influence the development of PYD characteristics starting at earlier ages and whether effects are sustained at later points in development would also be valuable.