Youth Empowerment Solutions: Evaluation of an after-school program to engage middle school students in community change

Abstract

We report on an effectiveness evaluation of the Youth Empowerment Solutions (YES) program. YES applies empowerment theory to an after-school program for middle school students. YES is an active learning curriculum designed to help youth gain confidence in themselves, think critically about their community, and work with adults to create positive community change. We employed a modified randomized control group design to test the hypothesis that the curriculum would enhance youth empowerment, increase positive developmental outcomes, and decrease problem behaviors. Our sample included 367 youth from 13 urban and suburban middle schools. Controlling for demographic characteristics and pretest outcome measures, we found that youth who received more components of the curriculum reported more psychological empowerment and prosocial outcomes and less antisocial outcomes than youth who received fewer of the intervention components. The results support both empowerment theory and program effectiveness.

Individually focused behavior change programs for youth have limitations when conducted without consideration of the environmental context within which unhealthy and unsafe behaviors occur (Branas & Macdonald, 2014). One reason for this limited effectiveness may be that most approaches to enhancing positive youth development fail to involve youth as agents of change. Empowerment theory (Zimmerman, 2000) provides a useful conceptual framework for understanding how after-school programs can enhance positive youth development (PYD) by engaging youth in developing confidence, skills, and behavioral strategies to achieve self-identified goals. After-school programs are empowering if they help youth develop a sense of agency to make a difference in the world, learn critical thinking skills, become independent decision makers, and practice their skills in real life contexts (Zimmerman, 2000). Empowering processes are consistent with the NRC/IOM (2002) characterization of successful positive youth development settings and should lead to empowerment outcomes (e.g., self-efficacy, critical awareness, participation in organized activities).

Zimmerman (1995) describes empowerment outcomes as having three interrelated components: 1) intrapersonal; 2) interactional; and 3) behavioral. The intrapersonal component includes beliefs regarding control and confidence that one can make a difference (e.g., perceived leadership ability, self-esteem). The interactional component includes critical awareness of forces that shape one’s life and an understanding of the resources and actions needed to create situations that are more aligned with one’s goals (e.g., resource analysis, instrumental support). The behavioral component refers to actions individuals take to make desired changes in their environment (e.g., involvement in prosocial activities). The specific measures used to operationalize psychological empowerment (PE) must be context-specific because different contexts require different efficacy beliefs, critical analyses, and actions to achieve goals (Zimmerman, 1995).

Empowerment theory has been applied effectively in programs to prevent HIV/AIDS among Mexican men (Zimmerman, et al., 1997), youth alcohol and substance use (Wallerstein & Bernstein, 1988), and youth violence (Reischl et al., 2011). Empowerment-based programs for youth have focused on providing supportive contexts where youth build assets, connect with local resources and adult role models, and engage in community change activities (Zimmerman & Eisman, in press; Gardner, Roth, & Brooks-Gunn, 2008; Zimmerman, 1995, 2000). Researchers have reported that constructive community engagement is associated with identity, well-being, self-perception, and educational attainment (Coatsworth et al., 2006; Haggard & Williams, 1992; Jacobs et al., 2005; Mahoney et al., 2006).

Researchers have noted that early adolescence, which is characterized by growing levels of autonomy (Busser et al., 1996; Collins et al., 1997) and identity development (Erikson, 1968), is an opportune time to intervene to promote positive outcomes. Providing opportunities for positive development during early adolescence prepares youth for successful futures (Pittman et al., 2003; Eccles & Gootman, 2002; Gardner et al., 2008; Mueller, et al., 2011; Pollard et al., 1999).

While many youth-focused interventions use the term empowerment, most fail to explicitly apply an empowerment-based theoretical framework that guides the intervention content or the theory of change expected in the program. In addition, few empowerment interventions have been evaluated with experimental or quasi-experimental designs. Some programs have aspects of empowerment theory as noted above, but we know of no program that incorporated all three components (intrapersonal, interactional, behavioral) into a curriculum that also examined both direct effects on youth outcomes and its mediating (indirect) effects through enhancing youth’s sense of (psychological) empowerment. The Youth Action Research for Prevention (YARP) intervention (Berg et al., 2009) and the Teen Empowerment Program (TEP; Pearrow, 2008) are two example of youth empowerment programs. YARP engaged youth in decision making that involved school policies and practices to improve social conditions (Berg et al., 2009). The TEP program hired youth to work with adults to review community conditions, recognize problems and develop solutions (Pearrow & Pollack, 2009). These two programs are informed by empowerment theory generally, but do not explicitly include all three components of empowerment theory (intrapersonal, interactional and behavioral) or consider the critical interdependence among components to enhance empowered outcomes. The evaluation of these programs also focused on process analysis and implementation issues (Phillips, et al., 2010) or non-experimental post-only assessments (Schutt & Gecker, 2013). These evaluations provided useful information about processes for empowering youth, but they were not designed to determine program effects with consideration for internal validity or examine the mechanism by which empowering processes may enhance psychological empowerment which in turn improves positive youth development outcomes.

Youth-led participatory action research (YPAR) is an empowerment approach designed to train youth in research skills so they may identify and analyze problems they define (Ozer, In press). YPAR training includes several factors associated with empowerment outcomes such as problem solving and critical thinking skills, communication, and teamwork. Although most research on YPAR is non-experimental, Ozer & Douglas (2013) used an experimental design to study the effects of YPAR on youth outcomes and found that youth in a YPAR class versus those in a peer education class reported more strategic thinking and motivation for change. Research on YPAR has focused on research skills and empowerment outcomes as opposed to more general application, and thus far has not examined empowerment as a mediating factor for positive developmental outcomes. Our study thus extends the prior literature on youth empowerment by testing a mediating model of youth empowerment that includes all three components of empowerment theory, examining positive youth development as the final outcomes studied, and focusing the empowerment process on community change activities (versus research as the vehicle for change).

The Youth Empowerment Solutions (YES)

We developed the Youth Empowerment Solutions (YES) program to engage middle school youth in promoting positive community change as a programmatic means to enhance positive development. YES incorporates empowerment theory through content focused on skill and confidence building (intrapersonal component), activities to help youth think critically about and develop a connection with their community (interactional component), and designing and implementing a community change project (behavioral component). YES expands research on YPAR by focusing on positive youth development-related outcomes, examining the mediating effects of empowerment, and incorporating a structured curriculum that broadens the focus for youth beyond research. YES and YPAR have similar approaches for helping youth gain voice in the process of change including the use of photovoice strategies, a focus on adultism, and collaborative work on a community change project. YES, however, also explicitly involves intergenerational change efforts and cultural identity, and is guided explicitly by empowerment theory.

YES was developed through an iterative process with community partners and youth participants in Flint, Michigan (Franzen, et al, 2009). The curriculum provides structured lesson plans and is responsive to local culture, creates opportunities for intergenerational collaboration, and prepares youth to plan and implement community change that they would like to see. Using empowerment theory to guide the curriculum content, YES applies an active learning approach organized around six units: 1) Youth as Leaders; 2) Learning About Our Community; 3) Improving Our Community; 4) Building Intergenerational Partnerships; 5) Planning for Change; and 6) Action and Reflection (Zimmerman et al, 2011). Table 1 provides a brief summary of each unit. The program also includes activities designed to help youth understand and embrace their cultural identity and to appreciate other cultures.

Table 1.

Curriculum description by unit

Unit	Unit Description
Youth as Leaders	Builds group norms and team identification Helps build leadership efficacy Engage in leadership roles Opportunities to complete small projects and engage early in program
Learning about Our Community	Provides participants skills to identify and assess community conditions contribute to or prevent healthy development Community windshield tour and Photovoice methods
Improving Our Community	Learn about ways to make a difference through community development Learn to think critically about how actions can help strengthen neighborhoods
Building Intergenerational Partnerships	Work in equal partnership with adults Develop confidence speaking and communicating with adults Learn about power differentials in social relationships Apply decision-making power Interview adult volunteers to work as adult advocates Intergenerational team building
Planning for Change	Learn to turn project ideas into concrete proposals that include project description, goals, budget, and timeline Build support for project among community
Action and Reflection	Project implementation Reflect on project and process Community celebration

Current Study

This study is an effectiveness evaluation of YES. We used a modified randomized control group design to test the hypothesis that the curriculum would enhance youth empowerment, increase positive developmental outcomes (e.g., school bonding; academic achievement; prosocial involvement) and decrease negative developmental outcomes (e.g., antisocial behavior). We hypothesized a mediation effect for YES such that we expected its effects on positive and negative developmental outcomes would operate indirectly through youth empowerment outcomes.

Method

Study Context

Participants were middle school students from Genesee County, MI. The city of Flint and surrounding area lost over 70,000 auto-industry jobs and over half its population in the last 50 years. Genesee County had higher unemployment levels than state and national averages for over a decade (Bureau of Labor Statistics, 2014) and Flint has been ranked as one of the most violent cities in the U.S. with a population over 100,000 (Weigley, Hess, & Sauter, 2013). The transition from a manufacturing-based to service-based economy has been difficult for residents, including young people.

Sample

We recruited 367 middle school youth, aged 11-16 (M=12.71;SD = 0.91; 60% female), who attended after-school programs at 13 schools in Flint and Genesee County. We implemented the program annually over a period of 4 years with a total of 29 different cohorts of youth across the 13 schools. Four schools implemented the program all four years, one school participated for three years, two schools for two years, and six schools participated in the study for one year.

Youth identified as 32% (n =117) white, 46% (n = 170) African-American, 22%; (n = 80) mixed race, Asian-American, Latino, Native American or other ethnic/racial group. Sixteen percent (n = 60) of the participants were from Flint public schools. The remaining participants attended middle schools in nearby suburban districts in Genesee County. All schools had high proportions of students in the free/reduced lunch program (93-99%).

Procedures

Parent consent and youth assent as approved by the university IRB were obtained before completion of the pretest questionnaire and assignment to conditions. Our goal was to recruit enough students at each school so we could randomly assign participants to YES or usual after-school programing (control). YES needs at least five participants in order to implement group activities in the curriculum. All project schools had only one group of youth who worked on a single final project.

Youth were randomized into YES (n=124) or usual after-school care (n=114) if sites had over 12 consented participants. If only 5 to 11 students were recruited at a school during a given year, all participants were enrolled in YES (n=74). If fewer than 5 participants were recruited, all participants were assigned to the usual after-school programs (n=4). Two exceptions to this protocol due to the limitations in the after-school setting were made. First, one school with 3 participants was assigned to YES. Second we recruited youth from schools that did not have usual after-school programming and enrolled them in YES (n=47). Thus, we consider this a modified random assignment because while many youth were randomly assigned (if schools had 12 or more consented youth in after school) some after-school cohorts were assigned to program or control (as noted above) because we had too few to fill both conditions or the school had no alternative after school program. Our group assignment protocol resulted in 249 YES participants and 118 usual after school control condition participants.

The pretest (Time 1) and end-of-program posttest (Time 2) data were used for this study. Participants completed their posttest 1 to 70 days after completing the YES or after-school program. This range of days for completing the posttests was mostly due to time to track down students, however, 79% of the sample had posttests completed within 30 days. Students were similarly distributed across condition (i.e., program & control) and four categories (1-30, 31-40, 41-50, 60+ days) of time to posttest (χ(3)=3.55; ns). Posttests were completed during after-school with group administration or over the phone. Attrition from Time 1 to Time 2 was 6% (n=23). We conducted an attrition analysis for all study variables at time 1 comparing those included in the study and those left out due to missing Time 2 data and found no differences for aggressive behavior, delinquent behavior, responsible decision making, and prosocial behavior, no differences by race/ethnicity or age.

Implementation Fidelity

Local after-school teachers were hired to implement YES. We monitored their implementation through observation and teacher reports of material covered for each session. We also conducted implementation fidelity assessments of randomly selected sessions and teachers for each school across all years of the study to assess how much facilitators followed the basic principles of the curriculum. Eleven trained observers reliably rated on 5-point Likert scales: 1) shared control between youth and facilitators; 2) youth given opportunities to lead; 3) facilitators used respectful language with the youth; and 4) facilitators used a respectful tone with youth. Fidelity observers were accompanied by a senior member of the program team the first time they rated implementation. They conducted independent assessments and discussed their ratings. We assessed inter-rater reliability by percent agreement of the two raters across 15 pairs of raters. If the observer needed to improve reliability, we duplicated the procedures for a second time with a different program team member. Percent agreement for all ratings across the 4 5-point Likert fidelity items ranged from 86-100% for being exact matches or within one point of the 5-point scale.

Measures

Descriptive statistics for all study measures are reported in Table 2.

Table 2.

Descriptive statistics for participant level variables

YES–Related Activities (Time 2)	% yes
Photovoice	19%
Work with a neighborhood advocate	30.5%
Learn cultural traditions	46.9%
Plan a summer project	57.7%
Identify community assets	41.4%
Empowerment subscales (Time 2)	Mean(SD)	Skewness
Intrapersonal component
Leadership efficacy	3.94(.63)	−0.58
Civic efficacy	4.26(.58)	−0.69
Self esteem	4.06(.68)	−0.91
Interactional component
Adult mentorship	4.32(.84)	−1.74
Adult resources	3.78(1.15)	−0.80
Resource mobilization	4.19(.52)	−0.90
Behavioral component
Leadership behavior	3.39(.88)	−0.43
Community engagement	3.50(.96)	−0.79
School engagement	4.05(.62)	−0.86
Outcomes	Mean(SD) Time 1		Mean(SD) Time 2	Skewness
Prosocial outcomes
Prosocial behavior	4.23(.66)	−0.94	4.27(.54)	−0.40
Responsible decision making	4.29(.68)	−0.95	4.29(.57)	−0.70
Academic effort	3.41(.53)	−1.12	3.53(.47)	−1.62
Antisocial outcomes
Aggressive behavior	1.91(.95)	1.14	1.62(.74)	1.60
Delinquent behavior	1.78(.88)	1.78	1.51(.69)	1.90

YES Program Dosage

Program dosage can be operationalized as dose delivered and dose received (Steckler & Linnan, 2002). Dose delivered refers to the amount of the program delivered by providers, whereas dose received examines the degree to which participants were receptive to, engaged with, or put into action program materials, trainings and resources (Baranowski & Stables, 2000; Steckler & Linnan, 2002). Our dose delivered measure was attendance reported by teachers.

Participants reported on the posttest if they participated (yes/no) in YES related activities in their after-school program as a proxy for dose received. These activities covered key components of YES (See Table 2—YES-Related Activities). Scores could range from 0-5.

Psychological Empowerment Components (Time 2)

Intrapersonal component

The intrapersonal component included measures for leadership efficacy, civic efficacy and self-esteem. Leadership efficacy was a 3-item 5-point Likert scaled measure adapted from Zimmerman and Zahniser (1991) that assessed respondents’ beliefs about being a leader in groups and organizing people to get things done (α = 0.55). We measured civic efficacy using three 5-point Likert items asking participants if they felt they could be involved in community change, make their community better by helping others and do things to make the world better (α = .69). Participants also indicated their level of agreement to six items from the Rosenberg (1965) self-esteem scale (α = .77).

Interactional component

The interactional component included measures for mentorship support, adult resources, and resource mobilization. Mentor support was measured by five 5-point Likert items (Vinokur & Van Ryn, 1993; α=0.88). Adult resources included three items (1 = None; 5 = Four or more) assessing how many adults young people knew who could help them with solving neighborhood, school, and city/town problems (α = 0.69). We measured resource mobilization with four 5-point Likert items (e.g., working with others makes a community project better; α = 0.70).

Behavioral component

The behavioral component included measures for leadership behavior, and community and school engagement. Leadership behavior included three items (α=0.73) adapted from Zimmerman and Zahniser (1991) that asked participants how often they were a leader in groups and organized people to achieve a goal (1 = Never; 5 = Always). Community and school engagement each included four items adapted from the U.S. Department of Education Community Engagement Scale (U.S. DOE, 2004) that asked about participating in neighborhood/school activities and helping people in need in their neighborhood/school (Community: α=0.86, School: α=0.77).

Youth Outcomes (Time 1 and Time 2)

Prosocial Outcomes

Prosocial behavior

We measured prosocial behavior using five 5-point Likert items adapted from Goodman (2001) asking participants how much they helped others or treated them well (α = 0.78).

Academic effort

We measured academic effort using six 4-point Likert items from the Social Skills Rating Scale (Gresham & Elliott, 1990). Items assessed study/work habits (α = 0.78).

Responsible decision making

We measured this variable using three 4-point Likert items from the Youth Asset Survey (Oman et al., 2002) addressing time management, treating others fairly, and making right choices (α = 72).

Antisocial Outcomes

Aggressive behavior

We measured aggressive behavior using ten items adapted from the California HealthyKids Survey (California Department of Education, 2004). Participants were asked how often in the past month they engaged in behaviors such as yelling at teachers, hitting or punching someone, and teasing another student (α = 0.88).

Delinquent behavior

We measured delinquent behavior using eight 5-point Likert items adapted from the Child Behavior Checklist (Achenbach & Rescorla, 2001). Participants were asked how often they engaged in delinquent behavior including breaking school rules and stealing things from stores or other people (α = 0.82).

Sociodemographic variables

Sociodemographic variables included self-reported sex, age and race/ethnicity. We calculated age from the reported month and year of birth. Racial/ethnic categories included Black, White, American Indian, Asian and Multiracial participants, but most individuals reported being Black or White. Therefore, we created a Black/non-Black dichotomous variable for race/ethnicity.

Data Analytic Strategy

Our structural equation modeling (SEM) approach included both direct (YES program on youth outcomes) and indirect effects (YES program effects on youth outcomes through empowerment) to test our mediation hypothesis. We used both latent factors and observed variables with Mplus Version 7.4 (Muthén and Muthén, 2015) using the weighted least squares means variance estimator (WLSMV). WLSMV allows missingness to be a function of observed covariates, but not observed outcomes as in ML estimation (Asparouhov & Muthén, 2010; Wang & Wang, 2012). We conducted tests of selective attrition to explore if those lost to follow up differed from those retained. We clustered by school/cohort/experimental group in all analyses to avoid potential biases in parameter estimates and standard errors from disaggregation (Pornprasertmanit, Lee, & Preacher, 2014). Following Kline (2014), we specified the model for the YES intervention and the parameters based on empowerment theory and used confirmatory factor analysis to examine the latent factors then examined the structural regression model to investigate relationships between YES, PE, and antisocial or prosocial outcomes. We examined direct and indirect effects of YES activities on outcomes to investigate alternative mechanisms of influence (Kline, 2014). We evaluated model fit indices using χ², Comparative Fit Index (CFI) values and Standardized Root Mean-Square Error of Approximation (RMSEA) with the associated 90% confidence interval. We used the “difftest” command in MPlus to obtain the correct chi-square difference test results when using WLSMV comparing nested models (Muthén & Muthén, 2015). In both measurement and structural models, we adjusted the model specification based on modification indices if they were consistent with our hypotheses and theoretical framework.

Results

Descriptive Statistics

Descriptive statistics for measurement model indicators and outcome variables are presented in Table 2.

Missing data

In addition to those lost to follow up between the two waves of data collection (n=27), we had 10 missing cases on prosocial outcomes and 11 on antisocial outcomes. We found no age, sex or race/ethnicity differences between students retained in the study and those lost to follow-up post-intervention or with missing data on positive and negative outcomes. We also found no differences in Time 1 (T1) variables between students with complete data and those lost to follow up.

Fidelity

Table 3 includes means and standard deviations for the four fidelity variables at the school/cohort level indicating high fidelity and limited variation across all four variables and observations.

Table 3.

Descriptive statistics for fidelity ratings

Fidelity ratings	mean	SD
Shared control	4.24	0.52
Youth opportunities	4.16	0.88
Respectful language	4.9	0.15
Respectful tone	4.88	0.2

Note: All 30 intervention schools and cohort implementations were observed at least three times.

Measurement Model

Measurement model results for PE are presented in Table 4 and 5. Following Geiser et al.’s (2013) approach, our measurement model alone with the 9 indicators for PE and 5 dichotomous indicators for YES dose received was a good fit with the data (X²:131.75, df: 76; RMSEA: 0.046, 90% CI:.033, .060; CFI: .95).

Table 4.

Prosocial outcomes model results.

	Unstandardized		Standardized
	Estimate	SE	Estimate	SE
Measurement model
YES activities -> A1 (photovoice)	1.00		0.50**	0.08
YES activities -> A2 (neighborhood advocate)	1.62	0.27	0.81**	0.04
YES activities -> A3 (cultural traditions)	1.60	0.24	0.80**	0.05
YES activities -> A4 (summer project)	1.75	0.31	0.88**	0.04
YES activities -> A5 (community assets)	1.74	0.27	0.87**	0.04
PE->I1 (leadership efficacy)	1.00		0.58**	0.04
PE->I2 (civic efficacy)	0.93	0.10	0.63**	0.03
PE->I3 (self esteem)	0.92	0.09	0.52**	0.04
PE->IN1 (mentor relationships)	0.55	0.19	0.25*	0.08
PE->IN2 (adult as resources)	1.43	0.22	0.47**	0.05
PE->IN3 (resource mobilization)	0.87	0.12	0.64**	0.04
PE->B1 (leadership behavior)	1.19	0.12	0.50**	0.03
PE->B2 (community engagement)	1.71	0.18	0.65**	0.03
PE->B3 (school engagement)	1.33	0.13	0.79**	0.02
Structural model✦
YES activities->PE	0.22	0.07	0.29**	0.05
Prosocial behavior T1->PE	0.12	0.04	0.21**	0.06
Academic effort T1->PE	0.19	0.04	0.28**	0.06
Responsible decision making T1->PE	0.15	0.03	0.27*	0.05
PE->Prosocial behavior (T2)	0.82	0.10	0.55**	0.04
PE->Academic effort (T2)	0.49	0.06	0.36**	0.04
PE->Responsible decision making (T2)	0.85	0.09	0.57**	0.04
Indirect effects
YES activities->PE->Prosocial behavior	0.18	0.053	0.16**	0.03
YES activities->PE->Academic effort	0.11	0.028	0.11**	0.02
YES activities->PE->Responsible decision making	0.19	0.049	0.17**	0.03

^*p<.05, p<.001

^✦We controlled for sex (male reference group), age, and race/ethnicity (African American reference group): African American participants and males were less likely to engage in prosocial behavior, all other sociodemographics were not associated with the outcomes. PE=psychological empowerment.

Table 5.

Antisocial outcomes model results

	Unstandardized		Standardized
	Estimate	SE	Estimate	SE
Measurement model
YES activities -> A1 (photovoice)	1.00		0.49**	0.09
YES activities -> A2 (neighborhood advocate)	1.60	0.32	0.79**	0.04
YES activities -> A3 (cultural traditions)	1.64	0.29	0.81**	0.05
YES activities -> A4 (summer project)	1.78	0.35	0.88**	0.04
YES activities -> A5 (community assets)	1.79	0.33	0.88**	0.03
PE->I1 (leadership efficacy)	1.00		0.57**	0.03
PE->I2 (civic efficacy)	0.95	0.08	0.62**	0.03
PE->I3 (self esteem)	0.87	0.10	0.48**	0.04
PE->IN1 (mentor relationships)	0.48	0.15	0.21**	0.06
PE->IN2 (adult as resources)	1.54	0.21	0.48**	0.05
PE->IN3 (resource mobilization)	0.93	0.10	0.66**	0.03
PE->B1 (leadership behavior)	1.33	0.13	0.54**	0.03
PE->B2 (community engagement)	1.82	0.17	0.59**	0.03
PE->B3 (school engagement)	1.28	0.10	0.76**	0.02
Structural model✦
YES activities->PE	0.30	0.09	0.41**	0.06
PE->Aggressive behavior (T2)	−0.18	0.10	−0.09✧	0.05
PE->Delinquent behavior (T2)	−0.25	0.06	−0.12**	0.03
Indirect effects
YES activities->PE->Aggressive behavior	−0.05	0.03	−0.04✧	0.02
YES activities->PE->Delinquent behavior	−0.07	0.02	−0.05*	0.01

^*p<.05, p<.001

^✦We controlled for sex (male reference group), age, and race/ethnicity (African American reference group): African American students were more likely to engage in aggressive and delinquent behavior, other sociodemographics were not associated with the outcomes. T1 outcomes were associated with T2.

^✧p<.10,

^*p<.05,

^**p<.001.

PE=psychological empowerment.

Structural Model with Dose Received

Prosocial Outcomes

We examined relationships between PE and prosocial outcomes (prosocial behavior, academic effort, responsible decision making). Results are reported in Figure 1 and Table 4. We controlled for sex, race/ethnicity, age, and T1 outcomes. We investigated both direct and indirect effects of YES activities on outcomes. Results indicated that direct paths from activities to outcomes were not significant and the model better fit the data without the direct paths; we tested the paths individually (results not shown) and collectively (X²_D=5.64, df_D=3, p<0.13). As these results were consistent with our a priori hypothesis regarding the mechanism of YES, we trimmed the direct paths. Modification indices suggested adding paths from T1 outcomes to PE which we included in the model because it seems plausible that success in school and prosocial involvement might naturally enhance students’ PE. The model fit significantly improved with the addition of these paths (X²_D=58.00, df_D=3, p<0.001). Table 4 reports the parameter estimates of this adjusted model. Consequently, our final model included paths from T1 outcomes to PE and only indirect effects. Estimates for the indirect effects indicate that PE does mediate the relationship between YES activities and prosocial behavior, academic effort and responsible decision making at T2; YES activities were associated with more PE, which, in turn, were associated with higher levels of prosocial behaviors. Sex, age and race/ethnicity were not associated with academic effort or responsible decision making at T2. We found no differences by age, but results indicate that African American youth and males were less likely to engage in prosocial behaviors at T2.

Figure 1.

Model of YES activities, psychological empowerment and prosocial outcomes. A1-A5: the 5 YES activities. I1-I3: Intrapersonal component, IN1-IN3: Interactional component, B1-B3: Behavioral component. Correlated errors between leadership efficacy and behavior and civic efficacy and resource mobilization indicators on PE latent factor. Not shown: controlled for sex, age, race/ethnicity and T1 outcomes on T2 outcomes and T1 outcomes on PE. Model fit indices. X²=278.39**, df: 202, CFI=0.93, RMSEA (90% CI)=0.034(0.023, 0.043), n=331. *p<.05, **p<.001

Antisocial Outcomes

We tested the antisocial outcomes model (see Figure 2 and Table 5). We controlled for demographics and outcomes at T1 on T2. We investigated both direct and indirect effects of YES activities on outcomes. Our results indicated no direct paths from activities to outcomes and the model better fit the data without the direct paths (X²_D=1.74, df_D=2, p<0.42). Consequently, our final model included only indirect effects. Estimates for indirect effects indicate that PE mediated the relationship between YES activities and delinquent behavior and aggressive behavior at T2 (one tailed tests). YES activities were associated with more PE, which, in turn, was associated with lower levels of delinquent behaviors. We found that sex and age were not associated with aggressive or delinquent behavior at T2. African American youth were more likely to report aggressive and delinquent behavior at T2.

Figure 2.

Model of YES activities, psychological empowerment and antisocial outcomes. A1-A5: the 5 YES activities. I1-I3: Intrapersonal component, IN1-IN3: Interactional component, B1-B3: Behavioral component. Correlated errors between leadership efficacy and behavior and civic efficacy and resource mobilization indicators on PE latent factor. Not shown: controlled for sex, age, race/ethnicity and T1 outcomes on T2 outcomes. Model fit indices. X²=225.80**, df: 174, CFI=0.95, RMSEA (90% CI)=0.030(0.017, 0.041), n=329. ^☨p<.05 one tailed test, *p<.05, **p<.001

Dose Delivered

Although researchers suggest that dose received is more predictive of outcomes, we also investigated the relationship between dose delivered (attendance) on PE and outcomes. We found that dose delivered did not predict PE or behavioral outcomes. We also conducted an intention-to-treat analysis by comparing youth initially assigned to the program with youth assigned to the control condition, but this comparison also indicated no intervention effects.

Discussion

Consistent with empowerment theory, the results of this study indicate that empowering processes enhance psychological empowerment outcomes (Zimmerman, 1995), which in turn has positive effects on youth developmental outcomes. This study builds on the small body of work on youth empowerment programs (e.g., YPAR approaches) in three key ways. First, we apply explicitly all three components of empowerment theory in both the content of the curriculum and in our analysis of the mediating effects of the program on youth outcomes. Second, we examined the mediating effects of empowerment outcomes on developmental outcomes. Third, our evaluation included a quasi-experimental design to test program effectiveness. The results supported our hypothesis that participation in YES components improved the middle school youths’ psychological empowerment, which in turn was associated with more prosocial and less antisocial behavior. Participating in YES did not have a direct effect on developmental outcomes as the indirect-effects-only-model fit the data better than a model that included both direct and indirect effects on prosocial and antisocial outcomes. The results are strengthened by the fact that the effects of program participation on empowered outcomes were found after controlling for Time 1 prosocial outcomes, we used multivariate statistics to test our hypotheses, and we conducted the evaluation over several years with different cohorts of youth, and in various community settings. These unique contributions of YES also support its wider applicability to the field of positive youth development.

Our confidence in the study results is strengthened by several key factors. First, structural equation modeling (SEM) provides a comprehensive method for quantifying and testing models based on substantive theory (Raykov & Marcoulides, 2006). The fact that the model fit better without direct effects for the program provides strong support for our hypothesized mediation model for empowering processes. The results are further strengthened because we controlled for Time 1 outcomes and demographic factors, and the results replicated for both prosocial and antisocial behavior. In addition, our analytic approach took into account the nested nature of the data within both school and cohort.

Second, we used self-reported participation in five different program components to define dose received. This was a useful indicator of program dosage because it considered what youth remembered about the program instead of how often they attended (i.e., dose delivered) (Steckler & Linnan, 2002). Researchers have challenged clear, causal linkages between attendance and program effects because exposure does not necessarily indicate participant engagement (Lauer et al., 2006; Lower, 2015; Roth, Malone, & Brooks-Gunn, 2010). Baydar, Reid, & Webster-Stratton (2003) reported that participant engagement was associated with program outcomes, but attendance was unrelated to program outcomes. Roth et al (2010) also found little predictive value of dosage delivered on program effects in their meta-analysis of after-school programs. It is also useful to note that our latent construct approach to assessing engagement provided an opportunity to evaluate the relative contribution of different program components and validate the notion that it was the common connections among the program activities rather than any one part alone.

Third, the external validity of the study was strong because it was implemented in the school context by teachers hired by the schools. The research team provided training for implementation that included the theory behind the program, the importance of letting youth have control over decisions, and practice sessions for implementing the curriculum. It is also notable that our fidelity measures indicated YES was implemented as intended.

These strengths notwithstanding, we also note three key methodological limitations of this evaluation. First, because the study recruitment and program implementation needed to adhere to local after-school program requirements, we had to modify the original randomized design in some schools when we did not have enough consented participants to randomly assign to the YES program and the usual after-school program control condition. Secondly, and a further deviation from our original design, we learned during our fidelity assessments that some teachers allowed youth in the control condition to attend some YES sessions for a variety of unintended reasons (e.g., lost track of who was in which condition). Although teachers received training in the experimental design, some control group members attended YES sessions and were exposed to YES activities. To address both of these limitations, we employed several strategies to improve our confidence in the internal validity of our study including considering within school and cohort variation, focusing on program dose received versus dose delivered, accounting for missing data in our analytic approach (even though it was minimal), and controlling for Time 1 outcomes and demographic variables. We also utilized SEM to create a multivariate measure of dose received and to test our mediation model applied to two sets of developmental outcomes. Although testing program effects in a real-world context with actual teachers (i.e., not project staff) reduces control of implementation and raises concerns for internal validity, the external validity of the study is improved substantially making it more useful to practitioners (Green & Glasgow, 2006; Onken, et al., 2013).

Third, our self-report measure of dose received is a limited assessment of program engagement. Even so, our limited measure of YES engagement did indicate that the program effectively enhanced youth empowerment outcomes that were related to positive developmental behaviors as hypothesized. Future research that collects detailed information about qualities of participant engagement such as how much they liked the activity, what they learned from the activity, and how they have applied what they learned will provide a more in-depth analysis of program engagement. Fourth, the range of time to posttest may raise some questions about comparability of youth with different posttest times. Our analysis of this issue indicated, however, that over three quarters of the youth completed posttests within 30 days of program completion and the delays in completing the posttest were similar for the YES program and control program participants, suggesting this issue does not explain the results.

Finally, our models did not explain much of the variation in the anti-social outcomes. The mediation effect of empowerment outcomes between YES activities and aggressive behavior, for example, was only significant for a one-tailed test, although a one-tailed test is appropriate given our directional hypotheses. The small effects on anti-social behavior, however, are not surprising. As Catalano et al. (2002) noted, it may be that programs focused on enhancing positive factors (like YES) are less likely to influence problem behavior because they do not address effects of risk exposure correlated with negative outcomes. Despite these small effects, we did find that empowered outcomes have an effect on anti-social behavior which suggests that enhancing positive factors in youths’ lives may have a salutary effect in reducing negative behaviors.

Implications for Practice

YES provides practitioners with an evidenced-based, theoretically-driven curriculum that engages youth in positive community change in an effort to promote healthy development. Although we described the evaluation results of one specific program, the YES curriculum has broad applicability to positive youth program development because it applies empowerment theory and integrates elements of other evaluated youth programs by building confidence, enhancing cognitive skills, and engaging youth in social change (Eccles & Gootman, 2002; Gardner et al., 2008; Mueller, et al., 2011). It was also tested across different ecological contexts (e.g., urban, suburban) with diverse cohorts of youth. The structured curriculum prepares youth to assume leadership roles, think critically about their community, and engage in community change in partnership with supportive adults to contribute to healthy developmental outcomes. The findings of this study are useful because they were established in a similar context to those in which the program may be implemented and not controlled exclusively by researchers. Finally, the fact that YES operates to enhance participants’ psychological empowerment, suggests that it can be applied to a variety of challenges facing adolescents. The program components may also be adapted for different age groups, communities and cultures.