Building Resilience After School for Early Adolescents in Urban Poverty: Open Trial of Leaders @ Play

Abstract

Leaders @ Play is a park after-school program for urban middle school youth designed to leverage recreational activities for social emotional learning. Mental health and park staff co-facilitated sports and games to teach and practice problem solving, emotion regulation, and effective communication. Additional practice occurred during multi-family groups and summer internships as junior camp counselors. We examined feasibility and promise via an open trial (n=3 parks, 46 youth, 100% African American, 100% low-income, 59% female, M=13.09 years old). Improvements in social skills and reductions in problem behaviors lend support to after school programs as a space for mental health promotion.

Early adolescents (ages 12 to 14) are developmentally ready for more autonomy and responsibility; hence, parental guidance, supervision, and authority wanes as peers assume more influence (Steinberg, 1990, 1999), presenting increased opportunities and pressure to experiment with risky behaviors. Frequent and severe risk-taking in turn may initiate a negative life trajectory with significant and prolonged consequences related to education, employment, relationships, and earnings in young adulthood (Caspi, Wright, Moffitt, & Silva, 1998). Correlates of economic disadvantage exacerbate risks, resulting in disproportionately poor outcomes for ethnic minority youth who are over-represented in poverty (Cappella, Frazier, Atkins, Schoenwald, & Glisson, 2008). This paper presents Leaders @ Play, a park after school program designed to strengthen resilience among urban, poor, minority middle school youth.

Risky Behaviors Increase during Early Adolescence

Fifty percent of adult mental disorders are diagnosed by age 14 (O’Connell et al., 2009), thus early adolescence is a critical period for building resilience. Literatures on smoking, alcohol and substance use, and sexual health illustrate the pathway from early initiation to persistent problems. Cigarette smoking begins, on average, at 12.6 years for African American youth and 11.6 years for White youth (CDC, 2012); early onset leads to more tobacco dependence and difficulty quitting as an adult (e.g., Khuder, Dayal, & Mutgi, 1999). Sexual activity begins by age 16 for 40% of youth (Cavazos-Rehg et al., 2009), and early sexual debut places youth at higher risk for substance use (Cavazos-Rehg et al., 2011) and delinquency (Armour & Haynie, 2007). Substance use begins as young as age 12, with 4% of 12 to 13 year-olds and 9.3% of 14 to 15 year-olds using illicit drugs (SAMHSA, 2010). Early experimentation with alcohol and drugs predicts poor academic performance, association with deviant peers and more stressful life events (Windle, Mun, & Windle, 2005) and longitudinally predicts poor outcomes in young adulthood (Flory, Lynam, Milich, Leukefeld, & Clayton, 2004). Together, these data illustrate the multiplicity and longitudinal negative trajectory associated with early risk-taking.

Poverty Exacerbates Risks and Poor Outcomes

Poverty exacerbates risks and poor outcomes for urban minority youth through multiple pathways including homes, schools, and neighborhoods (Cappella et al., 2008). Children in poverty are more likely to be African American (35%) and Latino (28%) than Caucasian (10%) and are more likely to live in single parent, female-headed households (43%) than two-parent families (9%) (http://www.childstats.gov/). Unresponsive and harsher, more punitive parenting, accompanied by low monitoring and high mobility, occurs more often among low-income families (Grant et al., 2003), beginning as early as infancy (e.g., McLoyd, 1998) and worsening as time spent in poverty grows (Miller & Davis, 1997).

Public schools in low-income communities are underfunded; deficient in environmental quality (e.g., heating), space (e.g., overcrowding), and educational materials (e.g., books) (Evans, 2004); and staffed by less experienced and more overwhelmed teachers with lower expectations for student learning (Weinstein, 2002). Urban elementary schools are characterized by less effective instruction (Pianta, Belsky, Houts, & Morrison, 2007), high teacher turnover, and frequent student mobility (Ingersoll, 2001), contributing to students’ experience of educational instability and discontinuity over time, and leading to poor academic outcomes.

Youth exposed to high rates of community violence exhibit school (e.g., poor attendance, low grades; Margolin & Gordis, 2004), conduct (e.g., aggression; Gorman-Smith, Henry, & Tolan, 2004), and internalizing (e.g., traumatic stress, depression; Fitzpatrick, Wright, Piko, & LaGory, 2005) problems, in particular during early adolescence and among urban low-income and ethnic minority youth (Gorman-Smith & Tolan, 1998). Community disorganization (i.e., crime, underemployment, substance abuse, physical decay) also predicts multiple negative youth outcomes, including aggression (Evans & Kantrowitz, 2002), anxiety and depression (Cooley-Quille, Boyd, Frantz, & Walsh, 2001), and poor academic performance (Schwartz, Gorman, Nakamoto, & Toblin, 2005), reflecting a robust body of literature that suggests a common set of heightened and prolonged risk pathways lead to a multiplicity of poor outcomes.

Promoting Resilience through Positive Youth Development

A rich literature on risk and resilience highlights contributions of neighborhood, family, and peer influences in shaping youth adjustment (Leventhal & Brooks-Gunn, 2000; Rankin & Quane, 2002). Fergus and Zimmerman (2005) review models for resilience that leverage these ecologies to alter trajectories from risk exposure to negative outcomes and help explain why some youth overcome risks via assets (i.e., individual coping skills or competencies) or resources (i.e., parents, social networks) (Beauvais & Oetting, 2002). Accordingly, youth development programs are designed to build competencies and facilitate mastery experiences across social, emotional, and behavioral domains (Boustani et al., 2014; Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2002) through individual and system-level change (Durlak & Weissberg, 2007). These competencies correlate with academic success (Zins, Bloodworth, Weissberg, & Walberg, 2007) and with educational and employment outcomes in young adulthood (Jimerson, 1999).

Leveraging Out of School Time for Urban Youth at Risk

After school time (2:00 to 6:00) reflects a substantial 20 hours per week (compared to an average 30 hours per week in school) that holds great opportunity for social emotional learning (SEL), especially among ethnic and racial minority youth in high poverty urban communities, for whom high quality, organized programs contribute most to social development (e.g., Roffman, Pagano, & Hirsch, 2001; Tebes et al., 2007). Past effort to integrate SEL into classrooms has been displaced by competing demands associated with standardized test scores. After school time offers an under-utilized and critical opportunity for building resilience, and the current work reflects a program of research focused on strengthening after school program quality through social context intervention and workforce development; integrating SEL more explicitly into recreation activities via teachable moments, family involvement, and staff consultation; and leveraging, while not over-extending, indigenous staff and resources.

Indeed, programs delivered in urban after school settings and designed to mitigate risk by promoting social and emotional competencies reveal positive impacts on youth, (Botvin & Kantor, 2000; Jain, Buka, Subramanian, & Molnar, 2010) including less experimentation with alcohol, marijuana and other drugs (Tebes et al., 2007). A growing literature cautions, though, that some program features can lead to poor outcomes in particular for adolescents. Gottfredson, Cross, and Soule (2007) examined associations between program structure and self-reported adolescent delinquency and victimization in 35 after school programs. Unsupervised socializing predicted victimization, supporting deviancy training (Dishion, McCord, & Poulin, 1999) as one mechanism by which risky behaviors may increase; in contrast, use of a published curriculum predicted lower reported substance use, supporting Gottfredson and colleagues’ earlier findings that programs emphasizing character development and social skills were associated with less delinquent behavior compared to programs without specific social development goals and suggest that youth benefit from structured, curriculum-based after school activities.

Building Resilience through Leaders @ Play

Leaders @ Play extends a longstanding partnership between the investigators and park district colleagues focused on strengthening program capacity to support skills acquisition, healthy relationships, and family involvement for enrolled youth. Leaders @ Play was a direct response to requests from park supervisors in communities of concentrated urban poverty to build a program for their early adolescents, who they perceived to be at highest risk for academic failure, risky behaviors, and gang recruitment. This request came while a national fiscal crisis and neighborhood gentrification were contributing to increased gang violence, dissolving social networks, and increasingly struggling schools, which together compromised the safe and healthy development of their youth. Hence, we developed a park program for middle school youth that leveraged the strengths and capacities of the park setting and its staff, integrated evidence-based tools for building social competencies into recreational activities, and fostered resilience among youth and families. In this paper, we describe the programmatic components of Leaders @ Play, and examine its feasibility and promise.

Method

Setting

This research reflects a decade-long partnership with the collaborating park district. Two of three parks had participated in previous studies, and the third was directed by a supervisor who also had participated previously (at a different park), but persistent needs minimized concerns about contamination. District managers identified parks for participation based on their high-risk status (i.e., location in neighborhoods of severe economic disadvantage, characterized by high rates of violence and risk exposure) and leadership by park supervisors enthusiastic to offer middle school programming and collaborate with our university research team.

Park #1

Park 1 is located in an urban African American community characterized by high rates of poverty, unemployment, and crime. It receives frequent national attention for youth victims of gun violence, including one of our junior counselors after the study had ended. It is one of the city’s top five most violent neighborhoods and on the list of poorest places to live (U.S. Census Bureau, 2010). Police statistics reveal a high rate of illicit drug activity (56%) and gang activity (11%). Park enrollment averages 45 youth after school and 35 during summer camp. This park is relatively small (6.18 acres), with few staff (one full-time physical instructor and one part-time recreation leader), few programs (dance and music), and minimal resources. Staff mobility was low; thus, staff was familiar and close with youth and families. The park is located across the street from the neighborhood school, easing concerns about safety getting from one to the other. Nevertheless, given the high rates of neighborhood crime and violence, a full-time city police officer was on duty after school until park closing.

Park #2

Park 2 is located in an African American community of severe economic disadvantage, with significant gang activity and chronic violence. This relatively large park (15.25 acres) maintains many active after school, weekend, and summer activities for children through seniors. Despite an expansive and well-maintained outdoor environment, the park supervisor was vigilant for gang activity on or near park property, and kids were often kept indoors. A gang-related park shooting during the study led to ongoing dialogue with park staff, youth, and families regarding the oppressive violence in their community, including systematic additions to Leaders @ Play focused on safety planning, conflict resolution, response to trauma, and violence prevention. Police statistics revealed a high level of illicit drug activity (34%), criminal and violent activity (40%), and gang activity (14%). The park enrolls on average 145 youth for after school programs (including youth in organized sports) and 40 youth for summer camp. There were 3 full-time physical instructors and 6 part-time recreation leaders, but the park had fewer overall resources compared to parks of similar size and enrollment. Low staff mobility contributed to positive relationships with youth and families. The park was located equidistant between two elementary schools, each approximately three blocks away, and many students walked from school or home to participate in programs. This presented a unique set of challenges and reduced attendance during weeks of heightened violence that predictably emerged in early spring when weather improved. A full-time city police officer was assigned to the park.

Park #3

Park 3 is located in a predominantly urban, African American, low-income community characterized by severe unemployment and chronic violence. Compared to Parks 1 and 2, police statistics revealed lower rates of crime and violence (29%) and gang activity (4%), but this community continued to rank within the top 20 (of 77) most violent neighborhoods in the city. The park enrolls on average 85 youth for after school programs and 100 youth for summer camp. This park is similar in size (15.64 acres) to Park 2, features a large field house (with indoor pool and two gymnasiums), natural savanna, several athletic fields, and playground. The park was located between two elementary schools, one adjacent to the park and another four blocks away, with a safe route between. In comparison to other parks in the same region of the city, Park 3 maintained higher youth enrollment and more resources, programs, and staff, including 6 full-time physical instructors and 6 part-time recreation leaders (100% African American). Prior to the study, this park received a new supervisor and one female instructor; during the study, one additional female instructor was hired to assist with Leaders @ Play.

Participants

Park staff (n = 7) included three park supervisors (Bachelor’s degrees), one full-time physical instructor (Bachelor’s degree) and three part-time recreation leaders (1 high school graduate, 1 with 3-years of college, and 1 Bachelor’s degree).. Staff reported several years of experience working with urban youth in community settings and working for the park district (Range: 6 months to 23 years, M = 14.9 years, SD = 8.80 years). All park supervisors endorsed the program and supported their staff’s involvement; only the supervisor at Park 2 actively participated in program implementation and study procedures and is therefore the only one reflected in staff-level data.

Middle school youth (n = 46, M = 13.09 years old, SD = 0.97; 59% female; 100% African American), their parents (n = 36; 10 families enrolled more than one youth; 66% mothers), and park staff (n = 5; 100% African American, 80% female) participated. Park 1 included 17 youth (M = 12.94 years old, SD = 1.00; 41% female), 11 parents, and 1 park staff. Park 2 included 15 youth (M = 12.80 years old, SD = 1.15; 40% female), 11 parents, and 2 park staff. Park 3 included 14 youth (M = 13.38 years old, SD = 0.51; 79% female), 13 parents, and 2 park staff.

Fifty-eight percent of families reported single-parent households, and participants were mothers (69%), grandmothers (11%), fathers (6%), or other adults (e.g., aunts 3%, 11% did not specify). Approximately 1 of 4 children had a grandparent living at home. Of 32 families that reported income data, n = 7 (22%) earned below $5,000, n = 12 (37%) earned between $5,000-19,999, n = 7 (22%) earned between $20,000-29,999, and n = 6 (19%) earned more than $30,000. A majority of youth (78%) received federal reduced-price or free lunch. Thirty-three families provided employment data: 39% were employed full-time, 19% part-time, and 33% were unemployed (among those, 25% had a full-time employed spouse or partner in the home). Thirty-three parents reported educational attainment: n = 7 had not completed high school, n = 8 received a high school diploma or GED, n = 11 attended some college, and n = 7 graduated from a 2-year college. Comparable education data were reported for the spouse or partner.

Measures

Baseline mental health need

Strengths and difficulties questionnaire (SDQ, Age 11-17 Form; Goodman, 2001)

Parents reported children’s mental health symptoms (25 items, 0 = Not True, 1 = Somewhat True, 2 = Certainly True). The SDQ provides a Total Difficulties score (0 to 40) and five clinical subscale scores (0 to 10): hyperactivity/inattention, emotional symptoms, conduct problems, peer problems, and prosocial behavior. Impairment is rated (0 = not at all to 2 = A great deal) across four domains: Home Life, Friendships, Classroom Learning, and Leisure Activities. Internal consistency for Total Difficulties was α = .84.

Self-report behavior index (SRBI; Brown, Clasen, & Eicher, 1986)

Youth reported their past month’s participation (never, once or twice, 3 or 4 times, pretty often, or almost every day) for 28 behaviors along two primary subscales: misconduct (15 items, α = 0.92, e.g., smoked a cigarette, got drunk, took something that didn’t belong to them) and peer involvement (10 items, α = 0.74, e.g., gone to a party, seen a movie with friends). Additional questions pertain to family involvement (5 items, α = 0.60, e.g., asked parents for advice) and school involvment (6 items, α = 0.10, e.g., done all homework, did well on a test or assignment).

Outcomes

Social skills improvement system (SSIS; Gresham & Elliott, 2008)

Parents and staff reported youth social skills (46 items) and problem behaviors (33 parent items, 30 staff items) along a 4-point scale (never to always). The SSIS has high internal consistency, test-retest and inter-rater reliability, and external validity with established measures of social, emotional and behavioral functioning (e.g., BASC-2). Our sample yielded high baseline reliability for both parent and staff report of Social Skills (α = 0.96 and 0.98, respectively) and Problem Behaviors (α = 0.94 and 0.97, respectively). Parent and staff report were not significantly correlated for Social Skills (r = 0.37) or Problem Behaviors (r = 0.34).

Procedure

This study was conducted in accordance with APA ethical guidelines and with approval from the university IRB for recruitment, informed consent, and data collection procedures.

Recruitment and data collection

Researchers attended pre-arranged park meetings to introduce the study, followed by individual meetings with park staff to obtain consent. One hundred percent of eligible staff (5 of 5) participated. Consented staff distributed recruitment flyers to eligible (6^th to 8^th grade) youth and families, including past or current participants in park programs. Researchers described the study and enrolled families during Family Nights at the parks (dinner and childcare provided). Parents and youth provided independent written informed consent and assent. Measures were administered at the parks at baseline, end of school (post-test), and end of summer camp (2 months follow-up). Parents received $20 and youth received a $10 gift card at each time point. Following park district guidelines, we provided lunch during data collection in lieu of individual staff compensation.

Intervention development and implementation

Leaders @ Play responded to requests from park supervisors for programs to support middle school youth, who were developmentally aging out of Kids @ Play after school activities for elementary students, but who were too young still for teen clubs. These concerns were reflected in our own city-wide data (n = 44 parks, 728 youth) that revealed after school K to 8 park programs were attended predominantly by elementary school children (M = 8.96 years old, SD = 2.17). Based on our team’s growing effort to leverage life skills opportunities inherent to recreation, we planned a curriculum that prioritized common elements of empirically-supported adolescent prevention programs (Boustani et al., 2014) - social problem solving, emotion regulation, and effective communication - which mirrored job skills identified by park staff and supervisors as critical for junior-counselors-in-training.

Content and materials were drawn from evidence-based mental health curriculum (e.g., Summer Treatment Program Manual; Pelham, Greiner, & Gnagy, 1997), prevention programs (e.g., Responding in Peaceful and Positive Ways; Meyer & Northup, 2002), recreation manuals (e.g., SPARK After School Physical Activity Program), and the park district’s Culture, Arts, and Nature Guides. Several meetings of the research team, project-funded mental health providers, and park supervisor and staff preceded implementation at each park. Meetings emphasized the underlying program rationale, goals, and objectives; integration of core skills with recreation (to ensure activities were both engaging and explicit with regard to skills building); and a vision for park and mental health staff to co-facilitate youth and family groups (with an eye toward sustainability). Shared decision-making related to selection of recreation activities, timeline and procedure for staff training and support, incentives for youth and families, and the role, responsibilities, rewards, and work schedule for junior-counselors-in-training was expected to increase feasibility. Intervention included the after school youth program (Leaders @ Play), multi-family groups (Families @ Play), and junior counselor-in-training summer camp internships (Junior Camp Counselors).

Leaders @ Play

The after school youth program was designed to meet for 10 weeks (corresponding to seasonal park programs), twice weekly at the parks for 90-minutes, co-facilitated by park staff and mental health providers. The program included didactic instruction, skills demonstration and discussion, role plays, and sports and recreation to provide practice with feedback. The first two sessions included team building activities; introduction to the Good Behavior Game (Barrish, Saunders, & Wolf, 1969; Embry, 2002) for maintaining engagement and minimizing disruptions; and orientation to the junior camp counselor internship. Staff reports on the SSIS were used to assign youth to same-gender pairs; youth demonstrating high social competence were paired with youth demonstrating difficulties, reflecting a peer-assisted learning model (Rohrbeck, Ginsburg-Block, Fantuzzo, & Miller, 2003) to minimize the risk of deviancy training and provide students at greater risk for problem behaviors systematic opportunities for interaction with positive peers. Intervention content emphasized social problem solving, emotion regulation, and effective communication (Table 1). The last two sessions included review, celebration, and preparation for summer camp. A typical agenda included: (a) welcome and review, (b) relaxation, (c) recreation activity (with integrated skills building), (d) didactic instruction, (e) recreation activity (with integrated skills building), and (f) wrap up.

Table 1.

Leveraging recreation to practice SEL skills.

Targeted Skill	Description	Sample Intervention Activities
Problem Solving	Define the problem Generate potential solutions Evaluate the feasibility and likely outcome for each alternative Select, implement, and evaluate success	Crash Landing (from RiPP Curriculum): An aircraft crashed on a tiny island without clean water. The crew survived, but two lost their sight and two lost use of their arms. Remaining is a radio transmitter, weather machine, pieces of the aircraft, and tape. The weather machine shows it will rain in 4 minutes and the crew needs to create cups to catch rain in order to survive until the rescue team arrives. Students were encouraged to work as a team and follow the problem solving sequence.
Emotion Regulation	Affect identification Relationship between feelings and physiology Relaxation Cognitive restructuring exercises	Basketball: Referee (designated confederate) makes strategic bad calls to generate frustration. Students were encouraged to identify and appropriately express their feelings, practice skills for tolerating frustration and remaining calm, and manage distress without initiating conflict.
Effective Communication	Introduction to verbal (e.g., active and reflective listening) and non-verbal communication Strategies for avoiding misunderstandings Discussing the value of asking questions Observing and praising peer positive behaviors.	Telephone: Youth are seated in a circle. One person is selected to begin by whispering something to the student seated next to them. This person then whispers whatever s/he hears to the next person, and so forth, until everyone in the group has received the message. There is no opportunity for repetition or questions. The last person in the circle reveals aloud what s/he heard which is then compared to the original statement. Students were encouraged to consider the importance of speaking clearly and listening closely, while recognizing potential for unintentional misunderstanding

Families @ Play

Multi-family groups comprised of youth, parents, and extended family were designed to meet twice per month for 90 minutes, co-facilitated by park and mental health staff. The format and content of family groups mirrored those of Leaders @ Play. The primary goal was to introduce a targeted skill (problem-solving, emotion regulation, or effective communication), accompanied by specific strategies by which families could model and reinforce them at home. A typical agenda included: (a) dinner and family games, (b) review of group rules and homework, (c) recreational activity (with explicit skills focus) (d) multiple family group activity, and (e) wrap-up and assign homework. Youth co-facilitated recreational activites, thereby demonstrating their new skills and providing opportunities for parents also to practice and praise. Dinner and childcare were provided, and families that submitted completed homework assignments from the prior group were entered into a raffle for small prizes (e.g., school supplies) for added incentive.

Junior Camp Counselor (JCC)

Youth were invited to intern during their park’s eight week summer camp as junior-counselors-in-training if they attended a minimum number of groups and/or demonstrated competence in core skills, in accordance with criteria set forth by each park supervisor. Youth worked in pairs (maintaining original pairs whenever possible) during specified blocks of time during the six week summer camp offered to students in grades K to 5. They assisted camp counselors in a variety of ways, including taking attendance, accompanying campers to the restroom, and setting up for and assisting during activities. In addition to camp responsibilities, youth participated in weekly supervision meetings with the research team and park staff to review skills, role, and responsibilities; provide and receive feedback; and problem-solve as necessary. Peer tootling strategies (e.g., a peer saying something helpful or friendly to another peer; Fantuzzo, Rohrbeck, Hightower, & Work, 1991) were encouraged to enhance postive interactions among JCC (e.g., “Great job!”) and to reinforce targeted skills (e.g., “I liked the way you remained calm when helping that frustrated camper”). Feedback from park staff and JCCs led to several iterations of Tootle Cards to guide supervision. At the conclusion of summer camp, each park honored their JCC with a celebration attended by family and friends.

Staff credentials, training, fidelity, and supervision

Mental health providers included two Caucasian female social workers (MSWs), both recent graduates with training in evidence-based interventions and after school park programs obtained during their first-year graduate school field placements with our investigative team. They were trained and supervised by an African-American female clinical psychology postdoctoral fellow (second author) and the principal investigator (first author), both with extensive experience working with urban racial minority youth and families in poverty. Initial training included an introduction to the role of mental health promotion during after-school programming, Good Behavior Game, and curriculum. Thereafter, providers received the majority of training on-site via real-time support by the psotdoctoral fellow (modeling and co-facilitation decreased over time). In addition, mental health providers and park staff met weekly to merge social emotional learning content with sports and recreation activities to provide the most engaging and explicit opportunities for learning and practicing skills.

Ongoing training was available during weekly 2-hour supervision meetings with the principal investigator. Weekly progress notes contained information about group attendance; intervention activities; qualitative descriptions of youth engagement; and individual, targeted goals (e.g., improve attendance; minimize disruptive behavior). Progress notes were used to monitor feasibilty of intervention implementation and to guide weekly supervision, which was informed by the principles, goals, and format of Multi-Systemic Therapy (Schoenwald, Sheidow, & Chapman, 2009) that our team had previously adapted for a related study (Omitted for blind review). Accordingly, supervision was highly structured and prioritized youth and families with greatest needs. Time was allocated to reviewing intervention implementation, problem solving, action planning, provider skill building, and sustaining collaboration.

Data Analytic Plan

Data were analyzed using paired sample t-tests and Cohen’s d effect size to examine changes in SSIS scores (social skills and problem behaviors) from baseline to post-test, post-test to follow-up, and baseline to follow-up for both parent and staff report. Given the small sample design and lack of control group, we also calculated the reliable change index (RCI) (Jacobson & Truax, 1991) to examine clinically significant improvement for each participant, as has been done in previous pilot work (Bagner, Rodríguez, Blake, & Rosa-Olivares, 2013; Chu, Colognori, Weissman, & Bannon, 2009). An RCI ≥ 1.96 represents a reliable change at α = 0.05. RCI data were computed for youth (n = 38) with at least two time points of data (baseline to post-test or baseline to follow-up) and are summarized as the percentage of youth exhibiting improvement, deterioration, or no change.

Results

Mental Health Need

Parent report at baseline revealed 85% of youth had SDQ Total Difficulties scores within normal range (≤ 13), 5% in the borderline range (13-16), and 10% in the abnormal range (≥ 17). Clinical subscale scores were Conduct Problems: 68% normal range, 15% borderline, 17% abnormal; Hyperactivity/Inattention: 88% normal range, 7% borderline, 5% abnormal; Emotional Symptoms: 88% normal range, 7% borderline, 5% abnormal; Peer Problems: 69% normal range, 12% borderline, 19% abnormal; and Prosocial Behavior: 78% normal range, 17% borderline, 5% abnormal. Impairment was reported for 24% of youth (10 with minor difficulties, 1 with definite difficulties), with over half (n = 6) exhibiting impairment for more than one year. Impact ratings ranged from 2 to 12 (M = 4.73, SD = 3.23); scores ≥ 2 are considered in the abnormal range. The most commonly endorsed problems, in order, occurred in Home Life (n = 10), Classroom Learning (n = 8), Leisure Activities (n = 5), and Friendships (n = 4). Baseline mental health need was used to assign youth to risk categories: Low Risk (Normal on all subscales and Total Difficulties, n = 14); Moderate Risk (Elevated on at least one subscale or Total Difficulties, n = 5); and High Risk (Abnormal on at least one subscale, Total Difficulties, or Impact score, n = 19). Together, 54% of youth were classified as moderate or high risk.

SRBI data revealed the most commonly endorsed misconduct items to include Done something your parents told you not to (Park 1: 47.1%, Park 2: 15.4%, Park 3: 33.3%), Gone cruising (driving around) (35.3%, 30.8%, 33.3%), and Made out (kissing, petting) (41.2%, 21.4%, 16.6%). Park 1 youth reported the most high-risk behaviors (had sex all the way 23.5%; drank beer or liquor 17.6%; smoked marijuana 11.8%; used hard drugs 6.3%) and delinquency (64.7% stayed out past curfew; 35.3% stolen something from someone; 23.5% stolen something from store; 17.6% vandalized something). Across parks, youth also endorsed many positive items related to peer, family, and school involvement.

Intervention Feasibility

Table 2 summarizes service delivery and service use data. As planned, Leaders @ Play groups were offered twice weekly for 90 minutes after school at all three parks, and Families @ Play groups were offered semi-weekly for 90 minutes during evenings. Variability in attendance reflected competing priorities after school (e.g., tutoring, sports, and family obligations) and neighborhood violence (i.e., parental concerns about children’s safe arrival from school to park), and groups sometimes were cancelled for inclement weather and escalating violence nearby. Variability in park resources and staffing resulted in significant differences across parks in the extent to which staff were available to co-facilitate groups. For instance, at Park 1, the park supervisor was unable to assign either of her staff to Leaders @ Play due to competing programs with high enrollments. Therefore, two project-funded mental health staff facilitated. At Park 2, two instructors were assigned to Leaders @ Play, but similarly reported that competing responsibilities interfered with their full participation. Although mental health staff were primary facilitators, park staff co-facilitated 52% of after school groups and 85% of family groups. Park 3 was able to assign one full-time instructor and one part-time recreation leader, who together co-facilitated, with the mental health provider, all of the Leaders @ Play groups. Park staff was unavailable for evenings, however, and the mental health providers independently facilitated Families @ Play.

Table 2.

Service delivery and service use across sites.

				Leaders @ Play
Park	Total # Groups	Minimum # Youth	Maximum # Youth	Mean # (%) Youth	# Groups Co-Facilitated by Park Staff	# Youth Completed 75% Groups	# JCCs
1	15	1	11	6.0 (35)	0 of 15 (0%)	7 of 17 (41%)	6
2	23	3	9	6.0 (40)	12 of 23 (52%)	10 of 14 (70%)	10
3	15	3	11	7.0 (50)	15 of 15 (100%)	6 of 14 (43%)	6

				Families @ Play
Park	Total # Groups	Minimum # Families	Maximum # Families	Mean # (%) Families	# Groups Co-Facilitated by Park Staff
1	2	5	6	5.5 (50)	0 of 2 (0%)
2	13	3	7	5.0 (45)	11 of 13 (85%)
3	8	3	6	4.5 (35)	0 of 8 (0%)

Across parks, 21 youth (46%) participated as Junior Camp Counselors during summer camp. Despite flexibility related to the JCC role, there was much overlap across parks in eligibility criteria (e.g., attended 75% to 80% of groups, demonstrated professional behavior) and JCC responsibilities. JCCs assisted a minimum of two times per week, during 2-4 hour shifts, with activity planning, preparation and clean up, and they served as positive role models, encouraged to utilize and model problem-solving, emotion regulation, and effective communication skills. They were paired (by schedule and perceived compatibility) and supervised weekly by a combination of summer counselors, full-time instructors, and mental health providers.

Overall Program Effects

Table 3 summarizes results, for individual parks and overall sample, from t-tests that examined parent- and staff-reported changes in youth social skills and problem behaviors from baseline to post-test, post-test to follow-up, and baseline to follow-up. Summarized here are the results for the total sample across parks. There were no significant changes in parent report of Social Skills over time: baseline to post-test: t₆₂ = −0.23, n.s., post-test to follow-up: t₅₀ = 1.19, n.s, and baseline to follow-up: t₆₄ = 1.08, n.s. Despite a trended increase in parent-reported Problem Behaviors from baseline to post-test, t₆₅ = −1.84, p = 0.56 (Cohen’s d = −0.46), these ratings returned to baseline levels by follow-up, t₅₀ = 1.83, p = .07 (post-test to follow-up) and t₆₅ 0.26, n.s. (baseline to follow-up).

Table 3.

SSIS Mean scores by parent and staff report across baseline, post-test, and follow-up assessments.

		Parent Report
		Baseline		Post-Test		Follow-Up
Park		Social Skills	Problem Behaviors	Social Skills	Problem Behaviors	Social Skills	Problem Behaviors
1		94.33(20.77)	15.87(11.26)	103.22(27.74)	30.50(32.39)	73.70(24.89)*bc	19.80(14.14)
2		95.23(30.42)	16.80(20.00)	100.50(23.91)	25.88(23.96)	97.44(33.24)	14.63(13.61)
3		108.36(20.82)	11.55(15.43)	95.75(11.78)	14.75(13.85)	106.50(21.69)	6.38(6.28)
Overall	n	39	41	25	26	27	26
	M	98.59(24.57)	15.05(15.77)	99.96(21.77)	24.23(25.25)	91.33(29.74)	14.08(12.92)
	d	--	--	−.06	−.46	.27	.07

		Staff Report
		Baseline		Post-Test		Follow-Up
Park		Social Skills	Problem Behaviors	Social Skills	Problem Behaviors	Social Skills	Problem Behaviors
1		-	-	84.58(20.19)	14.42(13.01)	94.75(17.67)	3.55(6.09)*b
2		88.69(22.32)	23.64(18.36)	-	-	92.54(6.77)	4.23(3.39)**c
3		86.29(8.53)	5.07(8.32)	120.79(22.78)***a	.36(.74)*a	100.38(19.75)*bc	.54(1.39)
Overall	n	27	25	26	26	38	37
	M	87.44(16.37)	13.24(16.33)	104.08(28.08)*a	6.85(11.22)	95.92(15.67)*c	2.73(4.19)*b***c
	d	--	--	−.72	.46	−.53	.88

Note. Standard Deviations reported in parentheses;

^abaseline to post-test

^bpost-test to follow-up

^cbaseline to follow-up

^*p < .05

^**p < .01

^***p < .0001. Due to data collection difficulties, baseline staff-report data were not available for Park 1, and post-test staff-report data were not available for Park 2.

Due to data collection difficulties, baseline staff-report data were not available for Park 1, and post-test data were not available for Park 2. Staff-reported Problem Behaviors for the total sample across sites showed no change from baseline to post-test, t₄₉ = 1.64, n.s., but declined significantly by follow-up, t₆₁ = 2.04, p < .05 (post-test to follow-up) and t₆₀ = 3.75, p < .0001 (baseline to follow-up). Staff-reported Social Skills improved from baseline to post-test, t₅₁ = −2.56, p = .01 and follow-up, t₆₃ = −2.11, p < .05, and gains were maintained from post-test to follow-up, t₆₂ = 1.49, n.s. Effect sizes based on overall means from the total sample showed staff-reported reductions in Problem Behaviors (d = 0.46 at post-test and 0.88 at follow-up) and gains in Social Skills (d = −0.72 and −0.53, respectively).

RCI data are illustrated in Figure 1. Across sites, 45% of youth exhibited improved social skills and 44% of youth exhibited fewer problem behaviors over time by parent report. These numbers were even higher according to staff report: 68% of youth showed improved social skills and 59% of youth showed reductions in problem behavior. Service use and reliable change data are summarized in Table 4 for youth (n = 38) classified as low, moderate, or high risk according to parent-report SDQ scores of baseline mental health need. Low and moderate risk groups showed an overall pattern of attendance and reliable change that resembled findings for the full sample. Highest risk youth attended the most groups overall, 40% showed improved social skills and reductions in problem behavior by parent report, and more than half showed improvements in both domains by staff report. Equivalent numbers of youth from each risk category (about 50%) met eligibility criteria and participated as JCCs.

Figure 1.

Percent youth demonstrating reliable change over time.

Table 4.

Service use and reliable change for youth at low, moderate, and high risk.

	Low Risk (n=14)
	Service Use
Intervention	Minimum # Groups Attended	Maximum # Groups Attended	Mean # Groups Attended	# JCCs
Leaders @ Play	0	14	6.8	6
Families @ Play	0	7	2.7

	Reliable Change
	Parent Report (n = 8)		Staff Report (n = 8)
Category	Social Skills	Problem Behaviors	Social Skills	Problem Behaviors
Improvement	4	3	6	2
Deterioration	3	2	1	0
No Change	1	3	1	5

	Moderate Risk (n=5)
	Service Use
Intervention	Minimum # Groups Attended	Maximum # Groups Attended	Mean # Groups Attended	# JCCs
Leaders @ Play	0	11	5.2	2
Families @ Play	0	7	2

	Reliable Change
	Parent Report (n = 4)		Staff Report (n = 5)
Category	Social Skills	Problem Behaviors	Social Skills	Problem Behaviors
Improvement	0	2	2	3
Deterioration	3	0	2	0
No Change	1	2	1	2

	High Risk (n=19)
	Service Use
Intervention	Minimum # Groups Attended	Maximum # Groups Attended	Mean # Groups Attended	# JCCs
Leaders @ Play	0	20	10.8	8
Families @ Play	0	13	4.8

	Reliable Change
	Parent Report (n = 17)		Staff Report (n = 18)
Category	Social Skills	Problem Behaviors	Social Skills	Problem Behaviors
Improvement	7	7	10	12
Deterioration	3	4	1	0
No Change	6	6	7	5

Discussion

Reflecting a public health framework, we sought to reduce emotional distress among adolescents (grades 6-8) through training, practice, and peer modeling in problem solving, emotion regulation, and effective communication and to prevent the emergence of behavioral, social, or emotional difficulties among children (summer campers in grades K-5), who were expected to benefit from improved counselor-to-camper ratios and modeling of targeted skills by JCCs. Feasibility data revealed enthusiasm among staff (100% of eligible staff consented to participate), families, and youth (Leaders @ Play enrollment exceeded number of 6^th-8^th grade peers enrolled in other park programs). Youth and family groups were offered according to the planned schedule (weekly and semi-weekly, respectively), with minimal cancellations for inclement weather and rising neighborhood violence. Attendance and completion rates hovered around 50%, mirroring national estimates in outpatient mental health service use (Gopalan et al., 2010) and exceeding average rates of retention and service completion for urban, low-income youth (McKay, Lynn, & Bannon, 2005). Program effects suggest the model holds promise, including for youth exhibiting elevated and clinical levels of mental health need. Aggregated data revealed overall increases in staff-reported social skills and decreases in staff-reported problem behaviors, with moderate to large effect sizes.

Parks Provide a Home for Social Emotional Learning

Findings lend support to recreational after school programs as a space and tool for social emotional learning. First, low mobility among staff and youth can facilitate adult-youth partnerships, mentoring, and supportive relationships, all indicators of high quality programs (Kahne et al., 2001). Second, students expressed a preference for recreation over didactic instruction. In fact, late arrival was a persistent problem, as several students strategically arrived just in time for sports or games, leading to frequent discussions among investigators and park colleagues regarding the extent to which such behavior warranted dismissal from the program. However, increasing violence surrounding the parks and staff commitment to keeping youth safe and engaged led to alternative revisions (e.g., starting and ending sessions with recreational activities) and consequences (e.g., rewards for arriving on time and remaining until the end).

Third, a large body of evidence highlights persistent challenges associated with engagement and retention of urban, ethnic minority, and low-income families in conventional outpatient mental health family groups (McKay et al., 2011). Extending our prior effort to engage families, with less traditional service formats (Omitted for blind review), Families @ Play used recreation (games, art, music) to introduce, practice, and reinforce specific skills. Family attendance approached 50% and participation included extended family (e.g., cousins, grandparents) suggesting opportunities for even greater reach. Few parent-reported outcomes revealed change, including scant evidence of movement in the wrong direction, suggesting more effort is needed to increase family participation and support youth’s application of skills across settings. Despite common barriers to participation (e.g., work, illness), families noted a preference for groups at the park (versus school or clinic), reflecting the parks’ history as community centers, where the risk of judgment or disapproval was low. Anecdotally, families were more likely to attend when their own child was scheduled to co-facilitate an activity, which has led us to revise the model such that all youth teach their own families the targeted skill at each group meeting and lead the associated recreational activity. Nearly 100% of families whose youth interned during summer as JCCs came together for a formal, end-of-summer graduation.

Limitations, Lessons Learned, and Future Directions

Program effects should be interpreted with caution given the absence of a control group and reporting by staff that also contributed to program implementation. Nevertheless, reliable improvement for individual youth at three levels of mental health need are encouraging and suggest the model holds promise for strengthening social skills and reducing problem behaviors, even for youth exhibiting clinical levels of distress and related impairment. Feasibility data revealed important challenges that are the focus of our ongoing work. These include increasing attendance and completion rates, and improving park staff comfort and capacity to co-lead and sustain youth and family groups. This pilot work is informing iterative revisions to the service model and research design for a larger, more rigorous and tightly controlled forthcoming trial. In particular, we are focusing on staff training, supervision, and fidelity measurement.

First, our park colleagues sought to engage middle school youth at heightened risk for misconduct, alcohol and drug abuse, and gang recruitment, but they were concerned about their capacity to manage and support youth with more severe mental health needs. From its inception, Leaders @ Play was designed to sustain by capacity-building among front-line program staff in order to (a) provide opportunities for modeling, demonstration, and practice with feedback, adhering to the most recent recommendations for empirically-supported training (Beidas & Kendall, 2010; Salas, Tannenbaum, Kraiger, & Smith-Jentsch, 2012); (b) maximize use of sports and recreation to teach, practice, and reinforce specific SEL skills, in particular by leveraging teachable moments; (c) enhance the likelihood that the program could be implemented by park staff and sustained with indigenous park resources, while identifying particular components that may require ongoing consultation from mental health providers and / or university partners; and (d) support the development of manuals for wider dissemination.

Despite significant enthusiasm by park supervisors and staff across all three sites, the extent to which they were involved in implementation varied significantly; mental health providers too often assumed primary responsibility for the groups, and park staff co-facilitated with less frequency than planned. Competing priorities, insufficient staff, and inadequate (unpaid) time for training or planning interfered with staff involvement. However, park and family enthusiasm for the program led to increased investment by park administrators over time, resulting in Park 3 assigning additional staff to implement the program, and by the end of the project, Leaders @ Play was listed among other after school programs (without a fee) at a small number of parks on the park district registration website. Among future priorities, we are working on a plan for workforce development that would prioritize the need for park staff and mental health providers to receive training and supervision together. We are borrowing from our prior work in which mental health trainees (rather than project-funded social workers) (Omitted for blind review) work closely with park staff to improve the generalizability and sustainability of the overall model while supporting an annual community-based field-placement for social work trainees.

Second, we have been developing a model for fidelity measurement that is both efficient (i.e., clinically useful) and effective (i.e., reliable and valid), per recent recommendations (Schoenwald et al., 2011). In the current trial, progress notes prepared and maintained by mental health providers were the primary source of fidelity measurement. They were designed to guide supervision related to attendance, activities planned and completed during groups, individual goals for youth and families, and problems and solutions regarding barriers to participation, engagement, or progress. For the summer internship, we developed Tootles Cards based on the concept introduced by Fantuzzo and colleagues (1991), and designed to leverage peers as sources of reinforcement (i.e., tootles as opposed to tattles). These cards provided systematic and structured opportunities for JCCs to observe, recognize, and praise their partner’s demonstration of specific skills or positive interactions with counselors, parents, or campers. This allowed them to capitalize on opportunities for peer modeling, practice with feedback, and fidelity measurement via permanent product. Feedback from JCCs and park staff led to iterative revisions, minimizing their usefulness as a fidelity tool in the current open trial but providing a measure for use in the subsequent planned randomized controlled trial.

Summary

During early adolescence youth independence and peer influences increase, parental supervision wanes, and opportunities to engage in risky behaviors become more plentiful and attractive. Leaders @ Play responded to local priorities and national recommendations (O’Connell et al., 2009) to help urban early adolescents reach developmentally and culturally appropriate milestones, including healthy relationships, school success, and workforce skills. Program effects are especially encouraging for youth at highest risk, whose attendance and outcomes demonstrated high engagement, skills acquisition, and reductions in problem behaviors over time. Findings encourage increased effort and investment to leverage the inherent capacity of after school recreation programs to mitigate risk and strengthen resilience for vulnerable youth.