An Outcome Evaluation of the Success for Kids Program

Short abstract

A nonreligious after-school program, Success for Kids (SFK) seeks to build resilience in children. Interestingly, it improves not just social and internal outcomes but also school-related outcomes, even though it is not an academic intervention.

Abstract

This article presents results from a multisite, quantitative evaluation of the international Success for Kids (SFK) after-school program. The program seeks to build resilience in children by teaching them to access inner resources and build positive connections with others. The SFK program is unlike most after-school programs both in its focus on spiritual development and in its emphasis on outcomes related to resilience rather than academics. The authors found that the program had beneficial effects on adaptive skills, behavioral problems, overexternalization of problems, and school problems, and the effects persisted at 12-week follow-up. Interestingly, the program positively affected school-related outcomes, even though SFK is not an academic intervention. Specifically, it improved reported study skills and reduced reported learning problems and attention problems. Overall, a major strength of the program appears to be its careful attention to uniformity of program delivery—in particular, its standardized curriculum, use of experienced teachers, and formal teacher-training program.

This article presents results from a multisite, quantitative evaluation of the international Success for Kids (SFK) after-school program. The program seeks to build resilience in children by teaching them to access inner resources and build positive connections with others. The level I SFK program, titled “The Game of Life,” is a ten-part course offered weekly in 90-minute lessons free of charge to children between the ages of 6 and 14 who are enrolled in existing after-school programs. The course uses structured games and activities to teach such concepts as cause and effect, how to control reactive behaviors, the value of sharing with others, and the importance of making an effort. Through its curriculum, the program aims to increase prosocial behaviors and reduce antisocial and problem behaviors. The SFK program is unlike most after-school programs both in its focus on spiritual development and in its emphasis on outcomes related to resilience rather than academics.

Although after-school programs in the United States receive significant financial backing from both public and private sources, the literature assessing their effectiveness has many methodological weaknesses. Among the better studies of programs addressing social behaviors (such as SFK), program effects have been shown to be small. This study attempted to avoid the pitfalls evident in the research literature by using a very rigorous methodology: randomization of program sites to treatment and control groups and repeated measurements over time. This research design was possible owing to a unique window of opportunity: SFK was rapidly expanding its operations in southeast Florida, which enabled us to randomly assign 19 new program sites to either immediate implementation of the program or delayed implementation after a 12-week waiting period. During the waiting period, the delayed-implementation sites formed a “control group” for the sites receiving immediate implementation (the “treatment group”). The 19 program sites were diverse both in their demographic composition and in their spatial location throughout Miami-Dade, Broward, and Palm Beach counties.

Behavioral outcomes were measured using the well-known Behavior Assessment System for Children, Second Edition (BASC-2). Children and their regular after-school–program teachers (not the SFK teachers) were assessed at pretest, posttest, and follow-up. An extensive analysis of reliability and validity revealed that the self-report data collected from children were plagued with inconsistencies, and thus conclusions are based on the data collected from teachers.

We found that the program had beneficial effects on virtually every domain covered by the BASC-2. In particular, the program had medium to large effects on adaptive skills (effect sizes [ESs] of 0.55 to 0.73),* which include adaptability, social skills, leadership, study skills, and communication skills. The program had small to medium effects on behavioral problems (0.19 to 0.37), especially attention problems and withdrawal, and small effects on overexternalization of problems (0.16 to 0.29). The program had small to medium effects on the reported incidence of school problems (0.32 to 0.48). There is suggestive evidence that many effects persisted at 12-week follow-up. Notably, the ESs exceeded the average ESs found for after-school programs targeting similar outcomes (0.19 for positive social behaviors and 0.18 for problem behaviors), as calculated by Durlak and Weissberg (2007) in a recent meta-analysis.

Of significant interest is the finding that the program positively affected school-related outcomes, even though SFK is not an academic intervention. Specifically, the program improved reported study skills and reduced reported learning problems and attention problems. SFK's success in improving school-related outcomes suggests that an extremely interesting follow-up study would be one that examined program effects on grades and subsequent standardized test scores.

Because the program continues to expand to not only new communities but different countries and settings, we recommend a follow-up study to test the replicability of the SFK model in other contexts. For example, the program was evaluated in southeast Florida, where it is delivered in school and after-school settings, but, in other countries, notably Mexico, Panama, and Malawi, it is delivered in family centers and orphanages. Likewise, in some U.S. and Latin American sites, it is delivered in Spanish rather than English.

Also, while many program effects were reasonably persistent at 12-week follow-up, there was some variation among the outcomes in whether treatment effects rose or fell with time. This suggests that follow-up programming could be used to support the treatment effects achieved with the level 1 course. We recommend evaluation of the SFK level 2 and level 3 courses to test whether they can support and perhaps even build on the effects achieved after the level 1 course. Future evaluations might also seek to extend the follow-up period beyond 12 weeks. Overall, a major strength of the program appears to be its careful attention to uniformity of program delivery—in particular, its standardized curriculum, use of experienced teachers, and formal teacher-training program.