Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Nov 5.
Published in final edited form as: Am J Community Psychol. 2014 Jun;53(0):491–502. doi: 10.1007/s10464-014-9656-0

Top-Down, Bottom-Up, and Around the Jungle Gym: A Social Exchange and Networks Approach to Engaging Afterschool Programs in Implementing Evidence-Based Practices

E P Smith, E Wise, H Rosen, S Childs, M McManus
PMCID: PMC4634874  NIHMSID: NIHMS734142  PMID: 24781678

Abstract

This paper describes the use of concepts from Social Networks and Social Exchange theories to implement an evidence-based practice in afterschool programs. The members of the LEGACY Together Afterschool Project team conduct collaborative research to design and deliver a behavioral strategy that has been documented to reduce disruptive behaviors in classroom settings to a new setting—that of afterschool programs. We adapted the Paxis Institute’s version of the Good Behavior Game (PaxGBG) to context of afterschool, which exhibits many differences from in-school settings, including more fluid attendance, multiple age groupings, diverse activities that may take place simultaneously, and differences in staff training and experience (Barrish, Saunders, & Wolf, 1969; Embry, Straatemeier, Richardson, Lauger, & Mitich, 2003; Hynes, Perkins, & Smith, 2009; Kellam et al., 2008). This paper presents the experiences of the three adult groups involved in the implementation process who give first-person accounts of implementation: 1) university-based scientist, 2) community partners who trained and provided technical assistance/coaching, and 3) an afterschool program administrator. We introduce here the AIMS model used to frame the implementation process conceptualized by this town-gown collaborative team. AIMS builds upon previous work in implementation science using four phases in which the three collaborators have overlapping roles: Approach/engagement, Implementation, Monitoring, and Sustainability. Within all four phases principles of Social Exchange Theory (SET) and Social Network Theory (SNT) are highlighted.

Topic Areas: Afterschool, Out-of-School time, Understanding communities, partnerships, coalitions, Implementation

Background and Introduction

Social network theory (SNT) can contribute to the understanding of the relational processes that contribute to quality implementation of evidence-based practices (Daly et al., 2009; Finnigan, Moolenaar & Daly, 2012; Neal, Neal, Atkins, Henry, Frazier, 2011). Social Exchange Theory (SET, Emerson, 1976) helps in the consideration of the circumstances in which stakeholders are willing to participate in an exchange of knowledge and practices. This paper uses SNT and SET to describe the LEGACY1 Together Afterschool Project, a university-community partnership of scientists and practitioners designed to foster implementation fidelity in the use of evidence-based practices. We use First Person Accounts from the three groups involved in implementing the strategy namely the: 1) scientist-practitioner, 2) community-based partner, and 3) afterschool program administrator. The First Person Accounts are valuable in that they allow us the opportunity to document in the literature the human relational processes in implementing large-scale prevention initiatives, perspectives that could be particularly valuable to community-based researchers. This paper will use the First Person Accounts to describe the experiences of the various stakeholders in implementing evidence-based practices in afterschool programs. These Accounts will highlight social processes germane to Social Network Theory and Social Exchange Theory. To organize our descriptions of these processes in the first two accounts, we describe them fairly sequentially, using our newly conceptualized AIMS model, akin to previous research (Glasgow, Vogt, & Boles, 1999; Lyon, Frazier, Mehta, Atkins, & Weisbach, 2011). The phases of the AIMS model are: 1) Approach and engagement which includes developing mutual familiarity and trust between the program staff and research team; 2) Implementation of the empirically-based practices in which the programs begin to receive training and technical assistance from a supportive coach; 3) Monitoring – the use of data to track progress in using PaxGBG; and 4) Sustainability and Continuous Quality Improvement – moving into a phase in which less frequent, but periodic support is provided to help programs self-assess and monitor the use of best practices. The last First Person Account (by the afterschool administrator) provides a voice across all four phases of implementation, but uses a less linear written style. Figure 1 gives a schematic overview of the AIMS implementation phases, specifies which of the three partners are intimately involved in each phase, and gives examples from SNT and SET that were used to enhance implementation of PaxGBG to the afterschool programs.

Figure 1.

Figure 1

Open in a new tab

The AIMS Model: Stakeholders and Theoretical Applications

This paper describes the “science migration” of the Paxis Institute’s version of a 40-year old behavioral management strategy developed for schools, the Good Behavior Game (PaxGBG) to afterschool programs (Barrish, Saunders, & Wolf, 1969; Embry et al., 2003; Hynes, Smith, & Perkins, 2009) GBG has been endorsed by OJJDP as an Effective Program and is acknowledged as a “Promising Program” in the Blueprints for Violence Prevention (Mihalic, Irwin, Elliott, Fagan, & Hansen, 2001). In past prevention research in school settings, when GBG was implemented in 1st and 2nd grade, it demonstrated reductions in aggression, delinquency, and substance abuse with effects lasting well into emerging adulthood for the most aggressive males (Ialongo et al., 1999; Kellam et al., 1994; Kellam et al., 2008).

Recently, Embry and colleagues (2003) at the Paxis Institute (Pax means “peace” in Latin) have developed a pre-packaged version of the 40+-year old game for classroom teachers, PaxGBG, complete with the manual and necessary materials (posters, timer, etc.), a package that we adapted for use with afterschool staff in collaboration with Dr. Embry and our community-based partners. The game builds upon youth social processes in that in order to “win the game” team members must inhibit off-task behavior, exhibit self-regulation and control, and appropriately encourage their team members to do the same using gentle verbal and symbolic reminders (e.g. thumbs up,/thumbs down). Upon winning in PaxGBG, youth receive group-based activity prizes--usually actions that they are admonished not to do during the program such as active dancing, yarn-ball fights, pencil tapping, or yelling for brief periods of 15 to 30 seconds. The youth and staff collaboratively nominate the activity prizes for their own site that are placed in a jar and selected by winning teams.

The goal of this effective behavioral strategy is to foster a sense of connectedness and empowerment among youth who positively influence each other. This sense of connectedness and empowerment has been termed “collective efficacy” and among adults in Chicago neighborhoods is related to reduced crime and delinquency (Sampson, Raudenbush, & Earls, 1997). Recent research is finding that collective efficacy among youth is related to attitudes less accepting of violence, less emotional maladjustment, and lower levels of delinquency and substance use (Odgers, Moffitt, Tach, Sampson, Taylor, Matthews, and Caspi, 2009; Smith, Osgood, Caldwell, Hynes, and Perkins, 2013). This research suggests that the idea of building a sense of community and power among youth in afterschool might bode well for youth behavioral outcomes.

The Significance of Afterschool Settings and Evidence-Based Practices

Afterschool is a promising setting for prevention activities for several reasons. Though a substantial number of prevention approaches focus on the family, peer, and school contexts, the after-school setting has been much less explored. The time afterschool is important in that 70% of all crime committed by youth occurs between the hours of 3–6pm (Snyder & Sickmund, 2006). Providing appropriate structure and supervision for children during out of school time (OST) is an important work-family issue enabling parents to work without the additional stress of worrying about the care of their children. Also, afterschool is a time in which youth can plan and be involved in more engaging and enriching opportunities that may affect their life trajectories (Eccles & Gootman, 2002; Larson, 2000; Lerner, Fisher, & Weinberg, 2005; Smith, 2007). Afterschool programs that are appropriately structured, engaging, and that use empirically-based practices are found to be most beneficial in terms of youth competence, behavior, and substance use outcomes (Gottfredson, et al., 2004; Durlak, Weissberg, & Pachan, 2010; Pierce, Bolt, & Vandell, 2010; Tebes et al., 2007). Thus, our project was focused upon helping the youth and staff of afterschool programs to implement an evidence-based practice, problem behavior.

Integrating Social Network and Social Exchange Theory into Implementation Science

Social Network Theory is helpful in understanding the group processes among the staff implementing evidence-based practices that deserve careful attention and consideration. The conceptual underpinnings of group social processes can be found in several foundational premises in psychology and sociology. At a time when psychology emphasized the role of individual motivation and emotion, Kurt Lewin (1947) conceptualized human behavior as a function of the interaction between individuals and their environment, including attention to both individual and group life spaces. He argued that the group was more than the sum of its parts and that group perceptions and decision-making could serve as a force-field either supporting or resisting change. In afterschool settings, the youth share a group space with each other, and with the adults with whom they interact. The adults share group space as well, in which they influence each other’s practices and beliefs regarding the best way to appropriately structure and support youth interactions in afterschool.

Theoretical premises in Social Networks Theory focus upon the patterns of relationships and interactions that influence human development. Granovetter (1973), a substantial contributor to social network theory, highlighted the possibility that not only strong but weak ties within social networks are important influences. Indeed, implementation science finds that more integrated networks including relationships with and among multiple actors, versus more central ones which are virtually connected to one actor, are most effective in promoting the use of evidence-based strategies among community practitioners (Feinberg, Riggs, & Greenberg, 2005). Thus, implementation is more likely to occur when there are relationships among multiple actors rather than dependency upon one central advocate for evidence-based practices. However, though the “power of many is important,” there is also support for the “power of one” in that having a leader who is a program advocate is beneficial to implementation fidelity (Butterfoss, Goodman, & Wandersman, 1993; Kam, Greenberg, & Walls, 2003). Thus both central and diffuse networks may play various but important roles in the supporting the process of adoption and implementation.

Social network theory also provides the foundation for attending to the patterns of social relationships and influence that can either hinder or promote change. School-based research has shown that the people whom one most often seeks for information about best practices are one’s colleagues with whom they interact on a daily basis, relationships that can either support or constrain the use of innovative approaches (Moolenaar, 2012, Moolenaar & Daly, 2012). It was the goal of this project to develop “professional learning communities,” groups of afterschool staff both within and across programs who were interested in supporting and collaborating together in the use of innovative and effective behavioral strategies in afterschool (Daly et al., 2009; Lieberman, 2000).

Combined with Social Network Theory (SNT), Social Exchange Theory (SET) examines networks of exchange in which others are willing to share and receive information and resources (Emerson, 1976; Cook & Emerson, 1978; Cook, Emerson, Gillmore, & Yamagishi, 1983). Social Exchange Theory has psychological, sociological, and even economic underpinnings. From a psychological framework, the idea of positive reinforcement and its impact upon increasing desired behaviors is important. However, this does not mean that a simplistic behavioral view is sufficient for understanding the role of social exchange in implementation science. Indeed, some organizational members may resist change that has economic benefits, especially if they view the potential costs to them, and the populations they serve, to outweigh the benefits (Battilana & Casciaro, 2013). On the other hand, strong affective connections, in which concerned parties know and trust each other, may be powerful in overcoming resistance, in ways that are particularly salient to “fence-sitters” and resistant staff who oppose organizational change (Battilana & Casciaro, 2013; Daly, Moolenaar, Bolivar, Burke, 2009). Often, group members need to feel that they share in power and decision-making more equitably--critical factors that may affect their receptivity to receiving and exchanging information among change agents (Cook & Emerson, 1978; Cook et al., 1983). Coupled with a sense of trust, the value of the potential benefits relative to the costs, are factors potentially affecting the degree to which members of a network will engage in an exchange of information, time, and resources (Cook & Emerson, 1978; Cook et al., 1983).

Our conceptual model (Figure 2) draws upon the premises of Social Exchange Theory (Emerson, 1976), highlighting the roles of trust, rapport, and appreciation of cost-benefits in engaging afterschool programs in implementing evidence-based practices. In the initial phases of Approach/engagement, we considered whether our project matched the goals and missions set forth by the organizations we planned to engage and the degree to which key personnel and resources aligned with these goals and objectives (Lyon, et al., 2011). As part of our research initiative, we were able to provide resources to incentivize and support training and implementation, a potential benefit to the afterschool programs. We endeavored to build trust with key decision-makers at multiple levels of the organization including management and on-line staff in ways that fostered opportunities for honest, clear, and authentic expression, so critical to building the partnerships. Trust and participation are more likely garnered when staff perceive benefits personally and collectively relative to the costs of being involved in the research and ongoing technical assistance initiatives. Further, we demonstrated ongoing commitment and support which went beyond the start and end dates of a research project in order to foster a truly supportive relationship with the potential for sites to continue incorporating the evidence-based organizational practices (Johnson, Hays, Center, & Daly., 2004; Lyon et al., 2011).

Figure 2.

Figure 2

Open in a new tab

A Social Exchange Model of Recruitment and Engagement

First Person Accounts: The Perspective of the Scientist-Practitioners

The university-based scientist-practitioners, brought over 20 years of prevention science experience in family, school, and afterschool settings. We had experience implementing GBG in school settings [author reference] and felt that it would be helpful to provide appropriate structure and support without overburdening afterschool staff with curriculum. GBG is a behavioral strategy, an overlay that can be used in concert with other planned activities allowing afterschool staff flexibility and fidelity in implementation. We begin by describing our efforts to engage afterschool programs in the project.

Phase 1: Approach/Engagement of the Afterschool Programs

The research team, and in particular the PI and Project Coordinator, identified afterschool programs in urban, suburban, and rural locales across southeastern Pennsylvania to participate in the project, gathering in-depth information about the socio-economic and racial-ethnic composition. In each locale, we searched on school district and school web-sites, to identify diverse afterschool program providers including school districts, 21st Century Community Learning Centers, private providers, local YMCA/YWCA’s, and local parks and recreation agencies.

The research team recognized that all of our actions would be interpreted by the community as either helping to build or undermine their potential trust in this project. Trust is an important dimension in the social exchange theory model (Emerson, 1976). However, for our last cohort, a national scandal at the university relevant to the protection of children occurred during our recruitment and engagement period. We were relieved that many sites still welcomed our team and even sought our guidance for how to foster the attainment of higher education for their youth. Apparently, they perceived potential benefits to community-based action research models designed to support their youth and programs (Cook & Emerson, 1978; Israel, Eng, Schulz, & Parker, 2005; Wallerstein, & Duran, 2006). In terms of Approach and engagement, we began with a “top-down” approach contacting program providers via email with a one-page Executive Summary stating the goals and giving a description of the project. A top-down approach (Sabatier, 1986) helped to ensure that the ultimate decision-maker was at the table when the project was described. Though we attempted to keep the program costs low in terms of monetary resources, it would require a significant commitment of time to involve staff in training and technical assistance for PaxGBG.

However, we recognized that the participation of staff in the decision-making process would lead to less opposition and more staff buy-in, speaking to the importance of bottom-up approaches as well (Sabatier, 1986). In the earliest stages of the project, some Program Directors did not include on-line staff in our initial meetings and this more centralized approach was less effective. Research from a social networks perspective debates the utility of a centrality approach that includes only program directors versus involving on-line staff as well, models of multiple more diffuse ties (Daly & Finnigan, 2011; Feinberg, Riggs, & Greenberg, 2005; Granovetter, 1976; Masquefa, 2008). In our experiences in those programs where only the directors were engaged, staff served more as obstacles than facilitators of the work.

In contrast, sites in which staff were engaged at the onset had more opportunities to consider the utility of the project, felt more knowledgeable, exhibited higher levels of attendance and participation in training and implementation, and literally helped to inform families of the youth about the benefits of participation. Thus, working top-down/bottom-up and “around the jungle gym,” resulted in more buy-in among after-school program directors, staff, and eventually the parents of the participating children.

At times, the PI communicated directly with parents who were concerned about the consenting process, the content of the surveys, observations, and/or the school-related data. We openly shared that information would be collected on sensitive problem behaviors among youth, such as vandalism, theft, and substance use, but that this information would be used to understand how and when these issues normally emerge for youth and ways to address them. We were able to secure the consent of an average of 84% of the parents of nearly 1,000 children across our various program sites.

Phase 2 – Implementation – Supporting Evidence-Based Practices in Afterschool Programs

We describe the elements of our project that sought to more effectively engage program participants in implementation.

The Project Kickoff

The project kickoff was the first event including multiple programs and staff all in one place to present the project, detailed calendars, expectations, and to entertain questions and suggestions. Understanding the demands upon afterschool staff, who are often part-time, or with multiple jobs, and little opportunity for training and or even collaboration with other staff within the program, the Project Coordinator worked to identify dates, times, and locations that were easily accessible to the programs and research team. Multiple programs were included at the project kick-off event and subsequent trainings, to facilitate social networks and learning communities of mixed ability and success who could share information on how to best integrate the practices into real-life afterschool programs.

Programs, were previously matched on size, socio-, and racial-ethnic composition, and staff flipped the coin to determine which sites would receive the innovative program versus the treatment-as-usual control site, another procedure designed to engender trust in that they determined and witnessed random assignment to condition (Emerson, 1976). Across all 76 of the program providers we approached, only had one provider refused (a YWCA with 5 program sites that was directed by an expectant mother at the time), and one other provider to drop-out (a brand new administrator of seven program sites who did not feel ready in her first year of on-the-job training).

Considering the Role of Leadership in Implementation

Centrality is a concept in social networks theory that consider the degree to which a “prevention advocate” might influence implementation for organizational members. Leadership that is supportive of empirically-based practices and sensitive to the needs and concern of staff can be instrumental in promoting implementation (Butterfoss, Goodman, & Wandersman, 1993; Johnson et al., 2004; Kam, Greenberg, & Walls, 2003). Research supports the idea that programs that are stronger in terms of leadership and capacity also do a better job of implementing evidence-based practices, and in particular, PaxGBG (Flaspohler, Stillman, Duffy, Wandersman, & Maras, 2008; Halgunseth, Carmack, Childs, Craig, Caldwell, and Smith, 2012; Wandersman et al., 2008).

Cultural Sensitivity

In considering the contributions, not only of program leaders but also other staff, we recognize that multiple relationships among staff might influence levels of implementation (Neal et al., 2011). In our engagement of afterschool sites, we found we had to be attuned to the values and sensibilities of the staff and their communities. Some older, often ethnic minority staff, objected to practices that would “reward” children We reframed the contingent group activity awards as allowing students to “earn” their activities, such as free-play and peer interaction time stressing the importance of both appropriate structure and adult support to children’s development, even for ethnic minority youth (Hudley & Taylor, 2006; McLoyd & J. Smith, 2002). Additionally, the use of activity rewards minimized the costs to staff and programs and promoted belonging among staff and students who enjoyed the activity rewards of a “Soul-Train line” or yarn-balls flying across a school cafeteria together.

Training the Trainers: The Implementation Team

We recruited and trained coaching staff using day-long, in-person trainings, readings, weekly/bi-weekly video-conferences that used weekly implementation data and problem-solving to support the coaches.

The next section presents the perspectives of the community-based partners describing the Implementation, Monitoring, and Sustainability phases in which they were involved.

Perspectives of the Community-Based Partners: Alison and Howard Rosen

Our organization, Hempfield Behavioral Health, has been delivering empirically-based services to youth and families for over 25 years. We have a history of being involved in empirically-based randomized trials, and brought several empirically-based strategies (Nurse-Family Partnership, PATHS, Multi-systemic Therapy, and Family Preservation) to our local area and sponsor an annual international conference on a Blue prints school-based socio-emotional learning strategy. When we first became a part of the LEGACY Together project, we were delivering 21st Century elementary afterschool programs. Our programs were unique in that we employed staff full-time so that they had time to connect effectively with both the parents and teachers of the children. The initial pilot project was conducted in our own afterschool sites. The project blossomed into us providing the training and technical assistance to other afterschool programs. The AIMS phases in which we were involved and applications of SET and SNT during these phases are shown in Figure 1.

There was a wide spectrum of leadership involvement, program vision, and knowledge of afterschool best practices among the afterschool administrators and supervisors who were involved. It was observed that the administrators and supervisors who were stronger in these areas had higher quality programs. LEGACY project staff kept in monthly contact with an assigned administrator from each afterschool program to share information on PaxGBG implementation. Administrators were also encouraged to observe PaxGBG implementation at their sites and to attend PaxGBG training with their staff.

There was also a wide range of structure in the afterschool programs that participated in the LEGACY project. The PAXGBG coaches made a few visits to their sites prior to the first training. This gave them an opportunity to assess how the afterschool site operated; the daily schedule, how the staff and students interacted, how the staff encouraged student leadership skills, and classroom management strategies used by the staff. This assisted the coaches in tailoring the PaxGBG trainings and their coaching visits.

Phase 2: Implementing PaxGBG

The Training Model

The model for training and coaching was refined as coaches received feedback from staff from previous years and observed which PaxGBG “kernels,” or strategies, staff had trouble implementing. Coaches also learned that afterschool staff often had little or no time to plan with other staff from their site. Staff at many of the sites were isolated and had little or no opportunity to share ideas with staff from other sites, even with other sites from within their own program.

The biggest concern was that many staff needed more time working on basic classroom management skills and the fundamental Pax Good Behavior Game kernels before implementing the more complex components of the Pax Game. These “kernels” included Pax Quiet (blowing an harmonica while staff and students raised two fingers indicating “peace”), Pax hands (keeping your hands to yourself), and Pax feet (walking quietly and in order). Introducing all of the intervention strategies during one training session was confusing and difficult for staff to understand and implement before these basics were mastered.

The following training techniques were used:

  1. Brief and periodic training sessions. Three two-hour training sessions were created. They were scheduled at one month intervals. Fewer PaxGBG kernels were introduced at each training session. In this way, after school staff could practice and master PaxGBG kernels with their coaches before learning new ones.

  2. Program Directors were strongly encouraged to attend training sessions. Program Directors were invited to attend all three training sessions. This gave staff a clear message that PaxGBG implementation was important and it was very helpful that the Program Directors understood what staff were learning. Some Program Directors became strong advocates for PaxGBG. PaxGBG staff continued to engage some of the less involved directors through periodic phone calls to update them on the project and to point out the positive changes in their staff and afterschool students.

  3. Built-in time for planning. Time was built into training for afterschool staff to plan when to implement the PaxGBG. This included when kernels would be used, and when GBG would be played in the daily schedule and which staff would take the lead. Time was also included for tasks such as developing teams and prizes.

  4. Time for sharing ideas and successes with PaxGBG. Training sessions included staff from multiple sites to provide time for discussion and sharing. Many afterschool staff are very creative and immediately had strong “buy in” to the PaxGBG. Being able to share ideas gave staff confidence and a feeling of pride in what they had accomplished. It also helped to motivate the more reluctant staff.

  5. Use of videos of afterschool staff. Videos were professionally created of staff and students demonstrating the PaxGBG in afterschool settings that were presented during training. This was very helpful and staff provided positive feedback in being able to watch implementation with other afterschool staff. Being able to see real afterschool situations where PaxGBG was being implemented strongly supported participant understanding.

  6. Incentives for participation. Staff were given incentives such as personal $50 gift cards and materials for their afterschool sites for participation during training. Also, our agency, Hempfield Behavioral Health, was certified to offer continuing educational units (CEU’s) for our trainings. In Pennsylvania, the Keystone Stars system awards staff pay incentives and program funding incentives for attending professional development trainings. Since our agency was accredited, staff and programs earned Keystone Stars credit for attending training which meant it helped them earn more professional development for staff, and more recognition and resources for their programs. Students also received string bags, water bottles and other prizes as incentives for participating in research surveys.

Collaborative visioning and goal-setting

A portion of the first training was spent conducting an activity called “My Wonderful Afterschool Program.” This is a visioning activity where participants were asked to imagine what they would see, hear, feel, and do more and less of in an ideal afterschool setting. The activity set the stage for developing shared rules and guidelines for staff and student interactions.

Weekly consultation and support

A Coach was assigned to intervention sites, usually with a load of 3–4 afterschool program sites. This Coach was consistent throughout the school year. Our Coaches were often called the “face” of our project in the afterschool programs; they helped the sites to make these practices come alive. Rapport was established by the Coaches early in the school year. Coaches began to visit their assigned after school sites prior to the first PaxGBG training session. Coaches assured staff that their observations and conversations would remain confidential.

Coaches would participate in the afterschool activities and assist when needed. Coaches developed a consistent coaching schedule with the site so staff knew when the Coach would be at the site. It was also very important that Coaches follow through on questions and requests from the staff and students. For example, if a site needed supplies for PaxGBG implementation that the Coach said he/she would bring out on their next visit, it was very important that the Coach follow through on this supply request. During these initial visits and during the first training coaches explained their role. They assured staff that they were not coming to the sites to evaluate or judge the staff. Their role as Coaches was to build a partnership with the staff. They were there to provide technical assistance and to assist staff in troubleshooting and individualizing PaxGBG to meet the needs of their students with fidelity. Coaches were also available to model if staff felt the need.

During site visits Coaches continued to support staff in their goals for implementation. Fidelity of implementation was critical but PaxGBG does leave room for individualization in areas such as prizes given to students, the time of the day when the game and kernels are used, and the number of kernels used by the staff. Although the LEGACY staff goal was that all sites use PaxGBG with a high level of fidelity, Coaches worked with staff to set their own goals and supported staff in increasing implementation.

Phase 3: Monitoring Implementation

Coaches completed a web-based Weekly Coach Observation tool after each site visit. Coaches marked “yes or no” if they observed each of 27 components of the Pax kernels and GBG used during the observation. Coaches also rated staff on a five point scale on staff attitude, enthusiasm, and creativity towards PaxGBG, openness to coaching, and student engagement in PaxGBG. Benchmarks were created to encourage staff to set implementation goals.

After the second training, afterschool staff were asked to complete a Weekly Game Calendar that documented information about PaxGBG kernels used during the week and information about the number of Good Behavior Games that were played (i.e. length of games, number of teams that won).

The research team created and shared graphs illustrating implementation and the Coaches were encouraged to share the graphs with afterschool program staff on their tablet devices. Being the “feet on the ground,” the Coaches helped the research team bear in mind the specific challenges, in terms of staffing, turnover, attitudes, or special- needs youth that might be influencing levels of implementation. Coaches helped the sites to address youth with high levels of misbehavior who might be impeding team wins for other youth. This is an example of how youth, like staff could be resistant to the game. In instances like these, Coaches could problem- solve with the implementation team and afterschool staff to diffuse youth resistance using praise to on-task youth, ignoring uncritical examples of misbehavior, and depriving misbehaving youth of the attention they desire. Because the Coaches worked with multiple sites, they "cross-pollinated" across afterschool programs, sharing effective strategies found in other sites (Figure 3).

Figure 3.

Figure 3

Open in a new tab

Social Exchange Networks in Supporting Implementation of Evidence-Based Practices in Afterschool: Potential Network Linkages

Coaches also provided staff with frequent reminders of the “bigger picture” of implementing evidence-based practices. By implementing PaxGBG, staff were teaching students to inhibit impulsive behavior, to work towards goals, to work together and cooperate as part of a team, and to develop leadership skills.

Phase 4: Sustainability of PaxGBG

A very important role for us as part of the coaching team was to support the sustainability of PaxGBG in the afterschool program after the duration of the research project. Coaches made sure that staff had all of their PaxGBG materials at the end of the school year. Newsletters were provided to staff on various topics such as using PaxGBG during summer camps (after the conclusion of the project) and a PaxGBG “Booster” which provided information to staff on starting up PaxGBG in their new school year. Directors were also given suggestions on how to sustain PaxGBG which included encouraging staff to continue to complete the weekly Game Calendars and to monitor this data. The Coaches visited less frequently moving to biweekly and eventually monthly visits. When they were on-site, Coaches took photographs of site implementation and documented creative ideas. Photographs were only taken with staff and parent permission. These were documented and distributed in newsletters to recognize staff and to continue to share PaxGBG ideas. Staff really enjoyed seeing their names in print and being recognized for their accomplishments.

It is typical that afterschool staff changes were frequent. Some sites had continuous staff changes throughout the school year. Coaches provided ongoing training for new staff. Coaches recruited the help of students to assist in the training for new staff. This provided an opportunity for students and helped make them feel competent.

The goals of the sustainability phase was to institutionalize PaxGBG within the site, empowering the directors and staff in continuing to play the game, helping youth to control their own behaviors, and positively influence their peers, so that everyone could be a PaxGBG “winner.”

Perspectives of an Afterschool Program Administrator: Peggy McManus

The process of implementing PaxGBG began with me as Site Administrator of a pair of afterschool programs in a medium-size school district just outside Philadelphia (See Figure 2 for phases of involvement). The initial buy-in occurred during the first meeting with the PI and the Project Coordinator The Project Coordinator not only arranged mutually agreed upon training dates, times, and places, but also personally stayed in contact with all of the afterschool program directors. When we all first met, my only perception of our large state university conducting the research, was that it was a party school. Seriously I am not saying that in jest, that was my true perception. From the first meeting I was excited and looking forward to working at the process. In hindsight, the relationship that we formed branched into much more than the GBG game, it included visits to the university and eventually my daughter enrolling in school as an undergraduate there. The components that helped were trust, realistic expectations, and a support system in place.

We had previously done GBG with a consultant who was operating solo and this process brought to light for me, the importance of having different teams to support data collection and weekly coaching and training. The coordinator of the University Survey Research Center who collected our data, the coaches, and their supervisors, Howard and Alison Rosen, were all key components of a research team that worked with us and helped us to gather the information and then put the new practices into place. The resources that I became aware of through this process were priceless. I will take them into much beyond GBG.

When you ask what does it take to get staff buy in? You need an Ahmal Martin, our coach, to connect with the staff at the site. The research team and I connected but once the program began, it took Ahmal to take it to the next level. All of our staff and youth literally greeted him with “Coach!” when he arrived onsite for his weekly visit. I truly believe that it was Ahmal that helped the afterschool staff to succeed. There was consistency, trust, a mutual respect on the part of the staff and Ahmal. They worked as a team but that is probably the next most important component, “they both worked.” Ahmal took his role very seriously and showed up when he said he would and did everything he could to help them succeed. On the other hand, the staff showed respect for his time and took what he said very seriously. They knew the expectation and they met it. The onsite director commented that having Ahmal come on site to actually see what they were dealing with in terms of space and the children, helped them to figure out ways to make PaxGBG work in their own programs. This experience tremendously changed the site director, an “old school” sort of guy who believes that discipline is critical. The support and direction enabled him to act independently and change a site in two years from the lowest functioning to the highest functioning and quickly becoming a model for others. He once commented that while afterschool programs are often despised by school teachers and staff for being unruly, he and the other afterschool staff began to feel respected by the teachers because when they encountered them, the children were well-behaved and polite. The teachers even wanted to know what we were doing differently to manage the children so well; all because we made behaving well into a game, played by staff and students alike. The children and staff at the site previously operated with great structure but much fear of making mistakes. Now with clear expectations the children have an environment that is safe, healthy and most of all fun. I am very grateful to have had the privilege to participate in this process.

Summary and Discussion

In the accounts of the various stakeholders, the principles of Social Exchange Theory and Social Network Theory are apparent. The ASP administrator emphasized several very important aspects of SET including exhibiting commitment, building trust and understanding, and providing psychological benefits to afterschool staff, such as a sense of respect within schools for their ability to manage and foster youth development. These insights point to the value of interpersonal processes consistent with SET in fostering implementation among afterschool staff.

The Community-based Partners who described the training and coaching model drew upon similar principles of SET including the coach efforts to support, build trust, and exhibit commitment to afterschool program staff and youth. They also drew upon concepts in social networks in that the training model was designed to expose afterschool staff to other staff and programs who might help them problem-solve and propose new, innovative ideas to use. Other concepts in SNT refer to the ability of the Coach to “cross-pollinate” by sharing ideas, photographs, and videos across multiple afterschool programs. On the other hand, some of the principles in implementing PaxGBG in afterschool were more practical than theoretical such as the need for shorter, more frequent trainings and providing technical assistance to hourly and part-time afterschool staff. Adapting coaching to the often free-flowing activity spaces of afterschool requires flexibility and creativity in order to have time to support and model for afterschool staff. Staff turnover is another practicality facing the implementation team. However, better application of SNT might help in developing groups of youth and staff who are committed to sustaining the use of evidence-based practices.

The scientist-practitioner team discussed the willingness of the research team to listen and consider the sociocultural values of the staff. We addressed them in ways that maintained the fidelity of the intervention; while allowing us to more equitably share in decision-making with afterschool staff from a SET perspective. Yet, the social networks approach also acknowledges that everyone will likely not buy-in and that nay-sayers and fence-sitters can influence the attitudes of others. Some implementation research has emphasized that having strong leadership, a prevention advocate, fosters the use of evidence-based practices. While we have developed some approaches to diffuse resistance, more systematic applied research could evaluate approaches to supporting buy-in and adoption (Gray et al., 2003). From a SNT perspective, the staff who did implement PaxGBG told us that participation in the LEGACY Together project, helped to create better relationships among the staff and children, and ultimately made them feel more respected within the schools in which they are operating.

Sustainability, is a challenge as matters of cost, benefits, and efficiencies can become critical for cash-strapped ASPs. Again, we would need to be sensitive to minimizing the costs and maximizing the perceived benefits (Emerson, 1976) of broader implementation, dissemination, and sustainability among afterschool programs. Sustainability requires the building of routines and systems to support and reward those routines that are part of quality practice (Fixsen, et al., 2005; Lyon et al., 2011). There is still a need for research identifying characteristics and relational processes among individuals, organizations, and strategies that use quality practices over the long-term along with research on effective methods and systems that support broader implementation and sustainability (Gray et al., 2003).

Beyond SET and SNT, helping afterschool program directors and staff to use web-based implementation data to inform practice is an area of future research and development that might contribute to broader implementation and sustainability. With evolving approaches to technology, more attention is needed to ways to build systems to gather and use data, not only on implementation, but also on attendance, program quality, and impact upon student academic and behavioral outcomes. The development of data systems to evaluate and support quality practices is an important aspect that could contribute to the sustainability of promising practices (Cummins, 2013).

This work has suggested some implications for future practice as well. From SNT they include 1) starting with the “top-down” centralized communication to make initial contact, and then quickly moving to include full staff with a “bottom-up” diffuse communication strategy for the remainder of the project; 2) hold joint trainings including multiple ASPs to foster and promote “diffuse ties across networks” as well as “strong ties across networks” within programs; and 3) use of the coaches to strengthen the person-to-person “weak” ties. From SET we recommend 1) recognizing cost-benefit considerations by providing incentives; 2) being attentive to sociocultural values and practices; 3) maintaining frequent communication to build trust; and 4) providing ongoing coaching, support and appreciation. These are elements upon which the various stakeholders concur in effectively implementing evidence-based practices in afterschool.

Footnotes

1

Leading, Educating, Guiding, A Community of Youth Together

Reference List

  1. Barrish HH, Saunders M, Wolf MM. Good Behavior Game: Effects of Individual Contingencies for Group Consequences on Disruptive Behavior in a Classroom. Journal of Applied Behavioral Analysis. 1969;2(2):119–124. doi: 10.1901/jaba.1969.2-119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Battilana J, Casciaro T. Overcoming resistance to organizational change: Strong ties and affective cooptation. Management Science. 2013;59(4):819–836. http://dx.doi.org/10.1287/mnsc.1120.1583. [Google Scholar]
  3. Biglan A, Mrazek PJ, Carnine D, Flay BR. The integration of research and practice in the prevention of youth problem behaviors. American Psychologist. 2003;58(6–7):433–440. doi: 10.1037/0003-066x.58.6-7.433. doi: 10.1037/0003-066X.58.6-7.433. [DOI] [PubMed] [Google Scholar]
  4. Butterfoss FD, Goodman RM, Wandersman A. Community coalitions for prevention and health promotion. Health education research. 1993;8(3):315–330. doi: 10.1093/her/8.3.315. [DOI] [PubMed] [Google Scholar]
  5. Cook KS, Emerson RM. Power, Equity and Commitment in Exchange Networks. American Sociological Review. 1978;43(5):721–739. doi: http://www.jstor.org/stable/2094546. [Google Scholar]
  6. Cook KS, Emerson RM, Gillmore MR, Yamagishi T. The Distribution of Power in Exchange Networks: Theory and Experimental Results. American Journal of Sociology. 1983;89(2):275–305. Doi: http://www.jstor.org/stable/2779142. [Google Scholar]
  7. Daly AJ, Finnigan KS. The ebb and flow of social network ties between district leaders under high-stakes accountability. American Educational Research Journal. 2011;48(1):39–79. [Google Scholar]
  8. Dumas JE, Lynch AM, Laughlin JE, Smith EP, Prinz RJ. Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the EARLY ALLIANCE Prevention Trial. American Journal of Preventive Medicine. 2001;20:38–47. doi: 10.1016/s0749-3797(00)00272-5. [DOI] [PubMed] [Google Scholar]
  9. Durlak JA, Weissberg RP, Pachan M. A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Community Psychology. 2010;45(3–4):294–309. doi: 10.1007/s10464-010-9300-6. doi: http://dx.doi.org/10.1007/s10464-010-9300-6. [DOI] [PubMed] [Google Scholar]
  10. Embry DD, Straatemeier G, Richardson C, Lauger K, Mitich JE. The Pax Good Behavior Game. Center City: Hazelden; 2003. [Google Scholar]
  11. Emerson RM. Social exchange theory. Annual Review of Sociology. 1976;2(1):335–362. doi: [Google Scholar]
  12. Feinberg ME, Riggs NR, Greenberg MT. Social network and community prevention coalitions. Journal of Primary Prevention. 2005;26:279–298. doi: 10.1007/s10935-005-5390-4. [DOI] [PubMed] [Google Scholar]
  13. Flaspohler P, Duffy J, Wandersman A, Stillman L, Maras M. Unpacking prevention capacity: An intersection of research-to-practice models and community-centered models. American Journal of Community Psychology. 2008;41(3–4):182–196. doi: 10.1007/s10464-008-9162-3. [DOI] [PubMed] [Google Scholar]
  14. Flaspohler P, Lesesne C, Puddy R, Smith E, Wandersman A. Advances in Bridging Research and Practice: Introduction to the Second Special Issue on the Interactive System Framework for Dissemination and Implementation. American Journal of Community Psychology. 2012;50(3–4):271–281. doi: 10.1007/s10464-012-9545-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Granovetter MS. The strength of weak ties. American Journal of Sociology. 1973;78(6):1360–1380. [Google Scholar]
  16. Gray DO, Jakes SS, Emshoff J, Blakely C. ESID, dissemination, and community psychology: A case of partial implementation? American Journal of Community Psychology. 2003;32(3–4):359–370. doi: 10.1023/b:ajcp.0000004754.37080.57. [DOI] [PubMed] [Google Scholar]
  17. Halgunseth L, Carmack C, Childs S, Caldwell L, Craig A, Smith EP. Using the interactive systems framework in understanding the relation between general program capacity and implementation in after-school settings. American Journal of Community Psychology. 2012;50(3–4):311–320. doi: 10.1007/s10464-012-9500-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hudley C, Taylor A. What is cultural competence and how can it be incorporated into preventive interventions? In: Guerra N, Smith EP, editors. Preventing youth violence in a multicultural society. Washington, DC: American Psychological Association; 2006. pp. 249–269. [Google Scholar]
  19. Hynes Kathryn, Smith Emilie Phillips, Perkins Daniel. Piloting a classroom-based intervention in after-school programmes: a case study in science migration. Journal of Children’s Services. 2009;4(3):4–20. doi: 10.1108/17466660200900014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Ialongo N, Werthamer L, Kellam S, Brown CH, Wang S, Lin Y. The proximal impact of two first grade preventive interventions on the early risk behaviors for later substance abuse, depression and antisocial behavior. American Journal of Community Psychology. 1999;27(5):599–641. doi: 10.1023/A:1022137920532. [DOI] [PubMed] [Google Scholar]
  21. Kam CM, Greenberg MT, Walls CT. Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science. 2003;4(1):55–63. doi: 10.1023/a:1021786811186. [DOI] [PubMed] [Google Scholar]
  22. Kellam SG, Rebok GW, Ialongo N, Mayer LS. The course and malleability of aggressive behavior from early first grade into middle school: Results of a developmental epidemiologically-based preventive trial. Journal of Child Psychology and Psychiatry. 1994;35(2):259–281. doi: 10.1111/j.1469-7610.1994.tb01161.x. [DOI] [PubMed] [Google Scholar]
  23. Kellam SG, Reid J, Balster RL. Effects of a universal classroom behavior program in first and second grades on young adult problem outcomes. Drug Alcohol Depend. 2008;95(Suppl 1):S1–S4. doi: 10.1016/j.drugalcdep.2008.01.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Israel BA, Eng E, Schulz AJ, Parker EA. Introduction to methods in community-based participatory research for health. Methods in community-based participatory research for health. 2005:3–26. [Google Scholar]
  25. Johnson K, Hays C, Center H, Daley C. Building capacity and sustainable prevention innovations: A sustainability planning model. Evaluation and Program Planning. 2004;27(2):135–149. [Google Scholar]
  26. Larson R. Toward a psychology of positive youth development. American Psychologist. 2000;55:170–183. doi: 10.1037//0003-066x.55.1.170. [DOI] [PubMed] [Google Scholar]
  27. Lerner RM, Lerner JV, Almerigi J, Theokas C, Phelps E, Gestsdottir S, von Eye A. Positive youth development, participation in community youth development programs, and community contributions of fifth grade adolescents: Findings from the first wave of the 4-H Study of Positive Youth Development. Journal of Early Adolescence. 2005;25(1):17–71. [Google Scholar]
  28. Lewin K. Frontiers in group dynamics: Concept, method and reality in social science; Social equilibria and social change. Human Relations. 1947;1(1):5–41. [Google Scholar]
  29. Lieberman A. Networks as learning communities shaping the future of teacher development. Journal of teacher education. 2000;51(3):221–227. [Google Scholar]
  30. Lyon AR, Frazier SL, Mehta T, Atkins MS, Weisbach J. Easier said than done: Intervention sustainability in an urban after-school program. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(6):504–517. doi: 10.1007/s10488-011-0339-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Masquefa B. Top management adoption of a locally driven performance measurement and evaluation system: a social network perspective. Management Accounting Research. 2008;19(2):182–207. [Google Scholar]
  32. McLoyd VC, Smith J. Physical discipline and behavior problems in African American, European American, and Hispanic children: Emotional support as a moderator. Journal of Marriage and Family. 2002;64(1):40–53. [Google Scholar]
  33. Mihalic S, Irwin K, Elliott D, Fagan A, Hansen D. Blueprints for violence prevention. Washington, DC: US Department of Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention; 2001. pp. 1–15. [Google Scholar]
  34. Moolenaar NM. A Social Network Perspective on Teacher Collaboration in Schools: Theory, Methodology, and Applications. American Journal of Education. 2012;119(1):7–39. doi: http://www.jstor.org/stable/10.1086/667715. [Google Scholar]
  35. Moolenaar NM, Daly AJ. Social Networks in Education: Exploring the Social Side of the Reform Equation. American Journal of Education. 2012;119(1):1–6. www.jstor.org/stable/10.1086/667762. [Google Scholar]
  36. Neal JW, Neal ZP, Atkins MS, Henry DB, Frazier SL. Channels of change: Contrasting network mechanisms in the use of interventions. American Journal of Community Psychology. 2011;47(3–4):277–286. doi: 10.1007/s10464-010-9403-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Odgers CL, Moffitt TE, Tach LM, Sampson RJ, Taylor A, Matthews CL, Caspi A. The protective effects of neighborhood collective efficacy on British children growing up in deprivation: A developmental analysis. Developmental Psychology. 2009;45(4):942–957. doi: 10.1037/a0016162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Prinz RJ, Dumas JE, Smith EP, Laughlin JE. EARLY ALLIANCE Prevention Trial: A dual design to test reduction of risk for conduct problems, substance abuse, and school failure in childhood. Controlled Clinical Trials. 2000;21:286–302. doi: 10.1016/s0197-2456(00)00051-9. [DOI] [PubMed] [Google Scholar]
  39. Sabatier PA. Top-down and bottom-up approaches to implementation research: A critical analysis and suggested synthesis. Journal of Public Policy. 1986;6(1):21–48. [Google Scholar]
  40. Sampson RJ, Raudenbush SW, Earls F. Neighborhoods and violent crime: A multilevel study of collective efficacy. Science. 1997;277(5328):918–924. doi: 10.1126/science.277.5328.918. [DOI] [PubMed] [Google Scholar]
  41. Smith EP. The role of after-school settings in positive youth development. Journal of Adolescent Health. 2007;41(3):219–220. doi: 10.1016/j.jadohealth.2007.06.010. Doi: http://dx.doi.org/10.1016/j.amepre.2003.09.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Smith Emilie Phillips, Osgood D Wayne, Caldwell Linda C, Hynes Kathryn, Perkins Daniel F. Measuring Collective Efficacy Among Children in Community-based Afterschool: Pathways toward Prevention and Positive Youth Development. American Journal of Community Psychology. 2013;52:27–40. doi: 10.1007/s10464-013-9574-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Snyder HN, Sickmund M. Juvenile offenders and victims: 2006 national report: Office of Juvenile Justice and Delinquency Prevention. 810 Seventh Street NW, Washington, DC 20531. Tel: 202-307-5911: 2006. Web site: http://ojjdp.ncjrs.org/publications/index.html. [Google Scholar]
  44. Tebes JK, Feinn R, Vanderploeg JJ, Chinman MJ, Shepard J, Brabham T, Connell C. Impact of a positive youth development program in urban after-school settings on the prevention of adolescent substance use. Journal of Adolescent Health. 2007;41(3):239–247. doi: 10.1016/j.jadohealth.2007.02.016. doi: http://dx.doi.org/10.1016/j.jadohealth.2007.02.016. [DOI] [PubMed] [Google Scholar]
  45. Tingstrom DH, Sterling-Turner HE, Wilczynski SM. The Good Behavior Game: 1969–2002. Behavior Modification. 2006;30(2):225–253. doi: 10.1177/0145445503261165. [DOI] [PubMed] [Google Scholar]
  46. Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health promotion practice. 2006;7(3):312–323. doi: 10.1177/1524839906289376. [DOI] [PubMed] [Google Scholar]

RESOURCES