Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Jul 30.
Published in final edited form as: J Emot Behav Disord. 2014 Mar 6;22(2):83–94. doi: 10.1177/1063426614522692

Guided by Theory, Informed by Practice: Training and Support for the Good Behavior Game, a Classroom-based Behavior Management Strategy

Jeanne M Poduska 1, Anja Kurki 1
PMCID: PMC4520537  NIHMSID: NIHMS705849  PMID: 26236144

Abstract

Moving evidence-based practices for classroom behavior management into real-world settings is a high priority for education and public health. This paper describes the development and use of a model of training and support for the Good Behavior Game (GBG), one of the few preventive interventions shown to have positive outcomes for elementary school children lasting through to young adulthood, ages 19–21, including reductions in the use of drugs and alcohol, school-based mental health services, and suicide ideation and attempts. We first describe the conceptual framework guiding the development of the model of training and support. Data on implementation of the model, from an ongoing trial of GBG being conducted in partnership with the Houston Independent School District, are then presented. We end with a discussion of the lessons learned and the implications for the next stage of research and practice.

Keywords: Coaching, Prevention, Schools, Teachers, Good Behavior Game, Classroom management

Teachers rate support with classroom behavior management as a pressing need (Evertson & Weinstein, 2006; Reinke, Stormont, Herman, Puri, & Goel, 2011; Walter, Gouze, & Lim, 2006). Yet, teachers have limited tested tools to help children and youth master the behavioral skills they need to succeed as students and classroom management is not a priority of preservice teacher training. In fact, the National Council for Accreditation of Teacher Education (NCATE) does not require proof of proficiency in this area for teachers to be certified (NCATE, 2008). The good news is that a number of preventive interventions aimed at aggressive, disruptive behavior, including the Good Behavior Game (GBG; Barrish, Saunders, & Wolf, 1969), have shown both short-term and long-term positive impact in epidemiologically-based randomized field trials. The challenge is that the outcomes seen in highly controlled studies are often not replicated when programs are moved into general practice in schools (Dusenbury, Brannigan, Hansen, Walsh, & Falco, 2005; Ringwalt, Vincus, Ennet, Johnson, & Rohrbach, 2004; Elliott & Mihalic, 2004).

Moving programs into practice requires an understanding that the programmatic intervention, for example, GBG, and the support system, that is, training and support, for the intervention are independent, though interrelated, components of a whole (Chen, 1998; Greenberg, Domitrovich & Bumbarger, 2001; Klein & Sorra, 1996). Indeed, research has shown that regardless of specific content or delivery mode, programs require some form of professional development or delivery strategy to be implemented effectively (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005; Lochman, et al., 2009; Noell, Witt, Gilbertson, Ranier, & Freeland, 1997).

Good Behavior Game: The Programmatic Base

GBG is a team-based classroom behavior management strategy that helps children master the role of student and be successful at the key demands of the classroom, including paying attention and working well with others. Through GBG, children work together to create a positive learning environment for all students by monitoring their own behavior as well as that of their classmates. As a universal preventive intervention, the focus of GBG is on strengthening the classroom environment and socializing children to the role of student.

In GBG classrooms, teams of children win when they meet behavioral expectations by not exceeding a criterion level of classroom rule infractions. There are four core elements: classroom rules/behavioral expectations, team membership, monitoring behavior, and reinforcement. The teacher and students work together to define classroom rules and the corresponding behavioral expectations. After observing students for several weeks, the teacher divides students into several teams, heterogeneous with regard to behavior, learning, and sex. During the game, students work together with their teammates to maintain good behavior. When a child does not meet a behavioral expectation/breaks a classroom rule, the team receives a point. At the end of the game, each team that has four or fewer points receives a reward. Teams do not compete against one another; all teams can win. Over the course of the year, the duration of the game increases and GBG is played at different times throughout the day, during different activities and instructional subjects, and in a variety of venues. Reinforcers shift from being tangible to intangible and are delivered on an intermittent schedule. Teachers may change teams during the school year to maintain balance or to provide students the opportunity to work with a broad range of peers. Of importance to the education sector is that as a strategy rather than a curriculum, GBG does not compete for instructional time.

GBG, tested through two highly controlled epidemiologically defined randomized field trials conducted in partnership with the Baltimore City Public Schools, (Dolan et al., 1993; Ialongo et al., 1999) has demonstrated positive impacts from elementary school through young adulthood. At first and second grades, there were reductions in aggressive, disruptive behaviour and increased time on task (Brown, 1993; Dolan, et al., 1993; Ialongo et al., 1999). In later elementary school, there were reductions in rates of conduct disorder diagnoses in students (Brown et al., 2008; Ialongo et al., 2001). And there was less aggressive/disruptive behavior through middle school (Ialongo, Poduska, Wethamer & Kellam, 2001; Kellam, Ling, Merisca, Brown & Ialongo, 1998). Results to young adulthood, ages 19–21, include reductions in the rates of antisocial personality disorder; drug and alcohol abuse and dependence; tobacco use (Kellam et al., 2008); the use of school-based services for problems with emotions, behaviors, and drug and alcohol use (Poduska et al., 2008); violence (Petras et al., 2008); and suicidal ideation (Wilcox et al., 2008). Positive impact has been greatest for males who entered first grade exhibiting higher levels of aggressive, disruptive behavior.

Conceptualizing a Model for Training and Support

A third epidemiological randomized field trial conducted in Baltimore City Public Schools included an aim to design a model of training and support focused on developing a district’s capacity to implement and sustain GBG (Poduska et al., 2009). We drew upon several literatures as we developed the model for GBG training and support. Mindful of the fact that GBG and the corresponding support system are independent, interrelated components, we focused on both individual and organizational change to understand the fit between the model of training and support, the implementer, and the broader context (Baldridge & Burnham, 1975; Giacquinta, 1973; Glasgow, Lichtenstein, & Marcus, 2003). The work of Leavitt (1965) and others (Robertson, Roberts, & Porras, 1993; Owens, 2004) provided a framework for understanding schools and districts as organizations. We also considered the extensive literatures on stages of program implementation and multi-level contextual factors that impede or enhance programs being moved into practice keeping in mind that different factors might be salient at different stages (Adelman & Taylor, 2003; Berman & McLaughlin, 1975, 1978; Burns & Hoagwood, 2005; Domitrivich et al., 2008; Dusenbury, Brannigan, Falco, & Hansen 2003; Fixsen et el., 2005; Greenhalgh et al., 2005; McLaughlin, 1990; Rogers, 2003; Rohrbach, Grana, Sussman, & Valente, 2006). With these literatures in mind, we focused on three areas: 1) the model of training and on-going support for teachers and local GBG coaches and trainers; 2) consideration of multi-level contextual factors; and 3) the use of data to monitor program implementation and inform professional development.

Theoretical Base for Coaching

Two strands of training and support were developed: one for teachers and one for local coaches. We considered the training teachers received through the prior randomized field trials and viewed it alongside the literature on professional development and coaching with the aim of developing a model that was feasible, relevant, and replicable. The literature on professional development for educators highlights the importance of providing opportunities for active learning through observation, meaningful discussion, practice, and reflection (Garet, Porter, Desimone, Birman, & Yoon, 2001; Lieberman, 1996; Loucks-Horsley, Hewson, Love, & Stiles, 1998). Research also suggests that effective professional development is an ongoing process rather than a single event; aligned with and incorporated into one’s daily professional work; and of long enough duration to ensure a thorough and working knowledge of the core program principles and how they translate into practice (Hargreaves & Fullan, 1992; Leach & Conto, 1999; Little, 1993; McCormick, Steckler, & McLeroy, 1995; Rose & Church, 1998).

Central to our model of training and support is a coach who works directly with the teacher in the classroom. GBG coaches support teachers in bringing together the new knowledge of GBG learned in group-based trainings with their craft knowledge of classroom teaching through relationship building (mentoring and providing feedback), data-driven decision making (observing, analyzing and interpreting data), and supporting teachers in becoming self-reflective and self-directed implementers of GBG (American Institutes for Research, 2005; Spouse, 2001; Taylor, 2007). Han and Weiss’s (2005) conceptual model of the process of teacher implementation and sustainability of practices places the coach at the center of “a self-sustaining feedback loop.” The primary purpose of coaching is to support teachers’ implementation of a program; this support is provided through observation, direct feedback, and modeling in keeping with Bandura’s social cognitive theory (1986). The goal of the initial GBG trainings is to provide the necessary knowledge about the intervention; coaching provides the necessary support for teachers to become independent self-directed learners who can sustain the intervention in their classrooms after the active coaching support is phased out. The second function of coaching is to promote teachers’ motivation to implement the new practice (Han & Weiss, 2005). For example, a coach can help a teacher with low self-efficacy with regard to classroom behavior management attribute small incremental successes to GBG. The self-sustaining loop occurs when teachers reach a high enough level of success and mastery that students’ classroom behavior is attributed to the program, thereby motivating the teacher to use the program.

Current Study

In this mixed-methods paper we draw upon quantitative data and qualitative interviews to understand the dynamic between the model of training and on-going support for GBG and implementation quality. The data were collected as part of a randomized field trial of GBG being conducted in partnership with the Houston Independent School District (HISD: Poduska, Gomez, Capo, & Holmes, 2012). An important feature of the trial is that we are implementing GBG under conditions which are not tightly controlled. Rather than research staff serving as GBG coaches who support teachers in the classroom, we trained staff of a local professional development provider as GBG coaches. As such, this study offers an important opportunity to examine how the training and support model operates outside of a highly controlled research trial in a large urban school setting. In the methods section we present the details of the randomized field trail, the model of training and support, and the results. Lessons learned, areas for future research, and implications for practice are then discussed.

Method

Design

This paper draws upon data from an ongoing, federally-funded randomized field trial testing the impact of GBG under two conditions of professional development against a control condition. Children entering first grade in 18 schools were randomized across first grade classrooms/teachers, balancing on gender, academic achievement, and behavior using procedures we have employed in two previous trials (Ialongo et al, 1999; Poduska et al., 2009). First grade teachers entered the study at the beginning of the 2010–11 school year or at the beginning of the 2011–12 school year and were randomized to one of three interventions conditions: GBG Basic, GBG with Coach, or Standard (control). Language of instruction was used as a blocking factor since in some classrooms the language of instruction was Spanish and in others it was English. The total sample includes 73 classrooms. The GBG Basic condition provided teachers with three days of group-based training: a two-day initial training and a one-day booster session, described below. In GBG with Coach, the group-based activities were enhanced by the presence of a coach who worked in the classroom with the teacher over the course of one school year, described below. Group-based training was conducted in English; training materials were in English. Of note is the fact that GBG with Coach is the model of training and support the American Institutes for Research (AIR) uses when working with schools and districts to develop local capacity for GBG.

Participants

The participants for this paper were the 26 teachers randomized to the GBG with Coach condition who taught in the same classroom for the entire school year during their first year of GBG implementation, and four local GBG coaches. Twenty-three of the teachers were female. Three (14%) of the 22 teachers who completed a background survey were younger than 30 years old; 13 (59%) were between 30 and 50 years old. Four (18%) were older than 50 and two person did not respond to the question. Fifteen (68%) teachers had completed a bachelor’s degree, five (23%) had completed a master’s degree, and one had a doctoral degree. Six (27%) had five or fewer years teaching experience; four (18%) had 6–10 years; six (27%) had 11–15 years; and five (23%) greater than 15 years; one person did not respond.

In keeping with our model of developing local capacity to implement and sustain GBG, AIR contracted with a local purveyor of professional development who provided staff to be trained and serve as local GBG coaches. As it was a time of leadership transition in the district, school district personnel and the teachers’ union suggested that we work with the purveyor to help ensure that local capacity for GBG would be sustained (Poduska et al., 2012). We interviewed and trained four local GBG coaches, all female. Each coach worked with between five and nine teachers. Two were fluent in Spanish and worked with the teachers in the bilingual classrooms. All had experience as classroom teachers. Three had a master’s degree and one had a doctorate. Three had special education backgrounds; one had a background in early childhood.

Training and Support for Teachers and Coaches

Teachers participated in two days of group-based initial training on GBG that included sessions on GBG core elements, demonstrations of strategies and procedures, and practice in the delivery of GBG. A one day group-based booster session was held after the mid-year break. During the school year, the teacher was supported in the classroom by a coach. The coaching process is cyclical—each cycle includes four biweekly visits, each 90 minutes in length. There were three coaching cycles across the school year. During the first visit in a cycle, the GBG coach conducts a classroom observation using the Fidelity Checklist and the GBG Scoreboard (described below). Together, these measures provide data on the teacher’s practices and student behavior in the classroom. The teacher and coach then develop a Professional Development Plan for the cycle. During the next three visits in a cycle (visits 2–4), the coach supports the teacher in GBG practices through activities such as modeling, guided practice, and visiting other classrooms to observe GBG implementation. Activities vary to meet the needs of individual teachers (AIR, 2005; Hasbrouck & Denton, 2005).

The coaching model combined a pre-determined amount of coaching support with support differentiated by teacher need. In the first semester, cycle one, coaches were expected to visit the classroom for 90 minutes every other week to support teachers as they first set up the conditions for GBG and then infused GBG into the classroom routine and the instructional day. During the second semester, cycles two and three, support could be differentiated. Teachers who were rated as “high” on the fidelity checklist and whose students demonstrated positive behavior could receive a check-in visit once a month rather than a bi-weekly visit. Teachers who rated low on the fidelity checklist and needed additional support could receive a greater number of visits from the coaches.

Local GBG coaches were trained in situ by training teachers and participating in professional development activities with AIR trainers over the school year. They participated in the group-based GBG trainings described above. They received additional training on using data to support effective GBG implementation and inform professional development; designing individual professional development plans and providing ongoing support to teachers; working with adult learners; and providing professional development to teams of teachers, other school-level staff, and principals. AIR trainers conducted two implementation audits during which the trainer and coaches visited classrooms. Coaches participated in regularly scheduled phone calls with an AIR trainer with an agenda driven by the needs articulated by the coaches. Email support was provided to individual coaches and the group, as needed.

Measures

GBG practice and support

During the first visit of each coaching cycle, the GBG coach completed a GBG Fidelity Checklist which provides information about teacher practices in classroom behavior management. Behavioral indicators are rated for occurrence (yes/no) and level of quality (scale of 1 being needs significant improvement to 6 exemplary); a global rating of classroom behavior management is also made (α = .80). A companion rubric guides the quality ratings. Teachers use the GBG Score Record to record when the game was played (date and time, instructional subject), the number of minutes played, and the result for each team (the number of rule infractions). Teachers and coaches also used the score record to log information about “probes”—instances where the teacher keeps a tally of infractions but does not announce the game to the students. A probe provides information about the extent to which students are generalizing the skills. The coach and teacher develop a Professional Development Plan (PD Plan) for each cycle by the end of the second visit. The Progress Report is completed by the coach at visits 2–4 and records the date and time of the visit, the focus of coaching, and the activities undertaken.

Self-report of multi-level contextual factors

Teachers and coaches completed surveys in the spring on a set of multi-level contextual factors hypothesized to influence implementation. The Evidence-Based Practice Attitude Scale (EBPAS; Aarons, 2005) is a 15-item measure that assesses the attitudes of mental health professionals toward adopting evidence-based practices which we adapted for use with teachers. Responses are made using a Likert scale of 1 (not at all) to 6 (a very great extent): (α = 0.77 for the full scale). EBPAS measures the extent to which a provider would adopt a program based on its intuitive appeal, its use by a colleague or that it is a requirement of the job; their willingness to try new interventions; and whether they see evidence-based practices as useful. The Perception of GBG scale measures perceived ease of implementation; ease with which GBG fits into daily practices; fit of GBG with one’s teaching philosophy; and perceived effectiveness of GBG using a six-point Likert scale ranging from 1 (not at all) to 6 (very much). The coefficient alpha for teacher reports’ for the full measure is 0.92. The Coach-Teacher Alliance scale has 23 items that assess the quality of the working relationship between a coach and teacher (α = 0.92). The measure incorporates 10 items from a teacher-coach alliance scale developed by members of the National Behavior Rating Coordination Center (Wehby, Maggin, Partin, & Robertson, 2012). Corresponding versions are completed by the teacher and coach. Principal Support rated by teachers and GBG coaches on six items includes tangible resources and encouragement (α = 0.86).

Qualitative interviews

To further inform our understanding of the coaching process, the coaches were interviewed about the support they provided to teachers and the training and support they themselves received. Thirty minute interviews were conducted by the AIR research team, guided by a detailed protocol with open-ended questions and probes. Interviews were audio-taped and transcribed; NViVo was used for coding.

Results

Analytic Approach

The results are organized by type of data and mode of measurement. We first describe implementation to the model of GBG training and support using the measures listed above under GBG practice and support, followed by exploration of multi-level contextual factors. We then present data from the qualitative interviews with the coaches.

Dose of Training and Support

All teachers and coaches received the group-based Initial GBG training which was held each year in late October after baseline data collection for the parent study was completed. They also attended the mid-year, group-based Booster session. Each school year had three coaching cycles: one in the first semester and two in the second semester. Overall, teachers received less coaching support as the school year progressed. Coaches reported a total of 225 classroom visits: 90 visits occurred in cycle one, 74 visits in cycle two, and 61 visits in cycle three. Of the 26 teachers who stayed in the study the entire year, 12 (46 %) received a total of between 10 and 12 coach visits; 6 (23 %) 7 to 9 visits; and 8 (31 %) 4 to 6 visits. In cycle one, 18 of the 26 teachers (69.2 %) received the expected four visits, 2 (7.6%) three visits, and 6 (23.1%) two visits. In cycle two, 8 teachers (30.8 %) received four visits, 7 (26.9 %) three visits, 10 (38.4%) two visits, and 1 (4.5%) one visit. The trend of fewer visits continued in cycle three, 3 teachers (11.5%) received four visits, and 9 (34.6%), 8 (30.8%), and 6 (23.1%) received three, two, and one visits, respectively.

Dose and Proficiency of Practice

Consistent with our coaching model, the reduction in the number of coaching visits that occurred was associated with an overall increase in teachers’ proficiency with GBG over the course of the school year as measured by the coaches’ global rating of quality on the GBG Fidelity Checklist, see Figure 1. At the beginning of cycle one, 1 teacher out of 26 received a rating of five or six (the criterion for moving from a biweekly coaching support schedule to a monthly check-in by the coach). Eight (30.8 %) and 17 (65.4%) teachers met this criterion at the beginning of cycles two and three, respectively. To examine patterns of proficiency we dichotomized the global rating into “high” for ratings of five or six and “low” for ratings of one to four. Three patterns emerged. One teacher (3.8%) was rated high in cycle one and retained that rating across the year. The majority of teachers (16 teachers: 61.5%) started with a rating of low and upon reaching a high rating, maintained it. Within this group, half (8: 30.7 %) were rated low in cycle one and high in cycles two and three; half were rated low in cycles one and two and high in cycle three. Nine teachers (34.6%) were rated low in all three cycles.

Figure 1. Global Quality Rating of Teachers’ GBG Practice across the Three Coaching Cycles.

Figure 1

Open in a new tab

Note: Global Quality rating is on a scale from 1 (poor) – 6 (excellent)

Adherence to the Coaching Cycle

Despite the fact that the majority of teachers were gaining proficiency with GBG over the course of the school year, there was a discrepancy between the number of classroom visits we would have expected to occur based on teacher need and the number of visits reported by the coaches. In cycle one, it was expected that all teachers would receive four classroom visits: 18 teachers (69.2%) received four visits. In cycles two and three, coaches were expected to conduct four classroom visits unless the teacher scored a five or six/high on the fidelity checklist at the first visit in the cycle. At the beginning of cycle two, 7 out of 26 teachers received a rating of high on the implementation checklist. Of the remaining 19 teachers, only 9 (47.4%) received four visits by coach report. At the beginning of cycle three, 17 out of 26 teachers received a rating of high; coaches reported visiting 3 of the remaining 9 teachers (33.3%).

We reviewed the coaches’ paperwork to examine their adherence to the coaching cycle. At the first visit of a cycle the coach was expected to complete a GBG Fidelity Checklist; by visit two a PD Plan was to be developed; a Progress Report and a GBG Score Report recording a probe were to be completed for the second through fourth visits. As mentioned, probes are times when teachers “play” the game without announcing it to the students and are a way to see how well the students are internalizing new skills and generalizing them to a variety of situations and contexts. Overall, coaches were consistent in that they began each cycle by observing the classroom, completing a fidelity checklist and developing a PD Plan. One teacher in each of cycles one and two did not have a Fidelity Checklist on file; three teachers in cycle two and one teacher in cycle three did not have a PD plan on file. However, the number of progress reports on file for visits 2–4 of each cycle varied widely between coaches. The data (not shown due to the small number of coaches) showed that some coaches visited some teachers far less than would be expected or that they were underreporting their coaching visits. With regard to probes, the a priori expectation was that coaches would conduct a probe at each of visits 2–4 in a coaching cycle. Keeping in mind that the number of visits was already less than expected, GBG Score Records for probes were recorded for 46 (71.8%) of visits 2–4 that took place in cycle one; and for 39 (81.2%) and 21 (60.0%) of visits 2–4 that took place in cycles two and three, respectively.

Follow-up discussions with the coaches revealed some potential influencing factors. First, we discovered that there may have been some confusion around the Progress Report since the local purveyor had GBG coaches complete a form called a Progress Report for their internal recordkeeping. Second, all GBG coaches reported difficulties in scheduling visits with teachers as the school year progressed. Scheduling class visits was most challenging for coaches that were full-time employees with job responsibilities beyond GBG because their schedules were less flexible than their colleagues who were only GBG coaches or were part-time consultants.

Focus of Professional Development

The top four professional development needs identified in the PD Plans for each coaching cycle are presented in Table 1, followed by the number and percent of teachers identified as having each specific need. In cycle 1, the focus is on teachers’ implementation of GBG: posting class rules; teaching behavioral expectations; assigning tasks to team leaders; announcing winners; using check, comment, redirect; and conducting probes. There is a broader range of professional development foci in cycles two and three. This is in keeping with a greater use of intangible and delayed reinforcement and the game becoming more challenging by playing longer and with greater variation as to length of time and when it is played during the day. Of note are the two areas for professional development that were identified in all three cycles: using check, comment, redirect and conducting probes. In GBG, when a behavioral expectation is not met, the teacher gives the team a check on a wall poster, neutrally comments on the rule that has been broken, and provides verbal reinforcement that serves to redirect the students to the desired behavior. That the use of check, comment, redirect was identified as an on-going professional development need is a reflection of the fact that many teachers struggle to provide students with consistent, neutral feedback regarding behavior. Conducting probes was also identified across the school year as a professional development need for teachers by coaches despite that fact that coaches themselves were not conducting probes at the level expected. We discuss this finding in the discussion.

Table 1.

Focus of Professional Development by Cycle: Items Ranked within Cycle

Focus Rank of Item as Focus of Professional Development : Number (%) of Teachers Identified as having the Specific Need
Cycle 1 Cycle 2 Cycle 3
Posting class rules 1: 16 (64%) 4: 5 (20%)
Implementing the “check, comment, redirect” system when behavioral expectations not met 2: 15 (60%) 1: 9 (36%) 1: 10 (40%)
Assigning tasks to team leaders 3: 14 (56%)
Teaching behavioral expectations 4: 10 (40%) 4: 5 (20%)
Conducting probes 4: 10 (40%) 2: 7 (28%) 3: 7 (28%)
Announcing winners 4: 10 (40%)
Limiting verbal and nonverbal feedback to “check, comment, redirect” during game 2: 7 (28%) 2: 9 (36%)
Use encouraging praise as positive reinforcement 2: 7 (28%) 1: 10 (40%)
Maintaining records of the Game results 3: 6 (24%) 3: 7 (28%)
Regularly providing appropriate feedback on behavioral expectations 4: 5 (20%) 4: 6 (24%)
Applying behavioral expectations across all situations 3: 7 (28%)
Using positive reinforcers 4: 6 (24%)
Open in a new tab

Teacher Report of Multi-Level Contextual Factors

Overall, teachers’ perceptions of evidence-based practices and of GBG were favorable. Teachers’ mean score on the EBPAS measuring the attitudes of teachers toward adopting evidence-based practices was 4.50 (Range 3.83–5.30). The mean score of perceived effectiveness of GBG on students’ skill development was 5.51 (Range 4.38–6.0) and 5.57 (Range 4.07–6.0) for GBG attributes: ease of implementation, ease with which GBG fits into daily practices, and fit of GBG with one’s teaching philosophy. While teachers tended to rate their relationship with their coach favorably (M=5.82, Range=5.00–6.00), there was considerable variation in the coaches’ ratings (M=5.30, Range-2.50–6.00) of the teacher-coach relationship. Administrator support for GBG implementation was rated more favorably by teachers (M=5.76, Range 4.81–6.00) than by coaches (M=2.63, Range=1.06–5.44).

Qualitative Interviews with Coaches

In the qualitative interviews, coaches reported that the training and support for teachers and 90 minutes for a coaching visit was adequate. Coaches noted two main challenges. First, all four coaches reported that scheduling classroom visits was challenging, particularly in cycle three when spring achievement testing takes place. The coaches who were employed full-time by the third-party purveyor and had responsibilities other than being a GBG coach had more difficulty scheduling classroom visits than their colleagues who worked as part-time consultants and had greater flexibility with regard to their schedules. Coaches also reported that teachers were less motivated if their principal was not actively supportive of GBG, illustrated by showing knowledge about the game, inquiring about how the game was being integrated into classrooms, and expecting that the game was being played. Coaches identified their colleagues and the AIR team as their primary resources for problem solving.

Coaches offered suggestions for the model of training and support. With regard to teachers’ professional development, coaches noted that some teachers would have benefited from additional support for GBG implementation at the beginning of the school year. With regard to their own professional development coaches liked that their initial training provided a comprehensive overview of their work. They suggested that the scope and sequence of the coach training be more finely aligned with the GBG implementation calendar for the year. A specific suggestion was that the coach/trainer telephone conferences focus on specific topics at times when they are particularly relevant. They suggested that the coach calls with AIR trainers be guided by a sequence of in-depth guided discussions over the course of the year. Each GBG coach mentioned the same two things when asked about the most rewarding aspects of their work: seeing the students’ progress with regard to awareness, self-regulation, and socialization and watching teachers grow and improve as practitioners.

Discussion

For evidence-based programs to be used by teachers, they must be appealing and feasible. In this study, teachers reported favorable impressions of GBG with regard to implementation, fit into daily practice, and effectiveness on children’s skill development. Teachers also rated the working relationship with their coach quite highly.

The challenges to implementing the model of training and support are instructive in that they provide us opportunity to learn and improve. In the next section we frame challenges in relation to our three foci: 1) the model of training and on-going support for teachers and local GBG coaches; 2) consideration of multi-level contextual factors; and 3) the use of data to monitor program implementation and inform practice. We discuss how the lessons learned from each challenge can inform the next stage of work moving evidence-based programs, GBG specifically, into practice.

Challenges related to the Model of Training and Support

Challenge 1

We reported a discrepancy between the number of classroom visits recorded by GBG coaches using the Progress Report and the visits that would be expected based on teacher proficiency with GBG in the second semester.

Response 1: Develop Clear Expectations for Training and Staffing Supports

In response to this discrepancy we now focus conversations in the pre-implementation readiness stage more specifically on the functions of the coach and the lessons we have learned about the daily life of a coach in contrast to providing only a job description. We have considered possible models for staffing additional GBG coaches to ensure both skill level and flexibility of schedule with stakeholders in Houston.

Challenge 2

We learned several lessons from our review of the coaches’ log data and experiences. For example, we found that we set the expectations for probe administration (i.e., when the game is played by the teacher but is not announced to the students and the results are not displayed) too high. The purpose of the probe is to see how well students are generalizing the skills used in GBG, and thus it is an important tool for assessing application of the GBG skills by students.

Response 2: Reconsider Expectations after Discussion

Conversations with the GBG coaches in Houston and at other sites revealed that our a priori expectation with regard to probe administration was too high. During a 60-minute classroom observation, coaches can reasonably watch a game or conduct a probe; it is rare that both occur in the same visit except as a directed professional development strategy. For example, in a classroom where the students are struggling to generalize behavioral skills beyond times when the game is being played, the teacher might play a game and a probe during the same coach classroom visit to provide data for reflection and discussion. More important than determining a set numerical expectation is developing a process to ensure that the teacher and the coaches are using probes and other data about teacher practices and student outcomes to inform GBG implementation.

Challenge 3

The feedback of GBG coaches regarding their training was consistent with the feedback of coaches in other locales (e.g., Oxfordshire, UK; Nebraska; Washington D.C.) and led to a refinement of their support.

Response 3: Adjust Sequence and Structure of Trainer/Coach Support

The trainer/coach calls are now more highly specified. They begin with a review of GBG Scoreboard and Fidelity Checklist data, followed by discussion of specific topics aligned with the scope and sequence of GBG in the classroom. Discussion topics at the start of GBG implementation include: Aligning the game with students’ individualized educational plans; Team membership; and Integrating GBG with classroom and school behavioral management strategies. Other topics include: Using GBG Score Record and Fidelity Checklist data to inform practice; Moving from tangible to intangible reinforcers; and Strategies for generalizing the game into other venues. Calls also include time for problem-solving amongst the trainer/facilitator and coach peers based on coaches’ immediate concerns in working with teachers. Coaches can email the trainer in advance to suggest topics given their work with teachers.

Challenges related to Multi-Level Contextual Factors

Challenge 4

Coaches reported lack of support by the school principal and leadership as a challenge to implementation in both surveys and interviews. We report elsewhere on developing a base for GBG in Houston (Poduska et al., 2012). Worth noting here is that during the pre-implementation/readiness stage, the district was undergoing leadership change and re-structuring at multiple levels. In addition to a new superintendent, there were several changes within the top central office positions, and a restructuring of the support and supervisory structure for schools. There were also many lay-offs in the principal and teacher workforce. In short, the context in Houston is similar to the context of many large urban systems across the country.

Response 4: Develop a Broad as of Community Support

For teachers to implement programs in a systematic and systemic fashion requires a shift in how many program developers and purveyors, and education policy makers at the local, state, and federal levels think about professional development for teachers. Moving programs into practice will likely require more time for school and district staff and community members to participate in information-sharing and buy-in activities than is currently funded by most schools and districts or offered by most program developers.

Challenges related to Using Data to Monitor Implementation and Inform Practice

Challenge 5

The follow-up conversations the research team had with Houston-based GBG coaches about “missing data” led to a series of conversations about the use of data to inform practice with a larger group that included AIR trainers, and GBG coaches and stakeholders from Houston as well as from other locales. While factors such as scheduling difficulties are real impediments to implementation, it became clear that a main reason GBG Score Records and Progress Reports were not being completed is that the data was not seen as useful by coaches in their daily practice working with teachers. GBG is quite elegant in that the playing of the game or conducting a probe generates data about how well the class is doing behaviorally. The GBG Score Record provides the teacher data by team on the number of checks/rule infractions as well as data on which classroom rules have been broken. The GBG Score Record might show that while a team is winning the game consistently, the team continues to break a specific rule during games. Similarly, data collected during probe administrations may show that students are struggling to generalize new behavioral skills during specific times of the day or during specific situations, e.g. transitions.

Response 5: Train to Ensure that Data informs Teachers’ Practice

To strengthen the use of data, we have added activities to the training for teachers and coaches so there is ample opportunity to practice interpreting data, determining “next steps,” and developing solutions in response to what the data reveal. Coach classroom visits and meetings with teachers are structured around a data review process.

Limitations

This study had two limitations. The small sample size of coaches, n = 4 makes it difficult to generalize to a larger population from this study alone. We were able to confirm many of the challenges reported in this study through conversations with coaches in other locales, for example, the UK, Baltimore, Colorado, Nebraska, and with AIR GBG trainers. In addition, the fact that some coaches did not make or underreported the expected number of classroom visits means that the information in the progress reports was biased towards the coaches that did conduct classroom room visits regularly.

Future Research

There are several lines of research that can guide us towards the goal of moving evidence-based programs into general practice so that we see positive outcomes over time and programs being sustained. As illustrated in this paper, the collection of both qualitative and quantitative data provided insight for improving implementation and the coaching process. We are in the process of analyzing extensive data from qualitative interviews with the teachers as well as data from classroom observations of teacher practice and student outcomes. Taken together, these data will provide additional direction as we create training and support that is differentiated by and responsive to teachers’ needs and learning styles.

We are also exploring ways in which web-based technology can help improve the implementation and practice for GBG (Brown et al, 2013). While terms such as “data-driven decision making” are common parlance in educational setting, we found that teachers and coaches need training and support to use data to inform and improve their practice. We are developing data tools, easy-to-use reports, and training that will support teachers becoming self-directed users of data regarding student behaviors and their own management practices. Web-based training and support may serve as an effective mode to increase the reach of GBG and similar programs by being accessible to a greater number of teachers than face-to-face coaching models, particularly in rural areas and dense urban locals.

Implications for Practice and Conclusion

It is a critical time in the field of school-based prevention and classroom behavior management. Increasingly, decision-makers at local, state, and federal levels require that funds be used for evidence-based programs, programs found to be effective through testing in studies employing scientifically rigorous designs. To ensure that positive impacts result and programs are sustained, it is essential that the field has effective models of training and support.

Developing structures to support the systemic and systematic use of evidence-based programs in schools requires an understanding of the multi-level organizational system of schools and districts as well as the broader social and political context. To that end, it is critical that researchers, practitioners, and policy-makers work together to solve problems of practice. The field needs models of researcher/practitioner partnering that promote program design at all stages as a shared activity. Such partnerships can ensure that programs and strategies along with support systems are feasible, relevant, and replicable, and that schools are ready to adopt and scale up programs and strategies as they prove effective.

Acknowledgments

We thank our many collaborators in this work in the Houston Independent School District community including principals, teachers, children and parents, and data clerks; central office staff; and the School Board. In addition, colleagues at the Harris County Department of Education and Zeph Capo of the Houston Federation of Teachers have generously shared their experiences and good thinking with us. We acknowledge the invaluable contributions of AIR staff members, past and present, including Gail Chan, Megan Lebow, Judith Littman, Carla Ford, and Natalie Keegan who have served as GBG trainers, and Megan Lebow and Jeanette Moses who led the fieldwork for the parent study. We also appreciate the critical read and comments provided by Catherine Bradshaw, James Taylor, and anonymous reviewers. The following grants have supported this work: National Institute on Drug Abuse (R01D030452) and Institute of Education Sciences (R305A090446). The American Institutes for Research provides training and support for the Good Behavior Game.

References

  1. Aarons GA. Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child and Adolescent Psychiatric Clinics of North America. 2005;14(2):255–271. doi: 10.1016/j.chc.2004.04.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Adelman HS, Taylor L. On sustainability of project innovations as systemic change. Journal of Educational and Psychological Consultation. 2003;14(1):1–25. [Google Scholar]
  3. American Institutes for Research. Conceptual overview of the coaching component of the reading professional development impact study. Washington, DC: Author; 2005. [Google Scholar]
  4. Baldridge JV, Burnham RA. Organizational innovation: Individual, organizational, and environment impacts. Administrative Science Quarterly. 1975;20(2):165–176. [Google Scholar]
  5. Bandura A. Social Foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall; 1986. [Google Scholar]
  6. Barrish HH, Saunders M, Wolf MM. Good behavior game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavior Analysis. 1969;2:119–124. doi: 10.1901/jaba.1969.2-119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Berman P, McLaughlin MW. Federal programs supporting educational change, vol. 4: The findings in review. Santa Monica, CA: Rand Corp; 1975. [Google Scholar]
  8. Berman P, McLaughlin MW. Federal programs supporting educational change, vol. VIII: Implementing and sustaining innovations. Santa Monica, CA: Rand Corp; 1978. [Google Scholar]
  9. Brown CH, Mohr DC, Gallo C, Mader C, Palinkas LA, Wingood G, Prado G, Kellam SG, Pantin H, Poduska J, Gibbons RDH, McManus J, Ogihara M, Valente TW, Wulczyn F, Czaja S, Sutcliffe G, Villamar J, Jacobs C. A computational future for preventing HIV in minority communities: How advanced technology can improve implementation of effective programs. Journal of Acquired Immune Deficiency Syndromes. 2013;63 (Suppl 1):72–84. doi: 10.1097/QAI.0b013e31829372bd. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Burns BJ, Hoagwood K, editors. Evidence-based practices Part II: Effecting Change. Child and Adolescent Psychiatric Clinics of North America. 2005;14(2):xv–xvii. doi: 10.1016/j.chc.2004.11.001. [DOI] [PubMed] [Google Scholar]
  11. Chen H. Theory-driven evaluations. In: Walberg HJ, editor. Advances in educational productivity: Evaluation research for educational productivity. Vol. 7. London: JAI Press; 1998. pp. 15–34. [Google Scholar]
  12. Dolan LJ, Kellam SG, Brown CH, Werthamer-Larsson L, Rebok GW, Mayer LS, et al. The short-term impact of two classroom-based preventive interventions on aggressive and shy behaviors and poor achievement. Journal of Applied Developmental Psychology. 1993;14:317–345. [Google Scholar]
  13. Domitrovich CE, Bradshaw CP, Poduska J, Hoagwood K, Buckley J, Olin S, et al. Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion. 2008;1:6–28. doi: 10.1080/1754730x.2008.9715730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Dusenbury L, Branningan R, Falco M, Hansen WB. A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research. 2003;18(2):237–256. doi: 10.1093/her/18.2.237. [DOI] [PubMed] [Google Scholar]
  15. Dusenbury L, Brannigan R, Hansen WB, Walsh J, Falco M. Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research. 2005;20:308–313. doi: 10.1093/her/cyg134. [DOI] [PubMed] [Google Scholar]
  16. Elliott DS, Mihalic S. Issues in disseminating and replicating effective prevention programs. Prevention Science. 2004;5(1):47–52. doi: 10.1023/b:prev.0000013981.28071.52. [DOI] [PubMed] [Google Scholar]
  17. Evertson CM, Weinstein CS. Classroom management as a field of inquiry. In: Evertson C, Weinstein C, editors. The handbook of classroom management: Research, practice, & contemporary issues. Mahwah, NJ: Lawrence Erlbaum; 2006. pp. 3–15. [Google Scholar]
  18. Fixsen DL, Naoom SF, Blasé KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network; 2005. (FMHI Publication #231) [Google Scholar]
  19. Garet M, Porter A, Desimone L, Birman B, Yoon KS. What makes professional development effective? Results from a national sample of teachers. American Education Research Journal. 2001;38(4):915–945. [Google Scholar]
  20. Giacquinta JB. The process of organizational change in schools. In: Kerlinger FN, editor. Review of research in education. Oxford: F. E. Peacock; 1973. pp. 178–208. [Google Scholar]
  21. Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health. 2003;93(8):261–1267. doi: 10.2105/ajph.93.8.1261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Greenberg MT, Domitrovich C, Bumbarger B. The prevention of mental disorders in school-aged children: Current state of the field. Prevention & Treatment. 2001;4 [Google Scholar]
  23. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Diffusion of innovations in health service organizations: A systematic literature review. Oxford: Blackwell; 2005. [Google Scholar]
  24. Han SS, Weiss B. Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology. 2005;33:665–679. doi: 10.1007/s10802-005-7646-2. [DOI] [PubMed] [Google Scholar]
  25. Hargreaves A, Fullan MG. Understanding teacher development. London: Cassell; 1992. [Google Scholar]
  26. Hasbrouck J, Denton C. The reading coach: A how-to manual for success. Denver, CO: Sopris West; 2005. [Google Scholar]
  27. Ialongo NS, Werthamer L, Kellam SG, Brown CH, Wang S, Lin Y. Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology. 1999;27(5):599–641. doi: 10.1023/A:1022137920532. [DOI] [PubMed] [Google Scholar]
  28. Ialongo N, Poduska J, Werthamer L, Kellam S. The distal impact of two first grade preventive interventions on conduct problems and disorder in early adolescence. Journal of Emotional and Behavioral Disorders. 2001;9:146–160. [Google Scholar]
  29. Kellam SG, Rebok GW, Ialongo N, Mayer LS. The course and malleability of aggressive behavior from early first grade into middle school: Results of a developmental epidemiologically-based preventive trial. Journal of Child Psychology and Psychiatry. 1994;35:259–281. doi: 10.1111/j.1469-7610.1994.tb01161.x. [DOI] [PubMed] [Google Scholar]
  30. Kellam SG, Poduska JM, Ialongo NS, Wang W, Toyinbo P, Petras H, et al. Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence. 2008;95:S5–S28. doi: 10.1016/j.drugalcdep.2008.01.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Klein KJ, Sorra JS. The challenge of innovation implementation. Academy of Management Review. 1996;21(4):1055–1080. [Google Scholar]
  32. Leach DJ, Conto H. The additional effects of process and outcome feedback following brief in-service teacher training. Educational Psychology. 1999;19(4):441–462. [Google Scholar]
  33. Leavitt HJ. Applied organizational change in industry. In: March JG, editor. Handbook of organizations. New York: Rand McNally; 1965. pp. 1144–1170. [Google Scholar]
  34. Lieberman A, McLaughlin MW, Oberman I, editors. Teacher learning: New policies, new practices. New York: Teachers College Press; 1996. Practices that support teacher development: Transforming conceptions of professional learning; pp. 185–201. [Google Scholar]
  35. Little JW. Teachers’ professional development in a climate of educational reform. Educational Revaluation and Policy Analysis. 1993;15:129–151. [Google Scholar]
  36. Lochman JE, Boxmeyer C, Powell N, Qu L, Wells K, Windle M. Dissemination of the Coping Power program: importance of intensity of counselor training. Journal of Consulting and Clinical Psychology. 2009;77(3):397–409. doi: 10.1037/a0014514. [DOI] [PubMed] [Google Scholar]
  37. Loucks-Horsley S, Hewson PW, Love N, Stiles KE. Designing professional development for teachers of science and mathematics. Thousand Oaks, CA: Corwin Press, Inc; 1998. [Google Scholar]
  38. McCormick LK, Steckler AB, McLeroy KR. Diffusion of innovations in schools: A study of adoption and implementation of school-based tobacco prevention curricula. American Journal of Health Promotion. 1995;9(3):210–219. doi: 10.4278/0890-1171-9.3.210. [DOI] [PubMed] [Google Scholar]
  39. McLaughlin MW. The rand change agent study revisited: Macro perspectives and micro realities. Educational Researcher. 1990;19(9):11–16. [Google Scholar]
  40. National Council for Accreditation of Teacher Education. Professional standards for the accreditation of teacher preparation institutions. Washington, DC: Author; 2008. Retrieved from http://www.ncate.org/LinkClick.aspx?fileticket=nX43fwKc4Ak%3D&tabid=474. [Google Scholar]
  41. Noell GH, Witt JC, Gilbertson DN, Ranier DD, Freeland JT. Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly. 1997;12(1):77–88. [Google Scholar]
  42. Owens RG. Organizational behavior in education: Adaptive leadership and school reform. 8. Boston: Pearson Allyn and Bacon; 2004. [Google Scholar]
  43. Petras H, Kellam SG, Brown CH, Muthen BO, Ialongo NS, Poduska JM. Developmental epidemiological courses leading to antisocial personality disorder and violent and criminal behavior: Effects by young adulthood of a universal preventive intervention in first- and second-grade classrooms. Drug and Alcohol Dependence. 2008;95:S45–S59. doi: 10.1016/j.drugalcdep.2007.10.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Poduska J, Kellam S, Brown CH, Ford C, Windham A, Keegan N, et al. Study protocol for a group randomized controlled trial of a classroom-based intervention aimed at preventing early risk factors for drug abuse: Integrating effectiveness and implementation research. Implementation Science. 2009;4:56. doi: 10.1186/1748-5908-4-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Poduska JM, Kellam SG, Wang W, Brown CH, Ialongo NS, Toyinbo P. Impact of the good behavior game, a universal classroom-based behavior intervention, on young adult service use for problems with emotions, behavior, or drugs or alcohol. Drug and Alcohol Dependence. 2008;95:S29–S44. doi: 10.1016/j.drugalcdep.2007.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Poduska J, Gomez M, Capo Z, Holmes V. Developing a collaboration with the Houston Independent School District: Testing the generalizability of a partnership model. Administration and Policy in Mental Health and Mental Health Services Research. 2012;39:258–267. doi: 10.1007/s10488-011-0383-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Reinke WM, Stormont M, Herman KC, Puri R, Goel N. Supporting children’s mental health in schools: Teacher perceptions of needs, roles, and barriers. School Psychology Quarterly. 2011;26:1–13. [Google Scholar]
  48. Ringwalt CL, Vincus A, Ennett S, Johnson R, Rohrbach LA. Reasons for teacher’s adaptation of substance use prevention curricula in schools with non-White student populations. Prevention Science. 2004;5:61–67. doi: 10.1023/b:prev.0000013983.87069.a0. [DOI] [PubMed] [Google Scholar]
  49. Robertson PJ, Roberts DR, Porras JI. Dynamics of planned organizational change: Assessing empirical support for a theoretical model. Academic of Management Journal. 1993;36:619–634. [Google Scholar]
  50. Rogers EM. Diffusion of innovations. 2. New York: The Free Press; 2003. [Google Scholar]
  51. Rohrbach LA, Grana R, Sussman S, Valente TW. Type II translation: Transporting prevention interventions from research to real-world settings. Evaluation and the Health Profession. 2006;29:302–333. doi: 10.1177/0163278706290408. [DOI] [PubMed] [Google Scholar]
  52. Rose DJ, Church RJ. Learning to teach: The acquisition and maintenance of teaching skills. Journal of Behavioral Education. 1998;8(1):5–35. [Google Scholar]
  53. Spouse J. Bridging theory and practice in the supervisory relationship: A sociocultural perspective. Journal of Advanced Nursing. 2001;33(4):512–522. doi: 10.1046/j.1365-2648.2001.01683.x. [DOI] [PubMed] [Google Scholar]
  54. Taylor JE. Instructional coaching: the state of the art. In: Mangin MM, Stoelinga SR, editors. Effective Teacher Leadership: Using Research to Inform and Reform. New York: Teachers College Press; 2007. [Google Scholar]
  55. Walter HJ, Gouze K, Lim KG. Teachers’ beliefs about mental health needs in inner city elementary schools. Journal of the American Academy of Child and Adolescent Psychiatry. 2006;45:61–68. doi: 10.1097/01.chi.0000187243.17824.6c. [DOI] [PubMed] [Google Scholar]
  56. Wehby JH, Maggin DM, Moore Partin TC, Robertson R. The impact of working alliance, social validity, and teacher burnout on implementation fidelity of the Good Behavior Game. School Mental Health. 2012;4:22–33. [Google Scholar]
  57. Wilcox HC, Kellam SG, Brown CH, Poduska JM, Ialongo NS, Wang W, et al. The impact of two universal randomized first- and second-grade classroom interventions on young adult suicide ideation and attempts. Drug and Alcohol Dependence. 2008;95:S60–S73. doi: 10.1016/j.drugalcdep.2008.01.005. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES