Teacher Consultation and Coaching within Mental Health Practice: Classroom and Child Effects in Urban Elementary Schools

Abstract

Objective

To examine effects of a teacher consultation and coaching program delivered by school and community mental health professionals on change in observed classroom interactions and child functioning across one school year.

Method

Thirty-six classrooms within five urban elementary schools (87% Latino, 11% Black) were randomly assigned to intervention (training + consultation/coaching) and control (training only) conditions. Classroom and child outcomes (n = 364; 43% girls) were assessed in the fall and spring.

Results

Random effects regression models showed main effects of intervention on teacher-student relationship closeness, academic self-concept, and peer victimization. Results of multiple regression models showed levels of observed teacher emotional support in the fall moderated intervention impact on emotional support at the end of the school year.

Conclusions

Results suggest teacher consultation and coaching can be integrated within existing mental health activities in urban schools and impact classroom effectiveness and child adaptation across multiple domains.

Children in high poverty urban schools experience mental health difficulties and academic underachievement at elevated rates (Atkins, Hoagwood, Kutash, & Seidman, 2010; Whipple, Evans, Barry, & Maxwell, 2010). Underfunded schools struggle to meet student needs and teachers report difficulties motivating and managing students (Reinke, Stormont, Herman, Puri, & Goel, 2011). Observations in elementary classrooms reveal inconsistent access to effective classrooms in schools with high proportions of economically disadvantaged students (Pianta, Belsky, Houts, Morrison, & NICHD Early Child Care Research Network, 2007). Effective classrooms are defined by the presence of supportive teacher-student interactions (instructional, emotional, and behavioral) which together create a classroom environment conducive to academic and social-emotional development (Hamre & Pianta, 2010). When children with adjustment problems are in effective classrooms, their achievement has been shown to match that of their typically developing peers (Hamre & Pianta, 2005). Promoting effective teaching practices in elementary classrooms in low-income schools may thus benefit the adaptation of all students – those with and without behavioral difficulties.

Recent efforts to promote effective teaching through teacher consultation and classroom coaching have been promising. Preschool teachers in high poverty schools who received training, coaching, and consultation improved in their management of behavior and provision of emotional support; and, children increased their self-regulation skills and academic performance beyond the skills and performance of comparison children (Raver et al., 2008; Raver et al., 2011). Similarly, in elementary schools assigned to receive training and coaching as a part of a social-emotional and academic learning program (4Rs; Brown, Jones, LaRusso, & Aber, 2010), intervention teachers demonstrated greater instructional and emotional support compared to control group teachers.

The promise of these models is tempered by concerns about sustainability. Across studies, researchers employ external coaches and consultants as primary interventionists (e.g., Jones, Brown, & Aber, 2011; Raver et al., 2008). These interventionists are unfamiliar to teachers at the start of the study and depart when the study ends. Recent scholarship suggests the importance of considering issues of intervention feasibility and sustainability from the start (Aarons, Hurlburt, & Horwitz, 2011; Wandersman et al., 2008). Employing individuals already working in the school to deliver novel interventions may increase their sustainability (Atkins et al., 2008). In addition, to advance the learning and development of children with behavioral difficulties and their classmates, school mental health services may need to be re-conceptualized to include a focus on universal prevention (Nastasi, 2004) and effective classrooms (Cappella, Frazier, Atkins, Schoenwald, & Glisson, 2008).

Accordingly, we designed and evaluated a teacher consultation and coaching program – Bridging Mental Health and Education in Urban Schools (BRIDGE; Cappella, Jackson, Wagner, Hamre, & Soulé, 2011) – to increase effective classroom interactions and improve the functioning of children with behavioral challenges and their classmates. Embedded within the regular activities of school mental health professionals, BRIDGE represents a step toward the broader goal of enabling school and community mental health resources to support effective classrooms. Rather than focus only on the whole classroom or students with behavioral problems, BRIDGE applies a two-tiered approach to improving classroom interactions for all children (universal) and children with behavioral difficulties (targeted). A randomized trial of the initial effects of this program illuminates the extent to which school mental health staff can support teachers to increase effective teaching and promote the academic, behavioral, and social adaptation of children in urban poor schools.

Classroom Interactions in Urban Poor Schools

The primary mechanisms of psychosocial and academic development in elementary school are proximal interactions between teachers and children in classrooms (Baker, 2006; Hamre & Pianta, 2010). Proximal interactions are the behavioral and verbal exchanges that communicate warmth and respect (e.g., teacher provides comfort and assistance), positive and clear expectations (e.g., teacher sets up clear rules and reinforcement), and engaging learning opportunities (e.g., teacher and students engage in instructional feedback loops). These interactions indicate what teachers and children do with materials, space, and curriculum (Pianta, LaParo, & Hamre, 2008), and predict preschool children’s social skills (Mashburn et al., 2008), and elementary age children’s social and academic competence (Brock, Nishida, Chiong, Grimm, & Rimm-Kaufman, 2008).

Schools in low income communities provide less consistent access to effective classroom interactions than schools in middle income communities (Pianta et al., 2007). However, these interactions are amenable to change, as evidence by improved classroom interactions following a universal social-emotional and academic learning program in urban schools (Brown et al., 2010). Moreover, the least effective classrooms and most struggling students may benefit most from intervention (Hamre & Pianta, 2005; Howes et al., 2008).

Public Health Approach to Classroom Support: Universal and Targeted

Public health models suggest a tiered approach to intervention, with universal strategies to improve a setting followed by targeted or indicated strategies to support children with high needs in that setting (Nastasi, 2004). However, rigorous classroom-based intervention trials tend to focus on universal or targeted/indicated strategies. For example, the effects of universal programs (e.g., PATHS; Domitrovich, Cortes, & Greenberg, 2007) and strategies (e.g., Good Behavior Game; Embry, 2002) are typically examined in separate trials from targeted/indicated programs for children with behavior problems (e.g., self-monitoring; Mooney, Ryan, Uhing, Reid, & Epstein, 2005).

The simultaneous deployment of universal and targeted support in classrooms is warranted. Teachers report difficulties dealing with student disruptive behavior and cite misbehavior as a reason for leaving the profession (Ingersoll, 2001; Reinke et al., 2011). The mainstreaming of children with emotional and behavioral problems into regular education classrooms (Norwich, 2008) occurs with minimal teacher training to address student needs. Targeted interventions, such as daily report cards, self-monitoring, and token economies, enhance interactions with students with behavioral difficulties (Rathvon, 2008). In addition, improved interactions with specific students may allow teachers to create more effective classrooms for all students.

At the same time, effective classwide teaching practices may benefit students with disruptive behaviors. Aggressive behaviors and peer problems are lower in classrooms with better behavior management, teacher responsiveness, and social and instructional climate (e.g., Donohue, Perry, & Weinstein, 2003; Thomas, Bierman, Thomson, Powers, & CPPRG, 2008). Well-structured routines and behavior management are linked to greater self-control and less off-task behavior (Rimm-Kaufman, Curby, Grimm, Nathanson, & Brock, 2009). Supportive teacher-student interactions build more capable and motivated students (Hughes, Luo, Kwok, & Loyd, 2008), which may help all children – including those with behavioral difficulties – to identify as successful learners.

Consultation and Coaching within School Mental Health

Consultation and coaching models have been successful at enhancing classroom practices, including emotional climate (Brown et al., 2010), behavior management (Raver et al., 2008), and rules and routines (Rimm-Kaufman & Sawyer, 2004). Intervention developers suggest these models work because they are responsive to teacher needs and strengths, provide supportive and specific feedback about practices, and offer coaching in context (e.g., Downer et al., 2009; Li-Grining et al., 2010; Mashburn, Downer, Hamre, Justice, & Pianta, 2010). Yet the interventionists are typically external to the context, a feature of implementation that raises concerns about sustainability.

Given the importance of implementation to the outcomes and sustainability of an intervention (Durlak & DuPre, 2008; Fixsen, Naoom, Blase, Friedman, & Wallace 2005), it is critical to examine whether effective teacher consultation and coaching can be implemented with regular staff who are partnered with or employed by the school. Approximately 60% of school districts have linkages with community agencies to provide school mental health services (Foster, Rollefson, Doksum, Noonan, & Robinson, 2005). These agencies assign clinicians to work on-site at specific schools as a component of the schools’ ongoing student support services. Beyond community-employed providers, most schools have district-employed mental health staff to evaluate and treat children whose emotional and behavioral difficulties interfere with learning (Adelman & Taylor, 2006a).

Although the regular presence of mental health staff in schools increases child access to and engagement with services, these services minimally impact children’s school functioning (Weist et al., 1999). Moreover, mental health activities are often isolated from the structure and mission of schools (Atkins et al., 2010). Broadening the role of school and community mental health staff to include a focus on improving effective teaching may increase the relevance and impact of school mental health. Indeed, these professionals are uniquely positioned to support teachers to interact effectively with children, particularly those with behavioral difficulties. Yet, their training—focused on children and families—rarely provides the language and tools to do so (Schaeffer et al., 2005). The extent to which such professionals, indigenous to the school and community, can help teachers improve classroom interactions and child outcomes in urban schools is an empirical question the current study evaluates.

BRIDGE Evaluation: The Current Study

As a step toward shifting school mental health resources to focus on classrooms, BRIDGE is a consultation and coaching intervention embedded into the regular activities of mental health professionals in urban schools. Classroom teachers and mental health staff are guided by a standardized and validated tool for understanding effective classrooms: the Classroom Assessment Scoring System (CLASS; Pianta, La Paro, et al., 2008). Mental health professionals receive training and support to consult with teachers around observed classroom interactions and target student behaviors, and to coach teachers to use classwide and targeted strategies aligned with the CLASS. Like other consultation and coaching models, BRIDGE promotes responsiveness to teachers, specific and supportive feedback, and coaching in context. Unlike other models, BRIDGE integrates a focus on universal and targeted support within a validated observational framework, and embeds these components into a sustainable delivery system.

The current study evaluated in an experimental pre-post design the effect of BRIDGE on classroom practices and child adaptation across one academic year. We hypothesized main effects of BRIDGE on classroom emotional support (positive and negative climate, teacher sensitivity, regard for student perspectives) and organization (behavior management, productivity, instructional learning formats), and the moderation of these effects by lower baseline levels of support and organization. We hypothesized main effects of BRIDGE across domains of child adaptation: behaviors (increased behavioral regulation), teacher-student relationships (increased closeness and decreased conflict), academic self-concept (increased identification as a successful student), and peer relationships (decreased aggression and victimization). Finally, we evaluated whether there are differential intervention effects for a subsample of children with behavioral difficulties.

Method

Design

The study incorporated a 2 × 2 factorial design. Thirty-six classrooms were randomly assigned within schools to intervention and control conditions. Classroom and child outcomes were measured pre- and post-treatment, and implementation fidelity indicators were obtained at regular intervals during the intervention. We conducted intent-to-treat analyses using multiple regression and multi-level models to examine intervention effects on classrooms and children.

Setting

Five public urban elementary schools were invited to participate. The schools were linked to a partner community agency for school-based mental health services on the basis of location (proximal to the agency) and economic disadvantage (free/reduced lunch eligibility: 89-99%). According to district records, schools enrolled mainly Latino and Black students (87% and 11%), with four of the five schools serving mainly Latinos (89-99%) and one school serving mainly Black students (69%).

Participants

Participants were mental health staff (i.e., consultants), teachers, and children. Twelve consultants delivered BRIDGE across the five schools. The consultants were female (82%), and identified as Latino (33%), White (50%), Black (8%), and mixed/other (8%). Roles in the school included counselor, social worker, and psychologist.¹ Consultants were employed by the school district (n = 7) or the community agency (n = 5) providing services at the school.

Thirty-six teachers from regular education (n = 23) and special education/combined classes (n = 13) consented, including similar numbers in younger grade (k-2^nd: n = 16) and older grade (3-5^th: n = 20) classes. The teachers were female (81%) and identified as Latino (47%), White (36%), or Black (14%). On average, teachers worked in their current school for 10.17 (SD = 7.33) years.

The child sample included 364 K-5^th grade children. Children’s mean age was eight years old (SD = 1.99) with comparable rates of boys and girls (43% female). A child subsample (n = 159) of target and comparison children was selected for specific measures (see Procedures below and Figure 1).² Teachers reported child race/ethnicity as Latino (90%), Black (7%), White (1%), and mixed/other (1%), and indicated that nearly all were eligible for free/reduced lunch (99%). These proportions matched the overall demographic composition of the schools. The child subsample did not differ significantly from the whole sample in age (M = 8.05, SD = 2.06) or gender (41% female).

Figure 1. BRIDGE Randomized Pilot Study Participant Flowchart.

a. Because of the time limitations of school-based consultants, only the 44 teachers who expressed initial interest and need were approached and considered for participation.

b. All children in consented classrooms were eligible for participation.

Research Procedures

All study procedures were approved by the university and school district institutional review boards. Informed consent was obtained from consultants, teachers, and parents, with assent from children (see Figure 1 for the flow of participation).

Selection of consultants

Consultants were recruited during informational meetings in the agency and schools. Interested individuals met with researchers to determine whether their time and role in the school were sufficient to accommodate the consultation practice, as well as to discuss intervention implementation and study design. Across the five schools, 12 of 18 eligible mental health professionals consented to participate.

Selection and randomization of teachers

A letter of invitation to participate in research was distributed to all classroom teachers (n = 154). Given consultant resources, a maximum of 48 teachers could be accommodated. Researchers and school designates met with teachers who expressed interest in the research within two weeks of receiving the invitation (n = 44). Of these, 36 consented to participate. Those who did not consent indicated concerns with time to complete research measures and lack of perceived need for classroom support. After Time 1 data were collected, researchers used a random numbers table to randomize teachers within schools to intervention and control conditions.

Selection of children

All children in consented teachers’ classrooms were eligible to participate. Researchers introduced the study to children through oral presentations; parental consent forms in English and Spanish were sent home to the families of 828 children. Consent forms were returned by 370 students: six declined to participate and 364 provided active consent. All consented children gave oral (K-1^st grade) or written (2-5^th grade) assent.

Among consented students, approximately four children per classroom were selected as a subsample for more extensive data collection (n = 159). The subsample included two children per classroom whom teachers identified with behavioral challenges (target) and two children per classroom randomly chosen among remaining consented classmates (comparison). At the time of fall data collection, the majority of target children (80%) were awaiting – or receiving – school-based services (e.g., evaluation, counseling, case management) for behavioral problems.³ On average, target students scored significantly higher than comparison students – and above the clinically elevated level – on teacher-reported behavioral difficulties scales prior to the intervention.

Data collection

Data were gathered from teachers, children, consultants, and observers. Time 1 data were collected in the fall and Time 2 in the late spring. Researchers were provided group training on all procedures and measures prior to each of the data collection periods.

Classroom observations

Classrooms were observed by a single coder blind to intervention condition. Observers were trained following standard procedures (Pianta, La Paro, et al., 2008): (a) two-day training with a certified trainer, (b) scoring within one point of “gold-standard” codes (scored by CLASS developers) on 80 percent of CLASS dimensions across four videos, (c) completion of a live observation with a master coder. A mid-year reliability test was conducted. For all observers, initial and mid-year reliability codings yielded overall inter-rater reliabilities of .90. Inter-rater reliability for the classroom practices subscales was .94 for Emotional Support (.83 - 1.00 for each dimension) and .96 for Classroom Organization (.95 - 1.00 for each dimension).

All observations took place during the first two hours of the school day. Observers sat in a location suggested by the teacher for an unobstructed view with minimal impact on classroom activities. Researchers observed for fifteen minutes and recorded scores for each of the CLASS dimensions in the final ten minutes. This procedure was repeated four times across the morning for approximately two hours of observations and coding.

Questionnaires and interviews

Teachers and consultants completed brief questionnaires about their demographic and educational backgrounds. Teachers filled out additional information about the child subsample in their classrooms. Surveys were administered primarily in paper and pencil format with a secured online option. At Time 1, 13 (of 36) teachers completed online surveys, whereas at Time 2, only one teacher chose to complete online surveys. Comparisons of scores on T1 measures revealed no significant differences by mode of completion.

All 2^nd to 5^th grade children completed paper and pencil surveys in classrooms or small groups during non-academic times suggested by teachers. Several researchers were present to read items aloud and answer questions. Students who did not have consent were provided word puzzles. Kindergarten and first grade students were interviewed individually outside the classroom. Two students identified as English language learners were interviewed by a Spanish-speaking researcher.

Following school district research board guidelines, schools received monetary gifts for participation and children were provided prizes worth less than $1. The community agency received a training stipend and mental health providers received gift cards of a modest monetary value to acknowledge their participation.

Intervention Procedures

BRIDGE intervention

BRIDGE was based in research on the importance of classroom interactions to children’s adaptation, the need for universal and targeted classroom support, and the effectiveness of consultation and coaching in urban schools. Two programs in mental health and education were integrated for BRIDGE: Links to Learning (L2L; Atkins et al., 2006; Atkins et al., 2008) and MyTeachingPartner (MTP; Pianta, Mashburn, et al., 2008). The former is a mental health model that aligns indigenous school and community resources around learning goals for children with disruptive behavior problems and includes universal and targeted classroom strategies (Cappella et al., 2008). The latter is a teacher support model that offers individualized, web-based consultation to pre-school teachers using video of the teachers’ classrooms and the CLASS as a common lens through which to understand effective interactions (Mashburn et al., 2010). Researchers and community members conducted participatory research to adapt these programs into a feasible and relevant intervention for urban schools and mental health (see Cappella et al., 2011).

BRIDGE goals are to promote effective emotional support and classroom organization, supportive teacher-student relationships, and children’s social, behavioral, and academic adjustment through a focus on: (a) live observation of classroom interactions and target student behaviors using the CLASS lens (Pianta, La Paro, et al., 2008), (b) individualized teacher consultation based on these observations (Pianta, Mashburn, et al., 2008), (c) supportive coaching to implement classwide and targeted strategies aligned with the CLASS, and (d) delivery of program components by mental health professionals indigenous to the school and community (Atkins et al., 2008).

Initial implementation

Consultants attended three half-day trainings at the agency or school. They received access to the CLASS website and interactive training on classroom observation, teacher consultation, classroom strategies, and coaching. They also received copies of the BRIDGE manual, with detailed information and concrete materials. The classroom strategies within the manual were derived from L2L (e.g., Good Behavior Game; Embry, 2002) and MTP (e.g., Random Positive Attention; Pianta, Mashburn, et al., 2008) as well as from an extensive review of evidence-based practices with demonstrated impacts aligned with the CLASS (e.g., Positive Peer Reporting; Skinner, Cashwell, & Skinner, 2000).

Teachers attended a two-hour workshop after school or during professional development time, and received access to the CLASS website. The website contained explanation of the CLASS dimensions of effective teacher-student interactions as well as accompanying video clips of K-5^th grade teachers using practices that depict effective interactions. The teacher workshop focused on reflective teaching, and the CLASS domains of emotional support and classroom organization. The workshops were derived from MTP and L2L and adapted in consultation with community clinicians and school staff (Cappella et al., 2011).

Consistent with the goal of sustainability, BRIDGE was delivered by school-based mental health staff. Teachers were paired with consultants through a resource mapping procedure aimed to match a teacher with a professional who currently served students in the teacher’s classroom, or had previously worked with the teacher. To obtain this information, researchers interviewed school administrators and other staff (Adelman & Taylor, 2006b). Consultants agreed to provide intervention to one to three classrooms depending upon their responsibilities and time in the school.

BRIDGE intervention cycle

Mental health consultants implemented the BRIDGE intervention with teachers in their schools from January to April. In January, consultants met with teachers to assess classroom strengths and needs, discuss target student behaviors, and build the consultation relationship (Initial Interview). Consultation meetings (3-5 meetings; 30 minutes each) and classroom observation/coaching (3-5 sessions; 30 minutes each) were scheduled. One CLASS dimension within the domain of Emotional Support (Positive Climate, Teacher Sensitivity, or Regard for Student Perspectives) or Classroom Organization (Behavior Management, Productivity, or Instructional Learning Formats) was chosen by the teacher-consultant pair as the focus of the first cycle. Then, consultants used the CLASS lens to observe teacher-student interactions and target student behaviors and recorded notes using an observation form (Initial Observation).

In February, the intervention cycles began and included the following steps: Preparation, Consultation, and Coaching/Observation. During preparation, consultants highlighted the effective and challenging teacher-student interactions they observed that fell within the focal CLASS dimension, and reviewed the menu of targeted and classwide strategies aligned with that dimension. During consultation, pairs met to discuss the observation, review relevant video from the CLASS website, and chose one or more strategies to implement to increase the effectiveness of interactions with the target student(s) and across the classroom. During coaching/observation, consultants modeled the strategy implementation for teachers and/or observed as teachers implemented the strategy. This cycle was designed to be repeated three to five times across the intervention period.

Supervision support to individual consultants was scheduled in monthly meetings with the university implementation team (4 meetings of 60 minutes each). Supervision occurred primarily in schools so that supervisors could observe and support the consultants in classrooms as needed. Content included: (a) discussion of observations, consultation, and coaching, (b) generation of solutions to implementation barriers, and (c) demonstration and modeling of intervention components.

Control intervention

Teachers randomly assigned to the control intervention attended the initial training on reflective teaching and dimensions of effective classroom interactions. Teachers also received access to the CLASS website with information and video on effective practices.

Measures

Intervention dosage was measured at regular intervals during the intervention period as described below. Intervention effect was assessed on classroom practices, teacher-student relationships, and child adaptation across three domains – social, academic, and behavioral.

Intervention dosage

Intervention dosage was assessed with four measures. Following the trainings, consented teachers (n = 36) and consultants (n = 12) completed questionnaires indicating their interest in and perceived knowledge of the material (5-point scale; 10-16 items). Intervention teachers (n = 18) and consultants (n = 12) completed checklists after each consultation meeting to indicate the BRIDGE content addressed (7 items). To track the specific strategies implemented across the intervention period, teachers completed monthly checklists indicating the strategies they implemented that month. Supervisors (n = 3) completed monthly ratings of the supervision support to consultants. Detailed description of dosage is presented in Cappella et al., 2011.

Classroom practices

A classroom observation measure – Classroom Assessment Scoring System (CLASS: Pianta, La Paro, et al., 2008) – was used to assess classroom practices. This is the same tool that guides BRIDGE intervention with teachers. Ten dimensions are scored on a seven-point scale ranging from 1 or 2 (low) to 6 or 7 (high). Each dimension contains a detailed overall description, behaviorally-anchored scale points, and behavioral indicators (see Mashburn et al., 2008). Each dimension was coded four times per teacher during one observational period. The dimension scores factor into three domains: Emotional Support, Classroom Organization, and Instructional Support (Pianta, La Paro, et al., 2008). High scores in these domains indicate effective teacher-student interactions and predict children’s academic and psychosocial development (Hamre, Pianta, Downer, & Mashburn, 2008). Analyses focused on the domains that guided the BRIDGE intervention: Emotional Support (Positive Climate, Negative Climate – reverse, Teacher Sensitivity, and Regard for Student Perspectives) and Classroom Organization (Behavior Management, Productivity, and Instructional Learning Formats). Current study internal reliabilities were α = .79 (T1) and α = .87 (T2) (Emotional Support) and α = .86 (T1 and T2; Classroom Organization).

Behavioral regulation

The Behavioral Regulation Index (BRI) of the Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 1996) was used to measure teachers’ perspectives on children’s behaviors. The BRIEF is a standardized scale allowing for parent and teacher report of executive behaviors among 5-18 year olds. The teacher report includes 29 items about Inhibition, Shift, and Emotional Control, in which the teacher indicates how often a child exhibits problems with a specific behavior (3-point scale: never to often). Recent studies suggest measures of executive behaviors are sensitive to underlying behavioral difficulties in early middle childhood that may not be detected using standard measures (e.g., CBCL; see Riggs, Blair, & Greenberg, 2004). Correlations between teacher and parent forms are high (r = .87; Gioia, Isquith, Guy, & Kenworthy, 2000), and current study reliability was strong (Cronbach α = .97 - .98).

Teacher-student relationships

Children’s relationships with their teachers were measured on the child subsample (n = 154) with the Student-Teacher Relationship Scale (STRS; Pianta, 2001). The 15-item, 5-point scale has two subscales: Closeness (7 items, e.g., “I share an affectionate, warm relationship with this child.”) and Conflict (8 items, e.g., “This child and I always seem to be struggling with each other.”). Teacher reports on these scales correlate with concurrent and future academic outcomes, school retention, and disciplinary infractions (Hamre & Pianta, 2001). Current study reliability was adequate at each time point (Closeness α = .78 - .82; Conflict α = .90 - .92).

Child adaptation

Second to fifth graders answered questions about their academic self-concept on the Self Perception Profile for Children (Harter, 1985). The SPPC is a common measure of self-esteem, with satisfactory internal and test-retest reliability (Muris, Meesters, & Fijen, 2003). The academic self-concept subscale consists of six pairs of statements (e.g., “Some kids do well at their school work BUT other kids don’t do well at their work.”). The child selects the statement that is “most like” him/herself and indicates if that statement is “really true” or “sort of true” for him/her (higher scores indicate higher self-concept). Current study alphas were .57 (T1) and .65 (T2).

The Social Behavior and Experience Questionnaire, adapted from Crick and Grotpeter (1995), was used to assess aggressive behavior and victimization. Second to fifth grade children circled the names of as many or as few classmates as fit behavioral descriptors. Aggressive behavior was assessed with six items (hits others; calls mean names; spreads rumors; will beat up classmates; tells mean lies; keeps out classmates); victimization was assessed with three items (left out; hit, pushed, or beat up; object of rumors). The scale scores were determined by summing the total nominations each participant received divided by the possible nominations. Standardized scores were calculated based on the classroom mean level of the subscale following standard procedures (Coie & Dodge, 1983). Internal reliability was strong (aggression α = .91 - .92; victimization α = .80 - .85).

Data Analyses

To examine the impact of BRIDGE on classrooms and children, a series of analyses were conducted at classroom and child levels. Prior to testing hypotheses, missing data were identified and classroom-level missing values were imputed.

Missing data analyses

For the classroom-level variables, no data were missing at T1 and one classroom was missing data on classroom observations at T2. Because of the need for a complete classroom-level dataset, missing data for this classroom were imputed using the EM algorithm (Dempster, Laird, & Rubin, 1977), a single-imputation procedure (SAS PROC MI). For the child-level variables, there was 0-10% missing data across study variables. Child-level missing data were not imputed based on conventions that missingness at less than ten percent is reasonable when restricted maximum likelihood estimation is used in multi-level models (Holland, 2006).

Classroom-level analyses

The first set of analyses focused on changes in classroom practices at the end of the intervention year controlling for baseline levels. In multiple regression models, each T2 classroom practice score from the relevant domain of the CLASS was modeled as a function of intervention condition and classroom covariates. Main effect analyses included the following classroom covariates: (a) T1 CLASS domain score (Emotional Support or Classroom Organization), (b) grade level, (c) class size, (d) classroom type, and (e) teaching experience. The specific main effects model fit was:

$$\begin{array}{l}{\mathit{\left(}T\mathit{2}\mathit{CLASS}\mathit{\right)}}_{\mathrm{j}}=b_{\mathit{0}}+b_{\mathit{1}}\mathit{(}\mathit{Intervention\; Condition}\mathit{)}+b_{\mathit{2}}\mathit{(}T\mathit{1}\mathit{CLASS}\mathit{)}+b_{\mathit{2}}\mathit{\left(}\mathit{Class\; Size}\mathit{\right)}\\ +b_{\mathit{3}}\mathit{\left(}\mathit{Classroom\; type}\mathit{\right)}+b_{\mathit{4}}\mathit{\left(}\mathit{Teaching\; Experience}\mathit{\right)}+{\epsilon }_j\end{array}$$

We evaluated variation in BRIDGE impact on classroom practices as a function of pre-existing classroom characteristics. Regression models evaluating moderation included the interaction between CLASS domain scores (centered on their T1 mean) and intervention condition.

Child-level analyses

The second set of analyses focused on changes in child adaptation as predicted by intervention condition. Random effects regression models were specified using SAS PROC MIXED, which is suitable for fitting multi-level models, including hierarchical linear models (Singer, 1998).⁴ Two-level models accounted for the hierarchical structure of the data: children within classrooms. Preliminary analyses with three-level models found no evidence of school variation when classroom was included and no residual between-school differences on any outcomes. Also, no significant associations were found for teacher demographics on any child-level outcome.

In each random regression model, T1 covariates were used to control for child-level (Level 1) covariates: baseline levels of the outcome and gender. All models also included classroom-level (Level 2) covariates: class size, grade, years teaching, and classroom type. Outcome variables and corresponding T1 variables were centered on their T1 mean. The behavioral regulation model was performed without four extreme outliers and with inverse transformed scores at both T1 and T2 due to skewness. Below is an example of the mixed model equation predicting T2 behavioral regulation:

$$\begin{array}{l}{\mathit{\left(}T\mathit{2}\mathit{Behavioral\; Regulation}\mathit{\right)}}_{\mathit{ij}}=b_{\mathit{o}}+b_{\mathit{1}}{\mathit{\left(}\mathit{Intervention\; Condition}\mathit{\right)}}_j+b_{\mathit{2}}{\mathit{\left(}\mathit{Class\; Size}\mathit{\right)}}_j\\ +b_{\mathit{3}}{\mathit{\left(}\mathit{Grade\; Level}\mathit{\right)}}_j+b_{\mathit{4}}{\mathit{\left(}\mathit{Classroom\; Type}\mathit{\right)}}_j+b_{\mathit{5}}{\mathit{\left(}\mathit{Child\; Gender}\mathit{\right)}}_{\mathit{ij}}\\ +b_{\mathit{6}}{\mathit{\left(}T\mathit{1}\mathit{Behavioral\; Regulation}\mathit{\right)}}_{\mathit{ij}}+{\mathrm{\eta }}_{\mathrm{j}k}+{\epsilon }_{\mathit{jk}}\end{array}$$

where η_jk represents the variance of classroom intercepts and ε_jk the residual variance.

Lastly, exploratory analyses evaluated the effects of BRIDGE on the child subsample of target children with behavior problems and comparison peers (on average, two target and two comparison children per classroom). Mixed effects regression models evaluating moderation of effects included an indicator variable of target status, the interaction term, and covariates from main effect analyses.

Proportion of variance and effect size calculations

For all significant findings, proportion of variance and effect sizes were calculated. Within-classroom proportion of variance – pseudo R² – was computed for multi-level models by comparing the variance components of the null model with those of the full models (Singer & Willett, 2003). Effect sizes were calculated following procedures by Feingold (2009) for pretest-posttest experimental trials with mixed effects regression models. Specifically, the model-based estimates of mean differences at T2 were divided by the within-group standard deviation of the raw scores of the outcome measure at T1, thereby yielding effect sizes in the same metric as classical designs, and facilitating comparisons across studies.

Results

This section begins with preliminary analyses to evaluate the baseline equivalence of randomly assigned groups, and change in the control group across the school year (i.e., normative change). Then, intervention dosage data are presented. Finally, we present results of the multiple regression and random effects regression models evaluating the impact of BRIDGE on classrooms and children, and whether child or classroom covariates significantly predict an outcome.

Preliminary Analyses

Time 1 intervention versus control classrooms

Chi-square statistics and independent sample t-tests showed no significant differences between groups for any study variable at T1. Intervention and control classrooms were similar in type, size, grade, teacher characteristics, and teaching practices. Children across groups were similar in gender and race, and there were no significant differences in any child-level variable.

Normative change in control classrooms

Paired sample t-tests indicated classroom practices did not significantly change over time in the control group. Emotional support (T1: M = 4.44, SD = .80; T2: M = 4.29, SD = 1.04) and classroom organization (T1: M = 4.36, SD = .83; T2: M = 4.48, SD = 1.03) remained stable. Significant change over time in control group children was found for only one outcome: teacher-student relationship closeness declined (t = -3.69(67), p < .001).

Intervention Dosage

BRIDGE training

All consented teachers and mental health consultants attended the BRIDGE trainings during the scheduled time or a planned make-up session. Mean ratings of teacher and consultant perceptions of content knowledge after the trainings were high, with item means ranging from 3.67 (SD = .49) to 4.82 (SD = .40) on a 5-point scale.

BRIDGE delivery

All teachers assigned to the intervention received an initial interview with their BRIDGE consultant. Teachers received an average of 4.47 (SD = 2.67) observation and coaching sessions, and 3.50 (SD = 2.22) consultation meetings at a duration of 25 to 30 minutes each. Teachers implemented an average of 5.50 (SD = 3.99) strategies over the intervention period, including a combination of targeted and classwide strategies to impact emotional support and classroom organization. Taken together, more than a third of the classrooms (n = 7) received 100% of planned intervention, and nearly all (n = 16) received more than 50% of planned intervention. There were no differences in delivery by classroom type, grade level, or consultant employer.

BRIDGE supervision

BRIDGE consultants received an average of five hours of BRIDGE supervision over the intervention period. The majority of the supervision meetings (69%) occurred at the school, and focused on implementation of classroom strategies.

Intervention Effect on Classrooms

Intent-to-treat analyses were conducted to examine whether the BRIDGE intervention had a positive effect on classroom practices. As indicated in Table 1, the BRIDGE main effect for emotional support was not significant; instead, its effect was moderated by emotional support at T1 (b = -1.26, p < .05, effect size [ES] = 1.80). This suggests that classrooms with lower emotional support before the intervention benefited more from assignment to the intervention than classrooms with higher T1 emotional support. BRIDGE classrooms low in emotional support at T1 did not improve to the levels of classrooms high in emotional support at T1 (see Figure 2). However, for classrooms that scored 1 point lower than the average in emotional support at T1, intervention classrooms scored 1.26 points higher than comparable control classrooms at T2. No main (b = -.07, p = .83) or moderated effect (b = -.08, p = .09) of intervention was found for classroom organization.

Table 1.

Classroom Emotional Support Model Coefficients

	Emotional Support		Emotional Support
	Main Effect Model (n = 36)		Moderation Model (n = 36)
Parameters	b	SE	b	SE
Intercept	-0.39	.98	-0.44	0.91
Intervention a	-0.13	0.34	-0.11	0.31
Class Size	0.03	0.03	0.04	0.03
Grade Level	-0.10	0.10	-0.14	0.10
Special Education/Combined b	-0.19	0.40	-0.15	0.37
Years Teaching	0.00	0.03	-0.02	0.03
T1 Emotional Support	0.75 **	0.30	1.05 **	0.31
Intervention X T1 Emotional Support			-1.26 *	0.54
R2		.25		.38
Adjusted R2		.10		.22
f		1.63		2.40*
Δ R2				.12
Δ f				5.53*

^*p < .05

^**p < .01

^aIntervention (1) versus control (0) conditions

^bSpecial education and combined classes (1) versus regular education classes (0)

Figure 2. Moderated Intervention Effect on Time 2 Emotional Support by Intervention Condition and Time 1 Emotional Support.

Note. Low T1 Emotional Support = 1 SD below the mean; Mean T1 Emotional Support = at the mean level (=4.41); Estimated effect includes all covariates (classroom type, classroom size, grade level, teaching experience)

Intervention Effect on Children

BRIDGE intervention effects on child outcomes (see Table 2 for details) were evaluated in multi-level models that included classroom and child covariates. Baseline scores on the outcome variable were significant predictors in all models; other classroom and child covariates were nonsignificant unless noted otherwise.

Table 2.

Observed Means, Standard Deviations, and Intent-To-Treat Intervention Main Effect for Child Outcomes

Outcome Variables	Control		Intervention				95% CI		Effect Size
	Time 1	Time 2	Time 1	Time 2	b	SE	LL	UL
Teacher-Student Closeness a	28.94 (6.71)	26.33 (5.58)	30.57 (4.70)	29.89 (5.33)	2.75*	.08	2.59	2.91	0.47
Teacher-Student Conflict a	14.65 (7.64)	14.04 (7.35)	14.71 (7.24)	16.34 (7.33)	1.60	.92	-0.24	3.64	-
Aggressive Behavior b	-0.09 (.83)	0.03 (.83)	-0.13 (.79)	-0.19 (.88)	-0.13	.12	-0.37	0.11	-
Victimization b	-0.08 (.89)	-0.07 (1.05)	-0.10 (.79)	-0.16 (.85)	-0.27*	.12	-0.51	-0.03	0.31
Academic Self-Concept b	2.87 (.66)	2.81 (.67)	2.84 (.63)	3.04 (.66)	0.20*	.09	0.02	0.38	0.31
Behavioral Regulation a c	58.71 (14.31)	58.08 (13.61)	58.95 (15.11)	62.86 (17.20)	-.00	.00	0.00	0.00	-

^*p < .05

^aSubsample participants (n = 159)

^bAll 2^nd to 5^th grade students (n = 226)

^cWithout 4 extreme outliers (n = 155)

Note. Covariates included in each model were class size, grade level, classroom type, years teaching, child gender, and Time 1 measures of the outcome variables.

Behavioral regulation

In the child subsample, no significant intervention effect was found for T2 teacher-reported behavioral regulation controlling for T1 behavioral regulation and child and classroom covariates (see Table 2).

Teacher-student relationships

A significant main effect of BRIDGE for the child subsample was found for teacher-student relationship closeness (b = 2.75, p < .05, pseudo R² = .15, ES = .47). At T2, there was on average a difference of 2.75 points on the closeness scale (range = 8 - 40) between children in intervention and control classrooms, controlling for T1 levels. No significant main effect of the intervention was found for relationship conflict (see Table 2).

Academic self-concept

There was a significant positive intervention effect on children’s academic self-concept (b = .20, p < .05, pseudo R² = .17, ES = .31). Children in BRIDGE classrooms reported, on average, a 0.20 higher academic self-concept score (scale from 1-4) than children in control classrooms, controlling for T1 levels and covariates (see Table 2).

Peer social experiences

Among second to fifth grade students (n = 226), there was a main effect of BRIDGE in predicting victimization (b = -.27, p < .05, pseudo R² = .05, ES = .31). Thus, the standardized victimization score at T2 was, on average, 0.27 points lower for children in intervention classrooms than children in control classrooms, controlling for T1 levels and covariates. In addition, victimization was negatively predicted by years teaching (b = -.03, p < .05), suggesting that children were less victimized in classrooms with more experienced teachers. No significant intervention effect was found for aggressive behavior (see Table 2).

Differential intervention effects

In preliminary analyses with the child subsample, differential impact on target children at T1 was found for victimization (b = -.95, p < .05), suggesting that children identified with behavioral problems benefited more from BRIDGE within this domain of social relations than their peers. Results indicated child scores on other outcomes were in the anticipated direction but did not reach statistical significance.

Discussion

This study evaluated the effects of a consultation and coaching model to enhance classroom interactions and improve child adjustment in poor urban schools. BRIDGE integrated universal and targeted supports in a classroom intervention that school and community mental health professionals, as part of their regular service to the school, coached teachers to implement. Unlike most coaching interventions, BRIDGE was designed to be a sustainable model (Cappella et al., 2011); unlike most mental health activities, BRIDGE enabled staff to provide feedback to teachers about their everyday classroom practices. BRIDGE had a positive impact on classroom interactions for classrooms with low levels of emotional support at the start of the year. Children in BRIDGE classrooms benefited in terms of their relational closeness to teachers, social experiences with peers, and academic self-concept. Findings were present across multiple reporters, beyond baseline levels of the outcomes and covariates, and within the context of underfunded, urban schools. Impacts on children’s relationships and academic self concept are notable given the short intervention period and the fact that all components were delivered at the teacher level, rather than the more typical mental health approach of direct service to children. Findings suggest that focusing school mental health resources on classrooms may have benefits for targeted students and their peers.

Changes in Classroom Practices

Teachers in BRIDGE classrooms who started the year with lower levels of emotional support made significant gains in these practices across the year. Although BRIDGE teachers who provided low levels of emotional support in the fall did not improve to the levels found in high support classrooms, gains are noteworthy because children in more supportive classrooms learn more (Pianta, et al., 2007) – particularly children at risk for social and emotional difficulties (Hamre & Pianta, 2005). These findings are consistent with recent work (e.g. Brown et al., 2010; Raver et al., 2008) demonstrating teachers can indeed become warmer and more responsive in their interactions with children. However, unlike other studies, we demonstrate this impact on teacher practices through an integrated consultation and coaching program (i.e., universal and targeted) delivered by members of the school mental health team.

BRIDGE did not impact classroom organization: behavior management, productivity, and engagement. Other studies examining the effects of teacher-focused interventions on classroom interactions similarly have failed to show changes in classroom organization (e.g. Brown et al., 2010; Domitrovich et al., 2007; Raver et al., 2008). The classroom organization domain of the CLASS measure, used in these studies, may not be sufficiently sensitive to capture small shifts in these practices. However, classroom management may be more difficult to change later in the school year, suggesting the need to train and support teachers early in the year (Marzano, Marzano, & Pickering, 2003). In addition, consultation and coaching may be more effective if preceded by quality pre-service training that is aligned with in-service support for classroom organization and management.

Behaviors and Relationships

No intervention effect was found for children’s behavioral regulation, despite an impact on emotional support. Thus, increases in emotional support without accompanying effects on behavior management may not be sufficient to improve child behaviors. Indeed, a teacher training and coaching model with external interventionists in urban preschools was found to improve regulation skills in young children in classrooms with increased emotional support and behavior management (Raver et al., 2011). In addition, the sample in the current study included children in middle childhood. Although no grade-level differences were found in intervention outcomes, child behaviors may become more entrenched as they grow older (Moskowitz, Schwartzman, & Ledingham, 1985), suggesting the need to supplement the classroom intervention with components focused on other contexts (e.g., home) and competencies (e.g., social skills) that influence behaviors.

Teachers in BRIDGE classrooms reported greater closeness with their students at the end of the year than teachers in control classrooms. Teacher closeness with their students relates to children’s engagement in school and competence across domains (Hamre & Pianta, 2001; Hughes & Kwok, 2006). Few intervention studies, however, have documented changes in the quality of these relationships as reported by teachers. Two studies conducted with preschool children, one focused on a social-emotional curriculum (Hamre et al., 2011), the other on improving teacher-student relationships (Driscoll & Pianta, 2010), similarly found changes in closeness. Preventing the observed normative decline in teacher-student closeness enables children (and teachers) to finish the school year on a positive note, which may influence subsequent motivation and engagement.

Similar to previous studies, no intervention effects were found in teacher-student conflict (Driscoll & Pianta, 2010; Hamre et al., 2011). Teachers’ perceptions of conflict with children are quite stable and may be harder to change than positive aspects of the relationship (Hamre et al., 2008). This finding aligns with the lack of impact on behavioral regulation: continued behavioral problems may contribute to continued experience of conflict. This may be particularly true if teachers are working to change behaviors but perceive no change. Recent work on teacher stress and coping (Brouwers & Tomic, 2000; Jennings & Greenberg, 2009) highlights the need for effective programs to help teachers cope with difficult behaviors, thus reducing their own reactivity to, and conflict with, students.

Academic and Social Experiences

The BRIDGE intervention impacted children’s sense of themselves as learners. Given that academic self-concept predicts subsequent achievement (Marsh & Yeung, 1997), this effect is promising. It is consistent with findings suggesting programs with social and emotional learning components impact achievement indicators (Durlak et al., 2011). Children may feel more successful in classrooms with emotional support (i.e., respectful and responsive interactions). Similarly, children with close relationships with their teachers may see themselves as their teachers want them to be – successful learners (Hughes & Kwok, 2007; Skinner & Belmont, 1993). Importantly, this finding is present in urban underfunded schools where children face risk for academic disengagement.

BRIDGE also reduced children’s victimization by their peers. This represents the first evidence that classroom-level intervention can impact such experience – prior findings have been reported only for school-level approaches focused on school climate and organization (Smith, Schneider, Smith, & Ananiadou, 2004). Although the mechanism of the effect is unclear, several possibilities warrant consideration. BRIDGE may increase interpersonal respect and sensitivity in classrooms, as these characteristics of classrooms have been linked to children’s social outcomes (Howes, 2000). Warm and positive relationships with teachers may protect vulnerable children from peer problems (Chang et al., 2007; Hughes, Cavell, & Wilson, 2001). Also, when individual children become more confident in their academic abilities, they may conduct themselves in ways that are more aligned with peer academic norms (Chen, Chang, Liu, & He, 2008) thereby eliciting less aggression (Taylor, Davis-Kean, & Malanchuk, 2007). Of interest, teachers with more teaching experience had less peer-reported victimization in their classrooms, suggesting greater awareness of, and/or actions to prevent, victimization – a finding to be replicated in future studies.

The intervention did not change student ratings of aggressive behavior. The lack of effect may reflect the stability of aggressive reputations across the year, even in the presence of behavior change (Peets, Hodges, Kikas, Salmivalli, 2007). Characteristics of the measure may attenuate the odds of detecting changes in aggressive behavior, as it indexes the presence or absence of particular behaviors, rather than the intensity of the aggressive acts, which may be more sensitive to change. Although additional research is necessary to understand whether and how the BRIDGE intervention affects aggressive behavior, it may be necessary to add intervention components that target predictors of aggression across levels, such as parenting practices (e.g., Webster-Stratton & Reid, 2010), school rules and routines (e.g., Bradshaw, Koth, Thornton, & Leaf, 2009), and child social and cognitive skills (e.g., Jones, Brown, & Aber, 2011).

Differential Impact on Target Students

Preliminary moderation analyses revealed that children with behavioral difficulties were less likely to be victimized at the end of the study in intervention versus control classrooms. Children with behavioral problems are at risk for a negative cycle of victimization by peers (Schwartz et al., 1999). Modifying these negative trajectories may be important to the subsequent mental health and school engagement of children with behavioral problems. Although we found no other significant child-level moderation findings, results were in the expected direction suggesting the need for larger sample sizes to detect differential intervention impact on target students.

Limitations and Future Directions

This study has several limitations. First, with a one-year pretest-posttest design, this study could not assess long-term effects. Replication of this study over a longer period would allow better understanding of intervention impact on change in trajectories for children and classrooms. Second, although there were advantages to the measures used, future research with standard clinical scales and multiple classroom observers would strengthen these findings. Third, classroom randomization provides a conservative test of intervention effect, but school level randomization is necessary in future studies to reduce the potential for contamination across groups. Fourth, the sample was small. Although schools were similar to one another in composition and structure, we did not have the power to evaluate whether school climate or consultant characteristics made a difference in results. In addition, the number of teacher and target child participants is a relatively low proportion of the available population, introducing the potential for selection bias. Motivated teachers or teachers struggling with student behavior problems may have consented at higher rates. Even though consented children were similar to the overall population and target children had behavior problems above clinical levels, it is possible children with the most severe difficulties were underrepresented. Thus, it is necessary to replicate these findings with additional teachers and children.

Despite these limitations, the findings have implications for policy- and practice-relevant research. First, the study demonstrated that a program focused on student behaviors and classwide practices can be embedded within the regular activities of school mental health professionals. Mental health staff with varied roles (social worker, counselor) and employers (school district, community agency) provided support to teachers across grades (kindergarten to fifth) and across classrooms (regular and special education). Future mixed method research is necessary to indicate whether the intervention model is feasible, engaging, and sustainable beyond one year (e.g., Palinkas et al., 2011). However, given the difficulties experienced in many urban underfunded schools, there is a profound need for classroom support that relies on indigenous resources and weaves its way into natural and sustainable routines. BRIDGE represents an important step in that direction.

Second, the current findings are consistent with previous research indicating that classrooms with low levels of emotional support can improve with consultation and coaching. This highlights the need to identify classrooms with poor emotional climate and take action to support them. Future research will determine whether new components or more time for the current components is necessary to enhance teachers’ ability to organize the classroom and improve students’ behaviors. Finally, after only a few months of intervention, promising – though modest – change in children’s adjustment had begun. It is encouraging that a consultation and coaching component of mental health practice based on actual interactions in the elementary classroom and effective strategies to improve these interactions promotes children’s functioning across domains in urban poor schools.