Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Feb 21.
Published in final edited form as: Psychol Sch. 2013 Nov 12;51(2):143–163. doi: 10.1002/pits.21741

EFFECTS OF A PROFESSIONAL DEVELOPMENT PROGRAM ON BEHAVIORAL ENGAGEMENT OF STUDENTS IN MIDDLE AND HIGH SCHOOL

ANNE GREGORY 1, JOSEPH P ALLEN 2, AMORI Y MIKAMI 3, CHRISTOPHER A HAFEN 4, ROBERT C PIANTA 5
PMCID: PMC5319794  NIHMSID: NIHMS847424  PMID: 28232767

Abstract

Student behavioral engagement is a key condition supporting academic achievement, yet student disengagement in middle and high schools is all too common. The current study used a randomized controlled design to test the efficacy of the My Teaching Partner-Secondary program to increase behavioral engagement. The program offers teachers personalized coaching and systematic feedback on teachers’ interactions with students, based on systematic observation of videorecordings of teacher-student interactions in the classroom. The study found that intervention teachers had significantly higher increases, albeit to a modest degree, in student behavioral engagement in their classrooms after 1 year of involvement with the program compared to the teachers in the control group (explaining 4% of variance). In exploratory analyses, two dimensions of teachers’ interactions with students—their focus on analysis and problem solving during instruction and their use of diverse instructional learning formats—acted as mediators of increased student engagement. The findings offer implications for new directions in teacher professional development and for understanding the classroom as a setting for adolescent development.

Despite the need to successfully engage adolescents in classrooms, there is a striking lack of empirically supported professional development programs targeting classroom engagement at the middle and high school level. In fact, research on effective teacher professional development has lagged despite an interest in establishing the benefits of sustained and intensive programs versus more traditional single session workshops (Carrol, Fulton, & Doerr, 2010). Research linking models of professional development to classroom-wide student outcomes is relatively new (e.g., Reinke, Lewis-Palmer, & Merrell, 2008). Yoon, Duncan, Lee, Scarloss, and Shapley (2007) reviewed more than 1,300 studies of the effect of teacher professional development on K–12 student outcomes, and found that only nine studies met What Works Clearing-house’s evidence standards without reservations. Importantly, the nine studies involved elementary school teachers, and none were at the middle or high school level. The authors call for rigorous empirical examinations of professional development at all levels of schooling. The current study answers this call with its randomized control design using well-validated measures of teacher behavior at the middle and high school levels. Specifically, we present results from an experimental study of the effects of My Teaching Partner–Secondary (MTP-S) on increased adolescent behavioral engagement via strengthened teacher-student interactions.

Engagement in the Classroom

Students engage in numerous, multifaceted ways in the classroom. Students participate in classroom activities (behavioral engagement), respond positively to teachers and assignments (emotional engagement), and thoughtfully problem solve on academic tasks (cognitive engagement) (Fredricks, Blumenfeld, & Paris, 2004). Recent measurement studies have distinguished between the behavioral, emotional, and cognitive subtypes of student engagement (Archambault, Janosz, Morizot, & Pagani, 2009; Betts et al., 2010). Yet the subtypes are likely interrelated (Fredricks et al., 2004). In fact, decades of psychological research have demonstrated that behaviors, emotions, and cognitions can be mutually reinforcing (e.g., Beck, 1976). For instance, undertaking an activity can lead to changes in the way one thinks and feels about the activity (Ellis, 1994). In classrooms, when students actively participate in assignments, their cognitive exertion and positive emotional responses to teachers may increase.

Whatever the directionality of influence between the three types of engagement, behavioral engagement has been studied the most extensively in relation to student achievement and graduation rates (Brophy, 1988; Fisher et al., 1981; Greenwood, Horton, & Utley, 2002; Wang, Haertel, & Walberg, 1997). Behavioral engagement can be observed when students contribute to classroom discussion, attend to an academic task, and demonstrate they are listening to teacher instruction. In a nationally representative sample of ninth grade students, Finn (1993) found that behavioral engagement predicted performance on standardized achievement tests. Numerous studies have corroborated that behavioral disengagement is a proximal risk factor for students and related to concurrent and later low academic achievement (e.g., Alexander, Entwisle, & Horsey, 1997; Fincham, Hokoda, & Sanders, 1989; Finn & Cox, 1992; Laffey, 1982; Spivack & Cianci, 1987; Swift & Spivack, 1969). Taken together, these studies suggest that increases in students’ behavioral engagement could have direct impact on raising student achievement; the more students actively participate in course activities, the more they will learn the course content.

Engagement is a relational process that hinges on teachers’ ability to synch with students’ developmental needs (Hamre & Pianta, 2010; Pianta, Hamre, & Allen, 2012). Accordingly, it is posited that improving teachers’ developmentally appropriate interactions with their students has the potential to increase their behavioral engagement. Interactions are defined as observable patterns of behavior between teachers and students that can occur during academic instruction, free time, or transitions. Positive teacher-student interactions are those that support students’ social, emotional, and cognitive needs. Through their interactions with students, teachers challenge and interest students; they afford rich learning opportunities; they provide appropriate support to scaffold student participation (e.g., Skinner & Belmont, 1993).

My Teaching Partner–Secondary

The MTP intervention explicitly targets increasing teachers’ use of positive, developmentally appropriate teacher-student interactions. Such interactions are intended to create an emotionally positive and cognitively challenging classroom characterized by sensitivity to individual students’ socioemotional and academic needs, which should ultimately result in improved classroom functioning for students. MTP was originally developed for pre-K and early elementary classrooms (MTP Pre-K) and has demonstrated effectiveness in improving teacher-student interactions and student outcomes. Specifically, in their controlled intervention study of MTP Pre-K, Pianta, Mashburn, Downer, Hamre, and Justice (2008) found that teachers in the control, or website-access only, condition actually demonstrated poorer interactions with their students over the course of the year whereas teachers receiving one-on-one consultancy and access to a website demonstrated improved interactions with their students. Intervention teachers’ students also made greater gains in receptive language skills during Pre-K compared to children whose teachers received access to the online video exemplars only (Mashburn, Downer, Hamre, Justice, & Pianta, 2010).

The secondary version of the program (MTP-S) for middle and high school classrooms shares similar intervention procedures and strategies with the MTP Pre-K program. Namely, MTP regardless of the targeted grade level is an intensive, year-long program that consists of a collaborative intervention process that increases teachers’ knowledge of and skills related to effective teacher-student interactions and applies this knowledge and skill with ongoing, personalized coaching and feedback (Pianta et al., 2003). MTP coaches provide direct feedback to teachers on their interactions with students based on a validated, observational assessment of teacher-student interactions, the Classroom Assessment Scoring System (CLASS; Pianta et al., 2008), via a web- and video-conference assessment and feedback process. Teachers send in videotaped instruction on a regular basis, which produces a stream of communication between coach and teacher based on observation of that teacher’s interactions with students in the classroom and a process of joint analysis and identification of strategies to improve those interactions, using the CLASS as the target and lens for analysis.

Four principles guide the consultancy approach: (a) build supportive and nonevaluative relationships with teachers; (b) use a collaborative approach to engage and challenge teachers; (c) provide individualized regular feedback based on specific behavioral examples seen in the classroom; and (d) base observations and feedback on the CLASS-S (My Teaching Partner-Secondary Consultancy Manual, 2010). Similar to behavioral consultation (Bergan & Kratochwill, 1990) and the Classroom Check-Up program (Reinke et al., 2008), MTP coaches identify specific, observable behaviors and systematically set goals for change.

Although similar intervention procedures are used in MTP-S and MTP-Pre-K, the specific types of teacher-student interactions encouraged are targeted to the developmental needs of the different age groups of students. These developmental differences are apparent in the CLASS instruments used to guide the consultation in each intervention (Pianta, Hamre, Hayes, Mintz, & LaParo, 2008). Specifically, the MTP-Pre-K version of the CLASS includes the provision of language and literacy activities whereas the MTP-S version of the CLASS focuses on adolescent autonomy, “real” world relevancy of the content, and higher-order problem solving.

The Classroom Assessment Scoring System–Secondary and Adolescent Needs

The CLASS-Secondary (CLASS-S) measure, which is the secondary school version of this instrument adapted for MTP-S, has three overarching domains, Emotional Support, Classroom Organization, and Instructional Support, which are each comprised of three to four dimensions (Hamre & Pianta, 2010). Predominant theories guiding the CLASS-S are self-determination theory (Ryan & Deci, 2000) and attachment theory (as applied to teacher-student relationships) (Bowlby, 1968; Hamre & Pianta, 2001), which assert that adolescents need to achieve a sense of relational support, a sense of autonomy and competence within settings, and a sense of the relevance and meaning of what is being taught (Allen, Hauser, Bell, & O’Connor, 1994; Allen, Kuperminc, Philliber, & Herre, 1994; Allen et al., 2002).

Related to the Emotional Support domain of classroom interactions, research has shown that when teachers learn to make modest efforts to form a personal connection with their adolescent students—such that students feel known—they can dramatically enhance student engagement in school and emotional functioning outside of school (Gregory & Weinstein, 2004; National Research Council [NRC], 2004; Skinner, Zimmer-Gembeck, & Connell, 1998). In terms of the need for autonomy and relevance in the classroom, developmentalists have long known that adolescents are particularly engaged by challenges that are within reach and that provide a sense of self-efficacy and control—experiences that offer challenges viewed as adultlike but for which appropriate scaffolding and support are provided (Bandura, Barbaranelli, Caprara, & Pastorelli, 1996; Eccles et al., 1993). Given adolescents’ need for relevancy in the classroom, drawing even distal connections between what occurs within high school and the larger “real world” can alter student behavior (Allen, Philliber, Herrling, & Kuperminc, 1997; Andriessen, Phalet, & Lens, 2006; Greene, Miller, Crowson, Duke, & Akey, 2004).

In the Classroom Organization domain, teachers structure students’ attention and activities which enables opportunities to demonstrate their competency (Emmer & Stough, 2001). Effective classroom managers set behavioral expectations, enforce classroom norms, and structure dynamic activities (Doyle, 2006). By so doing, effective managers guide student attention and help them stay engaged in academic tasks (Pianta & Hamre, 2009; Yair, 2000). Finally, teachers interact with adolescents through the Instructional Support domain. Studies of cognitive development have shown that students are more engaged when they have cognitively challenging tasks (Stodolsky, 1988), and opportunities to solve meaningful problems (Newmann, Wehlage, & Lamborn, 1992). Adolescents learn more when they are asked to demonstrate higher-order thinking skills (not simply memorizing discrete, disjointed facts) and apply knowledge and procedures to new problems (NRC, 2005).

In sum, based on the dimensions and domains of the CLASS-S (see Table 1 for the theoretical model of CLASS-S), the MTP-S program aims to help teachers offer relational support to adolescents (i.e., Positive Climate and Teacher Sensitivity in the Emotional Support domain) and capitalize on students’ autonomy strivings by providing them with choices of partners and group projects and increase the significance of what is taught in the classroom by making repeated, explicit ties between curricular material and real-world applications and by engaging peer group processes in learning (i.e., Regard for Adolescent Perspective in the Emotional Support domain). MTP-S helps teachers provide appropriate structure through clear routines and proactive monitoring of behavior (i.e., Behavior Management, Productivity in the Classroom Organization domain). And importantly, it targets the need for varied, engaging, and cognitively challenging instruction that fosters reasoning and conceptual understanding (i.e., Instructional Learning Formats in the Classroom Organization domain and Content Understanding, Analysis and Problem Solving/Quality of Feedback in the Instructional Support domain).

Table 1.

Theoretical Model of the Classroom Assessment Scoring System-Secondary (CLASS-S; Pianta, Hamre, Haynes, Mintz, & La Paro, 2006)

Domain Dimensions Description
Emotional Support Positive Climate The emotional tone of the classroom (e.g., warmth and connection among teachers and students).
Negative Climate The level of expressed negativity (e.g., irritability, frustration, anger).
Teacher Sensitivity The teacher’s responsiveness to academic and social/emotional needs of students.
Regard for Adolescent Perspectives The extent to which the teacher offers leadership, autonomy, and content relevance to students.
Classroom Organization Behavior Management Teacher’s use of effective methods to encourage desirable behavior and redirect misbehavior.
Productivity The teacher’s management of time to maximize instruction.
Instructional Learning Formats The teacher’s provision of interesting, varied lessons and materials.
Instructional Support Content Understanding The depth of lesson content and integration of facts, skills, concepts, and principles.
Analysis and Problem Solving The degree to which the teacher facilitates higher level thinking skills, problem solving, and metacognition.
Quality of Feedback The provision of feedback that expands or extends learning and understanding.
Outcome measure Student Engagement Students are consistently active in discussion and classroom tasks—they volunteer, ask questions, and show little off-task behavior.
Open in a new tab

Study Aims and Hypotheses

Few, if any controlled studies have established the efficacy of a teacher professional development program for increasing behavioral engagement in secondary schools (Fredricks et al., 2004). Aside from overall average impacts of professional development, it is also of interest whether program effects may be moderated by classroom characteristics (i.e., classroom poverty, enrollment size, achievement level, or racial composition) or teacher characteristics (i.e., teacher race, gender, age, level of education, and years of instructional experience), given the wide-ranging variation in U.S. secondary classrooms. Using an experimental design in which teachers were randomly assigned to the program or to “business-as-usual,” the current study asked:

  1. To what extent does participation in the My Teaching Partner-Secondary professional development program increase observed student behavioral engagement, after covarying teacher and classroom characteristics?

  2. To what extent do teacher and classroom characteristics moderate the effects of the program on observed behavioral engagement?

  3. To what extent do changes in specific dimensions of the CLASS-S mediate the effect of the program on observed behavioral engagement?

We hypothesized that the MTP-S intervention would increase behavioral engagement in classrooms across the school year regardless of teacher characteristics (e.g., years of teaching, gender) and classroom characteristics (e.g., percent of low income students). We considered the mediational analyses to be exploratory given there were no previous studies suggesting which aspects of developmentally appropriate teacher-student interactions (as measured by a validated observational system) help explain how a professional development program can shift adolescents’ behavioral engagement in the classroom.

Method

Participants and Settings

The 87 participating teachers taught in 12 different schools in Virginia. The number of teachers per school ranged from 1 to 15 (M = 7 teachers per school). About 61% of participants taught in middle schools and the remaining 39% of participants taught in high schools. Participants’ years of teaching experience ranged widely, with an average of 8 years of experience. The sample was composed of a majority of Caucasian female teachers.

Sixty-six teachers taught the same students across the school year, and 21 teachers switched students at mid-academic year. The year-long teachers had a “traditional” schedule, and the mid-year teachers had a “block” schedule, given that they taught two different blocks of students—one in the fall semester and one in the spring semester. Traditional teachers selected one course, and block teachers selected a fall and spring course in which (a) they were the primary instructor; and (b) an end-of-course standardized exam was administered to assess student learning. The selected courses became the “focal” classrooms for the intervention.

Procedures and Intervention

Subsequent to Institutional Review Board approval, a project team presented the MTP-S intervention and research to teachers in each school district. Teachers were asked to voluntarily consent for participation. During initial spring recruitment, 114 teachers initially expressed interest to participate in the study; however, 26 later decided not to participate (23 prior to any contact with the program, three after the initial workshop but before beginning the intervention or selecting a focal class). One intervention teacher did not participate in a single coaching cycle or mail in videotaped instruction because of medical illness.

Participating teachers received a modest monetary compensation for completing surveys and professional development credit for their participation, as issued by their school district. Parents of students in target classrooms were asked to provide informed consent and students were asked to provide assent if they were willing to voluntarily participate in the study. A majority of students (78.18% per classroom, 1669 students) consented to participate with an average of 17 students per classroom (SD = 5, range = 5–30).

As part of the randomization procedure, participating teachers were first grouped by district, school type (i.e., middle/high school), and their classroom subject (i.e., math/science, social studies/English). Teachers within each of the stratified groups were then randomly assigned to the intervention or control group. The control group received “business-as-usual” professional development. Intervention and control teachers did not significantly differ on a range of sociodemographic characteristics (e.g., gender, race, years of teaching experience; see Table 2). Their focal classrooms also did not significantly differ on a range of characteristics (e.g., student baseline achievement level, racial composition, block versus traditional scheduling; see Table 2). Intervention and control teachers were also equivalent in the fall, at the start of the school year, on student behavioral engagement and dimensions of the CLASS-S (see Table 3).

Table 2.

Teacher and Classroom Characteristics by Intervention vs. Control Group

Intervention
N 		Control
N 		Significance of Group Differences P
Teacher Gender Male: 13 Male: 19 .21
Female: 30 Female: 25
Teachers Education BA: 17 BA: 11 .15
Beyond BA: 26 Beyond BA: 33
TeacherRace/Ethnicity Asian: 0 Asian: 1 .40
Afr-American: 4 Afr-American: 4
White: 38 White: 35
Multi-racial: 1 Multi-racial: 4
M (SD) M (SD)
Teacher age 38.8 (11.7) 41.1 (10.8) .41
Number of Years Teaching 7.4 (6.6) 10.5 (11.2) .12
Classroom Grade Level 7.94 (1.5) 8.21 (1.5) .42
Students’ Prior Year Achievement Test Score 456.2 (44.6) 445.7 (52.6) .33
Average Class Size 21.3 (5.0) 22.6 (5.4) .25
% Male Gender 51.6% (13.4%) 51.0% (12.2%) .82
% Ethnic Minority 36.7 (22.0%) 39.6% (25.0%) .57
% Low Income 39.4% (26.7%) 38.9% (27.4%) .94
Open in a new tab

Note. % Low Income = Percentage of students qualifying for free and reduced priced meals.

Table 3.

Fall CLASS-S Dimensions by Intervention vs. Control Group

Intervention
M (SD) 		Control
M (SD) 		Significance of Group Differences P
Positive Climate 4.36 (.76) 4.08 (.65) .11
Teacher Sensitivity 4.61 (.73) 4.52 (.86) .63
Regard for Adol. Perspectives 3.45 (.76) 3.43 (.90) .92
Instructional Learning Formats 4.20 (.81) 4.27 (.75) .65
Analysis & Problem Solving 3.08 (.78) 3.07 (.92) .98
Behavioral Engagement 4.59 (.65) 4.66 (.70) .62
Open in a new tab

Intervention teachers were oriented to MTP-S coaching through a one-day workshop prior to the start of the first school year of the intervention. The workshop introduced teachers to principles of adolescent development and ways to improve their interactions with students, gave them a chance to discuss application of these principles to their classrooms, and introduced them to the MTP-S system of coaching.

The control teachers were asked to submit videotapes of six different instructional lessons across the school year. For research coding, intervention teacher tapes were selected so as to be synchronous in time with the submission of control teacher tapes. All teachers followed a standard protocol for taping, such as where to place the camera and the length of time to record (40–60 minutes of instruction).

The intervention teachers were asked to submit videotapes of instructional lessons every 2 weeks across the school year so that they could engage in a “coaching cycle,” with each cycle extending across a 2-week period (My Teaching Partner Secondary Consultancy Manual, 2010). The intervention teachers mailed videotapes to one of the two coaches, who were both experts in teaching and adolescent development. Receipt of the videotapes initiated a structured, manualized set of exchanges (based on the videorecording) between the teacher and the coach. Coaches were required to implement a five step exchange with the teachers: (1) For each coaching cycle, the consultant edits the videotaped instruction into at least two short clips that highlight one or more dimensions of the CLASS-S. The “nice work” clip helps the teachers see how their behavior in a particular interaction with students elucidate the principles of one of the dimensions of the CLASS-S. The “consider this” clip helps a teacher observe another interaction with students that typically reflects a CLASS-S dimension upon which the teacher needs to improve. (2) The coach sends the clips back to the teacher along with “nice work” and “consider this” written prompts, in which the coach clearly describes the observed teacher-student interaction in CLASS-S terms, along with how the teacher’s interaction with the student directly affects student responses. (3) The teacher reviews the clips and responds in writing to the prompts. (4) The teacher and the coach meet via telephone or computer for a conference to discuss the clips, written prompts, and responses. In addition to providing support, the coach also develops an action plan for improvement with the teacher and goals for the next cycle including which dimensions of the CLASS-S to consider implementing in subsequent interactions with students. (5) The coach summarizes the conference in writing including the action plan, which could include strategies to implement new behaviors reflective of a CLASS-S dimension in their upcoming instruction. An excerpted example of a coach’s written conference summary and goals for the next cycle is as follows:

Your first clip demonstrated how you give students the opportunity to play an integral role in class activities, which is an important facet of Regard for Adolescent Perspectives … Because you have given your students opportunities for leadership and autonomy throughout the year, they are able to take on a high level of responsibility for their own learning. In the next clip, you helped students apply their thinking to a real world situation, which helped them gain a better understanding of chemical reactions that they would see during the lab. We also discussed how clear it is that the students enjoy working with one another … which reflects the Positive Climate in the class. Next time, we will work on Instructional Learning Formats (using a variety of modalities, strategies, techniques, and materials). Please watch the video exemplars from the MTP-S website on this CLASS-S dimension.

As referenced in the above summary, intervention teachers also had ongoing access throughout the school year to the video and text examples of “high quality teaching” on the MTP-S website, which are organized around the dimensions of CLASS-S and provide an empirically based model of “best practice” in real classrooms. Typically, coaches asked teachers to view specific video examples on the MTP-S website as part of their goals for the next cycle.

Both coaches went through this same five step process in each coaching cycle for each teacher. The two coaches completed a similar number of cycles with their teachers (coach 1 M = 9.22, SD = 2.33; coach 2 M = 9.61, SD = 2.11). They each attended a clinic meeting once a week where they would discuss their work with their intervention teachers. An effort was made to move through the CLASS-S dimensions from cycle to cycle, though areas where a teacher was struggling were given more attention.

We considered program fidelity based on the five guidelines provided by Dane and Schneider (1998). Exposure to the program was high for teachers assigned to MTP-S, with 100% of teachers attending the workshop. Moreover, teachers completed an average of 9 (SD = 3; range = 2–16) coaching cycles and viewed an average of 14 (SD = 14; range = 0–60) clips of video exemplars on the MTP-S website. Adherence to treatment was demonstrated by the coaching content being similar to that in the manual, measured by 88% of teachers reporting that they agreed they had learned something new about the CLASS-S dimensions after the coaching cycles. In addition, 100% of MTP-S teachers said the coaching cycle was “worth the time it took,” an indicator of high participant responsiveness, and 98% of the teachers agreed that the cycles were “productive,” an indicator of good quality of delivery. Finally, regarding potential diffusion of MTP-related experiences to control teachers, the program was set up such that control teachers were given no access to the coaching web site, video library, or the coaches. In fact, the majority of control teachers (over 60%) reported receiving not even routine mentoring or coaching, including coaching completely unrelated to the intervention, during the school year.

Measures

Teacher and Classroom Characteristics

Data from teacher surveys and school records were used as covariates to ascertain if the effect of MTP-S on engagement held despite differences in teacher and classroom characteristics. Teachers completed a demographic survey at the beginning of the study in which they provided information on their years of teaching experience, their age, gender, and race/ethnicity. They also reported on a range of classroom characteristics including total number of students enrolled (i.e., class size), as well as the gender and racial/ethnic student composition of the classroom. In addition, school records were obtained for consented students to measure percentage of low income students and average baseline achievement. These covariates were taken into account in analyses given the link between students’ socioeconomic status, achievement, and engagement (Fredricks et al., 2004; Shernoff & Schmidt, 2008). Records showed each student’s eligibility for free and reduced priced meals (an indicator of low income status given families with incomes up to 185% of the federal poverty level qualify) and each student’s score on the prior year’s Standard of Learning (SOL) end-of-course exam in a subject similar to the MTP-S focal course. The SOL exams have been shown to have good test-retest reliability and concurrent validity with other accepted achievement tests (Hambleton, Crocker, Cruse, Dodd, Plake, & Poggio, 2000). The students’ SOL scores were aggregated in each classroom to provide an indicator of baseline achievement level at the start of the semester.

Observed Teacher Behavior and Student Engagement

A team of advanced undergraduate and graduate student coders (kept unaware of intervention or control condition) were trained in a two day workshop on the CLASS-S system. Coders learned to rate CLASS-S dimensions along a 1–7 scale, with a 1 or 2 indicating low quality; 3, 4, or 5 indicating mid-range quality; and 6 or 7 indicating high quality. At the end of the workshop, each coder passed a reliability test, in which they scored within one point of the master coded tapes on 80% of scores, across five video segments. The master coders had extensive knowledge of the CLASS-S instrument. Additionally, team members met regularly during the year to jointly code master tapes in order to prevent drift and increase coding agreement.

For the purposes of this study, analyses used coding of one 40–60-minute videorecording of instruction from the first eight weeks of teachers’ fall courses and one from the last weeks of their instruction in the spring. Each teacher’s videotaped instruction was divided into two 20-minute segments. Each segment was assigned randomly to two coders (2 segments X 2 coders per segment = 4 sets of CLASS-S scores). For the fall and spring, their four scores were then averaged to maximize the reliability of observation scores (Raudenbush, Martinez, Bloom, Zhu, & Lin, 2008).

A recent validity study using the same sample showed five dimensions of the CLASS-S were predictive of higher student achievement test scores at the end of the year, explaining 4–8% of the residual variance in these scores after accounting for the substantial effects of prior-year scores and relevant student and classroom characteristics (Allen et al., 2012). Given the validity results, we used only the five validated teacher behavior dimensions from the CLASS-S in the analyses. The dimensions used in these analyses include (a) three dimensions of the Emotional Support domain—Positive Climate (respectful/warm communications, shared positive affect), Teacher Sensitivity (teacher responsiveness to student needs), Regard for Adolescent Perspective (opportunities for students’ active, leadership roles and exposure to relevant course content), (b) one dimension from the Classroom Organization domain—Instructional Learning Formats (varied use of instructional modalities and strategies), and (c) one dimension from the Instructional Support domain—Analysis and Problem Solving (engagement in activities that require synthesis, evaluation, and novel application of knowledge (see Table 1).

The reliability of the coding was tested using intraclass correlations coefficients (ICCs), which were in the good to excellent range (ranging from .64 to .78), based on Cicchetti and Sparrow’s (1981) standards for interpreting ICCs. Other metrics of assessing reliability showed the coding was acceptable. Inter-rater agreement was acceptable—codes based on the same observations were within one point of each other 80.5% of the time.

Team members also observed and rated fall and spring Student Behavioral Engagement, which is the student outcome variable coded in the CLASS-S. Coders rated high engagement (6, 7) when they observed that most students were consistently active in discussion and activities (e.g., they volunteered, shared ideas, manipulated materials, asked questions, and showed minimal off-task behavior). In contrast, low ratings (1, 2) indicated that a majority of the students appeared consistently distracted or disengaged. The Engagement dimension used in these analyses had an ICC of .66, which is in the good range (Cicchetti & Sparrow, 1981). It had acceptable inter-rater agreement (codes based on the same observations were within one point of each other 81.1% of the time).

Missing Data

Incomplete data varied depending on the source. All of the 87 teachers mailed in at least one codeable 20-minute segment of videotaped instruction within the first 8 weeks of school in the fall. However, mailing dropped off by the end of the school year, resulting in nine teachers who did not send in a videotape during their last weeks of school and one teacher sent in segments that were uncodeable (11% missing). Importantly, teachers who mailed or did not mail in the last tape were not significantly different in terms of individual characteristics (gender, age, race/ethnicity, years of teaching experience) or classroom characteristics (class size,% low income,% ethnic minority, baseline achievement). They also did not significantly differ on their fall observed CLASS-S codes and whether they were in the intervention or control group.

Data Analytic Plan

Given that teachers were nested within 12 schools, we first calculated the ICCs to ascertain the amount of between-school variance. We found that the average between-school variance across the five CLASS-S dimensions and student behavioral engagement was minimal (M = 3.0%), however, we nevertheless used multilevel models (MLMs; classrooms at level 1 nested within schools at level 2) to control for any potential school-level effects on the classroom-level outcomes (Raudenbush & Bryk, 2002). There were no predictors at level 2, and our focus was on the significance of the level 1 predictor (classroom having MTP versus control group status) on the level 1 outcome (classroom average of observed student engagement). Given there were two coaches in MTP-S, we tested whether teachers’ intervention effects depended on which coach they were assigned. The effects did not depend on the assigned coach.

We undertook “intent-to-treat” analyses, which prioritize the randomization process and ignore nonadherence—a degree of which is to be expected in all intervention studies (Hollis & Campbell, 1999). In other words, we compared all teachers randomly assigned to each condition, without excluding those who did not meet expected levels of participation. This analytic approach incorporates teachers with variable levels of participation in the intervention, as incomplete or imperfect adherence is likely to occur in “real world” program implementation. Importantly, there were no statistically significant differences found between the intervention and the control group on teacher characteristics, classroom characteristics, or observed interaction dimensions at the start of the school year, when the intervention began. Whereas “intent-to-treat” analyses were the focus of our analyses, we also recognize that the fidelity of implementation varied across teachers. Given this variation, we tested whether number of completed coaching cycles moderated the effects of the intervention on student engagement.

For the primary analyses predicting behavioral engagement, we first entered teacher and classroom covariates into the model. Second, we entered the fall rating of engagement as a predictor of the spring rating. By entering the fall rating of engagement as a predictor of the spring rating, we showed the increase or decrease in engagement from the fall relative to the initial engagement in the spring (Cohen & Cohen, 1983). Third, we tested for intervention effects on change in student engagement. By using a series of MLMs, we were able to estimate the unique variance explained by the intervention. Similar to analyses for engagement, we tested whether the intervention resulted in change in each CLASS-S dimension. For all of the models, we took into account teacher and classroom covariates to ascertain whether the intervention effect existed after these covariates were taken into account.

Given the missing data, we used full-information maximum likelihood (FIML) methods through Mplus version 6 software (Muthén & Muthén, 2007), which enabled us to conduct analyses with a more complete dataset. Assumptions that missingness was random and not based on characteristics of the teacher were met—we found no statistical differences on the fall ratings of engagement and the CLASS-S dimensions for teachers who did or did not submit video in the spring. This finding suggests that the probability of missing data in the spring was not a function of the values of the data in the fall. In addition, for ease of interpretation of the interaction term estimates, all variables in the MLMs were standardized with a mean of zero and a standard deviation of one for all analyses (Aiken & West, 1991). During preliminary analyses and examination of the descriptives, we tested possible moderation of the intervention effect by entering a series of interaction terms into the MLMs (Aiken & West, 1991). In this manner, we tested whether the effects of the intervention depended on teacher or classroom characteristics.

Finally, for the CLASS-S dimensions with significant intervention effects, we then tested whether they were mediators of the change in engagement. We tested the significance of this indirect path using the MODEL INDIRECT option in Mplus. To estimate the confidence interval of the indirect effect, we used bootstrapping procedures and determined the proper confidence band around the indirect point estimate (MacKinnon, Fairchild, & Fritz, 2007; Preacher, Zyphur, & Zhang, 2010).

Results

Preliminary Analyses

Means and standard deviations for variables of interest in the study are reported in Table 4. Table 4 also presents a matrix of the bivariate correlations between the fall and spring observed instructional quality dimensions (i.e., teacher behavior ratings) and observed student engagement. A number of teacher characteristics (i.e., teacher years of experience, gender, age, ethnic minority status, and education level) and classrooms characteristics (i.e., classroom gender composition, grade level, class size, racial composition, and average fall classroom achievement at the start of the school year) were also examined for their relation to primary outcomes in MLMs; no significant relations were found between these covariates and student outcomes. Yet, the percentage of low income students was significantly related to a student outcome and was, thus, retained in the MLMs. Whether a teacher switched students mid-year (block teachers) or retained the same students the whole school year (traditional teachers) also was not predictive of change in engagement or any of the teacher behavior dimensions.

Table 4.

Descriptive Statistics and Correlations Among CLASS-S Dimensions

Mean SD 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.
1. Fall Positive Climate 4.2 .81 .47*** .56*** .38** .62*** .36** .53*** .38** .45*** .17 .48*** .39**
2. Spring Positive Climate 4.2 .87 – .33** .61*** .33** .57*** .40*** .50*** .06 .32** .30** .58***
3. Fall Teacher Sensitivity 4.6 .81 – .38** .52*** .27* .51*** .39** .43*** .18 .51*** .34**
4. Spring Teacher Sensitivity 4.4 .76 – .32** .51*** .39** .56*** .08 .48*** .42*** .61***
5. Fall Regard for Adol. Perspectives 3.4 .83 – .48*** .54*** .28* .62*** .22 .54*** .41***
6. Spring Regard for Adol. Perspectives 3.5 .70 – .29* .58*** .17 .44*** .29* .60***
7. Fall Instructional Learning Formats 4.2 .78 – .41*** .54*** .20 .72*** .43***
8. Spring Instructional Learning Formats 4.2 .80 – .10 .47*** .35** .70***
9. Fall Analysis & Problem Solving 3.1 .86 – .15 .53*** .14
10. Spring Analysis & Problem Solving 2.9 .87 – .17 .55***
11. Fall Behav. Engagement 4.6 .67 – .37**
12. Spring Behav. Engagement 4.5 .77 –
Open in a new tab

Note.

*

p <.05.

**

p < .01.

***

p < .001.

In addition, we ran MLMs that tested whether the intervention had varying effects on teachers depending on classroom and teacher characteristics as well as block/traditional status of teachers and number of coaching cycles completed. Putting interaction terms into MLMs to test the possible moderation, we found that the intervention effects did not differ depending on any of the classroom characteristics (e.g., baseline achievement, grade level), teacher characteristics (e.g., years of teaching experience), whether a teacher had a semester or full year schedule (i.e., block versus traditional) or the number of completed cycles. Thus, with the exception of the percent of low income students, classroom and teacher characteristics are not reported or discussed further.

Primary Analyses

Results from the MLMs predicting engagement show that teacher participation in the MTP-S intervention was associated with increased student engagement at the end of the school year, relative to the start of the school year (β = .23, p < .05, R-squared change = .04, p < .05; Table 5). Specifically, the intervention classroom would show a relative gain of approximately 1/4 of a SD in engagement by the spring, after controlling for fall engagement, and a control classroom would show a relative loss of approximately 1/4 of a SD in engagement by the spring, after controlling for fall engagement. Overall, there was a .46 SD difference in engagement by the spring between intervention and control classrooms.

Table 5.

Relation of the My Teaching Partner Intervention to End-of-Year Dimensions of the CLASS

Predicting End-of-Year Dimensions of the CLASS
Eng β PC β TS β RAP β ILF β AP β
Step 1:a
% Low Income − .06 − .16 − .08 − .13 − .25* − .02
R2b .01 .03 .01 .02 .06* .00
Step 2:
Fall Classroom Variable .33** .40** .39** .46** .37** .13
ΔR2 .11** .16** .15** .21** .11** .01
Step 3:
Intervention (1) Control (0) .23* .02 .08 .12 .22* .25*
ΔR2 .04* .00 .01 .01 .04* .06*
Open in a new tab

Note. Eng = Engagement, PC = Positive Climate, TS = Teacher Sensitivity, RAP = Regard for Adolescent Perspective, ILF = Instructional Learning Formats, AP = Analysis and Problem Solving.

a

Beta coefficients are from each step of the MLMs.

b

R-squared is actually a pseudo-R-squared.

*

p <.05.

**

p < .01.

***

p < .001.

Exploratory Mediational Analyses

HLM results for fall to spring change in Instructional Learning Formats and Analysis and Problem Solving showed that the intervention had a positive effect on two variables. Specifically, teachers in the intervention had a positive and significant change in Instructional Learning Formats (β = .22, p < .05, R-squared change = .04, p < .05) and Analysis and Problem Solving (β = .25, p < .05, R-squared change = .06, p < .05; Table 5) across the school year. These effects held when accounting for the percentage of low income students in the classroom. In contrast, results for the three teacher behavior dimensions in the Emotional Support domain (Positive Climate, Teacher Sensitivity, and Regard for Adolescent Perspective) indicate that the intervention facilitated increases in these areas (β = .02, β = .08, β = .12, respectively), but none was statistically significant. Also noteworthy is that the classroom covariate, percent of low income students, was associated with only one of the CLASS-S dimensions; classrooms with more low income students had lower classroom ratings of Instructional Learning Formats at the end of the school year (β = −.25, p < .05, R-squared change = .06, p < .05). In other words, “higher poverty” classrooms tended to have teachers who used less variety in instructional modality (e.g., small group activities versus lecture), fewer strategies for active facilitation, and more infrequent provision of novel materials compared to “lower poverty” classrooms.

We then tested whether changes in Instructional Learning Formats and Analysis and Problem Solving accounted for the changes in engagement, thereby potentially explaining the mechanism of action in the MTP-S intervention. The indirect effects for both Instructional Learning Formats, a × b = .13 (p < .05), 90% CI [.017, .249], and Analysis and Problem Solving, a × b = .12 (p < .05), 90% CI [.014, .213], were statistically significant, indicating that the intervention indirectly predicted changes in engagement through these two CLASS-S dimensions (see Figures 1 and 2).

Figure 1.

Figure 1

Open in a new tab

Instructional learning formats mediating the effects of MTP on engagement. Note. *p < .05. **p < .01. Although not depicted here, fall Instructional Learning Formats and Student Engagement were controlled for in the model.

Figure 2.

Figure 2

Open in a new tab

Analysis and problem solving mediating the effects of MTP on engagement.

Discussion

Teacher participation in the MTP-S program resulted in a modest increase in student behavioral engagement from the fall to the spring, as demonstrated by the results from this randomized controlled trial. The effects did not vary depending on the characteristics of the classrooms (class size, grade level, gender composition, percentage of low income and ethnic minority student, fall achievement level) or the teacher (gender, race/ethnicity, age, level of education, years in the teaching profession, number of MTP-S coaching cycles completed). This study is among the first experimental studies to establish the efficacy of a teacher professional development program in changing adolescent behavioral engagement in the classroom. It is striking that the program was able to significantly and positively shift engagement, albeit to a modest degree (explaining 4% of the variance), across a diverse range of middle and high school classrooms, whether high and low achieving, more or less ethnically diverse, or more or less economically disadvantaged. Additional exploratory mediational analyses offered possible mechanisms—improved Instructional Learning Formats and Analysis and Problem Solving—that accounted for the change in student engagement. Unexpectedly, the intervention did not significantly shift dimensions in the Emotional Support domain of the CLASS-S (e.g., Positive Climate, Teacher Sensitivity)—an intriguing finding that warrants future research at the secondary level.

The Promise of Increasing Engagement

The current study positively shifted classroom wide engagement from the fall to the spring—although the shift was modest, given that participation in the intervention explained 4% of the variance in change in engagement across the school year. The modest findings remain noteworthy in light of the ongoing quest to identify which characteristics differentiate teachers who alter student academic trajectories. Recently, researchers have turned to actual instructional practices in the classroom because more distal teacher characteristics, such as years of teaching experience or education level, have not differentiated high quality instructors (e.g., Shacter & Thum, 2004). As an intervention, MTP-S focuses on improving developmentally appropriate teacher-student interactions through a structured feedback process. The coach observes videorecorded tapes of the teachers’ instruction throughout the school year using a validated observational tool. The coach focuses on the interactions between teachers and students, as guided by the instructional quality dimensions of the CLASS-S. The focus on interactions is promising for future professional development programs and the video assessment and feedback process suggests new mechanisms for professional development. Validated, objective, standardized observational assessment of classroom interactions may be key to teacher professional development aimed at improving classroom effects on student development (Pianta & Hamre, 2009).

Exploring the Mechanisms of Change

For the intervention teachers, changes in a dimension in the Classroom Organization domain (Instructional Learning Formats) and in the Instructional Support domain (Analysis and Problem Solving) were associated with increased engagement from the start of the year to the end of the year. As measured by Instructional Learning Formats, intervention teachers were better able to structure their interactions with adolescents—they improved in their presentation of material and the provision of interesting lessons and materials. They communicated learning objectives, provided clear summaries, and sequenced material logically. The teachers learned to present information in multiple ways, using different materials and modalities (e.g., hands-on activities, peer collaboration). They became more skilled at actively facilitating student involvement by asking students questions, and scaffolding and extending their learning. As measured by observed Analysis and Problem Solving, these teachers also improved in their facilitation of students’ use of higher level thinking skills, such as reasoning and synthesis through the application of knowledge and skills. Teachers improved in how they interacted with students through their instruction. Specifically, they structured learning so students had opportunities to gain a deep level of understanding of the material. For instance, teachers used lessons that offered complex tasks, in which students needed to independently problem solve, generate, and test hypotheses, as well as examine relationships among ideas and synthesize material. Finally, improved teachers typically encouraged “thinking out loud” (meta-cognition) so that students reflected on their own process of learning. Taken together, classrooms high on these dimensions of the CLASS-S reflect a developmentally appropriate instructional context for adolescents.

Noteworthy is that the current study’s results corroborate basic research that has shown a link between the task characteristics of classrooms and student behavioral engagement (Fredericks et al., 2004). Students are more engaged when they use higher-order thinking skills as they tackle cognitively challenging tasks (National Research Council, 2005; Newmann, Wehlage, & Lamborn, 1992; Stodolsky, 1988). For instance, a recent study showed that in a sample of 24 schools students who reported that the work was cognitively challenging tended to be more engagement in classroom activities (Marks, 2000). Observational studies corroborate such findings. Students have been observed as more engaged in complex, hands-on tasks when teachers present novel classroom tasks (Helme & Clarke, 2001) and when teachers push for a deeper level of understanding (Blumenfeld & Meece, 1988).

Emotional Support in High School Classrooms

An unexpected finding was that the positive shift in the dimensions in the Emotional Support domain of the CLASS-S (e.g., Positive Climate, Teacher Sensitivity) for MTP-S intervention teachers did not reach statistical significance. The finding might be considered in light of the positive changes in the Organization and Instructional Support domains (i.e., Instructional Learning Formats, and Analysis and Problem Solving). We speculate that secondary teachers’ organization and implementation of their lessons may be more malleable than the socioemotional aspects of their interactions with adolescents. In other words, secondary teachers’ social and emotional competence may be a more stable and less malleable individual characteristic (Jennings & Greenberg, 2008). This postulation would need to be reconciled with the recent MTP Pre-K findings, in which the intervention was able to change teacher sensitivity, one of the dimensions of the Emotional Support domain (Pianta et al., 2008). The conflicting findings may be due to differences in the individual characteristics of pre-K versus secondary teachers or differing structural characteristics of their work settings. We speculate that different types of individuals gravitate to teach specific age groups—pre-K teachers may enter the profession more focused on nurturing relationships with young children, whereas secondary teachers may enter the profession more focused on fostering content knowledge with adolescents. As a result, pre-K teachers may be more open to learning how to strengthen emotional connections with their students and secondary teachers less so. Alternatively, pressures in pre-K schooling versus secondary schooling may provide more or less incentive for teachers to change in certain ways—pre-K administrators may place a greater value on improving the emotional climate of the classroom, and secondary school administrators may prefer improving subject area pedagogy and student performance on standardized tests. As a result, compared to pre-K teachers, secondary teachers may have less incentive to change the emotional climate in the classroom and more incentive to change instructional support. Future research is needed to further understand why certain aspects of teacher instruction may be more or less malleable depending on the grade level.

Limitations and Future Directions

A handful of limitations are important to consider. The mediational findings are only suggestive of possible mechanisms of change in the MTP-S program—these findings must be viewed with caution given design and measurement limitations. The design supports causal claims regarding the effects of the intervention on classroom engagement, but it does not support casual claims related to the mediators. The mediational analyses were correlational, and, as a result, cannot definitively confirm that changes in these CLASS-S dimensions caused engagement to change. Also related to the observational measure, the mediational findings must be considered in light of the problems of shared method variance. The engagement ratings and the instructional quality dimensions of the CLASS-S were based on observer ratings conducted on the same videorecording of teacher instruction, which suggests the mediational findings need to be corroborated by measures from multiple informants and sources in future studies. Given the limitations of the mediational analyses, we consider the results to be exploratory and a starting point for future research. Future research on mediation might examine whether increased student engagement helps explain why achievement gains occurred the subsequent year after the MTP-S intervention, as was recently demonstrated using data from the same sample in the current study (Allen, Pianta, Gregory, Mikami, & Lun, 2011).

The study used an observational measure of engagement, which offers a more objective view of participatory behaviors compared to self-report. That said, observational measures of behavioral engagement have been critiqued (Fredricks et al., 2004) given that observers cannot infer the quality of effort or thinking. Some students may be skilled at appearing engaged, while their minds wander. Other students may appear passive and inattentive, when in fact they are keenly participating. Future research might use self-reported cognitive and affective aspects of engagement and school record indicators of engagement (e.g., attendance, school completion) to accompany observed measures of behavioral engagement.

Intervention and control teachers, within a specified data collection window, selected the exact lesson to videorecord and mail to the researchers. What determined which lessons they selected is unknown. Hypothetically, intervention teachers could have watched a CLASS-S dimension exemplified in the video library and then selected a lesson in which they planned to model the same dimension. If implemented well, this might have resulted in higher codes on that dimension. That said, it is important to note that teachers were not explicitly coached in the outcome dimension of the study, student engagement. Thus, the teacher selection of the tapes would not inflate coding of the intervention outcome variable. Moreover, even if a teacher anticipated submitting a “best” lesson, the reality of teaching is such that a carefully planned lesson does not typically “roll out” as expected. In other words, the implemented lesson may not remotely resemble the plan. Alternatively, intervention teachers might have videotaped lessons they perceived as less polished if they felt they could be more vulnerable with a trusted and supportive coach as the intervention year progressed. Future research needs to examine teacher reports of their selection criteria for videorecording lessons or program designers might consider random selection of class periods for recording.

Overall, the MTP-S intervention made somewhat modest changes in engagement and in two of the instructional quality dimensions. In addition, by the spring, most teachers remained in the low quality range for Analysis and Problem Solving, which suggests that teachers have room for improvement. We might consider a range of explanations to understand the modest findings. Teacher behavior and student engagement may simply be hard to substantially shift across one school year. By high school and after years of sitting in classrooms, adolescents may develop somewhat rigid patterns in their own classroom behavior. Students who expect to be disengaged may need more than a single teacher to break such expectations and learned behavior—developmentally sensitive teacher-student interactions may need to occur throughout the school day and across courses (and teachers). In addition, some students may need more intensive remediation plans to help them address skill deficits (e.g., attentional difficulties) in order to increase their engagement.

Implications for School Psychologists

School psychologists typically spend most of their days addressing individual student difficulties (Bramlett, Murphy, Johnson, & Wallingsford, 2002). Yet the School Psychology: BluePrint for Training and Practice III report (Ysseldyke et al., 2006) calls for psychologists to demonstrate competency in systems-based service delivery and build capacity of systems in schools, including a focus on prevention in the classroom system. Barriers to a school psychology systems approach may include lack of practitioner access to empirically validated programs on classroom wide improvement. The current study offers promise in this arena; we showed that teachers can be systematically coached, which results in modest increases in student engagement. In the future, school psychologists could draw on this program and coach teachers in whole-class instructional practices, not just specially tailored interventions to individual children. By doing so, they may prevent individual referrals for special education while benefitting the class as a whole.

Summary

The current study contributes new knowledge about the efficacy of a teacher professional development program in increasing classroom engagement (to a modest degree) in middle and high schools. It is among the few randomized control trials to rigorously test whether ongoing, personalized coaching and feedback for teachers based on validated, observational assessment of the classroom environment affect student outcomes. Findings showed that intervention teachers, compared to controls, had somewhat higher observed engagement at the end of the school year, relative to the beginning of the school year. Noteworthy is that the findings held for a sample of diverse classrooms which varied considerably on class size, poverty, racial and ethnic composition, and fall achievement level.

Acknowledgments

This study and its writeup were supported by grants from the William T. Grant Foundation and the Institute for Education Science (R305A100367). The views in this article do not necessarily reflect policies or recommendations of the funding agencies. The authors are grateful for the teachers and students who participated in this study and the many research assistants and staff who contributed to this project, including Judith Wasserman, Sharon Deal, Marla Capper, Ruth Ferree, and Kathy Neesen.

Contributor Information

ANNE GREGORY, Rutgers University.

JOSEPH P. ALLEN, University of Virginia

AMORI Y. MIKAMI, University of British Columbia

CHRISTOPHER A. HAFEN, University of Virginia

ROBERT C. PIANTA, University of Virginia

References

  1. Aiken LS, West SG. Multiple regression: testing and interpreting interactions. Thousand Oaks, CA: Sage Publications; 1991. [Google Scholar]
  2. Alexander KL, Entwisle DR, Horsey CS. From first grade forward: Early foundations of high school dropout. Society of Education. 1997;70:87–107. doi: 10.2307/2673158. [DOI] [Google Scholar]
  3. Allen JP, Hauser ST, Bell KL, O’Connor TG. Longitudinal assessment of autonomy and relatedness in adolescent-family interactions as predictors of adolescent ego development and self-esteem. Child Development. 1994;65(1):179–194. doi: 10.1111/j.1467-8624.1994.tb00743.x. [DOI] [PubMed] [Google Scholar]
  4. Allen JP, Kuperminc G, Philliber S, Herre K. Programmatic prevention of adolescent problem behaviors: The role of autonomy, relatedness, and volunteer service in the teen outreach program. American Journal of Community Psychology. 1994;22:617–638. doi: 10.1007/BF02506896. [DOI] [PubMed] [Google Scholar]
  5. Allen JP, Marsh P, McFarland C, McElhaney KB, Land DJ, Jodl KM, et al. Attachment and autonomy as predictors of the development of social skills and delinquency during midadolescence. Journal of Consulting & Clinical Psychology. 2002;70(1):56–66. doi: 10.1037//0022-006X.70.1.56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Allen JP, Philliber S, Herrling S, Kuperminc GP. Preventing teen pregnancy and academic failure: Experimental evaluation of a developmentally based approach. Child Development. 1997;68:729–742. doi: 10.1111/j.1467-8624.1997.tb04233.x. [DOI] [Google Scholar]
  7. Andriessen I, Phalet K, Lens W. Future goal setting, task motivation and learning of minority and non-minority students in Dutch schools. British Journal of Educational Psychology. 2006;76:827–850. doi: 10.1348/000709906x148150. [DOI] [PubMed] [Google Scholar]
  8. Archambault I, Janosz M, Morizot J, Pagani L. Adolescent behavioral, affective, and cognitive engagement in school: Relationship to dropout. Journal of School Health. 2009;79:409–415. doi: 10.1111/j.1746-1561.2009.00428.x. [DOI] [PubMed] [Google Scholar]
  9. Allen JP, Gregory A, Mikami AY, Lun J, Hamre B, Pianta RC. Unpublished manuscript. University of Virginia; Charlottesville, VA: 2012. Observations of effective teacher-student interactions in secondary school classrooms: Predicting student achievement with the CLASS-S. [PMC free article] [PubMed] [Google Scholar]
  10. Allen JP, Pianta RC, Gregory A, Mikami AY, Lun J. An Interaction-based approach to enhancing secondary school instruction and student achievement. Science. 2011;19:1034–1037. doi: 10.1126/science.1207998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bandura A, Barbaranelli C, Caprara GV, Pastorelli C. Multifaceted impact of self-efficacy beliefs on academic functioning. Child Development. 1996;67:1206–1222. doi: 10.1111/j.1467-8624.1996.tb01791.x. [DOI] [PubMed] [Google Scholar]
  12. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986;51:1173–1182. doi: 10.1037//0022-3514.51.6.1173. [DOI] [PubMed] [Google Scholar]
  13. Battistich V, Watson M, Solomon D, Lewis C, Schaps E. Beyond the three R’s: A broader agenda for school reform. The Elementary School Journal. 1999;99:415–432. doi: 10.1086/461933. [DOI] [Google Scholar]
  14. Beck AT. Cognitive therapy and the emotional disorders. New York: International Universities Press; 1976. [Google Scholar]
  15. Bergan JR, Kratochwill TR. Behavioral consultation and therapy. New York: Plenum; 1990. [Google Scholar]
  16. Betts JE, Appleton JJ, Reschly AL, Christenson SL, Huebner ES. A study of the factorial invariance of the student engagement instrument (SEI): Results from middle and high school students. School Psychology Quarterly. 2010;25:84–93. doi: 10.1037/a0020259. [DOI] [Google Scholar]
  17. Blumenfeld PC, Meece JL. Task factors, teacher behavior, and students’ involvement and use of learning strategies in science. The Elementary School Journal. 1988;88:235–250. doi: 10.1086/461536. [DOI] [Google Scholar]
  18. Bowlby J. Attachment and Loss, Vol. 1: Attachment. Vol. 1968 New York: Basic Books; 1968. [Google Scholar]
  19. Bramlett RK, Murphy JJ, Johnson J, Wallingsford L. Contemporary practices in school psychology: A national survey of roles and referral problems. Psychology in the Schools. 2002;39(3):327–335. [Google Scholar]
  20. Brophy J. Classroom management as socializing students into clearly articulated roles. Journal of Classroom Interaction. 1988;33:1–4. [Google Scholar]
  21. Carroll T, Fulton K, Doerr H. Team-up for 21st century teaching and learning: What research and practice reveal about professional learning. Washington, DC: National Commission on Teaching and America’s Future; 2010. [Google Scholar]
  22. Cicchetti DV, Sparrow SA. Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency. 1981;86:127–137. [PubMed] [Google Scholar]
  23. Cohen J, Cohen P. Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum; 1983. [Google Scholar]
  24. Dane AV, Schneider BA. Program integrity in providing primary and early secondary prevention. Are implementation effects out of control? Clinical Psychology Review. 1998;18(1):23–45. doi: 10.1016/s0272-7358(97)00043-3. [DOI] [PubMed] [Google Scholar]
  25. Domitrovich CE, Greenberg MT. Introduction to the special issue: Preventive interventions with young children: Building on the foundation of early intervention programs. Early Education and Development. 2004;15:365–370. doi: 10.1207/s15566935eed1504_1. [DOI] [Google Scholar]
  26. Doyle W. Ecological approaches to classroom management. In: Evertson CM, Weinstein CS, editors. Handbook of classroom management. Mahwah, NJ: Lawrence Erlbaum Associates; 2006. pp. 127–156. [Google Scholar]
  27. Eccles JS, Midgley C, Wigfield A, Buchanan CM, Reuman D, Flanagan C, et al. Development during adolescence: The impact of stage-environment fit on young adolescents’ experiences in schools and in families. American Psychologist. 1993;48:90–101. doi: 10.1037/10254-034. [DOI] [PubMed] [Google Scholar]
  28. Ellis A. Reason and emotion in psychotherapy. New York: Birch Lane Press; 1994. Revised and updated. [Google Scholar]
  29. Emmer ET, Stough LM. Classroom management: A critical part of educational psychology, with implications for teacher education. Educational Psychologist. 2001;36:103–112. doi: 10.1207/S15326985EP3602_5. [DOI] [Google Scholar]
  30. Fincham FR, Hokoda A, Sanders R., Jr Learned helplessness, test anxiety, and academic achievement: A longitudinal analysis. Child Development. 1989;60:138–145. doi: 10.2307/1131079. [DOI] [PubMed] [Google Scholar]
  31. Finn JD. School engagement and students at risk. Washington DC: National Center for Educational Statistics; 1993. Retrieved from http://nces.ed.gov/pubs93/93470a.pdf. [Google Scholar]
  32. Finn JD, Cox D. Participation and withdrawal among fourth grade pupils. American Educational Research Journal. 1992;29:141–162. doi: 10.3102/00028312029001141. [DOI] [Google Scholar]
  33. Fisher CW, Berliner DC, Filby NN, Marliave R, Cahen LS, Dishaw MM. Teaching behaviors, academic learning time, and student achievement: An overview. Journal of Classroom Interaction. 1981;17:2–15. [Google Scholar]
  34. Fredricks JA, Blumenfeld PC, Paris AH. School engagement: Potential of the concept, state of the evidence. Review of Educational Research. 2004;74:59–109. doi: 10.3102/00346543074001059. [DOI] [Google Scholar]
  35. Greene BA, Miller RB, Crowson MH, Duke BL, Akey KL. Predicting high school students’ cognitive engagement and achievement: Contributions of classroom perceptions and motivation. Contemporary Educational Psychology. 2004;29:462–482. doi: 10.1016/j.cedpsych.2004.01.006. [DOI] [Google Scholar]
  36. Greenwood CR, Horton BT, Utley CA. Academic engagement: Current perspectives on research and practice. School Psychology Review. 2002;31:328–349. [Google Scholar]
  37. Gregory A, Weinstein RS. Connection and regulation at home and in school: Predicting growth in achievement for adolescents. Journal of Adolescent Research. 2004;19:405–427. doi: 10.1177/0743558403258859. [DOI] [Google Scholar]
  38. Guo S. Analyzing grouped data with hierarchical linear modeling. Children and Youth Services Review. 2005;27:637–652. doi: 10.1016/j.childyouth.2004.11.017. [DOI] [Google Scholar]
  39. Hambleton RK, Crocker L, Cruse K, Dodd B, Plake BS, Poggio J. Review of selected technical characteristics of the Virginia Standard of Learning (SOL) assessments. 2000 Retrieved from http://www.doe.virginia.gov/VDOE/Assessment/virginiareport.pdf.
  40. Hamre BK, Pianta RC. Early teacher–child relationships and the trajectory of children’s school outcomes through eighth grade. Child Development. 2001;72:625–638. doi: 10.1111/1467-8624.00301. [DOI] [PubMed] [Google Scholar]
  41. Hamre BK, Pianta RC. Classroom environments and developmental processes: Conceptualization and measurement. In: Meece J, Eccles J, editors. Handbook of research on schools, schooling, and human development. New York: Routledge; 2010. pp. 25–41. [Google Scholar]
  42. Hart P, Stroot S, Yinger R, Smith S. Meeting the teacher education accountability challenge: A focus on novice and experienced teacher studies. Mount Vernon, OH: Teacher Quality Partnership; 2005. [Google Scholar]
  43. Helme S, Clarke D. Identifying cognitive engagement in the mathematics classroom. Mathematics Educational Research Journal. 2001;13:133–153. [Google Scholar]
  44. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomized controlled trials. British Medical Journal. 1999;319:670–674. doi: 10.1136/bmj.319.7211.670. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Jennings PA, Greenberg MT. The prosocial classroom: Teacher social and emotional competence in relation to student and classroom outcomes. Review of Educational Research. 2009;79:491–525. doi: 10.3102/0034654308325693. [DOI] [Google Scholar]
  46. Laffey JM. The assessment of involvement with school work among urban high school students. Journal of Educational Psychology. 1982;74:62–71. doi: 10.1037/0022-0663.74.1.62. [DOI] [Google Scholar]
  47. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. doi: 10.2307/2529310. [DOI] [PubMed] [Google Scholar]
  48. MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Annual Review of Psychology. 2007;58:593–614. doi: 10.1146/annurev.psych.58.110405.085542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Marks HM. Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. American Educational Research Journal. 2000;37:153–184. doi: 10.2307/1163475. [DOI] [Google Scholar]
  50. Mashburn A, Downer J, Hamre B, Justice L, Pianta R. Consultation for teachers and children’s language and literacy development during pre-kindergarten. Applied Developmental Science. 2010;14:179–196. doi: 10.1080/10888691.2010.516187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. McCaslin M, Burross HL, Good TL. Change and continuity in student achievement from grades 3 to 5: A policy dilemma. Education Policy Analysis Archives. 2005;13(1) Retrieved from http://epaa.asu.edu/epaa/v13n1/ [Google Scholar]
  52. Muthén LK, Muthén B. Mplus: Statistical Analysis with Latent Variables, User’s Guide. Los Angeles: Muthén & Muthén; 2001. [Google Scholar]
  53. My Teaching Partner Secondary Consultancy Manual. Unpublished manuscript. University of Virginia; Char-lottesville, VA: 2010. [Google Scholar]
  54. National Research Council. Engaging schools: Fostering high school students’ motivation to learn. Washington, DC: National Academies Press; 2004. [Google Scholar]
  55. National Research Council. How students learn: History, mathematics, and science in the classroom. Washington, DC: National Academies Press; 2005. [Google Scholar]
  56. Newmann FM, Wehlage GG, Lamborn S. The significance and sources of student engagement. In: Newmann F, editor. Student engagement and achievement in American secondary schools. New York: Teachers College Press; 1992. pp. 11–39. [Google Scholar]
  57. NICHD Early Child Care Research Network [ECCRN] The relation of global first-grade classroom environment to structural classroom features and teacher and student behaviors. The Elementary School Journal. 2002;102:367–387. doi: 10.1086/499709. [DOI] [Google Scholar]
  58. NICHD Early Child Care Research Network. A day in third grade: A large-scale study of classroom quality and teacher and student behavior. The Elementary School Journal. 2005;105:305–323. doi: 10.1086/428746. [DOI] [Google Scholar]
  59. Pianta R. Standardized observation and professional development: A focus on individualized implementation and practices. In: Zaslow M, Martinez-Beck I, editors. Critical issues in early childhood professional development. Baltimore: Paul H. Brookes Publishing; 2005. pp. 231–254. [Google Scholar]
  60. Pianta RC, Hamre BK. Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher. 2009;38:109–119. doi: 10.3102/0013189x09332374. [DOI] [Google Scholar]
  61. Pianta RC, Hamre B, Allen J. Teacher-student relationships and engagement: Conceptualizing, measuring, and improving the capacity of classroom interactions. In: Christenson SL, Reschly AL, Wylie C, editors. Handbook of research on student engagement. New York: Springer; 2012. pp. 365–386. [Google Scholar]
  62. Pianta RC, Hamre B, Hayes N, Mintz S, LaParo KM. Classroom Assessment Scoring System–Secondary (CLASS-S) University of Virginia; Charlottesville, VA: 2008. [Google Scholar]
  63. Pianta RC, Hamre B, Stuhlman M. How schools can do better: Fostering stronger connections between teachers and students. In: Rhodes JE, editor. New directions for youth development: A critical view of youth mentoring. San Francisco: Jossey-Bass; 2002. pp. 91–107. [DOI] [PubMed] [Google Scholar]
  64. Pianta RC, Kinzie M, Pullen P, Justice L, Lloyd J, Fan X. Web training: Pre-k teachers, literacy, and relationships. Washington, DC: National Institute of Child Health and Human Development; 2003. Grant No. 1 R01 HD046061–01. [Google Scholar]
  65. Pianta RC, Mashburn AJ, Downer JT, Hamre BK, Justice L. Effects of web-mediated professional development resources on teacher–child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly. 2008;23:431–451. doi: 10.1111/j.1467-8624.2008.01154.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Preacher KJ, Zyphur MJ, Zhang Z. A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods. 2010;15:209–233. doi: 10.1037/a0020141. [DOI] [PubMed] [Google Scholar]
  67. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage; 2002. [Google Scholar]
  68. Raudenbush SW, Martinez A, Bloom H, Zhu P, Lin F. An Eight-step paradigm for studying the reliability of group-level measures. 2008 Report from the William T. Grant Foundation, Retrieved from http://www.wtgrantfoundation.org/publications_and_reports/featured_reports.
  69. Reinke WM, Lewis-Palmer T, Merrell K. The classroom check-up: A class wide teacher consultation model for increasing praise and decreasing disruptive behavior. School Psychology Review. 2008;37:315–332. [PMC free article] [PubMed] [Google Scholar]
  70. Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist. 2000;55:68–78. doi: 10.1037//0003-066X.55.1.68. [DOI] [PubMed] [Google Scholar]
  71. Schacter J, Thum YM. Paying for high- and low-quality teaching. Economics of Education Review. 2004;23:411–430. doi: 10.1016/j.econedurev.2003.08.002. [DOI] [Google Scholar]
  72. Shernoff DJ, Schmidt JA. Further evidence of an engagement-achievement paradox among U.S. high school students. Journal of Youth and Adolescence. 2008;37:564–580. [Google Scholar]
  73. Skinner EA, Belmont MJ. Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology. 1993;85:571–581. doi: 10.1037/0022-0663.85.4.571. [DOI] [Google Scholar]
  74. Skinner EA, Zimmer-Gembeck MJ, Connell JP. Individual differences and the development of perceived control. Monographs of the Society for Research in Child Development. 1998;63:1–220. [PubMed] [Google Scholar]
  75. Stodolsky SS. The subject matters: Classroom activity in math and social studies. Chicago: The University of Chicago Press; 1988. [Google Scholar]
  76. Swift MS, Spivack G. Achievement-related classroom behavior of secondary school normal and disturbed students. Exceptional Children. 1969;35:677–684. [PubMed] [Google Scholar]
  77. Wang MC, Haertel GD, Walberg HJ. Learning influences. In: Walberg HJ, Haertel GD, editors. Psychology and Educational Practice. Berkeley: McCutchan; 1997. pp. 199–211. [Google Scholar]
  78. Yair G. Educational battlefields in America: The tug-of-war over student’s engagement with instruction. Sociology of Education. 2000;73:247–269. doi: 10.2307/2673233. [DOI] [Google Scholar]
  79. Yoon KS, Duncan T, Lee SW-Y, Scarloss B, Shapley K. Reviewing the evidence on how teacher professional development affects student achievement. Washington, DC: U.S. Department of Education, Institute of Education Science, National Center for Education Evaluation and Regional Assistance, Regional Laboratory Southwest; 2007. (Issues & Answers Report, REL 2007-No. 033) Retrieved from http://ies.ed.gov/ncee/edlabs/regions/southwest/pdf/REL_2007033.pdf. [Google Scholar]
  80. Ysseldyke J, Burns M, Dawson P, Kelley B, Morrison D, Ortiz S, et al. School psychology: BluePrint for training and practice III. Bethesda, MD: 2006. Report from the National Association of School Psychologists. [Google Scholar]

RESOURCES