An Interaction-Based Approach to Enhancing Secondary School Instruction and Student Achievement

Abstract

Improving teaching quality is widely recognized as critical to addressing deficiencies in secondary school education, yet the field has struggled to identify rigorously evaluated teacher-development approaches that can produce reliable gains in student achievement. A randomized controlled trial of My Teaching Partner–Secondary—a Web-mediated approach focused on improving teacher-student interactions in the classroom—examined the efficacy of the approach in improving teacher quality and student achievement with 78 secondary school teachers and 2237 students. The intervention produced substantial gains in measured student achievement in the year following its completion, equivalent to moving the average student from the 50th to the 59th percentile in achievement test scores. Gains appeared to be mediated by changes in teacher-student interaction qualities targeted by the intervention.

In the context of education reform and efforts to raise student achievement, the development of effective teaching and teachers in secondary schools is of central importance. In large-scale testing programs, teacher quality is the greatest source of variation in what students learn as a function of attending school (1). Yet, teacher qualifications (e.g., degrees, experience, certifications, and teacher test performance) show only modest relations to student achievement (2, 3).

Despite the obvious importance of improving secondary school education, reviews by both the What Works Clearinghouse (4, 5) and the Johns Hopkins Best Evidence Encyclopedia (6) of published reports of teacher professional development efforts on secondary school student achievement find, respectively, either no programs or only two programs that document substantial impact on student achievement using fully rigorous designs. Even the two programs documenting substantial impact were limited solely to mathematics education.

In secondary schools, one of the largest potential mediators of academic outcomes is the extent to which students are motivated and engaged by their interactions with teachers, but this factor has received relatively little attention (7–10). Students themselves report interactions with teachers to be critical to their success and yet often of very poor quality (11, 12). Student motivation in school begins to decline as early as age 11, and by entry into high school more than half of students from all types of schools report that they do not take their school or their studies seriously (13, 14). Disengagement in the classroom is related to low academic achievement, disruptive and uncooperative behavior, missed instructional time, and ultimately to school failure (7, 15–17).

This study reports results of a randomized controlled trial of a coaching program—the My Teaching Partner–Secondary program (MTP-S)—focused on improving teacher-student interactions in secondary classrooms with students aged 11 to 18 so as to enhance student motivation and achievement. The program targets the motivational and instructional qualities of teachers’ ongoing, daily interactions with students. MTP-S is conceptualized within the Teaching Through Interactions framework (fig. S1), a content-independent framework that emphasizes the extent to which student-teacher interactions influence student academic motivation, effort, and achievement (18).

MTP-S uses the domains of the Classroom Assessment Scoring System–Secondary (CLASS-S) (19) to operationalize this framework by providing clear behavioral anchors for describing, assessing, and intervening to change critical aspects of classroom interactions. These domains focus on the extent to which interactions build a positive emotional climate and demonstrate sensitivity to student needs for autonomy, an active role in their learning, and a sense of the relevance of course content to their lives. Focus is also placed on bolstering the use of varied instructional modalities and engaging students in higher-order thinking and opportunities to apply knowledge to problems. Overall, the intervention is designed to enhance the fit between teacher-student interactions and adolescents’ developmental, intellectual, and social needs in an approach that aligns closely with elements of high-quality teaching that have been identified as central to student achievement (9).

The MTP-S intervention integrates initial workshop-based training, an annotated video library, and a year of personalized coaching followed by a brief booster workshop. During the school year, teachers send in video recordings of class sessions in which they are delivering a lesson. Trained teacher consultants review recordings that teachers submit and select brief segments that illustrate either positive teacher interactions or areas for growth in one of the dimensions in the CLASS-S. These are posted on a private, password-protected Web site, and each teacher is asked to observe his or her behavior and student reactions and to respond to consultant prompts by noting the connection between the two. This is followed by a 20- to 30-minute phone conference in which the consultant strategizes with the teacher about ways to enhance interactions using the CLASS-S system. This cycle repeats about twice a month for the duration of the school year.

We hypothesized that changes in the capacity of the teacher to generate high-quality teacher-student interactions would lead to student achievement gains. We expected changes to accumulate over the course of the year during which teachers were exposed to the intervention, with most student instruction time occurring before the point at which the greatest changes were expected. We thus focused our evaluation on whether changes in student achievement would be observed in the second year of the study, with a new class of students and no further coaching of the teacher, as a test of whether the intervention produced generalizable and sustainable changes in teaching. We also assessed whether program effects differed across subject matter or different populations of adolescents.

This study included 78 secondary school teachers (28 male and 50 female) from 12 schools who participated for 13 months in MTP-S and for a total of 2 years in the evaluation of the program. Teachers were randomly assigned to participate in either the intervention or regular in-service training. Participating teachers had an average of 8.7 years of teaching experience (SD = 8.8). Teacher racial and ethnic composition was 83% white, 8% African-American, 6% mixed ethnicity, and 3% other. Thirty-five percent of teachers had a terminal B.A. degree, and 65% had advanced education beyond the B.A. degree. There were no demographic differences between intervention and control group teachers in either year of the study. In the intervention year, 1267 students in 76 classrooms participated; in the post-intervention year, 970 additional students in 61 classrooms participated. There were no differences between intervention and control group students in terms of gender, middle versus high school attendance, racial and ethnic background, family poverty status, or baseline achievement test scores (table S1). Student achievement was assessed at the end of each course with the Virginia state standards assessment instrument applicable to the course being taught (20, 21); baseline achievement was assessed with performance on the standardized end-of-year test from the most comparable course in the previous year. We conducted assessments for both the intervention year and the post-intervention year.

Analyses used hierarchical linear models to account for the nesting of students within teachers’ classrooms. To assess the main effect of the intervention, we examined differences in end-of-year student achievement test scores for the intervention versus the control group, after first accounting for predictions from achievement test scores from the previous year and teacher and student demographic characteristics. Results indicate a nonsignificant effect of intervention on end-of-year test scores in the intervention year but a significant positive effect in the post-intervention year (table S2, and Fig. 1). Students in the MTP-S intervention had a significant net gain relative to the control group of 0.22 SD. This equates to an average increase in student achievement from the 50th to the 59th percentile for a student moved from the control condition to the intervention condition.

Fig. 1.

MTP-S effect on student achievement. Mean achievement test scores for Intervention and Control group students from the most comparable previous-year course (Pre-test) and the current year’s focal class (Post-test), adjusted for baseline demographic factors using hierarchical linear modeling (HLM) (table S2). Error bars reflect SEM from HLM.

The potential mediating role of observed teacher-student interaction qualities was assessed with a multilevel structural equation modeling framework (22). Interaction qualities were observed at the end of the intervention year, with analyses examining whether they potentially reflected an enduring change in classroom qualities that would mediate effects of the intervention on achievement for a new class of students in the post-intervention year. This analysis revealed a significant indirect effect of the intervention on student achievement in the post-intervention year through changes in teacher-student interaction qualities, consistent with a mediating role for these qualities (Fig. 2).

Fig. 2.

MTP-S effect in the post-intervention year as mediated by observed teacher-student interactions. *, P < .05; **, P < .01.

Results revealed no interaction of intervention effectiveness with subject area (e.g., math/science versus English/social studies) (all P’s > 0.10). This indicates that there was no evidence, albeit in a design with relatively modest power to detect interaction effects, that the effectiveness of the intervention differed depending upon the subject matter of the class in which it was implemented. Similarly, we found no evidence of differential intervention effectiveness for teachers who did or did not teach a different course (e.g., World History instead of U.S. History) in the second year of the intervention (all P’s > 0.10). Finally, no differences in the effectiveness of the intervention were observed across classrooms or teachers with different sociodemographic and structural characteristics.

These results show that a developmentally informed intervention can alter the nature of teacher-student interaction in secondary school classrooms to produce student achievement gains. The MTP-S program changed teacher behavior, and it led to gains in student achievement with a new class of students that had not been the focus of intervention efforts.

Mediation analyses that followed up on the primary study findings yielded results consistent with the interpretation that the operative mechanism of the intervention was indeed the specific qualities of teacher-student interaction that were the primary focus of the intervention. These qualities of teacher-student interactions were the direct targets of the intervention, they were predicted by participation in the intervention, and an indirect effect of the intervention on student achievement through these observed qualities was observed.

The finding that improved teacher-student interactions predicted improved student achievement regardless of the content area of instruction suggests the potential value of a focus on teacher-student interactions, apart from the specific content of knowledge being transmitted by teachers. This is in keeping with the fundamental theoretical assumption underlying the intervention: that increasing the extent to which interactions in secondary school classrooms are tailored to adolescents’ developmental needs will enhance both student motivation and achievement. These results suggest that, although it is obviously necessary to know math to teach math, in secondary school classrooms teaching math skillfully also involves successfully relating to and interacting with students so as to enhance their academic motivation (23).

A key feature of secondary education, too often overlooked, is that, unlike education in the primary grades, one cannot assume that adolescent students arrive at school with an intrinsic desire to please adult authority figures. On the contrary, autonomy struggles are a central facet of adolescent social development that can undermine teacher-student relationships unless handled sensitively (24, 25). Further, although students in primary grades can readily see how the ability to read, write, and perform basic arithmetic operations are used in the adult world, the links between the secondary school curriculum and daily adult life may appear more tenuous to adolescents. MTP-S directly targets the resulting motivational challenge and is a promising route for starting to tackle the seemingly intractable problem of adolescent underachievement in secondary school.

The effects of the intervention on teacher-student interactions at the end of the intervention year did not translate into statistically significant gains in student achievement until the post-intervention year. This result lends a cautionary note to these findings. It is, however, consistent with the idea that student gains in achievement would occur only after teachers had the benefit of a year’s worth of their own growth, such that students would actually experience enhanced teacher-student interactions over a substantial portion of their academic year. That these effects on teachers carried into the next year and new students, when there was no coaching and 30% of the teachers were teaching at least slightly different content material than in the first year, suggests that effects were driven by enduring change to the teacher and to the classroom as a behavior setting, not by student effects limited to the intervention year and class.

The intervention appears to be cost-effective. In terms of total teacher time, the intervention required approximately 20 hours of in-service training, spread across 13 months. The full cost for the teacher-consultants and video equipment was $3700 per teacher over this period. Such costs compare favorably to the annual $2000 to $7,000 typically spent each year on teacher in-service training (26). Given that effects were found in the next-year classroom, which was not the target of the intervention, we assume that effects might generalize across a teacher’s entire course load (typically five or more classes of 20 to 25 students each), thus reducing the per student costs to under $40 per student for a 9 percentile upward bump in academic performance.

Limitations to these findings should also be noted. The lack of effects on student achievement in the intervention year suggests the difficulty of rapidly changing classrooms in ways that leads to student achievement gains. Also, although the experimental design supports causal attributions regarding the effects of the intervention, the analyses of mediating processes extend beyond this experimental design; these analyses could thus disconfirm causal hypotheses about mediation but cannot directly confirm them. In addition, because teachers selected their focal class in the post-intervention year (albeit with clear guidance to select their most challenging course), it remains possible that this selection in some unmeasured way biased results of the study. Similarly, although analyses indicated no evidence of any attrition effects or initial sample differences impairing study validity, unmeasured biases due to such effects cannot be definitively ruled out. Finally, further replication within other school systems with different structural and demographic characteristics (e.g., class sizes and student socioeconomic status) is warranted.