Implementation quality impacts fourth grade students’ participation in physically active academic lessons

Abstract

Background:

It is important to assess implementation of active learning interventions to maximize their impact. Implementation quality, or how well one engages program participants, has been less studied than other implementation components (e.g., dose, fidelity). This cross-sectional, exploratory study examined associations between teacher engagement behaviors, quality of teacher engagement (i.e., teacher feedback), and student physical activity outcomes during active classroom lessons.

Methods:

This study used data from the Texas Initiatives for Children’s Activity and Nutrition (I-CAN!) randomized controlled trial. Fixed effects regressions investigated the impact of teacher engagement behaviors on student physical activity outcomes. Bivariate correlations examined associations between teacher feedback and student physical activity outcomes. A latent profile analysis explored whether there were subsets of teachers with similar feedback profiles.

Results:

The final analytic sample included 82 teachers. Teacher-directed changes and teacher participation in physical activity were each associated with higher ratings for how many and how often children were active during lessons. Teacher participation in physical activity was also significantly related to higher ratings for student physical activity intensity (all p<.05). Physical Activity Reinforcement and Technical Instruction feedback were positively associated with activity intensity (r=−.20, p<.05 and r=.34, p<.01, respectively). Technical Instruction feedback was positively associated with how many (r=.25, p<.01) and how often (r=.41, p<.01) students were active during lessons. Negative feedback was negatively associated with how often (r=−.25, p<.05) students were active and activity intensity (r=−.25, p<.05). Game Instruction was negatively related to how often students were active (r=−.23, p<.05). All teachers were represented by high levels of Game Instruction and Classroom Management feedback, moderate levels of Content Reinforcement and Content Instruction feedback, and low levels of Negative, Technical Instruction, and Physical Activity Reinforcement feedback. These data did not indicate the existence of multiple feedback profiles.

Conclusions:

Findings suggest that teacher engagement and feedback to students during active lessons can promote student physical activity. Teachers are primarily responsible for implementing school-based interventions, so it is critical to develop strategies that increase their ability to implement them successfully. Opportunities to maximize intervention delivery, such as co-designing with teachers, should be utilized when designing school-based, physical activity interventions.

Background

Physical activity (PA) is associated with numerous health benefits for children including lower body fat, stronger muscles and bones, improved cognition, academic tasks and attention control, reduced depression, and prevention of chronic disease (Piercy et al. 2018; de Greeff et al. 2018). PA guidelines recommend that children achieve 60 minutes of moderate-to-vigorous intensity physical activity (MVPA) daily (Piercy et al. 2018), yet only 42.5% of children aged 6–11 years meet these guidelines, declining to 5.1% by adolescence (Katzmarzyk et al. 2016). Children spend up to 30 hours each week at school and a majority (73%) of this time is spent sedentary. Thus, school-based interventions provide an optimal environment for targeting children’s PA (Carson et al. 2014).

Physical education (PE) is the foundation for promoting PA in schools (Centers for Disease Control and Prevention, 2013), with a recommendation of 150 min of weekly PE with 50% of time spent in MVPA (SHAPE America, 2015). Unfortunately, many elementary schools fall short, with students spending only 38.4% of time in MVPA during PE (McKenzie and Smith 2017). Additionally, opportunities for school PA have been decreased or eliminated (Sallis et al. 2012). In response, physically active learning has emerged as a strategy to increase children’s PA during school. This strategy incorporates PA within the regular curriculum and is implemented by teachers (Norris et al. 2015). Unfortunately, the impact of these interventions is variable, with increases in MVPA ranging from 2%−16% (standardized mean difference = 0.40, 95% CI: −0.15,0.95; Watson et al. 2017).

There are concerns about the effectiveness of school-based health interventions implemented outside of highly controlled studies or efficacy trials (Durlak 2015). Identification of factors that impact program implementation is essential to prevent “Type III error”, that is, concluding that a program is not effective when, instead, it was not implemented completely and/or correctly (Dusenbury et al. 2003). Thus, before a program can be widely disseminated, it is critical to study how, why, and under which conditions a program works.

The integrated model of program implementation (Berkel et al. 2011) postulates that fidelity (i.e., adherence to program), quality of delivery (i.e., how well implementers engage participants in the program), and adaptation (i.e., changes made to program during implementation), are facilitators of participant responsiveness (i.e., engagement or participation). Higher quality of delivery (hereafter referred to as implementation quality) is associated with greater participant engagement in school-based health interventions (Cross et al. 2010; Humphrey et al. 2018). Yet, outside of dose (Naylor et al. 2015) and fidelity (Watson et al. 2017), implementation quality is understudied among school-based PA interventions. As teachers are the primary implementers of active lesson interventions, their ability to engage students might contribute to variation in children’s PA outcomes across studies.

Teachers may engage students in active lessons through behaviors such as modeling (i.e., participating in activities with students), visual scanning (i.e., frequently scan their class looking for students engaging in appropriate and inappropriate behaviors), and moving around the classroom (i.e., walk among students, visit problem areas, and remain visible to all students), which are intended to prevent students from engaging in problem behaviors and reinforce appropriate behaviors (Gage et al. 2020). Modeling, visual scanning, and moving around the classroom have been directly linked to time students spend in MVPA during PE (Chow et al. 2009; Martin and Kulinna 2005; Fairclough et al. 2018). Attempts to assess implementation quality have typically relied on teachers’ self-reports. However, these can be biased (Low et al., 2014) – therefore, there is increased need to objectively measure movement integration in classrooms (Russ et al. 2017). Yet, only one known study has objectively examined associations between teachers’ engagement behaviors during an active lesson intervention and student PA (Gibson et al. 2008). Gibson and colleagues (2008) reported greater PA among students whose teachers participated in lessons (i.e., modeling). However, the limited number of students observed per classroom (n=3) likely missed important variations across student PA responses, and there was no consideration of other forms of teacher behaviors that might promote student engagement.

It is also important to capture verbal interactions between students and teachers during active lessons to assess the quality of teacher engagement. Previous studies of PE interventions indicate that teacher feedback impacts student responsiveness, with students engaging in more MVPA when PE teachers spend more time praising/reinforcing PA and less MVPA when feedback focuses on class management and general instruction (Chow et al. 2009; Martin and Kulinna 2005; Fairclough et al. 2018). Teachers’ patterns of delivery can vary considerably when implementing school-based interventions (Shin et al. 2014). It is crucial to learn how these patterns are related to variance in student PA. Recognizing latent delivery patterns may direct training efforts to maximize optimal delivery of active lessons.

Few tools exist to assess delivery of active lessons. We developed an observational tool, the Initiatives for Children’s Activity and Nutrition Active Lesson Implementation form (I-CAN! AIM), to guide observations of implementation quality and reduce reliance on self-reported behaviors. This tool assesses a wide variety of factors including the location and duration of lesson and types of PA performed during the lesson. The present study focuses on implementation quality (i.e., teacher engagement) and classroom-level student PA (i.e., how many students participated, how often, intensity). The purpose of this exploratory study was to use I-CAN! AIM to 1) Examine relationships between teacher engagement behaviors (e.g., modeling, moving around classroom) and student PA during active lessons; 2) Investigate relationships between quality of teacher engagement (i.e., feedback) variables and student PA; and 3) Explore relationships between quality of teacher engagement variables to determine whether teachers could be organized into groups with similar feedback profiles and examine possible associations between any resulting profiles and student PA. While teacher engagement behaviors and quality of engagement have been linked to student PA during PE lessons, it is unclear whether these behaviors exhibited by a non-PE teacher during active lessons would have the same impact on student PA. Therefore, no hypotheses were generated a priori.

Methods

Overview

This study used data from the Initiatives for Children’s Activity and Nutrition (I-CAN!) randomized controlled trial (RCT), collected across three academic years (i.e., 2012–13, 2013–14, and 2014–15). A complete description of the study design is provided elsewhere (Bartholomew et al. 2017). Briefly, the overall I-CAN! RCT compared the impact of 10–15 minute active lessons on 1,903 fourth grade students’ physical activity, academic achievement, and time-on-task, compared to standard academic lessons in 28 elementary schools (n=19 intervention and n=9 control) and across 149 fourth-grade teachers (n=99 intervention, n=50 control). Teachers in intervention schools were provided structured active lessons (math or spelling) designed to be easily modified for use across a variety of indoor and outdoor settings (Bartholomew and Jowers 2011) that included physical activities such as running, jumping jacks, and push-ups. All fourth grade students were eligible for inclusion in this study. Informed consent was obtained from all teacher participants, and both parental consent and student assent were obtained from all student participants included in the study. All study protocols were approved by the university’s Institutional Review Board. The current study focused only on data from teachers and students in the intervention arm of the I-CAN! study.

Participants

Data were collected from 82 fourth grade teachers and 99 classroom observations from schools assigned to the math (n=9) or language arts (n=10) intervention arms of the I-CAN! study. Some schools were departmentalized (i.e., different teachers for math and language arts), leading to some teachers being observed more than once. Consequently, 17 observations (n=5 math, n=12 language arts) were eliminated, leaving a final sample size of n=82 observations (See Online Resource 1).

Measures

Demographics.

Teachers self-reported sex, race/ethnicity, age, and years teaching. School-level demographics, including race/ethnicity and eligibility for free/reduced lunch (yes/no) were obtained through school records. Eligibility for free/reduced lunch was considered a proxy for socioeconomic status (SES) as students from families with incomes at or below 130% of the poverty income threshold are eligible for free lunches and those between 130–185% are eligible for reduced price lunch.

I-CAN! Active Lesson Implementation Form.

I-CAN! AIM is a direct observation tool to assess teacher engagement behaviors and student PA-related outcomes (Online Resource 2). Categories were derived from the Coaching Behavioral Assessment System (CBAS; (Smith et al. 1977) and the System for Observing Fitness and Instruction Time (SOFIT; (McKenzie et al. 1992). Specifically, teacher engagement behaviors such as modeling, visual scanning, and movement around the classroom were adapted from SOFIT. Quality of engagement content, teacher feedback related to fitness promotion, class management, and general instruction, were adapted from both the CBAS and SOFIT. Since active lessons incorporate PA into academic lessons, additional teacher feedback categories related to academic content (i.e., instruction, reinforcement) were developed and included in the observational tool. Student PA outcomes (e.g., intensity, frequency) were also adapted from SOFIT.

Research staff were trained to use the tool each year prior to field observations by one of the authors of the current study (VLE) whose ratings were used as the gold standard criteria for assessing the accuracy of research staff. These training sessions were conducted at a school not associated with the RCT. Research staff must have obtained an intra-class correlation (ICC) of at least 0.90 with the gold standard before participating in this study. Staff observations of I-CAN! lessons were conducted at least once for each class over the course of the school year. Two staff members continuously observed the teacher during each lesson. Research staff underwent group retraining each year and were randomly paired at each observation to reduce observer drift.

Teacher Engagement Behaviors.

Trained research staff observed whether teachers: 1) moved around the room, 2) scanned the room, 3) directed changes in PA performed during lesson (e.g., jumping jacks, push-ups), or 4) participated in PA during the I-CAN! lesson. Each item was rated dichotomously (Yes/No).

Quality of Engagement.

Two trained research staff observed teacher feedback, including: 1) Physical Activity Reinforcement (e.g. “Good high knees!”); 2) Technical Instruction (e.g. “During push-ups, keep your back straight”); 3) Content Instruction (e.g. “72 divided by 9 equals 8”); 4) Content Reinforcement (e.g. “Good answer!”); 5) Game Instruction (e.g. “You should be skipping right now”); 6) Negative (e.g. “You’re doing it wrong”); and 7) Classroom Management (e.g. “Please be quiet”). Research staff tallied the number of times the teacher gave a particular type of feedback to their students. Counts for each feedback category were summed and averaged across observers. ICCs for continuous variables ranged from 0.89 to 0.98, indicating excellent inter-rater reliability (Cicchetti 1994).

Student Physical Activity.

Student PA-related outcomes, collected using I-CAN! AIM, included: 1) how many children were active [(1) less than half of the class to (3) more than half of the class], 2) how often children were active [(1) not at all to (5) most of the time], and 3) intensity of movement [(1) standing still to (5) running]. For each class, observers’ ratings were averaged for each PA variable. ICCs were 0.94 for how many children were active, and 0.93 for how often children were active and intensity, indicating excellent inter-rater reliability.

Data Analysis

Means and frequencies for teacher and school characteristics, teacher behaviors (engagement and quality), and student PA outcomes were calculated using the Statistical Package for the Social Sciences. To identify potential covariates, independent samples t-tests and a one-way ANOVA were run to examine whether student PA outcomes differed by intervention condition (i.e., math or language arts) or year (i.e., 1, 2, or 3). Bivariate correlations examined associations between student PA outcomes and teacher characteristics (i.e., age, number of years teaching 4th grade, at present school, and overall). How often children were active during lessons differed by intervention year (F(2,79) = 4.54, p = .01). Classes in year 2 were more active than those in year 1 (p = .01). No differences were found between year 2 and year 3 (p = .19) or year 1 and year 3 (p = .60). Further probing showed year was highly correlated with school (r = 0.96) and we were unable to include it as a covariate due to collinearity. No associations between any other potential covariate and student PA outcome were found (Online Resource 3).

Due to the clustered nature of the data (i.e., classes within schools), ICCs were calculated to identify the amount of variance in student PA outcomes contributed by school. ICCs ranged from >1% (PA intensity) to 13.9% (how many children active). It has been found that in even with ICCs as low as 1%, the Type I error rate may be as high as .20, four times higher than the usual alpha of .05 (Huang 2018). Therefore, we followed best practices and accounted for clustering for all student PA outcomes. However, the number of clusters (i.e., schools; n=19) and cluster size (n=1/cluster) in the present study is less than recommended for a multilevel model to produce unbiased fixed effect standard error estimates (McNeish and Stapleton 2016). Thus, we used fixed effects regression models (FEM) to examine relationships between teacher engagement behaviors during active lessons and student PA outcomes. FEM accounts for variance associated at the cluster level by creating dummy coded variables for each school (referent = School 1) to include in the model (Huang 2016).

Using a series of FEMs (i.e., 12 models) for this exploratory study, we examined the individual impact of teacher engagement behaviors (i.e., moved around the room, scanned the room, directed changes in PA, and participated during active lessons) on each student PA outcome. Typically, p-value adjustments are made to decrease likelihood of type I error due to multiple testing. However, some statisticians argue that it is not necessary to adjust the p-value in exploratory research as corrections for multiple comparisons may reduce statistical power, and given the small sample size of this study, increase risk of type II error (Althouse, 2016; Matsunaga, 2007; Rothman, 1990). Therefore, we followed suggestions made by Althouse (2016) to (a) describe what was done in the study; (b) report the results and give explanations; and (c) let readers interpret the results according to the characteristics of the current research and the rationality of the results.

Bivariate correlations were run to examine relationships between teacher quality of engagement and student PA outcomes. A latent profile analysis (LPA) was conducted using MPlus 7.0 (Muthén and Muthén 2012) to determine whether relationships between quality of engagement can be used to organize teachers into groups (i.e., classes) with similar profiles. The goal of LPA is to identify the smallest number of classes that accurately describes the association amongst variables. Models are estimated with classes added iteratively to determine the best fit to the data. The optimal number of classes for the sample was determined based on several criteria: 1) use of the Lo-Mendell-Rubin Adjusted Likelihood Test (LMRT; (Lo et al. 2001); 2) the Akaike information criteria (AIC; (Akaike 1974); and 3) the sample size-adjusted Bayesian information criteria (sBIC; (Schwarz 1978). The LMRT compares the fit of a target model (e.g. 3-class model) to a comparison model that specifies one less class. If the p-value for the LMRT is less than .05, then the solution with more classes is a better fit for the data. The AIC and sBIC are descriptive fit indices where smaller values indicate better model fit.

Results

Teachers were predominantly female (95%), non-Hispanic (89%), White (94%), and had a mean age of 39.1 years with, on average, 9.6 years teaching. Schools in the intervention conditions were, on average, 33.1% Hispanic, 9.7% African American, and 46.5% White, with 35.4% of students classified as low SES. A majority of teachers’ engagement behaviors were scanning the room and directing changes in PA. Game Instruction and Classroom Management were the most frequently provided forms of quality of engagement. Teachers averaged less than two PA Reinforcement statements per lesson. Means and frequencies for teacher engagement behaviors, quality of engagement variables, and student PA outcomes are presented in Table 1.

Table 1.

Average teacher-, school-, and student-level sociodemographic characteristics, teacher behaviors and quality of engagement, and student physical activity outcomes of Texas I-CAN! observations^a

Teacher sociodemographic characteristics	M±SD
Age	39.11±9.65
Years teaching overall	9.57±6.48
Teacher engagement behaviorsb	% Yes
Directed changes in physical activities	68.7
Participation in physical activities	22.2
Scanned room	93.0
Moved around room	35.0
Quality of engagementc	M±SD
Game instruction	8.17±4.62
Classroom management	6.22±4.08
Content reinforcement	4.09±4.50
Content instruction	3.59±3.74
Technical instruction	2.00±2.26
Physical activity reinforcement	1.65±2.31
Negative feedback	1.43±2.30
School-level sociodemographic characteristics	%
Mean % low socioeconomic statusd	35.4
Mean % Race/ethnicity
Hispanic	33.1
White	46.5
African American	9.7
Student characteristics	%
Free/reduced priced lunchd	23.6
Female	51.0
Race/ethnicity
Hispanic	24.0
White	49.2
African American	8.3
Asian	4.4
American Indian/Alaska Native	1.4
Native Hawaiian	0.1
Multiple selected	4.8
Missing	7.9
Body mass index
Underweight	4.2
Normal weight	66.7
Overweight/obese	29.1
Student physical activity outcomes	M±SD
Class participatione	2.51±0.67
Often children active throughout lessonf	3.58±0.96
Intensity of movementg	3.62±0.95

^aTotal teachers observed = 82

^bMeasured as dichotomous (Yes/No)

^cMeasured as continuous counts

^dEstimated using eligibility for free/reduced lunch, measured as households with incomes ≤185% poverty income threshold

^epossible range 1–3, anchored from (1) less than half of the class to (3) more than half of the class

^fpossible range 1–5, anchored from (1) not at all to (5) most of the time

^gpossible range 1–5, anchored from (1) standing still to (5) running

Teacher Engagement Behaviors

A series of 12 FEMs were conducted to examine the individual impact of teacher engagement behaviors on each student PA outcome (See Table 2). FEMs indicated that teachers moving around the room during lessons was negatively associated with how many children were active (Model 1), but not with how often children were active nor with student PA intensity (Models 2 and 3). Scanning the room during lessons was not associated with how many or how often students were active throughout the lesson or student PA intensity (Models 4–6). Teachers directing changes in PA during lessons was significantly related to higher ratings for how many children were active and how often students were active throughout the lesson, and there was a strong trend for a positive association with PA intensity (Models 7–9). Additionally, teachers participating in the lesson was positively related to ratings for how many children were active, how often students were active throughout the lesson, and PA intensity (Models 10–12).

Table 2.

Associations between teacher engagement behaviors and student physical activity outcomes

Models 1 – 3	Model 1 How Many Children Active			Model 2 How Often Children Active			Model 3 Activity Intensity
	B (95% CI)	β	Adjusted R 2	B (95% CI)	β	Adjusted R 2	B (95% CI)	β	Adjusted R 2
During Lesson Movement	−.32* (−.61, −.02)	−.24*	.18*	−.30 (.21, .76)	−.15	.10	−.24 (−.75, .28)	−.12	−.09
Models 4 – 6	Model 4 How Many Children Active			Model 5 How Often Children Active			Model 6 Activity Intensity
	B (95% CI)	β	Adjusted R 2	B (95% CI)	β	Adjusted R 2	B (95% CI)	β	Adjusted R 2
During Lesson Scan Room	.16 (−.40, .71)	.07	.13	.71 (−.15, 1.57)	.19	.11	.32 (−.63, 1.27)	.09	−.09
Models 7 – 9	Model 7 How Many Children Active			Model 8 How Often Children Active			Model 9 Activity Intensity
	B (95% CI)	β	Adjusted R 2	B (95% CI)	β	Adjusted R 2	B (95% CI)	β	Adjusted R 2
During Lesson Direct Change in Physical Activity	.36* (.06, .67)	.28*	.19*	.60* (.12, 1.07)	.30*	.15*	.50+ (−.02, 1.02)	.26+	−.04+
Models 10 −12	Model 10 How Many Children Active			Model 11 How Often Children Active			Model 12 Activity Intensity
	B (95% CI)	β	Adjusted R 2	B (95% CI)	β	Adjusted R 2	B (95% CI)	β	Adjusted R 2
During Lesson Participation	.40* (.07 - .73)	.27*	.19*	.78** (.27, 1.29)	.34**	.19**	.63* (.07, 1.20)	.28*	−.02*

Quality of Engagement

Bivariate correlations between implementation quality and student PA outcomes are presented in Table 3. Technical Instruction had a positive association with how many children were active, how often students were active, and student PA intensity. Physical Activity Reinforcement was significantly and positively associated with student PA intensity but not how often or how many children were active. Negative feedback was inversely associated with how many students were active, how often students were active, and student PA intensity during the lesson. Game Instruction was negatively associated with how often students were active but not how many students were active or PA intensity. Content Reinforcement, Content Instruction, and Classroom Management were not significantly associated with any student PA outcomes.

Table 3.

Bivariate Correlations Between Quality of Engagement and Student Physical Activity Outcomes (n=99)

	1	2	3	4	5	6	7	8	9
Quality of engagement
1. Physical activity reinforcement	–
2. Game instruction	−.12	–
3. Content reinforcement	.05	.02	–
4. Technical instruction	.35**	−.18	−.11	–
5. Negative feedback	−.01	.18	<.01	−.03	–
6. Content instruction	.08	<.01	.44**	−.23*	.17	–
7. Classroom management	.11	.11	.25*	.06	.14	.08	–
Student physical activity outcomes
8. How many active	<.01	−.21	.04	.25**	−.14	.13	−.08	–
9. How often active	<.01	−.23*	−.20+	.41**	−.25*	−.17	−.21+	.70**	–
10. Physical activity intensity	.20*	−.10	−.20+	.34**	−.25*	−.01	−.13	.70**	.73**

Note.

⁺p < 0.10.

^*p < .05.

^**p < .01.

Latent profile models containing 2 and 3 classes were fit to the data (see model fit indices for each LPA in Online Resource 4). The LMRT indicated that the 3-class solution was not different from the 2-class solution (p = .19), and the 2-class solution did not differ from the 1-class solution (p = 0.07). Thus, a 1-class model was considered the best fit to the data, suggesting that one class represents the data. Teachers in the one class are represented by high levels of Game Instruction and Classroom Management, moderate levels of Content Reinforcement and Content Instruction, and low levels of Negative, Technical Instruction, and PA Reinforcement.

Discussion

Teachers’ engagement behaviors and the quality of their engagement during active lessons was associated with how many and how often children were active during the lesson as well as student PA intensity. Findings from this study align with the relationships between implementation quality and participant responsiveness in the integrated model of program implementation (Berkel et al. 2011). Teachers who implemented I-CAN! lessons with higher quality had students that engaged in greater PA than those who did not implement lessons with quality. Specifically, classes whose teachers exhibited engagement by directing changes in PA and participating in lessons had higher ratings for how many and how often children were active during lessons. Teacher participation in lessons was also associated with higher ratings for student PA intensity. Moreover, teachers moving around the classroom was negatively associated with how many and how often children were active during lessons. In regards to quality of engagement, Technical Instruction was positively associated with ratings for all student PA outcomes and PA Reinforcement was positively associated with PA intensity. In contrast, Game Instruction was negatively associated with how often students were active, and Negative feedback was associated with lower ratings for all student PA outcomes.

The results of this study are similar to those found in the Physical Activity Across the Curriculum intervention in which increased teacher active modeling during lessons was associated with increased student PA levels (Gibson et al. 2008). However, the present study indicates that teacher engagement behaviors differentially impact student PA. For instance, scanning and moving around the room during active lessons were not related to increased student PA. These findings suggest that in order to maximize the impact of active lessons on student PA, teachers need to participate in the lessons and direct changes in activity during the lessons. Efforts to increase these behaviors among teachers are needed. Only about 20% of observations found the teacher participating in PA with the students, and teachers did not direct changes to other types of PA in about 30% of observations.

Relationships between quality of teacher engagement and staff-rated, student PA during active lessons were also examined to determine which types of verbal feedback were indicative of a greater number of children active, more time spent being active, and higher PA intensity. Teachers’ patterns of program delivery when implementing school-based interventions can be very disparate (Shin et al. 2014). We speculated that multiple teacher feedback profiles would emerge because some teachers may have felt more confident providing PA-related feedback than others. These profiles would then be examined in relation to student PA outcomes. Using LPA allowed for the identification of discrete latent variables, from multiple observed variables, that best place individuals into groups based on shared characteristics that differentiate members of one group from another. However, these data did not indicate the existence of multiple feedback profiles. All teachers were represented by high levels of Game Instruction and Classroom Management feedback, moderate levels of Content Reinforcement and Content Instruction feedback, and low levels of Negative, Technical Instruction, and PA Reinforcement feedback. This profile is similar to findings from Weaver et al. (2016) which found that PE teachers devoted nearly 60% of lessons to instruction and management content and rarely promoted PA. As Texas elementary students are provided academic and PA instruction from separate instructors (i.e. classroom versus PE teachers), it is unsurprising that classroom teachers provided feedback primarily related to classroom management and academic content during active lessons. Though PA-related feedback (i.e. PA Reinforcement and Technical Instruction) was positively associated with greater PA intensity, number of students active, and time spent active, few teachers provided this feedback. Given these findings, it seems reasonable to conclude that most classroom teachers provide similar feedback for active lessons. However, their instructional style may not be ideal for achieving high levels of PA during active lessons. The high frequency of Classroom Management and Game Instruction feedback demonstrated in the present study indicates that the intervention may have imposed a structure (i.e., active lessons) that teachers are not comfortable with in the classroom. These findings are congruent with research examining the impact of integrating movement or PA in the classrooms, which found that elementary teachers perceive challenges keeping students on task, maintaining a safe classroom environment, and reining in students’ disruptive behaviors (McMullen et al. 2014; Stylianou et al. 2016). However, the present study did not examine the various styles strategies, and techniques that encompass classroom management and that could differentially impact PA. Moon et al. (2020) demonstrated that movement integration and classroom management practices are co-occurring, with incorporation of PA into regular classroom time positively associated with effective instructional management and proactive management strategies, but negatively associated with reactive management strategies and disruptive student behavior. Research is needed that explores the impact of teachers’ use of different management styles (e.g., proactive, reactive) during active lessons on students’ PA.

The present study’s findings reveal opportunities to maximize effectiveness of active lessons on child PA. For instance, one strategy to optimize implementation quality of active lessons in the classroom is to enhance teacher training by emphasizing teacher engagement behaviors, such as participating with children, and providing greater levels of PA-related feedback. Teacher trainings should incorporate strategies such as modeling and role play to illustrate the importance of providing PA Reinforcement and Technical Instruction feedback during active lessons. It may also be beneficial to provide teachers with mentors, that is, teachers who are successful implementers of active lessons in the classroom or PE teachers. These mentors can provide other teachers with different approaches they can use in the classroom to improve implementation quality, and specifically the forms of feedback used during active lessons. As elementary teachers often work collaboratively in teams, and meet regularly to discuss instructional practices, identifying a mentor from the team to facilitate weekly discussions surrounding active lesson implementation may provide opportunities to exchange strategies to successfully engage students in active lessons. Future research should identify the most impactful strategies to help teachers improve active lesson implementation quality.

Another, perhaps more essential strategy, is to incorporate teacher perspectives of classroom-based PA when designing active lesson interventions. Teachers are often the primary implementers of school-based PA interventions. Therefore, successful implementation, adoption, and sustainment of these interventions is highly dependent upon teachers’ ability to deliver them effectively. Unfortunately, teachers are rarely involved in active lesson intervention development (<35% of interventions; Vazou et al. 2020). Including teachers as collaborators during intervention development will enable researchers to identify implementation facilitators and barriers and increase the likelihood that teachers implement active lessons successfully (Daly-Smith et al. 2020). It is also important that teachers perceive that use of active lessons meet their class’s needs. Teachers may be more willing to implement and adopt active lessons if they understand their connection to behaviors important to student success, such as on-task behavior, and if they feel the lessons help effectively manage students (Lerum et al. 2019; Moon et al. 2020; Webster et al. 2013). Assessing teacher and organizational readiness could help identify factors related to program use (e.g., teacher training, program compatibility with educational philosophy) and help facilitate quality implementation of active lessons (Scaccia et al. 2015; Webster et al. 2013).

The limitations of this study should be noted. Although the parent I-CAN! study was a RCT, data for the present analyses were collected using a cross-sectional design during one day of a typical I-CAN! lesson, limiting the ability to make causal inferences. Additionally, the small sample size did not allow for multilevel modeling and use of FEM did not allow us to study the effects of school-level predictors on student PA outcomes. More research is needed that is sufficiently powered to confirm this study’s findings and assess the role of school-level factors on student PA during active lessons. Another study limitation is that teachers were not blinded to observations and may have been subject to social desirability bias, particularly regarding lesson delivery and involvement. Use of multiple observations or video recordings over the school year could help limit teacher reactance to observation and help determine whether teacher engagement during active lessons is stable over time. Further, teacher engagement behavior (e.g., participation in lessons) was observed using a binary (yes/no) response, which does not allow for finer exploration of the impact of teacher behavior on student PA. Finally, given the magnitude of I-CAN!, device-based measures of PA during lessons could not be correlated with observer-rated PA. Future studies should explore the relationship between teacher engagement and accelerometry-assessed PA.

In spite of these limitations, this study has numerous strengths. This study observed a large number of teachers relative to other studies in this area as well as a large, fairly diverse population of fourth grade students, which increases generalizability of results. However, research examining teacher behavioral strategies to promote classroom-based PA is needed in certain populations, such as older children and those with disabilities. Additionally, at least two observers were present per lesson to assess PA outcomes. Observers demonstrated excellent inter-rater reliability for these variables, limiting the likelihood of observer bias. Further, direct observation is considered an important tool for assessing lesson and teacher effectiveness (Bartholomew et al. 2018). This study also addresses limitations of previous work observing teachers’ behaviors as an indicator of implementation quality by considering a wider variety of teacher feedback during active lessons. For example, in the PE literature, teachers’ verbal promotion of PA was associated with students exhibiting greater time spent in MVPA, while instruction was associated with less time in MVPA (Fairclough et al. 2018). It is important to more granularly assess teacher feedback during active lessons and its relation to student PA.

Conclusions

Increasing PA participation among youth is paramount as activity levels decline from childhood into adolescence (Katzmarzyk et al. 2016; Troiano et al. 2008). I-CAN! has been shown to provide up to 20% of the recommended PA for children, with participation in active lessons occurring regardless of demographic subgroup (i.e., SES, race/ethnicity, BMI, fitness; (Bartholomew et al. 2018). This suggests the inclusiveness and benefit of active lessons for all children. Results from school-based, PA intervention studies help inform public health and policy decisions. Thus, it is important to assess implementation quality of such interventions to maximize their impact. The findings of this study suggest opportunities to maximize intervention delivery such as co-designing active lesson interventions with teachers and training teachers to engage in specific types of behaviors during active lessons to help promote student engagement in PA. As teachers are primarily responsible for implementing school-based interventions, it is critical to develop strategies that increase teachers’ ability to implement them successfully.

Data Availability:

All data is available upon request.