Predictors of on-task Behaviors: Evaluating Student-level Characteristics

Abstract

Objective:

In this study, we examined students’ fitness, body mass index (BMI), and demographics as predictors of observed time on-task (TOT) behaviors as an indicator of behavioral inattention.

Methods:

We collected demographics, fitness estimates, and BMI from 2020 fourth-graders (Mean age = 8.6 (SD = 0.5); 47% girls; 49% white) from 28 schools. We measured TOT through momentary time sampling observations. Three-level linear models were conducted to determine whether characteristics predicted differences in TOT. We tested interactions between characteristics and TOT.

Results:

Older students exhibited greater percent of TOT (estimate = 2.34, SE = 1.02, df = 919, t = 2.30, p < .05). Additionally, boys spent less percent TOT (estimate = −3.59, SE = 1.03, df = 906, t = −3.49, p < .05). There were no differences by race/ethnicity, SES, BMI, fitness, or time of day and percent TOT. Furthermore, none of the interactions were statistically significant (p > .15).

Conclusions:

Girls and older students spent more TOT. These findings are of interest to educators and psychologists working on the development of research-based guidelines aimed to support elementary students’ engagement in the classroom.

Effective education practice is often built around efforts to build student engagement. This is required, as elementary students have been shown to spend as much as 50% of time in off-task behaviors.¹ Behavioral inattention, a form of disengagement, encompasses a host of observable actions that are incongruent with a student’s focus on and engagement with teacher-assigned tasks. These actions are often assessed through teacher ratings of attention and have consistently been associated with lower academic performance.^2-4 Whereas consistent and excessive inattention are components of clinical diagnoses, such as attention deficit disorder,⁵ intermittent periods of behavioral inattention are common and associated with reduced academic performance in both subclinical populations⁶ and after controlling for other behavioral challenges.⁷ As such, the study of student behavior related to behavioral inattention and its antecedents is an important issue for the general student population.

There are a number of environmental factors, particularly in the elementary classroom, that have been shown to influence behavioral inattention. Peers are a common source of off-task behaviors.⁸ In addition, behavioral inattention is impacted by teacher behavior. Teachers who assert more control over student behavior, provide clear instructions, use quiet seated activities, and/or small-group activities^8,9 are associated with greater student engagement with academic material and less behavioral inattention. Research also suggests that elementary-aged students had high levels of attention during the afternoon.¹⁰ Given this importance, it is surprising that there is little known about student-level characteristics of behavioral inattention in sub-clinical populations. The most consistent findings have been for socio-economic status (SES) and gender, with both lower SES students¹¹ and males⁸ exhibiting greater behavioral inattention than their higher SES and female counterparts. Although race and ethnicity also have been studied as predictors of behavioral inattention, the findings are more varied, with no clear pattern of effect.¹²

A growing body of research has examined the impact of physical activity on behavioral inattention. Brief (10-15 minutes) periods of physical activity, implemented in the elementary classroom, have been shown to increase student attention,^13,14 with a particular benefit for overweight children.¹⁵ These data may help explain the findings that more active and physically fit children outperform their less fit counter parts on standardized testing,¹⁶ and that increases in activity and fitness are associated with improvements in academic performance.¹⁷ That is, children with greater levels of physical fitness, a state that reflects the capacity to perform physical activity,¹⁸ may be able to maintain greater behavioral control and resistance to distraction,¹⁹ which, in turn, allows them to outperform their less fit counterparts. For example, research shows that children with higher aerobic fitness – along with those who demonstrate increases in fitness – score higher on cognitively demanding tasks.²⁰ Additionally, elevated body mass index (BMI) has been associated with less efficient cognitive control²¹ and decreased cognitive function.²² Thus, one would hypothesize that low levels of fitness and unhealthy body composition would be correlated with behavioral inattention in the elementary classroom. For example, one study indicates that classrooms with at least 80% of children focused on teacher-assigned tasks had, on average, students with lower BMI and greater fitness than classrooms with greater behavioral inattention.²³ However, this study relied on teacher reports of classroom level engagement. This prevents an analysis at the individual level, which is critical for a test of student BMI and fitness on behavioral inattention. To our knowledge, this is the first study to examine fitness and BMI at the student-level as predictors of behavioral inattention.

Student achievement is one of the goals of education. Behavioral inattention in the classroom may inhibit the success of achieving this goal.²⁴ Thus, examining the student and environmental level characteristics that impact behavioral attention is important for teachers and administrators. Findings from such studies could play a role on how teachers structure their class day or materials taught in the classroom. For example, if students with higher fitness levels spent more time on a teacher-instructed tasks, then it would be highly beneficial for schools to increase fitness levels of their students.

Limitations in the measurement of behavioral inattention also have undermined research in this field. Whereas teacher ratings have long been considered the norm for educational research,²⁵ systematic and direct observations of student engagement have been established as reliable and valid.²⁶ Independent observers are also preferable to teacher ratings as they are less likely to reflect racial and ethnic^27,28 or gender bias.⁸ Moreover, it has long been recognized that children with early behavioral problems,²⁹ hyperactivity, and attention-deficit hyperactivity disorder³⁰ are likely to have their attention judged more harshly by teachers than independent observers. As a result, it is recommended that independent researchers, trained to be objective and nonjudgmental, conduct systematic and direct observations of student behavior to avoid bias.^31,32 A second concern is that the existing research has relied on relatively small samples or a minimally to moderately diverse set of students. Specifically, these studies often compare just one class or school with another. These both limit the ability to generalize data and prevent the opportunity to conduct a hierarchical model to distinguish between class- and student-level predictors.

Examining the impact of a range of student-level predictors of behavioral inattention in a large, diverse sample of elementary children would do much to support intervention design for the general student population. Our study fills this void as data were collected from a diverse sample of more than 2000 fourth-grade students drawn from 28 schools and 149 classrooms. To our knowledge, this is the largest study to date with researcher-led observations of student behavior, with sufficient power to model both class- and student-level variables.

METHODS

Participants

We recruited 28 elementary schools in Central Texas to participate in a randomized controlled trial, Texas Initiatives for Children’s Activity and Nutrition (I-CAN!), examining the impact of active learning on student outcomes, such as TOT and standardized test scores. Schools were randomized to receive the active lesson intervention (N = 19) or control (N = 9) with 149 teachers (N = 99 intervention, N = 50 control) and 2716 students (N = 1903 intervention, N = 813 control) participating between 2012 and 2015. For more information about the I-CAN! parent study, see Bartholomew et al.³³ The current study examines the baseline predictors of on-task behaviors for 2081 students, collected before implementation of the classroom intervention. Parents provided written, informed consent prior to the start of the study and students provided their assent.

Measures

Demographic information.

Student demographic information was obtained through school records. Gender was coded (1) male or (0) female. Free and reduced lunch was used as a proxy for SES where (1) eligible for free/reduced or (0) not eligible. Race/ethnicity was coded as (1) Hispanic, (2) American, Indian/Alaska Native, (3) Asian, (4) black, (5) Native Hawaiian/other Pacific Islander, or (6) white. For the purpose of this analysis, race/ethnicity was dichotomized into (1) white and (0) non-white. Student’s birthdate during the first semester of the I-CAN! intervention was used to calculate age.

Body mass index (BMI).

Student body composition was indicated by their BMI, the ratio of weight (kg) to height (cm) that is then normed based on the US Centers for Disease Control and Prevention (CDC) population-based growth charts.³⁴ These data were obtained through the school’s FITNESSGRAM^® records. The state of Texas requires FITNESSGRAM^®35 testing of all elementary students, from third through fifth grade. Physical education (PE) teachers measure student height and weight as a part of this test. All measures were taken during the same semester as the collection of TOT observations.

Fitness level.

The FITNESSGRAM^® also includes the Pacer test as a measure of cardiorespiratory fitness. This test requires students to complete a series of 20-meter laps, each lap in less than 30 seconds. Two failures to complete a lap in under 30 seconds ends the trial, with a score based on the number of laps completed. If a student reaches 190 laps, the test is halted. The Pacer is a valid and reliable measure of cardiorespiratory fitness among children of this age group.³⁵

Time on-task (TOT).

Research staff assessed students’ behavioral inattention and engagement by direct observations of student time spent focused on a teacher-assigned task. Research staff observed each classroom in pairs. Each research staff member observed a sub-set of participants over a 15-minute period using 5-second momentary time sampling. In this protocol, one student was observed for 5 seconds and marked as either on-(1) or off-(0) task. The next student was then observed for 5 seconds and so on. As soon as research staff completed one round of observations, further rounds were conducted until the end of the 15-minute period. Research staff conducted an average of 22 rounds of assessment for each student. To standardize the observation periods, researchers listened to a 15-minute recording of tones separated by 5-second intervals. The tones signaled that a 5-second observation period for a student had ended and that research staff should proceed to observe the subsequent student. Students considered on-task were quietly engaged and following teachers’ instructions or paying attention to the teacher for duration of the 5-second observation. Research staff marked students off-task when they were not following teachers’ instructions, usually by talking, laying their head on the desk, doodling, or gazing off. In addition, staff noted any instance where there was a class disruption during the assessment (eg, another teacher entering the room), and those observations were removed from analysis. This method has been established as a means to assess on- and off-task behaviors.^8,15,31,36 The percentage of time that students were engaging in on-task behaviors (TOT) during the 15-minute observation period was calculated. The time of day, whether observations of TOT were conducted before lunch (morning) or after lunch (afternoon), was also recorded.

Research staff were trained to conduct TOT observations in pairs within a separate set of non-participating elementary classrooms. Training is considered complete when inter-rater reliability (IRR) exceeded 90%. In an earlier study,¹⁵ our research team found 4 training sessions were required to achieve sufficient IRR of 92%. A 3-month, follow-up assessment indicated that IRR remained high at 94%.

Data Analysis

Descriptive statistics were conducted to determine differences between age, SES, gender, BMI, and Pacer score for percent TOT at baseline. Three-level linear models were run using SAS PROC MIXED to determine whether there were differences in percent TOT among students nested within classrooms and schools. Student-level predictors (level-1) included age, gender, race/ethnicity, SES, BMI, and Pacer score. BMI and Pacer score were grand-mean centered before estimating the models to ensure each variable had a meaningful zero.^37,38 Because time of day might impact student attention (lower attention later in the day), it was included as a predictor in the model at level-1 as a dichotomous variable – morning or afternoon. Classroom was included at level-2 and school was included at level-3. To address our research question, we estimated all models using maximum likelihood (ML) to examine model fit improvements using Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC). Smaller AIC and BIC values suggest a better fitting model. Given the nested design of this study, AIC and BIC indices help to identify differences in the random or fixed effects.³⁹

RESULTS

Descriptive Statistics

Table 1 presents the demographic information for schools and students participating in the study. On average, students were 32.9% Hispanic, 46% non-white, and 48.8% white. An average of 24.9% of students were eligible for free or reduced-priced lunch. We excluded 59 students due to the invalid TOT assessments (eg, classroom disruption), resulting in a final sample of 2020 students (Mean age = 8.6, SD = 0.5; 47% female; 49% white).

Table 1.

Characteristics of Students

	Percent	N
Race/ethnicity
White	48.8	1973
Non-White	46.0	1973
Hispanic	32.9	1963
Male	48.5	1976
Free/reduced priced lunch	24.9	1622
Morning Assessment of TOT	64.4	2079
	Mean (SD)	Range	N
Age	8.6 (.5)	7 – 10	1477
BMI	17.8 (3.3)	11.6 – 33.4	1551
Fitness level (Pacer Laps)	30.5 (18.5)	0 – 153	1709
%TOT	84.9 (16.6)	4.76 – 100	2020

Multilevel Models

The first model was an unconditional model with no predictors to estimate the interclass correlation coefficients (ICC) for each of the levels. The school level ICC accounted for 7% of the variance between schools, and the classroom ICC accounted for 14% of the variance between student’s percent TOT.

The second model included student-level predictors (age, gender, race/ethnicity, SES, BMI, fitness, and time of day) as fixed effects, which can be seen in Table 2. Age was a statistically significant fixed effect (estimate = 2.34, SE = 1.02, df= 919, t = 2.30, p = .02), suggesting that older students are more on-task. Additionally, gender was a statistically significant fixed effect (estimate = −3.59, SE = 1.03, df= 906, t = −3.49, p < .001), indicating that girls were more on-task compared to boys. There were no differences in race/ethnicity, SES, BMI, fitness level, and time of day on percent TOT. The model fit indices indicated that the second model was a better fit than the unconditional model. All predictors were tested for interactions; however, none were statistically significant (ps = .16 to .95).

Table 2.

Estimates from 3-level Model Predicting Percent Time on Task (N = 2020)

	Model 1 (unconditional)	Model 2 (fixed)	Model 3 (random)
Intercept	84.36** (1.03)	63.55** (9.05)	64.68** (8.96)
Age		2.34* (1.02)	2.34* (1.01)
Gender		−3.60** (1.03)	−3.71* (1.15)
Race/ethnicity		−0.24 (1.14)	−0.52 (1.13)
SES		0.35 (1.50)	0.30 (1.52)
BMI		−0.29 (0.18)	−0.26 (0.18)
Fitness level		0.05 (0.05)	0.08 (0.07)
Time of day		2.06 (1.47)	1.81 (1.53)
Error Variance
Level-1 (Student)	226.01** (7.32)	224.81** (10.72)	216.49** (10.83)
Intercept (Classroom)	35.58** (7.52)	34.91** (11.14)	19.86 (13.69)
Intercept (School)	16.49* (8.26)	10.28 (9.99)	5.11 (8.21)
Model Fit
AIC	16861.8	7911.8	7910.0
BIC	16867.1	7922.2	7924.1

Note.

Statistically significant

^*p < .05

^**p < .001

Values based on SAS PROC Mixed. Entries show parameter estimates with standard errors in parentheses. Estimation Method = ML; Satterthwaite degrees of freedom

The third model included random slopes for the same student-level predictors to test if the relationships between the student-level predictors and percent TOT varied between classrooms and schools. Neither the school-level slope (estimate = 5.11, SE = 8.21, p = .27) nor the classroom-level slope were statistically significant (estimate = 19.86, SE = 13.69, p = .07). Both age (estimate = 2.33, SE = 1.02, df = 901, t = 2.32, p = .02) and gender were statistically significant random effects of TOT (estimate = −3.71, SE = 1.15, df = 97.3, t = −3.22, p = .002), suggesting again that older students and girls were more on-task. Although the AIC was smaller in the third model, the BIC did not improve, indicating that the second model is the best fitting model.

DISCUSSION

This study was designed to examine student-level predictors (age, gender, race/ethnicity, BMI, fitness, SES, and time of day) on behavioral inattention as indicated by observations of on-task behaviors in the general, elementary population. In line with early findings, these data indicated that girls were more on-task compared to boys. Although we only assessed fourth-graders, those with earlier birthdates exhibited greater TOT than did younger students. Furthermore, our findings were consistent with research indicating that race/ethnicity, SES,⁸ and time of day were not predictors of TOT. In contrast to a previous study,²³ lower BMI and higher fitness levels were not predictors of time children spent on-task.

Findings revealed greater TOT for girls compared to boys. This replicates the most consistent finding for behavioral inattention.^8,40,41 Moreover, there is a body of literature that suggests that girls are generally more academically-engaged than boys.^42-45. Thus, this effect was not surprising. What was more interesting was the effect for age. Our findings coincide with studies among participants of a more diverse age range⁴⁶ despite a relatively narrow age range in our sample of fourth-grade children. The data also correspond with research indicating that entrance age for kindergarten, where some students start kindergarten at an earlier age compared to other students, has lasting negative effects on both academic performance⁴⁷ and social behavioral skills.⁴⁸ More research should directly test the impact of differences in TOT as a potential mediator for these apparently lasting entrance age-effects.

The literature has been mixed with regard to the impact of SES or race/ethnicity on attention.^12,49-51 This ambiguity is likely due to 2 issues. First, previous studies relied on teacher reports of attention, that may have been overly influenced by biased response toward minority and less affluent children.^27,28 Second, there is an extreme challenge in recruiting a sufficiently diverse and large group of children to allow for sub-group comparisons – particularly when sub-groups are not equally distributed in the sample. To address these issues, we recruited the largest sample to date for direct observations of TOT. Despite inequality in distribution such that only 24.9% were lower SES (ie, eligible for free and reduced-priced lunch) this still represented more than 400 children in our sample. This large sample provided sufficient power to test differences in TOT as a function of SES and race/ethnicity along with increased accuracy of assessment due to direct observation of individual TOT. Thus, our finding that there are no differences in TOT as a function of race/ethnicity nor SES is notable in the existing literature.

There are strong data indicating that elementary-age students with higher physical fitness have greater academic performance, and students who have higher BMI have poorer academic performance.⁵² One explanation is that children who have higher fitness levels have improved neurocognitive functioning and cognitive control.⁵³ TOT may reflect differences in cognitive control in a way that is positively associated with academic performance.⁵⁴ Similarly, self-regulation may be another potential mediator. One study showed that children that exhibited a compromised ability to self-regulate had high BMI z scores.⁵⁵ Self-regulation defined as a trait rather than a state may provide support that the inability to self-regulate extends to other behaviors such as behavioral inattention in the classroom. As such, it was expected that fitness and BMI would influence attentional focus and engagement. These hypotheses were not supported. Neither BMI nor physical fitness were predictors of on-task behaviors. Future studies should examine self-regulation as a mediating mechanism among elementary school children with a wider distribution of physical fitness and BMI levels compared to the current sample. Our findings may also indicate that there are other mechanisms, beyond student differences for TOT, that mediate the associations between fitness level, BMI, and academic achievement.

Limitations

Despite the large and diverse sample and strong, objective, measure of TOT, this study has limitations. We did not have access to clinical diagnoses such as attention deficit hyperactivity disorder, other cognitive/learning disorders, or pre-existing behavioral problems. All of these student characteristics are known to influence on-task behaviors. Additionally, we only evaluated on-task behaviors for 15 minutes. This may not be a sufficient indicator of attention control in fourth-grade students to reveal differences between students. Teachers often report that students can manage attention at the outset of a lesson, but decline as the lesson continues. This reflects our earlier data that found significant reductions in TOT during a teacher-assigned activity.¹³ In response, a longer assessment is warranted for future studies and, perhaps, an examination of TOT at different points throughout the school day. Additionally, the study was designed to evaluate only fourth-grade elementary students, who on average, had normal BMI and fitness levels, which limits our ability to generalize these findings to children in different grades.

Conclusion

Despite these limitations, this study included the largest, most diverse sample of elementary aged-students to examine predictors of individual time on-task behaviors. Additionally, this study used a rigorous protocol of observers using momentary time sampling to assess these behaviors. The findings replicate the impact of kindergarten entrance age and sex on TOT, but found no support for the role of BMI or physical fitness. These findings suggest the need to assess different mediators for the impact of fitness and BMI on academic performance as well as reinforce the need to consider interventions to reduce differences in behavioral inattention for younger students and boys. Moreover, this study adds to our understanding of the impact of student-level characteristics such as race/ethnicity, SES, and time of day on TOT, and suggests that future research examining disparities in academic outcomes should place more focus on classroom-level than student-level predictors for differences in these outcomes.

IMPLICATIONS FOR HEALTH BEHAVIOR OR POLICY

Student achievement is one of the main goals of education and an overarching goal of public health as education and student health are intertwined. Findings from the current study add to our understanding of the impact of student and environmental-level characteristics on student attention and engagement, a significant factor related to student achievement. One of the strengths of this study is that the results reinforce the long-standing finding that on task-behavior differs as a function of age and sex, while lessoning the focus on other student-level characteristics such as physical fitness and BMI. Implementing strategies and policies that increase student attention and engagement in the classroom is crucial to achieve the Health People 2030 objectives to increase the proportion of fourth-grade students whose math and reading skills are at or above the proficient achievement level for their grade.⁵⁶ Potential strategies to increase student attention include incorporating mindfulness as a classroom or school policy. Studies find that children who practice mindfulness in school have better self-regulation of attention and emotion.⁵⁷ Early implementation of mindfulness in the classroom may ameliorate the differences seen in attention due to age and sex. Another strategy is to require trained staff – including teachers, counselors, or school psychologists – to follow a specific plan to teach and support positive behavior through organizational training.⁵⁸ This training may include productive time management, planning skills, and ways to keep school materials organized to optimize student attention while reducing distractions.⁵⁹ Lastly, adding more frequent or longer physical activity breaks throughout the day has the promise increase attention and engagement in the classroom, thus improving academic achievement.³¹