The executive function-related learning and practicing activity analysis tool (EFRLPAAT): development of an observation system for assessing the effectiveness of physical education practice

Abstract

Objective

The teaching experimental process of exercise intervention for executive functions in children and adolescents lacks the necessary systematic observation tools, which cannot ensure the effectiveness of the intervention process. Therefore, the Executive Function-Related Learning and Practicing Activity Analysis Tool (EFPLRAAT) have been developed.

Methods

Invitations were sent to 27 experts (15 in the first round and 12 in the second round) from the fields of school physical education, psychology, and motor skill studies to evaluate and screen the preliminary established indicators. Kendall’s concordance coefficient, selection rate, and mean were used to reflect the content validity of the tool. The discriminant validity of the observation tool was established through systematic observation of 42 open and closed skill physical education classes. The concordance coefficient was used to reflect the reliability among observers.

Results

The EFRLPAAT consists of four primary and nine secondary indicators, mainly including movement status, movement scenarios, action structures, and interpersonal interactions. The expert advice has a medium degree of similarity (Kendall’ W = 0.491, χ² = 53.047, p < 0.01). Except for the selection rate of the secondary indicator “no movement” of “movement status”, which is lower (43.75%), all other indicators meet the development criteria. The proportion of time spent in unpredictable situations, interactive practices, and displacement movement states in open-skill courses is significantly higher than that in closed-skill courses, indicating a certain degree of discriminant validity. All of specialized lessons were coded at 85% or higher, with high interobserver agreement reliabilities.

Conclusion

The EFRLPAAT can be used as an observation tool to ensure the validity of the experimental process, helping the researcher to clarify the validity of their experiment, and it can also provide an effective observation tool for physical education teaching, pushing teachers to continuously feedback and optimize their teaching.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-025-24430-4.

Introduction

Executive functions refer to the higher cognitive processes by which individuals regulate a variety of basic cognitive processes in complex cognitive activities [1, 2]. The executive functions are somewhat related to the essential requirements of core literacy, both of which are oriented to problem solving in uncertain situations and involve higher-order literacies such as logical thinking, problem solving, analyzing, synthesizing, reasoning, deduction, induction, and decision making, as well as autonomous and self-aware actions, and intricate communication and exchange [3, 4]. Thus, executive functions have strong links to real-life logical reasoning [5], everyday decision-making [6] and social adaptation [7], regulating a wide range of cognitive, emotional and behavioral issues in daily life [8]. In addition, executive functions are significantly and positively associated with creative thinking and academic performance in children and adolescents [9–11]. Based on this, the promotion strategies for the executive functions of children and adolescents have gradually attracted the attention of researchers.

Physical exercise is a natural means of promoting the development of executive functions in children and adolescents [12, 13]. As research continues to progress, researchers [14–16] are gradually finding that there is skill-type variability in the benefits of exercise for executive functions, that is, open-skill exercise is a better intervention for executive functions than closed-skill. The so-called open skills are skills that perform movement tasks in unpredictable environments, where individuals need to react and adjust their movements according to changes in the environment, such as soccer, basketball, ping-pong, taekwondo; closed skills are skills that perform movement tasks in stable, predictable environments, where individuals are able to plan their movement routines in advance, such as tai chi, artistic gymnastics, swimming [17]. In addition, Shi et al. [16] categorized sports skills into sequential and continuous skills based on the complexity of movements. Sequential skills refer to the connection of multiple single movements in a certain order into a sequence of movements with a more complex structure, such as aerobic gymnastics, martial arts routines; continuous skills refer to the multiple repetitions of a single movement, and the movement does not have a clear beginning and end, and the structure of the movement is relatively homogeneous, such as swimming, bicycling, speed skating [17]. On this basis, Shi et al. [16, 18] and Feng et al. [19] found that sequential skill exercise interventions for executive functions in children and adolescents were more effective than continuous skills.

Maximizing the benefits of exercise on the executive functions of children and adolescents has always been a focus of researchers. Given the skill-type variability in the benefits of exercise on executive functions, researchers are increasingly interested in the underlying reasons for this variability. Ishihara et al. [20] showed that the benefits of exercise for executive functions depend not only on the skill types, but more importantly on the learning and practicing activities involved in the intervention. Learning and practicing activities, that is, physical learning activities used in the formation of sport skills, are practice activities that improve and standardize technical movements and form movement stereotypes. Learning and practicing activities are the process of practicing and applying executive functions repeatedly, and the richer the executive functions involved in learning and practicing activities, the more conducive they are to promoting their development [21]. For instance, Ishihara et al. [20] demonstrated that the time students spend on engaging in fun activities is significantly negatively correlated with reaction times in interference control tasks, while the time spent on coordination activities is significantly positively correlated with the accuracy of working memory tasks. Rodríguez-Negro and Yanci [22] found that tactical games can promote the development of students’ attention and creative thinking. Crova et al. [23] manipulated task and environmental constraint elements by altering the spatial dimensions of the field, game rules, and movement structures, and by setting up competitive activities, which induced students to flexibly apply the skills they had learned. This indicates that such cognitively challenging activities are beneficial for the development of executive functions, and can provide greater cognitive benefits even in shorter durations. Therefore, the executive function benefits of exercise can be effectively improved by rationally altering the learning and practicing activities of sport skills.

However, as far as the current intervention studies are concerned, although researchers have developed detailed exercise measures to promote executive functions in children and adolescents, it is difficult to ensure that the implementers of the intervention programs will follow the plans to the letter, and thus there may be shortcomings in the exercise promotion benefits. Systematic observation tools of the experimental process can compensate for this to a certain extent and play an important role in stimulating, sustaining and promoting the pedagogical motivation of the program implementers, in promoting the motivation of the participants to learn and practice, as well as in improving and increasing the effectiveness of the intervention [24, 25]. Systematic observation is the process of recording and analyzing the object or event to be observed by a trained observer according to certain procedures [26]. Currently, systematic observation tools in the field of physical education are widely used in teaching experiments and gradually serve as important tools to ensure the effectiveness of interventions. For example, systematic observation tools for teacher/coach coaching behavior such as ASUOI (Arizona State University Observation Instrument) [27], CAIS (Coach Analysis and Intervention System) [28], systematic observation tools for student physical activity such as SOFIT (System for Observing Fitness Instruction Time) [29], ASM (Simple Activity Measurement) [30], systematic observation tools for classroom interpersonal interaction such as TARE (Tool for Assessing Fitness Instruction Time) [31], ORRPETB (Observational recording record of physical educator’ s teaching behavior) [32], as well as the Physical Education Classroom Problem Behavior Observation Form [33].

However, after systematically reviewing the relevant tools, this study found a lack of necessary analytical tools for learning and practice activities related to executive functions, making it difficult to obtain valid data on the experimental process and to assess whether the subjects followed the interventions designed by the researchers. However, given the importance of executive functions in the development of children and adolescents, it is necessary to develop an executive function-related learning and practicing activities analysis tool (EFRLPAAT) to ensure that they can obtain as many executive function benefits as possible. Therefore, based on the integrated theory of promoting executive function benefits in children and adolescents through exercise [34], this study developed the EFRLPAAT. Through this study, we aim to provide an effective observation tool to ensure the validity of the experimental process. Additionally, it also serves as an effective observation tool for physical education class teaching, providing continuous feedback and optimization for teaching, and enhancing the educational benefits of school sports and motor skills.

Materials and methods

Participants

The participants in this study mainly include the invited experts and the observed students. The purpose of inviting experts is to conduct a content validity test of the tool; the purpose of observing students is to conduct a discriminant validity and inter-rater reliability test of the tool.

Firstly, this study invited experts based on the following inclusion criteria: (1) University teachers who have been engaged in the fields of psychology, motor skills, and school physical education for over 10 years, possessing extensive theoretical knowledge and research experience, and having an in-depth understanding of the relationship between motor skills and executive functions, as well as the principles by which exercise promotes executive functions; (2) Primary and secondary school teachers who have been engaged in the practice of school physical education for over 10 years, with a deeper understanding of learning and practical activities in physical education classes. This study invited six (40.0%) professors, two (13.3%) associate professors, four (26.7%) lecturers, and three (20.0%) secondary school physical education teachers to initially evaluate the effectiveness of the indicators. Based on the revision of the initial evaluation, four (33.3%) professors, three (25.0%) associate professors, and five (41.7%) secondary school physical education teachers were re-invited to conduct a content validity test. The experts invited for the second time were not the same as those invited for the first time.

Secondly, through the observation of 42 physical education classes, a preliminary understanding of the discrimination of the tool indicators for different types of skill learning and practicing activities was obtained. This study primarily includes skill training classes that focus on learning and consolidation for analysis. Exclusion criteria: (1) Physical fitness classes that do not focus on skill training; (2) Health education classes that are primarily focused on knowledge dissemination; (3) Sports skill test classes that are mainly centered on assessments. The observed classes came from 16 high schools in Shanghai, including 21 open skill courses and 21 closed skill courses (Table 1). This study conducted classroom observations on a class-by-class basis, primarily reflecting the learning and practice activities set in the teaching classroom through observing students’ behavior. The number of students in the observed classes was generally around 20, except that most students in the aerobics class were female, while the other classes included both males and females. Since this study is a natural classroom observational study, there is no restriction on gender. This study has been approved by the Scientific Research Ethics Committee of Shanghai University of Sport (102772023RT023) and observations were conducted with the informed consent of both parents and students.

Table 1.

The observed sports and their corresponding types of motor skills

Sports	Football	Basketball	Volleyball	Table tennis	Open skills
Num.	2	15	3	1	21
Sports	Martial arts	Aerobics	Track and field		Closed skills
Num.	8	4	9		21

Procedures

Theoretical basis for the selection of EFPLPAAT analysis indicators

Our early studies [34, 35] proposed an integrative theory, expounding how to manipulate the qualitative elements of exercise to enhance the promotional benefits of executive function (Fig. 1). The theory states that the enrichment of the sports environment, social interaction and the introduction of complex tasks can induce variability in movement practice, increase interest in sport skill learning, promote the development of neuromodulatory qualities such as agility and coordination, and thus facilitate the accumulation of motor experience and executive functional benefits [34]. The variable exercise means that the relative complexity of the practice movements, which involve numerous interconnected actions that frequently adapt to changes in external stimuli or task objectives. These exercise movements are characterized by complexity, diversity, novelty, creativity, and variability [36, 37]. Based on this, Pesce et al. [37] suggested that “variability of practice is a key interface between sport and cognitive development”. Therefore, this study focuses on extracting analytic indicators of EFRLPAAT from rich environments, social interactions, and complex tasks.

Fig. 1.

An integrative theory of the benefits of exercise for executive functions in children and adolescents

1. Movement status.

   Movement status is an important element of systematic observation tools such as SOFIT [29], SAM [30]. This study focuses on categorizing movement statuses into three types: no movement, in-situ movement, and displacement movement, based on the status of the student relative to some reference system while exercising. The no movement status refers to the target student being in a non-moving status, such as lying down, sitting, or standing. The in-situ movement status refers to the target student performing exercises in a fixed position, such as in-place warm-up exercises, in-place fitness exercises, and technical skill practice in place. The displacement movement status refers to the target student being in physical motion, such as jogging for warm-up, mobile fitness exercises, technical skill practice on the move, and games or competitions that involve displacement.

   Movement is beneficial, and both in situ and displacement movements improve executive functions in children and adolescents relative to the no-movement status, which is explained by the fact that motor manipulation shares brain area activation with cognitive tasks [38]. For example, repetitive learning of in situ-type technical movements, such as aerobics and gymnastics, can help improve inhibitory control and working memory in school-age children [39, 40]; displacement-type activities, such as running, can help improve attentional inhibition and executive functioning in school-age children [41, 42]. In addition, the reason for selecting the indicator of movement status was based on a combination of rich environment, social interaction and complex task elements. By adding elements of rich environments and social interactions as well as introducing complex tasks, it will definitely lead to a change in the movement status. For example, contextualized, game-based instruction can increase displacement-type activities; stimulus-response games that involve listening to commands in situ can increase the difficulty of the practice task and improve the variability of motor execution.

2. Sports environment.

   This study categorized the sports environment into two types: predictable and unpredictable environments. A predictable environment refers to a context where the target student’s movement process is foreseeable, such as routine warm-up activities, regular sports practice, and combinations of singular techniques or fixed tactics. An unpredictable environment, on the other hand, refers to a setting where the background of the target student’s movement process is unforeseeable, such as chase games, sports games, and games or competitions primarily focused on technical and tactical training.

   The changing and unpredictable sports environment is an ideal place to promote the development of executive functions [43], and the complexity and variability of this environmental information provides continuous stimulation of cognition, which in turn enhances executive functions benefits to a greater extent. The neoplastic cell proliferation induced by the enriched environment achieves about 50% of the effect of cardiovascular exercise and is more conducive to the integration of the newly generated neurons into the existing neural network [44]. In addition, experienced open-skill athletes have higher gray and white matter plasticity in the frontal coalition area [45] and show quality responses and increased prefrontal activation in general cognitive tasks [46].

3. Social interactions.

   In sports contexts, where more environmental stimuli are represented through social interaction processes, sports with peer participation are more effective in improving brain cognition in children and adolescents. Based on this, this study categorized social interactions into two types: individual practice/game/competition and interactive practice/game/competition. Individual practice/game/competition refer to the process where the target student practices without significant interpersonal interaction, such as individual sports skill practice, games or competitions for individual honors, and those with low interactivity. Interactive practice/game/ competition refer to the process where the target student practices with interpersonal interaction, such as interactive sports skill practice, games or competitions for team honors, and those with high interactivity.

   Wertsch [47] proposed that higher thinking functions developed in the context of interpersonal interactions; a study [48] also confirmed that completing tasks with peers improves children’s higher cognitive functions such as strategy selection and application, problem comprehension, and problem solving. Exercises that involve more elements of social interaction have a higher degree of environmental unpredictability, require participants to engage more of their awareness and thinking, and create higher cognitive challenges for the participant. Chen et al. [49] compared the acute intervention effects of cooperative versus solo rope skipping on executive functions in school-aged children, and preliminarily confirmed that peer-participation in a supportive activity is more conducive to promoting the development of executive functions.

4. Complex tasks.

   Wood [50] proposed three dimensions for analyzing task complexity, namely component complexity, coordination complexity, and dynamic complexity. Dynamic complexity refers to tasks that are dynamic and variable, which overlap to some extent with sports environment and social interactions; therefore, it will not be repeated here. In terms of component and structural complexity, multi-limbed movement practice with relatively complex structures and a relatively large number of segments increases the allocation of attentional resources, which helps to activate the relevant neural circuits and facilitates improvement in executive function prefrontal-dependent tasks [51, 52]. The diversity of the target task makes students constantly face new problems and challenges, and individuals need to coordinate and combine the original actions, integrate the existing declarative knowledge and experience to make more rational coordinated responses or create more novel actions, so that the relevant neural circuits in the cerebral cortex continue to establish connections [53, 54].

   Based on this, this study referred to the studies of Tian et al. [55] and Zhang et al. [17], and categorized the skill practices reflecting the complexity of motor structure into four types, namely, single-technique fixed practice, single-technique variation practice, fixed-combination technique practice, and multiple-technique stochastic practice; and the movements reflecting the complexity of the motor tasks were categorized into two types, namely, continuous and sequential movements. Single-technique fixed practice refers to the repetition of a fixed movement or sequence of movements by the target student, such as fixed-position passing and receiving in football, or the repetition of several movements in martial arts routines. Single-technique variation practice involves the repetition of a fixed movement or sequence of movements by the target student, but with different movement parameters during execution. For example, this could include passing and receiving from different distances in football, or kicking at different heights in martial arts. Fixed-combination technique practice involves the target student participating in prescribed and fixed multi-skill series practice, such as dribbling around poles and shooting in football, as well as martial arts routine practice. Multiple-technique stochastic practice refers to the target student practicing selecting appropriate skills for different situations in sports competitions, such as sports decision-making and skill selection during matches. Continuous movement refers to the target student participating in a relatively homogenous movement structure composed of multiple repetitions of a single movement, such as jogging warm-up, warm-up routines, and single-skill practice in both static and mobile positions. Sequential movement involves the target student participating in a more complex movement structure composed of multiple single movements connected together, such as agility and coordination quality exercises, rhythmic warm-up routines, and practice involving the connection of multiple skill movements.

Initial validation of EFRLPAAT

Based on the above theoretical basis, this study initially determined the dimensions and observation indicators of EFRLPAAT. This study used on-site distribution and email dissemination of expert consultation questionnaires to make initial judgments, adjustments, and supplements of indicators. In the questionnaire, the concept of executive functions was first introduced, and experts were informed that rich environmental stimuli, increased participation of peers, and complex exercise tasks help to improve the plasticity of the frontal lobe association area and enhance the behavioral performance of executive functions. On this basis, the dimensions, observation indicators, and related conceptual explanations of the physical education classroom systematic observation tool were provided (for details, see Sect. 2.2.1). Experts were asked to judge whether they agree or disagree with the dimensions and indicators of the observation tool based on the above concepts and theories. For example, for the predictable environment dimension of the sports environment, if an expert agrees that the indicator is an important observation indicator, they should mark a “√”; if they disagree, they should mark an “×”. In addition, experts were asked to revise the dimensions and indicators of the observation tool according to their own opinions, including the name, conceptual explanation, and any additional indicators that may be needed. The initial dimensions, observational indicators, and expert consultation tools of EFRLPAAT can be found in Supplementary Material 1.

The invited experts had a high degree of agreement on movement status and interpersonal interactions, recognizing that both dimensions are simple and straightforward, easy to observe and record, and have some association with executive function of children and adolescents. However, some problems were pointed out and some constructive modifications were provided in the three dimensions of sports environment, action structure and skills practice. Firstly, in the dimension of the sports environment, a number of experts suggested that “the environment refers to a wider range”. “From the perspective of school sports, the sports environment is a complex system that consists mainly of the natural, artificial and social environments of sports”, which is clearly incompatible with the notion of a manipulable environment in the PE classroom. Therefore, the experts recommended “replacing the sports environment with sports scenarios” and “predictable environment” and “unpredictable environment” with “predictable scenarios” and “unpredictable scenarios”. Secondly, in the dimension of action structure and skill practices, a number of experts felt that “the distinction between the two is not very strict”. Among them, single-technique fixed practices and single-technique variation practices correspond to continuous movements, both emphasizing the singularity of technical movements; fixed combination of technical practices and multiple techniques random practices correspond to sequential movements, both emphasizing the complexity of technical movements. Only the action structure observes not only skill practice time, but also includes preparation, relaxation, and other practice time. In addition, the dimension of skill practices was removed from this study in view of the experts’ suggestion that “skill practices are complicated to categorize and are not suitable for observation”. Finally, several experts suggested that the naming of continuous and sequential movements is too specialized. This study draws on the tool used by Fan [56] to categorize action structures into single and combined actions, which retains the original conceptual features and is easy for observers to understand. In addition, the concept of unpredictable scenarios was reformulated with reference to the suggestions of relevant experts, which suggested that such unpredictable scenarios not only originated from dynamic games or competitions, but also could be induced by signal stimuli, such as music, rhythms, colors, numbers, etc., to induce students to practice movement changes, which could promote the development of reaction speed, movement control, and cognitive flexibility. The revised analytical tool dimensions, indicators and concepts are explained in Table 2.

Table 2.

Dimensions, analytical indicators and conceptual explanations of the EFRLPAAT

Dimensions	Analyzing indicators	Conceptual explanations
Movement status	No movement	Students in the observed class were in a non-movement status, such as lying down, sitting, or standing.
	In situ movement	Students in the observed class were performing in-place movements, such as doing on-the-spot warm-up exercises, on-the-spot fitness training, and on-the-spot technical movement practices.
	Displacement movement	Students in the observed class were engaged in physical motion, such as jogging for warm-up, mobile fitness exercises, technical movement practices while moving, and games or competitions involving displacement.
Sports scenarios	Predictable scenarios	Students in the observed class were in sports with predictable environmental contexts, such as routine preparatory activities, routine physical exercises, and single techniques or fixed tactical routines.
	Unpredictable scenarios	Students in the observed classes played sports in unpredictable environmental contexts, such as chase games, “stimulus-response” games, and games or competitions based on technical and tactical training.
Action structure	Single actions	Students in the observed class participated in prescribed, fixed combinations of multiple technical drills, such as dribbling and shooting exercises in soccer and set drills in martial arts.
	Combined actions	Students in the observed class selected appropriate techniques to respond to situations during practice, such as making sports decisions and technical choices during games or competitions.
Interpersonal interaction	Individual practice/ games/ competitions	Students in the observed class had no interpersonal interactions during practice, such as individual sports skill practices, games or competitions for individual honors with low interactivity.
	Interactive exercises/ games/ competitions	Students have interpersonal interactions during exercises such as the practice of interactive motor skills, competing for team honors, games or competitions with a high level of interaction.

Content validity test of EFRLPAAT

The second round of expert consultation primarily focused on assessing the importance of analytical indicators to determine the content validity of the indicators. In the questionnaire, participants were also informed about the concept of executive functions, the elements that promote the development of executive functions, and the dimensions, observation indicators, and conceptual explanations of the classroom observation tool. Based on this, according to the opinions from the first round of expert consultation, the second round of expert consultation questionnaires were established using a 5-point Likert scale, assigning the values of 5, 4, 3, 2, and 1 to “Very Important,” “Important,” “Average,” “Unimportant,” and “Very Unimportant,” respectively. Experts were asked to score the dimensions and specific observation indicators of the observation tool according to their own opinions. For example, for the displacement movement indicator under the movement status, if an expert assigns a value of 4, it is considered an important indicator, and if a value of 1 is assigned, it is considered a very unimportant indicator. The expert consultation questionnaire can be found in Supplementary Material 2.

Final formation of EFRLPAAT

Based on the results of expert content validity, this study developed the EFRLPAAT instrument, which consists of 2 main parts: the basic information of the observation and the main content of the observation. Basic observation information includes date, school, specialization, and type of lesson, content of the lesson, age and gender of the teacher, number of students and gender ratio, and length of the observed lesson. Observations included executive function-related learning and practicing activity analysis indicators that required observers to make judgments about the elements involved in each minute of the PE classroom. This study referred to the coding principles of the system for observing the teaching of games in physical education (SOTG-PE) [57], and coded the level one and two indicators of the EFRLPAAT using an abbreviated format in English, with the observation recording time being 1 unit every 30s. Specific observation tool are shown in Supplementary Material 3.

Discriminant validity test of EFRLPPAT

After determining the content validity, EFRLPAAT is used to observe and record students’ learning and practice activities in physical education classes, and their discrimination is tested. The tool was developed based on our previously proposed theories [34, 35], which were distilled based on the effects of open and closed skill exercises on executive function and the motor attribute elements of these two types of skills. Therefore, logically, the tool should effectively distinguish the motor characteristics of open and closed skills. Based on this, this study conducted a discriminant validity test by observing open and closed skill courses. The observation includes 21 open-skill classes and 21 closed-skill classes, for details see Sect. 2.1 Participants. To prevent recording bias, this study used a Canon 6D video camera to film the teaching process in the classroom with the consent of the classroom teachers, during which no irritating, glaring sun rays and no interference from the shadows of obvious buildings were required. Classroom observations were conducted by the researcher and one master student trained in coding classroom observation tools.

Tnter-observer reliability of EFRLPAAT

This study examined the reliability of the EFRLPAAT instrument using inter- observer reliability. Systematic observation does not have specific requirements for the number of lessons to be observed, and the previous development of systematic observation tools mostly focused on about 10 lessons [58], so this study used the EFRLPAAT to conduct physical education classroom systematic observation on 10 lessons, including 5 open skill lessons and 5 closed skill lessons. In this study, 10 classroom videos were randomly selected from the aforementioned 42 for reliability analysis. The researcher and a professionally trained master’s student circled the codes for each activity to obtain the final time and proportion of each activity in the classroom.

Data analysis

This study used SPSS 25.0 software for statistical analysis and GraphPad Prism 8 software for data visualization. Firstly, Kendall’s coefficient of concordance was used to analyze the consistency of expert judgments, and the importance of expert indicator judgments was evaluated based on the selection rate and means. The selection rate refers to the proportion of invited experts who chose “very important” and “important”, and the larger the value, the more important the item is; the mean (M) refers to the simple arithmetic average of the experts’ judgmental scores, and the larger the mean value; the more important the item is [59]. This study drew on previous studies [59] and retained entries with a selection rate of ≥ 70% and M ≥ 3.5. Secondly, the normality test was conducted using the single-sample Shapiro-Wilk test, and it was found that the proportion of time spent on various activities in physical education classes did not conform to a normal distribution. Therefore, the Mann-Whitney U test was used for inter-group comparative analysis. Finally, referring to Roberts et al. [57], the total number of codes, consistent codes, and inconsistent codes of the observed courses were summarized, and the degree of inter-observer consistency was calculated based on [(consistent codes/total codes) × 100%].

Results

Content validity

The results of the second round of expert consultation consistency showed that Kendall W = 0.491, χ² = 53.047, p < 0.01, indicating that the expert opinions had moderate similarity. The results of the expert consultation on the indicators for the analysis of learning and practicing activities (Table 3) show that all the indicators meet the established criteria, except for the low selection rate of the indicator “no movement” (43.75%) for “movement status”.

Table 3.

Results of the expert consultation on the EFRLPAAT analytical indicators

Level one indicators	Level two indicators	Selection rate	M ± SD
Movement status		100.00%	4.81 ± 0.40
	No movement	43.75%	3.81 ± 1.11
	In situ movement	75.00%	4.19 ± 0.83
	Displacement movement	100.00%	4.81 ± 0.40
Sports scenarios		100.00%	4.75 ± 0.45
	Predictable scenarios	100.00%	4.63 ± 0.50
	Unpredictable scenarios	100.00%	4.69 ± 0.48
Action structure		100.00%	4.69 ± 0.60
	Single actions	87.50%	4.38 ± 0.72
	Combined actions	100.00%	4.88 ± 0.32
Interpersonal interaction		100.00%	4.69 ± 0.48
	Individual practice/ games/ competitions	93.75%	4.56 ± 0.63
	Interactive practice/ games/ competitions	100.00%	4.88 ± 0.32

Discriminant validity

The classroom observation results (Table 4) indicate that the proportion of time spent on displacement movement in open skill courses is significantly higher than that in closed skill courses (Z=-3.847, P < 0.001), while the proportion of time spent in no-movement status (Z=-2.535, P = 0.011) and in-situ movement (Z=-3.705, P < 0.001) is significantly lower than that in closed skill courses. The proportion of time spent on unpredictable scenario activities in open skill courses is significantly higher than that in closed skill courses (Z=-4.385, P < 0.001), while the proportion of time spent on predictable scenario activities is significantly lower than that in closed skill courses (Z=-4.385, P < 0.001). The proportion of time spent on composite movements in open skill courses, compared to closed skill courses, shows no significant difference (Z=-1.471, P >0.05), while the proportion of time spent on single movements is not significantly different compared to closed skill courses (Z=-0.549, P > 0.05). The proportion of time spent on interactive practice in open skill courses is significantly higher than that in closed skill courses (Z=-4.380, P < 0.01), while the proportion of time spent on individual practice is significantly lower than that in closed skill courses (Z=-4.059, P < 0.01). The above results generally support the previous interpretation that open skill exercises, which involve unpredictable situations and interpersonal interactions, have better intervention effects on executive functions than closed skill exercises [16, 18, 19], and have relatively good discrimination.

Table 4.

Results of a comparative analysis of classroom observations of open and closed skill physical education classes

Dimensions	Indicators	Open skills			Closed skills			Z	P
		M	SD	Rank Sum	M	SD	Rank Sum
Movement status	N	19.09	10.14	14.10	29.50	10.73	23.00	-2.535	0.011
	IS	20.80	22.90	12.52	54.67	16.99	25.54	-3.705	< 0.001
	D	59.57	25.29	22.67	16.87	16.40	9.15	-3.847	< 0.001
Sports scenarios	P	43.35	27.54	11.74	93.59	15.65	26.81	-4.385	< 0.001
	UP	55.46	27.99	23.26	4.94	12.07	8.19	-4.385	< 0.001
Action structure	S	21.25	18.29	18.24	17.41	15.23	16.31	-0.549	0.583
	C	58.47	24.44	19.48	51.29	20.39	14.31	-1.471	0.141
Interpersonal interaction	IV	23.49	18.86	12.05	62.93	19.23	26.31	-4.059	< 0.001
	IA	56.16	24.02	23.29	6.69	12.91	8.15	-4.380	< 0.001
Notes: M = Mean; SD = Standard Deviation; N = No movement; IS = In-situ movement; D = Displacement movement; P = Predictable scenario; UP = Unpredictable scenario; S = Single action; C = Combined action; IV = Individual practice; IA = Interactive practice.

Inter-observer reliability

Table 5 showed that there was 92.14% consistency in codes for open skills classes and 90.33% consistency in codes for closed skills classes; all special classes were coded at 85% or higher. With reference to Roberts et al. [57], it can be argued that the EFRLPAAT instrument has higher inter-observer agreement reliability.

Table 5.

Results of the inter-observer consistency assessment

Skill types	Observational course	Master code	Consistency code	Inconsistency code	Degree of consistency
Open skills		2596	2392	204	92.14%
	Basketball 1	396	353	43	89.14%
	Basketball 2	592	505	87	85.30%
	Volleyball 1	592	564	28	95.27%
	Volleyball 2	712	672	40	94.38%
	Football 1	304	298	6	98.02%
Closed skills		1872	1691	181	90.33%
	Martial arts 1	396	360	36	90.91%
	Martial arts 2	312	283	29	90.71%
	Martial arts 3	396	371	25	93.69%
	Aerobics 1	384	335	49	87.24%
	Aerobics 2	384	342	42	89.06%

Discussion

Interpretation of the EFRLPAAT

Based on the integrative theory [34, 35], this study has developed the EFRLPAAT for the first time, which specifically includes four dimensions and nine observation indicators: movement status, sports scenarios, action structure, and interpersonal interaction. The results show that the tool has good content validity, discriminant validity, and inter-observer reliability.

Firstly, the movement status includes three observation indicators: no movement, in stiu movement, and displacement movement. In the content validity test, this study found that the selection rate for the “no movement” status by the invited experts was generally low. However, this study still retained this observation indicator. Although in the relationship between movement and cognitive development, the “no movement” status is not closely related to the development of executive functions, this study, based on the comprehensiveness and systematicity of observation, referred to previous physical education classroom systematic observation tools [29, 30], and retained the indicator of “no movement”. Retaining this indicator is beneficial for researchers to fully observe the movement status of students in physical education classes, not just focusing on stationary and displacement movement states, which is beneficial for reflecting the students’ overall movement situation throughout the class. In addition, in the discriminant validity test, this study found that open skill classes have a higher proportion of displacement movement time, while closed skill classes have a higher proportion of no movement and in stiu movement time. This finding is consistent with previous research results [60]. This finding further suggests that one of the reasons for deficits in executive functioning among students in closed skill exercise interventions is the lack of necessary displacement sport, which is useful in providing guidance for subsequent instructional intervention practices.

Secondly, the sports scenarios include two observation indicators: predictable scenarios and unpredictable scenarios. Relevant studies [61, 62] have confirmed that a rich environment of stimuli has a stronger promotional effect on cognitive development, and executive functions are no exception [63]. These unpredictable sports scenarios, combined with physical exercise, are an important way for students to learn social rules and emotional expression. They not only promote the development of perceptual skills but also promote the development of executive control skills such as self-control and emotional regulation in solving complex problems [64]. Moreover, when students are placed in novel and challenging environments, they are more likely to explore the unknown, try new methods, and develop executive functions and creative thinking [65, 66]. Therefore, a study [67] has found that cognitive engagement in rich physical activities can promote the enhancement of executive functions. From the perspective of skill classification, open skills with rich environmental stimuli have been proven to have a stronger promotional effect on executive functions [16, 18]. The results of the discriminant validity test also confirm the advantages of open skill classes in promoting the development of executive functions, which has a positive significance for guiding teachers to carry out sports skill teaching in the future.

Thirdly, the action structure includes two observation indicators: single actions and combined actions. According to the integrative theory [34, 35], exercise methods with complex action structures can induce more neuroplasticity and promote the improvement of executive function task performance [51, 52]. However, the discriminant validity test did not find any significant differences in the action structures of students in open and closed skill classes. The reason for this is that the closed skill classes selected in this study include many martial arts routines and aerobics courses, which inherently have complex action structures. For example, the basic steps and hand movements of aerobics are relatively complex, the rhythm of the movements is fast, and there are many action segments involved, which helps to promote the improvement of executive functions [39]. Furthermore, Shi et al. [16, 18] found that skills such as aerobics and martial arts routines have a better promotional effect on children’s executive functions than single continuous movements like track and field, swimming, and cycling. Therefore, the action structure is not a unique factor that makes the promotional effect of open skills on executive functions superior to that of closed skills. However, this element of complex action structure is of positive significance for guiding the teaching of skills such as track and field, swimming, and cycling.

Finally, interpersonal interaction includes two observation indicators: individual practice/games/competitions and interactive practice/games/competitions. In physical exercise, especially in activities with strong interactivity, individuals not only need to respond quickly to the actions of their peers and opponents, but also need to remember complex game rules and strategies, and flexibly adjust cognition and behavior to choose the most appropriate course of action. Therefore, this adaptation to a variable environment can promote the development of executive functions. In addition, interpersonal interaction in sports activities also involves emotional communication, such as encouragement in team cooperation and challenges in competition. These emotional experiences can improve the brain’s ability to regulate emotions, thereby promoting the development of executive functions [35]. The discriminant validity test found that the proportion of interactive practice time in open skill classes is much higher than that in closed skill classes, supporting the view that open skills have a higher degree of interpersonal interaction [16, 18]. At the same time, this suggests that researchers can add elements of interpersonal interaction to closed skill classes to further enhance their effectiveness in promoting executive functions.

Implications for physical education practice

This study provides the following insights for future physical education practice. Firstly, the EFRLPAAT can assist researchers in effectively monitoring the learning and practicing activities of interventions during the experimental process, promote researchers to optimize and adjust specific measures of interventions based on the experimental process, and maximize the benefits of exercise in promoting executive functions among children and adolescents. Secondly, the EFRLPAAT provides teaching strategies for sport skills based on the perspective of executive function development, assisting physical education teachers in optimizing the organization and arrangement of classroom teaching according to movement status, sports scenarios, action structures, and interpersonal interactions.

Specifically, based on the main findings of the discriminant validity test, targeted changes can be made to the learning and practice activities of closed skills, thereby promoting the executive function benefits of students. Firstly, set up games that match the age of the students, create a lively teaching situation, and stimulate the interest of the students in learning and practicing the sport skills. For example, students are required to practice basic aerobics steps and the coordination of steps and upper limb movements according to different rhythms of musical accompaniment, and students are induced to think about how to improve their own movements in individual competitive kicking games of wushu, and to think about the use of pushing, pulling, leaning and other techniques in the two-player positional lunge pulling jousting games, and so on. Secondly, in the fitness practice to promote the development of sensitivity and coordination qualities of students, can be in situ or mobile state of the upper and lower limbs coordination practice or cross the opposite side of the limbs practice to improve the body coordination of students; can also create dynamic physical games, reduce the boring experience caused by the fixed practice. For example, in aerobics, single-footed jumps, double-footed jumps, parallel-footed jumps, open-and-close jumps, lunge jumps, and twisting jumps are randomly judged by accompanying music. Finally, the creation of complex situations containing a variety of solution ideas, through group inquiry learning, inspired students to change the orientation and rhythm of the technical movements, create and display the combination of learned movement routines, to cultivate a sense of autonomy, the spirit of innovation, and communication and collaborative problem-solving skills.

Limitations of this study

This study still has the following limitations. Firstly, due to limited resources such as manpower, financial support, and time, this study only conducted systematic observations on 42 physical education classes, with particularly few observation hours for small ball sports such as table tennis. This may have interfered with the accuracy of the research results, and further studies are needed to verify and explore the use of this tool. Secondly, although this study aims to provide an observation tool that can assist in demonstrating the effectiveness of experimental interventions, it has not yet provided detailed application examples. Therefore, further research is needed to promote, apply, and develop this tool. Thirdly, this tool was developed within the context of traditional Chinese culture and requires further cross-cultural examination to explore its universality. Finally, although the integrative theory, by synthesizing previous research, has found that introducing enriched environments, interpersonal interactions, and complex tasks can promote the enhancement of executive functions and the accumulation of sports experience, further research is still needed to verify this.

Conclusion

From the standpoint of enhancing executive functions through exercise, this research has pioneeringly created the EFRLPAAT, which demonstrates strong reliability and validity. This tool serves as a valuable aid for researchers to efficiently track the learning and practicing activities associated with intervention strategies throughout experimental procedures, thereby enhancing the efficacy of exercise-based interventions. In addition, the EFRLPAAT introduces a novel approach to the teaching of sports skills. It empowers physical education instructors to devise targeted learning and practicing activities that align with the developmental patterns of executive functions. By doing so, it not only fosters the enhancement of students’ executive functions but also accelerates the improvement of their sports skills. This innovative tool is a significant step forward in integrating cognitive development with physical education, offering a comprehensive strategy for educators and researchers alike.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

P. S. wrote and revised the manuscript. X. F. and B. W. processed the data and revised the manuscript. All authors participated in the intellectual content of the manuscript.

Funding

Not applicable.

Data availability

The data that support the findings of this study are available on request from the corresponding author upon reasonable request.

Declarations

Ethics approval and consent to participate

This study has been approved by the Scientific Research Ethics Committee of Shanghai University of Sport (102772023RT023). All participants gave written informed consent before enrolment in the study, which was conducted in accord with the principles of the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.