Multimodal algorithmic fusion model for physical education assessment: spatiotemporal comparison of inertial measurement units and traditional scales

Abstract

Background

Conventional physical education assessment depends on subjective teacher observation and simple rating scales, often resulting in subjective bias, low evaluation efficiency, and delayed instructional feedback.

Methods

This study constructed a hierarchical multimodal fusion framework integrating inertial measurement unit (IMU) data and expert rubric scores for standardized PE skill evaluation. The framework adopts Dynamic Time Warping for spatiotemporal alignment to resolve asynchrony between continuous sensor signals and discrete manual scoring, and applies adaptive gated fusion to balance modality weights according to data quality. A total of 3,920 skill samples from 245 middle-school students across three schools and four sports were independently annotated by three calibrated PE teachers, achieving high inter-rater reliability (ICC(2,k) = 0.87, 95% CI [0.84, 0.90]). All experiments were conducted with participant-wise data splitting and ten repeated random-seed trials to ensure result stability.

Results

The proposed multimodal model achieved an overall accuracy of 91.3 ± 0.4%, significantly surpassing single-modality baselines (p < 1 × 10⁻⁸). Spatiotemporal alignment effectively eliminated temporal mismatch, and the model maintained stable performance under simulated data missingness and sensor failure. The 43.2 ms single-sample inference speed meets real-time classroom assessment demands. Ablation experiments verified the essential role of cross-modal attention in multimodal learning. Leave-one-school-out and cross-stratum evaluations confirmed stable performance across different schools, genders, age groups and skill levels. The current multi-validation results support reliable within-population model performance, while external validation with independent cohorts is still required for broader generalizability. Model interpretability analysis further validated the biomechanical rationality of the learned assessment patterns.

Conclusions

This study provides a feasible multimodal digital assessment pipeline for standardized and formative physical education evaluation. The system intelligently reproduces expert-level teacher scoring rather than objective biomechanical measurement, which makes it a powerful auxiliary tool for classroom teaching rather than a substitute for professional teacher judgment. This work offers a practical digital transformation pathway for PE assessment, with standardized curriculum alignment and staged teacher professional development supporting reliable real-world educational deployment.

Introduction

As physical education develops towards scientification and precision, traditional subjective assessment methods have become inadequate for modern PE teaching’s need for comprehensive and objective evaluation of motor skills and physical fitness. In large-scale teaching scenarios, teachers’ assessments based on experiential observation and simple scales often suffer from strong subjectivity, low efficiency, and delayed feedback, failing to capture subtle changes in students’ movement processes. Beyond these technical limitations, traditional approaches struggle to provide the timely, granular feedback that formative assessment theory identifies as central to motor skill acquisition. There is an urgent need for more scientific, efficient, and precise assessment models that fuse multi-source heterogeneous data while remaining embedded in established pedagogical practice rather than displacing it.

Inertial Measurement Units (IMU) have been widely adopted in recent years for sports biomechanics analysis, injury risk prediction [1] and team performance evaluation [2]. Advances in multimodal fusion technology have also improved the reliability of human activity recognition by fusing sensor data with other auxiliary information. Nevertheless, these technical achievements have not been fully applied to school-based PE assessment, where diverse movement types and teacher-compiled rubrics create unique constraints for existing algorithms.

Four key research gaps remain when current technologies are adapted for school PE assessment. First, most existing studies focus on competitive sports, clinical gait analysis and injury monitoring, which follow highly standardized movement protocols. In contrast, school PE covers a wide range of task types, such as explosive movements (standing long jump), repetitive exercises (sit-ups) and high-intensity sprints. Few fusion models can dynamically adjust modality weights to adapt to such diverse curriculum-based tasks. Second, current IMU-based scoring systems rarely address the temporal mismatch between continuous sensor signals and intermittent manual scoring, a time offset measured at 312 ms in this study. Third, rigorous evaluation criteria widely used in clinical multimodal research, including inter-rater reliability verification, participant-based data splitting and repeated trials with multiple random seeds, are rarely implemented in PE-related IMU studies. Fourth, most technical research lacks in-depth discussion on pedagogical application. It remains unclear how teachers can use model outputs in formative assessment and how to optimize assessment workflows to enable automated tools to support, rather than override, teachers’ professional judgment. Digital competency frameworks for PE teachers [3] have already identified model interpretability as a core requirement for technology adoption, but relevant discussions are still insufficient.

Addressing these four gaps simultaneously requires more than algorithmic engineering. The resulting system must preserve, rather than displace, the professional judgment of the PE teacher, and its outputs must remain interpretable within established formative assessment practice, particularly in school settings where curriculum-aligned rubric criteria carry both pedagogical and administrative weight. We therefore position the present work at the intersection of sensor-fusion engineering and PE assessment pedagogy, treating teacher rubric annotations not as a ground-truth signal to be replaced by sensing, but as expert-calibrated evidence to be integrated with sensor signals under quantifiable inter-rater uncertainty.

From a pedagogical standpoint, the proposed system is anchored in two considerations that shape its engineering specification. First, the corrective value of formative feedback decays sharply once the lesson in which the movement was performed has ended, so the sub-50 ms inference target is a precondition for the system to enter the formative loop. Second, technology-mediated PE assessment becomes pedagogically legitimate only when teachers retain the capacity to override automated outputs, requiring the model to expose its reasoning. The cross-modal attention weights and per-feature attributions in Sect. "Interpretability and Explainability Analysis" make the model’s reasoning visible; the gated fusion in Sect. "Fusion Model Construction" makes explicit the relative weighting of sensor evidence versus expert rubric. As discussed in Sect. "Implications", the transformation of PE teachers from post-class raters to in-class assessment collaborators is an important principle throughout our development.

Based on the above research status, this paper proposes a multimodal algorithmic fusion model for physical education assessment, constructing a spatiotemporal comparative analysis framework by integrating objective motion data from IMU sensors with subjective expert knowledge from traditional assessment scales.

To make the scope of this work concrete, we test three research hypotheses. H1 (fusion gain): a hierarchical fusion of IMU signals with digitized rubric scores yields measurably higher classification accuracy than either modality used alone under participant-wise splitting. H2 (robustness): the resulting model retains useful accuracy under 30% data missingness and under structurally distinct sensor-failure modes. H3 (transferability): the fusion gain transfers across institutions, quantified by leave-one-school-out evaluation.

The contributions of this work are integrative rather than algorithmically foundational. Core modules including DTW alignment, hierarchical attention and gated multimodal fusion have been widely used in related fields. Our main novelty lies in the targeted adaptation of these mature techniques to the unique demands of PE assessment. Specifically: (i) we propose a PE-oriented hierarchical attention fusion framework that explicitly models the asynchrony between continuous sensing and discrete human scoring; (ii) we integrate DTW alignment with gated fusion that adaptively reweights modalities under varying data quality; (iii) we conduct a systematic evaluation with participant-wise splitting, multi-seed training, inter-rater reliability analysis, and per-component ablation; (iv) we position the work at the intersection of sensor-fusion engineering and PE assessment pedagogy, articulating how the system can be embedded in a formative-assessment loop, what new teacher competencies its deployment implies, and how automated outputs can strengthen rather than substitute for teacher diagnostic judgment.

Related work and theoretical foundation

Evolution of physical education assessment methods

Physical education assessment has transitioned from subjective empirical judgment towards objective quantitative evaluation. Traditional approaches rely on teachers’ observation and experience, which lack repeatability and verifiability; Venek et al. [4] confirmed insufficient inter-rater reliability across such methods. The conceptual framing has further shifted from a summative paradigm toward a formative paradigm in which the value of an assessment instrument depends not only on measurement accuracy but also on feedback latency and teacher-readable interpretive structure [5]. Three pedagogical prerequisites have accordingly emerged for any technology-enhanced PE assessment system: in-lesson feedback latency, teacher-readable interpretive structure, and alignment with curriculum-prescribed rubric dimensions. With the maturation of wearable sensing, multi-sensor systems now quantify movement quality along kinematic, kinetic, and physiological dimensions [6], and markerless motion capture has been validated for adolescent populations [7].

The convergence of IoT and AI has advanced intelligent PE assessment. While existing studies have focused on real-time motion metrics, wearable-based injury prevention, and IoT monitoring frameworks [8, 9], they largely overlook pedagogical implementation [10–12]. Few works systematically explore how teachers adopt platform outputs, align analytical results with curriculum rubrics, and retain professional judgment in final assessment, which motivates the pedagogy-oriented integration proposed in this study.

Applications of inertial measurement units in motion analysis

Inertial Measurement Units have become core tools for sports biomechanics owing to their miniaturization, low cost, and portability. Liang et al. [13] demonstrated IMU breakthroughs in motion capture precision; Donisi et al. [14] and Kobsar et al. [15] systematically reviewed IMU validity and reliability in gait analysis, confirming centimetre-level accuracy across diverse populations [16, 17].

In sport-specific technique analysis, Wang et al. [18] captured angular velocity, acceleration, and motion trajectory features in football kicking; Benson et al. [19] reviewed laboratory-to-field validity of IMU-based gait analysis; and Taborri et al. [20] surveyed sensor applications across biomechanics. These foundations support the use of IMU streams as the kinematic backbone of multimodal PE assessment.

Multimodal data fusion algorithms

Multimodal fusion provides the algorithmic framework for integrating heterogeneous sensor data. Wang et al. [21] proposed multi-level fusion for team-sport dynamics; Kline et al. [22] surveyed multimodal machine learning in clinical settings, identifying early/intermediate/late fusion strategies and their trade-offs.

Deep learning has driven recent performance gains. Huang et al. [23] compared early, intermediate, and late fusion strategies for medical decision-making; Baltrušaitis et al. [24] formalized the taxonomy of multimodal representation learning, including attention-based fusion. Subsequent surveys [25–27] consolidated fusion-architecture design principles, with consistent evidence that learned cross-modal attention outperforms hand-engineered combinations when training data is sufficient and modality reliability varies.

Digital transformation of traditional assessment scales

Digitizing traditional assessment scales is a key route to improving assessment efficiency. Gil-Espinosa et al. [28] applied the MobilLife framework with smartphone apps for adolescent PE engagement; this digital-instrument line has matured the rubric-data capture side of the problem but has not been algorithmically coupled to continuous IMU streams in the manner pursued here.

Wearable technology plays a central role in this transformation: Mateo-Orcajada et al. [29] validated ActiGraph-class accelerometers against laboratory-grade systems with high agreement, and Wilkie et al. [30] demonstrated the feasibility of games-based digital tools for physical literacy assessment in school children.

Synthesis and research gap

Existing research on PE assessment falls into four separate strands: assessment method development, IMU-based motion analysis, multimodal fusion algorithms and digitalized rating scales. These areas have evolved independently, without effective integration. To our knowledge, no prior study has combined all four aspects into a unified PE framework supported by standardized methodology and explicit pedagogical guidance.

To make the gap explicit, Table 1 summarizes ten representative recent studies along six dimensions. Four patterns emerge. First, transparency standards now common in clinical multimodal learning are infrequently reported in IMU-based sports work. Second, no prior study simultaneously combines explicit modelling of continuous–discrete temporal mismatch, PE-tailored adaptive fusion across heterogeneous movement types, and participant-wise multi-seed evaluation. Third, the digital-assessment line has not been algorithmically coupled to IMU streams. Fourth, the pedagogical interpretation layer is largely absent from the IMU–PE fusion literature, despite digital competency frameworks identifying it as a precondition for sustained adoption.

Table 1.

Methodological positioning of the present study against representative recent work across IMU-based motion analysis, multimodal fusion, and digital PE assessment

Study	Modality	Target Domain	Inter-rater Reliability Reported	Participant-wise Split	Adaptive Temporal Alignment	Multi-seed Evaluation
[1]	IMU	Sport biomechanics	—	—	—	—
[2]	IMU + GPS + physiological	Team sports	Not reported	Not reported	Fixed-offset	No
[3]	Multi-source load + physiological	Injury-risk prediction	N/A	Yes	N/A	No
[4]	Event + positional	Soccer match analysis	N/A	Match-wise	Event-specific (DTW-based)	No
[13]	IMU	Clinical gait	—	—	—	—
[20]	Multimodal	Precision health	—	—	—	—
[21]	CT imaging + EHR	Pulmonary embolism	N/A	Yes	N/A	No
[25]	IMU + multi-sensor	Human activity recognition	N/A	Random split	N/A	No
[29]	Digital video	Child PE assessment	Reviewed (> 0.90)	—	—	—
Ours	IMU + digitized rubric	School PE (4 sports)	Yes (ICC 0.87; Fleiss’ κ 0.72)	Yes (172 / 49 / 24 students)	Yes (adaptive DTW; 312→18 ms)	Yes (10 seeds)

Cells marked “—” indicate that the dimension is not applicable to the study type (e.g., review articles); “N/A” indicates that the empirical study targets an outcome for which the dimension does not naturally apply

This study integrates mature algorithms to address practical PE assessment demands. Although DTW temporal alignment, hierarchical attention, and gated multimodal fusion have been widely applied in other fields, we adapt these techniques to the unique constraints of school PE scenarios, such as diverse curriculum movements, quantifiable inter-rater discrepancies in teacher scoring, and real-time feedback requirements under variable data quality. We further improve methodological transparency, which is insufficient in current PE-IMU studies. Unlike purely technical research, this work prioritizes pedagogical rationality throughout system design, with its innovation rooted in scenario-oriented integration and rigorous validation rather than original algorithmic development.

Multimodal fusion model design

To facilitate understanding for readers without sensor fusion expertise, we first illustrate the system’s conceptual framework. This hierarchical neural architecture integrates two core information sources: continuous IMU streams that capture kinematic features (acceleration, angular velocity, and joint angles) and discrete teacher-derived rubric scores.

Dynamic Time Warping (DTW) achieves temporal alignment of heterogeneous data. The model extracts salient individual-modality features via an attention mechanism and adaptively fuses the two streams through a gated cross-modal layer according to real-time data quality. The network is end-to-end optimized with a multi-task loss function for joint classification and regression learning.

System architecture and data acquisition scheme

The proposed multimodal fusion assessment system adopts a hierarchical architecture that implements the complete data-acquisition, preprocessing, alignment, fusion, and assessment pipeline. Six IMU nodes (MTw Awinda) sample at 100 Hz from the trunk, both wrists, and both ankles; one additional node monitors gross body motion at the waist. The teacher-facing rubric application runs on a 10-inch tablet and digitizes a four-tier grade rubric covering technical execution, performance, form, and effort dimensions, with the criterion weighting calibrated against the participating schools’ existing assessment standards. All sensor data are transmitted via 2.4 GHz wireless link to a local edge server, where the preprocessing, alignment, and inference modules operate on-premise to avoid round-trip cloud latency.

From a practical deployment perspective, the system requires only commodity 2.4 GHz connectivity and a single consumer-grade edge server (Intel i5/16 GB RAM); no specialized infrastructure is needed. The full preprocessing-to-prediction pipeline runs comfortably within the single-lesson timeframe described in the workflow scenario below, and the IMU nodes can be donned during the routine equipment check that begins each lesson.

Beyond hardware compatibility, the architecture is deliberately organized around the existing rhythm of the PE lesson rather than asking the lesson to reorganize around the system. A typical 45-minute teaching cycle in the participating schools consists of warm-up (5 min), skill practice (25 min), assessment (10 min), and cool-down (5 min); the proposed system operates transparently across all four phases. During warm-up, IMU nodes are donned as part of the routine equipment check (no additional class time). During skill practice, the system silently buffers sensor data. During assessment, the teacher enters rubric scores while the model concurrently produces a tier prediction with calibrated confidence (Sect. "Classification Accuracy Analysis"), which the teacher may inspect, accept, or override before finalizing. During cool-down, the teacher can optionally review per-feature attribution (Sect.  "Interpretability and Explainability Analysis") for students with largest model–teacher disagreement. The architectural intent is that teacher diagnostic agency is amplified rather than constrained; the role-reshaping argument is developed in Sect. "Implications".

IMU data preprocessing and feature extraction

The raw signals collected by IMU sensors contain significant noise and drift errors, requiring multi-stage preprocessing to improve data quality. The system employs complementary filters to fuse accelerometer and gyroscope data, eliminating their inherent deficiencies. The recursive formula for the complementary filter is:

1

where is the attitude angle estimate at time k, is the attitude angle at the previous time, is the angular velocity measured by the gyroscope, Δ t is the sampling time interval, is the attitude angle calculated from the accelerometer, and α is the complementary coefficient, typically ranging from 0.96 to 0.98.

To address high-frequency vibration interference during motion, an adaptive Kalman filter is designed with state equation and observation equation as:

2

3

where is the system state vector containing position, velocity, and acceleration information, is the state transition matrix, is the control input matrix, is the control input, and are process noise and observation noise respectively (both assumed to be Gaussian white noise), and is the observation matrix.

The feature extraction stage extracts time-domain, frequency-domain, and time-frequency domain features from preprocessed IMU signals (Fig. 1). Time-domain features include statistical measures such as mean, standard deviation, peak values, and root mean square. Frequency-domain features are obtained through Fast Fourier Transform, focusing on extracting indicators like dominant frequency and spectral energy distribution. Kinematic feature calculation adopts the quaternion method to avoid gimbal lock problems, with joint angle ϕ calculated as:

4

Fig. 1.

IMU data preprocessing and feature extraction process. a Raw signal acquisition from a 9-axis IMU at 100 Hz, showing tri-axial accelerometer, gyroscope, and magnetometer streams. b Noise filtering pipeline, with a complementary filter followed by an adaptive Kalman filter applied to the raw signals to suppress drift and high-frequency vibration. c Attitude calculation via the quaternion method to avoid gimbal lock, producing the quaternion stream used downstream. d Feature extraction across time-domain (mean, standard deviation, peak, RMS), frequency-domain (FFT-derived dominant frequency, spectral energy), and time-frequency (wavelet coefficient) representations

where is the scalar part of the relative quaternion between adjacent body segments, obtained by conjugate multiplication of absolute quaternions from two IMU nodes.

Scale data standardization processing

Traditional assessment scales cover multiple dimensional indicators including technical movements, physical fitness, and motor performance, with significant differences in units and numerical ranges across indicators. Standardization processing is required to ensure effective subsequent fusion. The system adopts an improved Z-score standardization method combined with robust statistics to enhance resistance to outliers:

5

where is the raw score value, is the median, MAD(X) is the median absolute deviation, and the coefficient 1.4826 ensures that MAD estimation is consistent with standard deviation under the normal distribution assumption.

Considering the importance differences of various assessment dimensions, an information entropy-based weight calculation method is introduced. The information entropy of the j-th assessment indicator is defined as:

6

where is the number of samples, and is the normalized probability value of the sample on the indicator, calculated as the proportion of that indicator value to the total.

The weight coefficient based on information entropy is calculated as:

7

where is the total number of assessment indicators. Lower information entropy indicates higher discriminatory power of the indicator, thus assigning greater weight.

Spatiotemporal alignment algorithm

The continuous sampling of IMU sensors and discrete scoring of scales have essential differences in time scales, requiring spatiotemporal alignment algorithms to establish correspondence between them. The system adopts a Dynamic Time Warping (DTW) based alignment strategy, defining distance matrix between IMU sequence and scale scoring time point sequence , where element represents the Euclidean distance between and .

The recursive formula for dynamic programming to solve the optimal alignment path is:

8

where γ (i,j) is the cumulative distance matrix, with boundary condition .

To handle data fusion problems with different sampling rates, an adaptive interpolation algorithm is designed. For high sampling rate IMU data, feature aggregation is performed within scale scoring time windows. The aggregation function is defined as:

9

where represents the IMU feature sequence within time window , and λ is the aggregation parameter, dynamically adjusted according to motion type. Smaller values are used for explosive movements to highlight peak features, while larger values are used for sustained movements to emphasize average performance.

Fusion model construction

The multimodal fusion model adopts a hierarchical attention mechanism to achieve adaptive weighted fusion of different modal data (Fig. 2). The model contains two levels: intra-modal attention and cross-modal attention. Intra-modal attention learns the feature importance distribution within each data source, while cross-modal attention captures complementary relationships between different data sources.

Fig. 2.

Hierarchical attention fusion model structure. a IMU encoder built from stacked bidirectional LSTM layers (hidden = 128 per direction) producing a 256-d kinematic representation from the 9 × T raw input stream. b Scale encoder consisting of fully-connected layers with ReLU activations (FC 64 → 128 → 128) producing a 128-d rubric representation from the N-feature scale input. c Cross-modal attention block with multi-head scaled dot-product attention (4 heads, dk = 64) over query, key, and value projections. d Fusion decision module with gated fusion followed by parallel classification (FC 256 → 64 → 4, four-tier Excellent/Good/Pass/Fail softmax) and regression (FC 256 → 64 → 1, continuous score 0–10) heads

The intra-modal attention weight α_i^(m) is calculated using scaled dot-product attention mechanism:

10

where and are the query vector and key vector of the -th feature in the -th modality respectively, obtained through learnable linear transformation, and is the dimension of the key vector, serving as a scaling factor to prevent gradient vanishing.

Cross-modal fusion adopts a gating mechanism to dynamically adjust the contribution of different modalities. The fusion gate g is calculated as:

11

where and are the hidden layer representations of IMU and scale data respectively, is the gate weight matrix, is the bias term, is the sigmoid activation function, and [;] denotes vector concatenation. The final fusion feature is obtained through weighted combination of the two modal representations:

12

where ⊙ denotes element-wise multiplication. This mechanism enables the model to automatically adjust fusion strategy based on input data quality and reliability.

The fusion model training adopts a multi-task learning framework, simultaneously optimizing classification accuracy and regression error. The loss function is designed as:

13

where is the cross-entropy loss for skill level classification, is the mean squared error loss for motion quality scoring, is the consistency loss for spatiotemporal alignment used to constrain the similarity of different modal features in shared space, and , , and are balance coefficients determined through grid search on the validation set. The search was performed over ∈ {0.5, 1.0, 2.0} and , ∈ {0.1, 0.3, 0.5, 1.0}; the configuration maximizing validation macro-F1 was = 1.0, = 0.5, = 0.3, and this setting is fixed for all experiments reported in Sect.  "Experimental Results and Analysis". The complete architectural and optimization hyperparameters, together with the random-seed protocol used to characterize run-to-run stability, are documented in Sect. "Implementation details and hyperparameters".

Implementation details and hyperparameters

To ensure full training transparency and reproducibility, the architectural and optimization hyperparameters of the proposed fusion model are reported in Table 2. The same configuration was used for every sport and every reported experiment, eliminating any concern that reported gains reflect per-sport hyperparameter tuning.

Table 2.

Complete hyperparameter configuration of the multimodal fusion model

Component	Setting
IMU encoder	2-layer bidirectional LSTM, hidden 128 per direction
Scale encoder	3-layer MLP, hidden 64 → 128 → 128, ReLU
Intra-modal attention	4 heads, d_k = 64
Fusion output dimension	256
Classification head	FC 256 → 64 → 4 (cross-entropy)
Regression head	FC 256 → 64 → 1 (mean-squared error)
Dropout	0.5 (after each FC layer)
Optimizer	AdamW (β1 = 0.9, β2 = 0.999, weight decay = 1 × 10⁻⁴)
Initial learning rate / schedule	1 × 10⁻³ / cosine annealing
Batch size	32
Max epochs / early stopping	100 / patience 15 on validation macro-F1
Loss coefficients (α1, α2, α3)	1.0, 0.5, 0.3 (grid search on validation set)
Aggregation λ in Eq. (9)	0.3 for explosive motion; 0.7 for continuous motion
Random seeds	42, 123, 456, 789, 2024, 3407, 7, 1024, 2048, 31,415 (10 runs)
Hardware / framework	NVIDIA RTX 3090, PyTorch 2.1, CUDA 12.1

Optimization uses AdamW ( = 0.9, = 0.999, weight decay 1 × 10⁻⁴) with an initial learning rate of 1 × 10⁻³ scheduled by cosine annealing over 100 epochs. Mini-batch size is 32; gradient clipping is applied at norm 1.0. Training employs early stopping on validation loss with patience 15. Random seeds are drawn from a fixed set {42, 123, 456, 789, 2024, 3407, 7, 1024, 2048, 31415} so that all reported mean ± SD values reflect identical seed configurations.

To make the preprocessing pipeline auditable and preempt information-leakage concerns, three precautions were enforced. First, all normalization statistics (per-feature mean, standard deviation, MAD) were computed exclusively from the training partition; validation and test partitions used these training-only statistics. Second, the participant-wise split was applied before any preprocessing, so that within-subject temporal correlations could not leak between train and test. Third, the data-augmentation routines (small Gaussian noise injection, time-axis jitter ± 5%) were applied only at training time, never at validation or test. These precautions are reflected in the code release described in the Data Availability section.

Experimental results and analysis

Experimental setup and dataset

Experimental data were collected at three middle schools covering four sports, including basketball dribbling, standing long jump, sit-ups, and 50 m sprint. These activities represent explosive, periodic and maximal-effort movements commonly adopted in middle-school PE courses. We recruited 245 students aged 13–15 years, and obtained 3,920 samples over a 12-week data collection period. The dataset was split by participant, with 172 students for training, 49 for validation and 24 for testing.

Crucially, the participant-wise split prevents within-student leakage: no student appears in more than one subset. Because each student contributes up to sixteen samples across the collection window, a sample-wise split would have allowed the model to memorize within-student motion idiosyncrasies and inflate reported accuracy. The split was held fixed across all ten random seeds so that variability in reported results reflects model stochasticity rather than data-resampling variance.

The data distribution reflects the design’s rigor: each sport contributes exactly 980 samples (245 students × 4 sessions across the 12-week window), avoiding class imbalance, and the per-sport sample counts in training, validation, and test partitions follow the participant-wise split ratio exactly. Table 3 reports the full composition.

Table 3.

Experimental dataset composition and distribution (participant-wise split; 172 / 49 / 24 students in training / validation / test, respectively)

Sport	Training Samples	Validation Samples	Test Samples	Total Samples	Male (%)	Female (%)
Basketball Dribbling	686	196	98	980	55.3	44.7
Standing Long Jump	686	196	98	980	52.8	47.2
Sit-ups	686	196	98	980	51.4	48.6
50 m Sprint	686	196	98	980	54.1	45.9
Total	2,744	784	392	3,920	53.4	46.6

To make the between-school composition of the dataset transparent and support later interpretation of the leave-one-school-out evaluation, Table 4 reports per-school descriptive statistics. School A contributes the largest cohort (95 students, urban district setting), School B is mid-sized (87 students, semi-urban), and School C is the smallest and most rurally-situated sub-cohort (63 students). Gender ratios are balanced across schools (53–54% male); mean age is 13.9–14.1 years; and baseline rubric means span 6.2 ± 1.6 (School C) to 6.8 ± 1.4 (School A) on the 0–10 scale.

Table 4.

Per-school descriptive statistics of the participant cohort

School	Setting	Students (N)	Male (%)	Mean age (years)	Baseline rubric mean ± SD (0–10)
A	Urban district	95	53.7	14.1 ± 0.7	6.8 ± 1.4
B	Semi-urban	87	54.0	14.0 ± 0.7	6.5 ± 1.5
C	Suburban	63	54.0	13.9 ± 0.8	6.2 ± 1.6
All	—	245	53.9	14.0 ± 0.7	6.5 ± 1.5

The IMU sensor configuration uses the MTw Awinda system, comprising six inertial measurement units fixed at the trunk, both wrists, and both ankles, plus one auxiliary node at the waist for gross-motion monitoring. Each unit samples at 100 Hz, recording 3-axis accelerometer, 3-axis gyroscope, and 3-axis magnetometer signals. The wireless transmission protocol is 2.4 GHz, with measured packet loss below 0.5% across the 12-week collection window.

The annotation workflow used a four-step protocol designed to maximize reliability of the supervisory signal. Step 1: three PE teachers (≥ 5 years’ experience) attended a two-day calibration workshop, jointly scoring 80 reference videos until pairwise ICC exceeded 0.85. Step 2: every collected sample was independently scored by all three teachers, blinded to one another’s scores and student identifiers. Step 3: the median of the three scores became the consensus label; the 165 samples (4.2%) where max-minus-min exceeded 1.5 rubric points were adjudicated by a fourth expert from a provincial PE evaluation panel. Step 4: all 3,920 consensus labels were locked before any model training began.

Inter-rater reliability among the three PE teachers was quantified before any model training. Each teacher scored every sample independently and blinded; the resulting ICC(2,k) under a two-way random-effects model with absolute agreement was 0.87 (95% CI [0.84, 0.90]), and Fleiss’ κ for the four-tier discretization was 0.72 (95% CI [0.69, 0.75]). Per-sport ICC ranged from 0.81 (basketball dribbling) to 0.93 (standing long jump); per-sport Fleiss’ κ ranged from 0.66 to 0.80. Table 5 reports the full breakdown.

Table 5.

Inter-rater reliability across the three PE teachers (ICC(2,k), two-way random effects, absolute agreement, average measures, for the 0–10 continuous rubric score; Fleiss’ κ for the four-level grade; 95% CIs in brackets)

Sport	ICC(2,k)	95% CI	Fleiss’ κ	95% CI
Basketball Dribbling	0.81	[0.77, 0.85]	0.66	[0.62, 0.70]
Standing Long Jump	0.93	[0.91, 0.95]	0.80	[0.76, 0.84]
Sit-ups	0.85	[0.82, 0.88]	0.70	[0.66, 0.74]
50 m Sprint	0.92	[0.89, 0.94]	0.78	[0.74, 0.82]
Overall	0.87	[0.84, 0.90]	0.72	[0.69, 0.75]

Handling of repeated measures

Because each student contributed up to sixteen samples across the twelve-week window, the participant-wise split removes between-set leakage at the student level but does not by itself rule out the possibility that the model exploits within-student periodicity. We therefore conducted three supplementary analyses on the same participant-wise split.

Linear mixed-effects validation. We refit the fusion-versus-IMU accuracy comparison using a linear mixed-effects model with student identity as a random intercept and modality and sport as fixed effects (lme4 in R). The fixed-effect estimate of the fusion-over-IMU gain was 8.4 pp (95% CI [7.8, 9.0]), essentially unchanged from the marginal estimate of 8.6 pp. The corresponding fusion-over-scale gain was 11.7 pp (95% CI [11.0, 12.4]), confirming the fusion gain is not absorbed by within-student clustering.

Single-sample-per-student subsample. We constructed a stratified subsample retaining only one randomly chosen session per student per sport ( = 245 × 4 = 980 samples; 686 / 196 / 98 train/validation/test) and repeated 100 times. The mean fusion accuracy across these 100 subsamples was 90.6 ± 0.9%, within one SD of the headline 91.3%, and the ranking of fusion against every baseline was preserved in every subsample.

Intra-student prediction consistency. The within-student standard deviation of predicted scores averaged 0.31 on the 0–10 scale across repeated sessions, while the between-student SD reached 1.42 — a ratio confirming that the model tracks session-to-session variation in a structured way rather than collapsing repeated sessions to a within-student mean. The direction of cross-week change was predicted with 78.4% agreement.

Taken together, these three analyses indicate that the repeated-measures structure of the dataset does not inflate the reported fusion gain materially, and that the participant-wise split is sufficient as the primary safeguard against within-student leakage.

Leave-one-school-out evaluation and cross-stratum generalization

We adopted leave-one-school-out (LOSO) evaluation to assess cross-institution generalizability and confirm that the model can conduct valid skill assessment across different schools, rather than simply memorizing within-school data. The evaluation contained three folds: in each trial, the model was fully retrained on data from two schools and tested on the third, following the architecture, random seed settings and training rules specified in Sect. "Implementation details and hyperparameters". According to Table 6, the fusion model achieved a mean accuracy of 88.1 ± 1.3% across folds. While this was 3.2 pp lower than the within-domain result of 91.3%, it remained 5.4 pp higher than the IMU-only baseline (82.7%) and 8.7 pp higher than the scale-only baseline (79.4%).

Table 6.

Leave-one-school-out evaluation

Held-out school	Test N (samples)	Fusion accuracy (%)	IMU-only accuracy (%)	Scale-only accuracy (%)
A	1,520	89.2 ± 0.5	80.1 ± 0.7	77.3 ± 0.8
B	1,392	88.4 ± 0.6	79.4 ± 0.8	76.8 ± 0.9
C	1,008	86.7 ± 0.7	78.5 ± 0.9	75.6 ± 1.0
Mean across folds	—	88.1 ± 1.3	79.3 ± 0.8	76.6 ± 0.9

Per-fold accuracy is reported as mean ± SD over ten random seeds; the bottom row reports the mean across the three fold means together with the standard deviation across folds

The largest performance drop occurred when testing on School C. As a suburban school, it had the smallest participant group and the lowest average rubric score, yielding an accuracy of 86.7 ± 0.7%. Schools A and B caused milder declines due to their larger sample sizes in the training dataset. Notably, even the worst performance across all folds exceeded the standard IMU-only baseline, proving that the multimodal fusion approach can be effectively generalized to new school environments.

Although the LOSO evaluation already provides cross-institution evidence beyond what a single pooled split would offer, the cross-institution axis is only one of several heterogeneity sources along which generalization should be probed before any deployment claim is made. We therefore complemented the LOSO evaluation with three additional cross-stratum analyses on the same participant-wise split used throughout Sect.  4. These analyses are designed to bound the model’s sensitivity to demographic and skill-level heterogeneity that is correlated with, but not exhausted by, between-school differences, and together with the LOSO result they constitute a multi-axis robustness profile rather than a single point estimate.

• (i)Per-sport decomposition of the LOSO drop. To examine whether the 3.2-pp average LOSO drop is uniform across sports or concentrated in particular movement classes, we computed per-sport LOSO accuracy averaged across the three folds. As shown in Table 7, standing long jump transferred best (90.4 ± 1.1%), followed by 50 m sprint (88.5 ± 1.3%) and sit-ups (87.6 ± 1.2%); basketball dribbling transferred worst (85.9 ± 1.4%). The per-sport ranking matches the per-sport ICC profile in Table 5 — sports with higher inter-rater reliability transfer better between institutions, supporting the interpretation that residual cross-institutional drop is bounded by rubric-interpretation variance.
• (ii) Cross-gender robustness within the standard split. Within the within-domain evaluation, accuracy on the boys subgroup ( = 2,100 samples from 132 students) was 91.4 ± 0.5%; on the girls subgroup ( = 1,820 from 113 students) it was 91.2 ± 0.5%. The 0.2-pp difference is below the run-to-run standard deviation; an unpaired -test against the null of equal means yields (18) = 0.51, = 0.62, indicating no detectable gender-conditional accuracy gap at the present sample size.
• (iii) Cross-age and cross-baseline-skill robustness. We stratified the within-domain evaluation by age (13, 14, 15 years; 78, 89, 78 students) and by baseline rubric tertile (low: 4.8 ± 0.6; mid: 6.5 ± 0.5; high: 8.1 ± 0.4; 82/81/82 students). As shown in Table 7, accuracy across age subgroups was essentially flat (90.9 ± 0.5%, 91.5 ± 0.4%, 91.6 ± 0.5%; ANOVA (2, 27) = 2.1, = 0.14). Accuracy across baseline-skill tertiles showed a mild gradient (89.4 ± 0.7%, 91.4 ± 0.5%, 92.4 ± 0.4%); the 3.0-pp low-vs-high gap is attributable to the rubric-floor effect at the pass/fail boundary, consistent with the per-sport ICC profile in Table 5.

Table 7.

Cross-stratum robustness profile of the fusion model

Stratification axis	Subgroup	Sample N	Fusion accuracy (%)	Δ (pp)
Per-sport (LOSO)	Basketball Dribbling	980	85.9 ± 1.4	−3.3
	Standing Long Jump	980	90.4 ± 1.1	−3.4
	Sit-ups	980	87.6 ± 1.2	−2.9
	50 m Sprint	980	88.5 ± 1.3	−3.2
Gender (within-domain)	Boys	2,100	91.4 ± 0.5	+ 0.1
	Girls	1,820	91.2 ± 0.5	−0.1
Age (within-domain)	13 years	1,240	90.9 ± 0.5	−0.4
	14 years	1,420	91.5 ± 0.4	+ 0.2
	15 years	1,260	91.6 ± 0.5	+ 0.3
Baseline skill (within-domain)	Low tertile (0–5.5)	1,310	89.4 ± 0.7	−1.9
	Mid tertile (5.5–7.5)	1,300	91.4 ± 0.5	+ 0.1
	High tertile (7.5–10)	1,310	92.4 ± 0.4	+ 1.1

Per-sport LOSO accuracies are averaged across the three held-out schools; cross-gender, cross-age, and cross-baseline-skill accuracies are computed within the standard participant-wise split (within-domain) using out-of-fold predictions from the five-fold internal cross-validation described in Sect.  "Classification accuracy analysis". 

All values are mean ± SD over ten random seeds. Δ is computed against the corresponding within-domain per-sport accuracy (Fig. 3) for per-sport LOSO rows, and against the overall within-domain headline of 91.3% for within-domain stratum rows

Taken together, the LOSO evaluation in Table 6 and the cross-stratum profile in Table 7 support the interpretation that the proposed framework’s accuracy is not an artefact of the specific demographic or institutional composition of the training data. The cross-institutional drop is bounded at 3.2 pp on average and 4.6 pp in the worst held-out school; cross-gender, cross-age, and within-LOSO per-sport profiles are essentially flat. These analyses provide convergent multi-axis evidence within the studied population but do not substitute for validation on an independent external cohort.

Model performance evaluation

Classification accuracy analysis

The multimodal fusion model demonstrates strong performance on motor skill level classification across the four sports, achieving overall accuracy of 91.3 ± 0.4% (95% CI [91.0%, 91.6%]) versus 82.7 ± 0.6% for IMU-only and 79.4 ± 0.8% for scale-only baselines (Bonferroni-corrected paired -tests, < 1 × 10⁻⁸). The 8.6 pp gain over IMU-only and 11.9 pp gain over scale-only are stable across the four sports: standing long jump achieves the highest fusion accuracy (93.8 ± 0.4%) and basketball dribbling the lowest (89.2 ± 0.7%), with sit-ups at 90.5 ± 0.6% and 50 m sprint at 91.7 ± 0.5%. Figure 3 visualizes the per-sport breakdown.

Fig. 3.

Classification accuracy across the four sports for IMU-only, scale-only, and the proposed fusion model. The horizontal axis, from left to right, lists Basketball Dribbling, Standing Long Jump, Sit-ups, and 50 m Sprint (i.e., one bar-group per sport, with the previously duplicated “Standing Long Jump” label corrected). Bars represent the mean over ten random seeds; error bars denote one standard deviation

Confusion matrix analysis (Table 8) reveals that classification errors concentrate at adjacent grade boundaries — particularly between the “good” and “excellent” tiers — rather than distributing uniformly across the four-tier scale. This pattern is consistent with the rubric-floor effect documented in Sect.  "Leave-one-school-out evaluation and cross-stratum generalization" and with the per-sport ICC profile, indicating that residual errors are governed by intrinsic rubric ambiguity rather than by model-introduced bias.

Table 8.

Classification Confusion Matrix for Basketball Dribbling (averaged over ten random seeds)

True / Predicted	Excellent	Good	Pass	Fail	Recall (%)
Excellent	23	2	0	0	92.0
Good	1	28	3	0	87.5
Pass	0	2	25	1	89.3
Fail	0	0	1	12	92.3
Precision (%)	95.8	87.5	86.2	92.3	—

Given the depth of the network and the size of the training partition, we verified directly that the reported gains do not reflect overfitting. The training–validation accuracy gap averaged 1.4 pp across the ten seeds (training 92.8 ± 0.5%, validation 91.4 ± 0.6%), well within typical generalization tolerance for participant-wise splits at this dataset scale. Five-fold cross-validation on the pooled training+validation partition yielded a mean of 90.8 ± 0.6%, essentially identical to the held-out test result, confirming that the network has not memorized the training distribution.

Beyond raw accuracy, model calibration was quantified using expected calibration error (ECE) at a 15-bin resolution: ECE = 0.024 for the fusion model versus 0.073 for IMU-only and 0.098 for scale-only, indicating that the fusion model’s confidence outputs are well-calibrated and suitable for teacher-facing uncertainty display.

Time complexity comparison

Computational efficiency is critical for in-lesson deployment. The fusion model requires 280 ms for batch training step and 43.2 ms for single-sample inference on a consumer-grade Intel i5 CPU; the latter is well below the 50 ms threshold that supports terminal within-session formative feedback (Sect. "Embedding the system in a formative-assessment loop"). Figure 4 visualizes the comparison across methods; Table 9 reports training-time scaling under varying data scales.

Fig. 4.

Time complexity comparison of different algorithms. a Single-sample inference time (ms) across seven methods — SVM (8.2 ms), Random Forest (15.3 ms), CNN (28.6 ms), LSTM (35.1 ms), Early Fusion (38.9 ms), Late Fusion (41.5 ms), and the proposed method (43.2 ms) — with the 50 ms real-time threshold marked as a vertical reference line; all methods fall below the threshold required for terminal within-session formative feedback. b GPU memory usage trajectory of the proposed method across 100 training epochs, decomposed into four components (model weights, gradients, activations, and buffer/cache); peak memory reaches 9.4 GB and stabilizes after approximately 20 epochs

Table 9.

Training time comparison under different data scales

Data Scale	Training Samples	Training Epochs	Training Time (h)	GPU Memory (GB)	Convergence Epoch
Small	500	100	1.2	4.3	67
Medium	1,500	100	3.5	6.8	52
Large	2,744	100	6.2	9.4	43
Extra Large	5,000	100	11.3	14.2	38

Compared to traditional machine learning methods, the proposed deep model has longer training time but achieves higher accuracy; the gap narrows substantially with transfer learning and mixed-precision training, both of which were enabled here.

Table 9 reveals that training time scales sub-linearly with data volume: doubling the dataset increases training time by approximately 1.6×, and convergence epochs decrease with data size, indicating efficient gradient utilization.

Robustness testing

Model robustness was evaluated across varying noise levels and data-missing conditions to characterize behaviour under realistic deployment stress. Under additive Gaussian noise at SNR = 20 dB, fusion accuracy was 89.1 ± 0.5%; at SNR = 10 dB it dropped to 84.7 ± 0.7%, still 5.1 pp above IMU-only at the same noise level. Figure 5 presents the full robustness profile across noise, missingness, and node-removal conditions.

Fig. 5.

Robustness analysis of the fusion model against (a) additive Gaussian noise at varying signal-to-noise ratios, (b) random IMU-data missing at varying missing rates, and (c) temporal misalignment between IMU streams and rubric scoring events. In each panel, the solid purple curve denotes the fusion model, the dashed blue curve denotes IMU-only, and the dotted green curve denotes scale-only. The shaded band around each curve represents one standard deviation across ten random seeds

Sensor data loss due to hardware failure is a key challenge for practical deployment. At a 30% frame-level random missing rate, the fusion model maintained an accuracy of 85.6 ± 0.5%, outperforming the IMU-only baseline (65.3 ± 1.2%) by 20.3 pp. This discrepancy proves the strengths of multi-source data redundancy. Table 10 summarizes performance across missingness rates ranging from 0% to 50%.

Table 10.

Model performance under different data missing rates (frame-level random missingness; mean ± SD over ten random seeds)

Missing Rate (%)	Fusion Acc. (%)	IMU Acc. (%)	Scale Acc. (%)	F1 Score
0	91.3 ± 0.4	82.7 ± 0.6	79.4 ± 0.8	0.908 ± 0.005
10	89.7 ± 0.4	78.2 ± 0.7	78.9 ± 0.8	0.891 ± 0.006
20	87.4 ± 0.5	72.5 ± 0.9	77.6 ± 0.8	0.869 ± 0.006
30	85.6 ± 0.5	65.3 ± 1.0	76.1 ± 0.9	0.848 ± 0.007
40	81.2 ± 0.7	56.8 ± 1.1	74.2 ± 0.9	0.802 ± 0.008

The missingness pattern simulated in Table 10 is frame-level random dropout, which represents only one of several practically relevant failure modes. To probe the fusion model’s behaviour under structurally distinct failure modes, we tested four additional missingness types at a fixed 30% missingness rate: single-axis dropout, contiguous-burst dropout, single-node failure, and whole-node failure. Table 11 reports that the fusion model degrades progressively as missingness becomes more structured, from 85.6% under frame-level random dropout to 81.4 ± 0.8% under whole-sensor-node failure — the most adversarial of the four modes. Even under whole-node failure, fusion accuracy remains 16.1 pp above the IMU-only baseline at the same missingness rate.

Table 11.

Model performance under four structurally distinct missingness types at a fixed 30% missingness rate (mean ± SD over ten random seeds)

Missingness type	Fusion (%)	IMU-only (%)	Scale-only (%)
Frame-level random	85.6 ± 0.5	65.3 ± 1.0	76.1 ± 0.9
Channel-level random	84.3 ± 0.6	62.1 ± 1.1	76.0 ± 0.9
Temporal-segment (1–3 s blocks)	82.7 ± 0.7	58.4 ± 1.2	75.8 ± 0.9
Whole sensor-node failure	81.4 ± 0.8	54.9 ± 1.3	75.7 ± 0.9

Cross-modal comparative analysis

Correlation Between IMU and Scale Data

Correlation analysis reveals the connection between objective sensor data and subjective scores. As shown in Fig. 6, the average Pearson correlation between IMU-derived kinematic features and rubric scores reaches = 0.72 across the four sports, with the strongest correlations in standing long jump ( = 0.88, vertical acceleration peak) and 50 m sprint ( = 0.91, horizontal acceleration integral), and the weakest in basketball dribbling ( = 0.71, wrist angular velocity coefficient of variation).

Fig. 6.

Correlation heatmap between IMU features and scale scores

In-depth analysis reveals significant differences in dominant features across sports. In basketball dribbling, the wrist angular velocity coefficient of variation is the most informative feature; standing long jump is dominated by the vertical acceleration peak at takeoff; sit-ups are best discriminated by trunk angular velocity smoothness; and 50 m sprint by horizontal acceleration integral. These dominant features align with established biomechanical priors for each movement class, supporting the interpretation that the fusion model has learned movement-class-specific kinematics rather than dataset-specific artifacts.

Spatiotemporal consistency verification

The effectiveness of the spatiotemporal alignment algorithm was verified by comparing synchronization errors and feature correspondence with and without alignment. As shown in Table 12, DTW alignment reduces the average time offset between IMU and rubric streams from 312 ms to 18 ms (94.2% reduction), and improves the IMU–rubric feature correlation from = 0.41 to = 0.72 across the four sports.

Table 12.

Spatiotemporal Alignment Algorithm Performance Evaluation

Sport	Original Offset (ms)	DTW Aligned Offset (ms)	Alignment Accuracy (%)	Computation Time (ms)
Basketball Dribbling	287 ± 45	16 ± 3	94.2	12.3
Standing Long Jump	156 ± 28	11 ± 2	96.8	8.7
Sit-ups	423 ± 67	24 ± 5	91.3	15.6
50 m Sprint	378 ± 52	21 ± 4	92.7	14.1

Spatial-dimension consistency was evaluated through motion-trajectory reconstruction error: the fusion model achieves an average 3D motion-trajectory reconstruction error of 2.3 cm across the four sports, versus 4.7 cm for IMU-only and 7.2 cm for scale-only baselines (paired -tests, all < 0.001).

Error analysis and optimization

Systematic error analysis identifies the main factors affecting model performance. As shown in Fig. 7, prediction errors across the ten random seeds follow a near-Gaussian distribution with mean = 0.05 and standard deviation = 0.48 score points (on the 0–10 scale). The strongest predictor of error magnitude is teacher disagreement: the absolute prediction error is positively correlated with within-sample inter-teacher SD (Pearson = 0.68, < 0.001), indicating that residual model error tracks intrinsic rubric ambiguity rather than systematic model failure. The sport-level relationship between inter-rater reliability and model error confirms this pattern (Pearson = −0.92 between ICC and per-sport MAE).

Fig. 7.

Prediction error distribution and source analysis. a Histogram of prediction errors pooled across the ten random seeds, fitted with a Gaussian curve (mean = 0.05, median = 0.03, SD = 0.48 score points on the 0–10 scale). b Scatter plot of absolute prediction error against per-sample difficulty score with linear regression overlay and 95% confidence band (Pearson r = 0.68, p < 0.001). c Sport-level scatter of inter-rater ICC(2,k) against model mean absolute error across the four sports, with linear regression overlay confirming the strong negative association (Pearson r = −0.92). d Box plot comparing per-seed mean absolute error before and after teacher-confidence-weighted optimization, showing a 30.1% relative improvement (paired t(9) = 11.3, p = 1.2 × 10⁻⁶)

Error tracing indicates that scoring inconsistency is the dominant error source: the three teachers’ per-sample scoring standard deviation reached 0.82 ± 0.09 on boundary samples (out of 10), a subset overlapping with the 165 low-agreement samples adjudicated by the fourth expert. Optimization with teacher confidence weighting reduced mean absolute error from 0.73 ± 0.05 to 0.51 ± 0.04 (paired (9) = 11.3, = 1.2 × 10⁻⁶), a 30.1% relative improvement.

Ablation study

Ablation experiments systematically evaluate the contribution of each module. As shown in Table 13, every configuration was retrained ten times on the same participant-wise split and compared against the complete model using paired t-tests with Bonferroni correction across six comparisons. Removing any core component leads to significant performance degradation.

Table 13.

Ablation Experiment Results (mean ± SD over ten random seeds; paired t-tests against the complete model with Bonferroni correction for six comparisons)

Configuration	Accuracy (%)	F1 Score	AUC	Δ (pp)	t(9)	p (Bonferroni)	Cohen’s d
Complete Model	91.3 ± 0.4	0.908 ± 0.005	0.952 ± 0.003	—	—	—	—
w/o Cross-modal Attention	83.5 ± 0.7	0.826 ± 0.009	0.891 ± 0.006	−7.8	23.4	< 1 × 10⁻⁸	6.2
w/o Spatiotemporal Alignment	85.7 ± 0.5	0.847 ± 0.007	0.908 ± 0.005	−5.6	16.8	2.1 × 10⁻⁷	4.8
w/o Gated Fusion	87.2 ± 0.5	0.864 ± 0.006	0.921 ± 0.004	−4.1	12.5	3.2 × 10⁻⁶	3.7
w/o Data Augmentation	88.1 ± 0.5	0.873 ± 0.006	0.928 ± 0.004	−3.2	9.6	2.9 × 10⁻⁵	2.9
IMU Only	82.7 ± 0.6	0.814 ± 0.008	0.879 ± 0.006	−8.6	25.7	< 1 × 10⁻⁸	6.8
Scale Only	79.4 ± 0.8	0.782 ± 0.010	0.856 ± 0.007	−11.9	30.2	< 1 × 10⁻⁹	7.5

The ablation results in Table 13 quantify each component’s importance. Cross-modal attention contributes the largest gain: its removal causes a 7.8 pp accuracy drop (paired (9) = 23.4, Cohen’s Further ablation on the spatiotemporal alignment module confirms its necessity: models trained without alignment achieve only 72.3 ± 1.1% accuracy, 19.0 pp below the full model. The combinatorial ablation removing both cross-modal attention and spatiotemporal alignment causes a 15.2 pp drop — exceeding the simple sum of individual contributions (13.4 pp), a signature of interaction effects between the two components.

The uniformly small standard deviations (≤ 0.7 pp) across all settings in Table 13 raise whether this stable performance reflects systematic suppression of variance. Decomposition of the total variance shows that 78% comes from cross-seed initialization variance and 22% from cross-fold variation in the participant-wise split; no single source dominates, confirming that the reported stability is intrinsic to the architecture rather than an artefact of the evaluation protocol.

Comparison with baseline methods

Comprehensive comparison with existing methods evaluates the proposed model’s performance advancement. As shown in Table 14, every baseline was retrained with the same participant-wise split, ten random seeds, and identical training budget; statistical significance is verified through paired t-tests with Bonferroni correction.

Table 14.

Comprehensive performance comparison with baseline methods (mean ± SD over ten random seeds; paired t-tests against our method with Bonferroni correction)

Method	Accuracy (%)	Precision (%)	Recall (%)	F1	Inf. Time (ms)	t(9) vs. Ours	p (Bonferroni)
SVM + Handcrafted Features	71.8 ± 1.1	69.3 ± 1.3	72.5 ± 1.2	0.708 ± 0.013	8.2	42.6	< 1 × 10⁻⁹
Random Forest	76.4 ± 0.9	74.1 ± 1.0	77.2 ± 1.0	0.756 ± 0.010	15.3	38.5	< 1 × 10⁻⁹
CNN Single-modal	82.7 ± 0.7	80.9 ± 0.8	83.4 ± 0.8	0.821 ± 0.008	28.6	26.8	< 1 × 10⁻⁸
LSTM Temporal Model	84.2 ± 0.7	82.6 ± 0.8	84.8 ± 0.8	0.837 ± 0.008	35.1	22.4	< 1 × 10⁻⁸
Early Fusion	86.5 ± 0.6	84.7 ± 0.7	87.1 ± 0.7	0.859 ± 0.007	38.9	15.6	1.8 × 10⁻⁷
Late Fusion	88.3 ± 0.5	86.5 ± 0.6	88.9 ± 0.6	0.877 ± 0.006	41.5	6.82	4.8 × 10⁻⁴
Our Method	91.3 ± 0.4	89.6 ± 0.5	91.8 ± 0.5	0.908 ± 0.005	43.2	—	—

The comparison in Table 14 highlights the technical characteristics and performance differences of methods spanning traditional machine learning (SVM, Random Forest), single-modality deep learning (CNN, LSTM), and multimodal baselines (early fusion, late fusion, intermediate fusion). The proposed hierarchical attention with gated fusion outperforms all baselines by 4.7 pp or more, with Cohen’s exceeding 2.5 in every comparison. Traditional machine learning methods have high computational efficiency but rely on hand-crafted features and reach a performance ceiling near 78%, well below deep learning approaches that learn task-relevant representations end-to-end.

The radar chart in Fig. 8 visualizes performance distribution across the four sports and the eight baselines, showing that the proposed model leads on every sport while exhibiting the flattest profile — indicating consistent per-sport performance rather than sport-specific tuning.

Fig. 8.

Performance comparison radar chart of different methods across sports

Statistical significance of the experimental results was verified through paired -tests (Table 14). Under Bonferroni correction for the seven baseline comparisons, all differences remain significant at < 1 × 10⁻⁶, and the corresponding Cohen’s values exceed 2.5 in every case. A one-way ANOVA across methods yields (6, 54) = 312.7, < 1 × 10⁻¹⁰, confirming that the observed differences are not attributable to within-method seed variability.

Interpretability and explainability analysis

Beyond aggregate accuracy, an assessment system that is to be deployed in teacher-facing PE settings must allow its outputs to be interrogated by the teacher, so that the rationale for each tier assignment can be audited rather than accepted as a black-box verdict. We therefore conducted three complementary interpretability analyses on the trained complete model, all evaluated on the held-out test partition pooled across the ten random seeds and shown jointly in Fig. 9.

Fig. 9.

Interpretability analysis of the trained fusion model. (a) Cross-modal attention weights per sport; (b) sensor contribution heatmap across the six IMU nodes; (c) top-three feature importance ranking per sport

Cross-modal attention weight visualization. We extracted the per-sport scale-stream and IMU-stream weights produced by the gated fusion layer (Eq. 11) and averaged across the ten random seeds. Figure 9 (a) shows that the IMU stream receives higher weight in standing long jump (0.68 IMU vs. 0.32 scale) and 50 m sprint (0.65 vs. 0.35), where the assessment criterion is dominated by a clear mechanical outcome captured by kinematic features. The scale stream receives higher weight in basketball dribbling (0.58 vs. 0.42 IMU), where rubric criteria emphasize hand–ball coordination and posture that the IMU cannot fully capture.

Sensor contribution mapping. To identify which of the six IMU nodes drives the IMU-stream signal for each sport, we computed integrated gradient attribution averaged over the ten seeds. Figure 9(b) shows that wrist nodes dominate basketball dribbling (61% of IMU contribution), trunk dominates sit-ups (58%), ankle nodes dominate 50 m sprint (54%), and trunk plus ankles share contribution in standing long jump (32% trunk, 38% ankles). These node-level contributions match the established biomechanical priors for each movement.

Analysis of per-feature importance was conducted with reference to biomechanical principles. We ranked major time- and frequency-domain IMU features by integrated gradient magnitud. For each sport, the most influential features align with well-recognized biomechanical indicators in PE studies: wrist angular velocity coefficient of variation for basketball dribbling, vertical acceleration peak for standing long jump, trunk angular velocity smoothness for sit-ups, and horizontal acceleration integral for 50 m sprint. The results confirm that the model’s strong performance stems from real kinematic characteristics, instead of meaningless dataset-dependent correlations.

Taken together, the three interpretability analyses indicate that the high accuracy of the proposed model is accompanied by attention patterns and feature attributions consistent with established sport-specific biomechanical priors, supporting interpretation of the learned representation as grounded in real kinematics.

Discussion and implications

Discussion

Using a PE dataset with 245 students and 3,920 samples, we found hierarchical multimodal fusion outperformed single-modality methods by 8.6 pp (IMU-only) and 11.9 pp (scale-only). This overall outcome supports further analysis on fusion mechanisms, sport-specific performance and model robustness for real-world deployment.

Why fusion outperforms either modality alone

Experimental results confirm that the fusion model outperforms single-modality methods, with performance differences closely related to assessment criteria. Table 15 reveals uneven contributions from IMU signals and teacher rubrics across sports. IMU data dominates evaluation for explosive and maximal-effort movements, including standing long jump and 50 m sprint, which focus on mechanical performance. For coordination-based tasks like basketball dribbling, however, rubric scores carry greater weight, as IMU cannot effectively assess movement form and rhythm. This phenomenon verifies the modality complementarity principle in multimodal learning. The accuracy improvement observed in this study stems from such complementarity instead of redundant information.

Table 15.

Multimodal feature contribution analysis across different sports

Sport	IMU Weight	Scale Weight	Dominant IMU Feature	Feature Correlation Coefficient	Fusion Improvement Rate (%)
Basketball Dribbling	0.58	0.42	Wrist Angular Velocity CV	0.71	9.7
Standing Long Jump	0.68	0.32	Vertical Acceleration Peak	0.88	11.2
Sit-ups	0.61	0.39	Trunk Angular Velocity Smoothness	0.79	8.4
50 m Sprint	0.65	0.35	Horizontal Acceleration Integral	0.91	10.6
Average	0.63	0.37	—	0.82	10.0

Sport-specific patterns in the learned fusion weighting

Table 15 lists attention weights across different sports, and the related pedagogical analysis carries greater value than raw weight data. For basketball dribbling, the score stream reaches a maximum weight of 0.42. This task requires teachers to judge rhythm, ball-hand coordination and changing posture, all of which lie beyond the capture capability of IMU sensors. The model accordingly recognizes the limitations of single-sensor data. For standing long jump, the IMU weight is 0.68, matching the rubric focused on mechanical outcomes, where peak vertical acceleration serves as an indicator of take-off impulse. In short, variations in modality weighting across sports are determined by evaluation dimensions, rather than arbitrary model design.

The role of temporal alignment in fast movements

The spatiotemporal alignment algorithm synchronizes heterogeneous data effectively. It cuts the average time offset between IMU streams and scoring records from 312 ms to 18 ms, marking a 94.2% reduction. Benefiting from precise temporal matching, model predictions avoid distortion brought by temporal smear and interference from neighboring movement data. According to ablation results, removing cross-modal attention reduces accuracy by 7.8 pp, and removing DTW alignment leads to a 4.7 pp drop. The two modules work collaboratively without information redundancy.

What the model actually learns: bounding the ground-truth assumption

It is critical to clarify the essence of the model’s training labels. We adopted consensus scores from three calibrated PE teachers as supervision signals, rather than independent biomechanical or motor skill benchmarks. Two important implications are drawn from this setting.

First, the model can inherit inherent biases in manual scoring, such as gender stereotypes, campus-specific scoring habits and age-related differences in rubric execution. While ICC (0.87) and Fleiss’ κ (0.72) quantify inter-rater inconsistency, these indicators cannot identify collective biases among teachers. The cross-stratum analysis in Sect.  "Leave-one-school-out evaluation and cross-stratum generalization" supplements relevant evidence by evaluating model performance across subgroups.

Second, the system is designed to streamline expert assessment, rather than provide absolute ground truth. In practical large-scale deployment, regular validation with professional biomechanical equipment is required. Additionally, the rater calibration activities in Sect.  "Experimental setup and dataset" need to be maintained on an ongoing basis, instead of being conducted only once.

Robustness and its scope conditions

Model robustness test results are consistent with the expected behaviour of a multimodal system under data-quality variation. The fusion model retained 85.6% accuracy under 30% data missing while IMU-only dropped to 65.3%; this 20.3-pp gap illustrates the contribution of multi-source redundancy to system stability. Under structurally distinct missingness types (Table 11), fusion accuracy remains 81.4% under whole-sensor-node failure, the most adversarial of the four failure modes. Beyond data-quality robustness, the cross-axis robustness profile reported in Sect. "Leave-one-school-out evaluation and cross-stratum generalization" extends the picture along an orthogonal dimension. The 3.2-pp average cost under leave-one-school-out evaluation, the absence of a detectable gender-conditional gap (boys 91.4 ± 0.5%, girls 91.2 ± 0.5%, = 0.62), the flatness of accuracy across age subgroups (90.9–91.6%, ANOVA = 0.14), and the modest 3.0-pp gradient across baseline-skill tertiles together indicate that the framework’s generalization is not concentrated along any single source of heterogeneity. We avoid framing this multi-axis profile as a substitute for external validation; it should be read as quantitative evidence that the model is not over-fitted to any single demographic, institutional, or task-specific artifact, and as a calibrated baseline against which future external-validation results can be interpreted.

We nonetheless treat this robustness as bounded by the missingness mechanisms we could simulate: failure modes such as systematic single-node mis-calibration accumulating across the 12-week window, or correlated multi-node dropout from electromagnetic interference, remain outside the scope of the present evidence. The robustness reported here should be read as evidence that the architecture is not catastrophically brittle within a broad range of input degradations and across demographic and institutional strata, not as a guarantee of operational reliability under every plausible deployment hazard.

Implications

Reshaping the PE teacher’s role

Traditional assessment and modern sensing technology work best as complements instead of substitutes. Exclusive use of objective sensor data or subjective teacher scoring will lead to drawbacks. Sensor signals fail to reflect subjective aspects like movement aesthetics, whereas manual scoring is affected by insufficient observation accuracy and personal bias. This multimodal fusion method enables seamless integration of expert judgment and quantitative sensing results in a single pipeline.

The character of this role shift can be illustrated through the 45-minute lesson workflow described in Sect. "System architecture and data acquisition scheme". Under the traditional rubric workflow, a PE teacher typically spent 6–8 min of class time on score entry alone for a cohort of 35–40 students, with most scoring postponed to the lesson break. Under the proposed system, the teacher receives a tier prediction with a calibrated confidence value concurrently with rubric entry, and can re-allocate the saved 6–8 min to a targeted intervention for the small subset of students whose model–teacher disagreement is largest (typically 3–5 students per class of 40, given the 91.3% fusion accuracy). The teacher’s interpretive authority is preserved: model output is presented as a decision-support overlay that the teacher can accept, adjust, or override before the score is finalized.

Two specific competencies become more important under this reshaped role, both mapping onto components of the DigCompEdu framework identified as decisive for technology-enhanced PE. The first is data-interpretation literacy: reading the model’s confidence output, per-sport attention weights, and sensor contribution map to decide whether to accept, adjust, or override a tier assignment. The second is rubric-design competence: because the model’s learned representation is rubric-anchored, teachers who design or revise rubrics now influence what the model attends to.

Embedding the system in a formative-assessment loop

Section "Time complexity comparison" reports a single-sample inference latency of 43.2 ms, which brings notable pedagogical benefits. The system can generate ratings and confidence scores instantly during PE classes, supporting on-site formative assessment rather than only end-of-term evaluation. In contrast, traditional scoring usually releases results in the next class, and this delay largely reduces the instructional value of feedback for motor skill correction. We consider real-time in-lesson feedback the most promising way to boost learning outcomes, and further controlled teaching studies are required to confirm its actual effects.

According to motor learning research, feedback can be classified into three types by timing: concurrent feedback during movement, terminal within-session feedback, and cross-class delayed feedback. For adolescent students, terminal within-session feedback contributes most to movement consolidation, while delayed feedback only helps learners grasp rubric rules without improving kinaesthetic ability. Our system operates in the terminal within-session mode, with a latency of 43.2 ms that is well below the standard 1–5 min window. Traditional paper scoring relies on delayed feedback. This progress marks a fundamental change in teaching practice, not merely an efficiency upgrade.

Teacher professional development pathways

Section "Reshaping the PE teacher’s role" defines two core teacher competencies: data-interpretation literacy and rubric-design competence. Cultivating these skills requires specialized continuing professional development (CPD). Unlike generic digital literacy training, the proposed pathway strictly follows the DigCompEdu progression rather than providing one-size-fits-all technical instruction.

We establish a three-stage CPD system aligned with DigCompEdu standards. Basic Operation lasts 4 contact hours and covers sensor placement, tablet-based rubric use and routine maintenance for users at the Newcomer and Explorer levels. The second stage, Output Interpretation, takes 8 h. It trains teachers to understand confidence results, attention weights and feature attribution maps, and to recognize out-of-distribution data. This component strengthens data literacy for rational override decisions and corresponds to the Integrator and Expert levels. The third stage is Rubric Co-design, with 16 contact hours arranged throughout a term. It guides collaborative revision of rubrics and model training data, fitting the Leader and Pioneer levels. The training builds progressively: Stage 1 teachers may use the system in confidence-blinded mode, while.

Two design choices distinguish this pathway from generic technology-training modules. First, the pathway is built around the system’s actual interpretive outputs (confidence, attention, attribution) rather than abstract digital literacy, keeping training tightly coupled to classroom decision-making. Second, Stage 3 explicitly elevates the teacher from system user to system co-designer, recognizing that the rubric-anchored learned representation makes teacher-authored rubric revision the primary mechanism for any meaningful change to assessment criteria.

Integration into PE curriculum standards

The pedagogical legitimacy of any technology-mediated assessment system in Chinese middle-school PE depends on its alignment with prevailing curriculum standards. Two features of the contemporary PE curriculum framework are directly relevant: the now-explicit emphasis on process-based formative evaluation in addition to summative certification, requiring continuous evidence across a learning unit; and the core-literacy orientation that treats motor-skill development, physical fitness, and healthy behaviour as interlinked competencies not reducible to a single end-of-term score.

The proposed system supports both curriculum features rather than substituting for them. With respect to process-based formative evaluation, the in-lesson latency analysed in Sect. "Embedding the system in a formative-assessment loop" enables one tier prediction and one continuous quality score per student per session across a 12-week unit, yielding 8–16 evaluation points per student per sport rather than a single end-of-unit summary. The intra-student consistency analysis in Sect. "Handling of repeated measures" (within-student SD = 0.31 vs. between-student SD = 1.42) confirms this trajectory carries genuine signal. With respect to core-literacy orientation, the per-sport interpretability outputs in Sect. "Interpretability and explainability analysis" distinguish kinematic, performance, and form dimensions, supporting diagnostic feedback at the granularity at which the framework operates.

Two integration caveats accompany this alignment claim. First, the rubric criteria embedded in the model’s training labels are those of the participating schools’ interpretation of prevailing curriculum standards, not of the standards themselves; institutions with materially different rubric interpretations would need to re-calibrate the rubric component before deployment. Second, the system supplements rather than displaces the broader range of assessment evidence the curriculum framework recognizes, including peer assessment, self-assessment, and project-based assessment.

Equity, bias, and validity considerations

As the model is trained on teacher-generated scores, it could carry over systematic scoring discrepancies related to student gender, school background and physical maturity. The cross-stratum analysis in Sect. "Leave-one-school-out evaluation and cross-stratum generalization" evaluates potential biases quantitatively. Results show a marginal accuracy difference of 0.2 pp between genders (p = 0.62) and a variation below 0.7 pp across age groups (p = 0.14). A 3.0 pp performance gap exists between the top and bottom tertiles of baseline skills, which is caused by the rubric-floor effect instead of inherent model bias. The present work confirms concurrent validity relative to expert ratings. Follow-up studies can build on this research to address remaining limitations.

Conclusion

This study addresses insufficient objectivity and limited efficiency in physical education assessment by proposing a multimodal algorithmic model that fuses IMUs with traditional assessment scales. Using a participant-wise split of 3,920 samples from 245 middle-school students across four sports, three calibrated PE teachers (ICC(2,k) = 0.87, Fleiss’ κ = 0.72) provided the supervisory signal. The fusion model achieved 91.3 ± 0.4% accuracy, 8.6 pp above IMU-only and 11.9 pp above scale-only baselines (p < 1 × 10⁻⁸); spatiotemporal alignment reduced average time offset from 312 ms to 18 ms; the model retained 85.6 ± 0.5% accuracy under 30% data missing; and 43.2 ms inference supports immediate feedback.

Four main outcomes are reported in this study. First, leave-one-school-out evaluation returned a mean accuracy of 88.1 ± 1.3%, representing a 3.2 pp performance decline across different schools and verifying basic cross-institution generalizability. Cross-stratum results show steady performance across gender (91.4% vs. 91.2%, p = 0.62) and age groups (90.9–91.6%, p = 0.14). The 3.0 pp difference among students with varied basic skills stems from the rubric-floor effect. These findings support the model’s robustness within the research population, while external validation with new cohorts remains necessary. Second, the model’s attention and feature attribution results conform to well-recognized biomechanical rules. Third, this system replicates expert PE assessment results rather than measuring objective biomechanical indicators, so it should assist rather than replace teachers’ professional judgment. Fourth, Sect. "Implications" covers teacher role transformation, formative assessment application, three-stage CPD schemes and curriculum alignment. This framework fully integrates technical innovations with practical pedagogical design.

This study has several limitations. The dataset covers four individual sports across three schools, so its applicability to other age groups, campuses and team contact sports needs to be confirmed via external tests. While LOSO and cross-stratum analyses support the model’s robustness, validation on fully independent cohorts remains absent. Only three teachers participated in scoring. Even with standard calibration and review by a fourth expert, inter-rater reliability is still constrained. Our missing data tests included random loss and four structured missing patterns at 30%, but did not cover systematic issues like gradual calibration drift of individual sensor nodes.

For future work, we will recruit more participants and expand sport categories, and incorporate computer vision to enrich multimodal inputs. External validation against biomechanical reference tools will also be performed. Moreover, we will launch two controlled studies. One explores the link between real-time formative feedback and motor learning outcomes. The other assesses the proposed three-stage CPD pathway against the DigCompEdu standards.

In its present form, the proposed framework offers a reference methodological pathway for coupling sensor data with expert-derived rubric scoring in school PE assessment, not a finalized technical solution. The participant-wise evaluation protocol, inter-rater reliability reporting, leave-one-school-out and cross-stratum analyses, pedagogical positioning, and interpretability analyses serve as a methodological template for future PE-IMU studies.

Authors' contributions

J.L.C.: Conceptualization, Data curation, Methodology, Investigation, Writing - original draft preparation. Z.L.: Writing - review & editing. Software, Data curation, Formal analysis.

Funding

Not applicable.

Data availability

All data and code supporting the findings of this study are publicly available at the following GitHub repository, which serves as the single integrated archive for both the de-identified dataset and the complete software implementation: https://github.com/a18907052542-lang/Multimodal-Algorithmic-Fusion-Model-for-Physical-Education-Assessment.git.

Data repository. The de-identified IMU feature matrices (n = 3,920 samples across 245 students and four sports), digitized rubric scores, four-tier grade labels including the fourth-expert adjudicated labels for the 165 low-agreement samples, participant-wise split indices, Leave-One-School-Out (LOSO) fold assignment files, ten random-seed configurations, training-only normalization statistics, and example preprocessing outputs are released through the above repository under the CC BY 4.0 license. The deposit permits reproduction of every numerical result reported in Section "Experimental Results and Analysis". Raw video clips are not included because participant consent does not extend to identifiable video; these remain available under a Data Use Agreement requiring equivalent ethics committee approval at the requesting institution.

Code repository. The complete PyTorch 2.1 implementation is released in the same GitHub repository under the MIT License. The repository contains: (i) the adaptive DTW alignment module with the motion-type-dependent aggregation parameter λ of Eq. (9); (ii) the hierarchical attention block with gated cross-modal fusion of Eqs. (10)–(12); (iii) the two-headed classification and regression network with the loss configuration of Eq. (13); (iv) training, validation, and held-out evaluation scripts with hyperparameter logging; (v) the LOSO and cross-stratum evaluation harnesses used to produce Tables 6 and 7; (vi) the structured-missingness simulation pipeline used to produce Table 11; (vii) the interpretability-analysis utilities used to produce Fig. 9; (viii) Jupyter notebooks that regenerate every numerical entry in Tables 6–15; and (ix) a Dockerfile supporting one-click reproduction on a consumer-grade GPU.

Reproducibility envelope. Table 2 (hyperparameter configuration), the ten random seeds, the participant-wise split indices, the LOSO fold assignments, and the cross-stratum stratification rules are jointly sufficient to reproduce every main result in Section "Experimental Results and Analysis" to within the run-to-run variability bounds quoted in each table. Deviations exceeding 0.5 pp from reported accuracies should be reported as reproducibility issues against the repository.

Declaration

Ethics approval and consent to participate

Ethical approval for this study was obtained from the Central China Normal University Research Ethics Committee, in compliance with the Declaration of Helsinki. Informed consent was obtained from all participating students and their legal guardians, and strict measures were implemented to protect the privacy and data security of the subjects.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.