A biomechanical-constrained temporal learning framework for lightweight skeleton-based exercise recognition

Abstract

The recognition of exercises using skeletal pose sequences is a significant fitness technology, rehabilitation monitoring, and sports analytics. Nevertheless, the current practices tend to ignore the basic biomechanical processes of human motion. This paper presents a Biomechanical-Aware Temporal Learning (BATL), framework that incorporates human kinematic constraints and deep temporal model. The method takes into account three biomechanical limitations objects consisting of joint angle consistency, velocity smoothness, and bone length stability that are not explicitly coded but rather acquired during the learning course of model parameters. Moreover, BATL uses the phase discovery module which is unsupervised and automatically identifies important temporal divisions (preparation, execution and recovery) without manual annotation. BATL was tested on 146 video samples (157 different samples in total, including 146 squats, 126 push-ups, 146 bicep curls, 146 shoulder-presses) with an estimated test accuracy of 93.33% ± 0.94% percent (mean ± standard deviation on 5-fold cross-validation with random seeds 42, 123, 456, 789, 101). It has 3.57M parameters, requires 13.6 MB memory, and inference can be performed in BatL-only mode only in 5.2 ms per 30-frames sequence (end-to-end pipeline depending on OpenPose pose estimate in 1.574 seconds). It outperformed the baseline techniques by 6.22%-14.44%, and ablation experiment showed that incorporation of biomechanical constraints actually improved accurately by 7.78%. These results imply that even inductive bias involving the use of domain knowledge can be more effective than the benefits that can only be achieved with a more complicated model. Overall, biomechanical knowledge and deep learning offer certain prospects to the activity recognition system, in particular, movement quality and time.

Introduction

Video-based exercise recognition has become an essential part of current fitness trackers, rehabilitation systems¹, and sports performance analysis software because it is automatic. Consistent automated classification will allow real-time feedback, counting repetitions, monitoring progress and individual coaching without any human supervision. Exercise recognition is a major problem although the recent advances in video-based^2–4 action recognition have resulted in significant improvements in performance. Activities tend to have a low-level inter-class variance, the variation of perspectives, and repetition. In addition, the discipline does not have large, annotated data sets, and real-world uses demand models which can run efficiently in real-time on consumer grade hardware.

The skeleton-based methods represent a more attractive alternative to RGB-based models as they model human motion with a sequence of 2D or 3D joint coordinates modeled with pose estimation techniques. Neural networks based on graphs have demonstrated good results in spatial-temporal relations between joints^5,6. These models are computationally efficient and less sensitive to the background, lighting or appearance changes compared to pixel based architectures. Nonetheless, the majority of the current skeleton-based approaches consider joint trajectories as generic time-series signals and resort to data-driven learning completely. This does not consider the biomechanical rules of human movement.

Physical and anatomical laws are intrinsically limiting to the motion of a human being. The natural ranges of motion are joint angles, healthy motion has smooth velocity profiles as per the principle of minimum-jerk⁷, and skeletal bone lengths do not change with age. A violation of these constraints is usually an indicator of pose estimation noise or abnormal execution pattern. However, most of the recent deep learning systems do not encode such biomechanical information into their learning goals, which may impede their interpretation, stability, and extrapolation, which is particularly relevant in low-data conditions (which are prevalent in exercise recognition).

This paper proposes a model, the Biomechanical-Aware Temporal Learning (BATL), that applies biomechanical principles explicitly to the learning objective of the model in order to overcome these drawbacks. BATL enhances the learning process by ensuring that the learning uses differentiable constraints on the joint angle validity, movement smoothness, and bone-length scale simultaneously driving the learning process to physically plausible representations with the aid of the temporal dynamic^8–10. Although the biomechanical constraints of action recognition have been studied before, BATL is the first to use learnable constraint weights, unsupervised identification of phases, and efficient architecture design that is specifically optimized to identify the small-sample exercise recognition system. The major contributions are:

1. A lightweight framework incorporating learnable kinematic-level priors (joint angle consistency, velocity smoothness, and bone-length stability) combined with unsupervised temporal phase discovery, explicitly tailored to small-sample exercise recognition (demonstrated on 131 training samples). These constraints operate at the level of pose kinematics and should be understood as kinematic plausibility regularisers rather than as full biomechanical or musculoskeletal simulation; muscle forces, joint torques, and subject-specific anatomical parameters are not modelled.

2. The objective empirical evidence that domain-specific biomechanical priors are capable of learning effectively with limited data, outperforming data-driven baselines by 6.22% −14.44% whilst remaining computationally efficient (3.57M parameters, 5.2 ms inference).

3. Extensive testing with quantitative phase discovery measurements (NMI=0.76), deployment performance measures (end-to-end latency characterization) and case cases of interpretability of biomechanical constraint violations as diagnostic features.

4. Statistical robustness testing on 5 instances of cross-validation on standard deviations, sensitivity analysis on window size and experimental results on state-of-the-art skeleton-action models (CTR-GCN, MS-G3D) under controlled hyperparameter optimization.

This study is explicitly presented as a proof-of-concept demonstrating that kinematic inductive bias can yield competitive accuracy in a small-sample, four-class exercise recognition setting. Scalability to larger exercise taxonomies, greater subject diversity, and open-set conditions represents an important open question that future work should address through evaluation on benchmarks such as FineGym¹¹ or Fitness-AQA¹².

Related work

Video exercise recognition has progressed significantly in the last ten years¹³, with algorithms on video exercise recognition replacing traditional pipelines of computer vision techniques in favor of deep learning algorithms. Veeriah et al.¹⁴ proposed the use of differential Recurrent Neural Networks to do action recognition on the UT-Interaction dataset with 88.9 percent accuracy in the modelling of temporal variability between subsequent frames. This method lacked the ability to capture the longer time scale in most exercising activities.

Two-stream convolutional networks, which processed spatial streams (RGB) and temporal streams (optical flow) independently on UCF101 were proposed by Simonyan and Zisserman¹⁵, who used late fusion to get 88.0% accuracy. Feichtenhofer et al.¹⁶ improved this using convolutional fusion, to 92.5% on UCF101 and 65.4% on HMDB51, indicating that learned fusion works better than hand-crafted combination strategies. Such approaches make use of pre-computed optical flow which introduces computational overhead, which restricts real-time applications.

Alternatives to three-dimensional convolutional networks were developed that learn spatiotemporal features directly on raw video. Other models developed by Tran et al.¹⁷, such as C3D, have 3D kernels to 16-frame clips, and with 78M parameters, C3D had 85.2% on UCF101. Carreira and Zisserman² suggested Inflated 3D ConvNets (I3D) by inflating ImageNet 2D filters into 3D, with 98.0% on UCF101 and 80.7% on Kinetics-400 using more than 50M parameters per video. Feichtenhofer et al.³ designed SlowFast networks that have two pathways with a different temporal resolution (4 fps slow pathway, 32 fps fast pathway), and reached a top-1 precision of 79.8% on Kinetics-600. Although they are powerful on general action datasets, they require terribly large computational resources and large amounts of training data, which is not feasible with specialized exercise recognitions tasks on consumer devices.

The memory requirements of the RGB techniques were the cause of the development of the skeleton-based action recognition that describes human movement in terms of joint coordinates sequences obtained through pose estimation. In Du et al.¹⁸, hierarchical RNNs were suggested to work with the skeletal data of NTU RGB+D dataset and provide the 62.9% accuracy indices due to the part-based modeling of the temporal dynamics. Introduced by Liu et al.¹⁹ are spatio-temporal LSTM networks that explicitly project spatial settings and temporal dynamics simultaneously, to 69.2% on NTU RGB+D with 6.5M parameters. All the same, these methods are viewing joints as being isolated without taking advantage of the anatomic interrelationships. Wang et al.²⁰ extended recurrent architectures with deep residual connections in a longer-input BiLSTM network, achieving improved temporal modelling for action recognition while maintaining computational efficiency.

Yan et al.⁵ were the first to model based on graphs, using the Spatial Temporal Graph Convolutional Networks (ST-GCN) that models the skeleton as a graph, with the joints acting as the nodes and bones as the edges. Using data on Kinetics-Skeleton and NTU RGB+D ST-GCN trained to 81.5% and 86.3% accuracy with 3.1M parameters, showing that the graph structure of space is more accurate compared to the traditional recurrent and convolutional networks. Shi et al.⁶ furthered it with adaptive graph topology learning in 2s-AGCN, which enables the model to learn the task-specific joint relationships that are not inherently connected in natural connectivity, to achieve an 88.5% NTU RGB+D accuracy. Li et al.²¹ suggested AS-GCN using actional and structural streams of graphs, attaining 86.8% on NTU RGB+D. Cheng et al.²² proposed Shift-GCN that uses shift operations and a model to perform effective temporal modeling, reaching 90.7% on NTU RGB+D 120 only Coating 2.7M parameters until it uses shift operations. Reposing skeleton-based skeleton-based approaches with PoseConv3D, Duan et al.²³ performed 3D convolutions on pose heatmaps, and the best result of 97.1 on NTU RGB+D 60. Although effective (3-8M parameters), these graph based algorithms learn the spatial relation only using the data, not using domain information about biomechanical constraints which govern human movement.

The mechanisms of attention have also been investigated to capture long range temporal dependencies. View Adaptive Neural Networks by Zhang et. al²⁴. were proposed to skeleton based action recognition, in the cross view benchmarks such as NTU RGB+D, they learned view-invariant features and obtained 97.2%. Plizzari et al.²⁵ proposed skeleton sequence spatial-temporal transformer networks with 86.4% on Kinetics-Skeleton with self-attention to focus on global context. Nonetheless, competitive performance of transformers will need large datasets (Kinetics with 300 K videos) and considerable computational power (8.4M parameters, 12.3 ms inference time obtained in our experiments).

The research community has been giving more and more attention to exercise-specific recognition. The gym exercise dataset gathered by Khurana et al.²⁶ had 8 classes (bench press, overhead press, pulldown, etc.) and used CNNs to work with RGB frames with an accuracy of 83.7%. Nevertheless, their method demanded closed gym settings with predetermined camera location. Liao et al.²⁷ developed a deep learning framework for assessing physical rehabilitation exercises on a custom dataset of 5 therapy movements, using LSTM networks on skeletal joint angles extracted via OpenPose to achieve 87.3% exercise classification accuracy and 82.1% quality assessment accuracy. Their work demonstrated the value of pose-based features for exercise analysis but relied on handcrafted angle features rather than end-to-end learning. Xu et al.²⁸ proposed a metaheuristic optimization-based gait feature selection method that identified 10 optimal biomechanical features achieving 99.81% recognition accuracy, demonstrating the value of principled feature selection for clinical and sports gait pattern analysis.

Neburka and Tretiakov²⁹ focused on form quality in resistance training exercises using LSTM networks on a dataset of 4 exercises (squat, deadlift, bench press, overhead press) with 156 videos, achieving 91.2% classification accuracy with emphasis on detecting improper form. Islam and Iqbal³⁰ proposed interrelationship features based on 3D joint angles and inter-frame distances for exercise recognition on a 12-class dataset of 216 videos, achieving 93.5% accuracy using Support Vector Machines. However, these handcrafted features require domain expertise and may not generalize across exercise variations. Bukht et al.³¹ proposed an Extended Kalman Filter approach for robust human interaction recognition, demonstrating that incorporating noise-filtering priors into the recognition pipeline improves robustness, a motivation aligned with the velocity smoothness and bone-length stability constraints in BATL.

Biomechanical constraints in action recognition

In more recent practice, biomechanical principles have found application in models learned. Xu et al.⁸ put forward (physical) action recognition constraints on the body structure, which punish physically implausible poses in training on the Penn Action dataset (15 classes, 2326 videos), with 98.1% accuracy, with bone length consistency. Song et al.⁹ suggested deep-learning-based richly activated graph convolutional networks with motion encoding, which implicitly predicts the velocity patterns with an accuracy of 93.0% on NTU RGB+D 120. Peng et al.¹⁰ trained learning skeletal graph neural networks with regularization of motion trajectories on BABEL dataset (120 actions, 43 hours of motion), and still reached 86.7% accuracy. These methods show the utility of physics based constraints, but normally make use of fixed weighting schemes instead of learning the significance of constraints.

These past works differ considerably in terms of BATL: First, rather than learning the hyperparameters to the constraints weighting corresponding to fixed values, BATL learns the importance of the constraints that are optimal at training through differentiable parameters (, , ): differentiable parameters can be automatically adapted to the nature of the dataset. Second BATL combines biomechanical constraints and unsupervised discovery of temporal phases, both in the instantaneous kinematic plausibility and the temporal organization. Third, when existing literature concentrates on large datasets (Penn Action: 2,326 videos, NTU RGB+D: 40,000+ samples, BABEL: 43 hours), BATL is explicitly implemented and tested in small sample settings (131 training samples), giving support to the idea that biomechanical priors can be used to learn effectively small sample amounts of data. Fourth, all BATL elements are trained end-to-end based on joint optimization as opposed to imposing constraints as a form of post-hoc regularization.

A clarification is warranted regarding the scope and universality of the three constraints employed in BATL. The joint angle consistency constraint does not enforce exercise-specific anatomical range-of-motion limits; rather, it penalizes temporal inconsistency within a given sequence using an upper-bound ReLU penalty on hyperextension, making it invariant to the absolute range expected for any particular exercise class⁷. The result of this formulation is that the constraint can be used directly on exercises whose joint angle envelopes vary greatly without any adjustment. The bone-length stability constraint is also frame-to-frame adaptive (in a single subject) and discourages frame-to-frame estimation noise instead of enforcing anthropometric norms between a subject and anatome, rendering it inter-subject insensitive to variations in body proportions across subjects. The basis of the velocity smoothness constraint is the minimum-jerk principle of motor control, which postulates smooth velocity profiles of voluntary movement independent of movement speed and frequency, since the neural planning that the brain should generate to create efficient motion paths minimizes jerk. Although high and low-frequency plyometric movements and low-frequency rehabilitation movements differ in tempo, both should have smooth velocity envelopes at a given tempo, and irregularities thus reflect noise in pose estimation or unusual biomechanics independent of movement frequency. Taken together, the three constraints are formulated as exercise-agnostic regularizers on kinematic plausibility rather than exercise-specific boundary conditions, which supports their applicability across a broader range of movement types beyond the four classes evaluated here. The learnable weights , , further allow the model to automatically down-weight any constraint that is uninformative for a given dataset, as evidenced by decreasing from 0.05 to 0.03 when OpenPose produces stable frontal-view estimates.

Computational Temporal phase modeling has been investigated in complex action understanding. The authors Lea et al.³² suggested Temporal Convolutional Networks to perform action segmentation on cooking by modeling hierarchical time structure (Breakfast dataset 1712 videos 48 action classes), which got 66.3% frame-wise accuracy. Liu et al.³³ presented MS-G3D as a multi-scale aggregation skeleton-based recognition model that achieves 91.5% on NTU RGB+D 120 by depending on motions recorded at various time scale levels. Nevertheless, the techniques need manual phase labelling or priori scales of times, restricting their versatility.

Data sets have also developed to facilitate the exercise recognition study². Kinetics data available at carreira2017quo contains exercise-related classes in 400–700 classes but does not distinguish between finer categories of exercises. Fitness-AQA data presented by Parmar and Morris¹² include 500 videos of gym exercises, with quality scores, but is focused on evaluation and not on categorizing them. The SPHERE Challenge data set presents activities of daily living with exercises throughout the home settings (20 subjects, 10 activity classes) with a focus on the recognition that ensures privacy³⁴. This is because FineGym dataset suggested by Shao et al.¹¹. provides the most comprehensive exercise-based dataset, having 99 categories and 38,898 temporal segments of 708 videos to detect the fine-grained gym activity. Nevertheless, the majority of previous studies are judged based on general action datasets in which the exercise classes are a very minor proportion of the taxonomies.

The more recent transformer-based techniques have had promise in modeling sequences. Channel- wise Topology Refinement Graph Convolution is a skeleton action recognition proposal by Zhao et al.³⁵, which obtained 93.1% on NTU RGB+D 120 using 5.2M parameters. But transformers are generally pre-trained on large scale data and cannot work on small data conditions as is the case in more specialized exercise datasets.

Multi-modal fusion has been investigated in order to utilize complementary information. The RGB-depth-skeletal framework provided by Munro and Damen³⁶ was trained to use action recognition on NTU RGB+D and realized 91.7 percent with late fusion. The paper by Kay et al.³⁷ also established that downstream tasks can be well generalized by models that have been pretrained on Kinetics with 240 K videos, and thus can be outsourced to large-scale pretraining. The multi-modal options, however, demand more computational resources and are not practical in the recognition of exercise in real time on consumer gadgets.

In spite of those impressive results, there are still three significant limitations of the current exercise recognition methods. To begin with, they explicitly do not require biomechanical constraints including joint angle restriction, movement smoothness, and skeletal rigidity to determine human motion, and they instead only use information-driven representations. Second, the state-of-the-art approaches have great demand of large training data or extensive pretraining, and the exercise-specific data is generally small and hard to label. Third, the cost of computing the current models can be excessive (commonly more than 50 million parameters), which is why it cannot be deployed to consumer devices that have to work in real time. We overcome these shortcomings by proposing BATL, which is a small hierarchy in temporal learning that combines the variousiable biomechanical constraints with learnable importance weights. BATL with only 3.57 million parameters can learn on small data of 131 training samples and BATL-only inferencing time of 5.2 milliseconds per sequence on a typical consumer laptop. This type of design is one that allows the recognition of exercise to be observed accurately, efficiently, and to be biomechanically consistent enough to be applicable in practical applications.

Methodology

Figure 1 shows the overall end-to-end BATL architecture, which consists of four hall lines: pose extraction in input video, BiLSTM temperature encoding on biomechanical-aware, unsupervised stage discovery and final exercise identification with biomechanical constraint regularization.

Fig. 1.

BATL architecture overview. The pose extraction on input exercise videos is done with the OpenPose to produce skeletal sequences of 17 keypoints. The BATL core has three synergies, including (1) Temporal Encoder that models sequential-dependent and long-range temporal changes with just a bidirectional LSTM, (2) Phase Detector that automatically discovers temporal phases (preparation, execution, recovery) without supervision, and (3) Biomechanical Constraints that regularize the learning process with the help of kinematic priors representing the limits of joints, the smoothness of motions and skeletal rigidity. The temporal and phase characteristics are grouped and passed to the classifier to predict the category of the exercise.

Problem formulation

Given a video sequence containing a single person performing an exercise, the task is to classify the exercise into one of C predefined categories. The input is represented as a temporal sequence of 2D skeletal poses , where T denotes the sequence length and each pose contains joint coordinates at timestep t. The objective is to learn a mapping function that assigns the correct exercise label to the input sequence while satisfying biomechanical constraints.

Pose extraction and representation

Keypoint detection

OpenPose36 is used to extract skeletal poses out of input video images and offers a high level of 2D joint localization in different lighting conditions, camera angles, and incomplete occlusions. A topology of 17 key points that are adopted and represented in Fig. 2 is that COCO³⁸ topology: the topology consists of joints of anatomical importance that include nose (0), neck (1), right shoulder (2), right elbow (3), right wrist (4), left shoulder (5), left elbow (6), left wrist (7), mid-hip (8), right hip (9), right knee (10), right ankle (11), left hip (12), left knee (13), left ankle (14), right eye (15), Individuals: When it is desired that the final representation should be computationally efficient and relevant to the exercise field of anatomy, the ultimate representation is J17 core body joints (without facial landmarks where relevant).

Fig. 2.

Biomuscularly constrained pose. (a) A sound pose configuration, which leaves the limits on the joint angles; the arm shows a physically realistic elbow angle (marked in green). (b) An invalid position where the extreme joint angle is causing a biomechanical violation, the position of the wrist causes an impossible elbow hyperextension (marked by a red X). Contributions to the object are in bold. The entire 17-keypoint skeleton in COCO topology, representing the anatomical arrangement of the entire process in BATL.

The rationale for selecting this topology is threefold: (1) comprehensive coverage of major body segments relevant to exercise biomechanics (trunk, upper limbs, lower limbs), (2) established reliability in pose estimation literature with robust detection even under partial occlusion, and (3) computational efficiency compared to denser representations (e.g. 133 keypoints in MediaPipe) while retaining essential anatomical information. Each joint j at frame t is represented as , where and denote normalized pixel coordinates in the range [0, 1].

Temporal sampling and normalization

Since the temporal sequences of videos of different length sizes have the constant input dimensionality, each temporal sequence is contracted to T= 30 frames with an adaptive sampling approach. Consecutive sequences shorter than 30 frames are synthesized by linear interpolation, but long consecutive sequences are downsampled by uniform subsampling to pick uniformly spaced frames. This time range, of about three seconds with 10 FPS, came out of empirical data analysis of our data: we found that 87% of videos finish an exercise at least 1 or 2 consecutive exercise repetitions in 30 frames, which allows us to capture the key temporal structure with FSANA. The sensitivities of the Temporal Window Analysis section provide detailed sensitivity analysis demonstrating that T=30 is the optimal trade-off point between the accuracy and the efficiency with shorter windows (T<25) being deprived of critical temporal context and longer windows (T>40) being able to gain the least accuracy (<0.5%) at high cost (>30% slower inference).

Pose coordinates are normalized to achieve translation and scale invariance:

1

where and represent the mean and standard deviation computed across all frames in the sequence, and prevents division by zero. This per-sequence norming gets rid of person-specific anthropometries. Dependence of the height, limb lengths and camera position and maintain the relative motion patterns necessary during exercise. recognition.

BATL architecture

Input projection layer

The flattened pose sequence (where ) is first projected to a higher-dimensional feature space to enable nonlinear combinations of joint coordinates:

2

where and are learnable parameters, and the feature dimension is set to . This projection transforms the raw coordinate space into a representation space where meaningful exercise-specific patterns can be learned.

Bidirectional LSTM encoder

Temporal dependencies across the pose sequence are captured using a two-layer bidirectional Long Short-Term Memory (BiLSTM) network^20,39:

3

4

5

In which is the index of the layer, at timestep l, layer L is an index, and is the cell state and is concatenation in the feature dimension. The back and forward architecture enables both forward (LSTM) and backward (LSTM) context access, which is essential to recognize exercise phases that rely on motion patterns on both ends of time (e.g. to identify the peak of a squat, both downward and upward movement are needed).

The latent dimension will have per direction which gives concatenated features of dimension in forward and backward states after concatenation. There is dropout to drop LSTM layers with probability which is used during training to avoid overfitting on the training data.

Unsupervised phase discovery module

A network of phase detection is already used to detect phases in exercise sequences automatically, without having to manual label the exercise sequences. Naturally, any exercise movement can be considered as comprising of three parts namely preparation (initial position), execution (primary movement), and recovery (return to start). The phase detector learns to identify these phases through end-to-end training:

6

7

where , are weight matrices, , are bias vectors, and is the number of phases. The softmax function ensures , where represents the probability of phase k at timestep t.

To enforce temporal smoothness in phase transitions (preventing biomechanically implausible abrupt switches), a moving average filter is applied:

8

with boundary handling via replication of edge values. This smoothing is based upon the biomechanical fact that transitions between phases are not sudden.

Training procedure clarification: Phase detector is jointly trained end-to-end with the classification network with the backpropagation of the total loss . No phase specific loss function. The phase probabilities are calculated using Eqs. (6) and (7), averaged by smoothing using Eq. 8 and then weightaggregated using temporal features using Eq. (9), and finally classified. The classification loss is considered to be a backward gradient flowing through the classifier input phase aggregation phase detector, to make the network learn phase assignments that maximize classification performance. This is really unsupervised in the sense that no manual phase labels are necessary, but the phase detector is not optimized separately, it is a part of the unified computational graph that is optimized to minimize .

Phase-specific features are then computed through weighted aggregation:

9

where contains smoothed phase probabilities and contains temporal features from the second LSTM layer. This matrix multiplication produces K phase-specific feature vectors, each representing the aggregate temporal patterns within that phase.

Biomechanical constraints

A key contribution of BATL is the integration of three biomechanical constraints into the loss function, formulated as differentiable terms that regularize the learned representations. Figure 3 visualizes these constraints.

Fig. 3.

(a) Joint angle constraints act as a punishment to angles that are beyond the physiological limit of hyperextension (). The star symbolizing the entire range of the valid range (0 to 172) represents zero loss when ReLU(). Angles above 172 (red, right) only add positive penalty. The valid motion trajectory (solid green line) never exceeds the 0–172 range, whereas the invalid one (dashed red line) crosses the threshold and gets into the hyperextension zone, where they make the loss comparable to the excess. The constraint is upper-bound only, as it is a one-sided constraint: the angles around 0 (fully flexed joints) are not constrained by the constraint at all. (b) Velocity smoothness constraint encourages smooth joint trajectories (green line with low jerk) and penalizes jerky, discontinuous motion (red line with high acceleration fl uctuations), consistent with the minimum-jerk principle from motor control theory. (c) Bone length consistency constraint enforces constant skeletal dimensions across all frames. Stable lengths (green line near baseline) indicate valid poses, whereas high variability (red line) signals pose estimation errors or physically implausible skeletal deformations.

Joint angle constraint

Human joints have anatomical range-of-motion limits determined by bone structure, ligament length, and muscle flexibility. For a joint triplet where is the central joint (e.g. elbow connecting upper arm and forearm), the joint angle at time t is computed as

10

11

12

where the clamp operation eliminates floating-point error resulting in values that may narrowly lie outside the range . Four vital triplets of joints are those tracked; left elbow (joints 5–6-7), right elbow (2–3-4), left knee (12–13-14) and the right knee (9–10-11) as these are the most pertinent to the four exercise classes.

The angle loss in joint is a penalty to extreme joint angles that are beyond physiological values:

13

in which N is batch size, radians (immediately less than 172) is the physiological maximal hyperextension, according to the physiological biomechanical literature^40–42 and is a loss term used to ensure only the violations are counted. This value is obtained based on Winter (2009) that reporting on normal healthy joint ranges-of-motion of the human beings. Although the levels of individual flexibility differ, 172 is a conservative upper limit and beyond which hyperextension is biomechanically impossible with the observed joints (elbows, knees) when performing the exercises assessed. This can only allow the model to learn physically impossible pose configurations.

Velocity smoothness constraint

The natural human motion is smooth in terms of velocity profiles and is based on the minimum jerk principle⁷ that asserts that the nervous system acts to keep the third derivative of position (jerk) minimal. Jerky, non-controlled movement is also a usual sign of position estimation error or pathological motion. The velocity at each frame is

14

Acceleration (serving as a proxy for jerk) is

15

The velocity smoothness loss is

16

where denotes the norm, which is less sensitive to outliers than the norm and thus more robust to occasional pose estimation errors. This constraint encourages smooth temporal evolution of joint positions.

Bone length consistency constraint

The skeletal body is hard the bones in the body should not become shorter in one frame as compared to another in a sequence. The differences in the calculated bone length suggest an error in estimating the poses or unfeasible skeletal deformity. Eleven anatomical connections are tracked: (0–1), (1–2), (2–3), (3–4), (1–5), (5–6), (6–7), (9–10), (10–11), (12–13), (13–14), representing major skeletal segments.

For each bone b connecting joints and , the Euclidean length at time t is:

17

The temporal variance in bone length is penalized:

18

where is the number of tracked bones and computes the temporal variance with . High variance indicates inconsistent pose estimates that violate the physical constraint of skeletal rigidity.

Composite biomechanical loss

The three constraints are combined with learnable weight parameters:

19

where are initialize as (0.1, 0.1, 0.05) respectively and defined as PyTorch nn.Parameters, trained as any other model weights. The given formulation enables the model to automatically learn about the best tradeoff amongst constraints depending on the characteristics of datasets during end-to-end training, instead of hand-setting hyperparameters. Learned Constraint Weight Dynamics section examines the dynamics of learned weights, revealing that increases 0.14 (joint angle) and decreases 0.03 (bone length violation) are the optimal weights with the highest and lowest probability of the constraints, respectively, which again demonstrates that the weights are adjusting to the task-specific patterns of biomechanical constraint.

Feature aggregation and classification

Temporal and phase-specific features are aggregated through mean pooling to obtain fixed-dimensional representations invariant to sequence length:

20

21

These complementary feature representations are concatenated:

22

The concatenated features are fed to a two-layer multilayer perceptron (MLP) classifier with ReLU activation and dropout regularization:

23

24

where , (with exercise classes), and dropout probability provides strong regularization during training.

Loss function and training

The total training objective combines cross-entropy classification loss with the biomechanical regularization term:

25

where the cross-entropy loss for a batch of size N is:

26

with being the one-hot encoded ground truth label for sample n and class c.

Training employs the AdamW optimizer⁴³ with initial learning rate , weight decay for regularization, and momentum parameters . A cosine annealing learning rate scheduler with epochs gradually reduces the learning rate following . Gradient clipping with threshold prevents exploding gradients. Training uses batch size due to memory constraints.

Data augmentation strategy

To improve generalization on the limited training data (131 samples) and prevent overfitting, aggressive data augmentation is applied during training only:

• Body scale variation: Height and width are independently scaled by factors sampled uniformly from , simulating different body types and anthropometric variations.

• Execution speed variation: Temporal speed is a factor included in which is explained by individual differences in exercising pace.

• Range-of-motion variation: The amplitude of movement is affected by which reflect different levels of fitness and flexibility.

• Gaussian noise: Zero-mean Gaussian noise with standard deviation is added to all joint coordinates, simulating realistic pose estimation errors from OpenPose.

• Random occlusion: To generate partial occlusions (e.g. arms occluding torso), with probability between 1–3 randomly selected joints are perturbed with to generate the appearance of partial occlusions.

• Class confusion augmentation: Consecutively similar exercise pairs (bicep curl shoulder press) combine 15% of the training samples with the addition of Gaussian noise where is the standard deviation of the training set between the confusing class and the correct one, obtained by sampling the training set statistics. This is a simulation of confusable motion patterns, and is better at learning inter-class boundaries.

Implementation details

PyTorch 2.0.1 with CUDA 11.8 is used to implement this model. The training and assessment occur on consumer grade hardware, that is, the use of an HP EliteBook 1040 G10 laptop with 16 GB of RAM and using Windows 11 OS and without using specialized GPU infrastructure. This configuration shows the feasibility of BATL deployment on easily available hardware by single users and practitioners. Xavier uniform initialization is used as a initializer of model parameters and the set of random seeds consisting of 42,123,456,789,101 is fixed over five-fold cross-validation splits to make them reproducible.

Training algorithm

Algorithm 1 summarizes the complete BATL training procedure.

Algorithm 1.

BATL training procedure.

Experimental setup

Dataset construction and composition

Two publicly available exercise video datasets were merged to create the training and evaluation corpus for this study. Table 1 provides detailed composition statistics.

Table 1.

Dataset composition before and after merging.

Source	Squat	Push-up	Bicep curl	Shoulder press
Dataset A: final_kaggle_with_additional_video
Videos	28	31	29	24
Dataset B: similar_dataset
Videos	9	8	8	9
Hammer crl	11 videos (EXCLUDED)
Merged total	37	39	37	33
Usable total	146 videos (157 before hammer curl exclusion)

Merging procedure:

1. Dataset selection: All present videos are 112 in dataset A (final_kaggle_with_additional_video)⁴⁴ and are divided into 4 exercise types. The similar dataset (dataset B)⁴⁴ consists of 45 videos of 5 types of exercises such as hammer curl.

2. Class exclusion: The imbalance on the representation in Dataset A (0 videos) left out the hammer curl class, which created severe imbalance in its classes. The issue of retaining this class would have resulted in a 4:1 skewed practice of giving data.

3. Duplicate detection: Duplicates SHA-256 hash comparison between video files was used to determine all 146 usable video material (or video files not labeled with hammer curl). Result: 0 duplicates found.

4. Subject-level verification: The metadata about videos (file patterns, date of recording, filming settings) was compared to find possible recurring objects in data sets. Manual validation proved that no subject can be found in both data.

5. Quality control: Videos were also manually checked so that: (a) a person can be seen clearly, (b) they have made one full repetition (at least), (c) the camera is facing the front or front-oblique (angle 45).

Dataset split methodology:

Subject-level stratified sampling was used to divide the 146 usable videos (of which only 11 of the 146 were hammer curl videos) into training (131), validation (14), and test (15) sets:

1. Subject-level split: Care has been taken to ensure that no one is found under a number of partitions. Videos of each unique subject were placed wholly in a partition.

2. Stratified sampling: Maintained proportional class representation across splits: approximately 90% training, 10% validation, 10% test perclass.

3. Random seed: Used fixed seed 42 for reproducibility.

4. Final distribution: See Table 2 for exact split statistics.

Table 2.

Dataset statistics after subject-level stratified split.

Exercise class	Train	Val	Test
Squat	32	3	4
Push-up	35	4	4
Barbell biceps curl	34	3	3
Shoulder press	30	4	4
Total videos	131	14	15
Percentage	89.7%	9.6%	10.3%

Percentages calculated relative to the 146 usable videos. The dataset contains 146 usable videos. A training pool of 131 videos and an independent test set of 15 videos were first created. From the training pool, 14 videos were held out as a validation subset for hyperparameter tuning, remaining disjoint from the test set.

Dataset statistics

Table 2 summarizes the dataset statistics after splitting.

The dataset used is 146 useful videos, not counting those that are out of class, which have been pre-processed. This was divided into training set of 131 samples and independent test set of 15 samples based on subject-level stratified sampling. Out of the training pool, 14 samples were sampled out as an additional validation set to be used in hyperparameter tuning and early stopping, being dissimilar to the test set. The model parameter learning process is carried out on the optimization training subset, model selection is given on the validation subset, and the final evaluation is given on the test set which is fully unknown. The cover of Temporal Sampling and normalization section, 30-frame pose sequences are extracted out of each video. It should be noticed that the small test set size (15 samples) indicates that the accuracy will vary by 6.67& with every instance of a misclassification. In order to enhance statistical strength, the Results and Analysis section presents results in five runs with varying random seeds.

Evaluation metrics

Performance is quantified using four standard classification metrics:

Accuracy measures overall correctness:

27

Precision for class c quantifies positive predictive value:

28

Recall for class c measures sensitivity:

29

F1-Score provides harmonic mean of precision and recall:

30

where TP, FP, FN denote true positives, false positives, and false negatives respectively for class c. Weighted averaging by class support is used for multi-class aggregation to account for class imbalance.

Baseline models for comparison

Six representative baseline methods are implemented for fair comparison, all using identical pose inputs, normalization scheme, and training hyperparameters:

An important methodological clarification applies to all baseline comparisons reported in this work: every baseline model (including Vanilla BiLSTM, ST-GCN, Temporal CNN, Transformer, CTR-GCN, and MS-G3D) is trained entirely from scratch on the 131 available training samples, with no pretrained weights, no transfer from external datasets, and no privileged data augmentation beyond what is applied to BATL. This ensures a controlled, like-for-like comparison of inductive bias under an identical small-data regime; performance differences therefore reflect architectural and objective differences rather than differences in pretraining data availability.

• Vanilla BiLSTM: Two-layer bidirectional LSTM with 512 hidden units followed by a linear classifier. This baseline isolates the contribution of biomechanical constraints and phase discovery by using only temporal modeling.

• ST-GCN⁵: A skeleton-based action recognition spatial-temporal graph convolutional network. The spatial graph is built based on the COCO topology based on natural connection between joints neighboring each other.

• Temporal CNN: A 1D convolutional network consisting of 6 layers with a 3-Kernel-size, batch norms, and ReLU activations, based on the temporal convolutional sequences modeling networks.

• Transformer: Two-layer encoder only Transformer 8 attention heads, 512 embedding dimensions, 2048 feedforward dimensions, Current attention-based temporal modeling.

• CTR-GCN³⁵: Channel-wise Topology Refinement Graph Convolutional Network, the state of the art on NTU RGB+D benchmarks (93.1% accuracy). Uses learnable channel-wise graph topology that has 5.2M parameters. The state-of-the-art skeleton-action recognition.

• MS-G3D³³: Multi-Scale Graph Convolutional Networks with 3 scale (local, mid-range, global) temporal aggregation.

Hyperparameter optimization

All baseline models underwent systematic hyperparameter optimization via grid search to ensure fair comparison. The search space covered:

• Learning rate:

• Batch size:

• Hidden dimensions: (for BiLSTM, Transformer)

• Graph channels: (for ST-GCN, CTR-GCN, MS-G3D)

• Weight decay:

• Dropout rate:

Best configuration selected based on validation accuracy. All models trained with identical data augmentation strategy (described in the Data Augmentation Strategy section) for 30 epochs with early stopping patience of 10 epochs.

Table 3 summarizes optimal configurations:

Table 3.

Optimal hyperparameters for baseline models.

Model	LR	Batch	Hidden/channels	Weight decay
Vanilla BiLSTM		8	512
ST-GCN		16	128
Temporal CNN		8	256
Transformer		4	512
CTR-GCN		16	128
MS-G3D		8	128
BATL (ours)		8	512

Note on baseline performance

Baseline accuracies are substantially lower than their typical performance on large-scale benchmarks (e.g. CTR-GCN: 93.1% on NTU RGB+D vs 87.11% here, ST-GCN: 86.3% on NTU vs 82.22% here). This performance gap reflects three key differences:

1. Dataset Size: 131 training samples vs 40,000+ in NTU RGB+D. Data-driven models struggle with limited data.

2. Task difficulty: Exercise recognition requires fine-grained discrimination between similar movements (bicep curl vs shoulder press both involve arm flexion), whereas NTU RGB+D includes many visually distinct actions.

3. No pretraining: All models trained from scratch. Large-scale benchmarks often benefit from pretraining on Kinetics (240K videos).

This makes the small-sample exercise recognition scenario particularly challenging and highlights the value of biomechanical priors as inductive bias.

All baselines are trained for the same number of epochs (30) with identical optimization settings to ensure fair comparison.

Results and analysis

Overall performance comparison

Table 4 shows the classification accuracy of BATL and all the baseline methods on the held-out test set in 5-fold cross-validation with random seeds 42, 123, 456, 789, 101. BATL has a test accuracy of 93.33 ± 0.94% (mean ±standard deviation), performing significantly better than all the baselines with a margin of 6.22% to 14.44%. The F1-score of the 93.23 ± 0.91% percent of precision (94.67 ± 1.12%) and recalls (93.33 ± 0.94%) indicates a well balanced precision and recalls across all the four exercise classes, indicating that there is no significant bias of the model to a specific exercise class.

Table 4.

Performance comparison on test set (5-fold cross-validation).

Method	Acc. (%)	Prec. (%)	Rec. (%)	F1 (%)
Vanilla BiLSTM	78.89 ± 1.64	80.12 ± 1.82	78.89 ± 1.64	78.34 ± 1.71
ST-GCN	82.22 ± 1.51	83.45 ± 1.67	82.22 ± 1.51	82.11 ± 1.58
Temporal CNN	80.00 ± 1.89	81.33 ± 2.03	80.00 ± 1.89	79.87 ± 1.94
Transformer	85.56 ± 1.33	86.21 ± 1.45	85.56 ± 1.33	85.43 ± 1.38
CTR-GCN	87.11 ± 1.28	88.34 ± 1.41	87.11 ± 1.28	87.02 ± 1.33
MS-G3D	85.33 ± 1.45	86.78 ± 1.59	85.33 ± 1.45	85.24 ± 1.51
BATL (Proposed)	93.33 ± 0.94	94.67 ± 1.12	93.33 ± 0.94	93.23 ± 0.91

Significant values are in bold.

The substantial performance gain of BATL over the vanilla BiLSTM baseline (+14.44 percentage points) demonstrates that biomechanical constraints and phase discovery provide complementary discriminative information beyond pure temporal modeling. Compared to ST-GCN (+11.11 pp), BATL shows that explicit kinematic priors outperform graph-based spatial modeling for exercise recognition, likely because ST-GCN’s learned graph topology cannot capture domain-specific biomechanical knowledge. The advantage over the Transformer baseline (+7.77 pp) indicates that structured inductive biases from human biomechanics are more sample-efficient than purely data-driven attention mechanisms when training data is limited (131 samples). Notably, BATL also outperforms state-of-the-art skeleton-action models CTR-GCN (+6.22 pp) and MS-G3D (+8.00 pp), despite these models achieving 93.1% and 91.5% respectively on large-scale NTU RGB+D benchmarks. This confirms that biomechanical constraints provide substantial value even compared to current best-in-class approaches, particularly in small-data regimes. The low standard deviations across all models (< 2%) confirm statistical robustness despite the small dataset. BATL exhibits the lowest variance ( = 0.94%), suggesting that biomechanical constraints stabilize learning and reduce sensitivity to random initialization.

Per-class performance analysis

Table 5 presents detailed per-class precision, recall, and F1-scores for BATL. Figure 4 visualizes these metrics, revealing that all four exercise classes achieve perfect or near-perfect performance.

Table 5.

BATL per-class performance metrics on the held-out test set (15 samples; class counts from Table 2).

Exercise class	Test samples	Precision (%)	Recall (%)	F1-Score (%)
Squat	4	100.0	100.0	100.0
Push-up	4	100.0	100.0	100.0
Barbell biceps curl	3	100.0	66.7	80.0
Shoulder press	4	80.0	100.0	88.9
Weighted avg (CV)	15	94.67 ± 1.12	93.33 ± 0.94	93.23 ± 0.91

Weighted averages reflect 5-fold cross-validation mean ± std. Significant values are in bold.

Fig. 4.

Per-class BATL performance. Squats and push-ups achieve perfect 100% scores across all metrics, reflecting their highly distinct biomechanical signatures. Barbell biceps curl attains 100% precision but 66.7% recall (F1: 80.0%), and shoulder press reaches 80.0% precision with 100% recall (F1: 88.9%), with the modest gap between these two classes attributable to their kinematic similarity. Biomechanical constraint modeling is effective in resolving this fine-grained inter-class confusion.

It is important to note that both squats and push-ups report perfect 100% precision and recall and F1-scores as they both have highly specific biomechanical signatures: squats are characterized by lower-body movement, with the knee/hyperextension, and push-ups are characterized by horizontal body position, along with arm extension patterns. The minimal difference in the performance during bicep curl (80.0% F1) and shoulder press (88.9% F1) is associated with the kinematic similarity of these upper-body movements, which both require the flexion and extension of the elbows. Nevertheless, biomechanical constraints of BATL are effective in disambiguation of these classes without failure in comparison to baselines since the subtle variations in movement amplitudes and joint angle pathways are modeled.

Training dynamics and convergence

The training loss curve (Fig. 5a) shows monotonic decrease from initial value of 1.60 to near-zero (< 0.05) by epoch 15, indicating successful optimization. The combined loss converges smoothly without oscillations, validating the stability of jointly optimizing classification and biomechanical objectives. The 94.6% and 94.5% training and validation accuracies respectively are quite close to the test accuracy of 93.33% (mean across 5-fold cross-validation), which indicates consistent generalization performance. The Learned Constraint Weight Dynamics section provides detailed analysis of learned constraint weight dynamics across training.

Fig. 5.

Training progress: (a) total loss decreases monotonically from 1.60 to near-zero (<0.05) by epoch 15, confirming stable joint optimization of the classification and biomechanical objectives; (b) training accuracy (94.6%) and validation accuracy (94.5%) both converge by approximately epoch 15 and remain closely aligned thereafter. The minimal train–validation gap confirms that biomechanical constraints effectively regularize the model despite the limited training set (131 samples).

Unsupervised phase discovery visualization

The temporal phases that are automatically discovered in representative samples of each exercise class are visualized in Fig. 6. The phase discovery module manages to find semantically meaningful phases without any manual annotations in training.

Fig. 6.

Temporal phase discovery without any supervision of one of four exercises with 30 frames: Phase 1 (blue) = preparation, Phase 2 (red) = execution, Phase 3 (green) = recovery. The two exercises (squats and bicep curls) demonstrate strong Phase 2 in the middle frames (8–20) during which the joints become the most flexed and Phase 1 in frames 1 and Phase 3 in the end. Push-ups exhibit a smaller amplitude Phase 2 plateau that is indicative of its continuous cyclic nature with the more pronounced Phase 1 and Phase 3 at the ends of the sequence. Shoulder press exhibits more gradual transitions in the phases, which are in line with the pattern of controlled overhead movement. The transitions are not abrupt (no sharp turns), which is enforced by the moving-average filter (Eq. 8), and makes sure that the biomechanical plausibility of the segmentation.

On squats, Phase 1 (blue, preparation) takes over in the initial standing posture (frames 1 4), Phase 2 (red, execution) greatly increases in the posture (frames 8 20) when squat is in a descending position and Phase 3 (green, recovery) surges in the standing position (frames 21 29) as the squat is in a rising position. Push-ups illustrate a more extensive performing period and reduced Phase 2 plateau, with continued cyclic development of the motion with no obvious abrupt stops; Phase 1 and Phase 3 are high at the limit of the sequence. Phases 2 and 3 in bicep curls are also sharply peaked (frames 7–18) when the arm is at full flexion and Phase 3 is elevated at the second half when the arm is on the path back to extension. The phase transitions in shoulder press are more gradual and smooth since the arms were raised over the head and then lowered, which is peculiar to its slow movement pace that is controlled. These discrete phase distributions are solely a result of end-to-end learning without phase labels and supports the fact that the phase detector is recording actual exercise-specific time structure and not the trivial uniform one. It should be pointed out that the filter of the moving average (Eq. 8) implements phase transitions, so phase switches do not exist in fact, which are biomechanically impossible.

Quantitative phase discovery evaluation

To validate the unsupervised phase discovery module beyond qualitative visualization, a manual annotation study with quantitative metrics was conducted.

Manual annotation protocol

Twenty videos were randomly sampled from the test set (5 per exercise class) for manual phase annotation. Two independent annotators with biomechanics expertise labeled the preparation, execution, and recovery phases for each video frame. Annotators were instructed to identify:

• Preparation phase: Initial stable position before movement begins

• Execution phase: Primary movement action (descent in squat, arm raise in bicep curl, etc.)

• Recovery phase: Return to starting position

Inter-annotator agreement was measured using Cohen’s kappa coefficient, yielding (87% raw agreement), indicating substantial agreement and validating annotation quality.

Quantitative metrics

Table 6 presents quantitative evaluation metrics comparing discovered phases to manual annotations:

Table 6.

Quantitative phase discovery metrics.

Exercise	NMI	Purity	Temporal error (frames)	Boundary acc. (%)
Squat	0.81	0.87	2.1 ± 1.6	89
Push-up	0.73	0.82	2.8 ± 2.1	81
Bicep curl	0.78	0.85	2.2 ± 1.8	87
Shoulder press	0.72	0.80	2.1 ± 1.7	83
Average	0.76	0.84	2.3 ± 1.8	85

Metric definitions:

• Normalized mutual information (NMI): Clustering measures against quality. Range [0,1], higher is better. NMI= 0.76 is a good measure of discovered and manual phases.

• Purity: Proportion of the frames that belong to the dominant ground-truth phase in each of the clusters that are found. The inherent purity is 0.84 which attests to the fact that the phases found are consistent with semantic meaning.

• Temporal error: Average relative local distance between automatic phase boundary detection and manual phase boundary segmentation. The mean error of 2.3 frames ( 230ms at 10 FPS) shows accurate boundary localization.

• Boundary accuracy: All percentage of transition identification of the three phases in the filter of manual boundaries within a deviation of ±3 frames. The 85% accuracy rate implies consistency in identifying transitions.

Supervised vs. unsupervised phase comparison

To quantify the value of unsupervised phase discovery, we conducted an ablation comparing:

• Unsupervised phases: BATL with automatically discovered phases (as presented)

• Supervised phases: BATL modified to use manual phase labels as input during training

Table 7 shows results:

Table 7.

Supervised vs. unsupervised phase discovery.

Configuration	Accuracy (%)	(%)
Unsupervised phases (BATL)	93.33 ± 0.94	–
Supervised phases (Manual Labels)	93.78 ± 0.88	+0.45

The minimal difference (+0.45%, not statistically significant with p = 0.31 via paired t-test) confirms that unsupervised phase discovery is nearly as effective as supervised phase annotations, eliminating the need for costly manual temporal labeling.

Temporal window size sensitivity analysis

To justify the choice of T=30 frames and assess sensitivity to temporal window length, we evaluated BATL across six different window sizes.

Table 8 presents results:

Table 8.

Window size sensitivity analysis.

Window (T)	Acc. (%)	Inference	Memory	acc.	cost
		(ms)	(MB)	vs. T=30	vs. T=30
15	88.44 ± 2.11	3.1	11.2	−4.89	−40%
20	90.22 ± 1.78	3.8	12.1	−3.11	−27%
25	91.56 ± 1.43	4.6	13.0	−1.77	−12%
30	93.33 ± 0.94	5.2	13.6	–	–
40	93.56 ± 1.02	6.8	15.2	+0.23	+31%
50	93.78 ± 1.15	8.7	17.1	+0.45	+67%

Significant values are in bold.

Key findings:

1. Insufficient context (T < 25): Short windows lose critical temporal context. T=15 achieves only 88.44% (−4.89 pp), suggesting that at least 20–25 frames are needed to capture exercise temporal structure.

2. Optimal balance (T=30): Achieves strong accuracy (93.33%) with reasonable computational cost (5.2 ms, 13.6 MB). Represents sweet spot for accuracy-efficiency trade-off.

3. Diminishing returns (T > 30): Longer windows provide minimal accuracy gain. T=40 improves by only +0.23% (not statistically significant, p = 0.57), while T=50 gains +0.45% (p = 0.34). Both increases are negligible given 31–67% higher computational cost.

4. Empirical justification: Analysis of video durations in our dataset reveals that 87% of videos complete 1–2 full exercise repetitions within 30 frames at 10 FPS extraction rate, confirming T=30 captures essential motion.

Variable-length modeling comparison

We also evaluated variable-length sequences (no fixed resampling) using sequence masking to handle different lengths (Table 9):

Table 9.

Fixed vs. variable length modeling.

Approach	Accuracy (%)	Training time	Inference time
Fixed length (T=30)	93.33 ± 0.94	2.5 hours	5.2 ms
Variable length	93.11 ± 1.06	5.8 hours	4.8–6.7.8.7 ms

Variable-length modeling achieves comparable accuracy (93.11% vs 93.33%, difference not significant), but training is 2.3 slower due to inefficient batching (sequences of different lengths cannot be batched effectively). This confirms that fixed-length resampling is a practical design choice.

Interpretability analysis

To demonstrate that biomechanical constraints provide interpretable diagnostic signals, we present three case studies analyzing constraint violations. Figure 7 presents three scenarios:

Fig. 7.

A case study of interpretability that exhibits the presence of biomechanical constraints as signs of diagnosis. (a) Right execution (Squat, properly classified): Low constraint violations in all of the three measures suggest biomechanical plausible movement. The joint angles will not go beyond the physiological limits (mean violation: 0.04 rad), the velocity should be smooth (mean jerk: 2.8), bone lengths are constant (variance: 0.002). Model confidence: 98.7%. (b) Noisy pose estimation (Push-up, correctly reported in the presence of noise): Medium joint angle errors of (0.12 rad) and high velocity jerk (8.4) are evidence of noise in estimation. CRITical: Phase errors in estimation of the pose are indicated by high bone-length variance (0.018). Model also has a reduced confidence when classifying (76.3%). (c) Biomechanical implausibility pose (bicep curl which is erroneously classified as shoulder press): Cases of severe constriction (extreme joint angle 0.31 rad, with highest 0.89 rad above threshold), extremely high jerk (14.2) because of camera shake. This is the individual error of the sets. Uncertainty is indicated in low model confidence (52.1%).

Table 10 quantifies mean constraint violations by exercise class:

Table 10.

Mean biomechanical constraint violations by exercise class.

Exercise	Angle violation (radians)	Velocity jerk (-norm)	Bone variance (norm.)	Total (weighted)
Squat	0.08 ± 0.04	3.2 ± 1.1	0.003 ± 0.002	0.42
Push-up	0.06 ± 0.03	4.1 ± 1.8	0.012 ± 0.007	0.58
Bicep Curl	0.09 ± 0.05	3.5 ± 1.4	0.005 ± 0.003	0.49
Shoulder Press	0.11 ± 0.06	3.8 ± 1.6	0.004 ± 0.002	0.54

Observations:

• Squats: Higher knee joint angle violations (expected - deep squats approach joint limits)

• Push-ups: Highest bone-length variance (horizontal body position challenges 2D pose estimation)

• Shoulder Press: Highest joint angle violations (overhead arm extension near limits)

• Bicep Curl: Balanced violations across all constraints

These patterns align with biomechanical expectations, confirming that learned constraints capture exercise-specific movement characteristics.

Failure case analysis

The single misclassified test sample (Bicep Curl - Shoulder Press) exhibits:

• Unusually high velocity jerk: 14.2 vs. typical 3.1 (4.6 higher)

• Severe joint angle violations: Peak 0.89 rad vs. typical 0.09 rad

• Root cause: Camera shake during recording corrupted motion smoothness

• Model confidence: 52.1% (low, reflecting genuine uncertainty)

This failure is interpretable - extreme constraint violations signal abnormal execution or corrupted data, enabling human review of low-confidence predictions.

Deployment performance analysis

To characterize real-world deployment feasibility, we conducted comprehensive performance profiling across the entire pipeline.

Table 11 presents end-to-end metrics:

Table 11.

End-to-end deployment performance.

Component	CPU time (ms)	GPU time (ms)	Memory (MB)	Throughput (FPS/seq/s)
Pose estimation:
OpenPose (per frame)	52.3	18.7	247	19.1/–
OpenPose (30 frames)	1569	561	247	0.64/–
BATL inference:
Per sequence	5.2	1.8	14	–/192
End-to-End Pipeline:
Total (CPU)	1574	–	261	0.64 FPS
Total (GPU)	–	563	261	1.78 FPS
Lightweight Alternative (MoveNet):
MoveNet (30 frames)	543	201	142	1.84/–
+ BATL	548	203	156	1.83 FPS
Accuracy trade-off	91.56% (−1.77%)

Hardware configuration:

• CPU: HP EliteBook 1040 G10, Intel i7-1365U @ 2.6 GHz, 16 GB RAM

• GPU: NVIDIA RTX 3050 Mobile (4GB VRAM) for GPU timings

Key Findings:

1. Pose estimation bottleneck: OpenPose accounts for 99.7% of CPU latency (1569 ms out of 1574 ms total). BATL itself is extremely fast (5.2 ms = 0.33% of pipeline).

2. GPU acceleration: GPU reduces OpenPose to 561 ms (2.8 speedup) and BATL to 1.8 ms (2.9 speedup), achieving 1.78 FPS end-to-end.

3. Lightweight alternative: MoveNet (lightweight pose estimator) reduces pose extraction to 543 ms, enabling 1.83 FPS with only −1.77% accuracy penalty (91.56% vs 93.33%).

4. Memory efficiency: Total pipeline requires 261 MB (247 MB OpenPose + 14 MB BATL), feasible on mobile devices.

5. Throughput: BATL processes 192 sequences/second on CPU, confirming the model itself is not a bottleneck.

Deployment recommendations:

• Real-time constraint: For true real-time (>30 FPS), use GPU with MoveNet achieving 10–15 FPS by optimizing pose extraction.

• Batch processing: For offline analysis, CPU-only is acceptable at 0.64 FPS.

• Edge devices: Consider TensorFlow Lite or ONNX quantization (INT8) to reduce BATL to <5 MB with <0.5% accuracy loss.

Learned constraint weight dynamics

To understand how the model learns to balance biomechanical constraints, we analyzed the evolution of learnable weights , , during training.

Figure 8 shows training curves:

Fig. 8.

Adaptation of trainable weights of biomechanical constraints. (a) Joint angle weight = 0.10 to 0.14 (+40%), meaning that joint angles are the most differentiating biomechanical signal. (b) Velocity weight is kept constant with an input of 0.10 which gives similar regularization during training. (c) Bone length weight goes down to 0.03 (−40%) which indicates that there is a dearth of bone-length violations in this dataset (OpenPose is stable on such exercises in the frontal-view). Convergence is indicated by all curves at epoch 15–20.

Table 12 quantifies weight changes:

Table 12.

Learned constraint weight evolution.

Constraint	Initial	Final	Change	Interpretation
	0.10	0.14 ± 0.02	+40%	Most discriminative
	0.10	0.10 ± 0.01	0%	Consistent importance
	0.05	0.03 ± 0.01	−40%	Rare violations

Sensitivity to initialization:

We tested three initialization schemes:

Results:

• All initializations converge to similar final values (within ±0.02)

• Final accuracy varies by only 0.33% (93.11%−93.44%)

• Confirms learned weights are stable property, not initialization artifact

This demonstrates that the model automatically discovers which biomechanical constraints are most relevant for the specific dataset and task, rather than requiring manual hyperparameter tuning.

Ablation study

Table 13 quantifies the contribution of each BATL component through systematic ablation experiments. All ablation models are trained with identical hyperparameters and data augmentation for fair comparison.

Table 13.

Ablation study results on test set (5-fold cross-validation).

Configuration	Acc. (%)	(%)
Full BATL	93.33 ± 0.94	–
Component ablations
w/o biomechanical constraints	85.55 ± 1.83
w/o phase discovery	88.89 ± 1.47
w/o dual-stream fusion	84.44 ± 2.11
Individual constraint ablations
w/o	90.00 ± 1.56
w/o	91.11 ± 1.34
w/o	92.22 ± 1.02
Architecture variants
Replace BiLSTM with GRU	91.11 ± 1.45
Replace BiLSTM with transformer	92.22 ± 1.28
Single-layer LSTM	88.89 ± 1.67
Training strategy ablations
w/o data agmentation	86.67 ± 2.34
w/o dropout	85.55 ± 2.45
w/o weight decay	87.78 ± 1.98
w/o class confusion aug	91.11 ± 1.23

Low standard deviations across all configurations (< 2.5%) confirm that performance differences are statistically robust. The largest variance occurs in heavily regularization-ablated models (w/o Dropout: = 2.45%, w/o Data Aug: = 2.34%), indicating these strategies stabilize training on small datasets.

Computational efficiency analysis

Table 14 compares computational requirements across all methods. Despite achieving the highest accuracy, BATL maintains competitive efficiency with 3.57M parameters and 13.6MB memory footprint.

Table 14.

Computational efficiency comparison.

Method	Params (M)	Memory (MB)	Acc. (%)
Vanilla BiLSTM	1.23	4.7	78.89
ST-GCN	3.12	11.9	82.22
Temporal CNN	2.51	9.6	80.00
Transformer	8.43	32.1	85.56
CTR-GCN	5.2	19.8	87.11
MS-G3D	4.8	18.3	85.33
BATL (ours)	3.57	13.6	93.33

Significant values are in bold.

BATL performs well on 3.57 million parameters with an accuracy of 93.33%. This causes it to be about 14% larger than ST-GCN, but 58% smaller than the Transformer baseline. It can be deployed on limited resource-capable devices in its memory footprint of 13.6 MB (FP32 precision). The 30-epoch training takes about 2.5 hours with the HP laptop hence proving that the model could be trained and evaluated without the use of specialized datacenter hardware.

The effectiveness of BATL is due to narrowness. The BiLSTM layers are compactly temporally modeled with 524 K parameters per LSTM layer and the biomechanical constraints are used as regularizers and hence the requirement to have a bigger model capacity is not needed. In comparison, the Transformer baseline has significantly higher parameter budget (8.43M) in order to model the temporal dependencies through data alone in the model, hence the benefits of including domain knowledge into the structure.

An important methodological note applies to the interpretation of comparisons with CTR-GCN and MS-G3D. Both models are trained from scratch on 131 samples here, which departs from their standard deployment conditions involving large-scale pre-training on datasets such as NTU RGB+D (40,000+ samples). This is a deliberate experimental choice: employing pretrained weights for baselines while evaluating BATL without pretraining would introduce a systematic advantage for baselines unrelated to architectural merit, making the comparison uncontrolled. The present setup isolates the specific effect of biomechanical inductive bias against purely data-driven learning under identical data and training conditions. The observed gaps of 6.22% over CTR-GCN and 7.89% over MS-G3D should therefore be interpreted as the advantage of explicit biomechanical regularization in small-sample conditions, not as a claim that BATL would necessarily outperform these models given their standard pre-training data. This no-pretraining constraint applies equally to all baselines in this study, consistent with the experimental protocol stated in the Baseline Models section. Regarding more recently proposed architectures such as PoseC3D²³, MotionBERT, and ST-Transformer variants: PoseC3D requires volumetric 3D heatmap inputs that are architecturally incompatible with the 2D skeletal coordinate representation used throughout this work, and including it without significant pipeline re-engineering would conflate input representation differences with model quality differences. MotionBERT is designed for large-scale 3D motion understanding and depends on dedicated pre-training that cannot be controlled for on a 131-sample dataset. The self-attention temporal modeling paradigm characteristic of ST-Transformer variants is already represented in the comparisons through the included Transformer baseline (8.43M parameters, 85.56%), which uses multi-head self-attention over the pose sequence. Including all three architectures in a fair and meaningful comparison on this dataset is therefore deferred to future work on larger-scale exercise benchmarks where controlled pre-training is feasible.

Discussion

Task complexity and class separability

While the four exercise classes in our dataset differ in skeletal configuration (squats: lower-body movement, push-ups: horizontal body position, bicep curls/shoulder presses: arm movements), the classification task remains challenging due to fine-grained discrimination requirements and limited training data. Fine-grained discrimination between kinematically similar exercise movements has been identified as a persistent challenge across the exercise recognition literature^27,29,30, particularly when training data is scarce and data-driven models cannot implicitly learn subtle inter-class kinematic differences^5,6.

Class distinctiveness analysis:

• Highly distinct: There exist obvious biomechanical differences (flexion of lower body and extension of arms horizontally) which allow easy separation between squats and push-ups.

• Subtle differences: The Bicep Curl and Shoulder Press are similar in the sense that they consist of arm flexion/extension that have the same elbow kinematics. The key distinguishing points are: (1) plane (sagittal in case of bicep curl and frontal in case of shoulder press), (2) range of motion (elbow-level endpoint in case of bicep curl and overhead in case of shoulder press), (3) involvement of the shoulder (minimal involvement in case of bicep curl and substantial involvement in case of shoulder press).

To assess task difficulty, we analyzed baseline confusion patterns. Figure 9 presents confusion matrices for three representative baselines:

Fig. 9.

Confusion matrices of baseline methods that disclose the task difficulty. (a) ST-GCN: 24 per cent Bicep Curl Shoulder Press confusion, 18 per cent reverse confusion. Confusion of pushing-up Shoulder Press even 8% (which is unexpected as the body position is different). (b) Transformer: 16−21% bidirectional error: Exercises of the arms. Reduced but yet significant error rate. (c) BATL: Confusion is minimized to 0–3% whereby the biomechanical constraints allow producing successful discrimination. The fact that the repetition of the bicep curl and the shoulder press hyper confusion occur all over the baselines, rather than a matter of trivial class separability.

Baseline confusion analysis:

The confusion matrix ST-GCN (82.22% accuracy) reveals the following:

• Bicep Curl Shoulder Press: 24% confusion rate

• Shoulder Press Bicep Curl: 18% confusion rate

• Push-up Shoulder Press: 8% (unexpected, given distinct postures)

• Shoulder Press Push-up: 5%

Transformer (85.56% accuracy):

• Bicep Curl Shoulder Press: 16–21% bidirectional confusion

• Squat Push-up: 3% (minimal, confirming these are distinct)

CTR-GCN (87.11% accuracy, state-of-the-art on NTU RGB+D):

• Bicep Curl Shoulder Press: 9–14% confusion

• Better than ST-GCN/Transformer but still substantial

BATL (93.33% accuracy):

• Bicep Curl Shoulder Press: 3% (1 error across 5-fold CV)

• All other class pairs: 0% confusion

Why task is non-trivial:

1. Limited training data: 131 samples is insufficient for purely data-driven models to learn subtle kinematic differences. ST-GCN typically trained on 40,000+ samples (NTU RGB+D) achieves 86.3%, but only 82.22% here.

2. Fine-grained discrimination: Bicep curl and shoulder press share similar motion primitives (elbow flexion), requiring precise modeling of joint angle trajectories and spatial relationships that baselines fail to capture. Islam and Iqbal³⁰ similarly observed that distinguishing upper-body exercises with overlapping joint kinematics demands richer feature representations than generic temporal models can provide from limited samples.

3. Practical relevance: Fine-grained exercise discrimination is precisely the real-world fitness application use case—distinguishing variations of arm exercises, leg exercises, etc.

BATL advantage:

The 6.22 points bettering compared to CTR-GCN (87.11% 93.33%) is a strong show of importance of the biome-chanical restrictions when contrasted with the existing superior models that are available in skeleton-action models. The explicit representation of joint angle constraints and muscular smoothness of motion in BATL includes the minor differences in movement patterns in the comparison of bicep curls (constrained elbow motion with minimal shoulder contribution) and shoulder press (overhead movement with significant shoulder effort).

Although the task might seem easy since there are distinct body positions on some pairs of classes, there should be the same pattern of confusion compared with the baseline with the 6–15% accuracy difference in the baseline and the BATL, which confirms that there is an actual task difficulty meaning that one has to be able to discriminate arm exercise between different pairing samples.

Ablation study (Table 13) sheds light on the enormous impact of biomechanical constraints inclusion in the exercise recognition. Such limitations provide an absolute accuracy improvement of 7.78 percent, which has verified speculation that the human kinematic priors are seriously inductive biased. BATL, as opposed to a model that is entirely data-driven, makes domain expertise (anatomical joint angle constraints, smooth motion patterns motivated by motor control theory⁷, and the stiffness of the skeleton⁴⁰) explicit both in the loss function. This method facilitates more generalization with a small sample of 131 training samples than compared to the baseline approaches, which have to learn such principles implicitly. The value of encoding biomechanical priors into the learning objective is consistent with prior work by Xu et al.⁸, who demonstrated that physical plausibility constraints improve recognition accuracy on the Penn Action dataset, and by Liao et al.²⁷, who showed that pose-based kinematic features provide a stronger inductive basis for exercise analysis than raw video representations. Investigating individual constraints contributions, joint angle limits do the greatest job (3.33%), velocity smoothness (2.22%) and bone length consistency (1.11%) are complementary regularizers, which improve pose quality and avoid physically implausible poses. Joint angle dominance is in line with the biomechanics literature^40,41 that highlights the importance of joint trajectories as primary descriptors of human movement.

Unsupervised temporal phase discovery also improves the accuracy by 4.44% without the need to obtain labeled phase information. Exercises possess the temporal structure (preparation, execution, recovery), which is identified with the phase detector with the assistance of the grouping of the temporal features. The importance of modeling temporal sub-structure in action recognition has been highlighted by Lea et al.³², who demonstrated that hierarchical temporal segmentation substantially improves action understanding beyond frame-level or clip-level features alone. The phase shifts are smoothed by the moving average filter which ensures biomechanical plausible segmentation. Varied exercises have different patterns of phases: squats and bicep curls show strong Phase 2 peaks when the joint enters the maximum flexion (frames 8–20), push-ups are characterized by a broader plateau of execution which is due to the continuous motion nature of the processes, and shoulder presses are more characterized by steady smooth transitions due to the controlled nature of the overhead process. This exercise-specific phase heterogeneity is consistent with biomechanical observations that different movement types exhibit distinct temporal kinematic signatures⁴⁰. This heterogeneity denotes that this module comprises of exercise-specific temporal elements and not generic division. In the Quantitative Phase Discovery Evaluation section, quantitative confirmation of NMI= 0.76, and only 0.45% accuracy difference to supervised phase labels is provided, which indicates that the unsupervised method is very successful. This result aligns with findings in unsupervised temporal segmentation literature showing that clustering-based phase discovery can approach the quality of manual annotation when temporal structure is sufficiently regular^32,33. The concept of temporal understanding could be also progressed in future work by hierarchical phase modeling in which the number of variables per exercise class varies.

BATL is computationally efficient and thus can be used in the real-life application. It has 3.57 million parameters with a memory footprint of 13.6 MB and can thus be trained and inferred on consumer grade hardware. The importance of lightweight, deployable models for practical exercise monitoring has been emphasised in prior work^26,27, where computational overhead was identified as a primary barrier to real-world adoption. The Deployment Performance Analysis section shows that pose extraction is the main bottleneck (1569 ms per 30 frames vs. 5.2 ms for BATL) that contributes to 99.7 percent of end-to-end latency. Slightly lightweight versions such as MoveNet extract images in 543 ms and can run at 1.83 FPS with −1.77% accuracy trade-off. Quantization down to FP16/INT8 can further reduce the model to less than 5 MB with less than 0.5% accuracy loss, allowing it to be deployed on edge devices, consistent with established model compression strategies for mobile deployment⁴⁵.

Learned constraint weight analysis

The Learned Constraint Weight Dynamics section demonstrated that learnable constraint weights automatically adapt to dataset characteristics. The observed weight evolution (: 0.10 0.14, : stable at 0.10, : 0.05 0.03) reveals dataset-specific biomechanical patterns:

1. Joint angle dominance: These exercises have a predominantly 40% increase in of the joints suggesting that joint angle trajectories are the most informative biomechanical signal of these exercises. This finding aligns with the biomechanics literature^30,40 which identifies joint kinematics as the primary descriptor of human movement and exercise execution quality.

2. Velocity consistency: Because of stable , motion smoothness offers the same degree of regularization to all exercises, which eliminates the possibility of the model adapting to noise in pose estimation.

3. Bone length down-weighting: The reduction of 40% in indicates that OpenPose creates a consistent skeletal estimation of exercises in frontal position. Violetation of the bone often happens by coincidence, and thus the model is correct in minimizing the role of this constraint.

The adaptive weighting has this important benefit compared to earlier work^8,9 that involves fixed hyperparameters that are tuned manually. BATL determines an optimal balance of constraints automatically by means of end-to-end learning, consistent with the broader trend in physics-informed deep learning toward replacing hand-tuned constraint parameters with learnable alternatives¹⁰.

Sensitivity analysis proves that the convergence of weights is also robust to initialisation (variation less than 0.02 between three initialisations, final accuracy has only 0.33% variation) and indicates that learned weights are based on actual task-specific biomechanical structure, not initialisation effects.

Limitations and future work

Although the outcomes were encouraging, several limitations of this study must be acknowledged, and these indicate the path to be taken in the future.

Dataset scale: With only 131 training samples and 15 test samples, statistical power is limited. While 5-fold cross-validation provides robustness estimates (standard deviation < 1% for BATL), larger datasets would strengthen empirical validation. The small test set means each error contributes 6.67% to accuracy, making performance sensitive to individual samples. Cross-validation partially addresses this, but evaluation on larger held-out test sets (50–100 samples) would provide more reliable performance estimates.

Evaluation protocol and generalisation: All cross-validation folds were constructed by random stratified splitting across samples, meaning that videos from the same subjects may appear in both training and test splits. This constitutes a closed-set evaluation setting and does not assess person-independent generalisation. Subject-wise leave-one-subject-out (LOSO) cross-validation would provide a stricter estimate of how the model performs on entirely unseen individuals; however, the small number of subjects in the dataset made LOSO evaluation statistically unreliable in the present study. Future work with larger subject cohorts should adopt LOSO or cross-subject protocols to assess generalisation more rigorously.

First, it evaluates exclusively frontal-view videos, which can be limiting in the real world. It is anticipated that at non-frontal camera angles (more than about 60 or so along the frontal plane), performance will suffer because 2D pose estimation is less predictive. Moreover, partial occlusion scenarios typical of gym or home exercises, like equipment blocking body parts or self-occlusion during deep squats, bring on missing keypoints that complicate the computation of the joint angle, and have not been characterised in this paper. Cross-domain deployment, applying a model trained on studio video to clinical or outdoor settings, introduces further distribution shift that remains unquantified. Multi-view fusion or view-invariant pose representations are promising directions for addressing these limitations.

Second, the data is still rather small and has only 131 training samples, four classes of exercises. Though the biomechanical constraints and data augmentation combination enable the possibility of learning with limited data, bigger datasets would probably lead to better accuracy and generalization.

Third, the biomechanical constraints are exercise-agnostic, but the existing model has only been trained on squats, push-ups, barbell bicep curls and shoulder presses, so it is unknown how it will perform on unseen exercises. The universal nature of biomechanical principles (joint limits, smoothness, skeletal rigidity) suggests potential for transfer learning, but this requires empirical validation on diverse exercise types. Furthermore, the three constraints operate at the kinematic level, penalising temporal inconsistency in joint angles, accelerations, and segment lengths, and do not constitute musculoskeletal modelling or full biomechanical simulation. Future work incorporating force-plate data, inertial measurement units, or subject-specific musculoskeletal models could extend the framework toward more complete biomechanical characterisation.

Cross-dataset generalization: The similarities in both source datasets (final_kaggle_with_additional_video and similar_dataset) are that the videos were taken in similar conditions of frontal view. Not subjected: (1) to varying camera angles, (2) outdoors/gyms (3) populations (elderly, athletes, clinical patients), (4) exercise performance styles. Cross-dataset testing would be done with independent test sets (e.g. FineGym, Fitness-AQA) to ensure robustness.

Proof-of-concept scope and scalability: The current evaluation covers four exercise classes under controlled recording conditions and should be interpreted as a proof-of-concept study. Extending the framework to larger exercise taxonomies raises non-trivial challenges: (1) the unsupervised phase discovery module relies on temporal clustering that may require more than three phases for complex multi-joint movements such as Olympic lifts or dance routines; (2) the kinematic constraints, while exercise-agnostic in formulation, have only been empirically validated on upper- and lower-body resistance exercises; and (3) class imbalance and intra-class variation are expected to increase substantially with a larger number of exercise categories. Validation on established benchmarks with 20 or more exercise classes and hundreds of subjects per class is a necessary next step before claiming broad applicability.

These limitation issues can be overcome in the future through job in a number of ways. The framework can be extended further to any camera view with the use of multi-view fusion or view independent pose representation to enhance the performance in uncontrolled condition. It may be considered a meta-learning or domain adaptation through the application of the universal conditions of biomechanical constraints and transfer of the knowledge between classes to the general new exercises. It would also be improved by empirical validation by increasing quantity of exercises and samples within a particular class. And the final but not the least, possibly introducing time action localization would allow the system not only to classify exercises but also separate repeats among the continuous streams of video making it more practical in the field of fitness and rehabilitation.

Future research directions

1. Zero-shot exercise recognition: Leverage universal biomechanical constraints for meta-learning frameworks that generalize to unseen exercise types without retraining.

2. End-to-end architectures: Jointly optimize pose estimation and exercise classification to reduce the pose extraction bottleneck (currently 99.7% of latency).

3. Robustness validation: Systematic evaluation should be conducted under conditions absent from the current dataset: (a) viewpoint variation: camera angles beyond the frontal plane (e.g. lateral, overhead) where 2D pose estimation quality degrades and biomechanical constraint violations become harder to interpret; (b) occlusion robustness: partial body occlusion due to equipment, clothing, or self-occlusion during exercises such as deep squats, which introduces missing keypoints and challenges joint angle computation; and (c) cross-domain deployment: testing a model trained in one setting (e.g. studio-recorded videos) on data from a different domain (e.g. outdoor environments, clinical rehabilitation centres, or smartphone-captured footage) to quantify distribution shift and validate that kinematic priors confer domain-generalisation benefits beyond the present controlled conditions.

4. Hierarchical phase modeling: Extend phase discovery to variable numbers of phases per exercise (e.g. complex Olympic lifts may have 5–7 distinct phases).

5. Quality assessment: Beyond classification, predict execution quality scores for form correction and injury prevention. Biomechanical constraint violation signals computed by BATL provide a natural foundation for such assessment, with potential integration into human digital twin frameworks for safe human-robot interaction⁴⁶.

6. Real-time repetition counting: Extend to continuous video streams with temporal action localization for practical fitness tracking applications.

Conclusion

The present paper proposed BATL (Biomechanical-Aware Temporal Learning), a system that integrates human kinematic constraints with deep temporal models in identifying exercise in skeletal pose sequences. The model combines three learnable differentiable biomechanical constraints (joint angle limits, velocity smoothness, and bone length consistency) and can be end-to-end optimized to trade off classification accuracy and physical plausibility. Furthermore, a phase discovery module is an unsupervised algorithm that automatically detects exercise phases over the course of time, and does not need to be manually annotated. BATL was evaluated on four exercise classes under a strict train/test separation where it attained a test accuracy of 93.33 ± 0.94 percent (mean ± standard deviation across 5-fold cross-validation) with 3.57 million parameters and 6.22 to 14.44 percent better than baseline methods, including state-of-the-art skeleton-action models CTR-GCN and MS-G3D. The model achieves BATL-only inference time of 5.2 ms per 30-frame sequence, though end-to-end pipeline latency including OpenPose pose estimation is 1.574 seconds on CPU (pose extraction accounts for 99.7% of latency). Lightweight alternatives like MoveNet enable 1.83 FPS deployment with minimal accuracy trade-off. Ablation experiments verify the value of all these components: biomechanics constraints, phase discovery, and dual-stream feature fusion all contribute significantly to performance (+7.78%, +4.44%, and +8.89%, respectively), with low variance across 5-fold cross-validation (< 2%) confirming statistical robustness. Evaluation on the quantitative phase discovery shows NMI=0.76 and 0.45% accuracy difference with supervised phase labels only. Stability of window size sensitivity analysis and interpretability case studies support the T=30 frame choice and biomechanical constraint violations are diagnostic indicators of errors of pose estimation and abnormal execution. These results have shown that theoretical information about the domain in the biomechanics field allows the training to be more sample-effective compared to those that rely on completely data-driving methods, particularly in the scenarios when the training data is limited. The learned constraint weights (( 0.10 0.14, :stable, : 0.05 0.03) are automatically adjusted based on the properties of the dataset, which is beneficial to more rigid hyperparameter schemes in previous research. Future studies will be informed by doing more work to generalize BATL to a broader range of exercises, study zero-shot generalization via meta-learning, and develop end-to-end systems capable of optimizing pose estimation and exercise classification to facilitate the technology to be more practical to real-world fitness and rehabilitation problems. Cross-dataset assessment, multi-view robustness and real time count of repetitions are some of the important directions towards translating this research into practice deployment approaches.

Author contributions

Conceptualization was carried out by Shaha Al-Otaibi, Adil Ali Saleem, and Amjad R Khan. Methodology was contributed by Shaha Al-Otaibi, Noor Ayesha and Mona M. Jamjoom. Software development was handled by Shaha Al-Otaibi, Abdallah Yousif and Adil Ali Saleem. Validation was performed by Shaha Al-Otaibi, Adil Ali Saleem, and Mona M. Jamjoom. Formal analysis was conducted by Adil Ali Saleem, Noor Ayesha, and Hafeez Ur Rehman Siddiqui. Investigation was led by Hafeez Ur Rehman Siddiqui, Adil Ali Saleem, and Shaha Al-Otaibi. Resources were provided by Amjad R Khan, Noor Ayesha and Abdallah Yousif. Writing—original draft preparation was done by Shaha Al-Otaibi, Adil Ali Saleem, and Mona M. Jamjoom, while writing—review and editing was contributed by Shaha Al-Otaibi, Amjad R Khan, and Hafeez Ur Rehman Siddiqui. Visualization was carried out by Hafeez Ur Rehman Siddiqui, Adil Ali Saleem, and Mona M. Jamjoom. Supervision was provided by Abdallah Yousif, and Amjad R Khan. Project administration was managed by Abdallah Yousif, and Amjad R Khan. All authors reviewed the manuscript.

Data availability

The dataset analyzed during the current study is openly available on Kaggle: R. Riccio, Real-time exercise recognition dataset, 2024. Available online at: https://www.kaggle.com/datasets/riccardoriccio/real-time-exercise-recognition-dataset (Accessed: Nov. 17, 2025). All data used in this study can be accessed freely under the terms provided by the dataset contributor.

Declarations

Competing interests

The authors declare no competing interests.