Real-time yoga posture correction using deep learning for individuals with physical disabilities

Abstract

The rapidly increasing global adoption of yogic practice as a comprehensive inclination towards wellness has significantly increased the need for advanced as well as effective methods to recognize yoga postures. This study investigates a solution based on deep learning to deliver real-time yoga pose correction, tailored specifically for individuals with physical disabilities. For many, access to a yoga instructor or personal guide is not feasible, making artificial intelligence a practical alternative for offering pose guidance from home. Existing research on pose estimation in yoga primarily focuses on classifying poses for people without physical impairments. In our novel approach, we use the Thunder variant of the ultra fast MoveNet model, which enables real-time pose estimation, to identify and correct posture errors for individuals with lower-body disabilities. The proposed system applies deep convolutional neural networks (CNNs) to evaluate user posture through video inputs. This study enhances accessibility in chair yoga practices for people with physical challenges by providing accurate real-time pose corrections using deep learning methods, and by curating a state-of-the-art dataset for chair yoga poses, not worked upon before.

Introduction

In today’s contemporary, rapidly evolving world, people are often consumed by their daily routines, leading to a myriad of health concerns stemming from the neglect of physical and mental well-being. Yoga, originating in ancient India, offers a holistic approach to health²¹, that encompasses mental, physical, and spiritual dimensions. Mental health, crucial at every stage of life, is often overlooked despite its global impact, affecting millions around the world. The rise of fitness apps and wearable technology has spurred a surge in yoga practice, with individuals turning to digital platforms to incorporate wellness into their busy lives. Human posture classification¹⁹ has become a vital tool in understanding behavior and interactions, with technology playing a pivotal role in facilitating yoga practice anywhere, anytime. Smartphones and laptops have become ubiquitous, with projections indicating a steady increase in users globally. This technological advancement has paved the way for innovative approaches to yoga, such as automated pose detection systems. These systems utilize deep learning-based models such as MediaPipe³ and MoveNet to analyze and correct yoga postures. Proper posture is essential during yoga, as incorrect form can lead to injuries and negate its advantages.

Figure 1 depicts the domain of our study and provides a wholesome view of present state-of-the-art practices of Real Time Yoga Pose Detection. The significance of proper postures in yoga for the physically disabled cannot be overstated, as incorrect alignment may lead to adverse effects. As awareness grows about the benefits of mindfulness practices like yoga, there is a pressing need for accessible, technology-driven solutions to support practitioners maintain correct form, from the comfort of their homes. The popularity of yoga has grown significantly, particularly during challenging periods such as the Covid-19 pandemic, emphasizing its importance in fostering resilience and enhancing well-being.

Figure 1.

Problem/ domain description.

Despite the growing adoption of AI-based yoga systems, most existing approaches are primarily designed for able-bodied individuals and fail to address the unique needs of individuals with physical disabilities. In particular, chair yoga for users with lower-body mobility limitations presents distinct challenges, such as restricted limb movement, reduced range of motion, variations in seated posture, and altered pose definitions compared to traditional yoga practices. These factors significantly affect pose estimation accuracy and limit the applicability of conventional models.

From a rehabilitation perspective, chair-based exercises have been shown to improve mobility, balance, and overall well-being in individuals with physical impairments. However, access to trained instructors remains limited, creating a gap that intelligent assistive systems can address. Therefore, this study specifically targets individuals with physical disabilities and aims to develop a robust real-time pose correction system tailored to their requirements, thereby filling an important research gap in AI-assisted rehabilitation.

One of the significant challenges in yoga practice is the limited accessibility of qualified chair yoga instructors, making personalized posture guidance expensive and inconvenient. This is where AI-based applications can bridge the gap. Yoga posture alignment for wheelchair users is not widely discussed and does not have a data set on existing implementation. It is a novel contribution of this study, in which relevant data was collected and organized to create an AI-powered yoga trainer for physically disabled people. Artificial intelligence applications offer promising avenues for personalized yoga guidance, utilizing algorithms to first detect and then correct posture errors that occur in real-time. Deep learning techniques, exemplified by models like MoveNet ¹⁵, provide accurate pose estimation, empowering users to improve their practice and reap the full rewards of yoga.

Our proposed research aims to make apt use of state-of-the-art AI methodologies to develop an automated chair yoga pose detection system. Our study provides real-time posture guidance system regardless of location or instructor availability. By integrating MoveNet’s ¹⁴ Thunder model, we seek to further improve accuracy and efficiency in detecting chair-yoga poses and correcting any irregularities while performing them. Through rigorous experimentation and analysis, we aim to demonstrate the practical effectiveness of our approach on diverse chair yoga poses, facilitating a deeper understanding of its potential impact in empowering physically disabled people.

Our research introduces a novel approach to yoga pose correction, specifically tailored to the physically challenged–a segment often overlooked in existing solutions. The following key contributions highlight the innovation and impact of our work:

• Inclusive Pose Correction System: Unlike existing yoga correction systems, our model is the first to specifically address the needs of physically challenged individuals. This expands the scope of accessible fitness and rehabilitation technologies.

• Custom Dataset Creation: Since no prior dataset exists for disabled yoga practitioners, we curated and prepared a specialized dataset exclusively for this study, ensuring relevance and precision in model training.

Figure 2 illustrates the structure of the paper along with its various subsections.

Figure 2.

Table of content and paper organization.

Literature review

Recent research emphasizes the progression of human pose estimation from two dimensional to three dimensional models, advancing from single-person to multiple-person posture analysis. Techniques for evaluating human posture are typically split into two primary groups: top-down and bottom-up approaches. The top-down approach in Kumar et. al. ¹ involves detecting key points by first identifying human subjects, then applying a posture analysis algorithm. In contrast, the Bottom-up approach detects key points from multiple potential subjects and associates them with specific body parts using various data association techniques. This study introduces an approach for automatically identifying yoga asanas from both live and pre-recorded videos. The method leverages the OpenPose model to extract key points, followed by a combination of CNN and LSTM models for pose classification. The system achieves remarkable results, with an accuracy of 99.04% on individual frames and 99.38% accuracy after polling 45 consecutive frames. Furthermore, it attains 98.92% accuracy during real-time testing with 12 participants, highlighting its ability to handle diverse individuals and conditions effectively. The system operates seamlessly using input from a standard RGB camera. Potential enhancements include adding more asanas, expanding the dataset to encompass both images and videos, and adapting the system for deployment on portable devices for real-time predictions and self-guided training.

The study by B. Jo and S. Kim compares several pose estimation models, including OpenPose, PoseNet, and MoveNet ². OpenPose is a pioneering open-source framework, developed by Cao et al. ¹¹, for real-time multiple-person pose estimation. It employs a bottom-up approach to overcome the constraints of the top-down method. The model processes 2D images or videos, producing a confidence map from the input. After detecting an object, it draws a bounding box around it and assigns a probability score to the object. From this, OpenPose computes part affinity fields (PAFs). Finally, bipartite matching is applied using confidence maps and PAFs to produce full-body pose estimates. PoseNet, developed by Google Creative Lab, adopts the Top-down approach and is another open-source model capable of real-time human pose estimation¹². Multi-person pose estimation, requires additional parameters: the maximum number of detected poses, a confidence score threshold, and a non-maximum suppression radius. These extra inputs enhance the model’s accuracy in detecting multiple individuals. MoveNet¹³ and Thunder¹⁵ differ in input size and depth multiplier, with Lightning accepting a fixed-size image (192×192) with a depth multiplier of 1.0, while Thunder uses a (256×256) input size and a 1.75 depth multiplier. Among these models, OpenPose provides the most detailed body tracking, whereas MoveNet Lightning excels in speed. OpenPose is capable of estimating poses for multiple individuals simultaneously. PoseNet, on the other hand, achieves the highest accuracy. However, the study’s limitation lies in its comparison of model performance solely based on images, without evaluating results from live camera inputs.

Figure 3 provides a summarized comparison of the various pose estimation models discussed in this survey.

Figure 3.

Comparison of pose estimation models.

The objective of a study by Dill et al.³ is to take an initial step in quantitatively evaluating the performance of MediaPipe Pose, focusing on its accuracy in calculating metrics instrumental to physical therapy while identifying its advantages and limitations. In an experiment, pose data from multiple individuals performing physical exercises were collected using two cameras and a motion capture (MoCap) system. Shoulder and hip widths were derived from MediaPipe data by calculating the Euclidean distance between shoulder and hip joints, respectively. To further enhance accuracy, a logical progression would be to integrate pose data from the two cameras, which could minimize angle dependency and mitigate the effects of self-occlusion.

Sometimes there are difficulties in maintaining accurate yoga postures. The research by Borthakur et. al.⁴ discusses about creating a system capable of identifying correct yoga poses and delivering real-time feedback. This approach holds promise for further research and demonstrates significant potential for deployment on simple smartphones, offering real-time pose evaluation to users practicing yoga at home with minimal latency.

The research by Haque et. al.⁵ employs ExNET, a Convolutional Neural Network (CNN), to identify human poses from 2D images of human exercises. ExNET is a multilayer convolutional network designed to process grayscale images with an input size of 64 × 64. Their model was tested on a custom dataset comprising 2000 images, categorized into five classes. Several experiments were conducted on the test dataset, achieving a maximum accuracy of 82.68%. However, the study identified limitations, such as the model occasionally misinterpreting certain poses due to over-fitting.

The research paper by M. Goni⁶ concludes that yoga asana is an effective method for treatment, correction and prevention of every common postural deformity. It lists out various asanas that should be performed for specific physical deformities. Asanas for deformities like Bow/Knock Knees, Flat Foot, Scoliosis, Kyphosis and Lordosis are described in this paper. Limitations of this paper are that it follows a generalized approach towards deformities, without considering intensity of a particular deformity for suggesting an asana.

Study by Madhusmita et al.⁷ examines the n impact of incorporating a Yoga program alongside physiotherapy for individuals with paraplegia. Their findings reveal that Integrated Yoga and Physiotherapy program for just a month is more beneficial in managing paraplegia patients compared to physiotherapy alone.

The study by k. Yadav⁸ examines the therapeutic effects of Yoga asanas on individuals with spine curvature disorders. The key conclusion of this review indicates that practicing yoga can help reduce pain and enhance functional abilities in patients experiencing spinal discomfort. However, the research relies on observed responses and does not establish a mathematical correlation.

The system introduced by Kale et. al.¹⁰, known as e-YogaGuru, provides recommendations for necessary adjustments in postures for each iteration of a vision-based system. These adjustments focus on: (i) the sequence of Yogasana postures, (ii) body angles within the postures, and (iii) the duration for holding the main posture. The system attained a 98.29% accuracy rate in recognizing Yogasanas and a 93.46% accuracy rate in recommending adjustments. The reduced accuracy in amendment suggestions is attributed to the absence of a standardized database, which represents the primary limitation of this research.

The paper by Goyal et. al.¹⁷ explores the use of artificial intelligence in yoga. In contemporary society, yoga is widely recognized as a popular form of exercise, creating a growing need for guidance on performing yoga poses correctly. Incorrect execution of certain postures can result in injuries and exhaustion, emphasizing the importance of having a instructor. However, since many individuals lack access to a yoga trainer, AI can serve as an alternative, offering guidance on pose accuracy. The algorithm demonstrated an average error rate of 15%, which was attributed to minor variations in individual poses. However, these variations were found not to affect the correctness of the poses significantly. Table 1 provides a summary of some of the papers discussed in this literature review.

Table 1.

Overview of Literature review.

Paper	Short Summary	Result/Conclusion
Kumar and Sinha (2020)1	Study on advancements in human pose estimation from 2D to 3D, single to multi-individual. Introduces OpenPose and CNN-LSTM models to detect Yoga asanas	For a group of 12 distinct individuals, 98.92% accuracy in real time.
Kim and Jo2	The study compared OpenPose, MoveNet, and PoseNet for pose estimation.	OpenPose excels in multi-person tracking but is slower, PoseNet achieved the highest accuracy, MoveNet Lightning model excelled in speed.
Dill et al.3	The study assessed MediaPipe Pose’s accuracy for physical therapy metrics using data from cameras and motion capture. It highlights integrating data from multiple cameras	Accuracy declines when the exercise causes self-occlusion. Suggests to merge the pose information provided by two cameras
Borthakur et al.4	The study utilized machine learning and computer vision techniques to identify correct yoga poses in real-time, employing MLKit from Google for mobile application development.	Extremely randomized trees demonstrated highest accuracy at 91%, suggesting promising avenues for real-time feedback.
Haque et al.5	Introduced ExNET, a Convolutional Neural Network, for 2D human pose detection, on their dataset of 2000 images across 5 classes.	Achieved 82.68% accuracy, with plans to address overfitting by expanding the dataset.
Goni6	Explored yoga asanas’ effectiveness for treating, correcting, and preventing common postural deformities.	Specifics of poses for conditions like Bow/Knock Knees, Flat Foot, Scoliosis, Kyphosis, and Lordosis.
Madhusmita et al.7	Assessed the combined impact of Yoga and physiotherapy on paraplegia patients,	Found that a one-month integrated program has superior efficacy compared to physiotherapy alone.
Kale et al.10	Introduces e-YogaGuru, a system suggesting amendments in posture sequence, body angles, and hold time for yoga poses.	Achieved 98.29% accuracy in Yogasana recognition. Limitation in suggestion accuracy (9.346%) due to lack of data
Goyal and Jain17	Explored the application of AI in yoga, using Thunder version of Tensorflow MoveNet for real-time pose estimation.	Reports average 15% error in minor pose variations, which does not necessarily indicate incorrect poses, offering a promising avenue for pose correction.

Saurav et al.²⁹ proposed a mix of CNN-LSTM and 3DCNN-based models for real-time yoga pose recognition. Pre-trained C3D architectures were used. Their best model achieved a recognition accuracy of 99.65% and 31 FPS performance on GPU hardware.

Similarly, Rishan et al.³⁰ used OpenPose with a deep learning pipeline of CNN and LSTM to recognize six yoga poses in real-time reporting 99.91% accuracy. These approaches achieved strong results, but they rely on compute-heavy frameworks and are focused on a few regular yoga poses. In contrast, this paper uses MoveNet Thunder for lightweight pose detection, paired with an MLP classifier to achieve real-time detection without requiring specialized hardware. Moreover, the novel training dataset in this paper focuses on yoga poses for individuals with disabilities.

While prior studies have explored pose estimation and yoga classification, limited work exists in AI-assisted rehabilitation and real-time posture correction for individuals with physical disabilities. This highlights a critical research gap that the present study aims to address.

Dataset creation

The dataset illustrates 10 classes of chair yoga poses, as shown in figure 4 The details for each classess’ data is as follows: Back_Bend : 6 videos, 88 extracted frames. Cat_Cow : 13 videos, 307 extracted frames. Chair_Warrior_1 : 6 videos, 228 extracted frames. Chair_Warrior_2 : 2 videos, 116 extracted frames. Forward_Bend : 8 videos, 216 extracted frames. Mountain_Stretch : 9 videos, 154 extracted frames. Namaste_Stretch : 9 videos, 362 extracted frames. Pigeon_Stretch : 12 videos, 679 extracted frames. Side_Stretch : 8 videos, 385 extracted frames. Spine_Stretch : 9 videos, 415 extracted frames. Examples of each class is given in figure 4. Figure 5 gives an overview of the above data diagrammatically.

Figure 4.

All classes of chair yoga poses.

Figure 5.

Dataset Labels and counts.

The process of data collection was heavily time consuming. Video data for chair yoga on the internet is scarce, scattered and noisy. The initial task involved sifting through a vast array of content to procure relevant material. Subsequently, the acquired videos underwent a series of stages including segregation, pre-processing, and augmentation to ensure the dataset’s quality and suitability for analysis. Noisy/ faulty data was dealt with using the no_pose class. The details for each step performed are given in the subsections below :

Collecting the data

The dataset was constructed using publicly available chair yoga videos sourced from online platforms such as YouTube. It is important to note that these videos may include demonstrations by both trained instructors and individuals performing chair-based exercises. The dataset represents chair yoga postures rather than clinically verified individuals with physical disabilities.

All data used in this study is publicly accessible and utilized strictly for academic and research purposes. No personally identifiable information was collected or used. This approach ensures ethical compliance while maintaining the diversity and variability required for training robust models.

Prior to the start of data parsing, specific requirements were established for the data sources. Meeting these criteria helps to guarantee a highly diverse and high-quality initial dataset:

• Data must be fully accessible and open.

• Each target class should have at least several hundred frames and 10–20 videos available.

• Images should vary in quality, angles, and lighting conditions.

Preparing the data

A code was designed which utilizes the OpenCV ¹⁸ library to extract frames from a video file and save them as images. It sets up a loop to extract frames from the video.

• It reads each frame of the video.

• It checks if the frame was read successfully.

• It constructs a file name for each frame and saves the frame as an image (both the original and horizontally flipped version).

• It sets the video capture’s position to skip ahead by a certain time (‘time_skips‘), effectively skipping some frames.

• It increments the frame counter.

This process ensures the normalization process such that all frames are the same size. Once all frames are processed, we move on to data augmentation.

Dealing with faulty data and data normalisation

Despite careful preparation, not all recorded data proves to be usable. Faulty data may not always be outliers; they can include missing information or errors in the original dataset. These issues may arise from irrelevant video recordings, incorrect timestamps, or failure to adhere to the specified protocols by the participants involved. In our experience, many of the extracted image frames were found to be faulty which should thus not be used for training and testing. While removing all faulty data might seem like the most straightforward option, we retain certain segments labeled as ’no-pose’ and tag them appropriately. These segments are valuable for refining calibration techniques for our pre-trained model. The blurred, hazy and inaccurate faulty images are manually detected and deleted. This way, faulty data is dealt with.

Methodology

MoveNet is a model designed to accurately detect human poses in videos with intense motion. Unlike traditional pose estimation models, MoveNet is specifically optimized to handle dynamic and fast-moving scenarios, making it suitable for applications such as sports analysis, dance recognition, and gesture control.

At its core, MoveNet utilizes a deep neural network design as shown in Figure 6 that integrates convolutional neural networks (CNNs) with keypoint estimation modules. The model receives video frames as input and passes them through a series of convolutional layers to derive significant spatial characteristics. These features are then used to predict the locations of key body points. MoveNet leverages temporal information across video frames to improve pose estimation accuracy. By considering the temporal coherence of poses over time, the model can better handle motion dynamics and produce smoother pose trajectories. MoveNet incorporates mechanisms to handle occlusions and partial visibility of body parts.

Figure 6.

MoveNet Architecture.

MoveNet marks a major breakthrough in pose estimation technology, delivering cutting-edge performance in accurately identifying human poses, even in videos with rapid or complex movements. Its efficient design, resilience to occlusions, and capability to manage dynamic environments make it an essential asset for numerous computer vision and human-computer interaction tasks. Complementing MoveNet, a Multilayer Perceptron (MLP) model is employed due to its adaptability, allowing for the addition of more layers and neurons to effectively handle larger datasets. MLPs leverage multiple layers with nonlinear activation functions.

Multilayer Perceptron model

For each class, frames containing yoga poses are extracted from the videos in a sequence. Pose landmarks are detected using the MoveNet Thunder SinglePose model, which outputs 17 keypoints with their respective coordinates (x, y) and confidence score which indicates the confidence of the estimation of keypoint. These frames are normalized and converted to a numpy array. The model takes numpy array as input of size [1, 256, 256, 3] and produces output array of size [1, 1, 17, 3] which represent the 17 keypoints in the body and each keypoint of size 3. Invalid images such as those not in RGB format, are rejected. The minimum landmark confidence threshold of 0.4 was selected empirically based on preliminary experiments. Lower thresholds resulted in noisy keypoints and reduced model accuracy, while higher thresholds significantly reduced the number of usable frames. A threshold of 0.4 provided an optimal trade-off between data quality and quantity, ensuring stable model performance. Frames with pose landmarks of confidence scores below the threshold are discarded, and the remaining landmarks are flattened, and stored in CSV format. Separate CSV files are created for each class, and these files are combined to create the final training and testing CSV files.

Once the preprocessing is done, we move on to the calculation of Pose Embeddings. The pose landmarks are used to calculate pose embeddings, resulting in 34 embeddings per pose. Pose center and size are calculated using TensorFlow operations. The center (0,0) is computed by averaging the coordinates of left and right body parts, while the size is calculated by the gap between hip and shoulder centers or the maximum distance from pose center to each landmark, multiplied by a multiplier (2.5). Landmarks are then normalized to a constant pose size by subtracting the pose center from every landmark and dividing by the pose size. The normalized embeddings are flattened and converted into TensorFlow tensors for further processing.

Figure 7 given above shows a detailed workflow for the model. Once we have both the preprocessed data and the pose embeddings tensors in our hands, we finally build our Multilayer Perceptron (MLP) model²⁸. As the embedding size is 34, the input layer of the MLP²⁸ model receives inputs of size 34. The model comprises hidden layers with sizes of 128 and 64, respectively, with dropout layers (dropout rate of 0.5) added to prevent overfitting. The final output layer consists of 11 neurons, and the softmax function was used since it outputs confidence levels for all labels in the dataset. Table 2 shows the layers of the model. TensorFlow.js is employed to save the model for deployment on the frontend using React.js.

Figure 7.

Process Workflow.

Table 2.

Layers of MLP model.

Layer	No. of Params	Output
Input	0	34
Dense	4480	128
Dropout	0	128
Dense	8256	64
Dropout	0	64
Output	715	11

This comprehensive methodology outlines the pipeline for preprocessing data, calculating pose embeddings, training the MLP model, and deploying it for real-time pose classification, facilitating efficient interaction with the model in a web-based application scenario. The hidden layers utilize the Rectified Linear Unit (ReLU) activation function to introduce non-linearity, while the output layer employs the Softmax activation function to generate probability distributions across classes. The choice of 128 and 64 neurons in the hidden layers was made to balance model complexity and computational efficiency, while avoiding overfitting given the dataset size. The Complete steps are depicted in algorithm 1.

Algorithm 1.

Dataset Creation and Deep Learning Model Training

Posture correction mechanism

Beyond pose classification, the system incorporates a posture correction mechanism that compares detected keypoints with reference pose templates. Deviations in joint positions are analyzed to identify incorrect posture alignment.

Based on these deviations, the system provides corrective feedback in the form of visual cues and textual suggestions. For example, if arm elevation is insufficient, the system prompts the user to raise their arms higher. This feedback mechanism enhances usability by guiding users toward proper posture execution in real-time.

Result analysis

Results for Multi-layer Perceptron model are evaluated on metrics such as accuracy, precision, recall, F1-score and ROC curve. The training procedure was carried out on NVIDIA GeForce GTX 1650 GPU using tensorflow and NVIDIA CUDA toolkit.

Performance evaluation

Training was carried out with the Adam optimizer set to its categorical cross-entropy loss function and the default learning rate. An early stopping mechanism was applied with a patience value of 20. It was trained for 200 epochs using a batch size of 16 but due to early stopping, training stops at 100th epoch as there is no significant improvement in validation accuracy. Validation accuracy is calculated by splitting the training set into validation with a size of 0.15. The experimented model was able to achieve training accuracy of 90.02%, validation accuracy of 94.29% and 92% on test data. The training accuracy vs validation accuracy plot for MLP model shown in figure 8 depicts that the model performs well on training as well as validation data.

Figure 8.

Accuracy graph for MLP model.

Figure 9 shows the precision, recall and f1-score of each class predicted by the MLP model. F1 score for most of the classes is above 90%, indicating that the model achieves a good balance between precision and recall, it effectively identifies positive cases and reduces false positives and false negatives.

Figure 9.

Class-wise Precision, Recall, F1-Score for MLP model.

Figure 10 represents the class-wise accuracy of the MLP model. 80% of the classes are above 90% accuracy of which 3 classes achieved 100% accuracy, this shows how accurately the model can classify the poses. These metrics are calculated on the test data which contains overall 622 instances. Some classes (e.g., “chair_warrior_1”, “mountain_pose”, “spine_stretch”) have bars reaching close to 1.0 (100%), indicating that the model performs very well in recognizing these poses.

Figure 10.

Class-wise Accuracy for MLP model.

Certain classes (e.g., “no_pose”, “catcow”) have lower bars, which means the model struggles to accurately predict samples from these categories. These could represent challenging poses or might be due to factors like imbalanced data (fewer samples for these classes in training).

Figure 11 depicts the training loss v/s validation loss for MLP model. It is clearly shows that the model becomes more stable as the number of epochs increases. In this case, training for a higher number of epochs leads to improved results. From the graph, it’s evident that the training loss stabilizes after around 100 epochs, making 100 epochs sufficient for effective training. Additionally, since both the training and validation loss decrease consistently over the epochs, It suggests that the model is not overfitting and is achieving good performance on the test data.

Figure 11.

Loss graph for MLP model.

Confusion matrix is used to evaluate the effectiveness of this method, which measures the model’s performance across all instances. Accuracy, also referred to as the classification score, is calculated by dividing the count of correctly predicted instances by the overall number of instances.

1

The study utilizes a confusion matrix with 11 labels, resulting in an 11 * 11 matrix for evaluating the results. The rows correspond to the actual classes, whereas the columns represent the predicted classes. Figure 12 shows the confusion matrix for the test dataset. A total of 622 instances are analyzed, and a greater concentration of values along the diagonal signifies that the majority of predictions match the true classes.

Figure 12.

Confusion Matrix.

Figure 13 represents a multi-class Receiver Operating Characteristic (ROC) curve, which evaluates the performance of a classification model across multiple classes. Most classes have ROC curves that closely hug the top-left corner, with AUC values close to or equal to 1, demonstrating high performance. The dashed diagonal line represents a baseline for random guessing (AUC = 0.5), against which the model’s performance is benchmarked. Overall, this graph shows that the classification model demonstrates strong performance for most classes, with only slight differences in effectiveness. It underscores the model’s ability to accurately distinguish between various categories, though small variations in AUC scores suggest there may be room for optimization, particularly for the classes with slightly lower performance.

Figure 13.

Multi-Class ROC Curve for MLP model.

Analysis of the confusion matrix reveals that certain misclassifications occur between the “no_pose” class and dynamic poses such as “cat_cow.” This is primarily due to transitional frames where pose structures appear similar.

Additionally, class imbalance in the dataset, where some classes have significantly fewer samples (e.g., Chair_Warrior_2), affects model performance. Classes with fewer samples tend to have slightly lower accuracy. Future work will address this issue through data augmentation and balanced sampling techniques.

Figure 14 represents a multi-class precision-recall (PR) curve is a widely used metric for evaluating multi-class classification models. It is especially valuable for imbalanced datasets, where conventional accuracy measures can be deceptive. From the graph, it is evident that some classes, such as “chair_warrior_1” and “chair_warrior_2,” have perfect precision-recall curves with AUC values of 1.00 and an F1-score of 1.00, suggesting that these classes are classified with perfect accuracy. On the other hand, some classes, like “no_pose” and “mountain_pose,” have lower AUC values around 0.79–0.85, indicating that the model struggles more with these categories.

Figure 14.

Precision-Recall-F1 Curve for MLP model.

Comparison

The method’s effectiveness was assessed by comparing its accuracy against existing approaches for detecting yoga poses in fully functional individuals. Table 3 presents a comparison with the existing approach. Maddada et al. 2019 ²³ achieved an accuracy of 87.27% using Deep CNN, Parashar et al 2023 ²² reported accuracy of 86.34% using MoveNet and without preprocessing, and again Parashar et al 2023 ²² achieved an accuracy of 89.50% using MoveNet (MediaPipe and without Preprocessing). Chasmai et al 2022 ²⁴ analyzed over three public datasets and trained a random forest classifier with an accuracy of 91.21% whereas our study achieved far better accuracy of 92% using with Movenet thunder and MLP approach.

Table 3.

Evaluation of performance against current techniques for yoga pose detection.

Author	Year	Methodology	Accuracy (%)
Maddada et al	2019	Deep CNN and JADM	87.27
Parashar et al	2023	MoveNet (Non-MediaPipe) (no Preprocessing)	86.34
Parashar et al	2023	MoveNet (MediaPipe, no Preprocessing)	89.50
Chasmai et al	2022	AlphaPose	91.21
Proposed Method	2026	Posenet	87.11
Proposed Method	2026	Mediapipe	89.97
Proposed Method	2026	MoveNet (Thunder)	92.00

It also demonstrate the effectiveness of different pose estimation models on the same dataset (*), as existing approaches executed on different datasets. Among the evaluated approaches, PoseNet achieves an accuracy of 87.11%, while MediaPipe Pose improves the performance to 89.97%, indicating the benefit of more detailed landmark detection. The proposed method using MoveNet (Thunder) achieves the highest accuracy of 92.00%, highlighting its superior performance in real-time pose classification tasks with higher inference time. This improvement can be attributed to MoveNet’s optimized architecture, which provides a better balance between speed and accuracy compared to other models. Overall, the results indicate that lightweight and efficient pose estimation models such as MoveNet are more suitable for real-time chair yoga posture correction systems, especially when combined with an MLP-based classifier.

Deployment

TensorFlow.js is employed to save the model for deployment on the frontend using React.js. The model is hosted on a free cloud service provided by Vercel Blob. The final step is Real-time Model Deployment. The trained model is deployed for real-time pose classification on a React.js frontend using TensorFlow.js. The React-Webcam library is utilized to capture live video, and the MoveNet Thunder model is applied to extract landmarks. Landmarks with a keypoint score greater than 0.4 are detected in the live video, and the embeddings are passed to the above model for classification. The application records the duration for which poses are held and displays performance metrics such as the best time achieved. An example of deployed model is shown in Figure 15.

Figure 15.

Real-time pose validation using our trained model.

To validate the real-time capability of the proposed system, inference speed was evaluated on a system equipped with an NVIDIA GTX 1650 GPU. The model achieved an average processing speed of approximately 18–22 frames per second (FPS), demonstrating its suitability for real-time applications.

The average latency per frame was observed to be under 50 milliseconds, ensuring smooth user interaction during live pose detection. These results confirm that the proposed system meets the requirements for real-time deployment in practical scenarios.

Conclusion and future work

The proposed article provides a model and an application to successfully detect the correctness of 10 classes of chair yoga poses and 1 no pose class. It is the first and only study which provides pose detection for people with physical limitations, specifically chair yoga poses. The article also proposes a model for sequential detection of yoga pose frames throughout the performance of the pose. In the future, step by step instructions for each pose can be added to this model for better assistance to the user.

Future work in this field of study could include expansion of the dataset to increase the efficacy of the model. With higher data frames, other models can be used to increase the accuracy of detection by taking into account more number of data points in the body skeleton. After expert consultations, calibrated classes can be added to cater to the type and intensity of disability. This would ensure that the difficulty of pose correction is managed according to the degree of physical limitation of an individual. Moreover, higher computational power and more pre-processing of the dataset would help in distinguishing poses that are very similar to the no pose class with a better accuracy.

Author contributions

Vansh Aggarwal contributed to dataset collection, data preprocessing, model implementation, experimentation, and manuscript drafting. Vaibhav Yadav contributed to data curation, model training, performance evaluation, and result analysis. Saloni Chaudhary assisted in dataset preparation, literature review, experimentation, and visualization of results. Gaurav Singal contributed to methodology design, technical validation, critical review of the manuscript, and overall supervision of research activities. Riti Kushwaha contributed to conceptual refinement, literature analysis, experimental design validation, and manuscript editing. Lokesh Kumar Sharma conceptualized and supervised the study, guided the research methodology, validated the results, reviewed and edited the manuscript, and served as a corresponding author.

Funding

Open access funding provided by Manipal University Jaipur. This research received no specific grant/fund from any funding agency in the public, commercial, or not-for-profit sectors.

Data Availability

The processed keypoint dataset and implementation details will be made publicly available to support reproducibility. Source code and trained models will be shared via a GitHub repository upon publication.

Declarations

Consent for Use of Image/Video

The image used in this study depicts one of the authors and was recorded solely for demonstration purposes to illustrate the working of the proposed system. The image has been anonymized to ensure that no identifiable personal information is visible. The individual appearing in the image has provided consent for the use and publication of the image in this manuscript.

Human Subjects and Ethics Compliance

No external human participants were involved in this work. This demonstration does not constitute human subjects research according to the guidelines of our institution. Therefore, ethics committee/IRB approval was not required, and the study was considered exempt by the EC/IRB. All procedures were performed in accordance with institutional regulations and applicable guidelines.