A hybrid deep learning model with feature engineering technique to enhance teacher emotional support on students’ engagement for sustainable education

Abstract

Understanding students’ emotional conditions throughout the learning process is a significant feature of enhancing learning quality. In an academic setting, an extent of emotion is achieved physically or automatically by utilizing a computer. Emotions like curiosity, hope, interest, confusion, enjoyment, anger, pride, shame, frustration, anxiety, and boredom often arise throughout the learning procedure. In educational surroundings, emotions experienced have a robust relationship with a student’s academic attainment and personal development. However, developing an emotion detection model utilizing a harmless, contactless, and illumination-independent imaging modality is very challenging. Recently, the arrival of Artificial Intelligence (AI) and deep learning (DL) has opened up novel possibilities for tackling these tasks by automating the procedure of student emotion recognition through facial expression study. The DL-based techniques are utilized to improve the teacher’s emotional support for students’ engagement for sustainable education. This paper presents a Hybrid Deep Learning Model with Feature Engineering to Enhance Teacher Emotional Support on Students’ Engagement (HDLMFE-ETESSE) model for sustainable education. The aim is to progress an effective student emotion recognition to enhance student engagement and learning outcomes for sustainable education. Initially, the image pre-processing stage applies face normalization and facial alignment methods to improve image quality. Furthermore, the ptive separable convolutionX (AdaptSepCX) attention network system is utilized for feature extraction to identify and isolate the most relevant features from raw data. Finally, the hybrid of a convolutional neural network and a bidirectional gated recurrent unit (C-BiG) models is employed for the classification process. The experimental analysis of the HDLMFE-ETESSE approach is examined under the student-engagement dataset. The comparison study of the HDLMFE-ETESSE approach portrayed a superior accuracy value of 98.58% over existing models.

Introduction

The classroom is a place where students experience several kinds of emotions while doing actions, namely taking exams, finalizing projects, and constructing social relationships. Emotions, including curiosity, pleasure, hope, interest, anger, arrogance, shame, worry, frustration, confusion, and dullness, often arise in the learning procedure¹. Emotions experienced in educational institutions are closely linked to students’ academic performance and individual development. Experiencing positive emotions, like pleasure, while working on class projects can assist pupils in envisioning objectives, enhance inspiration and problem-solving, and aid self-regulation². In contrast, experiencing negative emotions, like worry, can impede educational achievement and harmfully impact mental as well as physical health. In education, the significance of emotions also applies to teachers, administrators, and authorities. Emotions encompass a set of mental processes, such as cognitive, emotional, motivational, physiological, and expressive modules³. Emotions are denoted, and humans can communicate their emotions through verbal language. Self-report is broadly employed as a technique to assess academic emotions. In the fast-changing domain of education, recognizing and addressing students’ emotions has become highly significant⁴. Students’ emotions typically play a vital part in learning, affecting motivation, focus, and overall mental processing. In conventional classroom environments, educators frequently depend on verbal and visual cues to assess students’ emotional states. Yet, in digital and remote learning contexts, in which face-to-face teacher-student communication is constrained, this task becomes far more difficult⁵. Emotion recognition based on facial expressions holds an essential place in current domains of artificial intelligence (AI) and computer vision (CV). Though emotions are identified and examined through wearable devices and are more significantly suitable to identify feelings in practical conditions, along with the intricacy and expandability of modern CV models, supported by powerful hardware to manage a large amount of data instantly, has encouraged the growth and use of AI-based facial emotion recognition⁶.

The rise of machine learning (ML) has introduced novel opportunities to address these issues by automating the emotion recognition process through facial expression examination. Emotion detection methods employ deep learning (DL) approaches to examine facial expressions and correctly assign them to various emotion types⁷. This method has the potential to reshape educational methods by allowing practical examination of student emotions, helping teachers to modify their teaching styles to suit the emotional and mental requirements of their students⁸. Current progress in DL, specifically Convolutional Neural Network (CNN), has drastically enhanced the precision of emotion recognition methods. CNNs are especially effective because they are strong at recognizing spatial patterns in visuals, which is vital for spotting slight changes in facial expressions⁹. Research has shown that CNN-empowered approaches can reach superior precision in emotion recognition tasks, often outperforming conventional ML models. Still, the issue lies in creating systems that are not only precise but also reliable across different groups and flexible to various educational settings¹⁰. Recent advancements in large language models (LLMs) such as ChatGPT have paved the way for better comprehension and emotional dynamics in the education sector. Educators can better interpret and respond to the psychological states of the studies by using these language models. Integrating virtual reality (VR) also presents immersive experiences that improve student engagement and personalized learning environments. These tools are motivating the utilization of hybrid DL techniques with a feature engineering model for enhancing emotional assistance, ultimately fostering sustainable education and student engagement.

This paper presents a Hybrid Deep Learning Model with Feature Engineering to Enhance Teacher Emotional Support on Students’ Engagement (HDLMFE-ETESSE) model for sustainable education. The aim is to progress an effective student emotion recognition to enhance student engagement and learning outcomes for sustainable education. Initially, the image pre-processing stage applies face normalization and facial alignment methods to improve image quality. Furthermore, the ptive separable convolutionX (AdaptSepCX) attention network system is utilized for feature extraction to identify and isolate the most relevant features from raw data. Finally, the hybrid of a convolutional neural network and a bidirectional gated recurrent unit (C-BiG) model is employed for the classification process. The experimental analysis of the HDLMFE-ETESSE approach is examined under the student-engagement dataset.

• The HDLMFE-ETESSE technique implemented facial alignment and normalization techniques for improving the quality of the images during the pre-processing phase, thus ensuring better consistency across samples. The mitigated variability caused by lighting differences ultimately assists in more accurate feature extraction and classification.

• The HDLMFE-ETESSE methodology utilized the AdaptSepCX attention network for selectively extracting and isolating the most relevant features from raw data, thus improving the ability of the model in concentrating on significant facial cues while reducing noise, thereby enhancing the robustness and accuracy of the recognition process.

• The HDLMFE-ETESSE model employs a hybrid model by incorporating the C-BiG approach for capturing both spatial and temporal patterns in the data, thus improving the accuracy of the classification process by effectively analyzing facial features and their sequential dependencies for more reliable emotion recognition.

• The novel integration of the AdaptSepCX attention network with a hybrid C-BiG model presents a robust framework that significantly enhances both feature extraction and temporal classification. This unique integration allows for accurate detection of crucial facial features, while effectually capturing their sequential relationships over time, resulting in improved robustness and accuracy in facial emotion recognition tasks.

The structure of the article is as follows: Section Review of literature for sustainable education provides a review of the literature, Section Algorithm and system design describes the proposed method, Section Model performance analysis presents the evaluation of results, and Section Conclusion offers the study’s conclusions.

Review of literature for sustainable education

In¹¹, a feature fusion-enabled hierarchical cross-modal spatial fusion network (HCSFNet) is introduced, which efficiently combines EEG and video features. By developing an EEG feature extraction system that relies on 1D convolution and a video feature extraction network that relies on 3D convolution, matching modality features are thoroughly extracted. To facilitate effective interaction between the dual modalities, a hierarchical cross-modal coordinated attention model is presented in this study. Moreover, to improve the network’s perceptual capability for emotion-related features, a multi-scale spatial pyramid pooling model is also created. Zhang and Chi¹² examined the relationship between modest teacher and student involvement in the creative procedure, based on Self-Determination Theory and Social Exchange Theory. Furthermore, it examines the step-by-step mediating functions of student trust and mental empowerment, along with active personality. Xie et al.¹³ introduced the MSC-Trans Engagement Detecting Network, a temporal multimodal data fusion architecture, which merges CNNs and a multi-layer encoder-decoder structure. This architecture comprises dual vital components: A multi-label classifier depending on Transformer and ResNet that inserts labels into image features extracted by the CNN for higher-accuracy classification by background inference, and a temporal feature fusion model that employs an encoder-decoder structure to combine multimodal features, allowing steady tracking of classroom engagement. Yun et al.¹⁴ proposed a teacher-leading Multimodal fusion network for ERC (TelME). TelME integrates cross-modal knowledge concentration to pass data from a language model, thus enhancing the effectiveness of weaker modalities. Next, merge multimodal features utilizing a shifting fusion method where student systems support the teacher. Multimodal ERC has constantly been viewed as a difficult task. Wei et al.¹⁵ proposed a new hierarchical cross-modal interaction and fusion network optimized by self-distillation (HCIFN-SD) for ERC. Particularly, it initially presents three diverse mask tactics to extract speaker-dependent cross-modal conversational context depending on the improved GRU model. Following that, the graph-attention-based multimodal fusion (MF-GAT) method creates three directed graphs to represent various modality spaces, executes detailed cross-modal interactions to propagate conversational contextual information, and develops a novel GNN layer for solving over-smoothing. In¹⁶, a new method for FER employing a homogeneous ensemble CNN, termed HoE-CNN, is suggested for online learning teaching. FER is challenging to study due to the presence of multiple use cases to account for, namely, education, adaptive user interfaces, robot integration, and games. HoE-CNN is utilized for improving classification performance and encompasses seven main multiclasses. Dong et al.¹⁷ proposed an AI technique to enhance teaching efficiency in art courses by a classroom behaviour analysis method. By leveraging image acquisition techniques and analytical software, this proposed technique allows real-time data extraction and thorough examination of students’ emotional states, thereby offering functional understandings to teachers for personalized pedagogical results. The designed emotion recognition approach, ERAM, provides a quick response rate and positions it as a strong tool for immediate feedback and adaptive teaching tactics.

Liu et al.¹⁸ recommended EmotionKD, a structure for cross-modal knowledge distillation, which instantaneously models interactivity as well as heterogeneity of GSR and EEG signals. Then, completely fused multimodal features are shifted to a unimodal GSR paradigm to enhance performance. Moreover, an adaptive feedback mechanism was introduced to allow a multimodal approach to be modified based on the unimodal model’s performance in knowledge distillation. Mazhar et al.¹⁹ proposed a model by utilizing DL and ML models, comprising deep neural networks (DNNs), 1D CNN, and bidirectional long short-term memory (BiLSTM) for precisely detecting student concentration levels from brain-computer interface (BCI) data. Al Shloul et al.²⁰ evaluated the efficiency of activity-based learning (ABL) compared to conventional teaching methods. Furthermore, the ChatGPT tool is also investigated to enhance personalized feedback. Mazhar et al.²¹ presented a model by incorporating CNN, random forest (RF), and extreme gradient boosting (XGBoost) for precisely anticipating student academic performance. Shah et al.²² utilized a comprehensive literature review and theoretical analysis to examine the integration of VR with established educational theories. Liang and Liang²³ developed a multi-attribute user evaluation model using DL (DL) methods. The model also used visualization tools such as heatmaps and diverse inputs. Hussain et al.²⁴ proposed a model by utilizing multi-layer topic modelling integrated with the Felder–Silverman learning style model (FSLSM) and fuzzy logic (FL) models to detect learners’ styles automatically. The method also utilizes web usage mining and DL. Furthermore, the latent Dirichlet al.location (LDA) technique is also implemented. Huang and Samonte²⁵ analyzed the behavioural engagement and emotional dynamics of non-English learners by employing a cross-modal adversarial learning framework that incorporates transformer, CNN, and long short-term memory (LSTM) models. The approach also improves the temporal dependencies and multimodal feature extraction. Mutawa and Sruthi²⁶ introduced a model to anticipate student satisfaction and emotional states in online education by using an ML model comprising an artificial neural network (ANN), RF, and backwards feature selection (BFS) techniques for providing personalized and adaptive learning pathways. Kannan, Dewan, and Murshed²⁷ introduced a model by utilizing hybrid DL methods such as CNN-BiLSTM, ANN-BiLSTM, and bidirectional encoder representations from transformers–BiLSTM (BERT-BiLSTM) technique. The approach integrates Interactive, Constructive, Active, and Passive (ICAP) and LDA methods for topic modelling. Shiri et al.²⁸ proposed a hybrid DL technique by incorporating EfficientNetV2-L with temporal models such as gated recurrent unit (GRU), Bidirectional GRU (Bi-GRU), LSTM, and Bi-LSTM models for classification and detection. Comparison analysis of existing Adoption of emotion recognition for sustainable education in Table 1.

Table 1.

A comparative study of diverse advanced models.

Author	Year	Objective	Method	Dataset	Merits, Demerits, and Findings
Xu et al.11	2025	Present self-distillation to multimodal emotion recognition depends on EEG and Video data, thus necessitating some resources and enhancing greater performance.	PMNet, VMNet, HCSFNet.	DEAP and MAHNOB-HCI datasets	Shows efficiency in modality fusion and attention, but real-time usage is affected by the model’s complexity; Accuracy of 97.78%, 60.59% on DEAP and MAHNOB-HCI datasets.
Zhang and Chi12	2025	To explore how humble teacher leadership influences student creative engagement.	Longitudinal Survey, PLS-SEM Analysis, SmartPLS 4.0	Survey dataset collected from 384 Chinese undergraduates	Merits include clear theoretical grounding with limitations on generalizability and self-reported bias; Positive leadership impact.
Xie et al.13	2025	To identify engagement utilizing video information, they usually depend on fixed feature integrations that fail to acquire the logical connection and temporal dynamics of the arrangement.	CNN	DAiSEE dataset	Temporal dynamics and feature selection are effectively captured, but may involve high computational cost; Achieved advanced engagement detection.
Yun et al.14	2024	Projected a model to deliberate diverse assistances of text and non-verbal modality to emotion detection for enhanced prediction. To improve the efficacy of weak non-verbal modality for the ERC challenges over cross-modal distillation.	Knowledge Distillation, Attention-based Modality Shifting Fusion	IEMOCAP and MELD	Weak models are strengthened, but find difficulty with intrinsic modality alignment; Accuracy of 70.48 and 67.37 on IEMOCAP and MELD datasets.
Wei et al.15	2024	Present an innovative multimodal structure for conversational emotion recognition and effectually combines a hierarchic cross-modal communication and fusion system with self-distillation to learn high-quality multimodal contextual models.	HCIFN-SD, GRU, MF-GAT, GNN	IEMOCAP and MELD	Rich cross-model context is efficiently captured, but scalability is affected; WAF1 of 73.42%, 64.71%, and WAA of 73.07%, 64.34% on IEMOCAP and MELD datasets.
Lawpanom et al.16	2024	Proposes a novel model for FER that employs a similar ensemble CNN to increase emotion recognition precision.	CNN, DCNN, EfficientNetB2, InceptionRestnetV2, Restnet50, Xception, DenseNet, VGG16	FER dataset	FER accuracy is improved, but encounters threats in real-time deployment; Accuracy of 75.51%.
Dong et al.17	2023	Introduce the employment of AI technology to enhance the teaching efficacy of the art course over behaviour identification models.	Emotional Recognition Model	FERPlus, CK+, ExpW datasets	Real-time emotion insights are enabled, but rely heavily on accurate image data. The accuracy is 11.06%.
Liu et al.18	2023	Introduce an innovative multimodal EmotionNet-Teacher that depends on a dual-stream transformer framework with an IMF module and intends an adaptive feedback model for cross-modal knowledge distillation.	IMF, Adaptive Feedback Knowledge Distillation	DEAP, HCI-Tagging datasets	Unimodal GSR performance is improved, but it depends on signal types. The accuracy of Arousal is 62.88% and 60.66% in the DEAP and HCI-Tagging datasets, respectively.
Mazhar et al.19	2025	To evaluate and compare DL and ML methods to detect students’ concentration levels.	DNN, BiLSTM, 1D CNN	Student brain signal data from Kaggle	Exhibits high accuracy, performance, and reduced ReLU and BiLSTM accuracy; DNN best (99.9%), 1D CNN stable (95%).
Al Shloul et al.20	2024	The goal is to evaluate the impact of activity-based learning and ChatGPT on student performance.	Comparative Analysis, Surveys and Trials	Student groups using activity-based learning and conventional methods	Show efficiency in personalized feedback, but it isn’t very objective; Improved achievement, interactive learning.
Mazhar et al.21	2025	To use predictive analytics and a hybrid ML to improve student achievement.	Hybrid CNN & RF, XGBoost	24,005 student records from Wollo University and Kombolcha Institute (2017–2022)	Depict high accuracy, but require extensive data and complex architecture; Improved prediction, key factors identified.
Shah et al.22	2024	To evaluate the integration of VR to improve learning experiences in higher education.	Literature Review, Synthesis of Studies, Theoretical Analysis	103 Research Works	Illustrates immersive, engaging learning, but shows constraint in cost and technical limit; Shows improved understanding and retention.
Liang and Liang23	2025	To develop a DL-based multi-attribute evaluation model.	Data Collection & Preparation, Model Construction, Visualisation Integration, Deployment & Refinement	Student physical, cognitive, emotional, and social activity data	Effectually scalable, comprehensive, but is sensor reliant and has high Computational load; High accuracy, adaptive
Hussain et al.24	2024	To automatically detect learners’ styles to improve personalized adaptive e-learning experiences.	Multi-Layer Topic Modelling, FSLSM, FL, Web Usage Mining, DL, LDA	Unlabeled student feedback and e-learning platform behaviour data	Precise personalized content and style detection, but may need intrinsic modelling and an initial questionnaire; Improved personalization, higher accuracy.
Huang and Samonte25	2025	To develop a cross-modal adversarial learning framework.	Transformer-CNN-LSTM, Adversarial Training Mechanism	Multimodal classroom engagement data (emotion & behaviour)	Shows high accuracy and prediction, but has restricted generalizability; 90% accuracy, 0.96 emotion recognition
Mutawa and Sruthi26	2024	To anticipate student satisfaction and emotions in online learning.	BFS, ANN, RF	Online survey with structured data (55 features reduced to 7 and 36)	Precise in prediction and efficiency, but limited to structured data and accuracy; Satisfaction: 81%, Emotion: 65.3%
Kannan, Dewan, and Murshed27	2025	The goal is to classify students’ discussion posts into cognitive engagement levels to improve online learning efficiency.	CNN-BiLSTM, ANN-BiLSTM, BERT-BiLSTM, BERT Embeddings, LDA	Online forum posts from educational platforms	Has high classification accuracy and engagement levels, but is computationally intensive and complex; Improved engagement, cognitive insights
Shiri et al.28	2024	To develop and evaluate hybrid DL models for automatic detection of student engagement levels.	EfficientNetV2-L Backbone, GRU, Bi-GRU, LSTM, Bi-LSTM, End-to-End Deep Models, Spatio-Temporal Classification	DAiSEE dataset	Excellence in detection and modelling, but it has a moderate accuracy range and generalizability; LSTM is most effective (62.11%)

While the existing studies exhibit advancement in multimodal fusion and emotion recognition models, several limitations remain in the studies. Several techniques concentrate on intrinsic models but mostly lack generalization across various real-world educational settings. The broad applicability is also affected by the dependence on specific datasets or modalities such as EEG and GSR. Furthermore, some methods address temporal dynamics and cross-modal interactions concurrently, which is significant for precise emotion recognition and engagement. The lightweight models have not been thoroughly explored, which may be suitable for real-time classroom deployment. Limitations also include a heavy dependence on specific datasets or modalities like EEG and GSR, which may mitigate generalizability. Also, while few techniques handle temporal dynamics and cross-modal interactions, lightweight models are not adequately studies, thus restricting practical applicability. The studies also emphasize the requirement for a more adaptable, efficient framework that can incorporate diverse modalities and capture temporal-spatial features effectively in varied educational environments.

Algorithm and system design

In this manuscript, the HDLMFE-ETESSE model is presented for sustainable education. The main intention of the HDLMFE-ETESSE technique is to develop an effective student emotion recognition to enhance student engagement and learning outcomes for sustainable education. It encompasses pre-processing, feature extraction, and hybrid classification. Figure 1 represents the complete workflow of the HDLMFE-ETESSE technique.

Fig. 1.

Complete workflow of the HDLMFE-ETESSE method.

Image pre-processing technique

Initially, the image pre-processing stage applies face normalization and facial alignment methods to improve image quality²⁹. Data pre-processing is essential to guarantee that the images provided are optimized, regularised, and aligned for useful training. The pre-processing is the initial stage, which is carried out utilizing important point detection methods. This process utilizes a regression model to approximate the coordinates of particular features. According to these coordinates, an affine space is used that aligns and rotates all images, as if the main points are reliable in the database. It is mathematically described by the transformation matrix T that fine-tunes the (x, y) coordinates of all images based on Eq. (1):

1

Here, represents associated coordinates, means the matrix of the affine, and embodies the new image coordinates.

The next stage in the pre-processing procedure is normalizing the pixel values. Meanwhile, the work is paramount with input values; the values of pixel intensity are measured between − 1 and 1, utilizing Eq. (2):

2

Whereas, denote the value of pixel intensity within the new image, and represent the minimal and maximal promising values (usually 0 and 255 for 8-bit images). This re-scaling guarantees that the discriminator and generator processing images increase training stability.

Another essential characteristic of pre-processing is lighting modification. In most cases, differences in lighting settings among images can present biases in learning attributes. To alleviate this issue, an illumination standardization approach was used that fine-tunes pixel intensity based on the image standard deviation and mean. The aim is to uniformize the lighting distribution, guaranteeing that modifications in expression are unaffected by variabilities in background or shadow lighting. The brightness correcti

3

on is expressed as shown:

Whereas denotes the fitted image, refers to pixel mean values within the images, and denotes the standard deviation.

Each of the images is measured to the even resolution of x pixels, utilizing a bicubic interpolation model. Pre-processing certifies that the images are fine-tuned, associated, and standardized in illumination and scale. It increases the data quality, improves convergence time, and enhances the method’s capabilities.

Feature extraction using AdaptSepCX attention network system

Furthermore, the feature extraction process is performed by the AdaptSepCX attention network system to identify and isolate the most relevant features from raw data³⁰. This model is chosen for its capability in effectively isolating and emphasizing the most pertinent factors from raw data, thus enhancing the focus and mitigating the noise interference. The technique effectually maintains high representational power and also employs adaptive separable convolutions, which improve computational efficiency. The attention mechanism (AM) model also allows dynamic weighting of feature maps, capturing fine-grained spatial details significant for tasks like facial emotion recognition. This methodology also presents a balanced trade-off between accuracy and speed, compared to standard CNNs or basic attention models, thus making the model more suitable for real-time applications and noisy environments. Figure 2 illustrates the structure of the AdaptSepCX attention network model.

Fig. 2.

Architecture of the AdaptSepCX attention network system.

The AdaptSepCX method with scalar dot-product attention is organized hierarchically and progressively, tailored to remove more and more composite features from input images. This approach starts with a first convolutional layer using the activation of GELU that decreases spatial sizes, while removing lower-level features. Next, another convolutional layer using the activation of GELU is combined. After this, 4 Blocks of Progressive Feature Refinement (PFR) are applied. Every block of PFR contains 2 Separable Convolution blocks, combining dual layers of consecutive SeparableConv2D with dropout, batch normalization (BN), Skip Connection, and GELU activation. Afterwards, the block of PFR, the Layer of Global Average Pooling (GAP) was applied to reduce the spatial size to lxl. This method also consists of the scalar dot-product AM, which routes the GAP feature maps for calculating an attention score and scaling the feature maps accordingly. At last, the dense layer output uses the activation function of softmax for classification. Here’s a detailed explanation of the AdaptSepCX Attention structure, separated into individual blocks:

Input layer: It includes images with magnitudes of pixels and three colour channels signifying RGB.

Initial Convolution layer: This method begins with this layer, intended to handle input images described by the shape (224,224,3). It applies a Conv2D process with 32 filters, using the Gaussian Error Linear Unit (GELU) activation function, with a kernel dimension of and a stride of . Furthermore, another layer of Conv2D with an activation function of GELU and 64 filters, with kernel dimensions of (3, 3) and a stride of (2, 2), is combined.

The Conv2D layer’s mathematical model, accompanied by the activation of GELU, was expressed as a composition of functions. Assume denotes an input tensor, represents a weight tensor, and represents a bias vector. The GELU activation function and Conv2D are described as explained in Eqs. (4) and (5).

4

Now, signifies the convolution process, and characterizes an element-wise activation function. denotes the outcome of the operation of.

5

PFR block: The method then progresses over the series of 4 SeparableConv Blocks. It is an essential portion of the technique and includes custom cells that gradually improve features. Every block includes dual SeparableConv layers with x kernels. It is accompanied by sequential application of a dropout layer, BN, activation of GELU, an optional Tensor adjustment stage, and lastly, skip connections for improved model connectivity. BN and the activation function of GELU are utilized after all convolutions to guarantee steady training. Fine-tuning the Residual Tensor (IR) stage ensures that the filter counts in the IR equal the filter counts from the 2nd SeparableConv. If they don’t match, a x SeparableConv was used to IR to fine-tune the filter counts. The block’s last output is gained by totalling the output of the 2nd SeparableConv and the familiar IR.

6

Now, indicates an outcome of SeparableConv at location shows the input image’s pixel value at position and indicates the horizontal and vertical kernel values at position and and signifies the function of Kronecker delta that is 1 if and or else.

GAP layer: Following the 4 PFR blocks, a Layer of GAP is presented. It carries out the GAP from the preceding blocks. The model decreases the spatial sizes of feature mapping to x. This stage organizes the output of the model by providing a brief description of the learned feature.

AM: Here, a progressive scalar dot-product AM was applied to allocate weights to the feature vector made by GAP. The succeeding section explains the functioning mechanism: Scalar Dot Product Calculation. Afterwards, the GAP process, a lambda layer was applied to calculate several values beside the channel axis, successfully generating a scalar for every sample within the batch. Now, signifies the feature channel counts, and embodies the channel value in the GAP feature tensor.

7

Softmax activation for attention weight: The calculated attention scores are converted using the activation function of softmax, providing attention weights as exemplified in Eq. (8) that sum to 1. These weights successfully indicate the significance assigned to every individual feature channel in the model.

8

Element-wise multiplication with feature vector: Finally, the attention weights were used element‐wise from the GAP layer, as exemplified in Eq. (9). This task fine-tunes the significance of all channels according to the calculated scores of attentions. It successfully attenuates or amplifies the support of every channel’s information in the final representation of the feature.

9

Therefore, this progressive scalar dot-product AM calculates attention scores by computing feature values through networks, using an activation function of softmax to get attention weights, and then uses these weights to fine-tune the feature vector. The AM permits the method to concentrate on the related data inside the feature vector and dynamically evaluate the contribution of dissimilar networks throughout inference.

Output layer: It concludes with two dense layers, one with 64 units with GELU and the other using 10 units with the activation function of softmax. The final layer of dense is essential and is assigned to creating the predictions of the model. It creates likelihood distributions above the stated amount of output classes. By utilizing the features, an output layer implements the real classification according to the input data. Therefore, the presented model structure was ordered into blocks; all custom PFR blocks consist of skip connections and separable convolution blocks. It gradually removed and improved features.

C-BiG-based hybrid classification process

Finally, the hybrid of the C-BiG model is used for the classification process³¹. This integrated model efficiently captures temporal dependencies and hence is suitable for sequential data like facial emotion sequences. The hybrid methodology also utilizes temporal context and spatial patterns for enhancing the accuracy of the classification in dynamic environments. This technique effectually handles long-term dependencies and reduces vanishing gradient problems, thus highlighting its merit over conventional RNNs. The BiGRU model is also computationally more effective compared to other recurrent models, such as LSTM. This incorporation results in a robust classifier that outperforms single-method models, specifically in intrinsic, time-dependent recognition tasks, justifying its selection over other techniques.

CNN is applied to remove spatial features from the input data, and Bi-GRU is used to capture the dynamic information of the feature sequences. The working principle of the convolution operation in the CNN of the method is signified by Eq. (10).

10

Whereas, and characterize the feature maps of the and - layers, correspondingly. denotes the activation function, and represent bias weights of the convolutional kernel of the layer, represents the input feature mapping counts, and specifies the convolution process.

The significant elements of the GRU network are the reset gate and the update gate, as described in Eq. (11). The update gate is assigned to determine the extent to which data from the preceding must be distributed to the following state. The reset gate is applied to select how much previous information must be discarded. The Bi-GRU train the network over dual parallel forward and backwards GRU channels, combining the features from either channel as the output. This 2-channel model allows the method to more successfully seize the temporal dependency relationship in the data, thus maximizing the use of data information.

11

Now, , and , correspondingly, characterize the present state variable, the state variable at the preceding time-step, and at the present time-step; and indicate the input gate, update gate, and reset gate, correspondingly; and signify weighted matrices for the candidate output, update gate, and reset gate; andcharacterize activation functions; and signifies the progression of dual vectors. Algorithm 1 specifies the C-BiG model.

Algorithm 1.

C-BiG technique.

Model performance analysis

The performance study of the HDLMFE-ETESSE model is examined under the student-engagement dataset³². This dataset consists of two main classes. The engaged class has 1076 images with three different subclasses: confused − 369 images, engaged − 347 images, and frustrated − 360 images. The not engaged class has 1044 images with three dissimilar sub-classes, namely looking away − 423 images, bored − 358 images, and drowsy − 263 images.

Figure 3 exhibits the confusion matrices formed by the HDLMFE-ETESSE approach on 80:20 and 70:30 of TRPHE/TSPHE. The results denote that the HDLMFE-ETESSE method efficaciously detects and classifies each class.

Fig. 3.

Confusion matrices of (a–c) TRPHE of 80% and 70% and (b–d) TSPHE of 20% and 30%.

Table 2; Fig. 4 demonstrate the classifier outcome of the HDLMFE-ETESSE technique on 80:20. On 80% TRPHE, the HDLMFE-ETESSE model obtains an average of 98.39%, of 95.19%, of 95.09%, of 95.12%, of 94.16%, and Kappa of 94.23%. Similarly, at 20% TSPHE, the HDLMFE-ETESSE model obtains an average of 98.58%, of 95.66%, of 95.70%, of 95.64%, of 94.82%, and Kappa of 94.88%.

Table 2.

Classifier outcome of HDLMFE-ETESSE model under 80%:20%.

Class Labels	Accuy	Precn	Recal	F1Score	MCC	Kappa
TRPHE (80%)
Confused	98.70	95.96	96.61	96.28	95.50	95.58
Engaged	98.88	96.10	97.13	96.61	95.94	96.00
Frustrated	98.41	95.76	94.76	95.25	94.30	94.36
Looking Away	97.76	92.49	96.39	94.40	93.03	93.09
Bored	98.11	97.16	91.95	94.48	93.40	93.48
Drowsy	98.47	93.69	93.69	93.69	92.82	92.87
Average	98.39	95.19	95.09	95.12	94.16	94.23
TSPHE (20%)
Confused	97.88	94.52	93.24	93.88	92.60	92.68
Engaged	98.82	95.65	97.06	96.35	95.65	95.71
Frustrated	98.82	96.00	97.30	96.64	95.93	95.99
Looking Away	98.82	97.78	96.70	97.24	96.49	96.56
Bored	98.58	98.21	91.67	94.83	94.08	94.16
Drowsy	98.58	91.80	98.25	94.92	94.17	94.22
Average	98.58	95.66	95.70	95.64	94.82	94.88

Fig. 4.

Average values of the HDLMFE-ETESSE model under 80:20.

Figure 5 exemplifies the training (TRAIN) and validation (VALID) of the HDLMFE-ETESSE method under 80:20 over 25 epochs. At first, both TRAIN and VALID rise fast, representing an effective pattern learning from the data. Around the epoch, the VALID exceeds the training accuracy slightly, implying good generalization without over-fitting. As training advances, it reflects maximum performance and a minimum performance gap between TRAIN and VALID. The close alignment of both curves in training suggests that the method is well-regularised and generalized. This shows the method’s stronger capability in learning and retaining valuable features across both seen and unseen data.

Fig. 5.

curve of HDLMFE-ETESSE technique under 80:20.

Figure 6 depicts the TRAIN and VALID losses of the HDLMFE-ETESSE approach under 80:20 over 25 epochs. Initially, both TRAIN and VALID losses are higher, denoting that the approach begins with a partial understanding of the data. As training evolves, both losses persistently reduce, demonstrating that the approach is efficiently learning and enhancing its parameters. The close alignment between the TRAIN and VALID loss curves during training implies that the approach has not over-fitted and retains good generalization to unseen data. This persistent and steady reduction in loss shows a reliable, well-trained, and stable deep learning model.

Fig. 6.

Loss curve of HDLMFE-ETESSE technique under 80:20.

Table 3; Fig. 7 present the classifier outcome of the HDLMFE-ETESSE technique at 70:30. Under 70% TRPHE, the HDLMFE-ETESSE model attains an average of 97.30%, of 91.96%, of 91.50%, of 91.67%, of 90.09%, and Kappa of 90.16%. Likewise, at 30% TSPHE, the HDLMFE-ETESSE model accomplishes an average of 97.33%, of 91.56%, of 91.23%, of 91.33%, of 89.78%, and Kappa of 89.84%.

Table 3.

Classifier outcome of HDLMFE-ETESSE approach under 70%:30%.

Class Labels	Accuy	Precn	Recal	F1Score	MCC	Kappa
TRPHE (70%)
Confused	96.63	90.62	89.92	90.27	88.24	88.31
Engaged	97.04	89.69	90.50	90.09	88.35	88.42
Frustrated	97.57	90.49	95.58	92.97	91.55	91.61
Looking Away	97.98	93.89	96.37	95.11	93.85	93.93
Bored	97.24	92.13	91.76	91.94	90.28	90.36
Drowsy	97.37	94.92	84.85	89.60	88.28	88.34
Average	97.30	91.96	91.50	91.67	90.09	90.16
TSPHE (30%)
Confused	97.33	95.19	89.19	92.09	90.56	90.61
Engaged	97.01	93.50	91.27	92.37	90.52	90.57
Frustrated	97.17	87.80	97.30	92.31	90.76	90.81
Looking Away	98.11	94.26	95.83	95.04	93.88	93.96
Bored	97.64	93.14	92.23	92.68	91.28	91.36
Drowsy	96.70	85.48	81.54	83.46	81.66	81.72
Average	97.33	91.56	91.23	91.33	89.78	89.84

Fig. 7.

Average values of HDLMFE-ETESSE approach under 70%:30%.

Figure 8 portrays the TRAIN and VALID of an HDLMFE-ETESSE model at 70:30 over 25 epochs. In the beginning, both TRAIN and VALID rise rapidly, specifying efficient pattern learning from the data. Around the epoch, the VALID slightly exceeds the training accuracy, indicating good generalization without over-fitting. As training progresses, it displays higher performance and a lower performance gap between TRAIN and VALID. The close alignment of both curves during training implies that the model is well-regularised and well-generalized. This reflects the model’s strong ability to learn and retain beneficial features across both seen and unseen data.

Fig. 8.

curve of HDLMFE-ETESSE model under 70:30.

Figure 9 illustrates the TRAIN and VALID losses of the HDLMFE-ETESSE approach at 70:30 over 25 epochs. Initially, both TRAIN and VALID losses are high, indicating the model starts with a partial understanding of the data. As training advances, both losses persistently decline, displaying that the model is efficiently learning and optimizing its parameters. The close alignment between the TRAIN and VALID loss curves during training suggests that the model hasn’t over-fitted and upholds good generalization to unseen data. This persistent and gradual decrease in loss denotes a well-trained, reliable, stable, and deep learning model.

Fig. 9.

Loss curve of HDLMFE-ETESSE model under 70:30.

Table 4; Fig. 10 depict the comparative study of the HDLMFE-ETESSE methodology with current methods under various metrics^33,34. The outcomes underlined that the HDLMFE-ETESSE model achieved higher , , and of 98.58%, 95.66%, 95.70%, and 95.64%, respectively. The present methodologies, namely Ensemble-DL, CNN-DALBPFE, TCN, Bag-of-State, EfficientNetB0, InceptionV3, and VGG-19 models, attain worse performance under various metrics. The findings exhibit that the Ensemble-DL model provides a balanced outcome, thus restricting its efficiency. The CNN-DALBPFE attains greater with low , highlighting imperfect detections. Also, the TCN model outperforms in F1-Score but has lesser . Moreover, the Bag-of-State model illustrates higher and lower overall . Additionally, the EfficientNetB0 and InceptionV3 methods emphasize robust outputs with minor variances in . The VGG-19 technique shows lesser performance. Finally, the HDLMFE-ETESSE method outperforms the existing model with greater accuracy and balanced metrics.

Table 4.

Comparative study of HDLMFE-ETESSE technique with existing methods.

Method	Accuy	Precn	Recal	F1Score
Ensemble-DL	75.80	82.25	69.94	82.42
CNN-DALBPFE	94.82	89.26	70.85	71.15
TCN	91.20	75.23	78.59	91.74
Bag-of-State Model	66.58	88.64	81.17	89.23
EfficientNetB0	95.30	94.68	94.70	94.58
InceptionV3	94.19	94.00	94.50	94.00
VGG-19	86.15	86.16	86.15	86.15
HDLMFE-ETESSE	98.58	95.66	95.70	95.64

Fig. 10.

Comparative analysis of HDLMFE-ETESSE model (a) , (b) , (c) , and (d) .

Conclusion

This paper presents an HDLMFE-ETESSE technique for sustainable education. The aim is to progress an effective student’s emotional recognition to enhance student engagement and learning outcomes for sustainable education. Initially, the image pre-processing stage applies face normalization and facial alignment methods to improve image quality. Furthermore, the feature extraction process is performed by the AdaptSepCX attention network system to identify and isolate the most relevant features from raw data. Finally, the hybrid of the C-BiG model is employed for classification. The experimental analysis of the HDLMFE-ETESSE approach is examined under the student-engagement dataset. The comparison study of the HDLMFE-ETESSE approach portrayed a superior accuracy value of 98.58% over existing models. The limitations include dependence on single dataset, which may not properly generalize across various cultural and educational settings. Also, contextual learning variables or physiological signals are not utilized in finding the engagement levels. Furthermore, the learning ability and fairness may be affected by sample size and class imbalance and also lacks real-time validation, restricting practical deployment. The user-specific factors were not considered and the annotation of engagement levels may involve subjective interpretations, reducing reliability. Future work may explore multi-source data integration, context-aware adaptation, and deployment in varied educational platforms for broader applicability.

Author contributions

Reema G AL-anazi: Conceptualization, methodology, validation, investigation, writing—original draft preparation, Nouf Mohammed Alhammad: Conceptualization, methodology, writing—original draft preparation, writing—review and editingMohammed Burhanur Rehman: methodology, validation, writing—original draft preparationNouf J. Aljohani: software, visualization, validation, data curation, writing—review and editingNesreen Almohammade: validation, original draft preparation, writing—review and editingReem Alsuhaibani: methodology, validation, Conceptualization, writing—review and editingMuhammad Alasmari: methodology, validation, original draft preparation.Abdulbasit A. Darem: validation, original draft preparation, writing—review and editing.

Funding

No Funding.

Data availability

The data that support the findings of this study are openly available in the Kaggle repository at [https://www.kaggle.com/datasets/joyee19/studentengagement] , reference number³².

Declarations

Competing interests

The authors declare no competing interests.