Adaptive G-UKT: a unified probabilistic framework for knowledge tracing via adaptive graph topology learning and uncertainty-aware Gaussian embeddings

Abstract

Dynamic State Tracking (DST) is pivotal for personalized recommender systems and user modeling, aiming to estimate users’ evolving latent states from sequential interactions. However, existing deep sequence paradigms predominantly treat interaction entities, such as semantic tokens and items, as isolated deterministic vectors. This approach often overlooks latent structural dependencies among entities, including knowledge graph topologies, and remains limited in quantifying the epistemic uncertainty inherent in stochastic user behaviors caused by random interactions and aleatoric noise. To address these dual challenges, we propose Adaptive G-UKT (Adaptive Graph-Enhanced Uncertainty-aware Knowledge Tracing), a unified probabilistic framework for temporal sequence modeling. Unlike traditional point-estimation models, we map hidden user states into Gaussian distributions, enabling the simultaneous tracking of semantic activation levels and estimation confidence through diagonal covariance. To mitigate data sparsity, we design an Adaptive Graph Learner that autonomously infers latent semantic correlations from raw data, coupled with an Adaptive Gaussian-HGNN that propagates uncertainty information across the dynamically learned topology. Furthermore, we introduce a Wasserstein attention mechanism to perform distribution-aware sequence retrieval and an uncertainty-guided contrastive learning strategy to enhance model robustness against noisy interactions. Extensive experiments on four large-scale real-world sequential datasets, namely ASSISTments2009, Bridge2Algebra2006, Algebra2005, and NIPS34, demonstrate that Adaptive G-UKT achieves competitive performance against state-of-the-art baselines, showing particularly significant gains in sparse data regimes. Crucially, visualization analysis confirms the model’s capacity to autonomously uncover intrinsic structural topologies, bridging the critical gap between high-precision deep sequence learning and interpretable knowledge graph reasoning.

Introduction

With the rapid proliferation of online learning platforms and adaptive inference systems, sequential latent state estimation over structured entity graphs has emerged as a central problem in representation learning¹. At the core of this problem lies Knowledge Tracing (KT)—formally, the task of sequentially inferring latent competency state distributions of entities from historical observation streams^2,3. By modeling how entities progressively update their latent states over structured concept graphs , downstream systems can perform confidence-aware inference and adaptive decision-making. Consequently, developing robust and accurate latent state estimation models is a key factor for reliable sequential inference pipelines.

In recent years, deep sequential encoders for KT, such as Deep Knowledge Tracing (DKT)⁴ and Attentive Knowledge Tracing (AKT), have demonstrated remarkable potential by capturing complex temporal dependencies via recurrent and attention-based architectures. Recent advancements have further integrated forgetting mechanisms⁵ and adopted simple yet competitive deterministic baselines⁶ to refine predictive accuracy. However, despite their empirical success, existing methods still face two fundamental limitations that compromise their reliability in complex real-world sequential inference scenarios⁷.

First, the challenge of Structural Sparsity and Node Isolation. Existing encoders predominantly adopt an entity-isolated scheme, modeling each concept node independently and failing to exploit the latent relational structure of the underlying concept graph via graph-based message passing^8,9. In practice, observation streams are frequently sparse: an entity may only interact with a small subset of nodes. Without propagating state information across topologically adjacent nodes, the model cannot infer latent states for unobserved entities. Recent attempts to incorporate structured relational priors¹⁰ or temporal extrapolation mechanisms¹¹ have shown promise, yet addressing the cold-start problem under severe observation sparsity remains an open challenge.

Second, the limitation of Deterministic Point Estimation. From a Bayesian perspective, existing models collapse the posterior distribution to a Dirac delta function, discarding all epistemic uncertainty encoded in the covariance structure¹². This renders them incapable of distinguishing high-confidence state estimates (low posterior variance ) from unreliable estimates corrupted by aleatoric noise—a critical failure mode for robust sequential inference. While some works have improved representational expressivity via question-centric encodings^13,14 or retrieval-augmented mechanisms¹⁵, explicit posterior uncertainty quantification remains underexplored.

While several recent works have attempted to incorporate graph structures or probabilistic embeddings separately, a unified framework that simultaneously addresses structural sparsity and epistemic uncertainty quantification remains an open research question. Graph-based methods often rely on static, pre-defined adjacency matrices that cannot adapt to data-driven relational patterns, while existing contrastive learning approaches^16–18 may not fully leverage uncertainty propagation across graph topology. Therefore, jointly modeling latent relational graph structure via adaptive topology learning and maintaining distributional state representations via Gaussian embeddings represents a critical avenue for advancing sequential latent state estimation.

More importantly, the novelty of our work does not lie in merely assembling adaptive graph learning, Gaussian embeddings, and contrastive learning as independent modules. Instead, Adaptive G-UKT couples these components within one probabilistic pipeline: the learned topology governs distributional message passing, Gaussian states provide a shared uncertainty-aware representation space, and Wasserstein attention together with uncertainty-guided contrastive regularization performs sequence retrieval and robustness enhancement directly in that space. This joint coupling allows the graph structure and posterior uncertainty to interact throughout state propagation, rather than being modeled in isolation as in prior uncertainty-aware KT or graph-based KT methods.

To address these challenges, we propose Adaptive G-UKT (Adaptive Graph-Enhanced Uncertainty-aware Knowledge Tracing), a unified probabilistic inference framework. Unlike traditional point-estimation encoders, we reformulate each entity’s latent state as a Gaussian Distribution , characterized by a posterior mean (state centroid) and a diagonal covariance (epistemic uncertainty bound). To address structural sparsity, we design an Adaptive Graph Topology Learner that infers latent relational dependencies end-to-end via differentiable sigmoid adjacency relaxation. Importantly, we propose an Adaptive Gaussian-HGNN that performs dual-channel distributional message passing across the learned topology, propagating both posterior means and diagonal variances via residual graph convolution, enabling inference of latent states for unobserved nodes through neighborhood aggregation. Furthermore, to enhance robustness against observation noise, we introduce an Uncertainty-Aware Contrastive Regularization strategy that leverages the learned covariance as a dynamic reliability filter for selective stochastic augmentation.

The main contributions of this paper are summarized as follows:

• Unified Probabilistic Inference Framework with Distribution-Aware Mechanisms: We propose Adaptive G-UKT, a unified framework that integrates Gaussian distributional state representations with an end-to-end differentiable Adaptive Graph Topology Learner. By utilizing a closed-form 2-Wasserstein distance kernel as the attention similarity metric, our model geometrically aligns the retrieval mechanism with the Riemannian manifold of Gaussian distributions, ensuring distribution-consistent latent state propagation under complex, high-variance observation regimes. Unlike prior uncertainty-aware KT methods that largely operate over fixed or implicitly modeled relations, and unlike graph-based KT models that mostly rely on deterministic point embeddings, our framework unifies adaptive structure learning, uncertainty propagation, and distribution-aware sequence retrieval in a single end-to-end probabilistic formulation.

• Uncertainty-Gated Contrastive Regularization: We introduce a dual-view contrastive learning strategy equipped with a covariance-gated augmentation mechanism. This approach leverages the learned diagonal variance as a dynamic reliability filter to perform selective stochastic masking, assigning higher perturbation probabilities to high-uncertainty observations, thereby enhancing model robustness and generalization under noisy and incomplete observation streams. Therefore, contrastive learning in our framework is not an auxiliary add-on, but a distribution-aware regularizer coordinated with graph-enhanced Gaussian state evolution.

• Extensive Empirical Evaluation and Distributional Interpretability: We conduct comprehensive experiments on four real-world benchmark datasets. The results demonstrate that Adaptive G-UKT establishes competitive state-of-the-art performance, particularly under sparse observation regimes. Beyond predictive accuracy, we provide in-depth visualization analysis demonstrating that the learned graph topology recovers interpretable relational structures such as modular node clusters and directed dependency hierarchies, offering a transparent “white-box” perspective on the latent relational manifold.

Related work

This section reviews the literature pertinent to our framework, categorizing prior research into three primary streams: Deep Sequential State Modeling, Graph-Structured Relational Learning, and Probabilistic Representation Learning with Contrastive Regularization.

Deep sequential state modeling

The transition from probabilistic graphical models, such as Bayesian Knowledge Tracing (BKT), to deep representation learning paradigms has fundamentally reshaped sequential latent state estimation. Deep Knowledge Tracing (DKT) pioneered this shift by employing Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) units to encode temporal observation sequences into latent state vectors, demonstrating superior capacity for modeling long-range dependencies compared to traditional HMM-based inference. Following this, Dynamic Key-Value Memory Networks (DKVMN)¹⁹ enhanced representational expressivity by decoupling static entity embeddings from dynamic latent state matrices via differentiable memory read/write operations. Recently, generative architectures coupling Variational Autoencoders (VAEs) with external memory mechanisms have further refined latent state disentanglement, enabling more accurate modeling of temporal forgetting and consolidation dynamics²⁰.

To overcome the limitation of RNNs in capturing non-consecutive temporal dependencies, Transformer-based attention mechanisms have been widely adopted for sequential latent state modeling. Attentive Knowledge Tracing (AKT) introduces a monotonic attention kernel that weights historical observations by temporal relevance to the current query state, implementing an inductive bias for forgetting dynamics. Recent work has further expanded the representational capacity of sequential encoders by integrating Large Language Models (LLMs) to extract semantic feature representations from exercise content²¹.

Despite their empirical effectiveness, these sequential encoders share a fundamental limitation: they adopt an entity-isolated encoding scheme, treating each concept node independently. Consequently, they fail to exploit the rich latent relational structure—such as directed dependency edges and topological similarities—that inherently exists within the underlying concept graph .

Graph-structured relational learning for entity state estimation

To address structural isolation in sequential encoders, Graph Neural Networks (GNNs) have been introduced to explicitly model relational dependencies between concept nodes . Early approaches constructed static adjacency matrices based on expert-defined prerequisite graphs or statistical co-occurrence probabilities. In these frameworks, latent state information propagates from observed nodes to topologically adjacent unobserved nodes via message passing, enabling inference over the full concept graph.

Recent years have seen a surge in dynamic and high-order graph representation learning. To capture temporal evolution of relational structure, DyGKT²² models observation histories as continuous-time dynamic graphs with time-stamped edge updates. To address entity heterogeneity, Heterogeneous Graph Neural Networks have been employed to construct unified multi-relational graphs over entity types (exercises, concepts, responses), enabling cross-type feature fusion via heterogeneous message passing^23,24. Furthermore, high-order relational dependencies have been explored via hypergraph-based architectures, modeling complex multi-entity “exercise-concept” hyperedges beyond simple pairwise connections²⁵.

Recent graph-structured KT studies further extend this line by introducing hyperbolic hypergraph transformers with knowledge-state disentanglement and knowledge-associated memory-aware embeddings, demonstrating the value of richer relational inductive biases for concept-state modeling. Recent work has further introduced structural causal models into graph learning pipelines to disentangle spurious statistical correlations from true causal dependency edges²⁶.

Beyond KT, recent GNN-related studies also highlight the importance of adaptive, personalized, incomplete, and heterophily-aware graph learning, including dynamic similarity fusion for recommendation²⁷, grouping-enhanced personalization in federated recommendation²⁸, class-imbalance and attribute-missing graph learning²⁹, and broader incomplete graph learning surveys³⁰. Related advances in adaptive graph meta-learning³¹, heterophilic graph learning under distribution shift³², flexible two-way flow message passing beyond homophily³³, holistic hypergraph message passing^34,35, temporal graph transformers for financial anomaly detection³⁶, hybrid GCN-based intrusion detection³⁷, and transformer-GGNN modeling for sarcasm detection³⁸ further suggest that real-world relational structures are often noisy, partially observed, distribution-dependent, and not necessarily homophilic. These observations motivate data-adaptive graph construction rather than reliance on fixed topology.

However, a critical limitation shared by most prior graph-based methods is their heavy reliance on pre-defined or static adjacency matrices . In practice, such graph structures are often unavailable, incomplete, or corrupted by noise³⁹, and fixed topology cannot adapt to the data-driven relational patterns emerging from observation streams. This underscores the need for end-to-end differentiable Adaptive Graph Topology Learning coupled with distributional uncertainty propagation mechanisms.

Probabilistic representation learning and contrastive regularization

Beyond deterministic point estimation, probabilistic representation learning recognizes that latent state distributions are characterized by inherent stochasticity arising from both observation noise and model uncertainty. Probabilistic extensions of sequential encoders have explored Variational Autoencoders (VAEs) coupled with Bayesian state-space models to decompose total predictive uncertainty into epistemic and aleatoric components¹². Closer to our work, the Uncertainty-Aware Knowledge Tracing (UKT) ⁴⁰ framework represents latent states as Gaussian distributions rather than deterministic point vectors, enabling explicit posterior variance estimation.

Simultaneously, InfoNCE-based Contrastive Learning (CL) has emerged as a powerful self-supervised regularization technique for improving latent representation robustness under sparse and noisy observations. Methods like CL4KT⁴¹ apply stochastic augmentation to construct positive pairs and maximize mutual information between augmented views via contrastive objectives. Recent strategies have evolved to address specific architectural challenges, such as integrating retrieval-augmented generation (RAG) pipelines to bootstrap structured relational graphs for multi-hop reasoning⁴². Others have introduced hard-negative mining strategies to improve discriminative capacity over semantically similar node representations. However, standard CL approaches treat all positive pairs uniformly, ignoring the heterogeneous reliability of samples induced by aleatoric observation noise. Our work advances this by leveraging estimated posterior variance as a dynamic reliability weight to perform uncertainty-gated selective augmentation. At a broader methodological level, this perspective also connects to uncertainty-aware learning beyond KT, including robust multimodal state modeling and fusion frameworks^43–46 and Bayesian or federated uncertainty quantification for distributed sequential inference^47–50. These studies collectively emphasize uncertainty as a structural signal that guides representation learning, information fusion, and robust decision-making under sparse or noisy observations, which helps position our framework within a wider uncertainty-aware learning landscape.

Research Gap and Our Approach Existing approaches typically address the above challenges in isolation. Sequential encoders lack relational inductive biases over ; graph-based methods rely on static, pre-defined adjacency matrices ; and probabilistic frameworks typically lack mechanisms for spatial uncertainty diffusion across graph topology. Adaptive G-UKT bridges these gaps by coupling an end-to-end differentiable Adaptive Graph Topology Learner with Gaussian Distributional State Representations, enabling robust sequential latent state estimation under both sparse observation regimes and high-noise environments. More specifically, compared with UKT-style uncertainty-aware models, Adaptive G-UKT introduces adaptive relational structure learning and graph-based uncertainty diffusion beyond standalone Gaussian state modeling. Compared with graph-based KT methods, our framework moves from deterministic node features to Gaussian latent states, and further performs sequence matching in the Wasserstein space. This distinction clarifies that our contribution is a unified probabilistic coupling of structure adaptation, uncertainty modeling, and sequence-level distribution matching, rather than a simple combination of existing ideas. In contrast to recent graph-centric KT models such as hyperbolic hypergraph transformers and memory-aware knowledge-associated embeddings^51,52, which mainly strengthen deterministic relational representation learning, Adaptive G-UKT treats graph topology learning as part of a probabilistic inference pipeline: the learned topology guides Gaussian message passing, while uncertainty-aware Wasserstein retrieval operates on the propagated distributional states.

Problem formulation

In this section, we formally define the sequential latent state estimation task, construct the graph-based relational representation over concept entities, and introduce the probabilistic distributional modeling objective.

Formal notation and relational graph onstruction

We consider a structured sequential inference setting comprising a set of entities , a set of query items , and a set of latent concept nodes . The relational mapping between query items and concept nodes is explicitly encoded by a binary association matrix , where an entry indicates that query item is associated with concept node . Each entity’s history is represented as a chronological observation stream , where each observation at time step t is a tuple comprising the queried item and the corresponding binary response .

Distinct from entity-isolated encoding schemes, we explicitly model the latent relational dependencies among concept nodes via a directed relational graph , where the node set corresponds to the concept entity set . The topological structure is parameterized by a weighted adjacency matrix , defined as:

1

where unlike methods relying on static, expert-defined structural priors, we formulate as an end-to-end learnable parameter matrix, enabling the framework to adaptively infer the intrinsic relational topology directly from observation data without requiring manual graph annotation.

Task definition and probabilistic odeling objective

Given the historical observation stream up to time step t, the fundamental objective of sequential latent state estimation is to predict the response probability for the next query item . This is formally expressed as estimating the conditional probability:

2

While standard sequential encoders represent the hidden latent state as a deterministic point vector , collapsing the posterior to a Dirac delta function, our framework extends this paradigm to full posterior inference. We aim to learn a distributional mapping that projects the observation stream into a Gaussian distributional representation:

3

In this formulation, the components are defined as follows:

4

where represents the concept-level posterior mean matrix aggregated over all concept nodes (state centroids), and denotes the corresponding diagonal covariance matrix encoding epistemic uncertainty bounds at the concept level. At the observation level, we use to denote the diagonal variance vector of a single interaction embedding , which is the direct output of the Gaussian projection head in Eq. (5). This distinction is maintained throughout: uppercase refers to concept-level aggregated posterior representations, while lowercase in Eq. (13) refers to observation-level diagonal variance vectors. More precisely, stacked concept-level quantities such as and are treated as matrices collecting node-wise parameters, whereas are per-interaction vectors; whenever a true covariance matrix is needed at the interaction level, we write it explicitly as .

Methodology

While traditional sequential state estimation approaches predominantly rely on deterministic point encoders and static graph structures, they fail to account for the stochastic nature of latent state distributions and inter-node relational dependencies. To circumvent these limitations, we propose Adaptive G-UKT, a unified probabilistic inference framework that transitions from deterministic point estimates to full posterior Gaussian distributional representations . By explicitly quantifying epistemic uncertainty via diagonal covariance estimation and inferring adaptive graph topology end-to-end from observation data, our framework facilitates robust distributional state propagation under both sparse observation regimes and high-noise environments.

As illustrated in Fig. 1, the proposed architecture operates through three integrated components. The pipeline initiates with a Probabilistic Graph Encoding module (Sect. 3.1 which projects discrete observations into Gaussian distributional embeddings and propagates them over a learned relational topology via dual-channel message passing. Subsequently, the Distribution-Aware Sequential Modeling module (Sect. 3.2) captures temporal dynamics of latent state evolution via a closed-form 2-Wasserstein attention mechanism. Finally, the framework is optimized via a Joint Optimization Objective (Sect. 3.3) that synergizes the primary prediction loss with a dual-regularization strategy, incorporating both feature-level distributional consistency and sequence-level uncertainty-gated contrastive constraints.

Figure 1.

Schematic overview of the proposed Adaptive G-UKT framework. The model operates through a cohesive pipeline: (Left) The Probabilistic Graph Encoding module projects discrete observations into Gaussian distributional embeddings and propagates them over a differentiable learned topology via dual-channel residual message passing; (Center) The Distribution-Aware Sequential Modeling module captures temporal dynamics of latent state evolution via a closed-form 2-Wasserstein Transformer backbone; (Right) The Joint Optimization Objective simultaneously optimizes three terms: the primary prediction loss (), a feature-level distributional consistency loss () to align raw and graph-enhanced embedding views, and a sequence-level Wasserstein NCE loss () to enforce posterior distributional robustness.

For readability, Table 1 summarizes the core symbols used throughout the methodology, including Gaussian distribution parameters, graph learning variables, and optimization terms. This glossary is intended to help readers quickly track the formal definitions and their task-specific meanings before proceeding to the technical derivations.

Table 1.

Summary of the main mathematical symbols used in the methodology.

Symbol	Formal definition	Meaning in KT
	Historical interaction sequence	A student’s observed response history
	Interaction tuple	Exercise-response pair at step t
	Concept set	Knowledge concepts / KCs
	Q-matrix	Exercise-concept association matrix
	Soft / thresholded adjacency	Learned concept graph before / after sparsification
	Gaussian mean vector	Latent mastery representation of one interaction
	Diagonal variance vector	Interaction-level uncertainty of one observation
	Diagonal covariance matrix	Covariance matrix induced by
	Stacked mean matrix	Concept-level latent state centroids at time t
	Stacked diagonal covariance representation	Concept-level uncertainty representation at time t
	Layer-l graph states	Mean/variance states after l rounds of message passing
	2-Wasserstein distance	Distribution-level similarity between two historical states
	Balance coefficient in Eq. (11)	Relative weight of mean and variance mismatch
	Attention coefficient	Importance of historical interaction j to query i
	Prediction loss	Supervision from next-response prediction
	Contrastive loss	Uncertainty-guided representation regularization
	Contrastive loss weight	Trade-off between prediction and contrastive objectives
	Temperature parameter	Softness of contrastive similarity distribution

To further improve readability, we provide an intuitive summary of how the main components interact in the proposed framework. Gaussian state representation answers how certain the model is about the current latent state by explicitly encoding both semantic activation and uncertainty. Adaptive graph propagation answers where information should flow by transferring both posterior means and uncertainty signals across related concepts under the learned topology. Wasserstein-based sequential retrieval then answers which historical states are most relevant by matching past interactions in the same Gaussian distribution space. In this sense, uncertainty modeling and graph propagation are not independent add-ons: the graph controls how uncertainty-aware information is propagated, while the downstream retriever selects history according to both semantic similarity and confidence compatibility.

Probabilistic graph encoding via gaussian distributional message passing

A fundamental limitation of deterministic sequential encoders is the conflation of high-confidence state estimates with noisy, stochastic observations under a single point representation , collapsing the posterior to a Dirac delta function. This representation is geometrically inconsistent with the Riemannian manifold of probability distributions, and renders the model incapable of distinguishing reliable state estimates (low posterior variance) from unreliable ones corrupted by aleatoric noise. To address this limitation, we transition from deterministic point encoders to full distributional representations. Specifically, we project discrete observations into a continuous probabilistic embedding space, employing diagonal multivariate Gaussian distributions to explicitly disentangle the posterior mean (state centroid) from the diagonal covariance (epistemic uncertainty bound).

Gaussian distributional embedding initialization

We model the initial state of an interaction involving exercise as a Gaussian distribution . Formally, given the input feature vector , we employ two parallel projection heads to map it to the latent mastery and uncertainty spaces:

5

where denotes the diagonal entries of , and is a small constant added for numerical stability, ensuring strictly. The activation guarantees strict positive definiteness of the diagonal covariance matrix and prevents numerical degeneracy in the subsequent computation of within the Wasserstein distance, where near-zero variance would otherwise cause instability. In this disentangled distributional representation, encodes the posterior state centroid, while serves as a dynamic epistemic reliability metric—consistent observations converge to low posterior variance, whereas stochastic, noisy observations yield higher uncertainty bounds.

End-to-end differentiable graph topology learning

Effectively capturing latent relational dependencies among concept nodes is crucial for enabling distributional state propagation via graph message passing. However, relying on static, expert-annotated adjacency matrices limits adaptability to data-driven relational patterns. To bridge this gap, we propose an Adaptive Graph Topology Learner that infers the intrinsic relational structure end-to-end directly from observation data via differentiable parameterization.

We define the latent relational graph with node set corresponding to concept entities . Instead of assuming fixed edges, we model the global relational dependency structure via a learnable parameter matrix . The soft adjacency strength is derived via a sigmoid relaxation:

6

To enforce sparsity and filter noise, we apply a hard thresholding operation, resulting in a refined sparse adjacency matrix where if , and 0 otherwise. This learned sparse topology serves as the relational inductive bias for the subsequent distributional message passing over the concept graph.

Dual-channel Gaussian distributional message passing

With the learned structure , we introduce a dual-channel Adaptive Gaussian-HGNN to enable the simultaneous diffusion of mastery and uncertainty. Unlike standard GCNs that smooth deterministic features, our layer propagates probabilistic states while preserving their Gaussian properties via linear transformations. The propagation at layer l follows a residual update process across two distinct channels:

• Posterior Mean Channel: This channel aggregates state centroid signals via neighborhood mean propagation:

  7
• Epistemic Variance Channel: Simultaneously, this channel diffuses diagonal variance vectors (equivalently, the diagonal entries of the covariance matrices) across the graph topology:

  8

Here, denotes the matrix of diagonal variance entries at layer l, consistent with the parameterization introduced in Eq. (4). The residual update preserves the positivity of inherited from the ELU initialization, as the learned weight matrix and the non-negative adjacency jointly ensure that the propagated increments do not violate the strict positivity constraint . This dual-channel propagation scheme implements a principled form of uncertainty diffusion over the relational graph. When the topology learner identifies a strong directed dependency edge (high ), low epistemic uncertainty at the source node (small ) naturally propagates via neighborhood aggregation to reduce the posterior variance at the target node . Consequently, the model can infer reliable latent state estimates for unobserved nodes through graph-regularized posterior update, even in the absence of direct observation evidence.

Distribution-aware sequential latent state modeling via Wasserstein attention

The theoretical motivation for introducing Wasserstein attention is that the latent states in Adaptive G-UKT are modeled as Gaussian distributions rather than deterministic point vectors. Under this representation, a similarity function should jointly reflect discrepancies in both semantic location and uncertainty structure. Standard dot-product attention is defined in Euclidean vector space and mainly compares first-order features, whereas the 2-Wasserstein distance provides a distribution-level metric that is geometrically compatible with Gaussian states. Compared with KL divergence, Wasserstein distance is also more suitable for our sequence retrieval setting because it is symmetric in measuring pairwise discrepancy and admits a closed-form expression for Gaussian distributions that explicitly decomposes similarity into mean and covariance discrepancies. This makes it better aligned with our uncertainty-aware retrieval objective and more convenient for stable end-to-end optimization. Importantly, the graph topology and Wasserstein retrieval are not assumed to share an identical metric form. Instead, they remain geometrically consistent because the adaptive graph only governs where information is propagated, while the downstream Wasserstein kernel measures similarity over the Gaussian states produced after this propagation. Since the dual-channel message passing updates both means and diagonal covariances within the same probabilistic latent space, the representations consumed by the sequence retriever remain distributionally well-formed, making topology-aware propagation and Wasserstein-based retrieval compatible parts of one unified Gaussian geometry.

Building upon the relational inductive biases captured by the adaptive graph topology, we employ a Transformer backbone to model the temporal evolution of distributional latent states . However, standard scaled dot-product attention is geometrically inconsistent with our probabilistic framework: it computes relevance scores based solely on mean vectors, completely discarding the epistemic uncertainty information encoded in covariance matrices . This induces a metric mismatch, as Euclidean inner products are ill-suited for measuring similarity on the Riemannian manifold of Gaussian distributions. To address this limitation, we introduce a Wasserstein Attention Mechanism that replaces the Euclidean inner product with a closed-form 2-Wasserstein distance kernel, ensuring geometric consistency between the similarity metric and the distributional embedding space.

To incorporate sequence order within this probabilistic framework, we first inject learnable sinusoidal positional encodings into the Gaussian means:

9

Subsequently, we compute the attention weights between a query interaction i and a historical key interaction j. Diverging from standard scalar products, we derive the similarity score using the negative 2-Wasserstein distance (). The negative sign is employed to transform the distance metric into a similarity kernel:

10

Intuitively, this formulation means that the model assigns larger attention weights to historical interactions that are closer to the current query in the distribution space. In other words, a past interaction is considered more relevant only when it is similar not only in latent semantic state but also in uncertainty level, which makes the retrieval process more robust than standard point-wise attention under noisy observations.

where the normalization factor accounts for the full dimensionality of the concatenated Gaussian representation , serving as a theoretically grounded scaling factor analogous to the scaling in standard dot-product attention, and preventing the Wasserstein distance from growing disproportionately large with increasing embedding dimension. For the diagonal Gaussian distributions employed in our framework, the squared Wasserstein distance decomposes into a closed-form solution comprising two distinct components:

11

Here, denotes the element-wise square root of the diagonal variance vector , and the norm replaces the Frobenius norm since is diagonal, under which holds exactly. The scalar is a balance coefficient that reconciles the potentially different magnitude scales of the mean and variance terms; in our implementation is empirically set to 1.0, as the two terms are observed to be of comparable magnitude after the initialization. The analytical decomposition above offers rigorous geometric interpretability: the Content Similarity term measures the distance between posterior mean vectors (state centroids), while the Uncertainty Alignment term implements a covariance-aware reliability filter that penalizes similarity scores between observations with heterogeneous epistemic uncertainty levels, geometrically encoding the distributional divergence between Gaussian embeddings on the Wasserstein manifold.

From a reader-friendly perspective, Eq. (11) can be understood as comparing two aspects of historical states at the same time: the first term asks whether two interactions indicate similar knowledge states, while the second term asks whether these states are estimated with comparable confidence. Therefore, even if two observations appear semantically similar, the model will still downweight them when their uncertainty patterns differ substantially.

The final normalized attention coefficients are computed via the Softmax operation:

12

Geometric Interpretation. The analytical decomposition in Eq. (11) provides rigorous geometric interpretability of the attention mechanism. The Content Similarity term quantifies the squared distance between posterior mean vectors, capturing the degree of latent state alignment between two observations. More critically, the Uncertainty Alignment term implements a covariance-gated reliability filter: it assigns lower similarity scores to observation pairs with heterogeneous posterior variance profiles, thereby penalizing high-uncertainty matches and anchoring latent state updates on historically low-variance, high-confidence observations. This decomposition ensures that the retrieval mechanism is sensitive not only to what the latent state centroid encodes, but also to how reliably that state was estimated from the observation stream.

Joint optimization: BCE loss with uncertainty-gated contrastive regularization

While InfoNCE-based Contrastive Learning (CL) provides effective regularization under sparse observation regimes, applying vanilla CL to sequential latent state estimation is suboptimal due to its uniform treatment of all positive pairs. Specifically, standard CL objectives assign equal augmentation probability to all observations, ignoring the heterogeneous reliability induced by aleatoric noise—high-uncertainty observations (corrupted by stochastic perturbations) are incorrectly treated as equally reliable positive samples. To mitigate this, we propose an uncertainty-gated augmentation strategy that leverages the learned diagonal covariance as a dynamic reliability filter to perform selective stochastic masking over the latent representation space.

We construct two complementary views of the observation stream: a Raw View () derived from base interaction embeddings and a Graph-Enhanced View () enriched via GNN-based relational message passing. Crucially, the generation of employs a covariance-proportional stochastic perturbation mechanism, where masking intensity scales with the estimated posterior variance. Formally, for each interaction t, we define a masking probability based on the Frobenius norm of its covariance:

13

where the normalization is performed via min-max scaling over all T interactions within the current sequence. Here, denotes the interaction-level diagonal variance vector of interaction t (as defined in Eq. (5)), which is distinct from the concept-level variance matrix in Eq. (4). Since is diagonal, the Frobenius norm reduces to the norm of the diagonal entries, i.e., . The contrastive loss is defined as:

14

where denotes cosine similarity, is a temperature parameter, and B denotes the batch size. For each anchor , its graph-enhanced counterpart serves as the unique positive sample, while all remaining graph-enhanced representations within the same training batch serve as negative samples. This in-batch negative sampling strategy is computationally efficient and avoids the need for an explicit negative sample queue, as the batch size provides sufficient contrastive signal in practice. Simultaneously, for the primary prediction task, we bridge the gap between our probabilistic latent space and the binary response requirement. Since the predicted mastery and uncertainty are concept-level tensors, we first aggregate them over the concept dimension via a learnable attention-weighted pooling:

15

where is a learnable query vector and denotes the attention weight assigned to concept c, reflecting its relevance to the current prediction. The aggregated representation jointly encodes the posterior mean (state centroid) and diagonal covariance (epistemic uncertainty bound) across all concept nodes via attention-weighted pooling over the concept dimension. We then pass through an MLP followed by a sigmoid activation to obtain the scalar response probability:

16

where the MLP consists of two fully-connected layers with ReLU activation, projecting , and denotes the sigmoid function that maps the output to a valid probability range.

The final objective function is a weighted sum of the prediction loss (negative log-likelihood) and the contrastive regularization:

17

By jointly optimizing this composite objective, Adaptive G-UKT simultaneously maximizes predictive accuracy and calibrates posterior uncertainty estimates to reliably distinguish between high-confidence latent state estimates and observations corrupted by aleatoric noise, yielding well-calibrated distributional representations across diverse observation regimes.

Experiments

Datasets

To strictly evaluate the generalizability and robustness of our proposed Adaptive G-UKT framework, we conducted extensive experiments on four real-world benchmarks: AS2009, AL2005, BD2006, and NIPS34. The statistical details are summarized in Table 2. These datasets were strategically selected to cover a wide spectrum of sequential observation characteristics, ranging from dense, long-sequence regimes to sparse, short-sequence diagnostic settings.

Table 2.

Statistical summary of the benchmark datasets.

Dataset	Sequences	Questions	KCs	Interactions	Avg. Len
AS2009	4661	17,737	123	3,374,115	723.9
AL2005	4712	173,113	112	884,098	187.6
BD2006	9680	129,263	493	1,824,310	188.5
NIPS34	9401	948	57	1,399,470	148.9

Sequences refers to the number of entities (observation sources); Q and KC denote the total number of distinct query items and concept nodes, respectively; Avg. Len indicates the average observation sequence length per entity.

Representing a dense observation regime, AS2009 is characterized by exceptionally long interaction sequences and stable inter-node relational dependencies. It serves as a standard benchmark for evaluating the model’s capacity to perform long-horizon sequential latent state estimation over dense observation streams. In contrast, AL2005 and BD2006 present the challenge of fine-grained step-level state modeling. By recording granular step-level binary responses, their structure necessitates that the model capture subtle local state transitions and fine-grained structural variations across densely interconnected concept nodes. Finally, to rigorously evaluate robustness under sparse observation conditions, we employ the NIPS34 dataset. Featuring significantly shorter observation sequences, this dataset reflects a sparse diagnostic regime, serving as a critical testbed for verifying the effectiveness of our posterior uncertainty modeling in mitigating the cold-start problem under severely limited observation evidence.

Baselines

To strictly evaluate Adaptive G-UKT, we benchmark against a comprehensive set of representative methods stratified into three paradigms:

• Sequential and Robustness-Aware Encoders: This category evaluates temporal latent state modeling capacity and observation noise resistance. We include AKT⁵³, which incorporates temporal decay inductive bias into the attention kernel, and SimpleKT⁵⁴, which captures latent query item difficulty via disentangled representations. To benchmark noise robustness, we employ ATKT⁵⁵ and RobustKT⁵⁶, which utilize adversarial perturbation training or noise-decoupled loss objectives. However, these methods fundamentally rely on deterministic point encoders, collapsing the posterior to a Dirac delta and discarding all epistemic uncertainty.

• Graph-Structured Relational Models: We compare against GKT⁵⁷, DGEKT⁵⁸, and DyGKT²² to isolate the contribution of relational graph structure learning. Notably, GKT formulates the task as time-series node-level classification using statistical co-occurrence transition matrices as static adjacency priors. Comparisons against this spectrum validate whether our end-to-end differentiable Adaptive Graph Topology Learner recovers more meaningful latent relational dependencies than statistical approximations or fixed predefined graph structures.

• Probabilistic Distributional Models: UKT⁴⁰ serves as our most direct competitor, representing state-of-the-art posterior uncertainty modeling for sequential latent state estimation. Comparing against UKT is pivotal to isolate the specific gains derived from our closed-form 2-Wasserstein attention kernel and dual-channel distributional message passing, distinguishing our framework from probabilistic encoders that lack graph-topology-aware spatial uncertainty diffusion.

Experimental settings

We implemented Adaptive G-UKT and all baseline models using the PyTorch framework, accelerated on an NVIDIA GeForce RTX 3090 GPU. The model parameters were optimized via the Adam optimizer with a batch size of 64. The learning rate was initialized at and scheduled via a ReduceLROnPlateau strategy (decay factor 0.5, patience 5). To prevent overfitting and ensure robust convergence, we applied a dropout rate of 0.2 and utilized an early stopping mechanism with a patience of 10 epochs based on validation loss. Regarding model hyperparameters, the dimension of the Gaussian embeddings was fixed at to balance model expressiveness with computational efficiency. The Wasserstein balance coefficient was set to , the contrastive temperature to , and the contrastive loss weight to ; these settings are further examined through the hyperparameter sensitivity analysis reported in Table 8. To strictly assess predictive performance, we employ two standard metrics: Area Under the Curve (AUC) and Accuracy (ACC). Specifically, AUC evaluates the model’s discriminative capability between correct and incorrect responses, being less sensitive to class imbalance, whereas ACC measures the overall prediction correctness using a threshold of 0.5. To further evaluate the quality of uncertainty estimation, we additionally report Negative Log-Likelihood (NLL) and Expected Calibration Error (ECE) in the revised manuscript. NLL measures the likelihood quality of probabilistic predictions, while ECE evaluates the consistency between predicted confidence and empirical accuracy; lower values on both metrics indicate better uncertainty quality and calibration. For data partitioning, each dataset was randomly shuffled with a fixed random seed (1024) to ensure reproducibility, and subsequently split into training, validation, and test sets following an // ratio, with held out as the test set and the remaining further divided into for training and for validation. This partitioning strategy is applied consistently across all four datasets to ensure fair comparison. To ensure statistical reliability, all reported results represent the average of 5 independent runs with distinct random seeds.

Table 8.

Sensitivity analysis of key hyperparameters, including the Wasserstein balance coefficient , contrastive temperature , and contrastive loss weight .

Parameter	Value	AS2009 AUC	AS2009 NLL	NIPS34 AUC	NIPS34 NLL
	0.1	0.8822	0.3657	0.7817	0.5531
	0.3	0.8841	0.3664	0.7819	0.5533
	0.5	0.8841	0.3635	0.7835	0.5516
	1.0	0.8848	0.3607	0.7812	0.5549
	2.0	0.8857	0.3576	0.7823	0.5532
	0.05	0.8833	0.3676	0.7816	0.5535
	0.10	0.8877	0.3578	0.7845	0.5497
	0.50	0.8988	0.3440	0.7872	0.5489
	1.00	0.9006	0.3402	0.7874	0.5515
	2.00	0.9022	0.3365	0.7889	0.5516
	0.00	0.9021	0.3349	0.7911	0.5426
	0.01	0.8990	0.3415	0.7866	0.5474
	0.02	0.8958	0.3441	0.7864	0.5495
	0.05	0.8900	0.3498	0.7850	0.5506
	0.10	0.8848	0.3607	0.7812	0.5549
	0.50	0.8722	0.3862	0.7760	0.5604

Results and analysis

Table 3 presents the comparative results of Adaptive G-UKT against a comprehensive suite of baseline methods—ranging from classical graph models (GKT) to robust sequence learners (AKT, RobustKT)—across four benchmark datasets. As evidenced by the quantitative metrics, our proposed framework achieves superior performance across all four benchmark datasets against external baseline methods, establishing competitive state-of-the-art results particularly in data-rich environments and large concept-space settings. Crucially, the reported standard deviations indicate that our model achieves this high precision with remarkable stability, exhibiting low variance across independent runs.

Table 3.

Overall performance comparison on four benchmark datasets (mean ± standard deviation).

Model	AS2009		AL2005		BD2006		NIPS34
	AUC	ACC	AUC	ACC	AUC	ACC	AUC	ACC
GKT
AKT
ATKT
SimpleKT
DGEKT
DyGKT
RobustKT
UKT
Adaptive G-UKT

All metrics are reported in percentage (%). The best results are highlighted in bold. Adaptive G-UKT consistently outperforms baselines with high stability.

The performance advantage of Adaptive G-UKT is most conspicuously observed on the AS2009 dataset, which represents a data-rich environment with long interaction sequences. As shown in Table 3, our model achieves a remarkable AUC of 90.62%, surpassing the strongest attention-based baseline, AKT (84.74%), by nearly 6.00%. This result offers a critical insight: while advanced attention mechanisms capture monotonic temporal dependencies, they remain fundamentally bounded by the point-estimation constraint—collapsing the posterior to a Dirac delta and discarding covariance information. In contrast, our Gaussian-Wasserstein mechanism leverages dense observation streams to perform geometrically consistent distribution-level similarity computation on the Wasserstein manifold, maximizing the utilization of high-density observation data for a substantial performance gain.

Beyond dense scenarios, our framework demonstrates exceptional precision on the fine-grained, step-level datasets (AL2005 and BD2006). Notably, on AL2005, Adaptive G-UKT achieves a commanding AUC of 93.37%. It is worth noting that while pure graph baselines like GKT (81.54%) perform decently, they struggle to match the precision of our model. This validates that static, expert-annotated adjacency matrices introduce irrelevant structural noise that degrades message passing quality. In contrast, our end-to-end differentiable Adaptive Graph Topology Learner dynamically filters spurious relational edges via sigmoid relaxation and hard thresholding, yielding consistent accuracy gains even over highly competitive baselines like AKT (92.94%). It is worth noting that on BD2006, our model’s margin over the probabilistic baseline UKT is narrow (AUC 81.82% vs. 81.78%), suggesting that uncertainty modeling alone already provides strong structural inductive bias on fine-grained step-level data. Nevertheless, the consistent improvements in both AUC and ACC across all datasets confirm that the integration of adaptive graph propagation and Wasserstein attention yields reliable gains beyond point-estimation and vanilla probabilistic baselines alike. For BD2006 in particular, we believe the marginal gain of the adaptive graph module is mainly due to two factors. First, this dataset already allows UKT to achieve a relatively strong baseline through uncertainty-aware sequential modeling alone, leaving less room for structural augmentation. Second, BD2006 contains fine-grained algebraic steps with highly local and repetitive response patterns, so much of the useful dependency signal is already captured by short-range sequential context and Gaussian uncertainty modeling. Under this condition, the adaptive graph learner still acts as a structural regularizer, but its contribution is more incremental than in datasets where longer-range or sparser relational cues play a larger role.

Furthermore, the robustness of our approach is rigorously tested under the data-scarce conditions of the NIPS34 dataset. Here, Adaptive G-UKT maintains a consistent lead with an AUC of 80.55%. A particularly interesting comparison is against RobustKT (78.54%) and ATKT (76.65%), which employ adversarial perturbations to handle data noise. The superiority of our probabilistic framework over these methods suggests that explicit posterior uncertainty quantification via diagonal Gaussian covariance is a more principled strategy for mitigating the cold-start problem under sparse observation regimes than implicit adversarial perturbation training. By utilizing the closed-form 2-Wasserstein metric to geometrically calibrate distributional similarity, Adaptive G-UKT achieves an optimal balance between representational stability and posterior plasticity, ensuring robust generalization under severely limited observation histories. Across all four datasets, Adaptive G-UKT consistently outperforms its most direct probabilistic competitor UKT (e.g., +4.99% AUC on AS2009, +0.17% on AL2005, +0.04% on BD2006, +0.20% on NIPS34), demonstrating that the proposed adaptive graph propagation and Wasserstein attention contribute complementary gains beyond uncertainty modeling alone.

At the same time, we acknowledge that UKT remains highly competitive on several datasets, especially AL2005 and BD2006 where the margins are relatively small. We attribute this phenomenon to the fact that UKT already captures a strong portion of the task benefit through Gaussian uncertainty modeling alone, particularly on fine-grained algebra datasets where local response patterns are informative and the baseline performance is already high. Under such settings, the adaptive graph learner mainly plays a regularizing and structural refinement role, so its gain is naturally more modest than in AS2009, where long interaction sequences allow Wasserstein-based retrieval and graph-enhanced uncertainty propagation to accumulate clearer benefits. Therefore, the comparison with UKT suggests that our contribution should be understood not as replacing uncertainty-aware KT, but as extending it with adaptive structural modeling that yields consistent, though dataset-dependent, improvements.

Beyond discrimination metrics such as AUC and ACC, we further evaluate uncertainty quality using NLL and ECE on representative datasets using the same abbreviations as the main results tables where directly comparable probabilistic and deterministic baselines are available. As shown in Table 4, Adaptive G-UKT consistently achieves lower NLL than AKT and also outperforms UKT on all three reported datasets, indicating that the proposed graph-enhanced probabilistic framework improves the quality of probability estimates in addition to predictive accuracy. Although AKT attains the lowest ECE on two datasets, Adaptive G-UKT still maintains competitive calibration performance while delivering markedly stronger likelihood quality, suggesting that the proposed uncertainty-aware structural modeling improves probabilistic reliability in a broader sense rather than only sharpening confidence values.

Table 4.

Uncertainty quality evaluation on representative datasets.

Model	AS2009		AL2005		BD2006
	NLL	ECE	NLL	ECE	NLL	ECE
AKT	0.4656	0.0424	0.4914	0.0462	0.3860	0.0063
UKT	0.3495	0.0104	0.5036	0.0555	0.3807	0.0153
Adaptive G-UKT	0.3399	0.0086	0.4850	0.0485	0.3456	0.0083

Lower values indicate better probabilistic fitting quality and calibration performance. The best results are highlighted in bold.

We further analyze the computational overhead of Adaptive G-UKT to clarify the practical cost associated with adaptive graph topology learning, dual-channel Gaussian message passing, and Wasserstein attention. As summarized in Table 5, the additional cost over UKT remains consistently limited on both representative datasets. On AS2009, Adaptive G-UKT increases the parameter count from 3.291M to 3.407M, which corresponds to only about 3.5%, while the training time rises from 9.66s to 10.03s per epoch (about 3.8%), the inference time from 1.78s to 2.12s (about 19.1%), and GPU memory from 2719MB to 2922MB (about 7.5%). On BD2006, the parameter count rises from 17.803M to 18.148M (about 1.9%), the training time from 31.27s to 31.64s (about 1.2%), the inference time from 6.60s to 6.65s (less than 1%), and GPU memory from 2958MB to 3203MB (about 8.3%). These results show that the main computational burden is still inherited from the underlying probabilistic KT backbone, whereas the graph-enhanced components introduce only a small marginal overhead relative to UKT. Therefore, the proposed framework improves structural modeling capacity without materially compromising practical efficiency.

Table 5.

Computational complexity and efficiency comparison on representative datasets.

Model	Dataset	Params(M)	Train/Epoch(s)	Infer(s)	GPU Mem(MB)
Adaptive G-UKT	AS2009	3.407	10.03	2.12	2922
UKT	AS2009	3.291	9.66	1.78	2719
AKT	AS2009	1.529	0.10	0.11	98
Adaptive G-UKT	BD2006	18.148	31.64	6.65	3203
UKT	BD2006	17.803	31.27	6.60	2958
AKT	BD2006	2.019	0.28	0.27	369

Lower values indicate better efficiency.

Ablation study

To rigorously verify the contribution of individual components within Adaptive G-UKT and dissect the interplay between probabilistic modeling and structural learning, we conducted a comprehensive ablation study. We selected two datasets with distinct characteristics—AS2009 (dense, long sequences) and NIPS34 (sparse, diagnostic)—to evaluate robustness across different data regimes. We compared the full framework against four variants: removing the uncertainty-guided regularization (w/o Contrastive Loss), disabling the GNN module (w/o Adaptive Graph), reverting to deterministic point estimates (w/o Gaussian), and retaining Gaussian embeddings but replacing the distance metric with standard dot-product attention (w/o Wasserstein Attn). The comparative results (Mean ± Standard Deviation) are summarized in Table 6.

Table 6.

Ablation study comparison on AS2009 and NIPS34 datasets (mean ± SD).

Variant	AS2009		NIPS34
	AUC	ACC	AUC	ACC
Adaptive G-UKT
w/o Contrastive Loss
w/o Adaptive Graph
w/o Gaussian
w/o Wasserstein Attn

All metrics are reported in percentage (%). The best results are marked in bold. The degradation in variants confirms the necessity of each component.

On the dense AS2009 dataset, characterized by long observation sequences and high interaction density, the distribution-aware retrieval mechanism proves to be the dominant performance factor. The most severe degradation is observed in the w/o Wasserstein Attn variant (AUC 89.15%), confirming that Wasserstein distance is the dominant factor for data-rich sequential tasks. An intriguing observation is that the w/o Gaussian variant achieves unexpectedly high performance on AS2009 (AUC 90.28%). We attribute this to AS2009’s dense and long interaction histories, where a deterministic representation may still capture sufficient sequential patterns without full distributional modeling. This highlights a critical theoretical insight: probabilistic representations must be coupled with a mathematically aligned distance metric. When standard dot-product attention is applied to Gaussian embeddings, the model fails to effectively utilize the learned covariance (uncertainty) during retrieval. Consequently, the variance parameters devolve into redundant noise that interferes with the optimization process rather than serving as validity filters. This metric mismatch—applying Euclidean dot-product similarity on a non-Euclidean distributional manifold—prevents the model from distinguishing between high-confidence state estimates and high-variance stochastic observations, confirming that the 2-Wasserstein kernel is geometrically indispensable for unlocking the full potential of distributional uncertainty modeling in retrieval-heavy sequential inference scenarios.

Conversely, the performance dynamics shift significantly on the NIPS34 dataset, which features shorter and more diagnostic sequences. In this sparse regime, the w/o Gaussian variant suffers the most severe degradation among all variants (AUC 72.09%), which reveals a critical finding: on sparse diagnostic sequences, probabilistic embeddings are indispensable. Without Gaussian distributional representations, the model collapses to a deterministic point encoder paradigm, losing the ability to maintain posterior uncertainty bounds over limited observation histories and failing to distinguish high-confidence state estimates from noisy, aleatoric observations in short sequences. A further notable observation is that the w/o Wasserstein Attn variant also yields substantial degradation (AUC 77.29%, a drop of 3.26%), confirming that precise distribution-aware retrieval remains a critical bottleneck when interaction histories are limited. Furthermore, the w/o Contrastive Loss variant exhibits a sharper performance drop on NIPS34 (AUC drop of 1.42%) compared to AS2009 (AUC drop of 1.19%), validating the hypothesis that uncertainty-guided contrastive regularization is particularly vital for preventing overfitting when training data is limited.

In conclusion, the ablation study reveals that the components of Adaptive G-UKT are not merely additive but synergistic. The Gaussian embedding provides the fundamental uncertainty representation required for robust modeling in sparse contexts (NIPS34), while the Wasserstein attention ensures this information is correctly utilized for precise retrieval in dense contexts (AS2009). Together with the end-to-end differentiable graph topology learner and uncertainty-gated contrastive regularization, these components form a cohesive probabilistic inference framework capable of generalizing across diverse observation regimes ranging from dense to severely sparse sequential data distributions.

To further examine the design choice in Sect. 3.1.3, we additionally evaluate the sensitivity of the dual-channel Gaussian message passing module to HGNN layer depth. As reported in Table 7, a single graph propagation layer yields the best overall trade-off between relational aggregation and representation preservation. As the number of layers increases, AUC gradually decreases on both AS2009 and NIPS34, while NLL also shows no consistent improvement. This trend suggests that stacking deeper Gaussian message-passing layers tends to over-smooth concept representations and over-mix uncertainty information across the learned topology, thereby weakening the discriminability of knowledge states. Therefore, the single-layer design adopted in Adaptive G-UKT is not arbitrary, but is supported by both empirical performance and the need to avoid excessive uncertainty diffusion.

Table 7.

Sensitivity analysis of HGNN layer depth in the dual-channel Gaussian distributional message passing module.

#Layers	AS2009 AUC	AS2009 NLL	NIPS34 AUC	NIPS34 NLL
1	0.8932	0.3465	0.7887	0.5457
2	0.8926	0.3454	0.7872	0.5478
3	0.8893	0.3506	0.7862	0.5480
4	0.8886	0.3513	0.7849	0.5508

We further conduct a systematic hyperparameter sensitivity analysis for three key coefficients: the Wasserstein balance coefficient , the contrastive temperature , and the contrastive loss weight . All experiments are performed by varying one hyperparameter at a time while keeping the remaining settings unchanged. As shown in Table 8, Adaptive G-UKT is generally stable across different settings on both AS2009 and NIPS34. The Wasserstein coefficient has the smallest influence, with AUC varying only within a narrow range, indicating that mean discrepancy and uncertainty discrepancy are naturally balanced after Gaussian state normalization. The temperature mainly affects the sharpness of contrastive alignment; larger values improve single-run AUC in this sweep, while the default is retained because it was selected under the original validation protocol and provides a conservative low-temperature setting for stable contrastive optimization. The loss weight is more sensitive: excessive contrastive strength, especially , clearly degrades both AUC and NLL, while the default keeps the contrastive objective active without overwhelming the prediction loss. Overall, the default configuration represents a practical trade-off between predictive performance, uncertainty quality, and training stability.

To further validate the benefit of the proposed Adaptive Gaussian-HGNN design, we replace the graph propagation backbone with several representative alternatives, including GCN, GAT, and GraphSAGE, while keeping the remaining components unchanged. As shown in Table 9, all graph backbones achieve broadly comparable performance, indicating that the overall framework is relatively robust to the specific GNN operator. At the same time, the proposed HGNN remains competitive on both datasets and is particularly well-matched to our formulation because hypergraph-style propagation is more natural for capturing higher-order concept relations induced by multi-concept exercises. Therefore, the advantage of the proposed backbone should be understood not only in terms of raw AUC differences, but also in terms of its structural suitability for modeling complex concept interactions within the probabilistic KT framework.

Table 9.

Comparison of different GNN backbones within the proposed framework.

Backbone	AS2009 AUC	AS2009 ACC	BD2006 AUC	BD2006 ACC
Proposed (HGNN)	0.8943	0.8301	0.8146	0.8501
GCN	0.8948	0.8301	0.8178	0.8519
GAT	0.8971	0.8320	0.8180	0.8510
GraphSAGE	0.8949	0.8304	0.8134	0.8500

We further study the threshold used in the adaptive graph topology learner to verify whether the sigmoid relaxation followed by hard thresholding is a reasonable design choice. As reported in Table 10, the model remains relatively stable across a wide range of thresholds, while the learned edge density changes substantially from dense to extremely sparse regimes. This indicates that the adaptive graph learner is not overly sensitive to a single threshold value. In practice, moderate thresholding preserves a compact and interpretable topology without causing material degradation in predictive performance, which supports the rationality of using hard thresholding to control graph sparsity.

Table 10.

Sensitivity analysis of the threshold used in adaptive graph topology learning.

Threshold	AS2009 AUC	AS2009 Edge Density	BD2006 AUC	BD2006 Edge Density
0.0 (soft)	0.8947	0.323	0.8150	0.023
0.3	0.8927	1.000	0.8150	1.000
0.5	0.8951	0.323	0.8130	0.023
0.7	0.8941	0.002	0.8156	0.000
0.9	0.8963	0.000	0.8143	0.000

To further justify the graph structure learning strategy itself, we additionally compare several alternatives, including a no-graph variant, a fixed graph, adaptive soft graph learning without hard thresholding, and the full adaptive-thresholded design. As shown in Table 11, these variants achieve broadly comparable AUC values, but their calibration behaviors differ more noticeably. In particular, the thresholded adaptive strategy maintains competitive predictive performance while producing an explicit sparse learned topology and avoiding any reliance on pre-defined graph priors. This result suggests that the main value of the proposed graph learner lies in offering a practical trade-off among data-driven structure discovery, sparsity control, and stable uncertainty-aware prediction, rather than in maximizing AUC alone.

Table 11.

Comparison of different graph structure learning strategies.

Strategy	AS2009 AUC	AS2009 ECE	BD2006 AUC	BD2006 ECE
No Graph (UKT)	0.9013	0.0302	0.8162	0.0087
Fixed Graph	0.8955	0.0105	0.8137	0.0060
Adaptive (soft)	0.8966	0.0140	0.8144	0.0256
Adaptive+threshold	0.8944	0.0161	0.8148	0.0120

Visualization of learned relational graph topology

To analyze how Adaptive G-UKT recovers latent relational dependencies among concept nodes and to validate the structural interpretability of the learned adjacency matrix , we visualize the concept-node correlation weights learned by the Adaptive Graph Topology module. Figure 2 presents the heatmaps of the concept-concept correlation weights extracted from the best-performing epochs, where the x and y axes denote the indexed Knowledge Components (KCs) within each dataset. The distinct topological patterns observed across these sub-figures corroborate the specific characteristics of each learning regime, providing strong qualitative evidence that our model effectively recovers data-driven relational graph structures without explicit structural supervision or pre-defined adjacency priors.

Figure 2.

Visualization of learned relational graph topology . The heatmaps depict the learned edge weights between concept nodes across four benchmark datasets, with brighter regions (yellow) indicating higher relational dependency weights. (a) NIPS34 exhibits a selective, sparse topology reflecting high-confidence edge filtering to prevent over-smoothing under sparse observation regimes. (b) BD2006 displays modular block-diagonal cluster structure representing latent relational node communities with high intra-cluster edge density. (c) AS2009 shows a pronounced diagonal activation pattern reflecting strong temporal locality and sequential node dependency chains. (d) AL2005 reveals complex off-diagonal activations representing non-linear, cross-cluster relational dependencies and multi-hop graph connectivity.

A striking contrast emerges when examining the topological density between diagnostic and extensive learning scenarios. As shown in Fig. 2a, the heatmap for the sparse NIPS34 dataset exhibits a highly selective and discrete topology. This indicates that the Adaptive Graph Topology Learner functions as a sparse relational filter, propagating distributional state information only across high-confidence edges to avoid over-smoothing on the node feature manifold—a critical property for maintaining inference robustness under sparse observation regimes. In stark contrast, Fig. 2c for the AS2009 dataset displays a pronounced and continuous diagonal structure. This pattern aligns with the strong temporal locality and sequential prerequisites inherent in the dataset, suggesting that the model prioritizes self-reinforcement and immediate neighbor propagation to maximize knowledge tracing stability in data-rich environments.

Furthermore, the model demonstrates a remarkable capability in recovering fine-grained latent relational dependencies and modular graph cluster structures from observation data alone. For the BD2006 dataset (Fig. 2b), the heatmap reveals distinct “block-diagonal” clusters. These blocks visually represent latent skill modules or problem clusters that typically co-occur within the same algebraic topic, confirming the model’s ability to perform implicit skill clustering. Similarly, Fig. 2d for AL2005 shows widespread off-diagonal activations. This indicates that the model has captured intricate, non-linear dependencies and cross-topic knowledge transfer required to solve complex multi-step problems, moving beyond simple one-to-one mappings. Collectively, these visualizations confirm that the Adaptive Graph Topology Learner is a dynamic, data-driven module that converges to mirror the intrinsic relational manifold of the underlying observation graph without any structural annotation.

Geometric analysis of learned gaussian posterior representations

While the topology heatmap confirms the structural adaptability of the graph module, it does not fully characterize the internal distributional state of individual concept nodes. To validate the posterior uncertainty quantification capability of Adaptive G-UKT, we project the learned Gaussian embeddings into a 2D subspace via Principal Component Analysis (PCA). As illustrated in Fig. 3, each knowledge concept is depicted as an ellipse, where the centroid corresponds to the mean vector () and the spatial extent (radius along the axes) reflects the diagonal covariance matrix induced by the variance vector (). This geometric representation offers profound insights into the model’s latent manifold.

Figure 3.

Visualization of learned Gaussian embeddings via PCA projection. The diagrams illustrate the latent semantic space across four datasets. Each ellipse represents a Knowledge Component (KC), where the center denotes the semantic mean vector and the radius represents the scope of the corresponding diagonal covariance. (a,b) BD2006 and AL2005 exhibit dense, overlapping clusters, reflecting the high semantic similarity among fine-grained sub-skills. (c) AS2009 shows a balanced, compact distribution due to higher data abundance. (d) NIPS34 displays a dispersed topology with variable ellipse sizes, demonstrating the model’s ability to quantify uncertainty in sparse data regimes.

Posterior Variance Dynamics and Epistemic Uncertainty Calibration. The variation in ellipse sizes serves as a tangible proxy for the model’s epistemic confidence. In the sparse NIPS34 dataset (Fig. 3d), we observe a dispersed topology with several prominent, large-radius ellipses. This phenomenon indicates that for concepts with sparse interaction histories, the model avoids collapsing node representations into deterministic point estimates. Instead, it maintains wide posterior covariance bounds , effectively implementing a probabilistic uncertainty buffer that accounts for insufficient observation evidence at the node level, effectively creating a probabilistic buffer that accounts for insufficient evidence. Conversely, for the data-rich AS2009 dataset (Fig. 3c), the ellipses are notably more concentrated and uniform. This demonstrates that increased interaction density successfully “shrinks” the uncertainty boundaries, as the model accumulates sufficient evidence to enhance its state-estimation certainty.

Latent Semantic Clustering and Inter-Node Representational Similarity. The spatial arrangement further reveals the semantic granularity of different educational domains. For BD2006 (Fig. 3a) and AL2005 (Fig. 3b), the ellipses form dense, overlapping clusters. This significant overlap reflects the fine-grained nature of algebraic topics, which consist of numerous highly similar sub-steps. The model successfully maps pedagogically related skills (e.g., consecutive steps in equation solving) into proximal latent regions, which is essential for facilitating cross-concept positive transfer and capturing the high semantic density of these domains.

To complement the PCA visualization in Fig. 3, we further report three quantitative clustering indicators, including Silhouette Score, Davies-Bouldin Index (DBI), and Calinski-Harabasz (CH) score, as summarized in Table 12. Although the absolute values are modest, which suggests that the learned concept states are organized as soft and continuous semantic manifolds rather than sharply separated hard clusters, the metrics still provide a quantitative view of the latent structure shown in Fig. 3. In particular, BD2006 achieves the highest CH score among the evaluated datasets, indicating relatively clearer hierarchical organization, while the near-zero Silhouette scores on other datasets are consistent with the strong overlap observed in the PCA projection.

Table 12.

Quantitative evaluation of the latent semantic organization visualized in Fig. 3.

Dataset	Silhouette	DBI	CH
AS2009	− 0.0034	3.878	1.53
AL2005	− 0.0019	3.843	1.58
BD2006	0.0064	5.992	3.14
NIPS34	0.0000	2.753	1.45

Higher Silhouette and CH values and lower DBI values indicate better clustering structure.

In summary, Fig. 3 validates that the Gaussian projection head successfully disentangles the posterior mean (state centroid , encoding semantic identity) from the diagonal covariance (epistemic uncertainty bound , encoding estimation confidence). This dual-factor distributional representation provides a geometrically principled foundation for sequential latent state estimation that deterministic Euclidean point encoders fundamentally fail to capture.

Discussion

The consistent empirical superiority of Adaptive G-UKT across diverse benchmarks provides strong empirical validation for transitioning sequential latent state estimation from deterministic point encoders to full posterior Gaussian distributional representations. By synthesizing our experimental findings, we observe that the framework’s success is not merely attributable to individual components, but rather to the synergistic interplay between relational structural inductive bias and epistemic uncertainty quantification. In sparse observation regimes such as NIPS34, the Adaptive Graph Topology Learner functions as a critical relational regularizer. This aligns with recent findings that explicit end-to-end graph structure learning effectively compensates for observation data scarcity by constraining the hypothesis space via relational inductive biases ⁴⁰. By recovering latent relational dependencies—visualized as the sparse yet high-confidence edges in Fig. 2a—the model successfully propagates posterior state information from observed nodes to unobserved graph neighbors via message passing, effectively alleviating the cold-start problem under limited observation evidence. Crucially, this structural propagation is regulated by the posterior uncertainty quantification mechanism to prevent noise amplification along low-confidence edges. As evidenced by the modular block-diagonal cluster structure in Fig. 2b, the model successfully identifies latent node communities, through which the 2-Wasserstein attention kernel selectively propagates only robust, low-variance signals as a dynamic covariance-gated filter. This creates a self-correcting equilibrium: the learned graph topology provides the relational pathways for latent state diffusion, while the posterior uncertainty quantification regulates signal traffic, ensuring that only high-confidence distributional representations are propagated across the concept graph.

From a practical system deployment perspective, Adaptive G-UKT offers principled uncertainty quantification signals directly applicable to downstream adaptive inference pipelines. A primary contribution is the introduction of a confidence-aware decision boundary via explicit diagonal covariance estimation. Traditional deterministic encoders exhibit systematic over-confidence by outputting scalar point predictions without uncertainty bounds, leading to unreliable downstream decision-making. By explicitly estimating the posterior covariance , our framework enables downstream systems to distinguish between high-confidence state estimates (, low posterior variance) and unreliable, high-entropy observations (, high aleatoric noise), facilitating principled confidence-thresholded inference strategies. Furthermore, the framework demonstrates significant potential for automated relational graph mining. The modular block-diagonal cluster patterns autonomously recovered from fine-grained datasets (e.g., Fig. 2b) demonstrate that the Adaptive Graph Topology Learner can identify latent node communities and directed dependency hierarchies from observation data alone without structural annotation. Recent studies have validated that such data-driven graph topology mining can significantly reduce reliance on expensive manual graph annotation⁵⁹, offering a scalable solution for dynamically evolving relational graph structures⁶⁰. From this broader perspective, Adaptive G-UKT can be viewed as a task-specific instantiation of uncertainty-aware sequential modeling and probabilistic graph representation learning, echoing themes from robust multimodal reasoning and state assessment systems and Bayesian or federated learning frameworks that foreground uncertainty quantification for distributed inference. Relative to these directions, our contribution is to instantiate the same uncertainty-aware modeling principle within KT by jointly coupling adaptive graph discovery, Gaussian posterior state tracking, and Wasserstein-based sequence retrieval in one end-to-end framework.

Despite these contributions, we acknowledge certain limitations that motivate future research directions. Specifically, regarding the interpretability of the learned graph topology , while the adaptive adjacency matrix reveals strong statistical co-occurrence patterns between concept nodes, it does not inherently guarantee causal graph structure in the structural causal model sense. An edge implies statistical dependency between nodes and , but not necessarily a strict directed causal relationship (i.e., that high posterior mean at node causally induces high posterior mean at ). Looking ahead, we envision two promising technical directions. First, to address the cold-start problem under severely sparse observation regimes, future work could integrate pre-trained Large Language Models (LLMs) to provide semantic feature priors for Gaussian embedding initialization, thereby enriching posterior mean initialization with linguistically grounded semantic representations^61,62. Second, to address the limitation of purely statistical relational learning, we propose incorporating Causal Discovery algorithms to prune the learned adjacency matrix . By moving beyond statistical co-occurrence to identifiable causal graph structures⁶³, we aim to ensure the recovered topology reflects robust causal dependency paths rather than spurious statistical associations⁶⁴.

In addition, the present work mainly provides geometric and modeling-level justification for Wasserstein attention, rather than a full formal treatment of convergence or uncertainty calibration guarantees; establishing such stronger theoretical results remains an important direction for future work.

Conclusion

In this paper, we proposed Adaptive G-UKT, a principled probabilistic inference framework that addresses the fundamental limitations of conventional deterministic point encoders in sequential latent state estimation. By coupling an end-to-end differentiable Adaptive Graph Topology Learner with a closed-form 2-Wasserstein attention kernel, our framework achieves geometric consistency between the similarity metric and the Riemannian manifold of Gaussian distributions, bridging the gap between predictive accuracy and distributional interpretability. We fundamentally shift the sequential modeling paradigm from static deterministic point vectors to dynamic posterior Gaussian distributions , ensuring that epistemic uncertainty—distinguishing high-confidence state estimates from aleatoric noise—is explicitly quantified, propagated, and calibrated throughout the inference pipeline.

Extensive empirical evaluations across diverse benchmarks (AS2009, AL2005, BD2006, NIPS34) demonstrate that Adaptive G-UKT consistently outperforms state-of-the-art baselines across all evaluation metrics. Crucially, our results confirm that explicit posterior uncertainty quantification via diagonal Gaussian covariance acts as a robust distributional regularization mechanism, significantly mitigating performance degradation under noisy observation streams and cold-start sparse regimes. The framework exhibits superior cross-dataset stability and generalization, demonstrating that incorporating end-to-end differentiable relational graph topology learning into probabilistic sequential encoders is essential for robust latent state estimation over sparse, structured observation data.

Beyond predictive performance, Adaptive G-UKT provides a transparent “white-box” perspective on the learned latent relational manifold. Our work demonstrates that the model can autonomously recover the latent relational graph topology and distributional state representations without explicit structural supervision or manual graph annotation, identifying node community structures and directed dependency hierarchies purely from observation streams. This interpretable distributional representation provides downstream systems with trustworthy, granular posterior uncertainty estimates at the node level, paving the way for reliable confidence-aware inference pipelines in adaptive sequential decision-making systems.

Future Work. Building on these findings, future research will explore extending this probabilistic distributional inference framework to multimodal heterogeneous observation streams, integrating textual feature representations from pre-trained language models with behavioral sequential signals via cross-modal Gaussian fusion. Additionally, we plan to couple Adaptive G-UKT with Large Language Models (LLMs) to provide linguistically grounded semantic priors for posterior mean initialization , further enhancing cold-start robustness under sparse observation regimes and facilitating causal graph structure discovery via identifiable structural causal model inference.

Author contributions

N.J. conceived the study and designed the methodology. N.J., W.S., and J.X. developed the model and conducted the experiments. W.S. and J.X. collected and processed the data. S.Z. contributed to the analysis and interpretation of the results. Y.X. supervised the research and provided critical revisions to the manuscript. N.J. wrote the main manuscript text, and all authors reviewed and approved the final manuscript.

Funding

This research was supported by the Natural Science Foundation of Hebei Province (Grant No. G2024507002) and the Technical Research Program of the Ministry of Public Security (Project No. 2023JSYJC20).

Data availability

The datasets generated and/or analysed during the current study are available in the Adaptive-G-UKT repository, https://github.com/OyaaSu/Adaptive-G-UKT.

Declarations

Competing interests

The authors declare no competing interests.