Cross-modal adaptive reconstruction of open education resources

Abstract

Matching vast online resources to individual learners’ needs remains a major challenge, especially for adults with diverse backgrounds. To address this challenge, we proposed a Dynamic Knowledge Graph-enhanced Cross-Modal Recommendation model (DKG-CMR) to solve the problem. This model utilizes a dynamic knowledge graph—a structure organizing information and relationships—that continuously updates based on learner actions and course objectives. DKG-CMR focuses on three key improvements: (1) Aligning meaning across different data types (e.g., text, video, user behavior logs). (2) Maintaining the knowledge graph’s real-time relevance. (3) Reducing the cognitive demand of recommendations (optimizing cognitive efficiency). Our approach employs contrastive learning (a technique for similarity learning) with an enhanced algorithm. It achieved high accuracy (F1-score = 0.912) in multimodal understanding, significantly outperforming baselines (+ 33.7%). The dynamic knowledge graph improved recommendation accuracy by 35.5% while achieving low system latency (1.45 s average, 99% of responses ≤ 1.8 s). Evaluation with 1,520 adult learners demonstrated significant improvements: Participants reported a 40.5% reduction in perceived mental workload (measured by NASA-TLX, p < 0.001). Resource screening time decreased by 56.8%. Mediation analysis identified reduced cognitive load as a primary mediating factor, explaining 47.6% of the total effect variance. We established a Cognitive-Friendly Recommendation (CFR) criterion balancing accuracy with operational efficiency. Implemented in an electronics course restructuring, this work provides an effective framework for techno-cognitive collaborative optimization. Integrating cognitive science insights with cross-modal AI demonstrates significant potential for enhancing resource accessibility and personalization in open education.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-025-15200-8.

Introduction

The rapid digital transformation of open education is fundamentally reshaping lifelong learning ecosystems. In China, platforms offering Massive Open Online Courses (MOOC) are at the forefront of this change. These platforms now host 76,800 courses–more than triple the number available in 2019–and support over 1.277 billion users¹. However, this rapid increase in resources has created a surprising problem known as the ‘resource abundance paradox’: while there are more learning materials than ever, adult learners—especially those juggling work and study—often feel overwhelmed (experience substantial cognitive overload). This happens because they struggle to find and make sense of relevant information scattered across many different, unconnected sources (when navigating disjointed and heterogeneous resources)². Data from China’s National Survey of Student Engagement (NSSE-China) reveal that 67.3% of adult learners dedicate fewer than 5 h weekly to productive study³ yet spend approximately 230 min daily screening redundant materials. The cognitive load index (terminology consistency) measures 0.58 (threshold = 0.4)⁴ with this inefficiency directly contributing to course completion rates below 22% in vocational programs.

To address personalized resource delivery, existing research predominantly utilizes collaborative filtering and deep learning-based recommendation systems⁵. While these methods demonstrate strong interpretability and improve relevance prediction accuracy through higher-order feature extraction, they face two critical limitations: (1) Modal-isolation problem (hyphen added)—over 80% of existing models (clarification added) process text, video, and behavioral logs independently, failing to capture cross-modal semantic coherence⁶; (2) Static knowledge representation—conventional knowledge graphs (KGs) lack capacity for dynamic updates to learner profiles and resource ontologies (parallel structure)⁷. These technical limitations embody a novel instantiation of Sweller’s cognitive load theory in digital education contexts: when semantic discrepancies across multimodal resources intensify working memory demands, they inevitably result in the resource overload paradox. This phenomenon underscores the imperative to establish robust interdisciplinary synergies between cognitive science and artificial intelligence.

Recent advancements in cross-modal learning (e.g., vision-language pre-training⁸ and dynamic knowledge graph embedding⁹ have provided novel pathways for technological innovation. However, their application in open educational environments remains underexplored. Inspired by AI-generated content (AIGC)-driven adaptive learning¹⁰ this study proposes a Dynamic Knowledge Graph-enhanced Cross-Modal Recommendation (DKG-CMR) model. The model’s innovations are threefold:

1. Cross-modal alignment: A bidirectional transformer architecture jointly encodes textual (course materials), visual (instructional videos), and behavioral data (clickstream logs), addressing modal isolation through contrastive learning (CL).

2. Dynamic knowledge graph evolution: Real-time entity-relationship extraction updates learner competency profiles and resource dependency networks, enabling adaptive resource reorganization.

3. Cognitive-efficiency co-optimization guided by CFR: A multi-objective optimization algorithm balances recommendation accuracy (F1-score, the harmonic mean of precision and recall) and cognitive load reduction.

In evaluations using a dataset of 1,520 adult learners from an Open University’s Mechanical and Electrical Engineering program, the DKG-CMR achieved 89.7% recommendation accuracy. Key contributions include:

1. A novel framework integrating cross-modal learning with dynamic knowledge graphs for open education;

2. Empirical validation of cognitive load reduction via adaptive resource reorganization.

Theoretical framework and hypotheses development

Cognitive load reduction through cross-modal semantic alignment

The evolution of open education has generated abundant learning resources, yet it continues to confront the resource abundance paradox. Specifically, the excessive variety of learning resources in open education—such as text materials, videos, and audio courses—can lead to decision paralysis. This means learners often fixate on certain resources (e.g., repeatedly watching the same video) while ignoring other useful content, ultimately harming their learning results. Data from China’s National Survey of Student Engagement (NSSE-China) reveal that learners face multimodal resource heterogeneity—including text, video, and audio—that exceeds their cognitive processing capacity. Consequently, learners expend excessive time filtering resources, which diminishes learning efficiency. This inefficiency manifests as wasted time navigating redundant content and heightened anxiety from being unable to quickly identify target resources¹¹.

Sweller’s Cognitive Load Theory (CLT) posits that learners possess finite cognitive resources; suboptimal information presentation or information overload may exceed their cognitive load threshold, thereby impeding learning¹². In open education contexts, cross-modal semantic alignment and integration demands exacerbate cognitive load¹². For instance, learners must alternate between textual and visual modalities to comprehend intermodal relationships—a process that consumes substantial cognitive resources. When resource volume is excessive or alignment suboptimal, learners risk cognitive overload, characterized by attentional fragmentation and comprehension deficits.

Hypothesis 1 (H1)

Cross-modal semantic alignment significantly reduces cognitive load via the mediating role of resource screening time reduction (p < 0.05). The mechanism operates as follows: Cross-modal semantic alignment establishes intermodal semantic linkages, enabling learners to rapidly assess resource relevance. For example, contrastive learning⁹ and attention mechanisms¹³facilitate image-text semantic congruence. When searching for topic-specific resources, learners can precisely locate target text-image pairs without exhaustive review of irrelevant materials. Such technologies enhance learning efficiency, mitigate cognitive load, and allow learners to focus on content mastery. The cross-modal cognitive load transmission model is illustrated in Fig. 1.

Fig. 1.

Cross-Modal Cognitive Load Transmission Model.

Dynamic bidirectional update mechanism enhances resource matching accuracy

Static knowledge graphs (SKGs) exhibit significant limitations in processing dynamic data. Relying on fixed data structures, SKGs fail to effectively capture the dynamic evolution of entities and relationships, leading to suboptimal prediction and recommendation performance when handling real-world dynamic information⁷. For instance, in spatiotemporal data processing, SKGs often face challenges in scalability and semantic comprehension, which limits their applicability to large-scale datasets. To address these limitations, this study proposes a bidirectional update mechanism for dynamic knowledge graph (DKG):

1. Behavior-driven real-time optimization: Entity Relationship Evolution Driven by Learner Behavioral Data

This mechanism propels the evolution of entity relationships through real-time learner behavioral data. Specifically, learners’ interactive behaviors (e.g., clicks, browsing, searches) generate continuous data streams, which are dynamically integrated to update entities and relationships within the knowledge graph. For example, when learners repeatedly search for or interact with a specific node in the dynamic knowledge graph, the system automatically adjusts the weight and relevance of that node. This real-time recalibration optimizes the graph’s structural integrity, ensuring it adapts to learner preferences dynamically.

• (2)Top-Down Update Mechanism: Curriculum-Driven Resource Priority Adjustment

Guided by predefined curriculum objectives, DKG dynamically adjust resource priorities. These objectives define the overarching learning direction and focus, enabling the system to strategically evaluate and reallocate resources in alignment with pedagogical goals. For example, if a curriculum emphasizes knowledge in a specific domain, the system elevates recommendations for domain-related resources and reweights associated entities in the knowledge graph.

Hypothesis H2

The dynamic nature of DKG achieves a ≥ 15% improvement in resource-matching accuracy.

A rigorous randomized controlled trial (RCT) was designed to validate the impact of dynamic updates on matching accuracy. Participants were randomly divided into two groups: an experimental group using DKG and a control group using SKGs, implemented within an A/B testing framework. Resource matching accuracy between the two groups was compared to evaluate the effectiveness of DKG dynamism. Key steps included: (1) hypothesis formulation, (2) selection of appropriate statistical tests, (3) determination of significance levels (α), (4) data collection and computation of statistical metrics, (5) p-value calculation based on test statistics, and (6) statistical inference. If the computed p-value fell below α, the null hypothesis was rejected, indicating that the dynamic nature of DKG significantly enhances resource-matching accuracy.

Cognitive-efficacy synergy optimization under CFR principles

Cognitive-Friendly Recommendation (CFR) principles constitute a multi-objective optimization framework that reconciles recommendation accuracy with user cognitive load, establishing an integrated optimization paradigm for recommendation system enhancement. Specifically, CFR principles achieve this through the simultaneous optimization of two metrics: F1-score (recommendation accuracy) and NASA Task Load Index (NASA-TLX) (cognitive load assessment)¹⁴. The CFR framework addresses a persistent limitation of conventional recommendation systems—their over-prioritization of accuracy metrics at the expense of user cognitive experience—thereby enhancing holistic user engagement. The multi-objective optimization problem is formalized as:

1

2

3

In this formulation, X denotes the decision variable vector, which encapsulates adjustable parameters within the recommendation system (e.g., weights for resource allocation across user segments). F (X) represents a multi - objective optimization vector, integrating core objectives such as recommendation accuracy (quantified by F1 - score) and cognitive load (assessed via NASA - TLX index). f₁ (X) denotes the F1-score (accuracy metric), f₂ (X) represents the NASA-TLX (cognitive load index), while g_i (X) and h_j (X) correspond to inequality constraints and equality constraints, respectively. To solve this multi-objective optimization problem, several methods can be employed, such as the linear weighting method or minimax method¹⁵. The linear weighting method transforms the multi-objective problem into a single-objective formulation by assigning predefined weights to each objective, which is mathematically expressed as:

4

Here,ω_k denotes the weighting coefficients, subject to the condition To solve the multi-objective optimization problem, this method minimizes the maximum value of the objective function, formulated as:

5

For F_k (X), denoting the k-th objective function (e.g., minimizing recommendation error as f₁ (X) or reducing cognitive load as f₂ (X)). By employing this approach, the Pareto optimal solution can be identified, which optimizes one objective without degrading the performance of others.

The decision architecture enables dynamic multi-objective optimization (balancing recommendation accuracy and user cognitive load) via a three-level hierarchical decision mechanism. Decision triggers use dual thresholds: if NASA - TLX (cognitive load) > 70, simplification mode activates to reduce load by lowering interface information density; if the recommendation accuracy (F1-score) is lower than 0.8, the expansion mode is activated to enhance the matching effect by increasing the diversity of candidate resources; the default state is the standard mode, which maintains the balance between accuracy and load. The dynamic adjustment mechanism features quantitative evaluation logic as its optimization kernel. If the load improvement is less than 15% after the first adjustment, the system adjusts the weight coefficient in steps of ω + 0.1 and re-enters the state evaluation process. The closed-loop design operates via real-time feedback: after each parameter adjustment, NASA-TLX and F1 indicators are re-evaluated, forming a continuous optimization cycle of status monitoring, decision implementation, and effect verification (Fig. 2).

Fig. 2.

Optimization decision diagram.

Hypothesis H3

CFR optimizes learning effectiveness and system efficacy by dynamically balancing recommendation accuracy (F1) and cognitive load (NASA-TLX).

This chapter focuses on the open education resource adaptation problem, and constructs a three-level progressive hypothesis system of “cognitive load alleviation - resource matching accuracy enhancement - dual-objective optimization”. Cross-modal alignment breaks information barriers, dynamic knowledge mapping enables real-time updates, and the CFR criterion core-optimizes resource recommendation, forming a closed-loop of “theoretical hypothesis-technological path-optimization goal. Hypotheses H1-H3 will be empirically tested in chapters four and five.

Research methodology

Cognitive-Friendly recommendation system architecture

Building on the theoretical framework, this study proposes a three-tiered closed-loop architecture (“Perception–Decision–Feedback”) to establish an AI-driven recommendation engine for open education scenarios (Fig. 3). The architecture integrates multimodal data fusion and dynamic knowledge evolution through layered mechanisms.

Fig. 3.

Three-tiered “Perception–Decision–Feedback” closed-loop architecture.

The Perception Layer captures real-time learner data—including text interactions, video viewing patterns, and behavioral streams—via multimodal sensors, employing an LSTM-attention network¹⁶ to model temporal behavioral patterns. This layer generates cognitive load estimates calibrated to NASA Task Load Index (NASA-TLX) scores¹⁵ (prediction error: ±2.3 points, R² = 0.81, 95% CI [0.76, 0.86]) and dynamically quantifies behavioral feature importance through attention-weighted mechanisms.

The Decision Layer forms the recommendation core via a cross-modal resource pool and dynamic knowledge graph (DKG). The resource pool maintains semantic indexing of multimodal resources through resource semantic association, and automatically prunes content inactive for > 90 days or with relevance weights < 0.2. The DKG supports bidirectional updates: bottom-up entity relation adjustments every 15 min via behavioral logs and eye-tracking data¹⁷ (250 Hz sampling), and top-down curriculum-constrained resource prioritization with weight tweaks capped at ± 0.05 for stability. A multi-objective optimization function balances recommendation accuracy and cognitive load, constraining single-recommendation information entropy to ≤ 3.2 bits (per Miller’s Cognitive Capacity Law¹⁷ and optimizing cross-modal switching frequency to 1.8–2.4 transitions/minute based on eye-tracking experiments¹⁸.

The Feedback Layer addresses open education challenges through adaptive interventions. For commuting scenarios, it triggers text summarization (compression rate ≥ 60%) and increases video keyframe density by 40%¹⁹. Attention drift detection simplifies mobile interfaces to ≤ 5 components per screen when the standard deviation (SD) of interaction time exceeds 28s¹². Meanwhile, cognitive overload mitigation activates simplified recommendations (e.g., prioritizing ‘Circuit Fundamentals’ before ‘Motor Control‘²⁰ if the NASA-TLX equivalent score exceeds 70. This closed-loop architecture establishes a scalable technical paradigm for developing cognitive-aware AI educational systems aligned with human cognitive principles. The architecture supports seamless integration into existing online education platforms through a microservices decoupling design (see the Section "Engineering practice of system deployment" for more details).

Dynamic knowledge graph (DKG) construction process

This study employs a bidirectional updating mechanism for DKG construction:

1. Bottom-Up: Behavior-Driven Entity Relationship Evolution

The DKG captures learners’ fine - grained interaction patterns (e.g., knowledge point searches, video replays, annotations) via an Apache Kafka - based event - driven architecture⁹. The Kafka-based architecture enables real-time interaction data processing, facilitating timely knowledge graph updates with learner behaviors via high-throughput and low-latency features. When a knowledge point’s query frequency exceeds a dynamic threshold (exponentially weighted moving average, λ = 0.1), the system normalizes real-time behavioral data using Siprocal (α). The weight adjustment rule is:

6

where ω_old and ω_new represent the original and updated association weights of knowledge graph entities (e.g., related nodes of a knowledge point). x represents the occurrence rate of learners’ interaction events (e.g., searching or replaying a knowledge point). This mechanism constrains weight increments for high-frequency entities to ≤ 30% to prevent overfitting¹⁶. For instance, when learners consecutively access the “Circuit Fundamentals” knowledge point three times, the system strengthens the association weights of related nodes.

• (2)Top-Down: Curriculum-Objective-Constrained Resource Prioritization

Under curriculum objectives, the DKG dynamically adjusts resource priorities using a goal-oriented evaluation algorithm. For example, if a curriculum emphasizes the “Motor Control” domain, the system prioritizes domain-relevant resources and reinforces intra-domain entity associations. Structural outliers (detected via the Louvain community detection algorithm with modularity < 0.3) are merged to maintain topological coherence¹². For adult education scenarios, a resource recency control is introduced:

7

In this formula, t is the number of days that a resource (e.g., an instructional video) remains unaccessed. where resources (e.g., instructional videos) unaccessed for > 45 days undergo a 20% priority reduction⁷ which maintains consistent knowledge representation through dynamic resource weighting.

The DKG dynamic update timing diagram in Fig. 4 visualizes the behavioral event-driven knowledge graph incremental update process. The real-time update path uses Kafka to receive behavioral events, Spark Streaming for feature extraction/weight computation, and triggers Neo4j incremental updates only when knowledge-associated weight changes exceed ± 0.1. The system also integrates a nightly batch processing mechanism to automatically execute knowledge freshness assessment and obsolete resource elimination strategy. After update completion: The closed-loop feedback mechanism pushes newly generated associated resources to users in real time. This forms a “behavior acquisition–intelligent calculation–graph update–resource push” loop, ensuring the knowledge graph’s dynamic timeliness and recommendation accuracy.

Fig. 4.

DKG dynamic update timing diagram.

Multimodal data processing protocol

The cross-modal learning theory emphasizes enhancing cognitive outcomes through synergistic interactions of multimodal data, with its core lying in semantic alignment and feature fusion across modalities. The proposed DKG-CMR framework optimizes the entire multimodal data processing workflow through standardized protocols, with specific technical pathways as follows:

Cross-modal data collection and preprocessing

In open educational scenarios, multimodal data encompasses audiovisual content (visual and auditory modalities), textual courseware, and learner behavioral logs. The visual modality includes instructional scenes, course diagrams, and experimental demonstrations, while the auditory modality comprises lecture narration, operational sound effects, and background music. Textual courseware provides structured information such as syllabi, knowledge points, and case analyses. Behavioral logs record interaction patterns—including resource access modalities (text/video/audio), dwell times, assignment completion rates, test scores, and engagement metrics (e.g., comments, queries).

Preprocessing involves two key stages. Video enhancement mitigates compression artifacts arising from network fluctuations (e.g., frame drops, pixelation) and applies adaptive noise reduction filters to suppress audio interference (e.g., static, ambient noise)²¹. Text normalization standardizes formatting inconsistencies (e.g., encoding errors, and markdown syntax variations) and employs domain-specific lexicons to optimize semantic coherence.

Feature extraction and cross-modal alignment

The cross-modal alignment module, serving as the core innovation of the DKG-CMR framework, achieves semantic unification of heterogeneous data through coordinated multimodal representation learning. This hybrid architecture integrates Bidirectional Encoder Representations from Transformers (BERT), Vision Transformer (ViT), and Long Short-Term Memory (LSTM) models, incorporating dynamic attention modulation and contrastive learning strategies.

Its technical implementation can be broken down into the following steps:

Multimodal feature extraction employs domain-specific encoders

Visual modality: A ViT-LSTM dual-stream encoder extracts spatial features via a 12-layer Transformer and temporal dependencies through LSTM, generating a 256-dimensional spatiotemporal semantic vector. This architecture achieves an F1 - score of 89.2%—12.7% higher than traditional Convolutional Neural Network - Recurrent Neural Network (CNN-RNN) baselines—on mechanical structure recognition benchmarks.

Auditory modality: The Wav2Vec 2.0 framework, integrated with 13-dimensional Mel-frequency cepstral coefficients (MFCC), constructs a joint acoustic representation space. In Chinese educational automatic speech recognition (ASR) tasks, this approach reduces character error rate (CER) by 30% absolutely via contrastive learning.

Textual modality: A whole-word masking BERT (BERT-wwm) model enables knowledge-enhanced encoding via bidirectional attention mechanisms, improving educational entity relationship recognition accuracy by 18.3%.

Dynamic semantic calibration mechanism

A bimodal collaborative controller, leveraging mutual information theory, is proposed²². When the text-video mutual information metric MI (x_t, x_v) falls below the empirically validated threshold of 0.4, a cross-modal attention module activates²³. This module employs a differentiable region localization algorithm to dynamically map text semantic units to video spatiotemporal features, formalized as:

8

Where:

A_cross denotes the cross-modal attention matrix, capturing the semantic alignment between textual and video modalities.

Q_t (query matrix): Represents query demands of the source modality (e.g., textual or temporal signals), extracted via learnable weightsW_Q.

K_v (key matrix): Encodes feature distributions of the target modality (e.g., textual or temporal signals), extracted via learnable weightsWQ, visual/auditory signals through weights W_K, establishing cross-modal semantic correlations.

V_v (value matrix): Embeds contextual information from the target modality using weights W_V, generating fused representations.

(scaling factor): Stabilizes gradient magnitudes during dot-product operations, with d denoting the key matrix dimension.

Contrastive optimization framework

The proposed framework constructs a triplet contrastive learning space using a modified InfoNCE (Noise-Contrastive Estimation) loss function, integrating temperature-scheduled curriculum learning strategies and dynamic memory bank sampling mechanisms²⁴. This design achieves a 37% improvement in inter-class separability of cross-modal similarity matrices while retaining the computational complexity of O (n).

This module achieves cross-modal semantic equivalence mapping at the parameter space level through three mechanisms: hierarchical feature abstraction, dynamic semantic calibration, and contrastive optimization. It establishes a unified representational framework for downstream knowledge graph construction. Ablation experiments demonstrate that the dynamic alignment mechanism accounts for 62% of the total performance gain, validating the architectural efficacy (Fig. 5).

Fig. 5.

ViT-LSTM cross-modal alignment flowchart.

The architecture adopts a three-modal parallel processing and dynamic fusion mechanism: The video stream, leveraging ViT and key frame extraction, reduces complexity while extracting spatial features; The behavioral stream, using LSTM, models temporal behavior sequences to capture interaction dynamics; The textual stream, via BERT, enables semantic encoding. This multi-modal design ensures comprehensive feature representation. The fusion center dynamically weights and fuses three-modal features via cross-modal attention, then optimizes them with a contrast loss function to output semantically aligned unified feature representations. The architecture realizes semantic consistency modeling of multimodal data through hierarchical feature extraction and attention fusion.

Multimodal feature fusion strategy

Cross-modal data fusion critically enhances analytical accuracy through two strategies: early fusion and late fusion, each with distinct advantages and limitations.

Early Fusion integrates multimodal data at the feature extraction stage by merging raw inputs (e.g., images, audio, text). This approach exploits inherent cross-modal correlations to capture interaction patterns, enriching fused features with comprehensive semantics. However, its performance is highly sensitive to data quality—noise or errors in any modality (e.g., blurred images, erroneous text) may severely degrade results²⁵. In open education, early fusion excels for tightly coupled modalities with stable data quality, such as aligning mathematical formula images with their LaTeX textual explanations.

Late Fusion independently extracts modality-specific features: e.g., video features via convolutional neural networks [CNNs]; Audio spectrograms via Mel-frequency cepstral coefficients [MFCCs]; Text semantics via word embeddings. These features are combined at the decision level using weighted summation, concatenation, or attention mechanisms. This strategy minimizes intermodal interference while preserving modality-specific information, making it suitable for weakly correlated modalities (e.g., asynchronous video demonstrations and voice narrations). Although late fusion sacrifices some cross-modal interactions captured by early fusion, it demonstrates robustness in heterogeneous modality scenarios.

To systematically select fusion strategies, this study proposes a three-tier decision framework (Fig. 6):

Fig. 6.

Three-Tier Decision Framework.

​Data Quality Assessment Layer: Quantifies signal-to-noise ratios (SNR) per modality and inter-modal quality balance using semantic relevance metric calculations and fault tolerance metrics.

​Modality Relevance Analysis Layer: Evaluates semantic complementarity via cross-modal attention mechanisms, temporal synchronization via Dynamic Time Warping (DTW), and feature redundancy via Canonical Correlation Analysis (CCA).

Learning Context Adaptation Layer: Dynamically adjusts strategy selection based on task objectives, resource constraints, and interpretability requirements.

The decision function can be formalized as follows: when the quality assessment score (Q), modality relevance (C) meets the threshold and the real-time requirement (R) is low, early fusion is chosen; otherwise, late fusion is adopted. In open education, this framework prioritizes early fusion for high-correlation pairs (e.g., formula images-LaTeX text) and recommends late fusion for weakly aligned combinations (e.g., video demonstrations asynchronous audio). The decision logic is visualized through a flowchart that maps data inputs to strategy outputs, ensuring traceability and transparency.

In this section, we construct the Dynamic Knowledge Graph Enhanced Cross-Modal Recommendation (DKG-CMR) model framework, and realize a technological breakthrough through a three-layer synergistic mechanism. The closed-loop architecture of “Sensing - Decision Making - Feedback” is adopted, and the perception layer uses the LSTM-Attention network to quantify the cognitive load; The decision-making layer integrates the cross-modal resources and the dynamic The perception layer uses LSTM-Attention network to quantify the cognitive load; The decision-making layer integrates cross-modal resources and Dynamic Knowledge Graph (DKG) to balance the recommendation accuracy and cognitive cost with multi-objective optimization; And the feedback layer reconfigures the resources based on scene adaptation. A two-way update mechanism is proposed to construct the DKG, which evolves in real-time based on event-driven bottom-up, and dynamically adjusts top-down based on course objectives and knowledge freshness. A standardized multimodal flow boosts semantic mapping F1-score by 12.7% using BERT/ViT/LSTM joint coding and improved InfoNCE loss. Feature fusion is adaptive to quality, relevance, and real-time needs. This framework provides methodological support for subsequent research on solving the conflict between overload and personalization of open educational resources.

Each component plays a sequential, complementary role in this system: cross-modal alignment provides a unified knowledge foundation; DKG enables real-time resource matching based on this foundation; CFR converts matched resources into adaptive guidance; and the recommendation engine coordinates the entire process. Empirical ablation tests confirm their interdependence—removing any component undermines the system’s effectiveness.

Experimental design and results

Datasets and participant selection

This study used stratified random sampling to recruit 1,520 adult learners from the Mechanical and Electrical Engineering program at the National Open University (NOU). These learners were enrolled during the 2022–2023 academic cycle and had a mean daily learning duration of ≥ 30 min. The multimodal dataset includes:

Instructional resources: PDF lecture notes (textual content), MP4 practical demonstration videos (resolution: 1080p, frame rate: 30 fps), and timestamped clickstream logs (sampling frequency: 1 Hz).

Learner metadata: Pre-course aptitude assessments (metrics: logical reasoning, spatial visualization) and longitudinal weekly behavioral logs (variables: login frequency, resource dwell time). All experimental protocols were approved by the named institutional committee of The Open University of Sichuan. Informed consent was obtained from all subjects. All methods followed CNIES (Chinese National Institute of Educational Sciences) ethical guidelines and OUSC (Open University of Sichuan) 2023 Revised Data Governance Framework for educational technology research.

Model training and optimization

The proposed BERT-ViT-LSTM multi-modal encoder was implemented using PyTorch 1.13 and optimized via a modified InfoNCE loss function to facilitate cross-modal contrastive learning. The training pipeline integrated dynamic batching (batch size = 256) and multi-threaded data loading (8 workers) to accelerate training. Regularization techniques—15% random token masking and order perturbation—were adopted to enhance generalization, following the data augmentation paradigm of SimCLR (Simple Contrastive Learning of Visual Representations).

For parameter initialization, stage-specific strategies were applied. The first six layers of the BERT text encoder were frozen to mitigate overfitting; The ViT image encoder was fine-tuned from ImageNet-21k pre-trained weights using a 16 × 16 patch size; And the LSTM temporal encoder underwent Xavier-normal initialization with a hidden dimension of 512. These protocols, validated in prior contrastive learning frameworks such as CEED (Contrastive Estimation for Embedding Distillation), ensured robust feature extraction and stability.

The training protocol incorporated three key hyperparameters: (1) a temperature coefficient τ = 0.07 to modulate the contrastive intensity between positive and negative sample pairs; (2) a negative queue of 65,536 samples to improve embedding discriminability; and (3) gradient clipping (max norm = 1.0) combined with automatic mixed precision (AMP) to balance computational efficiency and training stability.

After 15 training epochs, the model achieved loss convergence at 0.213 (training) and 0.225 (validation), with a contrastive accuracy of 78.4% on the validation set. These results align with the Decoupled Contrastive Learning (DCL) theory²⁶which emphasizes the reduction of spurious correlations between positive-negative pairs through systematic decoupling mechanisms.

A/B testing

To validate the DKG-CMR model’s efficacy, a randomized controlled trial (RCT) was conducted following the CONSORT guidelines, with three-phase experimental controls to ensure causal validity and generalizability. Stratified random sampling allocated 1,520 active learners (mean daily engagement ≥ 30 min) from an open education platform into two groups:

The experimental group (n = 760) received DKG-CMR interventions featuring (a) real-time behavior-triggered knowledge graph updates (λ = 0.1 decay factor) and (b) multimodal matching algorithms (θ = 0.4 cosine similarity threshold). The control group (n = 760) utilized traditional methods combining LDA-based topic modeling (K = 30 topics) with collaborative filtering (Top-50 nearest neighbors). Baseline equivalence was verified through stratified variables—prior knowledge (MoocMeta pre-test scores), device type (PC/mobile), and learning styles (Kolb inventory clusters)—with chi-square tests confirming no significant differences (χ²(3) = 2.17, p = 0.54).

During the 12-week intervention, four metrics were tracked:

​Recommendation precision (dynamic F1-score weighted by λ = 0.85 time decay);

​Response latency (millisecond-resolution delay from trigger to delivery);

​Cognitive load (weekly Chinese NASA-TLX ratings, Cronbach’s α = 0.87, 95% CI [0.83, 0.91]);

Course completion (≥ 75% core knowledge mastery and ≥ 95% assignment submission).

Data collection followed double-blind protocols: participants were unaware of group assignments, while analysts handled SHA-256-hashed anonymized data. A Flume-Kafka-Spark pipeline performed real-time log cleansing, with Tukey’s fences (k = 3.0) filtering outliers. To mitigate confounders, three safeguards were implemented: A/B-balanced allocation, ε-differential privacy (ε = 1.2) for behavioral data, and ​backup cohorts retaining 97.3% data integrity post-attrition.

Statistical analysis employed intention-to-treat (ITT) principles with ANCOVA (Analysis of covariance) adjusting baseline variances, reporting effect sizes as Cohen’s d. All inferences used Bonferroni-corrected thresholds (α = 0.01, two-tailed). The study received IRB approval, complying with informed consent and GDPR-aligned data security.

This chapter adopts a stratified random sampling RCT research design, divides 1520 learners into experimental group and control group, and monitors them by four indexes. resource matching accuracy, system response delay, cognitive load level, and learning completion rate. To ensure research validity, we integrated differential privacy and blinding principles. Data processing adopted a Flume-Kafka-Spark streaming pipeline. Rigorous experiments, scientific monitoring, and advanced data processing guarantee result reliability and universality, empirically validating technical solutions for practical promotion.

Results analysis

Hypothesis H1 validation: cognitive load effects of cross-modal alignment

The DKG-CMR framework significantly reduced learners’ cognitive load through cross-modal alignment, as evidenced by a 40.5% decrease in NASA-TLX scores from baseline (M = 55.2, SD = 7.1) to post-intervention (M = 32.8, SD = 5.1; F = 102.37, p < 0.001, η² = 0.12, Cohen’s d = 1.87, 95% CI [1.52, 2.22])(Table 1). The temporal demand dimension showed the largest reduction (72.3 → 34.1), aligning with the NSSE-China educational engagement benchmark (136 min/week resource screening time).

Table 1.

Comparison of key cognitive load indicators (repeated measures ANOVA, η² = 0.12).

Indicator	Experimental Group	Control Group	Reduction (%)	Cohen’s d (95% CI)
NASA-TLX Total Score	32.8	55.1	40.5***	1.87[1.52,2.22]
Resource Screening Time (min/week)2.04 [1.68, 2.40]	136	315	56.8	2.04[1.68,2.40]
Temporal Demand Dimension	34.1	72.3	52.8	1.91[1.55,2.27]

*p < 0.05, **p < 0.01, ***p < 0.001; Experimental group (n = 760) vs. control group (n = 760); NASA-TLX total scores by Mauchly’s test of sphericity (p > 0.05) with Greenhouse-Geisser correction; Reduction=(control group-experimental group)/control group × 100%.

Critically, this reduction was observed exclusively in the experimental group utilizing cross-modal alignment (CMA), while the control group employing conventional recommendation methods (LDA + CF) showed no significant change (Control NASA-TLX: M = 55.1, SD = 6.9), confirming CMA as the differentiating factor.

Structural equation modeling (SEM) identified two significant pathways mediating cognitive load reduction. Cross-modal alignment decreases cognitive load by reducing resource screening time (β = 0.58, p < 0.001, 95% confidence interval [0.51–0.65]). Also, cross-modal alignment reduces cognitive load by increasing recommendation accuracy (β = 0.49, p = 0.003, 95% confidence interval [0.38–0.60]).

These pathways collectively explained 76% of the variance (R² = 0.76). The dominance of the first pathway (β = 0.58 vs. β = 0.49) indicates that CMA’s primary cognitive benefit stems from reducing information foraging costs, independent of final recommendation quality. Technical improvements further demonstrated dose-response relationships: a 0.1-unit F1-score increase predicted an 11.2-point NASA-TLX reduction (β = −11.2, p < 0.01, R² = 0.73, 95% CI [−14.1, −8.3]), while 1-second latency reduction increased course completion rates by 14.3% (OR = 1.35, 95% CI [1.21–1.51]), underscoring the interdependence of algorithmic performance and cognitive experience optimization.

Hypothesis H2 validation: precision enhancement via DKG dynamics

The Dynamic Knowledge Graph (DKG) significantly enhanced recommendation precision and system efficiency through its real-time evolution mechanism. Compared to the control group (LDA + collaborative filtering), the experimental group achieved a dynamic F1-score of 0.912 (95% CI [0.894, 0.930], t (758) = 24.91, p < 0.001), representing a 33.7% improvement (Table 2), while reducing system response latency by 53.9% (2.97s → 1.45s). The high-concurrency stress test shows that the sharded architecture allows the system to maintain P99 latency ≤ 1.8 s under 5,000 QPS load.

Table 2.

Comparison of DKG-CMR and baseline model performance (Independent samples t-test, cohen’s d = 1.24).

Metric	DKG-CMR	Baseline	Improvement	Mean Diff 95% CI	Cohen’s d (95% CI)
F1-score	0.91***	0.68	33.7%	[0.89,0.93]	1.24[1.01,1.47]
Response time (s)	1.45***	2.97	51.2%	[1.43, 1.59]	1.68[1.41,1.95]

F1-score comparisons were tested by Levene’s chi-square test (F = 2.17, p = 0.14); *p < 0.05, **p < 0.01, ***p < 0.001; response latency is in seconds (s); performance improvement = (DKG-CMR-baseline)/baseline × 100% (refer to IEEE Performance Reporting Standard, 2023)²⁷.

Behavioral metrics further validated these improvements: practical resource click-through rates increased by 42% (χ²(1) = 31.57, p < 0.001); Dwell time extended to 58 s (Cohen’s d = 1.12, p < 0.05 large effect); User-reported accuracy satisfaction improved from 3.2 to 4.2 (Pearson’s r = 0.68, p < 0.01).

The stratified analysis demonstrated DKG’s adaptive capacity across learner cohorts (Table 3):

Table 3.

Multidimensional analysis of learner satisfaction (paired sample t-test, bonferroni correction).

Dimension	Pre-test Mean (SD)	Post-test Mean (SD)	Effect size (r)	95%CI	t-value
Accuracy	3.2 (0.8)	4.2 (0.6)	0.68**	[0.82,1.18]	12.37
Richness	3.0 (1.1)	3.9 (0.7)	0.61*	[0.65,1.13]	9.84
Targeting	3.0 (0.9)	4.1 (0.5)	0.73**	[0.91,1.29]	14.25
Understandability	3.5 (0.7)	4.3 (0.6)	0.64*	[0.71,1.27]	10.92

Based on a 7-point Likert scale (1 = extremely dissatisfied, 7 = extremely satisfied); pre- and post-intervention comparisons for the same learners (n = 760); *p < 0.05, **p < 0.01, ***p < 0.001.

Basic learners exhibited a 49% reduction in preconception errors (β = 0.83, p < 0.01, 95% CI [0.75–0.91]);

​Advanced learners saw cutting-edge paper recommendation F1-scores rise from 0.62 to 0.84 (Δ + 35.5%, p < 0.001).

These results confirm DKG’s ability to dynamically balance algorithmic precision and pedagogical relevance for diverse learners, as evidenced by both system metrics and behavioral outcomes.

Hypothesis H3 validation: dual-objective optimization via CFR criterion

The Cognitive-Friendly Recommendation (CFR) criterion achieved dual optimization of algorithmic precision and system efficiency through cognitive load mediation. Bootstrap-based mediation analysis demonstrated that cross-modal alignment (CMA) significantly improved learning outcomes (total effect = 0.63, 95% CI [0.51, 0.75]), with cognitive load mediating 47.6% of this effect (Table 4). The mediating effect share was calculated using the bias-corrected percentile Bootstrap method^28,29 with 5000 samples. This share is the ratio of the standardized indirect effect (β_a×b = 0.3, 95% CI [0.22, 0.38]) to the standardized total effect (βc = 0.63). The 95% bias-corrected confidence interval for the standardized indirect effect for the cognitive load-mediated pathway was [0.62, 0.94] (excluding 0), confirming its significance. Strikingly, the mediation effect was 51.3% (β = 0.83) in the basic learners —those with limited cognitive resources—versus 38.7% (β = 0.64) in advanced learners. This highlights differential reliance on cognitive regulation pathways.

Table 4.

Decomposition of cognitive load mediation effects (Bootstrap method, 5,000 samples).

Effect Type	Estimate	95% CI	p-value	Proportion (%)
​Total Effect	0.63	[0.51, 0.75]	0.001	100.0
​Direct Effect	0.33	[0.18, 0.48]	0.006	52.4
​Mediation Effect	0.30	[0.22, 0.38]	0.009	47.6

Standardized effects estimates; total effects model R² = 0.76; mediator share = indirect effects/total effects × 100%; Bootstrap confidence intervals are bias-corrected; *p < 0.05, **p < 0.01, ***p < 0.001.

Modular course design (15-minute knowledge units) enhanced the mediation effect to 0.36 (SE = 0.05), a 67.8% improvement over traditional designs, empirically validating personalized pedagogy. Concurrently, the DKG-CMR framework achieved:

​Academic-practical synergy: Theoretical scores rose from 78.3 to 94.1 (Cohen’s d = 3.02, 95% CI [2.75, 3.29], scale:0-100), while practical performance increased from 54.9 to 71.2 (d = 2.94), both exceeding conventional large-effect thresholds (d > 0.8).

​Computational efficiency: 72% reduction in CPU core hours and 41% lower peak memory usage, showcasing Neo4j-PyG integration advantages.

These results confirm that dynamic knowledge graph orchestration enables technical-cognitive co-optimization, redefining design principles for educationally grounded recommender systems.

As shown in Table 5, the DKG-CMR system achieves significant resource consumption reduction through a three-level optimization architecture (lightweight inference-stream processing-cold/hot storage). Specifically, it reduces CPU core hours by 72.4%, peak memory usage by 41%, and P99 latency by 48.6%. These results validate the effectiveness of engineering strategies including knowledge distillation, Kafka sharding, and the Neo4j-MinIO hybrid storage model. The findings provide empirical evidence for efficient deployment of open education platforms.

Table 5.

Comparison of system resource consumption during A/B testing (data collection period: 2023.03-2023.06).

Metrics	Experimental Group	Control Group	Reduction (%)	Cohen’s d (95% CI)
CPU Core Hour/Day	2.4 ± 0.3	8.7 ± 1.1	72.4%	7.81[7.52,8.10]
Memory Peak (GB)	3.2 ± 0.5	5.4 ± 0.7	41%	3.62 [3.45, 3.79]
Response Latency P99 (s)	1.8 ± 0.2	3.5 ± 0.4	48.6%	5.38 [5.16, 5.60]

Mann-Whitney U test; Data is presented as mean ± standard deviation; *p < 0.05, **p < 0.01, ***p < 0.001.

Learner application scenario cases

Based on the empirical findings of the above cognitive-performance synergistic optimization mechanism, the DKG-CMR system shows remarkable adaptability in real learning scenarios. Take “Li Ming”, a mechanical and electrical engineering learner, as a typical example. The system’s technological response logic in dynamic resource interaction is visualized via multi-temporal scenario restoration:

7:30 AM fragmented study scenario of commuting

The system detects that Li Ming is in the mobile device login state and the network environment is stable, based on the scenario adaptation mechanism in the feedback layer (see the Section"Cognitive-friendly recommendation system architecture"for details). Leveraging the feedback layer’s scenario adaptation mechanism (The Section"Cognitive-friendly recommendation system architecture"), the system detects Li Ming’s mobile login and network instability. It then automatically triggers the text summary generation function. The “Principles of Motor Control” chapter content was compressed to 40% of its original length, and key frames were marked in video resources. This enables Li Ming to review core knowledge points during his 25-minute commute, shortening reading time by 56.8% compared to the traditional mode.

Evening 19:00 home system learning scenario

Based on DKG’s course objective constraint mechanism (The Section "Dynamic knowledge graph (DKG) construction process"), the system analyzes the knowledge dependency graph and finds that Li Ming made four circuit conceptual errors in the pre-test of the chapter “Motor Control”. The system then prioritizes and pushes the prerequisite video resources such as “Kirchhoff’s Law” and “Ohm’s Law”, and strengthens the connection of knowledge points through dynamic weight adjustment. This reduced Li Ming’s error rate in the conceptual tests of the subsequent chapters, verifying the top-down updating mechanism’s moderating effect on cognitive load.

21:15 exercise scenario

When the system detects Li Ming has spent over 12 minutes on the “Three-Phase Asynchronous Motor Troubleshooting” exercise and clicked the “Hints” button ≥ 3 times, it triggers the NASA-TLX (NASA Task Load Index) threshold response mechanism in real time (Section. 3.1). The system simplifies the mobile interface to 3 elements per screen and pushes the ‘Motor Winding Inspection’ pre-laboratory video via the knowledge graph. This adjustment reduces Li Ming’s cognitive load score and improves task completion efficiency.

The above data are derived from real user logs (desensitized). Through the two-way mapping of timeline and technology modules, the cognitively friendly adaptability of the DKG-CMR system in multiple scenarios, such as commuting and living at home, is visually presented. This visualization provides concrete practical evidence for the theoretical framework.

In this chapter, the triple hypotheses are tested through empirical studies. The results show that cross-modal alignment reduces cognitive load by 40.5% (H1); Dynamic Knowledge Graph (DKG) technology significantly improves resource matching accuracy by 33.7% (H2); Optimization strategy based on Cognitive Feedback Reconciliation (CFR) achieves both learner performance enhancement (standardized effect size d = 3.02) and resource consumption savings of 72% (H3). The mediation effect analysis further confirmed that the cognitive load reduction contributed 47.6% to the total effect, validating the theoretical mechanism.

Discussion

Theoretical and practical implications

This study redefines cross-modal educational theory through the DKG-CMR framework, which resolves modal isolation by synergizing a BERT-ViT-LSTM encoder with InfoNCE contrastive learning. The framework’s cross-modal alignment module captures semantic relationships between text, video, and behavioral data, achieving ​dual optimization of recommendation accuracy (F1-score = 0.912) and cognitive efficiency (NASA-TLX Δ = 40.5%). Critically, the dynamic knowledge graph (DKG) validates a bidirectional updating paradigm—combining real-time behavioral triggers with daily batch processing—that mediates 47.6% of total learning outcome improvements (β = 0.30, p = 0.009; Table 4). This mechanistic insight advances methodological debates on resource-personalization in MOOCs.

By reducing learners’ weekly resource screening time to 136 min—meeting NSSE-China engagement benchmarks—the DKG-CMR framework demonstrates the scalable potential for adult upskilling initiatives. Its generalizability across disciplines (engineering → healthcare/economics) stems from ontology-agnostic knowledge graphing, enabling open universities to accelerate digital transformation while maintaining pedagogical fidelity (learner satisfaction r = 0.73; Table 3).

Limitations of the study

DKG-CMR significantly improves learning outcomes in electromechanical courses. Specifically, it achieves a 33.7% higher F1-score and reduces NASA-TLX task load by 40.5%. However, its structured knowledge dependency assumption encounters adaptation challenges in humanities scenarios and resource-constrained environments.

First, the humanities domain encounters semantic ambiguity issues. Due to the highly subjective and weakly causal nature of humanities resources (e.g., literary critiques), the mutual information value of cross-modal alignment modules decreases significantly (typically falling below the empirically validated threshold of 0.4), thereby impairing knowledge alignment accuracy. This performance degradation is particularly pronounced in domains characterized by high polysemy, interpretive subjectivity, and lack of clear hierarchical knowledge structures, representing a significant boundary condition for the current model.

Second, the current framework suffers from knowledge graph coverage bias. On one hand, it exhibits implicit infrastructure dependencies—for instance, insufficient support for learners with disabilities and a lack of optimized sign language video feature extraction in speech-text alignment models. On the other hand, low-bandwidth environments cause significant system performance degradation. Specifically, the ViT encoder requires a stable bandwidth of ≥ 2.3 Mbps (identified as a critical performance threshold), which may not be available in rural areas with limited network capacity. Consequently, in high-latency scenarios (> 500 ms, identified as a threshold where cognitive load rebound becomes significant), learners experience a rebound in cognitive load. These infrastructure requirements (bandwidth ≥ 2.3 Mbps, latency ≤ 500 ms) constitute crucial boundary conditions for maintaining optimal system performance and user experience.

Future research directions

This study will subsequently carry out in-depth exploration around three dimensions: interdisciplinary adaptation, educational equity and lifelong learning ecology.

Interdisciplinary knowledge representation model construction

Aiming at the dispersive characteristics of knowledge in humanities, it is proposed to construct a specialized representation model based on Probabilistic Knowledge Graph (PKG). The core design principle leverages Bayesian inference mechanisms to explicitly model uncertainty and confidence in entity relationships (e.g., associating a literary motif like “the moon” with potential themes like “loneliness” or “reunion” with probabilistic weights, rather than binary links). Key challenges include: (1) Acquiring high-quality, domain-specific probabilistic annotations for training and validation, which may be scarce in literary or historical studies; (2) Designing effective prior distributions and likelihood functions within the Bayesian framework that accurately capture the nuances of humanistic interpretation; (3) Integrating probabilistic reasoning efficiently into the real-time recommendation pipeline without excessive computational overhead. Preliminary validation plans involve: (1) Curating a benchmark dataset for humanities cross-modal retrieval and recommendation, focusing on tasks like poetry imagery mapping or multi-perspective historical event analysis; (2) Conducting small-scale pilot studies comparing PKG-based recommendations against the current DKG-CMR and traditional methods within literature or history courses, measuring alignment accuracy (e.g., F1-score for theme/motif matching), recommendation acceptance rates, and learner cognitive load (NASA-TLX). This approach aims to overcome the constraints of traditional deterministic relationship modeling in subjective domains.

Design of education equity enhancement mechanisms

Construct a closed-loop system of “De-biased Knowledge Graph - Compensation Algorithm - Evaluation Matrix - Resource Rebalancing” to enhance education accessibility in three aspects: Adaptation for Special Groups: Develop multimodal interaction solutions (voice-activated logging system, tactile feedback mechanism), and optimize the interface in accordance with the WCAG 2.1 standard. This includes implementing robust voice-activated logging systems for motor-impaired users. Exploring tactile/haptic feedback mechanisms for conveying complex information to visually impaired learners, and optimizing the interface for diverse needs. Crucially, enhance the cross-modal alignment module to include optimized feature extraction and semantic mapping for sign language videos, addressing a current gap.

Optimization of Technologies for Weak Network Areas: Under strict constraints, priority should be given to guaranteeing the realization of functions and core learning needs. For low-income and rural areas, technical optimization can be achieved by implementing aggressive context-aware data compression, such as smarter video keyframe selection and enhanced text summarization. The second is to leverage intelligent local caching and prefetching (utilizing DKG’s ability to predict learner paths). Third, with a full-featured offline recommendation engine that is able to run the core logic locally and synchronize it periodically. However, this process may face some challenges. The main ones include how to keep the relevance of the recommended content as well as the freshness of the knowledge under intermittent network connectivity, while also minimizing the cost.

Cognitive Load Regulation: An enhanced “Cognitive Assistance Mode” needs to be embedded, which is designed to follow the principle of “Active Simplification and Guidance under Pressure”. This mode is activated more aggressively in scenarios with weaker network conditions, or when targeting specific groups of users. Reduce choice overload by simplifying decision-making options. Extend interaction thresholds to reduce time pressure. And leveraging multi-modal interpretation systems - for example, prioritizing the presentation of text or audio summaries over video when bandwidth is low.

Lifelong learning ecosystem dynamic evolution system

Design the knowledge graph dynamic evolution mechanism around the full-cycle path of “Academic Education - Career Development - Elderly Learning”. Focus on structured course modeling in the academic education stage, integrate industry certification resources in the career development stage, and embed ageing cognitive regulation modules in the elderly learning stage. With the help of blockchain technology, a cross-platform credit bank is constructed to realize the credible deposit and consistent application of full-cycle learning data.

The above research directions adhere to a three-level argumentation framework: problem definition, technical path, and quantitative indicators. Some key technologies—such as Professional Knowledge Graph (PKG) construction and lightweight model optimization—build on prior research, ensuring the feasibility and systematicity of subsequent exploration.

The above research directions follow a three-tier argumentation framework. That is, problem definition, technology path, and quantitative metrics. Some key techniques, such as probabilistic knowledge graph (PKG) construction and lightweight model optimization, are based on previous research, which guarantees the feasibility and systematicity of subsequent exploration.

Engineering practice of system deployment

In the engineering practice of system deployment, this research adopts a three-tier architecture to realize efficient data processing and resource management. The lightweight inference engine layer receives Learning Management System (LMS) requests via an API gateway, processes multimodal data using TinyBERT and MobileNetV3 in a dual-encoder architecture, and ensures high inference efficiency through knowledge distillation and model compression. The stream processing middle layer utilizes Kafka cluster and Spark Streaming to achieve load balancing and real-time updating of the knowledge graph to ensure the timeliness and accuracy of data processing. In the data storage layer, Neo4j manages the hot data and MinIO archives the cold data through the hot and cold separation strategy to improve the storage efficiency.

In terms of computing cost optimization, a multi-layer optimization system is constructed. The inference model reduces parameters via layered distillation. A dynamic update mechanism—combining daytime incremental updates with nighttime full batch processing—significantly decreases CPU load. The storage solution employs a Neo4j-MinIO hybrid architecture, significantly optimizing memory utilization. Practical results demonstrate reduced hardware costs, fewer servers and GPUs, and significant cost control benefits.

The scalability guarantee mechanism designs flexible solutions for different application scales. In large-scale scenarios, the “slice-and-relay” architecture realizes load balancing and efficient streaming. In small-scale scenarios, it is simplified to modular expansion and reduces configuration complexity. Under different load conditions, system resource consumption is reasonable and performance is stable.

In terms of heterogeneous platform integration, the adapter system is designed to be compatible with the requirements of multiple scenarios. The API gateway adapts to multiple LMSs, the data intermediary layer effectively reduces the network load, and the integrated privacy protection technology meets the requirements for desensitization of educational data. In addition, lightweight containerized deployment through Docker Compose further enhances the flexibility and adaptability of the system.

Although DKG-CMR achieved a high overall accuracy (F1-score = 0.912), critical failures requiring manual intervention occurred in 1.2% of the sessions (n = 18/1,520). Based on the A/B test logs, we categorized the critical failures into two types: cultural polyglot and device stability, and the following are typical cases. Case 1 is a catastrophic mismatch caused by a cultural context gap. When a learner inquired about “three-phase motor wiring for delta configurations” (text), the system recommended a video on “delta river ecosystem protection”. This is because the system only mapped “delta” to its geographic meaning, but ignored its electrical engineering context. This situation resulted in the learner wasting 12 min watching irrelevant content, reporting a NASA-TLX value of 82, which is well above average, and manually skipping 3 subsequent recommendations. Case 2 is a cascading failure triggered by noisy sensor data. A sudden failure (loss of fixation) of the eye-tracking device caused the system to misdiagnose that the learner was distracted while watching the circuit theory video. In turn, the circuit topic was downgraded to “low priority” and the arithmetic tutorial video was pushed. In the end, learners abandoned the course after receiving 6 consecutive off-topic recommendations, and the course completion rate dropped by 40%. In response to these problems, we have taken corresponding measures. On the one hand, we set up a user feedback mechanism. When a user flags a bad recommendation (e.g., clicking the “Not Relevant” button), the system will trigger a knowledge graph reweighting within 15 min to further optimize the recommendation effect. On the other hand, the system implements the “Behavioral Outliers Removal” mechanism. If the difference in dwell time is > 3σ, the system will immediately pause the knowledge graph update and revert to course-based recommendation.

Conclusion

In summary, this study addresses core challenges in open education resource management by constructing the Dynamic Knowledge Graph Enhanced Cross-Modal Recommendation (DKG-CMR) framework. This framework effectively tackles the two core problems introduced in the introduction. At the resource optimization level, the cross-modal semantic alignment technique reduces the resource screening time by 36%, effectively alleviating the contradiction between resource overload and mismatch of personalized needs. In response to the limitations of the traditional knowledge graph, a two-way dynamic evolution mechanism is innovatively proposed to improve knowledge freshness up to 45 days significantly. Experimental data demonstrate that the framework achieves two key breakthroughs: reducing the cognitive load index and improving course completion rates. This offers empirical evidence for addressing the dilemma of applying Sweller’s cognitive load theory in digital education scenarios. The framework holds significant theoretical and practical implications.

Practical verification shows that the framework features a lightweight design. With a memory footprint of less than 8GB per single node, it demonstrates high efficiency. Moreover, it seamlessly integrates with mainstream Learning Management Systems (LMS), such as Moodle and Canvas, via standardized connections. These characteristics collectively demonstrate its excellent engineering practicability and compatibility.

Looking ahead, the research team will focus on optimizing the adaptability of the DKG-CMR framework in different educational scenarios; Explore its application potential in complex learning environments; Sustain theoretical and technical support for intelligent education, while promoting innovative practices in educational resource management.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

T.S. : Conceptualization, Methodology, Investigation, Writing – original draft, Funding acquisition F.L. : Software development, Data curation, Formal analysis, Visualization, Validation Z.W. : Experiment design, Resources acquisition, Project administration X.Z. : Theoretical framework construction, Writing – review & editing, Supervision All authors contributed to manuscript revision, interpretation of results, and approved the final version.

Data availability

The datasets used in this study are available from the Sichuan Open Education Platform but restrictions apply to public sharing due to institutional data governance policies. De-identified data supporting key findings are available from the corresponding author on reasonable request.

Declarations

Competing interests

The authors declare no competing interests.