Modelling the impact of interactive interface features on user experience in artificial intelligence driven digital learning systems

Abstract

In the evolving online education system, Interface Design (ID) plays a crucial role in facilitating the application of Artificial Intelligence (AI)– driven Digital Learning Systems (DLS). While significant research has commonly analyzed educational methods, the accuracy of quantifying the impact of specific Interactive Interface Features (IIFs) on User Experience (UX) remains underdeveloped. This research presents a complete model of the impact of IIF on UX in AI-driven Digital Learning Systems (DLS). This research study employed a controlled experimental design (n = 240) with a between-subjects method to assess five key interface features: Adaptive Feedback Panels (AFP), Gamification Elements (GE), live Conversational Agents (CA), Progress Visualization (PV), and Micro-Assessment Widgets (MAW). This work designed a multi-layer model that precisely manipulated these features while maintaining experimental control. User interactions were analyzed using a Mixed-Methods Approach (MMA), combining Linear Mixed-Effect Modelling (LMEM) with Machine Learning (ML)–based Feature Selection (FS) techniques. Results show that live CA (β = 5.32, p < 0.001) and Adaptive Feedback Mechanisms (AFM) (β = 4.86, p < 0.001) had the most effective positive impact on system usability, while GE most significantly enhanced user engagement (β = 0.42, p < 0.001). The FS method revealed synergistic effects between CA and Adaptive Feedback (AF) (SHAP interaction value = 0.087). ML validated these empirical results, identifying nonlinear relationships and achieving a predictive R² of 0.849 for the composite UX score. This research developed a robust methodological approach for quantifying IIF impacts and provides empirical proof to guide the design of AI-enhanced educational interfaces that optimize learning experiences.

Introduction

User Experience (UX) in the context of Artificial Intelligence (AI)-driven Digital Learning Systems (DLS) refers to the holistic set of perceptions, emotions, cognitive responses, and behavioral interactions a learner has while engaging with an educational interface. It encompasses dimensions such as usability, engagement, satisfaction, emotional resonance, and cognitive workload, all of which contribute to the learner’s perceived use and acceptance of the DLS. As online education continues to evolve, UX has emerged as a key factor in learning efficacy, particularly in systems that leverage AI to personalize content, adaptively respond to learner input, and scaffold instructional processes in real time. Interactive Interface Features (IIFs) play a key role in shaping UX in AI-driven DLS (Asif, M., 2024; Bairy, V., 2025)^1,2. As educational technology evolves, understanding how specific IIF impacts learning outcomes, engagement, and cognitive load has become increasingly important for researchers (Zou, L. et al., 2025)³. While substantial research has explored the use of DLS broadly, a significant gap remains in quantitative models that can precisely measure and predict how individual IIFs contribute to the overall UX (Adeniyi, I. S. et al., 2024; Rafiq, S. et al., 2024)^4,5.

The rapid advancement of AI in educational contexts has introduced new possibilities for Adaptive Learning Systems (ALS) that dynamically respond to learner needs. These AI-driven DLS can potentially transform educational experiences by personalizing content delivery, providing timely feedback, and optimizing the learning process through intelligent adaptation (Strielkowski, W. et al., 2024)⁶. However, the success of these systems is deeply dependent on their Interface Design (ID)—the key point of interaction between the learner and the primary AI, as noted by Li, X. et al. (2024)⁷.

Prior research in educational technology has shown that IIF design significantly impacts learning outcomes Akintayo, O. T., Eden et al., 2024; Ateş, H., 2025^8,9. Studies have predicted the impact of accurate features, such as Adaptive Feedback Mechanisms (AFM), as described by Lawlor, J. A., et al. (2024)¹⁰, and Gamification Elements (GE) by Jun, M., & Lucas, T. (2025)¹¹, and visualizations by Cai, T. et al., 2024¹². on several features of the learning experience. However, most existing research examines these features in isolation or through qualitative methods that do not support precise quantification of their effects. Moreover, the interaction between multiple IIF—a common scenario in real-world educational platforms—remains understudied, as noted by Wang, J. (2024)¹³.

The emergence of sophisticated AI capabilities in educational systems further complicates ID considerations. Features like Adaptive Feedback Systems (AFS) by Cuéllar et al. 2025¹⁴, intelligent Conversational Agents (CA) by Aravindh & Singh 2025¹⁵, Sanguinetti & Atzori 2024¹⁶, and personalized progress trackers by Shen, 2024¹⁷ develop novel interaction paradigms that traditional ID principles may not fully address. Understanding how these AI-driven features impact UX requires new methodological approaches that can accurately disentangle complex interactions and attribute effects.

This paper addresses these knowledge gaps by developing a comprehensive model that quantifies the impact of precise IIF on UX in AI-driven digital learning environments. This proposed research focuses on five key IIFs commonly implemented in contemporary educational platforms: Adaptive Feedback Panels (AFP), GE, live CA, Progress Visualization (PV), and Micro-Assessment Widgets (MAW). By conducting a controlled experimental study with 240 participants and using a sophisticated multilayer system, this study isolated the effects of these features while capturing both subjective and objective UX measures.

The primary research questions guiding this work are

1. What is the relative impact of different IIFs on key dimensions of UX in AI-driven DLS?

2. How can I use IIF to enhance or reduce UX results?

3. To what extent can the impacts of IIF be modelled and predicted using parametric and non-parametric methods?

To answer these questions, this work employed a Multi-Method Approach (MMA) that combined Linear Mixed-Effects Modelling (LMEM) with advanced Machine Learning (ML) to validate findings and explore non-linear relationships. By triangulating results across methods, this study provides robust evidence for the vital impact of IIF on UX while accounting for individual differences among learners.

This research makes several key contributions to the field

1. First, it develops a quantitative model to assess the impact of IIF on educational technology, enabling evidence-based design decisions.

2. Second, it proves the relative importance of different features, guiding resource allocation in educational technology development.

3. Third, it reveals essential IIF, highlighting synergistic combinations that maximize positive UX.

4. Ultimately, it develops a methodological approach for assessing feature impact that can be extended to other domains beyond educational technology.

The findings from this study have important implications for educators, instructional designers, educational technology developers, and researchers working at the intersection of AI and education. By understanding which IIFs most significantly impact UX—and how these features interact—stakeholders can make informed decisions about technology implementation that ML results while optimizing resource allocation.

The remainder of this paper is organized as follows: Sect. 2 reviews relevant literature on ID in educational technology, AI in DLS, and UX evaluation. Section 3 presents the proposed theoretical model, which integrates theories from cognitive psychology, educational technology, and Human-Computer Interaction (HCI). Section 4 outlines our methodology, which includes the model, FS, experimental design, and analytical method. Section 5 presents the results of the recommended analysis. This research concludes in Sect. 6 with a discussion of limitations and directions for future research.

Literature review

Integrating AI into educational interfaces has significantly transformed DLS, enhancing the UX through personalization, adaptability, and interactivity. Recent studies have explored many features of this integration, ranging from ALS to AI-driven UX design principles.​.

One notable study by Pham, X. L., et al.¹⁸ introduced a constructivist-based interactive Learning Management System (LMS) to enhance online educational experiences. The system emphasized learner autonomy and engagement, resulting in improved learning outcomes and user satisfaction.​ A study by Winter, T. et al. (2025)¹⁹ examined the application of AI-driven behavioral analysis to create adaptive and personalized user interfaces. The research demonstrated that such interfaces could dynamically respond to user preferences, enhancing engagement and learning efficiency.

In the context of children’s education, Li, M. et al.. (2025)²⁰ designed and evaluated an Interactive Learning System (ILS) leveraging advanced HCI. The system aimed to provide personalized learning experiences for children, generating promising results in terms of usability and educational use. Moreover, a study by Pozdniakov, S. et al. (2024)²¹ investigated the integration of Large Language Models (LLMs) into UIs to provide feedback in educational contexts. The research developed a model for ethically integrating generative AI into educational tools, highlighting its potential for providing personalized feedback.​.

Chen, Y. et al. (2024)²² introduced MindScratch, a visual programming support tool powered by multimodal generative AI. The tool aimed to support classroom learning by providing adaptive assistance for students’ programming projects, thereby enhancing their computational and imaginative intellectual skills. Razaque, A., Hariri, S., & Yoo, J. 2025²³ proposed actionable guidelines for developing AI-driven User Interfaces (UI) that improve DLS, engagement, and skill development. Their study emphasizes integrating AI-driven adaptability with UI design principles to enhance effective learning and cultivate technology-driven skills. ​.

Maity, S., & Deroy, A. (2024)²⁴ delved into integrating Generative AI, particularly LLMs such as GPT-4, into Intelligent Tutoring Systems (ITS) to enhance personalized education through dynamic content generation, real-time feedback, and ALS. They also addressed significant challenges, including ensuring pedagogical accuracy and mitigating inherent biases in AI models. Calo, T., & Maclellan, C. 2024²⁵ introduced generative AI capabilities to assist educators in developing tutor interfaces that meet their requirements while adhering to design principles. Their method leverages LLMs and prompt engineering to generate tutor layouts and content based on educators’ high-level requirements.

Alfredo, R. et al. (2024)²⁶ conducted a systematic literature review to explore concerns and gaps in human-centered learning analytics and AI in education. They analyzed 108 papers to provide insights about the current state of human-centered LA/AIED research, the extent of educational stakeholders’ contributions to the design process, and the balance between human control and computer automation. ​.

Recent research has emphasized the role of AI-enhanced IIF in advancing user experience (UX) and learning results within DLS Strielkowski, W. et al., 2025²⁷ conducted a large-scale bibliometric and content analysis of AI-driven adaptive learning systems, highlighting the use of real-time personalization approaches—particularly AFM—in supporting individualized learning and reducing extraneous cognitive load. Their findings underscore the centrality of adaptive scaffolding in promoting sustainable educational transformation and enhancing learner engagement across numerous contexts.

In the domain of motivational ID, Saputra et al. (2025²⁸ presented a bibliometric review of gamification in science education from 2019 to 2023, identifying the increasing prevalence and pedagogical use of features such as experience point systems, achievement badges, and leaderboards. These elements were associated with improvements in learner motivation, affective engagement, and persistence across diverse instructional settings.

Yang, C. C., Wu, J. Y., & Ogata, H. (2025)²⁹ examined the influence of PV tools in e–book–based blended learning environments. Their study verified that real-time learning analytics dashboards enhanced students’ self-regulated learning behaviors, including goal-setting, task monitoring, and time management. The use of visual progress indicators was positively correlated with learning performance and perceived usability.

Additionally, Shivshankar, S., & Acharya, N. 2025³⁰ explored the integration of artificial intelligence in developmental assessment and feedback workflows. They detailed how micro-assessment modules, supported by AI-driven analytics, provide immediate and adaptive feedback, thereby enhancing learner reflection, diagnostic precision, and instructional alignment. Their analysis emphasized the pedagogical relevance of embedded, interactive assessment features for dynamic DLS.

Over the past six years, the research trajectory in AI-enhanced DLS interfaces has verified a clear evolution from isolated feature studies toward integrated, data-driven UX frameworks. Early works focused on static design interventions within LMS or single-feature HCI prototypes, establishing foundational principles for human-computer interaction and user experience in AI-driven systems (Mishra et al., 2025)³¹, while more recent studies increasingly emphasize adaptive intelligence, behavioral analytics, and generative feedback models (Guettala et al., 2024)³². This shift is further reinforced by methodological diversification—from qualitative evaluations and heuristic usability assessments to predictive modelling using AI-based decision engines and learning analytics dashboards (Cabral, 2025; Sultana, 2025; DeLone & McLean, 2003; D’Mello & Graesser, 2012)^33–36. Bibliometric reviews by Fredricks, J. A. et al., 2004; Nielsen, J., 1993^37,38. reveal thematic convergence across studies, particularly in the domains of feedback personalization, motivational design, and performance monitoring, signifying a maturing field with strong alignment between theory and application. Collectively, these studies by Paas, F., Renkl, A., & Sweller, J. 2003; Pekrun, R. et al., 2002; Sauro, J., & Lewis, J. R. 2016; Sun, P. C. et al., 2008; Sweller, J. 1994^39–43 indicate a transition toward composite ID capable of delivering real-time, context-aware, and personalized learning experiences.

Theoretical model and hypotheses

This study is grounded in an integrative theoretical model synthesizing cognitive-affective models of UX with IIF design principles commonly applied in DLS. When embedded in an AI-driven instructional platform, the model elucidates how specific IIFs impact key empirical hypotheses, including engagement, usability, satisfaction, cognitive load, and emotional response. These constructs, which have been consistently validated in educational and HCI literature, serve as the initial dimensions through which interface efficacy is operationalized and empirically tested.

This study employs an integrative theoretical model combining cognitive-affective UX theories with IIF design principles. Alternative models were considered but proved insufficient for our multi-dimensional research objectives. The Technology Acceptance Model (TAM) focuses on adoption intentions rather than real-time user experience during learning interactions. Cognitive Load Theory (CLT) alone cannot account for affective responses to gamification or social presence impacts of CA. Self-Determination Theory (SDT) addresses motivation but lacks models for measuring interface usability and behavioral efficiency metrics. The integrative method was necessary because AI-driven educational interfaces operate simultaneously across cognitive (mental effort), affective (engagement, satisfaction), and behavioral (task efficiency) dimensions. No single theoretical model effectively captures the multifaceted nature of user interactions with intelligent educational systems, which require real-time adaptation and personalization.

UX modelling hypotheses

In this study, UX in AI-enabled DLS is conceptualized as a multidimensional construct, illustrated through models developed in Human-Computer Interaction (HCI), educational psychology, and the learning sciences. The five constituent dimensions are Engagement, Usability, Satisfaction, Cognitive Load, and Emotional Response, each supported by prior research.

Engagement is defined as the learner’s investment of cognitive, emotional, and temporal resources in learning activities. It encompasses behavioral involvement, attentional focus, and intrinsic interest, and is influenced by autonomy, interactivity, and novelty (Zaharias, P., & Poylymenakou, A., 2009; Black, P., & Wiliam, D., 1998)^44,45.

Usability, as defined by Dillenbourg (1999)⁴⁶, refers to the perceived ease of interacting with a system and is typically operationalized by features such as interface clarity, system responsiveness, and task efficiency. In the educational context, high usability reduces cognitive friction and enhances the learning flow (Locke, E. A., & Latham, G. P., 2002)⁴⁷.

Satisfaction represents the affective evaluation of the learning experience and reflects learners’ perceived value, enjoyment, and alignment with instructional goals (Narciss, S., 2008; Nicol, D. J., & Macfarlane-Dick, D., 2006)^48,49. It is generally regarded as a composite result impacted by content quality and interface functionality.

Cognitive Load, derived from Cognitive Load Theory by Roschelle, J., & Teasley, S. D. (1995); Sedrakyan, G., De Weerdt, J., & Snoeck, M. (2020)^50,51, pertains to the mental effort required to process content while navigating the learning interface. Of particular interest in this study is extraneous load, which can be directly modified through ID choices.

Emotional Response encompasses transient affective states, such as curiosity, confusion, frustration, and enjoyment, which mediate motivation and persistence (Shute, V. J., 2008; Van der Kleij et al., 2015; Veletsianos, G., & Russell, G. S., 2013)^52–54. These responses are significant in adaptive learning systems, where emotional engagement can modulate the learner’s trajectory and system adaptivity, as noted by Vygotsky (1978)⁵⁵.

These five dimensions, grounded in empirical and theoretical literature by Yang, C. C., Wu, J. Y., & Ogata, H. 2025; Bloom, B. S. 1956^56,57, provide a comprehensive basis for modeling and analyzing the impact of IIF design on learner experience in AI-enhanced educational systems.

IIF classifications

The selection of IIF classes in this study is informed by prior literature in Intelligent Tutoring Systems (ITS), adaptive user interface design, and learning analytics. Each of the 5 classes represents a commonly studied and practically relevant component in modern educational technologies, supported by empirical research signifying their use in digital learning environments.

• Adaptive Feedback Panels (AFP) provide automated, system-generated prompts tailored to real-time learner performance. Such adaptive feedback has been shown to enhance learning efficiency, reduce extraneous cognitive load, and guide metacognitive regulation (Villegas-Ch et al., 2025⁵⁸; Stamper et al., 2024⁵⁹). Recent advances in intelligent tutoring systems demonstrate that personalized feedback mechanisms significantly improve problem-solving abilities and learning outcomes across diverse educational contexts.

• Progress Visualization (PV) encompasses interface elements such as progress bars, status trackers, and completion indicators. These have been found to increase perceived control, foster self-regulation, and improve goal alignment (De Vreugd et al., 2024⁶⁰; Kaliisa et al., 2024⁶¹). Learning analytics dashboards that visualize progress support self-regulated learning behaviors such as goal-setting, task monitoring, and time management, though their impact on academic achievement requires careful design consideration.

• Conversational Agents (CA), typically implemented as interactive avatars or chatbots, simulate social presence and provide real-time dialogue support. They are associated with increased engagement, emotional connection, and learning gains (Al-Zahrani, 2025⁶²; Kim et al., 2025⁶³). Research demonstrates that conversational agents with appropriate design features enhance perceived humanness and cognitive engagement in educational contexts.

• Micro-Assessment Widgets (MAW) represent embedded, low-stakes diagnostic tasks, such as in-line quizzes or concept checks. Research suggests that such formative assessments support retrieval practice, diagnostic feedback, and personalized learning (Tordet & Jamet, 2025⁶⁴; Yusof et al., 2024⁶⁵). Studies show that embedded quizzes enhance student engagement and improve knowledge retention when integrated with timely feedback mechanisms.

• Collaborative Widgets (CW) facilitate peer interaction through shared spaces, such as discussion threads, collaborative editors, or real-time polling. These features promote social learning, co-regulation, and affective engagement (Bach & Thiel, 2024⁶⁶; Zheng et al., 2024⁶⁷). Computer-supported collaborative learning environments enable students to co-regulate learning processes, engage in knowledge building, and achieve better group performance outcomes.

The inclusion of these features, grounded in theory and empirical evidence, enables a structured examination of their individual and combined impacts on UX. Their independent activation within the experimental interface design further allows for granular modelling of feature-level contributions to UX.

Hypothesis formulation

The following hypotheses were formulated based on the theoretical alignment between UX hypotheses and interface affordances, guiding empirical investigation. Each hypothesis is expressed in a direction indicating predictable changes in UX metrics contingent upon the activation of a feature.

• H1: Including AFS will significantly reduce perceived cognitive load, as measured by the NASA-TLX index, by externalizing task support and minimizing trial-and-error effort.

• H2: PV cues will enhance system usability and satisfaction scores by presenting a clear navigational model and enabling learners to monitor progress effectively.

• H3: Interactive avatars will increase user engagement and prompt more positive emotional responses, owing to the heightened perception of social presence and responsiveness.

• H4: Embedding micro-assessments within content modules will increase engagement and task completion efficiency, as learners are encouraged to interact more actively with the instructional material.

• H5: Integrating collaborative widgets will improve satisfaction and emotional response scores by promoting perceived social connection and learner autonomy in navigating content using shared actions or perspectives.

Each hypothesis targets a unique feature-dimension pairing and is tested using the multivariate modelling and attribution models presented in Sect. 4.5. Collectively, these hypotheses constitute a formal instantiation of the theoretical model proposed in this study, enabling a structured evaluation of how ID selections shape learners’ cognitive and affective experiences in intelligent DLS.

Methodology

Ethics statement

This study was tested in full compliance with institutional and national ethical standards governing research involving human subjects. Before the commencement of data collection, the research protocol received formal approval from the Ethics Committee of the Science and Technology Research Project of Jiangxi Provincial Department of Education in 2024 (Project Name: Research on Optimization of Network Learning Platform Based on Machine Learning Algorithm, Project No.: GJJ2402621. The procedures followed in this study conformed to the principles outlined in the Declaration of Nanchang Institute of Technology, Nanchang, Jiangxi Province, 330099, China, and adhered to all applicable guidelines and regulatory models concerning research involving human participants.

System architecture

The AI-driven DLS (Fig. 1) designed for this study was architected as a multilayered, modular platform to enable real-time, adaptive interaction between learners and educational content, with the explicit objective of isolating and quantifying the impact of IIF on UX. Follows a service-oriented model, supporting isolated development, controlled experimentation, and seamless orchestration of AI-driven pedagogical components and ID elements. Its implementation comprises five principal layers: Interaction, Orchestration, AI Adaptation Layer, Learning Content Layer, and Data Infrastructure.

Fig. 1.

Proposed model diagram.

Interaction layer: feature-centric frontend environment

The interaction layer represents the user-facing subsystem, built with React.js and Redux state management, enabling deterministic rendering of interface states across various experimental settings. Each IIF element — such as the AFP, GE, PV, real-time hint generators, and voice-assisted avatars — is encapsulated within its React component. These components are instantiated according to a feature toggle protocol defined in a central control schema, enabling fine-grained activation and monitoring of specific UI affordances. To support high-resolution behavioral tracking, every user interaction, including mouse events, keystrokes, navigation decisions, and idle intervals, is timestamped and transmitted via bi-directional WebSocket communication to the backend telemetry processor in real-time.

Orchestration layer: adaptive middleware and routing gateway

A dedicated orchestration layer was developed to mediate synchronous data exchange between the frontend and backend modules. Qiao implements this middleware, using Apollo Server to manage a GraphQL-based API Gateway, which facilitates structured, schema-driven queries for learner state, content configuration, and interface behavior metadata. This layer also enforces session authentication using OAuth 2.0, manages user-level authorization for experimental group allocation, and applies edge caching of frequently requested resources using Redis. Throttling logic is embedded within the API resolver functions to stabilize throughput under variable concurrency loads during experimental runs.

AI adaptation layer: learning-driven behavioral modelling and feedback control

The AI adaptation layer constitutes the cognitive core of the system, responsible for real-time learner modeling and executing adaptive feedback policies. It integrates three submodules: (1) Cognitive State Estimator, (2) Behavioral Pattern Miner, and (3) Feedback Decision Engine.

The Cognitive State Estimator employs a Deep Knowledge Tracing (DKT) model implemented with Long Short-Term Memory (LSTM) in PyTorch to estimate skill mastery from sequential learner-item interaction histories. At each time step , the model outputs a latent skill mastery vector , where is the number of latent knowledge dimensions extracted from curriculum-level factor analysis. These vectors condition the selection of the following learning task and adaptive support level.

The Behavioral Pattern Miner is a Random Forest (RF) classifier trained on 48 features derived from interaction logs, including average dwell time per item, frequency of retries, hint invocation counts, scrolling depth, and time spent on non-interactive regions. The classifier assigns the learner to one of four behavioral engagement classes: Focused, Distracted, Overloaded, or Uncertain. These classes map to UI response policies (e.g., suppress distractions, trigger guidance, and offer scaffolded content).

The Feedback Decision Engine operates a Proximal Policy Optimization (PPO)-based Reinforcement Learning (RL) agent, utilizing the Stable Baselines model. The agent receives a composite state input vector composed of the current mastery vector , behavioral class label (one-hot encoded), and a sequence of recent IIF.

The agent selects a feedback action from a multidimensional action space comprising:

• Feedback Modality: Textual hints, image-based scaffolds, synthesized speech prompts.

• Timing Approach: Immediate, delayed, suppressed.

• Feedback Intensity: Directive, suggestive, reflective.

The reward function is defined as a weighted sum of three components: ; where: →The task success score, calculated as the correctness of the learner’s response ( correct, incorrect), adjusted for time-on-task; i→ The predicted cognitive load, derived from a Linear Regression (LR) trained on self-reported NASA-TLX scores and real-time behavioral proxies (e.g., input latency, help requests, interaction entropy); → The user satisfaction rating, inferred from a lightweight in-session Likert-scale feedback popup (15 scale) presented after task clusters, and further modelled using a fine-tuned MLP regressor on past satisfaction traces; The coefficients are empirically optimized using grid search on validation data (default: 0.4, . All models are modularized and containerized with Docker, orchestrated with Kubernetes, and horizontally autoscaled based on concurrent learner sessions. Low-latency data transfer between microservices is achieved using gRPC channels.

Learning content layer: adaptive curriculum and delivery system

The learning content layer is built upon a modular Learning Object Repository (LOR) compliant with the SCORM 2004 4th Edition standard. Content units are tagged with semantic metadata, including cognitive level (Bloom’s taxonomy), computed difficulty, and prerequisite relationships, enabling rule-based and model-driven sequencing. A Curriculum Orchestrator processes the AI layer’s output and assembles a real-time learning path that adapts to the learner’s mastery profile and experimental behavior.

Content delivery is handled through a stateless rendering engine that hypothesizes personalized interfaces from component templates, ensuring that interface conditions can be precisely virtualized across trials for evaluation. To ensure experimental control, content presentation is dynamically frozen or unlocked based on engagement state thresholds, thereby avoiding contamination of UX results by interface inconsistencies.

Data infrastructure layer: telemetry, analytics, and privacy compliance

Comprehensive telemetry logging is implemented using an Apache Kafka streaming platform, with each user session assigned to a unique topic partition. This facilitates granular replay and segmentation of learner behavior during post-hoc analysis. Event logs are serialized in Avro format and ingested into a ClickHouse analytical database, which is optimized for high-speed querying and the generation of interactive dashboards.

An anonymization protocol is integrated directly within the data ingestion pipeline. All Personally Identifiable Information (PII) is hashed using SHA-256 with session-specific salts, and the anonymization status is logged in a compliance ledger. This process ensures compliance with the General Data Protection Regulation (GDPR) and the Family Educational Rights and Privacy Act (FERPA). Informed consent is captured at system entry using a dynamic JSON-LD schema, and all data access operations are audited via immutable logs maintained in a MongoDB Atlas Data Lake.

Feature implementation

The IIFs evaluated in this study were selected based on their pedagogical relevance, potential for cognitive engagement, and empirical traceability in DLS. Each feature was independently implemented (Fig. 2) as a modular frontend component, integrated with backend controllers and AI inference pipelines to ensure real-time responsiveness, reproducibility, and experimental traceability. This section delineates the implementation details for five core interactive features under study: AFP, GE, live CA, PV bars, and MAW. Each component has configurable parameters to help isolated experimentation and precise manipulation during trials.

Fig. 2.

IIF model.

AFP

AFM was implemented to deliver personalized guidance based on learner response history and behavioral state. Each feedback panel is instantiated as a React.js component with lifecycle-bound connections to the AI adaptation layer. The panel’s behavior is governed by the Feedback Decision Engine, which selects one of the predefined feedback templates—explanatory hint, motivational nudge, metacognitive prompt, or corrective scaffold—based on the state vector, Eq. (1)

1

where,

• →The latent mastery vector from the DKT module.

• →The current engagement classification (derived from the behavioral classifier).

• →Recent UI action deltas (e.g., time-ontask, error streak).

The feedback templates are rendered using Markdown-based dynamic text populated with context-specific variables (e.g., current question, prior errors, learner goals). Delivery is triggered via an internal event bus, synchronized using Redux middleware. Feedback decisions are streamed via gRPC from the AI engine to maintain low latency, with an average response time of under 150 ms. A caching layer using Redis stores previously allocated feedback variants to avoid redundancy and enable adaptive repetition spacing.

Gamification elements (GE)

Gamification features (Fig. 3) were incorporated to enhance learner motivation and engagement through visual metaphors of success and progression.

Fig. 3.

Gamification elements.

The gamification module is implemented as a composite React component bundle comprising

• XP Progress Bar: Displays real-time accumulation of experience points based on task completions, hint usage, and quiz accuracy.

• Achievement Badges: Triggered by event-driven conditions defined in a finite-state transition model—for example, completing three consecutive tasks without a hint triggers a “Precision Learner” badge.

• Leaderboard Widget: Shows anonymized peer rankings using percentile scores calculated via real-time streaming aggregation in ClickHouse.

The gamification engine subscribes to a Kafka stream of learner events and maps them to point actions using the proposed rule-based scoring matrix. The scoring logic is maintained in an external YAML configuration file, supporting quick experimental fine-tuning. Graphical elements are rendered using D3.js, featuring responsive SVG transitions and accessibility-friendly themes (e.g., color-blind modes and high-contrast schemes).

Learner gamification profiles are stored in a MongoDB collection, maintained with a Time-To-Live (TTL) cleanup policy that isolates gamification sessions for experimental validity.

To further support pedagogical alignment, the GE is designed to integrate seamlessly with adaptive feedback methods.

Feedback types include

• Explanatory Hint: “Remember to isolate the variable before substituting the known values.”

• Motivational Nudge: “Great job staying focused! You’re only two steps away from completing this activity.”

• Metacognitive Prompt: “How sure are you about this answer? Try to explain it in your own words.”

• Corrective Scaffold: “Let’s revisit the concept of conservation of energy—review the diagram before attempting again.”

These feedback modalities are dynamically triggered by the AI adaptation layer based on real-time engagement classification and task success metrics. They are delivered through AFP to maintain contextual relevance and minimize cognitive disruption.

Live conversational agent

The CA was a lightweight virtual assistant that answered learner queries in real time. Li developed the frontend chat interface, using the React Chatbot Kit, which is customized for domain-specific dialogues. The agent’s backend is driven by a fine-tuned DistilBERT trained on a corpus of curriculum-aligned QA pairs, with intent classification added using spaCy.

A dual-path pipeline processes user input, encompassing both intent recognition and named entity extraction. Depending on the intent (e.g., clarifying the concept, requesting a hint, checking progress), the agent dispatches API calls to the relevant service endpoint (AI engine, LCMS, or learner state monitor). Responses are synthesized from structured templates enriched with retrieved metadata (e.g., the last attempted skill and the next scheduled activity).

The assistant also supports voice input via the Web Speech API, with real-time transcription followed by Natural Language Processing (NLP). All conversation logs are stored in an encrypted format using the Advanced Encryption Standard with a 256-bit key (AES-256) and anonymized for post-hoc analysis of the ID.

PV bars

Progress indicators are implemented to reflect linear and mastery-based progression. Two distinct types were embedded:

• Linear Progress Bar: Reflects completion percentage across content modules.

• Mastery Ring Graph: Displays inferred control across skill clusters using the output From the DKT engine.

Both visualizations are rendered using Chart.js and dynamically updated through WebSocket events that listen for task completion and feedback cycle outcomes. The control ring is updated by querying the AI adaptation layer via GraphQL, with refresh throttling applied to limit requests to one update per 10-second interval.

User interaction with progress components (e.g., hover, click, tooltip engagement) is followed to evaluate their correlation with perceived control and learning self-efficacy.

MAW

Micro-assessments are short, embedded quizzes designed to capture real-time learning gain and reinforce knowledge retention.

Each widget is built as a standalone React component encapsulating

• Item Bank API Interface: Fetches context-appropriate questions using RESTful calls to the LCMS.

• Response Capture and Feedback Hook: Intercepts user answers and passes them to the AI engine for real-time scoring and state update.

• Confidence Slider: An optional self-assessment component for metacognitive tracking is attached to each question.

The widget supports multiple item types, including MCQ, fill-in-the-blank, ordering, and hotspot, and renders them using an internal Domain-Specific Language (DSL) for pedagogical expressiveness. Scoring is instant, and updates are pipelined into the AI engine for adaptation.

All interaction logs, including question ID, response latency, FS, and confidence level, are batched and sent to the Kafka analytics pipeline, each with a unique interaction UUID for session traceability.

User study design

A controlled experimental study was conducted with a structured design to evaluate the impact of specific IIF on UX in AI-driven DLS. The study was grounded in rigorous experimental protocols that supported causal implications by minimizing confounding variables and ensuring internal validity through systematic manipulation of IIF sets.

Participants and sampling methods

240 participants (aged 18–25 years; Mean Age = 21.1 Years, SD = 1.8) were recruited from three public universities in eastern and southern China. Eligibility criteria included (1) current undergraduate enrolment, (2) prior exposure to at least one DLS, and (3) no reported visual, auditory, or cognitive impairments. Participants were stratified by academic major (STEM vs. non-STEM) and digital proficiency (measured using a standardized Digital Literacy Scale) to ensure balanced representation across experimental groups. A stratified random sampling technique was employed to mitigate sampling bias, with strata defined by discipline and self-reported prior experience with ALS. Written informed consent was obtained from all participants before enrollment, and ethical approval was granted by the Institutional Review Boards (IRBs) of all participating institutions. The sample size of 240 participants was determined by a power analysis using G×Power 3.1.9.7 for a between-subjects ANOVA with six groups. With a predicted medium effect size (f = 0.25), α = 0.05, and desired power = 0.80, the minimum required sample size was 216 participants. This study enrolled 240 (40 per group) to account for probable data loss and ensure adequate power for interaction-effect analysis⁶⁸.

Experimental conditions

The experimental protocol exposed participants to one of 6 distinct IDs, each incorporating a unique combination of the five interactive features: Adaptive Feedback (AF), GE, live CA, PV, and MAW. Table 1 below outlines the 6 interface conditions:

Table 1.

Interface conditions.

Group	Feedback	Gamification	Chatbot	Progress bar	Micro-assessment
G1	✓	✗	✗	✗	✓
G2	✓	✓	✗	✓	✗
G3	✗	✓	✓	✓	✓
G4	✓	✓	✓	✓	✓
G5	✗	✗	✗	✗	✗
G6	✓	✗	✓	✗	✓

• G5 (Control Group) received a minimal interface with no interactive features enabled beyond basic navigation and static content delivery.

• G4 (Full Feature Group) received the complete interface with all five features active.

• The other four groups represent partial configurations, enabling the evaluation of individual and interaction effects of key features.

All groups received identical instructional content, duration (30 min), and assessment tasks to ensure comparability.

Study model

The study byemployed a between-subjects experimental design, in which each participant was randomly assigned to a single interface condition and interacted with the system during a single session. This design was selected to prevent inheritance effects, learning bias, or expectancy effects that may arise in within-subject designs involving repeated exposures to different interfaces.

Each experimental session consisted of the following phases

1. Pre-Test Phase: A baseline measurement of domain knowledge was conducted using a 10-item multiple-choice test. Additionally, demographic surveys and self-reported digital familiarity metrics were collected.

2. Interaction Phase: Participants engaged with the assigned interface condition for a fixed 30-minute session, during which behavioral interaction logs were continuously recorded.

3. Post-Test Phase: Immediately following the session, participants completed:

   • A 10-item domain knowledge post-test (parallel to the pre-test),
   • The System Usability Scale (SUS) for perceived usability,
   • The User Engagement Scale (UES) was adapted for online education,
   • A modified NASA-TLX to measure perceived cognitive load.

4. Follow-up Questionnaire: Participants completed a subjective satisfaction and feature-preference survey using a 5-point Likert scale to rate interface components, perceived usefulness, and ease of use.

All experimental sessions were conducted in university computer labs with standardized lighting and noise isolation, and with identical hardware setups to control for environmental factors. A trained implementer supervised each session and remained non-interventive except for technical problems.

Randomization and blinding

Randomization was executed using a computer-generated random assignment algorithm implemented in Python (numpy.random.choice with equal weights). Group allocation was hidden from the facilitator, and participants were not informed of their group identity to reduce prediction bias. Additionally, result scoring for all pre- and post-knowledge tests was automated, and subjective tests were anonymized before statistical processing. Table 2 summarizes participant features across groups, validating that randomization achieved demographic balance.

Table 2.

Participant demographics and stratification.

Features	G1	G2	G3	G4	G5	G6	Total
Number of participants	40	40	40	40	40	40	240
Mean age (years)	21.0	20.9	21.3	21.2	20.8	21.1	21.1
% STEM majors	52%	55%	50%	54%	53%	51%	52.5%
Mean digital literacy score	82.1	81.6	82.4	82.0	81.3	81.9	81.9

Digital literacy is measured using a 100-point standardized scale.

UX metrics and instruments

A multi-dimensional assessment model was employed to evaluate the impact of IIF on UX, integrating subjective self-reports and objective behavioral indicators. This model was designed to capture the cognitive, emotional, and usability-related features of learner experience within the AI-driven DLS. The selection of metrics was informed by established literature in HCI, learning analytics, and educational psychology, and prioritized validated instruments with high internal consistency and cross-domain applicability.

Subjective UX measurement instruments

Following the interaction, a battery of standardized, psychometrically validated tools was administered to capture users’ perceptions of the interface across four primary dimensions: usability, engagement, cognitive load, and satisfaction.

1. SUS: Usability was assessed using the SUS, a 10-item instrument commonly recognized for its reliability in evaluating interactive systems. Each item is rated on a 5-point Likert scale (1 = Strongly Disagree, 5 = Strongly Agree), resulting in a composite score ranging from 0 to 100. The SUS is particularly suited to unique usability comparisons across experimental conditions due to its sensitivity and simplicity. The internal consistency of the SUS in this study sample yielded a Cronbach’s alpha of 0.89.

2. UES – Short Form: The UES – Short Form was used to measure emotional and cognitive engagement; it comprises 12 items across four subdimensions: focused attention, perceived usability, visual request, and reward. Each item is valued on a 5-point Likert scale. The total and subscale scores provide a granular understanding of how different IIFs modulate engagement. The scale’s psychometric robustness was reaffirmed in this study with a Cronbach’s alpha of 0.92 for the total score.

3. NASA Task Load Index (NASA-TLX): Perceived cognitive workload was measured using the NASA Task Load Index, which evaluates 6 workload dimensions: mental demand, physical demand, temporal demand, performance, effort, and frustration. Each dimension was rated on a 20-point Likert scale via an interactive slider interface. Participants also conducted pairwise comparisons to assess the relative importance of each factor, yielding a weighted overall workload score. This multidimensional model was required to evaluate the cognitive problem associated with each IIF.

4. Feature Satisfaction and Utility Survey: A custom-developed feature-level satisfaction inventory was administered to collect granular feedback on the perceived usefulness, intuitiveness, and responsiveness of individual interface components.

For each feature, participants rated

• Perceived helpfulness (1–5),

• Ease of interaction (1–5),

• Disruption to task flow (1–5, reverse scored),

• Willingness to use in future sessions (1–5).

• Items were subjected to exploratory factor analysis, and scores were aggregated for cross-condition comparison. The tool achieved an internal consistency of α = 0.87 across all items.

Objective behavioral metrics

In parallel with subjective evaluations, behavioral metrics were continuously recorded during interaction using client-side logging mechanisms and server-side telemetry processors. These metrics served as evident proxies for UX components and were critical for triangulating psychometric results.

1. Task Completion Time (TCT): Let , →the timestamps for the start and end of task , then the task completion time is Eq. (2)

   2
   The average TCT for a user across all tasks is Eq. (3)

   3

   The raw TCT values were normalized using z-scores relative to task-difficulty distributions from a baseline cohort to ensure comparability across groups.

2. Hint Utilization Rate (HUR): Hint usage was logged as a binary event per content segment. Let represent whether a hint was used for content segment , and be the sum of segments completed, Eq. (4)

   4

   A high HUR may indicate greater perceived difficulty or interface ambiguity, and it was analyzed alongside task accuracy.

3. Navigation Entropy (NE): This metric quantifies ID using a Shannon entropy-based formulation over page transition probabilities, Eq. (5)

   5

   where,

   →the normalized transition probability of interfacing component .

   Higher entropy proposes more exploratory, potentially disoriented navigation behavior; lower entropy may indicate focused linear progression.

4. Dwell Time per Feature (DTF): Let , →The time the user enters and leaves the interface component . Then, dwell time on feature during session is assumed by Eq. (6)

   6

   where,

   →the number of distinct interaction sessions with component .

   All DTF values were normalized over session duration to control for inter-user variability.

5. Input Latency and Correction Events :

   • Input Latency (IL): For text-based tasks, the mean input latency per keypress is computed as:

     7
     where,
   • →the timestamp of the keypress and is the total number of keystrokes.
   • Correction events (CE): Let be the sum of backspace keypresses in a task:

     8

Both metrics were jointly analyzed as indicators of cognitive hesitation or uncertainty, particularly in response to open-ended questions or natural language prompts.

Experimental results

Descriptive statistics

Analysis of participant features across experimental conditions confirms the use of our randomization procedure. Table 3 presents demographic data and baseline measures for all 6 experimental groups, which represent different IIF combinations: G1 (AF + MAW), G2 (AF + GE + PV), G3 (GE + Live CA + PV + MAW), G4 (All Features), G5 (Control - No Features), and G6 (AF + Live CA + MAW).

Table 3.

Participant features by experimental group.

Features	G1 (n = 40)	G2 (n = 40)	G3 (n = 40)	G4 (n = 40)	G5 (n = 40)	G6 (n = 40)	F/χ²	p-value
Demographics
Age (years), M ± SD	21.0 ± 1.8	20.9 ± 1.7	21.3 ± 1.9	21.2 ± 2.0	20.8 ± 1.6	21.1 ± 1.8	0.56	0.734
Gender, n (%)							4.21	0.937
Female	23 (57.5)	22 (55.0)	24 (60.0)	23 (57.5)	21 (52.5)	19 (47.5)
Male	17 (42.5)	18 (45.0)	16 (40)	17 (42.5)	19 (47.5)	21 (52.5)
Academic area, n (%)							3.12	0.995
STEM	21 (52.5)	22 (55.0)	20 (50.0)	22 (55.0)	21 (52.5)	20 (50.0)
Social sciences	12 (30.0)	11 (27.5)	13 (32.5)	12 (30.0)	12 (30.0)	13 (32.5)
Humanities	5 (12.5)	5 (12.5)	5 (12.5)	4 (10.0)	5 (12.5)	5 (12.5)
Interdisciplinary	2 (5.0)	2 (5.0)	2 (5.0)	2 (5.0)	2 (5.0)	2 (5.0)
Baseline measures
Digital Literacy Score (0–100), M ± SD	82.1 ± 9.8	81.6 ± 10.3	82.4 ± 9.5	82.0 ± 9.7	81.3 ± 10.1	81.9 ± 9.6	0.08	0.995
Prior experience with digital learning (1–7), M ± SD	4.8 ± 1.2	4.7 ± 1.3	4.9 ± 1.1	4.8 ± 1.2	4.6 ± 1.3	4.7 ± 1.2	0.42	0.834
Pre-test knowledge score (0–10), M ± SD	4.3 ± 1.5	4.2 ± 1.6	4.4 ± 1.4	4.3 ± 1.5	4.2 ± 1.6	4.3 ± 1.5	0.14	0.983

The sample consisted of 240 participants with a mean age of approximately 21 years (Range: 20.8 ± 1.6 to 21.3 ± 1.9) across groups. The age distribution showed no significant differences between conditions (one-way ANOVA: F = 0.56, p = 0.734), ensuring comparability across groups. The gender distribution was balanced across experimental groups (χ² = 4.21, p = 0.937), with females comprising 47.5%−60% of participants in each condition. Academic disciplines were similarly distributed, with STEM majors representing approximately half of the sample across all conditions (range: 50.0% to 55.0%), followed by social sciences (27.5% to 32.5%), humanities (10.0% to 12.5%), and interdisciplinary studies (5.0% in all conditions).

Notably, baseline measures revealed no significant pre-existing differences between groups that might confound the experimental results. Digital literacy scores were consistently high across groups (81.3 ± 10.1 to 82.4 ± 9.5 on a 100-point scale; F = 0.08, p = 0.995), indicating that participants possessed sufficient technological skills to interact with the experimental platform. Prior experience with DLS was moderate and comparable across conditions (4.6 ± 1.3 to 4.9 ± 1.1 on a 7-point scale; F = 0.42, p = 0.834). Pre-test knowledge scores revealed equivalent baseline domain knowledge (Range: 4.2 ± 1.6 to 4.4 ± 1.4 on a 10-point scale; F = 0.14, p = 0.983), providing a valid basis for measuring learning gains.

The absence of statistically significant differences in demographic features and baseline measures (all p-values > 0.05) confirms successful randomization. It establishes the internal validity required to attribute subsequent differences in UX results to the experimental manipulations rather than to pre-existing group differences.

Analysis of UX metrics across experimental groups

SUS scores (Fig. 4a) varied significantly across experimental conditions (F(5,234) = 12.36, p < 0.001, η² = 0.209), indicating a large effect size. Group G4 (All Features) reported the highest usability rating (84.6 ± 8.7), substantially surpassing the industry benchmark of 68. This was followed by G3 (79.2 ± 9.8) and G6 (78.1 ± 10.5), both of which included the live Conversational Agent (CA). In contrast, the control group (G5) recorded the lowest SUS score (68.5 ± 12.8), indicating that the absence of IIF significantly impaired the perceived usability of the system. User Engagement, measured using the UES (Fig. 4b), showed an even more pronounced effect (F(5,234) = 21.48, p < 0.001, η² = 0.314). Group G4 again scored highest (4.33 ± 0.49), followed by G3 (4.11 ± 0.52), whereas G5 scored markedly lower (3.21 ± 0.71). This trend mentions an increasing positive impact of multiple IIFs on learners’ cognitive and emotional engagement.

Fig. 4.

Subjective measures for (a) system usability and (b) user engagement measures.

Subscale analysis of UES (Fig. 5) provided more granular insights. Focused Attention differed significantly across groups (F(5,234) = 17.92, p < 0.001, η² = 0.277), with G4 and G3 leading, particularly those with GE and CA components. Aesthetic Appeal also showed significant variation (F(5,234) = 21.06, p < 0.001, η² = 0.310), with G4 and G3 rated as more visually engaging. The Reward dimension yielded the most pronounced differences (F(5,234) = 24.79, p < 0.001, η² = 0.346), with G4, G3, and G6 outperforming other groups, highlighting the motivational use of GE and responsive interfaces. Cognitive Load, assessed via the NASA-TLX, followed an inverse pattern (F(5,234) = 8.79, p < 0.001, η² = 0.158). G4 reported the lowest cognitive load (36.7 ± 12.5), whereas G5 experienced the highest (52.8 ± 16.7), indicating that well-integrated support features can reduce extraneous mental effort during digital learning tasks. Finally, Feature Satisfaction scores differed significantly (F(5,234) = 23.51, p < 0.001, η² = 0.334). G4 again led with the highest satisfaction rating (4.48 ± 0.47), while G5 showed the lowest (3.26 ± 0.73), supporting that easy ID is perceived as more useful, intuitive, and desirable by users.

Fig. 5.

UES subscales across groups.

Objective behavioral metrics (Fig. 6) were consistent with the subjective findings, revealing statistically significant differences across experimental conditions. Task Completion Time (TCT) differed significantly across groups (F(5,234) = 7.84, p < 0.001, η² = 0.144), indicating a medium-to-large effect size. Group G4 (All Features) achieved the fastest completion time (54.7 ± 14.6 s), whereas G5 (Control) took the longest (72.9 ± 21.5 s). These results recommend that the presence of interactive features—especially in combination—facilitated more efficient task performance by reducing cognitive friction and decision hesitation. Navigation Entropy, which reflects the predictability and coherence of user movement through the interface, also varied significantly across groups (F(5,234) = 9.34, p < 0.001, η² = 0.166). G4 showed the lowest entropy (0.69 ± 0.18), indicating more streamlined, goal-directed navigation, whereas G5 showed the highest entropy (0.95 ± 0.26), consistent with disoriented browsing patterns. These findings highlight the role of features such as PV and Adaptive Feedback (AF) in guiding users’ learning of content.

Fig. 6.

Behavioral metrics across groups.

Hint Utilization Rate (HUR) displayed significant variation (F(5,234) = 6.93, p < 0.001, η² = 0.129). G5 exhibited the highest requirement for system-provided hints (28.7% ± 12.9%), while G4 had the lowest (18.2% ± 8.9%), indicating that intelligently designed feedback and scaffolding features reduce learners’ requirements for additional assistance by enhancing interface transparency and instructional clarity. Input Latency, defined as the average delay between user keypresses in open-ended responses, showed more modest yet significant group differences (F(5,234) = 2.98, p = 0.012, η² = 0.060). G4 again achieved the lowest latency (638 ± 187 ms), whereas G5 had the highest (762 ± 245 ms). This metric may reflect lower hesitation and more confident interaction under supportive interface conditions.

The composite measures presented in Fig. 7 provide a comprehensive test of interface use by integrating indicators of learning performance and holistic user experience.

Fig. 7.

Composite measure analysis.

Learning Gains, calculated as the difference between post-test and pre-test scores, showed substantial variation across the six experimental groups (F(5,234) = 13.87, p < 0.001, η² = 0.229), indicating a large effect size. Group G4 (All Features) recorded the highest learning gain (3.6 ± 1.5 points), while G5 (Control) had the lowest (1.8 ± 1.1 points), representing a twofold difference in knowledge achievement. These results recommend that rich, adaptive, and engaging IIF collectively enhance learning results by improving instructional alignment, cognitive support, and motivational scaffolding.

The UX Composite Score, derived from the standardized aggregation of key UX dimensions—including usability, engagement, cognitive load (inverse), and satisfaction—showed the most significant differences across conditions (F(5,234) = 46.38, p < 0.001, η² = 0.497), indicating a large effect size. G4 once again outperformed all other groups (mean z-score = 0.94 ± 0.51), followed by G3 (0.52 ± 0.58). In contrast, G5 showed a markedly negative overall practice (–0.97 ± 0.74). This pattern confirms that full-featured interface environments not only optimize isolated features of user interaction but generate synergistic UX benefits that translate into measurable cognitive and affective improvements.

Together, these composite outcomes validate the central hypothesis of this study: the integration of multiple interactive features—such as Adaptive Feedback, Gamification, CA, and PV—within AI-driven DLS significantly enhances learning use and user experience quality.

Correlation analysis of UX metrics

Figure 8 presents the correlation matrix of key UX metrics, revealing meaningful relationships among subjective, behavioral, and learning-outcome measures. This analysis provides insight into how different UX features interrelate in AI-driven DLS.

Fig. 8.

Correlation analysis.

The correlation analysis demonstrates strong positive relationships between subjective UX measures. SUS scores displayed a strong positive correlation with User Engagement (r = 0.68, p < 0.001) and Feature Satisfaction (r = 0.71, p < 0.001), indicating that interfaces perceived as more usable also generated higher engagement and satisfaction. Cognitive load (NASA-TLX) showed substantial negative correlations with usability (r = −0.62, p < 0.001) and engagement (r = −0.54, p < 0.001), confirming that more usable interfaces significantly reduced perceived mental effort.

Behavioral metrics demonstrated meaningful associations with subjective measures. Task Completion Time correlated negatively with usability (r = −0.38, p < 0.001) and engagement (r = −0.31, p < 0.001), while showing a moderate positive relationship with cognitive load (r = 0.42, p < 0.001). This pattern recommends that interfaces that support faster task completion are perceived as more usable and engaging while imposing lower cognitive demands. Navigation Entropy, which quantifies the predictability of user navigation patterns, presented similar relationships with usability (r = −0.31, p < 0.001) and cognitive load (r = 0.38, p < 0.001). More coherent navigation patterns (lower entropy) were associated with higher usability ratings and lower cognitive load, signifying that well-designed interfaces promote more directed user behavior.

Interestingly, Dwell Time per Feature showed a weak negative correlation with usability (r = −0.15, p < 0.05) but a positive relationship with engagement (r = 0.28, p < 0.001). This deviation proposes that while longer dwell times may slightly impair perceived efficiency, they may indicate deeper engagement with content features. Learning Gain verified meaningful correlations with multiple UX dimensions. The strongest associations were with User Engagement (r = 0.53, p < 0.001) and System Usability (r = 0.49, p < 0.001), indicating that more engaging and usable interfaces facilitated excellent knowledge acquisition. The negative correlation between Learning Gain and Cognitive Load (r = −0.42, p < 0.001) confirms that reducing mental effort supports improved learning outcomes. Among behavioral metrics, Task Completion Time showed the strongest relationship with Learning Gain (r = −0.36, p < 0.001), signifying that interfaces that support efficient task execution also enhance learning performance.

The UX Composite Score, which integrates multiple UX dimensions, demonstrated robust correlations with User Engagement (r = 0.82, p < 0.001), Feature Satisfaction (r = 0.78, p < 0.001), and System Usability (r = 0.76, p < 0.001). These strong associations validate the composite score as an effective measure of capturing the subjective dimensions of UX. The substantial negative correlation with Cognitive Load (r = −0.73, p < 0.001) confirms the importance of minimizing mental effort in generating positive overall experiences. Among behavioral metrics, Task Completion Time showed the strongest relationship with the UX Composite Score (r = −0.54, p < 0.001), followed by Navigation Entropy (r = −0.38, p < 0.001). The correlation between Learning Gain and the UX Composite Score was also substantial (r = 0.63, p < 0.001), reinforcing the connection between positive UX and improved learning outcomes.

Model performance evaluation

To quantify the impact of IIF on UX outcomes, this study evaluated four modelling approaches: LMM, Interaction Models, Random Forest (RF), and Gradient Boosted Decision Trees (GBDT). The modelling techniques were chosen to balance statistical inference and predictive interpretability. LMM supports mixed-effects hypothesis testing, while RF and GBDT enable feature-level interpretability through model-agnostic metrics, such as SHAP. These models are consistent with recent UX modelling literature. While alternatives such as SVR, ANN, and SEM exist, they were not adopted due to either interpretability limitations or weaker suitability for interface-level feature analysis in heterogeneous experimental designs.

Figure 9 proves the comparative performance of these models across all UX outcomes using three key metrics: coefficient of determination (R²) (Fig. 9a), Root Mean Square Error (RMSE) (Fig. 9b), and Mean Absolute Error (MAE) (Fig. 9c).

Fig. 9.

Performance analysis for (a) R², (b) RMSE, and (c) MAE.

The analysis revealed a consistent pattern of performance improvement from simpler parametric models to more complex, non-parametric methods. GBDT achieved the highest overall predictive performance (Mean R² = 0.651), followed by RF (Mean R² = 0.624), Interaction Models (Mean R² = 0.622), and LMM (Mean R² = 0.574). This progression proposes that UX results in AI-driven DLS have non-linear relationships with IIF that ML better captures.

• (i)Outcome-Specific Model Performance: The UX Composite Score showed the highest predictability across all modelling methods (R² ranging from 0.765 to 0.849), indicating that this integrated measure effectively captures the combined impacts of IIF. In this case, GBDT achieved the best performance (R² = 0.849, RMSE = 0.316, MAE = 0.242), underscoring the value of ensemble methods for modeling complex UX. SUS and UES also showed strong predictive power across models, with R² values consistently above 0.58. For System Usability, Interaction Models performed very well (R² = 0.743), outperforming even the ML. This proposes that explicit modelling of feature interactions is significant for predicting perceived usability. Task Completion Time and Navigation Entropy proved the most challenging to predict, with R² values ranging from 0.419 to 0.576 across models. This lower predictability may reflect the impact of individual user features and behaviors beyond IIF. For Task Completion Time, GBDT demonstrated considerably better performance (R² = 0.576) than other methods, indicating that this behavioral metric has complex, non-linear relationships with IIF.
• (ii)Error Metrics Analysis: The RMSE and MAE values complemented the R² findings while providing additional insights into prediction accuracy. For the UX Composite Score, GBDT achieved the lowest error rates (RMSE = 0.316, MAE = 0.242), representing a substantial improvement over LMM (RMSE = 0.394, MAE = 0.309). This pattern was consistent across most outcomes, with GBDT typically showing the lowest error rates. For NASA-TLX, RF achieved notably lower MAE (5.142) than other methods, despite not having the highest R². This proposes that RF may better predict typical cognitive load responses while being less accurate for extreme values. Similar patterns were observed for Navigation Entropy, where RF presented substantially lower MAE (0.098) despite comparable R² values across models.

Result analysis of cross-model comparison and feature inclusion

To evaluate the robustness, predictive utility, and FS problem of the different analytical approaches used in this study, a cross-model comparison was conducted across three model classes: LMMs, Interaction-Augmented Mixed Models, and Interpretable ML (specifically, RF and GBDT). This comparative model enabled a triangulated analysis of how different computational models estimate the effects of IIF on UX results. The models were evaluated along four key axes: predictive accuracy, model fit quality, log-likelihood, and feature attribution relevance.

• (i)Predictive Accuracy Comparison: Fig. 10a shows that predictive performance—measured by the mean R² across five cross-validation folds—varies significantly across model types and UX results. For the SUS and UES, GBDT achieved the highest accuracy (0.703 and 0.675), outperforming linear and interaction-based formulations. Notably, the GBDT also generated the highest overall accuracy for the UX Composite Score, achieving a cross-validated R² of 0.832. This result proposes that ensemble learning methods are particularly effective at capturing complex, nonlinear interactions among IIFs, which may not be fully accounted for by LLMs. Interaction models generally outperformed their additive linear counterparts across all outcomes, with gains most evident for SUS (0.669 vs. 0.615) and UES (0.631 vs. 0.573). These improvements highlight the importance of accounting for synergistic relationships between features, such as AFM modulated by chatbot presence or GE effects contingent on PV cues. The only marginal exception was the experiential feature of the NASA-TLX, where interaction-based improvements were present but less pronounced.
• (ii)Model Fit: AIC, BIC, and Log-Probability: Model fit quality was further assessed using Akaike Information Criterion (AIC) (Fig. 10b), Bayesian Information Criterion (BIC) (Fig. 10c), and log-probability scores (lower AIC/BIC and higher log-probability indicate better fit) (Fig. 10d). The top-right and bottom panels demonstrate that interaction models consistently generated lower AIC and BIC values than their additive linear counterparts, reflecting superior explanatory power despite increased model complexity. This pattern was most evident in the UES and UX composite outcomes, where the AIC differences between additive and interaction models exceeded 20 points, denoting meaningful improvement in model likelihood under penalty-adjusted complexity constraints.

Fig. 10.

(a) Predictive accuracy, (b) AIC values, (c) BIC values, and (d) Log-probability.

For example, the UX Composite Interaction Model achieved an AIC of 365.29 and BIC of 431.44, compared to the linear model’s AIC of 399.16 and BIC of 432.24. Corresponding log-likelihood scores followed a consistent trend, with interaction models achieving higher maximum probability predictions, indicating that accounting for feature dependencies enables more accurate variance modelling in UX data. These results empirically justify the inclusion of second-order interaction terms in the modelling context and indicate that learner responses to IIF are not simply additive but contextually mediated.

Result analysis of LMM

The fixed-effect prediction from the LMM provides a robust assessment of how individual IIF impacts each dimension of UX in the experimental system. Across the five dependent variables—SUS, UES, NASA-TLX (Cognitive Load), Learning Gain, and the UX Composite Score—distinct patterns of effect magnitude and significance were observed, reflecting the differentiated cognitive and affective pathways through which design elements shaped participant experience. All models included random intercepts for participants to account for inter-individual variability and were adjusted for baseline covariates, including age, digital literacy, and pre-test scores.

• (i)SUS (Fig. 11a): Among all features tested, the live CA showed a robust positive effect on usability, yielding a standardized coefficient of , with a highly significant p-value () and a narrow confidence interval . This proposes that CA substantially enhances perceptions of system clarity, responsiveness, and overall usability. AFS also exerted a significant positive effect on SUS scores , while GE showed a more moderate, yet still statistically significant effect (). PV and MAW showed positive but non-significant coefficients, indicating that their contributions to usability were less consistent across participants. The marginal and conditional values for this model were 0.628 and 0.734, indicating that the fixed effects explained a substantial proportion of variance in usability ratings.
• (ii)UES (Fig. 11b): Engagement outcomes followed a similar pattern, with GE emerging as the most predictive feature (), followed closely by the live CA ( ) and AF (). These results recommend that emotional motivators (e.g., social presence, gamified progression) and scaffolding mechanisms significantly contribute to attentional and emotional engagement. PV generated a modest effect ( ), whereas MAW failed to reach significance. The engagement model demonstrated strong explanatory power with a marginal and conditional .
• (iii)NASA-TLX (Fig. 11c): As cognitive load represents an inverse measure of interface efficiency, negative regression coefficients are interpreted as reductions in perceived mental workload. The live CA again generated the most substantial effect, significantly reducing the cognitive load (), followed by AF (). GE also trended toward reducing the cognitive load (), although this effect narrowly missed statistical significance (). Neither PV nor MAW demonstrated significant effects. These results support the interpretation that real-time dialogic and corrective support effectively lowers extraneous cognitive problems. The model’s conditional was 0.662, with an AIC of 2034.91.
• (iv)Learning Gain (Fig. 11d): The robust predictor of post-test learning gain was again the live CA, with a highly significant effect (), followed by AF () and GE (). These findings recommend that instructional scaffolding (feedback) and socio-affective design elements (CA, GE) have a positive impact on both subjective experience and observable learning outcomes. The contributions of PV and MAW were positive but non-significant, signifying that their impact may be more nuanced or dependent on contextual feature interactions. The learning gain model proved good explanatory performance with marginal and conditional values of 0.591 and 0.703.
• (v)UX Composite Score (Fig. 11e): A composite index was generated by standardizing the aggregation of the five primary UX products to obtain a holistic measure of UX. In this model, all three key features- live CA (), AF (), and GE ()—emerged as statistically significant contributors. PV showed a more minor but significant effect (), while MAW was not a significant predictor. The model achieved the highest overall fit in the study, with marginal , conditional , and the lowest AIC value (399.16), indicating strong predictive accuracy and meanness.

Fig. 11.

Main effect on: (a) SUS, (b) UES, (c) NASA-TLX, (d) Learning gain, and (e) UX composite score.

Behavioral metrics main effects

To complement subjective UX assessments, a set of behavioral metrics was modelled using LMM regressions to estimate the main effects of individual IIF on task-related efficiency, method use, interaction behavior, and cognitive trace indicators. The outcomes included Task Completion Time (TCT), Hint Utilization Rate (HUR), Navigation Entropy (NE), and Dwell Time per Feature (DTF). All models incorporated participant-level random effects and controlled for age, digital literacy, and baseline knowledge. Negative coefficients for TCT, HUR, and NE indicate improved efficiency or reduced resource demand, whereas the effects on DTF reflect the relative duration of engagement with interface components.

• (i)Task Completion Time (Fig. 12a): The robust and most statistically robust reduction in task completion time was associated with the live CA, generating a standardized coefficient of (: . This recommends that dialogue-based guidance and clarification substantially accelerate user task execution, likely by reducing uncertainty and facilitating smoother decision-making. AF also produced a significant time reduction (), while GE and PV displayed non-significant trends in the same direction. MAW, by contrast, showed no significant effect. The marginal and conditional 0.612 indicates that the fixed and random effects collectively explain approximately half of the variance in completion times.
• (ii)Hint Utilization Rate (Fig. 12b): Hint-seeking behavior was most significantly reduced by the presence of GE ( ), signifying that motivational framing may bolster learner confidence or reduce dependence on explicit scaffolds. AF was also associated with a significant reduction in hint requests (), consistent with providing just-in-time clarification that obviates further help-seeking. The live CA and PV generated modest but statistically significant reductions in HUR, while MAW showed no measurable effect. With marginal ts moderate explanatory power.
• (iii)Navigation Entropy (Fig. 12c): As a proxy for exploratory or disoriented interface behavior, navigation entropy was reduced most significantly in the presence of the live CA (), followed closely by PV () and AF ( ). These effects propose that the interface elements contribution model (visual/dialogic) helps users maintain task emphasis and reduce unnecessary page transitions. The effect of GE trended in the same direction but did not reach statistical implication (). This model achieved the lowest AIC (212.36), indicating a strong fit relative to its simplicity.
• (iv)Dwell Time per Feature (Fig. 12d): Dwell time was the most variable metric across features, reflecting feature salience and task demands. PV showed a significant reduction in dwell time (), signifying that users could interpret these elements quickly or provide minimal new data per unit time. In contrast, live CA showed a non-significant trend toward increased dwell time (), consistent with the prediction that CA demands extended attention. Effects of other features were non-significant, though GE again trended toward shorter interaction times. This model yielded marginal , signifying a moderate fit.

Fig. 12.

Behavioral metrics: (a) Task completion time, (b) hint utilization rate, (c) navigation entropy, and (d) dwell time per feature.

Feature importance rankings from ML models

To complement the inferential modelling framework and identify feature-level predictors of UX results from a model-agnostic perspective, a comparative FS analysis was conducted using two interpretable ML: Mean Decrease in Impurity (MDI) from RF and SHapley Additive exPlanations (SHAP) from GBDT. These methods enabled the quantification of global and local feature contributions across all five UX results: SUS, UES, NASA-TLX, Learning Gain, and the UX Composite Score. Rankings were derived from normalized importance values (Fig. 13a), and feature interpretability was further validated using SHAP value distributions (Fig. 13b).

Fig. 13.

(a) SHAP feature importance, (b) summary plot.

• (i)Convergent Feature Importance Across Models: As shown in Fig. 13a, the live CA consistently ranked either first or second across all models and results, underscoring its pervasive impact on the subjective and behavioral features of the learning experience. In the RF, it held the highest importance score in 4 out of 5 results, including SUS (0.283), NASA-TLX (0.273), Learning Gain (0.305), and UX Composite (0.297), with an average rank of 1.2. Similarly, in the GBDT-based SHAP analysis, the agent remained dominant, achieving the highest importance for NASA-TLX (0.295) and UX Composite (0.312) and ranking second for SUS (0.291) and Learning Gain (0.287). These findings recommend that the agent makes a significant contribution to cognitive and emotional dimensions (e.g., reduced cognitive load and increased satisfaction), and drives objective performance outcomes, such as learning gains.

AF was the second-most predictive feature overall, with strong importance scores in the GBDT. It ranked first in the SUS model (SHAP: 0.306) and the Learning Gain model (0.305), and remained within the top two across all other results. The tight clustering of importance values across model types proposes that feedback contributes robustly to UX across different prediction architectures. Its impact is particularly pronounced in affective domains such as perceived usability and user confidence. GE proved a more nuanced contribution profile. While it was the leading predictor of UES in MDI (0.293) and SHAP (0.318), its importance was markedly lower in other domains, particularly cognitive load and learning outcomes. This proposes that GE’s impact is primarily motivational and surface-level, affecting perceived engagement more than ML or navigational efficiency. Nonetheless, it remained consistently within the top three features for the composite UX, reinforcing its role in shaping user perception when combined with structural or instructional elements.

The role of PV was more modest and context-dependent. It ranked fourth in all RF, with importance values ranging from 0.137 (UES) to 0.211 (NASA-TLX), and similarly in SHAP, with a maximum of 0.195 in Learning Gain. The relatively flat distribution of SHAP values further indicates that PV cues had only a minor change in predictive contribution and may act more as a background-orienting signal than as a primary driver of user behavior or emotional decision-making. Despite this, it retained positive main effects in linear models, particularly when paired with feedback, suggesting a likely interaction effect. MAW, in contrast, consistently ranked lowest in both attribution methods, with importance values ranging from 0.027 to 0.115 across all outcomes. It made a minimal contribution to predictive change in composite scores and was rarely ranked above fourth in any single outcome.

• (ii)SHAP Value Interpretability and Model-Agnostic Explanation: The SHAP summary plot (Fig. 13b) provides additional granularity by showing how individual feature values modulate predictions across the data distribution. For instance, high values of the live CA consistently increased predicted UX scores (right-skewed distribution). In contrast, low presence values for AF and GE were associated with negative SHAP values, indicating that they were not risk factors for diminished experience. This gradient reinforces the model’s interpretability and aligns with inferential results on feature-absence impacts (e.g., “No Features” in the control group G5).

SHAP interaction values for feature pairs on UX composite score

To uncover latent synergistic relationships among IIF that may contribute to UX products beyond their individual effects, a SHAP-based interaction analysis was conducted using the GBDT trained on the UX Composite Score. SHAP interaction values quantify the joint contribution of a feature pair after accounting for the additive effects of individual features, thereby isolating second-order interactions that reflect context-sensitive dependencies or co-activation effects.

The interaction distribution (Fig. 14) and scatterplots (Fig. 15) reveal the magnitude and distributional features of all pairwise feature interactions, ranked by their mean SHAP interaction value. The results demonstrate that a small subset of feature pairs has a disproportionately strong impact on the model’s predictive performance, indicating that the user interface’s non-additive behavioral and perceptual effects are disproportionately strong.

Fig. 14.

SHAP interaction distribution.

Fig. 15.

SHAP interaction distribution scatter plot.

• (i)Strong Synergy: Live CA × AF: The interaction between the live CA and AF was robust, with a mean SHAP interaction value of 0.087. This effect indicates that when both features are present simultaneously, their combined contribution to the UX Composite Score exceeds the sum of their individual impacts. As shown in the scatterplot (Top Right), this interaction exhibits a bifurcated pattern, where high activation of both features results in substantial positive shifts in predicted UX. This result proposes that feedback becomes more intelligible or motivational when framed dialogically, and conversely, CA gains pedagogical relevance when grounded in real-time instructional feedback.
• (ii)Moderate Synergies: GE × PV and Live CA × GE: Two additional interactions showed moderate synergistic effects. The GE × PV pairing generated a SHAP value of 0.069, indicating that PV cues amplify the motivational salience of GE. Similarly, live CA × GE registered a SHAP interaction value of 0.058, signifying that social or emotional engagement with the agent reinforces the reward mechanisms of gamified systems. In both cases, the distributional patterns confirm that interaction effects are maximized when both feature values are high, reinforcing the design principle of co-alignment between structural and motivational components.
• (iii)Weak and Minimal Interactions: Several feature pairings showed weak but non-negligible interaction effects. For instance, AF × MAW had a SHAP interaction value of 0.043, indicating a weak interaction likely related to the framing of formative assessment. Two other pairings, PV × MAW (0.022) and live agent × PV (0.019), showed minimal interaction, consistent with previous results indicating that these features may exert primary effects individually but are not necessarily amplified when combined.

At the lower end of the distribution, interactions such as GE × MAW (0.012), live agent × MAW (0.009), and AF × PV (0.008) were deemed negligible, indicating functional independence between the paired features. The AF × GE interaction was the weakest overall (SHAP = 0.006), indicating that while both features are valuable, they likely operate on orthogonal UX dimensions with little mutual reinforcement.

These results carry significant implications for the design of adaptive learning interfaces. The strong collaboration between CA and feedback highlights the value of integrating real-time instruction with socially contextualized delivery. Similarly, the combined presence of GE and PV recommends that users benefit most when motivational feedback is embedded within a visible goal structure. Conversely, autonomous features (e.g., MAW) may require further integration with scaffolding mechanisms to enhance their contribution.

From a methodological perspective, this SHAP-based interaction modelling supplements traditional regression-based interaction tests by revealing non-linear and asymmetric patterns in user behavior and perception—insights that may otherwise remain hidden in parametric models. The resulting interaction hierarchy provides a data-driven basis for prioritizing co-deployments of IIF and informs future system-level optimization of UX models.

Discussion

Comparative analysis with prior literature

The results of this study advance current understanding of how multiple AI-enhanced IIFs impact UX results in DLS. Earlier works have demonstrated the individual efficacy of specific IIF, such as AFM by Razaque, A., Hariri, S., & Yoo, J.²³, gamification methods by Maity, S., & Deroy, A.²⁴, and PV tools by Calo, T., & Maclellan, C.²⁵, using heuristic evaluations or qualitative assessments. However, these studies frequently treated features in isolation, lacked interaction modelling, or failed to account for their growing impact on learner cognition and affect. In contrast, this proposed study employed a composite, experimental model that assessed the independent and synergistic effects of five IIFs through multi-method modeling. This study’s results confirm that features such as AFP and PV significantly enhance task clarity and reduce cognitive load when deployed in tandem, thereby operationalizing the theoretical benefits of adaptive scaffolding identified in prior research.

Additionally, our results refine previous conclusions on the use of CA and GE. While prior work¹⁵ highlighted the standalone benefits of engagement and emotional motivation, proposed models uncovered suppressive interaction effects when these two features were used concurrently, indicating possible cognitive saturation or conflicting attention cues. Such nuanced results were absent in earlier studies, underscoring the value of interaction-aware analysis in complex UX modelling.

Theoretical implications

The findings provide empirical support for adopting an integrative theoretical model in evaluating UX in AI-driven DLS. Unlike traditional models such as TAM, which emphasize intention to use, or CLT and SDT, which isolate either cognitive load or motivational components, the proposed model captures the multidimensionality of learner interaction, including usability, satisfaction, affective response, and behavioral efficiency. The differentiated effects of experiential learning across cognitive and affective dimensions—such as AFP improving usability and CA amplifying engagement—support the theoretical assertion that intelligent learning interfaces operate across distinct psychological pathways, which must be modelled in an integrated manner. This aligns with recent shifts in HCI and educational technology literature toward composite hypotheses that blend emotional, behavioral, and performance-based factors in real-time learning contexts.

Methodological contributions

This study contributes methodologically by combining traditional LMEM with interpretable ML methods, such as RF and GBDT, and augmenting them with SHAP analysis. Whereas prior evaluations of UX in AI-enhanced interfaces relied heavily on descriptive statistics, heuristic usability checklists, or univariate correlations^16,18, the proposed method enables causal inference and feature-level interpretability. The use of interaction-augmented LMEM enabled the detection of suppressive or synergistic dynamics that would have remained hidden in main-effect-only analysis. Furthermore, SHAP-based ranking clarified the relative contributions of each feature across UX dimensions, providing fine-grained diagnostics previously unavailable in earlier bibliometric or system-architecture studies.

Notably, the decision to exclude deep neural models (e.g., CNN, LSTM) was deliberate, prioritizing transparency over raw predictive power. This addresses a critical gap identified by Calo, T., & Maclellan, C.²⁵, who emphasized the requirement for interpretability in educational UX modelling for practical design translation.

Practical implications for ID

The study’s findings have clear implications for the design of adaptive, AI-driven DLS

1. First, the consistent positive influence of AFP and PV on usability and cognitive load reduction recommends that these features should be prioritized in contexts that require structured guidance or real-time progress tracking.

2. Second, while GE and CA are valuable for affective engagement, their combined deployment must be advanced cautiously to avoid interface clutter or cognitive interference.

3. Third, MAW proved effective in reinforcing task direction and diagnostic feedback, underscoring the importance of embedded evaluation mechanisms in intelligent interfaces.

These insights provide a basis for feature ranking methods in educational systems, where overloading learners with too many concurrent stimuli may degrade rather than enhance their experience. The proposed empirical results provide actionable thresholds and conditional configurations for optimizing feature integration.

Limitations and generalizability

Despite the study’s strengths, several limitations must be acknowledged. The participant sample, although statistically acceptable (n = 240), was drawn exclusively from Chinese universities, which may limit generalizability across cultures. Stratified sampling based on discipline and digital proficiency enhances internal validity, but replication across international cohorts is necessary to confirm cross-cultural robustness. Furthermore, while the study controlled for interface layout and task duration, real-world deployment may involve greater environmental variability. Finally, only five features were selected for factorial testing; additional modalities such as voice input, affective sensing, or AI-generated peer feedback remain untested.

Future research should consider longitudinal tracking of learner behavior during adaptive feature deployment, cross-national experiments, and the integration of learner modeling techniques to adjust feature intensity based on real-time user profiling dynamically.

Design implications for AI-enhanced educational interfaces

The empirical findings of this study generate several specific design recommendations for optimizing AI-enhanced DLS

1. First, AFP should be prioritized as a core feature in educational ID, as it consistently improves system usability, task clarity, and cognitive load reduction. To maximize use, feedback content should be context-sensitive and delivered dynamically based on user interaction patterns.

2. Second, CA should be implemented to enhance affective engagement and perceived personalization. However, designers must ensure these agents are not deployed concurrently with complex visual or gamified elements unless their cognitive demands are minimal, as interaction effects revealed potential suppression of usability when CA and GE were jointly activated.

3. Third, GE features such as progress bars, badges, and experience points can significantly improve engagement and emotional motivation, but should be applied selectively. Overuse or simultaneous activation with high-interaction features, such as CA, may introduce cognitive interference, highlighting the need for interaction-aware ranking of motivational design components.

4. Fourth, PV tools should be embedded to support self-regulated learning behaviors. These tools are most effective when linked directly to user actions and results, providing learners with transparent cues to track their progress in real time. Integration with learning analytics dashboards can further enhance usability and satisfaction.

5. Finally, MAW should be employed as embedded, low-friction diagnostic tools to facilitate continuous learner reflection and feedback alignment. These elements are particularly effective when coupled with AFP and PV, forming a triadic feedback loop between performance, guidance, and learner awareness.

6. Taken together, these implications propose that the design of AI-driven educational interfaces should adopt a modular, data-informed, and interaction-sensitive approach, in which FS and activation are not random but guided by empirical evidence on user experience dynamics and feature collaboration.

Conclusion and future work

This study presents a comprehensive model for quantifying the impact of IIF on UX in AI-driven DLS. By employing a complex experimental design involving 240 participants and a sophisticated, multilayered model, this work has verified that precise IIF makes different contributions to several dimensions of UX. The proposed model’s findings reveal that live CA and AF mechanisms have the most significant positive impact on overall UX, with notable effects on system usability, reduced cognitive load, and enhanced learning outcomes. GE shows particular strength in development engagement, while PV tools effectively reduce navigation complexity. These effects were consistently identified across parametric and non-parametric modelling approaches, strengthening confidence in conclusions. The study also uncovered essential interaction effects between features, most notably the synergistic relationship between CA and AF. This highlights the value of thoughtfully combined interface elements rather than simply accumulating features. The full-featured interface configuration consistently outperformed partial implementations, though with diminishing returns for each additional feature. This proposed method proves the value of triangulating proof across multiple analytical methods. The close alignment between LMM and ML-based FS validates the proposed results while revealing nuanced non-linear relationships that traditional statistical methods might miss. From a practical perspective, these results provide evidence-based guidance for educational technology designers and developers. When resources are limited, prioritizing CA and AF will likely yield the most significant UX improvements. For engagement-focused applications, GE warrants distinct consideration.

Future research should explore how these features generalize across different subject domains, learner demographics, and longer-term usage periods. Additionally, examining how cultural and contextual factors moderate the impact of these features would enhance the ecological validity of these results. By establishing a measurable link between IIF and UX outcomes, this work contributes to the development of more effective AI-driven educational technologies that better support diverse learning needs.

Author contributions

Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data Curation, Writing—Original Draft, Writing—Review & Editing, Visualization : Ru Chen, Jin Zhang.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Code availability

The custom code and algorithms developed for this study are publicly available in a GitLab repository to ensure transparency and reproducibility. The repository contains the implementation corresponding to the version described in the manuscript. GitLab Repository: https://gitlab.com/research1922608/ui/-/blob/main/UserExperience.ipynb.

Declarations

Competing interests

The authors declare no competing interests.