Skip to main content
Scientific Reports logoLink to Scientific Reports
. 2026 Apr 3;16:15847. doi: 10.1038/s41598-026-44804-x

Mamba-based modulated fusion model for video moment retrieval

Bing Yu 1,2,✉,#, Jingyu Li 1,2,#, Youxian Di 1,2, Yingran Liu 1,2, Youdong Ding 1,2, Dongjin Huang 1,2
PMCID: PMC13195039  PMID: 41932980

Abstract

Video Moment Retrieval (VMR) serves as a fundamental task in video understanding, bridging vision and language by localizing the most relevant temporal segments in untrimmed videos according to a textual query. However, existing approaches excel at fine-grained alignment but often fail to capture global temporal context effectively, particularly in long-form videos. To address this challenge, we propose Hybrid Mamba Network (HM-Net), a two-level fusion architecture which unifying the strengths of attention and sequence modeling. Especially, its core lies in the Hybrid Modulated Bi-Mamba (HMB) Block, which integrates the powerful temporal modeling capability of Mamba into the VMR framework to achieve effective long-range temporal reasoning. Extensive experiments on the challenging TACoS and QVHighlights benchmarks show that HM-Net consistently outperforms existing approaches, achieving 3.84% improvement in R1@0.5 (TACoS) and 1.65% in mAP (QVHighlights), demonstrating notable gains in localization accuracy, particularly on long-form videos.

Keywords: Video moment retrieval, Cross-modal fusion, State space models

Subject terms: Engineering, Mathematics and computing

Introduction

The rapid growth of digital media has made video a primary tool for communication, driving a strong demand for effective video retrieval. However, finding specific details within untrimmed videos remains challenging due to their massive scale and complex visual content. Video Moment Retrieval (VMR) tackles this problem by localizing the exact video segment that matches a natural language query. By bridging vision and language at the temporal level, VMR makes large-scale video databases semantically searchable. Furthermore, it serves as a foundational technique for real-world applications such as content-based video search, automatic summarization, and human-computer interaction.

Early VMR approaches primarily adopted a two-stage propose-and-rank paradigm1. However, such methods inherently suffer from a strong dependence on the quality of initial proposals, imprecise boundary predictions, and inefficiencies stemming from multi-stage pipelines. Motivated by the success of DEtection TRansformer (DETR)2 in object detection, recent research has transitioned to end-to-end proposal-free frameworks3–5. These approaches directly regress temporal boundaries from a small set of learnable queries via a Transformer encoder-decoder, where the decoder’s cross-attention implicitly aligns each query with relevant video content. Although Transformer-based methods have achieved notable improvements in cross-modal alignment, they still face fundamental challenges in modeling long-range temporal dependencies. The standard attention mechanism is inherently permutation-invariant, treating a video as an unordered collection of frames rather than a structured temporal sequence. This limits the model’s ability to capture causal progression, action ordering, and temporal state transitions. These elements are critical for understanding complex, multi-step queries and comprehending long, compound actions. As illustrated in Fig. 1, for queries involving long and compound action sequences such as “After getting out the juicer, he juices the first orange half”, the target moment spans multiple sub-actions performed by the subject. However, the query only explicitly describes the first and last steps. Consequently, these Transformer-based methods typically capture only part of the action sequence mentioned in the query, failing to comprehend the full event structure and resulting in inaccurate predictions.

Fig. 1.

Fig. 1

Illustrations of the difficulty of retrieving long, multi-step events from partially described queries. The query mentions only the first and last actions, while the actual moment spans three sequential steps.

To overcome these challenges, we propose the Hybrid Mamba-based Network (HM-Net), a novel framework for cross-modal fusion in VMR, aiming to inject powerful temporal reasoning into efficient fusion architectures. At its core lies the Hybrid Modulated Bi-Mamba (HMB) block, which uniquely couples the precise query-video alignment of cross-attention with the sequence modeling of State Space Models (SSMs). This hybrid design enables the model to reason over the global narrative flow and bidirectional dependencies, essential for the complex, long-duration events in datasets. As illustrated in Fig. 1, HM-Net approach successfully models the entire multi-step event, even when the query provides only incomplete descriptions. The HMB block introduces three key innovations to realize this synergy. First, it acts as a hybrid module that couples cross-attention for fine-grained token-level query-video grounding with a bidirectional Mamba core capable of robust long-range temporal reasoning. This enables the model to align textual cues with visual evidence while maintaining awareness of the narrative flow across the full event span. Second, it incorporates dynamic adaptability through two modulation mechanisms: a query-aware contextual signal that adjusts fusion behavior according to global query semantics, and a conditionally activated positional bias that enhances stability across varying sequence lengths. Finally, the bidirectional Mamba architecture employs independent forward and backward state models to capture comprehensive temporal context around each moment, ensuring both past and future dependencies are effectively modeled.

In summary, our main contributions are threefold:

  • We propose HM-Net, a novel two-level fusion architecture which enhances video representations with query guidance before cross-modal fusion, enabling more effective alignment between textual and visual modalities and improving localization in long and complex videos.

  • We propose a Hybrid Modulated Bi-Mamba (HMB) Block that unifies cross-attention with bidirectional Mamba modeling for fine-grained query-video alignment and causal progression and action ordering.

  • We further introduce an adaptive Contextual and Positional Modulation that dynamically guides cross-modal fusion with query semantics and selectively applies positional information based on video length, improving flexibility and robustness.

  • Extensive experiments demonstrate that HM-Net gains significant improvements in localization accuracy, particularly in long-form videos.

Related works

Video moment retrieval

Early works in VMR largely followed a two-stage, propose-and-rank paradigm1. Candidate temporal segments were first generated using sliding windows6 or dedicated proposal networks7,8, and then ranked against the text query by a cross-modal matching model9. However, this pipeline suffered from inherent limitations. The final performance was heavily reliant on the quality of the initial proposals, the methods often struggled with precise boundary prediction, and their multi-stage nature could be inefficient and prone to error propagation10,11.

Inspired by DETR’s success in object detection2, research has shifted toward end-to-end, proposal-free frameworks. Architectures such as Moment-DETR12 reformulated VMR as a direct set prediction task, employing a Transformer encoder-decoder to regress moment boundaries from a small set of learnable queries. In these methods, the decoder’s cross-attention provides implicit query-video alignment. To improve multimodal correlation, later works introduced explicit fusion modules13,14, contrastive learning objectives15,16, and fine-grained alignment networks17,18. Broadly speaking, the necessity for robust multi-modal interaction and target-distractor discrimination is a shared challenge across various cutting-edge localization tasks, inspiring advanced fusion paradigms in recent visual tracking approaches as well19,20. Beyond video retrieval, the necessity for robust multimodal alignment extends to various domains, inspiring advanced techniques such as semantic segmentation21 and cross-domain sequential recommendation22–24.

Despite their success in alignment, a core challenge persists for these Transformer-based models: effectively modeling long-range temporal relationships. The standard attention mechanism is fundamentally permutation-invariant, treating the video as a bag-of-frames rather than a structured sequence. While powerful for connecting any two points in time, this design is not inherently suited to capturing the sequential narrative, ordering of events, and causal dependencies that are crucial for understanding long videos. To overcome this issue, some works segment videos into shorter units instead of processing them as a whole. SpotVMR25 avoids dense feature extraction across the entire video and instead employs a lightweight clip retrieval module to quickly locate potentially relevant temporal regions conditioned on the query, followed by fine-grained localization using a VMR model. EventFormer26 groups consecutive visually coherent frames into higher-level “events” and performs hierarchical encoding and reasoning over these events, enabling more effective comprehension of complex activities in long videos. Although these strategies partially mitigate long-range dependency issues, they disrupt the holistic temporal structure of videos and limit the model’s ability to reason over global narrative flows. Other works introduce LLMs to enhance the understanding of complex natural-language queries27,28. However, these methods typically rely heavily on large-scale pretraining and still encounter challenges in efficiency and scalability when dealing with long videos. To address this, we aim to effectively capture long-range temporal dependencies while maintaining fine-grained cross-modal alignment.

Vision state space model

State Space Models (SSMs) have recently emerged as a highly promising alternative to Transformers for sequence modeling. Originating from classical control theory, modern SSMs like S429 have demonstrated exceptional performance on long-sequence tasks, primarily due to their ability to model dependencies with linear or near-linear complexity. More recently, Mamba30,31 has pushed the boundaries of SSMs by introducing a selection mechanism that allows the model to be content-aware, dynamically focusing on or ignoring inputs based on their relevance. This, combined with a hardware-aware parallel algorithm, enables Mamba to match or exceed the performance of Transformers on various sequence modeling benchmarks with linear complexity.

The success of Mamba in 1D sequence modeling has spurred its adaptation to visual data. Vision Mamba(Vim)32 and VMamba33 were among the first to demonstrate that a Mamba-based architecture can be a strong backbone for image classification by flattening image patches into a sequence and applying bidirectional scanning. This concept has been extended to other vision tasks, including object detection and segmentation34,35 Recent research has successfully explored semantic-guided Mamba fusion for robust object detection in challenging real-world environments36, showcasing the potential of SSMs as a general-purpose vision backbone. Video Mamba37 further demonstrates the powerful temporal modeling capabilities of the Mamba architecture in the video field. We integrate a Mamba-based encoder into a robust VMR framework, inheriting the effective and long-range modeling capabilities of SSMs while preserving the precise cross-modal alignment required for this task. To our knowledge, our model is one of the first to design and validate a dedicated Mamba-driven architecture for the specific challenges of Video Moment Retrieval.

Method

Overall framework

Given an untrimmed video input V and a natural language query Q, the goal of the VMR task is to localize the most relevant temporal segment Inline graphic. Following common practice in VMR research3,38, we first extract features via offline encoders, representing the video as Inline graphic, and the queries as Inline graphic, where L and N is the number of video frames and tokens in a single query, and K is the number of queries paired with a video. Both feature sequences are first projected into a shared embedding space via learnable MLPs.

The architecture of HM-Net, depicted in Fig. 2, is an end-to-end pipeline that transforms raw video and query inputs into precise temporal localizations. At its core lies a two-level fusion framework built upon the Hybrid Modulated Bi-Mamba (HMB) blocks. The first level, a Unidirectional Enhancement Encoder, refines video representations to Inline graphic by integrating semantic cues from the textual query. The second level, a Symmetric Co-fusion Encoder, conducts deep bidirectional interaction between modalities, producing the final fused feature Inline graphic. To further enhance semantic consistency, the query-side output of HMB is aggregated through masked mean pooling and incorporated into Inline graphic. Additionally, following39, we introduce a segment-sentence reconstruction objective as an auxiliary task to regularize training and improve cross-modal alignment. Finally, Inline graphic is passed to a Transformer decoder, which, under the DETR paradigm, employs learnable queries to directly predict temporal spans. This overall design unifies the fine-grained alignment capacity of attention with the long-range modeling of Mamba, leading to a powerful and scalable framework for temporal moment retrieval.

Fig. 2.

Fig. 2

The proposed HM-Net architecture. Video and text features are processed by a two-level fusion pipeline composed of our HMB blocks. The resulting features are fed into a Transformer encoder-decoder network to predict the final temporal spans.

Preliminaries

Building on the traditional state space model29,40,41, Mamba30 represents a recent advancement that achieves remarkable efficiency and performance in long-sequence modeling. Unlike Transformers, whose self-attention scales quadratically with sequence length, Mamba models temporal context with linear complexity. It operates by mapping a 1D input sequence x(t) to an output y(t) via a latent state h(t), defined by:

graphic file with name d33e509.gif 1
graphic file with name d33e513.gif 2

where Inline graphic is the state matrix that governs the temporal evolution of the hidden state, Inline graphic is the input matrix controlling how the input influences the hidden state, and Inline graphic is the observation matrix mapping the hidden state to the output. To facilitate application in deep learning, Mamba adopts the Zero-Order Hold (ZOH) principle, discretizing the continuous formulation. Let Inline graphic denote the time-scale parameter and I the identity matrix. The discretized state parameters Inline graphic and Inline graphic are given by:

graphic file with name d33e547.gif 3
graphic file with name d33e551.gif 4

The key innovation of Mamba lies in its selection mechanism, which makes the state-space parameters (B, C, and a timestep parameter Inline graphic) input-dependent. This allows Mamba to be content-aware, enabling it to selectively emphasize relevant information while filtering out noise across long sequences. Such dynamic, efficient, and expressive sequence modeling makes Mamba a compelling alternative to the computationally expensive self-attention mechanism in temporal video retrieval tasks.

Hybrid Modulated Bi-Mamba block

To achieve fine-grained and efficient cross-modal interaction, we propose the HMB block, which serves as the fundamental building block for both our enhancement and fusion encoders. As illustrated in Fig. 3, the HMB block synergistically integrates cross-attention for multi-modal alignment with a bidirectional Mamba architecture for temporal dependency modeling. This integration is further augmented by a novel modulation mechanism that incorporates robust contextual information and adaptive positional cues.

Fig. 3.

Fig. 3

The detailed architecture of HMB block. The block first establishes fine-grained alignment and injects global semantic guidance via parallel cross-attention and robust context pooling modulation. This fused representation, optionally enhanced with a conditional positional bias, is then processed by a Bidirectional Mamba core to efficiently capture long-range temporal dependencies before a final residual update.

Cross-modal alignment Given a primary sequence Inline graphic (e.g., video features) and a context sequence Inline graphic (e.g., text features), the HMB block first establishes an initial alignment. This is accomplished via a standard multi-head cross-attention mechanism, where the main sequence acts as the query and the context sequence provides the keys and values:

graphic file with name d33e595.gif 5

where Inline graphic denotes Layer Normalization. This step allows each element in the main sequence to aggregate relevant information from the entire context sequence.

Hybrid feature modulation Next, to introduce global contextual guidance, we adopt a Robust Context Pooling strategy. Traditional approaches often rely on a single [CLS] token to summarize the context sequence. However, this design is inherently brittle: the first token may be noisy, non-informative, or insufficient to represent the full context. To obtain a more stable and semantically reliable summary, we compute a global context vector Inline graphic using a mask-weighted average over the context tokens:

graphic file with name d33e615.gif 6

where Inline graphic is the padding mask that filters out invalid tokens, Inline graphic is the context length. This pooling ensures that only valid tokens contribute to the global representation. The vector Inline graphic is then projected through a learnable matrix Inline graphic to produce a global modulation term Inline graphic, which is injected into the main sequence through a gated residual connection controlled by the learnable scalar Inline graphic:

graphic file with name d33e645.gif 7
graphic file with name d33e649.gif 8

where Inline graphic is the normalized main-sequence input and Inline graphic is the output of the cross-modal interaction module.

This mechanism provides a holistic conditioning signal that complements token-level attention and enhances overall semantic consistency. For example, given the query from Fig.1, After getting out the juicer, he juices the first orange half, standard attention attempts to explicitly align the nouns, such as orange and juicer, with specific frames. However, this token-level matching can be brittle if the objects are temporarily occluded by the subject’s hands or motion-blurred. The global modulation term Inline graphic addresses this vulnerability by acting as a sequence-level semantic bias. It pools the entire sentence to extract the intent, making juice, and uniformly shifts the entire video feature space to reflect this continuous contextual atmosphere. Consequently, even during intermediate frames where specific objects are obscured, this holistic baseline ensures the model maintains stable awareness of the ongoing event, providing a robust foundation for the subsequent Mamba sequence modeling.

To further incorporate explicit temporal cues, we introduce Adaptive Positional Modulation, which is particularly beneficial for shorter sequences that lack inherent structural cues. Specifically, we maintain a learnable positional bias matrix and dynamically interpolate it to the target length of the main sequence. This interpolated bias is added to the sequence under the control of a gating scalar Inline graphic. Importantly, the positional modulation is activated only when the main-sequence length Inline graphic is smaller than a fixed hyperparameter threshold Inline graphic (empirically set to 200). This thresholding strategy ensures that explicit positional priors are applied only to short-term dependencies where they are most effective, preventing overly rigid biases from being introduced when the video is sufficiently long to encode its own temporal structure.

Bidirectional sequence modeling Finally, to effectively model the temporal dependencies within the modulated sequence, we employ a bidirectional Mamba architecture. Unlike the original unidirectional Mamba, we process the sequence in both forward and backward directions using two independent Mamba modules Inline graphic with separate weights. This allows the model to capture a more comprehensive temporal context, integrating information from both past and future frames. The outputs are combined and added to the sequence through a learnable gate Inline graphic, followed by a residual connection, where Inline graphic denotes sequence reversal operation:

graphic file with name d33e705.gif 9
graphic file with name d33e709.gif 10
graphic file with name d33e713.gif 11

This gated bidirectional design enables HMB block to efficiently model long-range dependencies while being dynamically conditioned on cross-modal information, forming a powerful and versatile fusion primitive.

Two-stage hierarchical fusion

The HMB block serves as the core component within our proposed two-level fusion pipeline, which is designed to progressively refine and integrate cross-modal features. This hierarchical approach consists of an initial enhancement stage followed by a deep, symmetric co-fusion stage, facilitating deeper and more effective integration between the modalities.

Unimodal feature enhancement The first level, termed the Frame-Word Enhancement Encoder, aims to enrich the raw video features with contextual cues from the text modality. As depicted in our overall framework, this encoder is composed of a stack of HMB blocks. It takes the projected video features Inline graphic as the main sequence and the projected text features Inline graphic as the context sequence. Each video frame representation is updated by attending to the entire textual query. The output is an enhanced video feature sequence Inline graphic, where the frame representations are now semantically conditioned on the textual description, making them more discriminative for the subsequent localization task.

Symmetric co-fusion Following the initial enhancement, the enhanced video features Inline graphic and the projected text features Inline graphic are fed into the second level, the Symmetric Co-fusion Encoder. Unlike the first stage, this encoder performs deep, iterative, and bidirectional fusion. It consists of multiple layers, where each layer contains two parallel HMB blocks. One block updates the video features using the text features as context, while the other simultaneously updates the text features using the video features as context. This symmetric co-reasoning process can be formulated for a layer l as:

graphic file with name d33e753.gif 12
graphic file with name d33e757.gif 13

where Inline graphic. This iterative refinement allows for a more profound and fine-grained alignment between the two modalities. The final output of this pipeline is the deeply fused video feature sequence from the last layer, which encapsulates rich cross-modal information and serves as the key-value memory for the subsequent prediction head. After the final symmetric co-fusion layer, we perform a masked mean pooling operation on the updated text feature sequence to produce a global query representation. This holistic vector, representing the ultimate understanding of the textual query, is then broadcast and added to the video feature sequence. This final conditioning step allows each frame’s representation to be globally calibrated by the fully contextualized query, leading to a more refined and query-aware memory for the decoder.

Training objective

Prediction head Following the classic DETR paradigm2,3, our prediction head is a Transformer-based decoder that operates on the fused cross-modal representations generated by our two-level fusion pipeline. The deeply fused video feature sequence serves as the memory for a standard Transformer decoder. A fixed number of learnable embeddings, known as object queries, are input to the decoder. These queries are iteratively refined through layers of self-attention and cross-attention, ultimately producing a set of output embeddings. Each output embedding is then passed through two parallel feed-forward networks to generate the final predictions. Specifically, a classification head predicts the foreground/background logits, and a regression head predicts the temporal span parameterized by its center coordinate and width: [center, width].

Loss function The overall training objective is a composite loss that mirrors the successful formulation of prior works in moment retrieval. First, following DETR2, a bipartite matching between the set of predictions and ground-truth moments is established using the Hungarian algorithm3,42. The primary moment localization loss, Inline graphic, is then calculated only on the matched pairs. This loss is a linear combination of: (1) a classification loss Inline graphic for the query labels2 (2) an Inline graphic loss on the predicted span parameters and (3) a scale-invariant Generalized IoU Inline graphic loss to measure the overlap quality43. The combined localization loss is:

graphic file with name d33e826.gif 14

In addition to the primary localization objective, we incorporate two auxiliary losses to enhance representation learning. First, a contrastive saliency loss Inline graphic is employed to improve the model’s ability to distinguish correct video-text pairs from negative ones44. It is formulated as a max-margin objective:

graphic file with name d33e840.gif 15

where Inline graphic and Inline graphic are the saliency scores for a positive and a negative pair, respectively Inline graphic, and Inline graphic is a predefined margin. This encourages the model to map semantically similar inputs closer together in the embedding space while pushing dissimilar ones apart. Second, following MESM39, we include multi-level reconstruction losses Inline graphic, which promote fine-grained alignment at both the frame-word and segment-sentence levels.

The final training objective is a weighted sum of all these components:

graphic file with name d33e872.gif 16

where the Inline graphic terms are hyper-parameters balancing the influence of each objective.

Experiment

Experimental settings

Datasets We validate our approach on two standard benchmarks to demonstrate its effectiveness and generalization: the large-scale, in-the-wild QVHighlights dataset12; and the challenging TACoS cooking dataset45, which is known for its long videos and complex language queries.

Metrics To ensure a fair comparison with prior work, we follow standard evaluation protocols. Performance is primarily measured by Recall@1 at multiple Intersection over Union (IoU) thresholds (R1 @ Inline graphic, where Inline graphic), the mean IoU (mIoU) to assess overall localization accuracy, and mean Average Precision (mAP) to evaluate the ranking quality of the retrieved moments.

Implementation details We utilize standard pre-extracted features: SlowFast46 and CLIP47 for QVHighlights, and C3D48 and GloVe49 for TACoS. Our model architecture is configured with a hidden dimension of 256. Specifically, within the HMB block, the cross-attention mechanism employs 8 parallel heads, and the Mamba state space dimension (Inline graphic) is set to 32. The core fusion pipeline consists of a 2-layer enhancement encoder and a 2-layer symmetric co-fusion encoder. The model is trained end-to-end using the AdamW optimizer with a peak learning rate of Inline graphic and a weight decay of Inline graphic. The learning rate follows a cosine annealing schedule after a 5-epoch linear warmup. For the training objective, the balancing coefficients are set as Inline graphic, where Inline graphic, Inline graphic, Inline graphic, Inline graphic and Inline graphic. The logic behind this configuration follows a principled hierarchy: the localization sub-weights inherit standard DETR-based configurations for stable span regression, while Inline graphic is equally weighted to ensure a discriminative embedding space. Importantly, Inline graphic is scaled down to 0.1 to guarantee that the multi-level reconstruction acts strictly as an auxiliary regularizer, promoting fine-grained alignment without overwhelming the primary localization task. All experiments are conducted on a single NVIDIA GeForce RTX 3090 GPU.

Experimental results

We compare our proposed HM-Net with several state-of-the-art (SOTA) methods on two challenging and widely-used benchmarks: TACoS and QVHighlights. For a comprehensive evaluation, we report results across multiple standard metrics, including Recall@1 at various IoU thresholds (R1Inline graphic), mean Average Precision (mAP) at different thresholds, and the overall mean IoU (mIoU). The detailed results are presented in the tables.

TACoS As shown in Table 1, the TACoS dataset poses significant challenges due to its long video durations and complex activity descriptions. On this benchmark, HM-Net achieves new state-of-the-art results across the most critical metrics. Specifically, it delivers the best performance on R1@0.5 (43.16%) and the stricter R1@0.7 (27.94%), surpassing the strong DETR-based baseline CG-DETR by 2.77 and 2.95 points, respectively. HM-Net also attains the highest overall localization accuracy with an mIoU of 40.04, as well as the best mean Average Precision, achieving both the highest average mAP (25.01%) and a substantial gain on the more challenging mAP@0.75 (23.15%). These improvements underscore the effectiveness of our hybrid fusion approach, which combines the precise alignment capabilities of attention with the efficient long-range modeling of Mamba. The results suggest that HM-Net is particularly adept at handling the complex temporal structures and nuanced language queries characteristic of TACoS, leading to more accurate and reliable moment retrieval.

Table 1.

Comparison results on the TACoS dataset.

Method R1 mAP mIoU
@0.3 @0.5 @0.7 @0.5 @0.75 Avg.
MMN50 37.19 26.39 13.02 – – – –
Uni-VTGInline graphic51 48.62 35.39 17.76 40.17 17.57 20.26 32.69
LLMEPETInline graphic28 46.80 32.30 17.09 38.07 15.13 17.97 31.58
CICR52 53.09 38.94 20.09 42.15 15.79 19.68 35.76
MESM39 53.91 38.69 22.64 40.40 19.69 21.03 37.46
CG-DETR38 53.11 40.39 24.99 45.16 20.69 23.56 37.14
MRNetInline graphic53 55.14 39.32 22.24 – – – 38.77
HM-Net(Ours) 55.71 43.16 27.94 44.13 23.15 25.01 40.04

Inline graphic denotes methods that rely on large-scale pretraining. For fairness, all methods are trained only on the target dataset. Inline graphic denotes methods that additionally incorporate audio features. The best performace in each column is highlighted in bold.

QVHighlights HM-Net further demonstrates strong generalization on the large-scale, in-the-wild QVHighlights dataset. As shown in Table 2, while the improvements here are more modest than on TACoS, the gains remain consistent. For example, HM-Net yields incremental improvements over strong baselines such as UVCOM and CICR. This outcome may reflect the nature of QVHighlights, where shorter, action-dense videos reduce the impact of self-attention’s quadratic complexity, allowing Transformer-based models to remain competitive. Nonetheless, HM-Net surpasses previous best method UVCOM by 0.97%, 0.23% and 1.65% in terms of mAP@0.5, mAP@0.75 and average mAP, highlighting the effectiveness of its fine-grained alignment and modulation even in shorter sequences. These results suggest promising future directions in adaptively balancing attention and Mamba mechanisms based on video characteristics.

Table 2.

Comparison results on the QVHighlights dataset.

Method R1 mAP mIoU
@0.3 @0.5 @0.7 @0.5 @0.75 Avg.
EaTR54 69.68 59.23 44.65 60.54 40.25 39.99 54.56
LLMEPETInline graphic28 63.61 47.10 27.48 50.37 26.67 27.98 44.49
Uni-VTGInline graphic51 72.58 59.81 42.71 51.15 33.80 33.39 54.24
QD-DETR3 74.52 62.06 46.32 62.17 41.34 40.96 57.38
MESM39 74.00 62.52 46.39 62.20 40.88 40.74 57.53
UVCOM55 74.00 63.74 47.68 62.17 43.23 41.52 58.01
CICR52 75.35 63.35 47.23 62.62 41.96 41.51 58.26
HM-Net(Ours) 74.77 62.77 48.13 63.14 43.70 43.17 57.88

Inline graphic denotes methods that rely on large-scale pretraining. For fairness, all methods are trained only on the target dataset.  The best performance in each column is highlighted in bold.

Ablation study

To validate the contribution of each key component in our proposed HM-Net, we conducted a series of ablation studies on the TACoS dataset, with the results presented in Table 3. Our investigation reveals that while both the Mamba backbone and the Cross-Attention (C-Attn) module are crucial, their combination yields the most significant performance gain. Specifically, a model equipped with both Mamba and C-Attn achieves an R1@0.5 of 41.14%, substantially outperforming variants that use only one of them. This demonstrates a powerful synergy where Mamba provides robust temporal context that is effectively leveraged by C-Attn for precise cross-modal alignment. To theoretically and empirically justify our architectural choice of sequence modeling, we further introduce a unidirectional Mamba variant (Inline graphic) for comparison. VMR requires a holistic understanding of the entire event sequence. A unidirectional model only compresses historical context, which is often insufficient for determining the exact end boundary of a moment to confirm action transitions. Our empirical results strongly support this intuition. Unidirectional variant leads to a notable performance drop, decreasing R1@0.5 from 43.16% to 41.06%. Furthermore, the integration of our Context Pooling (C-Pool) layer on top of this strong bidirectional foundation provides a final, meaningful boost, elevating the R1@0.5 and mIoU to their peak values of 43.16% and 40.04%, respectively. This final improvement confirms C-Pool’s effectiveness in aggregating the fused features for more accurate localization. In essence, these ablation results strongly suggest that all three components are integral and contribute cumulatively to the superior performance of our HM-Net.

Table 3.

Ablation studies on the core components of HM-Net on the TACoS dataset. For Mamba, we compare the original bidirectional design (Bi) with a unidirectional variant (Uni) to validate the importance of bidirectional modeling.

Components Performance metrics
Mamba C-Attn C-Pool R1@0.5 mIoU mAP@Avg.
Inline graphic 38.69 37.46 21.03
Inline graphic 35.77 33.72 19.71
Inline graphic Inline graphic 41.14 37.82 23.95
Inline graphic Inline graphic Inline graphic 41.06 38.52 23.12
Inline graphic Inline graphic Inline graphic 43.16 40.04 25.01

The best performace in each column is highlighted in bold.

We further investigated the impact of the depth of our enhancement module (Enh Layers) and co-fusion module (Co-fusion Layers) on model performance, with results summarized in Table 4. For the Enhancement module, performance peaks at 2 layers on R1@0.5 (43.16%), surpassing the base model (0 layers) by 2.52%. Similarly, the Co-fusion module achieves optimal results with 2 layers, outperforming both shallower and deeper configurations. In both cases, increasing depth to 3 layers leads to performance degradation, likely due to overfitting or optimization difficulties. Consequently, we set both modules to 2 layers for the final HM-Net architecture.

Table 4.

Ablation studies on the depth of different modules on the TACoS dataset.

Module Layers R1@0.5 mIoU mAP@Avg.
Enhancement 0 40.64 38.57 24.11
1 41.31 38.39 25.05
2 43.16 40.04 25.01
3 42.54 39.43 24.15
Co-fusion 1 40.74 38.05 22.70
2 43.16 40.04 25.01
3 42.84 39.71 24.89

 The best performance within each group is highlighted in bold.

Finally, we trained a variant of HM-Net by removing the reconstruction loss (Inline graphic). Our model achieves an R1@0.5 of 40.14%, which still significantly outperforms MESM (38.69%). While Inline graphic provides a complementary regularization benefit, the primary performance gain is driven by the robust temporal modeling capabilities of the proposed HMB block.

Analysis on long-form videos

To validate our model’s enhanced capability for understanding complex and long-duration events, we conduct both a qualitative and quantitative analysis.

In Fig.4, we present qualitative results on challenging long-form video examples from both the TACoS and QVHighlights datasets. Evidently, our model demonstrates superior capability in long video scenarios, accurately capturing complex temporal dynamics and subtle semantic cues. In the TACoS example, the video lasts over ten minutes. For short action segments, both the baseline and our model can roughly localize the correct moments; however, HM-Net achieves noticeably higher precision. In contrast, for a long and complex action lasting more than two minutes (“The person scoops the seeds from that half of the pomegranate onto the plate”), the baseline prediction deviates substantially, overemphasizing visual regions merely related to the pomegranate while failing to integrate the full text-video semantics. Similarly, in the QVHighlights example, our model exhibits strong performance. It not only identifies short event segments with high precision, but also captures the semantic relevance between talk and jewelry with a highly semantically relevant query, producing a well-aligned temporal boundary that closely matches the ground truth. These qualitative differences highlight the superior ability of our method to model long-range temporal dependencies and reason over semantically intricate events.

Fig. 4.

Fig. 4

Qualitative comparison on long videos from TACoS dataset test split (top) and QVHighlights dataset val split datasets (bottom). The color bars in the boxs represent the predicted windows of the ground-truth and models. Our model demonstrates superior capability in capturing complex temporal dynamics and subtle semantic cues in long video scenarios, leading to more accurate moment localization.

To verify that the superior performance observed in Fig.4 is a systematic advantage and not an isolated case, we provide a quantitative comparison on designated long-video subsets of both TACoS and QVHighlights. Here, MESM39 is adopted as our baseline for fair comparison. As shown in Table 5, our model consistently outperforms the baseline across all long-video subsets. HM-Net achieves 6.14% and 2.15% mAP improvements in TACoS long and super long duration dataset, and 5.0% mAP improvements in QVHighlights long duration dataset. This quantitative data strongly corroborates our qualitative findings, confirming that the enhanced sequential modeling capability provided by the HMB block translates into a robust and generalizable performance gain on long-form videos. This systematic improvement proves that our architecture is better equipped to handle the long-range temporal dependencies that are prevalent and challenging in real-world video content.

Table 5.

Quantitative comparison on long-video subsets. “Long”: videos longer than 30 seconds for both TACoS and QVHighlights. “Super-long”: videos longer than 150 seconds for TACoS.

Dataset Duration Method mAP Avg.
TACoS Long MESM 15.81
HM-Net (ours) 21.95
Super-long Baseline 10.47
HM-Net (ours) 12.75
QVHighlights Long MESM 45.61
HM-Net (ours) 50.61

 The best performance within each group is highlighted in bold.

Limitations and future work

Failure case analysis To evaluate robustness, we visualize typical failure cases in Fig.5. First, in the TACoS example, the model struggles to localize the micro-action “He puts the yolk in the other small cup” within a repetitive cooking sequence. We attribute this to the state smoothing effect of the Mamba architecture, which prioritizes global contextual flow but may inadvertently blur the high frequency boundaries of momentary actions. Second, in the QVHighlights example, both our model and baseline model fail to distinguish the subtle transition from chatting to singing. Due to the recurrent nature of the HMB block, the accumulated historical state tends to carry over visual semantics. It may require additional mechanisms to enhance sensitivity to fine-grained temporal cues for micro-actions and semantically ambiguous boundaries.

Fig. 5.

Fig. 5

Failure cases from TACoS dataset test split (top) and QVHighlights dataset val split datasets (bottom). The color bars in the boxs represent the predicted windows of the ground-truth and models.

Computational complexity We profiled the inference latency and GPU memory usage of the complete end-to-end model (including the DETR head). As shown in Table 6, HM-Net introduces a marginal latency overhead compared to the Transformer models (CG-DETR and MESM). This is primarily due to the maturity gap between current SSM hardware kernels and highly optimized FlashAttention. However, notably, HM-Net maintains an identical peak memory footprint to the baseline. The memory bottleneck remains determined by the quadratic complexity of the shared DETR prediction head, rather than our linear fusion module. Consequently, HM-Net achieves significant performance gains on long-form videos without exacerbating the peak memory cost, representing a favorable performance-efficiency trade-off.

Table 6.

Comparison of inference latency and peak GPU memory (batch size=2).

Length (Inline graphic) CG-DETR MESM HM-Net (ours)
Latency(ms) Memory(MB) Latency(ms) Memory(MB) Latency(ms) Memory(MB)
256 20.96 72.83 35.01 120.74 49.12 120.74
512 21.79 109.37 34.03 157.69 48.16 157.69
1024 22.05 225.17 34.14 276.99 48.31 277.00
2048 27.85 651.97 35.41 709.34 48.84 709.34
4096 46.04 2275.40 69.61 2342.02 92.88 2342.02
8192 133.59 8591.69 251.57 8681.38 292.13 8681.39

Conclusion

In this paper, we introduced HM-Net, a novel Hierarchical Hybrid Mamba Network designed to achieve fine-grained modal alignment with robust long-range temporal reasoning in VMR. At the heart of our framework is the HMB block, combining the precise alignment power of cross-attention with the powerful temporal modeling capabilities of a bidirectional Mamba. Through experiments on the TACoS and QVHighlights datasets, we demonstrated that HM-Net achieves new state-of-the-art performance, showing a particular advantage in high-precision localization. Our comprehensive ablation studies systematically validated the effectiveness of each component in our design, from the hybrid nature of the HMB block to the hierarchical architecture. While our empirical profiling indicates a slight latency trade-off due to current hardware constraints, the architecture successfully avoids additional memory bottlenecks.

Future work will explore fully end-to-end linear architectures to unlock the complete efficiency potential of SSMs and investigate adaptive fusion strategies that dynamically adjust to video complexity. Furthermore, integrating Large Vision-Language Models (LVLMs) to deepen semantic comprehension is a highly promising direction56.

Acknowledgements

This work was supported by the Shanghai Municipal Fund for Promoting the Development of the Cultural and Creative Industries (2025020022) and the Shanghai Natural Science Foundation (25ZR1401130).

Author contributions

Bing Yu and Jingyu Li wrote the main manuscript text. Bing Yu and Jingyu Li did experients of the manuscript. Youxian Di and Yingran Liu preparesd all figures. All authors read and approved the final manuscript.

Data availability

The datasets used and analysed during the current study are available in the following repositories: The TACoS dataset is available at https://zenodo.org/records/15379789. The QVHighlights dataset is available at https://github.com/jayleicn/moment_detr.

Declarations

Competing interests

The authors declare no competing interests.

Footnotes

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Bing Yu and Jingyu Li contributed equally to this work.

References

  • 1.Liu, M., Nie, L., Wang, Y., Wang, M. & Rui, Y. A survey on video moment localization. ACM Comput. Surv.55, 1–37. 10.1145/3556537 (2023). [Google Scholar]
  • 2.Carion, N. et al. End-to-end object detection with transformers. In Computer Vision—ECCV 2020: 16th European Conference 213–229 10.1007/978-3-030-58452-8_13 (2020).
  • 3.Moon, W., Hyun, S., Park, S., Park, D. & Heo, J.-P. Query-dependent video representation for moment retrieval and highlight detection. In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 23023–23033 10.1109/cvpr52729.2023.02205 (2023).
  • 4.Liu, Y. et al. Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection. In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 3042–3051 10.1109/cvpr52688.2022.00305 (2022).
  • 5.Jang, J., Park, J., Kim, J., Kwon, H. & Sohn, K. Knowing where to focus: Event-aware transformer for video grounding. In Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV) 13846–13856 10.1109/iccv51070.2023.01273 (2023).
  • 6.Gao, J., Sun, C., Yang, Z. & Nevatia, R. Tall: Temporal activity localization via language query. In Proc. of the IEEE International Conference on Computer Vision (ICCV)10.1109/iccv.2017.563 (2017).
  • 7.Xu, H., Das, A. & Saenko, K. R-c3d: Region convolutional 3d network for temporal activity detection. In Proc. of the IEEE International Conference on Computer Vision (ICCV)10.1109/iccv.2017.617 (2017).
  • 8.Xu, H. et al. Multilevel language and vision integration for text-to-clip retrieval. In Proc. of the Thirty-Third AAAI Conference on Artificial Intelligence10.1609/aaai.v33i01.33019062 (2019).
  • 9.Rossi, E. et al. Temporal graph networks for deep learning on dynamic graphs. In ICML 2020 Workshop on Graph Representation Learning (2020).
  • 10.Xiao, S., Chen, L., Shao, J., Zhuang, Y. & Xiao, J. Natural language video localization with learnable moment proposals. In Proc. of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) 4008–4017 10.18653/v1/2021.emnlp-main.327 (2021).
  • 11.Nan, G. et al. Interventional video grounding with dual contrastive learning. In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2765–2775 10.1109/cvpr46437.2021.00279 (2021).
  • 12.Lei, J., Berg, T. L. & Bansal, M. Qvhighlights: Detecting moments and highlights in videos via natural language queries. In Proc. of the 35th International Conference on Neural Information Processing Systems(NeurIPS)10.5555/3540261.3541167 (2021).
  • 13.Xu, Y. et al. Mh-detr: Video moment and highlight detection with cross-modal transformer. 2024 International Joint Conference on Neural Networks (IJCNN) 1–8.10.1109/IJCNN60899.2024.10650814 (2024).
  • 14.Ma, H., Wang, G., Yu, F., Jia, Q. & Ding, S. Ms-detr: Towards effective video moment retrieval and highlight detection by joint motion-semantic learning. In Proc. of the 33rd ACM International Conference on Multimedia 4514–4523 10.1145/3746027.3755484 (2025).
  • 15.Zhang, H. et al. Video corpus moment retrieval with contrastive learning. In Proc. of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval 685–695 10.1145/3404835.3462874 (2021).
  • 16.Panta, L. et al. Cross-modal contrastive learning with asymmetric co-attention network for video moment retrieval. In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), 617–624, 10.1109/wacvw60836.2024.00071 (2024).
  • 17.Gao, J. & Xu, C. Fast video moment retrieval. In Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV), 1523–1532, 10.1109/iccv48922.2021.00155 (2021).
  • 18.Zeng, Y., Zhang, X. & Li, H. Multi-grained vision language pre-training: Aligning texts with visual concepts. In Proc. of the 38th International Conference on Machine Learning (ICML) (2021).
  • 19.Xue, Y. et al. Fmtrack: Frequency-aware interaction and multi-expert fusion for rgb-t tracking. IEEE Trans. Circuits Syst. Video Technol.10.1109/TCSVT.2025.3601598 (2026). [Google Scholar]
  • 20.Xue, Y. et al. Target-distractor aware uav tracking via global agent. IEEE Trans. Intell. Transp. Syst.10.1109/TITS.2025.3581391 (2025). [Google Scholar]
  • 21.Wu, W. et al. Adaptive patch contrast for weakly supervised semantic segmentation. Eng. Appl. Artif. Intell.10.1016/j.engappai.2024.109626 (2025).41383559 [Google Scholar]
  • 22.Wu, W. et al. Image fusion for cross-domain sequential recommendation. In Companion Proceedings of the ACM on Web Conference 2025, 10.1145/3701716.3717566 (2025).
  • 23.Wu, W. et al. Tag-enriched multi-attention with large language models for cross-domain sequential recommendation. IEEE Trans. Consum. Electron.10.1109/TCE.2025.3620527 (2025). [Google Scholar]
  • 24.Wu, W. et al. Llm-enhanced multimodal fusion for cross-domain sequential recommendation. ArXivarXiv:2506.17966, 10.48550/arXiv.2506.17966 (2025).
  • 25.Fang, X. et al. Fewer steps, better performance: efficient cross-modal clip trimming for video moment retrieval using language. In Proc. of the Thirty-Eighth AAAI Conference on Artificial Intelligence10.1609/aaai.v38i2.27941 (2024).
  • 26.Hou, D., Pang, L., Shen, H. & Cheng, X. Event-aware video corpus moment retrieval. arXiv preprintarXiv:2402.1356610.48550/ARXIV.2402.13566 (2024).
  • 27.Liu, W. et al. Context-enhanced video moment retrieval with large language models. IEEE Transactions on Multimedia 6296–6306 10.1109/TMM.2025.3581797 (2025).
  • 28.Jiang, Y. et al. Prior knowledge integration via LLM encoding and pseudo event regulation for video moment retrieval. In Proc. of the 32nd ACM International Conference on Multimedia (ACM MM), 10.1145/3664647.3681115 (2024).
  • 29.Gu, A., Goel, K. & Re, C. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations (ICLR) (2022).
  • 30.Gu, A. & Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprintarXiv:2312.00752 (2023).
  • 31.Dao, T. & Gu, A. Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality. In International Conference on Machine Learning (ICML)10.5555/3692070.3692469 (2024).
  • 32.Zhu, L. et al. Vision mamba: Efficient visual representation learning with bidirectional state space model. In Forty-first International Conference on Machine Learning (ICML)10.5555/3692070.3694654 (2024).
  • 33.Liu, Y. et al. VMamba: Visual state space model. In The Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS), 10.52202/079017-3273 (2024).
  • 34.Wang, Z., Li, C., Xu, H., Zhu, X. & Li, H. Mamba yolo: a simple baseline for object detection with state space model. In Proc. of the Thirty-Ninth AAAI Conference on Artificial Intelligence10.1609/aaai.v39i8.32885 (2025).
  • 35.Li, H. et al. Cfmw: Cross-modality fusion mamba for robust object detection under adverse weather. IEEE Trans. Circuits Syst. Video Technol.10.1109/tcsvt.2025.3587918 (2025). [Google Scholar]
  • 36.Lan, P., Xian, Y., Shen, T., Lee, Y. & Zhao, Q. Semantic-guided mamba fusion for robust object detection of tibetan plateau wildlife. Electronics10.3390/electronics14224549 (2025). [Google Scholar]
  • 37.Li, K. et al. Videomamba: State space model for efficient video understanding. In Computer Vision—ECCV 2024: 18th European Conference, 237–255, 10.1007/978-3-031-73347-5_14 (2024).
  • 38.Moon, W., Hyun, S., Lee, S. & Heo, J.-P. Correlation-guided query-dependency calibration in video representation learning for temporal grounding. arXiv preprintarXiv:2311.08835 (2023).
  • 39.Liu, Z. et al. Towards balanced alignment: Modal-enhanced semantic modeling for video moment retrieval. In Proc. of the AAAI Conference on Artificial Intelligence38, 3855–3863. 10.1609/aaai.v38i4.28177 (2024).
  • 40.Gu, A. et al. Combining recurrent, convolutional, and continuous-time models with linear state-space layers. In Proc. of the 35th International Conference on Neural Information Processing Systems (NeurIPS)10.5555/3540261.3540305 (2021).
  • 41.Gu, A., Dao, T., Ermon, S., Rudra, A. & Ré, C. Hippo: recurrent memory with optimal polynomial projections. In Proc. of the 34th International Conference on Neural Information Processing Systems (NeurIPS), 10.5555/3495724.3495849 (2020).
  • 42.Liu, S. et al. DAB-DETR: Dynamic anchor boxes are better queries for DETR. In International Conference on Learning Representations (ICLR) (2022).
  • 43.Rezatofighi, H. et al. Generalized intersection over union: A metric and a loss for bounding box regression. In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/cvpr.2019.00075 (2019).
  • 44.Wu, W., Luo, H., Fang, B., Wang, J. & Ouyang, W. Cap4video: What can auxiliary captions do for text-video retrieval? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 10704–10713 10.1109/CVPR52729.2023.01031 (2023).
  • 45.Primus, P., Schmid, F. & Widmer, G. Tacos: Temporally-aligned audio captions for language-audio pretraining. In 2025 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)10.1109/waspaa66052.2025.11230997 (2025).
  • 46.Feichtenhofer, C., Fan, H., Malik, J. & He, K. Slowfast networks for video recognition. In Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/iccv.2019.00630 (2019).
  • 47.Radford, A. et al. Learning transferable visual models from natural language supervision. In Proc. of the 38th International Conference on Machine Learning (ICML) (2021).
  • 48.Tran, D., Bourdev, L., Fergus, R., Torresani, L. & Paluri, M. Learning spatiotemporal features with 3D convolutional networks. In 2015 IEEE International Conference on Computer Vision (ICCV), 4489–4497, 10.1109/iccv.2015.510 (2015).
  • 49.Pennington, J., Socher, R. & Manning, C. D. Glove: Global vectors for word representation. In Proc. of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1532–1543, 10.3115/v1/d14-1162 (2014).
  • 50.Wang, Z., Wang, L., Wu, T., Li, T. & Wu, G. Negative sample matters: A renaissance of metric learning for temporal grounding. In Proc. of the AAAI Conference on Artificial Intelligence 2613–2623 10.1109/cvpr52729.2023.01031 (2022).
  • 51.Lin, K. Q. et al. Univtg: Towards unified video-language temporal grounding. In Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV) 2794–2804 10.1109/iccv51070.2023.00262 (2023).
  • 52.Tang, K., He, L., Dang, J. & Gao, X. Boosting temporal sentence grounding via causal inference. In Proc. of the 33rd ACM International Conference on Multimedia10.1145/3746027.3755624 (2025).
  • 53.Hu, J. et al. Maskable retentive network for video moment retrieval. In Proc. of the 32nd ACM International Conference on Multimedia (ACM MM), 10.1145/3664647.3680746 (2024).
  • 54.Jang, J., Park, J., Kim, J., Kwon, H. & Sohn, K. Knowing where to focus: Event-aware transformer for video grounding. In Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/iccv51070.2023.01273 (2023).
  • 55.Xiao, Y. et al. Bridging the gap: A unified video comprehension framework for moment retrieval and highlight detection. arXiv preprintarXiv:2311.1646410.1109/cvpr52733.2024.01770 (2023).
  • 56.Chen, B. et al. From global to granular: Revealing iqa model performance via correlation surface, 10.48550/arXiv.2601.21738 (2026) [DOI] [PubMed]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The datasets used and analysed during the current study are available in the following repositories: The TACoS dataset is available at https://zenodo.org/records/15379789. The QVHighlights dataset is available at https://github.com/jayleicn/moment_detr.


Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES