Comprehensive evaluation of structural variation detection for germline and somatic analysis with long-read sequencing data

Abstract

Detecting structural variations (SVs) via long-read sequencing remains difficult due to algorithmic variations and genomic complexity, alongside a shortage of comprehensive benchmarks for somatic variants. We present a unified benchmarking framework covering both germline and somatic SV detection, which evaluates 14 long-read callers across 20 core datasets from Pacific Biosciences (PacBio) Continuous Long Reads (CLR), Circular Consensus Sequencing (CCS), and Oxford Nanopore Technologies (ONT) platforms. Performance was analyzed across 12 dimensions using metrics including baseline artefact rate, Mendelian discordance rate (MDR), and Mendelian inheritance error rate (MIER). For germline SVs, DeBreak, cuteSV2, and SVDF showed stable and accurate detection. cuteSV2 and SVHunter maintained consistent genotyping accuracy across sequencing depths, whereas Severus and cuteSV2 performed well in identifying complex SVs such as inversions, duplications, and translocations. In tumor datasets, tools designed specifically for somatic SVs outperformed germline callers. Severus and SAVANA showed strong overall somatic performance, and nanomonsv showed distinct advantages at low coverage. These results offer practical references for tool selection under various sequencing scenarios, supporting future algorithmic development. The code and resources are available at https://github.com/model-lab/LR-SV-Benchmark.

Introduction

There are three main types of differences in human genes: single-nucleotide variations (SNVs), small insertions and deletions (indels), and structural variations (SVs). Among them, SVs are large genomic alterations spanning at least 50 base pairs and include deletions (DELs), insertions (INSs), duplications (DUPs), inversions (INVs), and translocations (TRAs) [1]. The number of SNVs in the human genome is greater than that of SVs, but the total number of nucleotides altered by SVs is larger. Therefore, SVs are the main cause of genetic differences between individuals [2, 3]. These large-scale genomic variations significantly affect human traits and also alter an individual’s risk of developing diseases [4]. The functional consequences of SVs are often context-dependent. While inherited germline SVs are linked to developmental disorders, neurological conditions, and congenital defects [5–7], somatic SVs acquired over time drive oncogenesis by disrupting genomic stability, activating oncogenes, or inactivating tumor suppressors [8].

Accurately identifying SVs helps researchers understand the pathogenesis of complex diseases. Short-read sequencing offers high accuracy for detecting SNVs and small indels [9]. However, its limited read length constrains its performance in highly repetitive genomic regions and complex structural rearrangement sites [10]. Long-read sequencing technologies help overcome these limitations. Prominent examples include Pacific Biosciences (PacBio) [11] and Oxford Nanopore Technologies (ONT) [12]. These platforms generate extended, continuous reads typically spanning tens of kilobases to over 1 megabase (Mb). Such read lengths improve genome assembly reliability and breakpoint resolution [13]. PacBio platforms utilize single-molecule real-time (SMRT) sequencing to produce Continuous Long Reads (CLR) up to 200 kb, alongside High-Fidelity (HiFi) reads (also called Circular Consensus Sequencing, CCS) with base accuracies reaching 99.9% with base accuracies reaching 99.9%. In contrast, ONT sequencing yields ultra-long reads exceeding 1 Mb, while single-molecule sequencing accuracy continues to advance [14].

Advances in long-read sequencing have driven the rapid development of SV detection algorithms, yielding various tools tailored for specific research contexts [15]. Germline SV methods, including SVIM [16], cuteSV2 [17], and Sniffles2 [18], primarily target variation signals in diploid genomes. Recently, deep learning–based frameworks, including NanoVar [19], SVision [20], and SVHunter [21], have been developed to improve accuracy. Meanwhile, the complexity of cancer genomes has prompted the development of specialized somatic SV tools like nanomonsv [22] and SAVANA [23]. These tools utilize tumor-normal paired analyses to filter out background noise and resolve detection challenges related to subclonal structures. Additionally, tools supporting dual-mode detection, such as Delly2 [24] and Severus [25], support dual-mode detection for both germline and somatic contexts. However, due to genomic complexity and platform-specific sequencing noise, the performance of these algorithms varies significantly, often exhibiting low concordance between tools [26]. These issues highlight the need for a systematic benchmarking framework to guide method selection and algorithmic optimization across research applications [27].

Previous benchmarking studies on SV detection tools have notable limitations. For example, early work by Kosugi et al. [28] focused exclusively on short-read algorithms without testing long-read SV callers. Similarly, Duan et al. [29] compared only a limited selection of early tool versions. Although Liu et al. [30] recently evaluated alignment and assembly methods, they did not incorporate family-based data or verify Mendelian inheritance consistency. Because acquiring high-confidence truth sets remains difficult, metrics such as the Mendelian discordance rate (MDR) and the Mendelian inheritance error rate (MIER) serve as practical alternatives for evaluating genotyping reliability [31]. Furthermore, few studies have tested algorithms specifically designed for somatic SVs; for instance, Aydin et al. [32] evaluated several standard tools on tumor samples using germline algorithms. Because these algorithms are unsuitable for somatic data, they performed poorly when processing mixed tumor signals. The field still lacks a unified benchmarking framework that integrates dual analysis pipelines to test algorithms using stable germline trio data alongside mixed somatic datasets. For germline evaluation, resources are provided by Genome in a Bottle (GIAB) [33], the Human Genome Structural Variation Consortium (HGSV) [34], and the Human Pangenome Reference Consortium (HPRC) [35]. For somatic evaluation, several validated standards are available. These standards include experimentally validated cell lines such as HCC1395 [36] and COLO829 [37], as well as the HG008 benchmark set generated by NIH/NCI laboratories through multi-algorithm integration [25].

To address current methodological gaps, we establish a systematic evaluation framework leveraging diverse PacBio and ONT datasets to benchmark 14 long-read SV callers released between 2012 and 2026. Performance is assessed across 12 key dimensions, which range from breakpoint precision and genotyping accuracy to Mendelian inheritance consistency. Our findings provide four primary insights.

• A unified evaluation framework is developed to simultaneously benchmark long-read algorithms across both germline and somatic workflows, addressing the limitation of previous studies being restricted to single scenarios.

• Leveraging trio datasets from diverse platforms, the analysis incorporates metrics such as the MIER. This integration extends beyond traditional precision-recall metrics, enabling researchers to directly compare the genotyping accuracy of different algorithms.

• Through systematic testing of specialized somatic SV tools across multiple tumor datasets, their empirical detection capabilities in cancer samples are systematically characterized, with Severus and SAVANA demonstrating the best overall performance in tumor analysis.

• Synthesizing performance data across all 12 evaluation dimensions yields benchmark-derived guidance designed to facilitate workflow optimization across diverse research scenarios.

Materials and methods

Overview of the benchmarking framework

In this study, a unified evaluation framework encompassing both germline and somatic SV detection was developed to systematically assess the performance of various callers (Fig. 1). Using PacBio (CLR/CCS) and ONT data, we assessed five core SV types (DEL, INS, INV, DUP, and TRA). We benchmarked 14 mainstream tools on aligned reads, categorized by application: nine germline-specific tools spanning deep learning, machine learning, and heuristic methods (NanoVar (v1.8.3) [19], SVision (v1.4.0) [20], SVDF (default) [38], SVHunter (v1.1.0) [21], Dysgu (v1.7.0) [39], SVIM (v2.0.0) [16], cuteSV2 (v2.1.3) [17], DeBreak (v1.0.2) [40], and Sniffles2 (v2.6.3) [18]); two somatic-specific tools (nanomonsv (v0.8.0) [22] and SAVANA (v1.6.0) [23]); and three versatile dual-mode callers (SVision-pro (v2.5.0) [41], Delly2 (v1.5.0) [24], and Severus (v1.6.0) [25]). Callsets were generated under uniform baselines, standardized, and evaluated against ground-truth datasets using Truvari (v5.4.0) [42] and Minda (v0.0.2) [25] to compute precision, recall, and F1 scores. Given that most callers output TRA variants as breakends (BND), BND records in the Variant Call Format (VCF) files were uniformly extracted to represent TRA variants for benchmarking, following the approach described by Jiang et al. [17]. The supplementary note provides comprehensive methodologies, detailed tool specifications (Supplementary Table S1), and a discussion of their algorithmic evolution. To ensure a fair and objective comparison, each dataset was processed using its optimal and standard reference genome and aligner, maintaining a strictly unified testing environment for all evaluated tools within that dataset. Given that upstream inputs, evaluation rules, and post-processing standards (via Truvari and Minda) were held constant within each benchmark condition, the SV caller served as the sole comparative variable. Consequently, the observed performance differences among tools primarily stemmed from their distinct algorithmic architectures and internal computational logic.

Figure 1.

Benchmarking workflow of germline and somatic SV detection tools based on long-read sequencing. The framework utilizes PacBio (CLR, CCS) and ONT datasets from germline and tumor-normal paired samples. It evaluates 14 mainstream SV callers across two major analysis modes (germline and somatic): nine germline-specific, two somatic-specific, and three versatile multi-scenario tools. Systematic performance is assessed through multi-dimensional downstream analyses.

SV benchmark datasets and ground truths

To comprehensively evaluate the SV callers, we utilized a combination of simulated and real sequencing datasets across data from three long-read types (CCS, CLR, and ONT) for both germline and somatic SV detection. Briefly, simulated datasets were generated based on authentic SV signals and platform-specific error profiles to establish precise ground truth. Real germline datasets included the Ashkenazi and Chinese trios (e.g. HG002), as well as the CHM13 cell line. For somatic SV evaluation, we constructed synthetic tumor-normal pairs using CHM1/CHM13 mixtures and employed three representative real tumor-normal paired cell lines: HCC1395, COLO829, and HG008. The ground-truth SV sets were derived from simulation seed variants, widely accepted consortium baselines (e.g. GIAB), de novo assemblies, or consensus benchmark sets from published multi-omics studies. Detailed dataset construction parameters, simulation workflows, and ground-truth generation methodologies are provided in the Supplementary Notes and Supplementary Tables S2–S4.

Performance evaluation of germline SV calling

For simulated datasets, we adopted the evaluation framework by Jiang et al. [17]. Predicted DELs, INSs, INVs, and DUPs were classified as true positives (TPs) if they met equation (1):

(1)

Where subscripts , , , and denote the start coordinate, end coordinate, length, and variant type, respectively; “ and “bench” represent the predicted variant and baseline gold standard. Translocations (TRAs; 368 reciprocal TRAs comprising 1472 breakends) were evaluated at the breakpoint and breakend levels. A predicted TRA was a TP at these levels if it satisfied equation (2) or (3), respectively:

(2)

(3)

Where , , , , , and represent breakpoints, chromosomes, and connectivity. Variants failing equations (1)–(3) were false positives (FPs), while unmatched baseline SVs were false negatives (FNs) [43–47]. Performance metrics were calculated as [48]:

(4)

(5)

(6)

TPs with correct genotypes were designated as genotyping TPs (TP-GT) and evaluated identically. These workflows were implemented using custom Python scripts. For real datasets, Truvari benchmarked detected SVs (DEL/INS) against baselines to compute precision, recall, and F1 scores.

Breakpoint and length deviation analysis

To evaluate the fidelity of callers in resolving precise SV breakpoints and lengths, we assessed the proportions of zero-breakpoint deviation () and zero-length deviation () across CHM13 datasets (CLR, CCS, ONT). Using Truvari, we applied stringent thresholds: breakpoint distance (--refdist) was restricted to 0 bp to verify base-level coordinate accuracy, and size similarity (--pctsize) was set to 1.0 to confirm exact length matching between predicted and baseline variants. Metrics were calculated as follows:

(7)

(8)

Where and represent true positive variants meeting zero-distance and equal-length criteria, respectively, and is the total number of detected variants. Additionally, a two-dimensional grid search on the CHM13 CCS dataset profiled F1 score fluctuations by tuning size similarity (from 0 to 1.0, 0.1 increments) and reference distance (from 0 to 1 kb, 100 bp increments).

Baseline artefact rate determination

The present study utilizes the human T2T-CHM13 genome, which is validated by multiple orthogonal technologies and widely recognized as a high-quality standard, as the reference sequence [49]. Ideally, if raw CHM13 sequencing reads are mapped back to this reference, an SV detector should yield no variant calls. However, in practical analysis, observed residual calls may result more from mapping artefacts, repeat collapse, or assembly-specific representation differences rather than genuine algorithmic false discoveries. Given this, the current benchmarking test aims to evaluate the anti-artefact and background noise suppression capabilities of each tool when dealing with complex genomic backgrounds. In other words, an excellent detection algorithm should minimize such residual calls when processing this isogenic (self-versus-self) dataset. To ensure a strictly unbiased comparison, identical parameter thresholds were applied to all evaluated tools when processing both the T2T-CHM13 and GRCh38 reference genomes. By mapping multi-platform sequencing data (CLR, CCS, ONT) to the T2T-CHM13 sequence and performing a normalized comparison with the calls detected using the GRCh38 reference genome, we quantitatively evaluated the baseline artefact ratio for each detection tool:

(9)

Where and are the total SVs detected when aligning CHM13 data to the T2T-CHM13 and GRCh38 genomes, respectively. Following the studies by Denti et al. [50], we use as the denominator to normalize the residual detection results, thereby enabling a standardized quantitative evaluation of each tool’s ability to resist artefacts and suppress background noise. This metric should be interpreted as a relative indicator reflecting background noise consistency, rather than a false discovery rate in the traditional statistical sense.

Although CHM13 is highly homozygous, callers may erroneously detect heterozygous SVs due to in vitro passaging artefacts or trace inherent heterozygosity. We evaluated whether these calls were true variants using the T2T consortium’s high-confidence heterozygous dataset [51]. Heterozygous precision (Het Precision) was calculated as:

(10)

Where represents the T2T-detected SVs intersecting with the validated heterozygous ground truth set.

Mendelian discordance rate and inheritance error rate calculation

SVs in offspring typically adhere to Mendelian inheritance patterns, with de novo mutations occurring at a low frequency, averaging 0.160–0.206 events per genome [52–54]. Thus, evaluating Mendelian discordance and error rates in pedigrees effectively assesses caller consistency and genotyping accuracy. We utilized two standard trios (Ashkenazi: HG002/HG003/HG004; Chinese: HG005/HG006/HG007) sequenced at 30× depth across CLR, CCS, and ONT platforms. The MDR, representing the proportion of offspring SVs lacking parental matches, is calculated as follows:

(11)

Where denotes the total number of SVs identified exclusively in the offspring without matching calls in either parent, and represents the total number of SVs identified in the offspring. A matching failure occurs if the breakpoint distance exceeds 1 kbp or size similarity is <0.7 equation (12). A lower MDR indicates a reduced false discovery rate.

(12)

Genotyping accuracy was quantified using the MIER:

(13)

Where represents SVs with valid trio genotypes, and denotes Mendelian violations (e.g. 0/0 parents yielding a 0/1 offspring). We integrated trio-based SV callsets using the SURVIVOR (v1.0.7) [55] merge function with a 100 bp threshold, keeping only variants with non-missing genotypes. A lower MIER typically indicates a higher genotyping accuracy of a detection tool, reflecting greater reliability in distinguishing between homozygous and heterozygous genotypes. The aforementioned evaluation workflows were implemented using custom Python scripts.

Implementation and evaluation of somatic SV calling

For somatic SV analysis (≥50 bp), SVision-pro, Delly2, SAVANA, and Severus simultaneously processed paired tumor-normal Binary Alignment Map (BAM) files using default parameters. For nanomonsv, the default “parse” step was followed by the “get” step (-use_racon, -single_bnd). For Sniffles2, as the “Mosaic” mode reduced F1 scores, a multi-sample mode incorporating tandem repeat annotations [56] was adopted; tumor-exclusive variants were then filtered from the merged Sniffles Native Format (SNF) output. An “independent detection and filtering” strategy was applied to germline callers (cuteSV, DeBreak, SVIM, SVDF, SVHunter, NanoVar, SVision, Dysgu), following the benchmarking practice described by Keskus et al. [25]. Following default paired detection, Jasmine (v1.1.5) [57] merged the callsets (--dup_to_ins --max_dist 500), and BCFTools (v1.22.1) [58] extracted tumor-exclusive SVs. Performance was evaluated using Minda (-tolerance 500). All complete command-lines parameters and execution scripts are provided in the Supplementary Materials.

Somatic SV detection performance under varying tumor purity and sequencing depth

Normal cell contamination in heterogeneous tumor samples poses a significant challenge to the accurate detection of somatic SVs [59, 60]. To systematically evaluate the robustness of existing algorithms and sequencing platforms under complex conditions, we constructed a series of in silico mixed datasets to quantify the impact of tumor purity and sequencing depth on SV detection performance. Using high-quality COLO829 and HG008 cell line data, we utilized Samtools (v1.22.1) [58] to admix tumor and matched-normal sequencing reads at various ratios to simulate tumor purities ranging from 20% to 100% (with a controlled total sequencing depth of ~30×). These datasets were then comprehensively evaluated against the benchmark truth set described earlier. Furthermore, to investigate the effect of sequencing depth on somatic SV detection, we downsampled the 100% purity samples across a gradient from 10× to 60×.

Somatic SV detection via the tumor-only strategy

In detection scenarios lacking paired normal samples, the “tumor-only” modes provided by the somatic structural variant callers Severus and SAVANA were evaluated. Severus was executed using the “-PON” parameter to incorporate the panel-of-normals dataset from the 1000 Genomes Project (PoN_1000G_hg38.tsv.gz). Conversely, SAVANA was run via its “savana to” mode, specifying the 1000 Genomes Project reference VCF file through the “--g1000_vcf” parameter.

Runtime and memory consumption calculation

In the germline SV benchmarking, an HG002 sample with a mean sequencing depth of 5× was used, spanning three platforms: CLR, CCS, and ONT. The somatic SV evaluation was based on the CCS and ONT sequencing data of the COLO829 tumor sample and its paired normal sample, COLO829BL, with mean sequencing depths of 66× and 49×, respectively. All computational tasks were executed on a server equipped with dual Intel Xeon Platinum 8168 processors (totaling 48 cores and 96 threads, clocked at 2.70 GHz) and 1.0 TB of RAM. For tools that support multi-threading, the number of threads was uniformly set to 16. Computational resource consumption was monitored using the Linux command “/usr/bin/time -o <output> -v <caller command>,” which records elapsed wall-clock time and maximum resident set size. Each benchmark command was executed independently in triplicate.

Results

Germline SV calling performance on simulated datasets

We evaluated 12 germline-capable SV callers across five variant types using 15 simulated long-read datasets (Fig. 2A–F, Supplementary Tables S5–S22). SVision (lacking BND output) and DeBreak (lacking orientation information) were excluded from BND evaluations. Performance varied significantly by variant type. For DEL variant detection, SVision consistently led (F1 ≥ 0.94), with SVIM (CLR: 0.94), Sniffles2 (ONT: 0.93), and NanoVar/SVHunter (CCS: 0.90–0.92) performing strongly. For INS variant detection, SVIM excelled (CCS: 0.88), while Sniffles2, SVHunter, and SVision clustered (F1: 0.84–0.86). For INV variant detection, SVision dominated CLR (F1 = 0.77), whereas Delly2 led on ONT/CCS (F1 = 0.71–0.73). DUP variant detection favored Delly2 (ONT/CCS: F1 = 0.84–0.85) and Severus (CLR: F1 = 0.83). Regarding BND variant detection, Delly2, cuteSV2, and Severus consistently topped breakpoint-level detection (F1 = 0.94–0.96); performance declined at the breakend level, with Delly2 (F1 ≈ 0.80) and Severus (F1 ≥ 0.76) remaining superior.

Figure 2.

Benchmarking of SV callers on simulated and real datasets. (A–F) Radar charts of F1 scores for 12 callers across six dimensions: three sequencing platforms (PacBio CLR, PacBio CCS, ONT) and their genotyping-incorporated modes (suffix “-GT”). Panels denote variant types: (A) DEL, (B) INS, (C) INV, (D) DUP, (E) TRA at the breakpoint level, and (F) TRA at the breakend level. (G–L) Genotyping F1 scores for DEL (G–I) and INS (J–L) across varying sequencing depths using HG002 and CHM13 datasets.

Precision and recall patterns reflected distinct algorithmic strategies. For DEL variant detection, NanoVar (precision: 0.97) and SVision (precision: 0.96) prioritized accuracy, while Dysgu, SVIM, and SVision maximized sensitivity (recall: 0.91–0.94), with SVision achieving a balanced precision-recall profile (Supplementary Fig. S1). For INS variant detection, Delly2 (precision >0.92) demonstrated exceptional accuracy, while SVIM and SVision led in sensitivity (recall: 0.92–0.93, Supplementary Fig. S2). In summary, SVision, SVIM, and cuteSV2 are optimal for simple variants (DEL/INS), while Delly2 and Severus provide superior detection for complex rearrangements (INV, DUP, BND). NanoVar and Delly2 achieve top precision via stringent false-positive suppression, whereas Dysgu and SVIM prioritize high recall.

Germline SV calling performance on real datasets

We further evaluated callers on real human samples, HG002 and CHM13. For HG002, SVDF, cuteSV2, and DeBreak demonstrated superior performance (Fig. 3A–F, Supplementary Tables S23–S25). In CLR (69×) datasets, SVDF and DeBreak led DEL detection (F1 = 0.98), while cuteSV2 excelled in INS (F1 = 0.93). In CCS (28×) datasets, performance converged (F1: 0.94–0.96). In ONT (48×) datasets, SVDF and DeBreak led DEL and INS detection (F1 = 0.97 and 0.95, respectively). SVHunter maintained stable performance. Notably, performance evaluation on the standard HG002 benchmark dataset mapped to both GRCh37 and GRCh38 revealed that most SV callers achieved slightly higher F1 scores on GRCh38 (Supplementary Tables S26–S28), likely reflecting the improved assembly quality and reduced artefacts in the newer reference genome. Regarding the CMRG benchmark, DeBreak, SVDF, and Delly2 proved most reliable for complex clinical SVs, consistently achieving F1 > 0.90 (Supplementary Fig. S3 and Table S29).

Figure 3.

Precision-recall curves for germline SV detection across three long-read sequencing platforms. (A–F) Results for the HG002 sample; (G–L) Results for the CHM13 sample. Rows 1 and 3 display DELs, while rows 2 and 4 display INSs. Enlarged insets are provided to resolve caller performance in high-density regions.

CHM13’s complex genomic background elevated detection difficulty, causing universal F1 attenuation (Fig. 3G–L, Supplementary Table S30). In CLR (54×) datasets, Sniffles2, SVDF, and DeBreak led in detection performance (F1 > 0.65). In CCS (33×) datasets, SVIM, NanoVar, and Sniffles2 led (F1 > 0.75), while in ONT (123×) datasets, SVDF, NanoVar, and SVIM consistently exceeded F1 = 0.70. Notably, SVision-pro performed suboptimally on CLR due to a lack of platform-specific optimization. Precision and recall analysis reveals distinct strategies: SVDF, Sniffles2, and Severus prioritize precision (DEL and INS precision ranging from 0.73 to 0.74), while NanoVar, SVIM, and SVision emphasize recall (recall >0.73). Synthesis of these benchmarks shows that transitioning to the complex CHM13 background highlights the stability of SVDF, DeBreak, and Sniffles2. For clinical CMRG regions, DeBreak, SVDF, and Delly2 remain highly effective. Ultimately, SVDF, Sniffles2, and Severus function as conservative, high-precision pipelines, whereas NanoVar, SVIM, and SVision operate as sensitive tools prioritizing comprehensive variant discovery.

Genotype calling accuracy for germline SV

Accurate genotyping is vital for downstream SV analysis. We assessed genotype concordance across 12 callers, excluding Severus (somatic-specific) and SVIM (TRA genotyping unavailable). On simulated datasets, Delly2 delivered balanced performance, outperforming others in genotyping complex SVs (INV, DUP, TRA) (Fig. 2A–F, Supplementary Tables S5–S22). For DEL and INS variant detection, cuteSV2, Sniffles2, and SVHunter achieved robust accuracy (F1 > 0.81). Specifically, Sniffles2 led in DEL genotyping on CLR/ONT (F1 = 0.91), while SVHunter excelled on CCS (F1 = 0.88). For INV variant detection, Delly2 maintained a distinct advantage (F1 > 0.70), while most tools clustered at 0.40–0.60. DeBreak led DUP genotyping (F1 = 0.64), closely followed by Delly2. For BND variant detection, Delly2 and NanoVar significantly surpassed other callers.

On real-world sequencing datasets and the complex CHM13 benchmark, cuteSV2 and SVHunter demonstrated exceptional robustness across sequencing platforms and depths (Fig. 2G–L, Supplementary Tables S31–S34). PacBio CCS datasets provided the most reliable foundation for SV genotyping, with top-tier tools routinely exceeding 0.90 in F1 score at standard sequencing depth. Specifically, cuteSV2 attained peak performance in DEL and INS genotyping, achieving F1 scores up to 0.96 on high-depth CLR/ONT datasets and maintaining high accuracy under low-depth conditions (e.g. >0.70 at 5×). In parallel, SVHunter and DeBreak exhibited strong consistency and strong performance for INS detection, particularly outperforming other tools on long-read platforms. While high-depth standard datasets allow tools like cuteSV2, Sniffles2, and Delly2 to reach near-optimal accuracy (F1 > 0.90), performance on the challenging CHM13 benchmark generally drops to 0.70–0.76 for DEL and INS, highlighting ongoing hurdles in segmental duplications and repetitive genomic regions. Overall, cuteSV2, Sniffles2, and SVHunter provide the highest overall accuracy and resilience for standard DEL/INS genotyping, whereas specialized tools like Delly2 and Severus remain valuable options for resolving complex variant structures.

Impact of SV size, sequencing depth, platforms, and supporting reads on germline SV calling

SV size critically dictates detection accuracy by challenging alignment continuity and the ability of algorithms to capture variant features (Fig. 4A–F, Supplementary Tables S35–S37). Detection performance for both deletions and insertions typically peaks at 200–500 bp, a size range that optimally fits long-read window spans while avoiding short-read ambiguity and ultra-long alignment fragmentation. For short-to-medium variants (50 bp–2 kb), SVDF, cuteSV2, and Sniffles2 consistently achieve F1 scores above 0.95. For SVs greater than 5 kb, CCS F1 scores drop sharply (around 0.20), primarily constrained by library size selection bias and local depth drops over large structural breaks, whereas SVDF maintains ~1.00 for ultra-large DELs (10–50 kb) on CLR and ONT. INS performance declines substantially beyond 1 kb; for 10–50 kb INS on ONT, only SVDF (0.78) and cuteSV2 (0.77) remain reliable.

Figure 4.

Performance evaluation of SV callers across varying depths and length intervals. (A–F) F1 scores of 12 SV callers across various length intervals for deletion (A–C) and insertion (D–F) detection under three sequencing technologies. Gray bars indicate the baseline SV count distribution per interval. (G–L) Impact of sequencing depth on caller performance, illustrating F1 score variations for deletion (G–I) and insertion (J–L) detection.

Increasing sequencing depth substantially enhances F1 scores (Fig. 4G–L). CLR and ONT exhibit maximum gains from 5× to 10× (e.g. DEL F1 for SVDF increases by 15.1% and 7.6%, respectively) and plateau beyond 20×. In contrast, CCS performs robustly at lower depths, achieving ~0.90 DEL/INS F1 at 10× with marginal gains at 28×. While 10× (CCS) or 20× (CLR/ONT) suffices for cost-effective benchmarks, 15× (CCS) or 35× (CLR/ONT) is recommended for optimal INS detection.

Platform choice impacts INS more than DEL detection (Supplementary Fig. S4 and Supplementary Tables S5–S25, S30). Across simulated and HG002 data, CLR slightly outperforms others for DELs (F1 up to 0.98), whereas CCS universally dominates INS detection (F1 > 0.90). On CHM13, CCS yields the highest overall accuracy. This platform-dependent discrepancy is driven by distinct sequencing error profiles: the exceptionally high raw consensus accuracy of CCS prevents small insertions and deletions from being obscured by sequencing noise, whereas the higher random error rates of CLR and ONT complicate the precise localization of insertion breakpoints.

Evaluating minimum supporting read thresholds (2–10) reveals notable parametric sensitivity driven by the interplay between error profiles and algorithmic designs (Supplementary Fig. S5 and Table S38; auto-optimizing callers such as Delly2, Sniffles2, and SVHunter were excluded). On HiFi CCS data, lower thresholds (2–3) maximize F1 scores by preserving genuine variant signals embedded within low error rate alignments, whereas stricter cutoffs discard true positives due to local coverage drops. Conversely, in noisier CLR and ONT datasets, elevated thresholds (8–10) effectively suppress false positives originating from random sequencing errors and alignment artefacts, thereby safeguarding precision for tools like DeBreak and SVDF. Notably, SVDF demonstrates exceptional cross-platform parametric resilience.

The accuracy of SV breakpoints and lengths

Using a high-confidence assembly-derived variant set, we systematically evaluated caller precision on the CHM13 dataset via Truvari. For CLR data, breakpoint accuracy (zero-deviation) for deletions (DEL) remained highly stable, led by Severus (82.70%), SVDF (82.25%), and DeBreak (81.32%) (Fig. 5A–C, Supplementary Table S39). In contrast, insertion (INS) precision was comparatively lower, with only SVIM (53.85%) and Delly2 (38.24%) maintaining acceptable rates. Benefiting from HiFi reads, CCS breakpoint accuracy improved markedly, with Severus (81.19%) and cuteSV2 (78.80%) heading DEL detection, while SVDF (77.25%), Sniffles2 (73.56%), and SVIM (72.98%) excelled in INS. On ONT, SVDF (76.25%) achieved the highest DEL precision, whereas SVIM (62.97%) and Sniffles2 (61.15%) led in INS.

Figure 5.

Evaluation of SV breakpoint accuracy and parameter sensitivity. (A–F) Proportions of zero-breakpoint (A–C) and zero-length (D–F) deviation for SVs (DEL and INS) called by 12 tools across three sequencing platforms. (G) Heatmap of F1 scores from a grid search on the CHM13 CCS dataset, evaluating varying length-similarity (pctsize, y-axis: 0.0–1.0, interval: 0.1) and breakpoint distance thresholds (refdist, x-axis: 0–1000 bp, interval: 100 bp).

Regarding zero-length deviation (Fig. 5D–F, Supplementary Table  S40), CCS significantly outperformed other platforms in DEL performance, spearheaded by Severus (72.12%), Sniffles2 (70.11%), and SVDF (69.87%). Conversely, CLR DEL zero-deviation dropped below 16% for the majority of callers, with Delly2 representing a notable exception at 36.01%. For INS evaluation, SVIM (68.01%) and Dysgu (65.02%) dominated on CCS, while SVIM sustained a superior 42.39% rate on ONT. Grid searches on CCS data further revealed distinct parametric sensitivities (Fig. 5G). Enforcing strict length similarity (pctsize = 1.0) triggered a precipitous drop in F1 scores—for instance, cuteSV2 and NanoVar declined from 0.62 and 0.71 (at 0.90) to 0.31 and 0.35, respectively, while SVision dropped from 0.55 to 0.22. Meanwhile, tightening the breakpoint distance threshold (refdist) from 100 bp to 0 bp exerted minimal impact on Sniffles2, SVIM, and SVDF. These patterns indicate that precise length reconstruction, rather than mere breakpoint positioning, constitutes the primary performance bottleneck. Overall, Sniffles2, SVDF, and SVIM exhibit robust adaptability across diverse sequencing platforms and parametric constraints.

Baseline artefact rates of SV callers

To assess caller robustness, we evaluated the baseline artefact rate by mapping raw CHM13 reads to the T2T-CHM13 haploid assembly. Since this represents a perfectly homozygous reference, any detected SVs are categorized as mapping artefacts, residual noise, or cell line–specific discrepancies rather than true variants. Callers varied substantially in their capacity to suppress these artefacts (Fig. 6A–C, Supplementary Table S41). For CLR data, Severus achieved the lowest baseline artefact rate (0.01%), followed closely by DeBreak (0.02%) and Delly2 (0.26%). CCS baseline artefact rates were generally improved (<1%), with NanoVar performing best (0.10%). On ONT data, Sniffles2 (0.58%) and NanoVar (0.62%) maintained the lowest levels of background noise, whereas others exhibited higher counts of residual calls. Notably, NanoVar consistently maintained a baseline artefact rate <1% across all platforms, and Severus yielded the lowest absolute number of residual calls (1, 24, and 572 calls, respectively).

Figure 6.

Evaluation of baseline artefact rates, Mendelian inheritance consistency, and computational resource consumption. (A–C) Baseline artefact rates (lines) and total detected SVs (bars) derived from aligning CHM13 reads to the T2T-CHM13 genome. (D–I) MDRs (D–F) and MIERs (G–I) for 12 callers across three platforms using two pedigree datasets. (J–M) Resource consumption of germline and somatic callers, detailing runtime (J, K, min) and peak memory (L, M, GB).

To distinguish these artefacts from rare, genuine heterozygous loci, which potentially stem from somatic cell line mutations or residual heterogeneity, we cross-validated these calls against the T2T Consortium’s high-confidence heterozygous set (Supplementary Fig. S6 and Supplementary Table S41). In CCS, SVDF achieved the highest precision regarding genuine variants (Het Precision: 36.36%, 16/44 overlaps), followed by SVision (35.56%). In ONT, SVision outperformed others by identifying 108 known heterozygous variants (Het Precision: 20.45%). Ultimately, Severus and DeBreak excel at filtering background noise, while SVision, despite a higher baseline artefact rate, demonstrates a unique capability to capture genuine, naturally occurring heterozygous variants.

Mendelian discordance rate and inheritance error rate across callers

We evaluated MDR and MIER in two real trios to assess caller consistency and genotyping accuracy (Fig. 6D–I, Supplementary Tables S42–S53). SVHunter, SVIM, and cuteSV2 demonstrated robust MDR performance. SVIM led on CCS datasets (MDR: 5.31%, 5.55%), SVHunter excelled on CLR (15.54%, 18.93%), and SVision-pro topped ONT (10.11%, 7.36%). Notably, the low MDR values of SVision and DeBreak were due to a lack of TRA calls and missing essential breakpoint-orientation information, respectively. MDR was influenced by both data type and pedigree: HiFi CCS reads significantly reduced MDR compared to CLR. Furthermore, the Chinese trio yielded a higher MDR than the Ashkenazi trio (e.g. the MDR for cuteSV2 increased from 22.32% to 49.66% on CLR). While this preliminary observation may stem in part from inherent genomic differences, such as the generally lower burden of complex structural variants in Ashkenazi populations [61] compared with the higher burden of diverse rearrangements reported in East Asian cohorts [62], these architectural variations can complicate sequence alignment and zygosity calling. This, in turn, may contribute to the observed differences in MDR across pedigrees. Nevertheless, we acknowledge that a single-trio design per population is insufficient to conclusively isolate population-level effects. Other confounding variables, including individual-specific genetic features and technical artefacts introduced during sequencing or library preparation, likely contribute to these differences as well. Consequently, these findings warrant cautious interpretation, and broader cohort studies will be essential to rigorously disentangle population-specific genomic complexity from technical and sample-specific noise.

Regarding MIER, most callers maintained errors below 5%. Dysgu achieved the lowest and most stable MIER across all datasets (CLR: 0.49%, 0.38%; CCS: 0.60%, 0.82%; ONT: 2.15%, 0.51%). SVHunter ranked second (prominent in CLR/ONT), and Sniffles2 performed strongly on ONT/CCS. Severus was excluded due to its somatic-optimized genotyping logic, and SVIM’s low MIER was partially attributed to lacking TRA genotyping outputs. Across CCS datasets, INS exhibited higher MDR and MIER than DEL, while complex SVs (DUP, INV) yielded much higher errors than simple variants (Supplementary Fig. S7). Ultimately, Dysgu excels in genotyping accuracy, whereas SVHunter and SVIM provide superior detection stability and false-positive control.

Comparative evaluation of deep learning–based SV calling

Deep learning algorithms have become widely adopted for SV detection in long-read sequencing. Here, we evaluate and summarize these deep-learning tools, including NanoVar, SVDF, SVHunter, SVision, and SVision-pro, outlining their respective performance profiles. Results on simulated datasets show that SVision and SVision-pro achieve high performance in detecting deletions and insertions (Supplementary Fig. S8A and B). For complex SVs such as inversions, duplications, and translocations, SVision, SVDF, and NanoVar show strong accuracy and stability, with NanoVar outperforming other tools in translocation detection (Supplementary Fig. S8C–F). On real datasets, SVDF exhibits robust overall performance, maintaining high precision and F1 scores across all tests, whereas NanoVar and SVHunter show advantages only on specific sequencing platforms or for particular variant types (Supplementary Fig. S9). For genotyping, SVision-pro and SVision perform best overall, yielding higher F1 scores across different sequencing platforms and depths, while SVDF maintains stable, middle-tier performance (Supplementary Fig. S8G–L). Analysis across variant fragment lengths (Supplementary Fig. S10A–F) shows that SVDF adapts well to all sequencing platforms and exhibits high robustness across various length intervals, particularly for medium-to-large fragments and 10–50 kb deletions. SVHunter ranks second, showing balanced performance in detecting small-to-medium SVs and large insertions on specific ONT platforms. Evaluations across sequencing depths (Supplementary Fig. S10G–I) show that SVDF consistently yields high F1 scores with minimal fluctuation; even at an ultra-low sequencing depth of 5×, it suffers the smallest performance drop and maintains strong robustness. Regarding computational resource use (Supplementary Fig. S10J and K), SVision-pro and SVDF show high computational efficiency, with SVision-pro featuring low memory usage, though SVDF incurs higher memory overhead on the CLR platform. Conversely, SVHunter requires significantly longer execution times overall.

Somatic SV calling performance on real data

Assessing somatic structural variants remains a formidable task given the intrinsic complexity of cancer genomes [63, 64]. To address this, germline and somatic-specific callers were systematically benchmarked across three matched tumor-normal datasets. In HCC1395, performance metrics derived from ONT sequencing universally lagged behind CCS data, a disparity likely attributable to constrained sequencing depth (19×) (Fig. 7A and D, Supplementary Fig. S11A and B, and Supplementary Tables S54–S56). When CCS reads were used, somatic-tailored algorithms (Severus, SAVANA, and nanomonsv) substantially outperformed conventional germline callers, achieving respective F1 scores of 0.42, 0.42, and 0.40, whereas methods lacking dedicated somatic filters struggled to maintain F1 scores above 0.20. This performance divergence became even more pronounced within the COLO829 dataset (Fig. 7B and E and Supplementary Tables S57–S60). On CCS data, germline tools successfully captured numerous variants, yielding recall rates of 88.24% for SVDF and 76.47% for SVIM, yet excessive false positives severely depressed precision (<4.20%) and F1 scores (<0.08). In contrast, somatic-specific callers performed robustly. Although SAVANA, Severus, and nanomonsv reached high F1 scores of 0.74, 0.67, and 0.63, respectively, their overlapping 95% confidence intervals indicate comparable overall utilities rather than definitive rank separations (Supplementary Fig. S11C and D). On the ONT platform, SAVANA achieved high precision (56.82%) and F1 score (0.64).

Figure 7.

Performance evaluation of somatic SV detection tools. (A–F) Precision-recall curves for three real tumor datasets (HCC1395, COLO829, HG008) on CCS (A–C) and ONT (D–F) platforms. (G, H) Impact of sequencing depth on somatic F1 scores, illustrating the performance of various tools across sequencing depths ranging from 10× to 66× on PacBio CCS (G) and ONT (H) platforms. (I, J) Impact of tumor purity on somatic F1 scores, showing the F1 score trends of detection tools under tumor purity levels ranging from 20% to 100% using the COLO829 dataset on PacBio CCS (I) and ONT (J) platforms.

These general trends were consistently mirrored in the HG008 dataset (Fig. 7C and F and Supplementary Tables S61–S63). Under CCS conditions, somatic tools exhibited high recall, ranging from 87.82% for Severus to 92.31% for Delly2 and SAVANA, alongside well-controlled false positive counts. Severus, SAVANA, and Delly2 clustered closely with F1 scores of 0.80, 0.76, and 0.71, respectively, where intersecting confidence intervals denote shared tier-level efficacy rather than absolute superiority (Supplementary Fig. S11E and F). For ONT sequencing in HG008, SAVANA attained high precision (86.54% with 21 false positives) and an F1 score of 0.62, whereas Severus yielded high recall (94.87%) and an F1 score of 0.82, with overlapping confidence intervals indicating statistically comparable utility. Ultimately, these evaluations indicate that SAVANA and Severus show complementary strengths in precision and recall, respectively, while their overall performance remains statistically comparable when accounting for their confidence intervals. The performance contrast between somatic-specific and germline callers underscores the fundamental requirement for rigorous germline background filtration in somatic workflows.

Impact of sequencing depth and tumor purity on somatic SV calling

In this study, we performed a benchmark evaluation of somatic SV detection tools across a gradient of tumor purities and sequencing depths. Sequencing downsampling results (Fig. 7G and H) demonstrate that SAVANA and Severus maintain stable and competitive performance across multiple sequencing depths, including both high-depth and 10× coverage scenarios, while nanomonsv shows comparable competitiveness in the ONT dataset. Given the overlapping 95% confidence intervals across multiple depth intervals, the overall performance of SAVANA and Severus is statistically comparable. Conversely, Delly2’s detection efficacy depends on sequencing coverage; its F1 scores drop sharply as coverage decreases, indicating its limited utility for low-depth tumor samples.

Tumor purity gradient tests across different sequencing platforms (Fig. 7I and J) reveal tool-specific adaptation features. On the CCS platform, both SAVANA and Severus exhibit robust performance in the moderate-to-high tumor purity range of 100%–40%, though SAVANA’s performance declines significantly at an ultra-low purity of 20%. In contrast, Severus shows strong performance in low-purity scenarios, achieving F1 scores comparable to or higher than others at 20% purity (F1 score: 0.71). On the ONT platform, SAVANA and Severus demonstrate complementary performance across purity intervals, with SAVANA maintaining robust metrics in the 100%–60% moderate-to-high purity range (peaking at 0.69), whereas Severus shows stable values in the 40% and 20% low-purity intervals, yielding F1 scores of 0.70 and 0.59, respectively; these variations fall within intersecting confidence intervals and should be interpreted with caution. Meanwhile, nanomonsv demonstrates strong competitiveness in the 40%–60% intermediate purity range. Data from Supplementary Table S64 explain the unique performance pattern of Severus: reduced tumor purity decreases FP, thereby significantly improving precision. When the increase in precision outweighs the decline in recall, the composite F1 metric rises or remains stable despite the drop in purity. Purity gradient validation experiments on the HG008 dataset (Supplementary Fig. S12 and Table S65) yield conclusions highly consistent with the COLO829 sample: both SAVANA and Severus exhibit stable detection results within the 100%–40% moderate-to-high purity range (with SAVANA reaching an F1 score of 0.86), while Severus maintains robust performance under ultra-low purity conditions, achieving an F1 score of 0.85 at 20% purity on the CCS platform, with confidence intervals indicating comparable performance. Collectively, the results of all gradient tests indicate that SAVANA and Severus are effective for SV detection under varying sequencing depths and tumor purity conditions, while their overall performance remains statistically comparable when accounting for their respective uncertainty intervals. Note that all downsampled datasets and tumor purity gradients are derived from the same single sample, serving as technical replicates to evaluate tool performance under varying sequencing conditions rather than as independent biological replicates.

Somatic SV calling performance on CHM1-CHM13 synthetic datasets

Subsequently, we evaluated these tools using a simulated somatic dataset based on a synthetic CHM1/CHM13 mixture (Supplementary Fig. S13A and Supplementary Table S66). On this dataset, somatic-specific tools yielded F1 scores comparable to those of selected germline callers. Specifically, the highest overall performance was achieved by SVDF (F1 = 0.54) and Delly2 (F1 = 0.44), whereas nanomonsv (F1 = 0.27) and SAVANA (F1 = 0.25) displayed lower metrics. The performance drop of somatic tools on this synthetic dataset mainly stemmed from the structural bias inherent in the benchmark itself. As noted in [65], this dataset consists mainly of germline-like indels, with about half located in variable number tandem repeat (VNTR) regions. This composition does not capture the complex mutational features typical of cancer genomes, which often involve clustered SVs, chromothripsis, and translocations.

Somatic SV calling performance in tumor-only mode

In clinical and practical settings, matching normal samples are often unavailable, which restricts somatic variant detection. To address this limitation, we tested SAVANA and Severus in tumor-only mode. Across three CCS datasets (HCC1395, COLO829, and HG008), SAVANA produced large numbers of false positives (5025 to 30 122), dropping its precision below 0.07 (Supplementary Table S67). In contrast, Severus used population-scale database filtering to restrict false positives (333 to 624), yielding higher F1 scores (0.24 to 0.43) than those of SAVANA (below 0.11) (Supplementary Fig. S13B). These results indicate that Severus provides better precision and specificity when paired normal controls are missing.

Computational resource consumption in germline and somatic SV calling

We tracked the runtime and peak memory consumption of different structural variant tools across sequencing platforms (Fig. 6J and K, Supplementary Table S68). In the germline evaluation (HG002), Sniffles2 and cuteSV2 processed data most efficiently in under 3 min with <1.2 GB of memory, whereas SVHunter required much longer execution times. Most tools kept memory usage under 10 GB, with exceptions for SVDF, which consumed 24.16 GB on CLR data, and NanoVar, which used ~15 GB on CCS and ONT data. Execution times generally followed a descending order of ONT > CLR > CCS. Somatic SV calling on COLO829 consistently required heavier computational resources than germline screening. Severus and SAVANA completed somatic runs in the shortest times, ranging from 20 to 36 min (Fig. 6L and M, Supplementary Table S69). For somatic memory footprints, nanomonsv and SVision-pro recorded the lowest usage below 3.1 GB, whereas both SAVANA and Severus reached ~23 GB. Additionally, we designed user recommendation tables based on the benchmarking results to highlight well-performing tools across different scenarios (Supplementary Tables S70–S73). The manifest linking of the benchmark pipeline and the corresponding resource links is provided in Supplementary Table S74, and the abbreviations and nomenclature used in this study are summarized in Supplementary Table S75.

Discussion

In this study, we established a comprehensive dual-pipeline benchmarking framework, using long-read sequencing data, to systematically delineate the performance profiles of 14 SV detection tools at both the germline and somatic levels. In the germline pipeline, building upon previous benchmarking practices, we implemented a 12-dimension evaluation matrix to systematically assess current algorithms, including those based on deep-learning architectures. To address the common limitation where somatic analyses have largely relied on germline-focused tools, our somatic pipeline systematically benchmarks state-of-the-art tumor-specific callers across diverse cancer samples. By moving beyond isolated single-tool evaluations, this work establishes a comprehensive reference framework that addresses key knowledge gaps and provides practical guidance for tool selection in long-read cancer genomics.

Similar to previous reports, all algorithms experienced a 10%–30% drop in F1 scores when analyses moved from simulated to real datasets. This performance drop became especially pronounced on the CHM13 dataset and under low-depth sequencing conditions [28]. In addition, performance varied notably depending on the variant type. Most callers identified deletions and insertions accurately, yielding F1 scores between 0.80 and 0.94. Tools such as SVision, SVIM, and cuteSV2 showed the most reliable performance for these simpler variants. In contrast, performance dropped for inversions and duplications, with F1 scores clustering between 0.50 and 0.80. Unlike deletions and insertions that alter sequence length, inversions and duplications preserve fragment size and frequently occur in repetitive regions, making accurate boundary definition difficult. For translocations, although most tools mapped breakpoints correctly, only Delly2 and Severus successfully resolved complex topological structures and breakend patterns [17, 31]. Specifically, Delly2 combines paired-end mapping with split-read analysis to precisely define structural variant breakpoints and evaluate translocation breakend patterns. Meanwhile, Severus constructs a phased breakpoint graph and leverages long-read connectivity to resolve complex topological structures and multi-breakpoint rearrangements. These results indicate that untangling complex SV topologies continues to pose a major technical challenge.

Here, we evaluate performance using metrics such as base accuracy, breakpoint localization, and unbiased variant-length reconstruction, offering detailed insights compared with earlier studies [26, 28–32]. Because precise length and genomic coordinates are critical for functional annotation [66], our evaluation indicates that most algorithms capture deletion breakpoints more reliably than insertion breakpoints, as deletion signatures are directly reflected in continuous Concise Idiosyncratic Gapped Alignment Report (CIGAR) strings, whereas insertions rely heavily on local realignment heuristics. Severus, SVDF, and SVIM showed stable breakpoint positioning through robust graph-based or localized clustering filters, whereas SVIM performed particularly well in restoring inserted fragment lengths owing to its refined local consensus realignment module that corrects alignment drift around insertion sites. Two-dimensional grid optimization showed that precise sequence restoration remains difficult, as performance dropped for cuteSV2, NanoVar, and SVision when we tightened the length similarity threshold due to their lack of base-level sequence error-correction mechanisms. In contrast, Sniffles2, SVIM, and SVDF maintained stable performance under stricter breakpoint offset thresholds by employing rigorous filtering heuristics, reflecting their reliability in capturing precise signals.

Using the T2T-CHM13 reference assembly and high-confidence heterozygous datasets, we addressed certain limitations found in the GIAB reference set. DeBreak and Delly2 kept false positives under control, maintaining a low baseline rate. Cross-validation indicated that deep-learning–based tools like SVision and SVDF detected more variants and performed well in heterozygous variant detection. These results imply that many filtered false positives might actually represent somatic mutations or natural heterozygous variations. In addition, family-based analyses of MDR and MIER highlighted SVHunter and SVIM as the most stable tools. For genotyping, Dysgu and SVHunter worked best in separating homozygous and heterozygous states, keeping MIER consistently below 5%. Operationally, Sniffles2 and cuteSV2 suit population cohort studies well because of their high processing speed and low memory use, whereas deep-learning tools like SVHunter and NanoVar demand heavier computational resources. Furthermore, performance variations among deep learning–driven variant callers are partly driven by domain shifts separating their initial training sets from empirical test data. A large number of deep-learning architectures rely on static pretrained weights, which are optimized on mixed datasets of synthetic genomes and real biological samples. When these models face unseen long-read sequencing cohorts in practice, data distributions often mismatch. Such intrinsic biases undermine feature generalizability, leading to compromised accuracy in complex genomic landscapes compared with that of traditional heuristic algorithms.

Detecting somatic SVs accurately is crucial for identifying cancer driver genes and supporting long-read cancer studies [67]. Although previous studies have evaluated somatic SV detection, most applied germline variant callers directly to somatic datasets. Systematic benchmarks specifically for dedicated somatic SV tools remain scarce. Using three cancer cell lines, HCC1395, COLO829, and HG008, we established a benchmark evaluation system for somatic SVs. Our findings show that somatic-specific tools, such as Severus, SAVANA, nanomonsv, and Delly2, achieve higher overall F1 scores than germline tools due to their built-in germline-filtering algorithms. Among the tested tools, Severus and SAVANA performed best overall. In contrast, germline callers like SVDF lack targeted filtering strategies for somatic variants, leading to high recall but extremely low precision. Their F1 scores generally fall below 0.20. Algorithmically, Severus and SAVANA overcome tumor heterogeneity, subclonal structures, and high background noise through tailored error-correction and filtering mechanisms. Conventional germline tools struggle to eliminate mosaic noise, sequencing artefacts, and complex background interferences in tumor samples, producing many false positives. Conversely, SAVANA combines feature encoding with machine learning models (using conformal prediction for confidence estimation) to suppress background noise while preserving high precision. Meanwhile, Severus constructs a phased breakpoint graph and uses long-read connectivity across complex regions to resolve graph tangles from overlapping structural variants, achieving high recall for subclonal rearrangements with low variant allele frequencies (VAFs). These different designs drive their respective strengths: SAVANA excels in precision control, whereas Severus excels in complex rearrangement recall. Notably, tumor data analysis demands heavier computational resources; in this study, Severus and SAVANA detected variations within 40 min, though this came at the cost of significantly higher memory usage.

In addition, obtaining paired normal controls is not always feasible due to high sequencing costs or limited sample availability. To address this, several somatic SV tools provide a tumor-only mode that uses public population-variation databases to filter out germline background variants offline. In the tumor-only evaluation, Severus used the population-structure variation dataset from the 1000 Genomes Project [68] to identify and filter germline variants. It showed strong false-positive control, successfully reducing background interference. Among the tested tools, it achieved the highest overall F1 score. This result highlights the feasibility of using public population databases for germline filtering, offering useful references for long-read cancer studies. Although our framework focuses on traditional single-reference alignment algorithms, emerging pangenome technologies are reshaping structural variant detection. Unlike linear genomes that are prone to reference bias and miss complex variants, pangenome graph approaches encode genetic diversity directly into graph topologies [10]. This shift from linear mapping to graph traversal allows for a more robust capture of complex structural variants across populations. Although current benchmarking focuses on single-reference genomes, future evaluations should expand to pangenome-scale graph models as these technologies mature in population genomics.

Several limitations of this study should be noted. First, because high-confidence benchmarks for DUPs, INVs, and TRAs are scarce in real samples, we relied on simulated data, which may introduce simulation bias. Second, public long-read cancer datasets remain small in scale, restricting the generalizability of our somatic benchmarks to broader oncological contexts. Additionally, due to experimental constraints, we did not perform a fine-grained stratified evaluation of complex VAF distributions or subclonal architectures. Future work will examine how VAF thresholds and subclonal heterogeneity affect detection sensitivity across algorithms. Furthermore, our evaluation of the tumor-only mode was limited to Severus and SAVANA. Because most somatic-specific SV callers, such as nanomonsv, strictly require paired normal data, our benchmarking could not include a broader range of algorithms in this setup. This highlights the need for future tools to support long-read SV detection in tumor-only scenarios. Future research should use larger cohorts and broader metrics to improve the applicability of these methods across diverse genomic applications. In summary, our benchmarking framework offers practical guidance for tool selection across diverse sequencing scenarios, facilitating more reliable structural variant detection in complex genomic applications.

Conclusion

This study presents a benchmarking framework for long-read SV detection that integrates germline and somatic assessments into a unified analytical system. For germline evaluation, we assess diverse SV callers, including both traditional and deep-learning–based architectures, to establish stability benchmarks across varying sequencing technologies and genomic contexts. For somatic SVs, we analyze state-of-the-art tumor-specific algorithms using both simulated and real-world datasets. Our results indicate that variant complexity, sequencing platforms, and algorithmic designs heavily influence SV detection performance. In germline analyses, DeBreak, cuteSV2, and SVDF exhibit robust cross-platform stability while balancing precision and recall. For tumor genomes, somatic-specific tools like Severus and SAVANA outperform germline callers due to dedicated background-filtering mechanisms. Additionally, Sniffles2 and cuteSV2 show high computational efficiency, rendering them practical for large-scale cohort studies. Based on these results, we provide structured tool-recommendation tables tailored to specific application scenarios. Identifying reliable workflows for complex structural variants helps improve analytical accuracy in genomic studies. Ultimately, these findings offer a practical reference for method selection and future tool developments.

Key Points

• We establish a comprehensive, dual-pipeline benchmarking framework that systematically evaluates 14 long-read SV callers across 20 core datasets and 12 analytical dimensions for both germline and somatic SV detection.

• This study represents the first extensive benchmark of somatic SV detection tools on both real and synthetic tumor data, demonstrating that dedicated somatic-targeted algorithms possess distinct advantages through effective germline filtration.

• Given that the performance of SV callers in long-read sequencing analysis is highly scenario-dependent, comprehensive benchmarking is conducted to establish structured tool-recommendation tables, providing actionable strategies and reliable guidance for tool selection across diverse applications.

Author contributions

Hua Shi (Methodology, Resources, Writing—review & editing, Funding acquisition), Yihang Lin (Data curation, Writing—review & editing, Visualization), Dachen Liu (Validation, Formal analysis), Hongfeng Wu (Validation, Investigation), Nor Ashidi Mat Isa (Writing—review & editing, Supervision), Leyi Wei (Writing—review & editing, Supervision), Quan Zou (Writing—review & editing, Supervision), and Xiang Chen (Conceptualization, Resources, Writing—review & editing, Supervision)

Funding

This work was supported by the National Natural Science Foundation of China (No. 62372392 to HS) and the Natural Science Foundation of Zhejiang Province (LMRY26H200005 to XC).

Data availability

All datasets utilized in this study are publicly available, with detailed descriptions and accession links provided in Supplementary Tables S2–S4. The human reference genomes GRCh37 and GRCh38 can be accessed via http://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/hs37d5.fa.gz and http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa, respectively. The T2T-CHM13 v2.0 assembly is available at https://s3-us-west-2.amazonaws.com/human-pangenomics/T2T/CHM13/assemblies/analysis_set/chm13v2.0.fa.gz. All data generated or analyzed during this study are included in this published article and its supplementary data. All source code and SV callsets for the benchmarking tools evaluated in this study are available at https://github.com/model-lab/LR-SV-Benchmark. All tables presented in this study were regenerated directly from the audit manifest (see Manifest linking.xlsx in the Supplementary Materials) after correction to ensure complete traceability.