NanoTS: a deep learning tool for accurate SNP calling in nanopore long-read transcriptome data

Abstract

Accurate variant detection using nanopore long-read transcriptome data remains challenging. Here we present NanoTS—a deep learning-based tool for single nucleotide polymorphism detection from diverse types of nanopore transcriptome sequencing data. NanoTS outperforms existing methods, achieving F₁ scores above 0.980 and 0.966 on nanopore direct RNA and cDNA sequencing data, respectively, for single nucleotide polymorphisms with at least five supporting reads. Notably, NanoTS shows strong improvements over existing methods for allelically imbalanced variants. We also demonstrate that NanoTS enables accurate detection and genotype calling of pathogenic variants underlying Mendelian disorders, highlighting its potential clinical utility.

NanoTS facilitates nanopore long-read transcriptome sequencing based SNP detection and genotype calling using deep learning.

Main

Nanopore transcriptome sequencing has transformed RNA research by enabling characterization of full-length transcripts and direct detection of RNA modifications¹. Recent advances in Oxford Nanopore Technologies (ONT) platforms have further improved sequencing throughput and accuracy, expanding its cost-effectiveness and potential for variant calling. However, compared to Illumina short-read sequencing and Pacific Biosciences (PacBio) HiFi long-read sequencing, the higher error rate of nanopore sequencing remains challenging for accurate variant detection, particularly in transcriptome sequencing data.

Compared to genome sequencing data, transcriptome sequencing data inherently complicates variant detection due to highly uneven read coverage across genes, transcript isoform diversity, allele-specific isoform expression and library artifacts^2,3. These factors hinder the performance of standard long-read DNA variant callers on transcriptome sequencing data⁴, necessitating specialized transcriptome-based variant-calling methods. To address these challenges, we developed NanoTS (nanopore transcriptome SNP caller)—a deep learning-based tool for accurate single nucleotide polymorphism (SNP) calling from nanopore direct RNA (dRNA) and cDNA sequencing data. NanoTS introduces an allele-phased pileup strategy and integrates haplotype information by phasing across reads to improve variant-calling accuracy. Across diverse benchmarking datasets, NanoTS consistently outperformed existing nanopore transcriptome-based variant callers, including Clair3-RNA⁵ and LongcallR⁶. Furthermore, NanoTS enables accurate detection and genotype calling of pathogenic variants underlying Mendelian disorders, further highlighting its potential for RNA-based genomic diagnosis.

To train and evaluate NanoTS, we used high-confidence variant calls and regions from the Genome in a Bottle (GIAB) database—a widely used benchmarking resource⁷. Homozygous alternative (ALT) and heterozygous ALT sites were used as gold-standard positive references, whereas homozygous reference (REF) sites were used as gold-standard negative references (Fig. 1a). Consistent with how other nanopore transcriptome-based variant callers were trained, we used nanopore dRNA⁵ and cDNA data generated from the HG002 sample—a widely used benchmark reference for variant calling—to train NanoTS.

Fig. 1. NanoTS training, feature extraction and variant-calling workflow.

a, Diagram illustrating the generation of NanoTS training set. Variants supported by at least two ALT reads and ALT read proportion ≥0.05 in GIAB high-confidence (HC) regions were used for training. b, Schematic of feature integration in NanoTS. Reads with a mismatch rate <0.1 were selected for feature extraction and categorized into four groups based on the target variant: FWD REF, REV REF, FWD ALT and REV ALT. Features include (1) local (around ±30 bp of SNP candidates) base fraction of A/C/G/T for the four read groups, incorporating reference sequences (seq); (2) entropy calculated from the local base fractions across all reads; (3) normalized local base coverage; (4) normalized distance to read end; (5) normalized maximum coverage and (6) base fraction of REF/ALT/other alleles at the 20 nearest SNP candidates. c, Diagram of NanoTS unphased and phased model training. Training was performed on autosomes, excluding chr. 20, which was reserved as the validation set. A softmax function was applied to infer the probabilities of three possible SNP genotypes (0/0, 0/1, 1/1). d, The NanoTS variant-calling pipeline comprises five steps: (1) converting ONT current data (Pod5) to FASTQ using Dorado; (2) alignment to the reference genome using Minimap2¹⁸; (3) initial unphased SNP calling; (4) phasing of variants with WhatsHap and (5) final SNP calling with the phased model. e, Sequencing depth of samples, including dRNA, cDNA and cDNA TEQUILA-seq. HG002 dRNA and cDNA sequencing data were used for model training. Datasets marked with an asterisk were generated in this study. f, Autosomal regions covered by at least five reads, excluding regions where reads skip splice junctions. g, Base error rate of samples, estimated by the number of mismatches per mapped base. Het, heterozygous; Hom, homozygous.

Within NanoTS, we implemented an allele-phased pileup strategy, which categorizes reads into four groups based on the allele type of a given variant and strand direction (Fig. 1b). To account for systematic sequencing errors and library artifacts—such as strand-biased errors, positional biases within sequencing reads, localized mismatch clusters and error-prone sequence contexts—six additional features were incorporated: (1) base fraction of A/C/G/T, (2) entropy of base fraction, (3) normalized base coverage, (4) normalized distance to read ends, (5) normalized maximum coverage and (6) base fraction of REF/ALT/other alleles for the 20 nearest SNP candidates.

To further enhance the performance of NanoTS in regions affected by allelic imbalance—a major challenge for long-read transcriptome-based variant callers⁴—we incorporated haplotype information into the variant-calling framework (Fig. 1c). NanoTS includes both an unphased and a phased model. The unphased model utilizes features derived from unphased data, whereas the phased model integrates features from unphased data, haplotype 1, and haplotype 2, enabling more accurate SNP calling. When using NanoTS, variant calling initially employs the unphased model (Fig. 1d). High-confidence variants (QUAL ≥ 10) are then phased using WhatsHap⁸, which partitions reads into haplotype 1 and haplotype 2. Finally, SNP calling is conducted using the phased model.

For model network architecture, NanoTS employs a combination of convolutional and fully connected neural network layers to integrate the six features within a subnetwork (Supplementary Fig. 1). The unphased model utilizes a single subnetwork based on unphased data. The phased model incorporates three subnetworks: one for unphased data, one for haplotype 1, and one for haplotype 2, with an additional fully connected layer to link nodes across all three subnetworks.

To evaluate the performance of NanoTS comprehensively, we applied it to whole-transcriptome dRNA sequencing data from HG004 and HG005⁵, as well as in-house generated whole-transcriptome cDNA sequencing data from HG001, HG004 and HG005 (Supplementary Table 1). Furthermore, we tested NanoTS on in-house generated targeted cDNA sequencing data from HG001 by TEQUILA-seq⁹—a targeted nanopore cDNA sequencing method applied in this case to a panel of 422 genes related to inborn errors of immunity (IEI; Supplementary Tables 1 and 2). We used the TEQUILA-seq data to evaluate NanoTS on targeted nanopore transcriptome sequencing data with high read coverage across target genes. The sequencing depth ranged from 5.7 million to 72.5 million reads (Fig. 1e). We observed that whole-transcriptome cDNA sequencing data exhibited approximately 3-fold greater genomic region coverage than whole-transcriptome dRNA sequencing data, independent of sequencing depth, probably due to intrinsic differences in these two sequencing methods^10,11 (Fig. 1f). We also observed modest variation in base error rate across samples and protocols (Fig. 1g).

Across varying ALT read count thresholds, NanoTS achieved an F₁ score above 0.980 and 0.966 in dRNA and cDNA data, respectively (≥5 ALT reads and ≥0.05 ALT read proportion per variant site), outperforming Clair3-RNA and LongcallR (Fig. 2a–c). It also achieved precision and recall both greater than 0.979 in dRNA data and both greater than 0.957 in cDNA data (Fig. 2d,e). Notably, in cDNA data, NanoTS improved the F₁ score by 0.047–0.088 and 0.127–0.159 over Clair3-RNA and LongcallR, respectively (Fig. 2c). NanoTS also outperformed competing tools in genotype zygosity calling. For example, in HG004, for distinguishing the 0/0 (REF/REF), 0/1 (REF/ALT) or 1/1 (ALT/ALT) genotype from the other two genotypes respectively, NanoTS achieved F₁ scores of 0.988, 0.957 and 0.957, respectively, in dRNA data, and 0.993, 0.940 and 0.954, respectively, in cDNA data (Supplementary Fig. 2). Furthermore, across a range of ALT read proportions in both cDNA and dRNA data, NanoTS consistently demonstrated better performance in both variant detection (Fig. 2f and Supplementary Fig. 3a) and genotype zygosity calling (Supplementary Fig. 4). Notably, for allelically imbalanced variants with ALT read proportions between 0.05 and 0.20 in cDNA data, NanoTS improved the F₁ score for variant detection by 0.338–0.478 and 0.497–0.635 compared to Clair3-RNA and LongcallR, respectively (Fig. 2f).

Fig. 2. Performance evaluation of NanoTS and validation of known pathogenic variants underlying Mendelian disorders.

a,b, F₁ score comparison of NanoTS, Clair3-RNA and LongcallR across different ALT read thresholds in dRNA (a) and cDNA (b) datasets, including whole-transcriptome sequencing (WTS) and targeted sequencing (TEQUILA-seq with the IEI gene panel). c, F₁ score comparison across different samples with ≥5 ALT reads and ≥0.05 ALT read proportion. d,e, Precision-recall plots for dRNA (d) and cDNA (e) datasets, both filtered using the same criteria as in c. f, F₁ score comparison of NanoTS, Clair3-RNA and LongcallR in cDNA data under varying ALT read proportions (≥5 ALT reads). g, F₁ score of cDNA HG004 across different genomic contexts, filtered using the same criteria as in c. h, Number of gold-standard positive SNPs identified by NanoTS in annotated exonic regions of IEI genes for HG001 based on TEQUILA-seq data and WTS data that were downsampled to the same sequencing depth (5.7 million reads). SNP identification was evaluated across varying total read coverage thresholds (2× to 100×), focusing on SNPs with ≥2 ALT reads and ≥0.05 ALT read proportion. i, Detection of 32 known pathogenic variants within annotated exonic regions from TEQUILA-seq data of 25 patients with Mendelian disorders using NanoTS, Clair3-RNA and LongcallR. Diseases included CDG and PMD. j,k, Variant calling results in two CDG samples, where NanoTS identified pathogenic variants in PIGN (j) and PMM2 (k). l, Sashimi plot of haplotype-resolved MPV17 splicing patterns in TEQUILA-seq data for patient PMD-014 using ggsashimi¹⁹ (v1.1.5). Read coverage tracks and the first ten read alignments for each haplotype are shown as displayed in Integrative Genomics Viewer²⁰ (v2.18.2). j–l, Raw alignment data were phased into haplotype 1 (H1) and haplotype 2 (H2) based on variants identified by NanoTS.

Across different genomic contexts, sequence composition influences variant-calling accuracy from nanopore transcriptome sequencing data, with homopolymer regions posing a known challenge due to sequencing errors involving repetitive bases⁴. We assessed the performance of NanoTS, Clair3-RNA and LongcallR across different genomic contexts (Fig. 2g and Supplementary Fig. 3b). All three tools had lower accuracy as homopolymer length increased. By contrast, performance was highest in coding sequences (CDS), which are depleted for homopolymers and simple repeats¹². NanoTS consistently achieved the highest F₁ score across different genomic contexts in both dRNA (Supplementary Fig. 3b) and cDNA (Fig. 2g) data.

To determine which features had the greatest impact on the performance of NanoTS, we conducted a feature shuffling analysis (Methods). Base fraction and base coverage within ±30 bp of SNP candidates were identified as the most influential factors for accurate SNP calling in both dRNA and cDNA data (Supplementary Fig. 5). As expected, the phased model consistently improved genotype zygosity calling, yielding an improvement in F₁ score of 0.010–0.023 in calling variants as heterozygous (Supplementary Fig. 6).

We also trained and evaluated NanoTS using PacBio HiFi cDNA sequencing datasets from GIAB samples (Supplementary Table 1). On these datasets, NanoTS achieved F₁ scores comparable to those of Clair3-RNA and higher than those of LongcallR across varying ALT read count thresholds (Supplementary Fig. 7 and Supplementary Table 3). Consistent with our evaluations on nanopore data, NanoTS outperformed both Clair3-RNA and LongcallR in detecting allelically imbalanced variants with low ALT read proportions (Supplementary Fig. 7c).

Moreover, we evaluated the performance of NanoTS on data generated by targeted nanopore cDNA sequencing, which has emerged as an important strategy for long-read transcriptomics when tailored to specific biological or clinical settings^1,9,13. We assessed NanoTS, Clair3-RNA and LongcallR on TEQUILA-seq data of 422 IEI genes in HG001. NanoTS outperformed Clair3-RNA and LongcallR by a substantial margin across all performance metrics (Fig. 2b,c,e). As expected, at the same overall sequencing depth, substantially more gold-standard positive SNPs in IEI genes were identified from TEQUILA-seq data compared to whole-transcriptome cDNA data (Fig. 2h).

To investigate whether NanoTS can detect clinically relevant variants in patients with Mendelian disorders, we applied it to in-house TEQUILA-seq data generated using separate panels targeting genes implicated in congenital disorders of glycosylation¹⁴ (CDG; 466 genes) and primary mitochondrial diseases¹⁵ (PMD; 359 genes) (Supplementary Tables 4 and 5). We performed variant calling using all three tools (NanoTS, Clair3-RNA and LongcallR) on cDNA TEQUILA-seq data generated on fibroblasts from three patients with CDG and 22 patients with PMD (Supplementary Table 1), who altogether harbored 32 known pathogenic variants in annotated exonic regions based on previous genetic testing.

Across these samples, NanoTS correctly detected and genotyped 30 of 32 variants, outperforming Clair3-RNA (27 of 32) and LongcallR (24 of 32) (Fig. 2i and Supplementary Table 6). For example, in patient CDG-002, who carries compound heterozygous variants (chr18:62095902, C>T; chr18:62157743, G>C) in PIGN, all three tools correctly identified both variants and their genotype zygosity (Fig. 2j). By contrast, in patient CDG-003, who carries compound heterozygous variants in PMM2, only NanoTS correctly identified both variants and their genotype zygosity (Fig. 2k). LongcallR misclassified one variant (chr16:8811153, G > A) as homozygous ALT, probably due to strong allelic imbalance in transcript expression (ALT:REF = 0.91:0.09). The second variant (chr16:8811146, G>A) was not detected by either Clair3-RNA or LongcallR. This variant disrupts an exonic splicing enhancer within PMM2 exon 5, leading to exon skipping and the introduction of a premature termination codon that triggers nonsense-mediated decay¹⁶. The reduced transcript abundance from this allele results in strong allelic imbalance, probably explaining why Clair3-RNA and LongcallR failed to detect the variant.

Finally, in one patient (PMD-014) with compound heterozygous variants in MPV17, all three tools correctly identified and genotyped a known pathogenic missense variant in exon 4 (chr2:27312768, G>A). Although the second pathogenic variant (chr2:27312489, C>A) was not included in our evaluation due to its intronic location (+5 donor position of intron 5), NanoTS was able to detect and correctly genotype this variant, as it activated a cryptic donor splice site that extended exon 5 by 88 bp (Fig. 2l), consistent with the predicted effect of the variant on splicing (SpliceAI¹⁷ ∆score = 0.65). Although Clair3-RNA and LongcallR also detected this variant, both tools reported it incorrectly as homozygous ALT, probably due to strong allelic imbalance in transcript expression (ALT:REF = 0.90:0.10). Notably, the ability of NanoTS to help phase long RNA-seq reads allowed us to characterize the cis effects of this intronic variant on MPV17 transcripts. Transcripts expressed from the haplotype carrying this intronic variant showed multiple splicing defects, including exon 5 skipping and activation of several cryptic donor splice sites, providing additional molecular evidence of disease.

Taken together, the ability to detect and help phase both genetic variants and transcript alterations highlights the potential of NanoTS, together with targeted nanopore cDNA sequencing, for RNA-based genomic diagnosis of rare Mendelian disorders. Although NanoTS is currently limited to calling germline variants, future work could extend NanoTS to somatic variant calling, which can further broaden its utility.

Methods

Patient cohort and clinical samples

The institutional review board (IRB) of the Children’s Hospital of Philadelphia provided study approval and oversight for research protocols enabling skin punch biopsies for fibroblast cell line establishment and genetic/transcriptomic analysis of rare disease patients (IRB no. 14-0112233 for CDG; IRB no. 08-6177 for PMD). All patients and/or guardians of patients provided informed consent before inclusion in the studies. This study was performed in accordance with ethical principles for medical research outlined in the Declaration of Helsinki of 1975, as revised in 2013.

Dataset preparation

High-confidence variant calls and genomic regions were obtained from the GIAB database. GIAB integrates multiple sequencing technologies and variant-calling methods to generate high-confidence variant sets, ensuring accuracy through consensus-based validation and orthogonal sequencing approaches.

ONT R10.4.1 dRNA FASTQ files (HG002, HG004, HG005) were downloaded from the NCBI Sequence Read Archive (accession ID PRJNA1169852). ONT R10.4.1 cDNA datasets (HG001, HG002, HG004, HG005, HG001 IEI TEQUILA-seq, CDG TEQUILA-seq for 3 samples, and PMD TEQUILA-seq for 22 samples) were generated in house. PacBio HiFi MAS-seq cDNA FASTQ files (HG002, HG004, HG005) were downloaded from the GIAB database (https://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data_RNAseq). Details of all samples are provided in Supplementary Table 1.

Cell culture

Four lymphoblastoid cell lines were purchased from the Coriell Institute: GM12878 (HG001), GM24385 (HG002), GM24143 (HG004) and GM24631 (HG005). Cells were cultured in RPMI-1640 medium (Gibco, cat. no. 11875085) supplemented with 10% fetal bovine serum (FBS; Corning, cat. no. 45000-734) and 1% penicillin–streptomycin (Gibco, cat. no. 15140122) at 37 °C in a humidified incubator with 5% CO₂.

Patient-derived skin fibroblast cell lines (n = 25) were cultured in Dulbecco’s Modified Eagle Medium (DMEM; Gibco, cat. no. 11885084), supplemented with 10% FBS, 1% GlutaMax (Gibco, cat. no. 35050061) and 100 U ml⁻¹ penicillin–streptomycin. The cells were maintained at 37 °C in a humidified incubator with 5% CO₂.

RNA isolation

Total RNA was extracted from 2–3 million cells per cell line using TRIzol reagent (Invitrogen, cat. no. 15596018) following the manufacturer’s protocol. RNA was purified by chloroform extraction and isopropanol precipitation, followed by a 75% ethanol wash. The RNA pellet was air-dried and resuspended in RNase-free water. RNA integrity and concentration were assessed using a Nanodrop spectrophotometer and an Agilent RNA ScreenTape (Agilent, cat. no. 5067-5576) on a 4200 TapeStation before proceeding to library preparation.

Full-length cDNA library generation

A 200-ng aliquot of total RNA was used as the template for cDNA synthesis. Reverse transcription was performed using an oligo-dT primer (5′-AAGCAGTGGTATCAACGCAGAGTACT[30]VN-3′, Integrated DNA Technologies (IDT)) and a template-switching oligo (5′-AAGCAGTGGTATCAACGCAGAGTACATrGrGrG-3′, IDT) with Maxima H Minus reverse transcriptase (Thermo Scientific, cat. no. EP0753) at 42 °C for 90 min, followed by 85 °C for 5 min. First-strand cDNA was amplified using the PCR primer (5′-AAGCAGTGGTATCAACGCAGAGT-3′, IDT) and KAPA HiFi HotStart ReadyMix (Roche, cat. no. KK2602) under the following conditions: 95 °C for 3 min; 11 cycles of 98 °C for 20 s, 67 °C for 20 s and 72 °C for 5 min; with a final extension at 72 °C for 8 min. PCR products were purified with 0.8× volumes of DNA Clean Beads (Beckman Coulter, cat. no. B23318). Amplified cDNA was quantified using the Qubit dsDNA High Sensitivity assay (Invitrogen, cat. no. Q32854) and its quality assessed using the Agilent High Sensitivity D5000 ScreenTape assay (Agilent, cat. no. 5067-5588).

TEQUILA probe synthesis and hybridization capture

TEQUILA probes for the IEI, CDG and PMD gene panels (Supplementary Tables 2, 4, 5, 7, 8 and 9) were generated using a previously established protocol⁹. For the hybridization capture step, approximately 500 ng of amplified cDNA was denatured at 95 °C for 10 min and then hybridized with 100 ng of TEQUILA probes at 65 °C for 12 h. This hybridized mixture was subsequently incubated with 50 μl of M-270 streptavidin beads (Invitrogen, cat. no. 65306) for 45 min at 68 °C. After a series of washes following the IDT xGen Lockdown protocol using xGen Lockdown Reagents (IDT, cat. no. 1072281), the beads were resuspended in 40 μl of TE buffer (Invitrogen, cat. no. 12090015). The captured cDNA was amplified directly on beads using KAPA HiFi HotStart ReadyMix with the following cycling conditions: an initial denaturation at 95 °C for 3 min; 14 cycles of 98 °C for 20 s, 67 °C for 20 s and 72 °C for 5 min; and a final extension at 72 °C for 8 min. Final PCR products were purified with 0.7× SPRIselect beads and quantified using both the Qubit dsDNA High Sensitivity assay and the Agilent High Sensitivity D5000 ScreenTape assay.

Nanopore library preparation and sequencing

A DNA library for barcoding and ligation sequencing was prepared following protocols ONT SQK-NBD114.24 and SQK-LSK114. Briefly, 120 ng of cDNA was repaired and dA-tailed using the NEBNext Ultra II End Repair/dA-tailing Module (NEB, cat. no. E7546L) followed by purification with DNA Clean Beads. For multiplexing, the repaired DNA was barcoded using the Native Barcode kit with NEB Blunt/TA Ligase Master Mix (NEB, cat. no. M0367S) and purified again with DNA Clean Beads. Barcoded samples were pooled in equimolar amounts. Single samples without barcodes or pooled barcoded samples were then ligated to an ONT adapter using the NEBNext Quick Ligation Module (NEB, cat. no. E6056S) and purified with DNA Clean Beads. Finally, the prepared DNA library, combined with sequencing buffer and beads, was loaded onto a PromethION R10.4.1 Flow Cell and run on a P24 sequencer. The cDNA data were basecalled using MinKNOW (v6.2.8), invoking the high-accuracy mode of Dorado. Reads with a quality score of at least 9 were labeled as ‘PASS’ and used for subsequent analysis.

Alignment of long-read transcriptome data

All long-read transcriptome data were aligned to the hg38 reference genome, excluding alternative contigs, using Minimap2¹⁸ (v2.24-r1122) with parameters ‘-ax splice -ub -k 14 -w 4 --junc-bed gencode.bed --secondary=no’ for nanopore data and parameters ‘-ax splice:hq -ub --junc-bed gencode.bed --secondary=no’ for PacBio data. The gencode.bed file was generated from the GENCODE (v40) GTF file using the command ‘paftools.js gff2bed gencode.gtf > gencode.bed’.

Identification of SNP candidate sites

Before variant calling, NanoTS filters out unmapped, secondary and supplementary alignments, and considers only reads with mapping quality ≥20. At each genomic position, counts of A, T, C and G bases across all reads are calculated. NanoTS currently supports biallelic SNP calling and considers only a single ALT allele at each site. The ALT allele is defined as the non-REF base with the greatest read support. In rare cases where two or more non-REF bases have identical read support, NanoTS applies a deterministic tie-breaking rule that selects ALT alleles based on a fixed priority order (A > T > C > G) informed by the distribution of alternative allele frequencies reported in the dbSNP database (v151)²¹. Such variants are annotated explicitly with the tag ‘alt_tie’ in the final output. Total read depth is defined as the total number of reads supporting the REF and ALT alleles. The ALT read proportion is computed as ALT/(REF + ALT). A site is designated as a SNP candidate site if it has an ALT read count ≥2 and an ALT read proportion ≥0.05.

Model training

Individual NanoTS models were trained on different long-read RNA-seq datasets generated on sample HG002 (GIAB), including nanopore dRNA data (1,200,253 sites), nanopore cDNA data (4,275,726 sites) and PacBio HiFi MAS-seq data (1,922,461 sites) (Supplementary Table 10). To enhance the training data and improve model generalizability, we implemented a data augmentation strategy by downsampling the original FASTQ files. Specifically, three separate subsets were created, containing 100%, 20% and 10% of randomly selected reads, respectively. This approach helps the model learn from varying coverage depths, reducing potential biases caused by overrepresented transcripts and improving its robustness in detecting variants across different expression levels. For each dataset, we performed the following steps:

1. Determine the ALT allele: NanoTS only considers a single ALT allele. The ALT allele was determined based on the highest number of supporting reads. If multiple bases have the same highest number of supporting reads, a fixed priority order (A > T > C > G) was used for ALT allele selection.

2. Variant selection: variants were required to have at least two ALT reads and an ALT read proportion ≥0.05. Furthermore, only variants located within GIAB high-confidence regions were retained.

3. Genotype classification: if a variant with a determined ALT allele was present in the GIAB variant set, it was labeled as 0/1 or 1/1 according to the GIAB genotype. Otherwise, it was labeled as 0/0.

After genotype classification, the three augmented datasets were combined. The final dataset was then split into a training set (chrs. 1–19, 21 and 22) and a validation set (chr. 20).

For unphased model training, features were extracted solely from raw alignment data. For phased model training, raw alignment data were phased into haplotype 1 and haplotype 2 using GIAB variants, and features were extracted from the raw, haplotype 1 and haplotype 2 alignment data.

During each epoch, training samples were shuffled randomly and processed in batches of 50,000 variants. Model optimization was performed using an Adam optimizer with a learning rate of 0.001 and cross-entropy loss, with SNP genotype classification (0/0, 0/1, 1/1) as the evaluation metric. We set the maximum number of training epochs to 300 as an upper bound to ensure that all NanoTS models reached full convergence. The model with the highest validation performance was selected.

Feature extraction

The NanoTS unphased model utilizes six features extracted from unphased alignment data, whereas the phased model incorporates these features from unphased, haplotype 1 and haplotype 2 alignments. Only reads with a mismatch rate (number of mismatched bases divided by read length) <0.1 are used for feature extraction:

1. Base fraction of A/C/G/T: base fractions are computed from the ±30-bp flanking regions of each target SNP candidate. Reads are categorized into four read groups: forward reads containing the reference allele (FWD REF), forward reads containing the alternative allele (FWD ALT), reverse reads containing the reference allele (REV REF) and reverse reads containing the alternative allele (REV ALT). At each nucleotide position within the flanking region, the fraction of each base (A, C, G, T) is calculated separately for each group, yielding a 4 (read groups) × 4 (bases) × 61 (positions) matrix. The reference sequence is one-hot encoded to produce a final 5 × 4 × 61 matrix.

2. Entropy of base fraction: the entropy is calculated from the base fractions of all reads within the ±30-bp flanking regions, quantifying the variability at each position. The entropy H at each position is computed using the Shannon entropy formula: $H=-{\sum }_{\left\{b\in \{\mathrm{A},\mathrm{C},\mathrm{G},\mathrm{T}\}\right\}}{p_b\log }_2{p}_b$. In the formula, p_b is the fraction of reads containing base (A, C, G or T) at a given position. If p_b = 0, the term p_b log₂ p_b is defined as 0. If the sum of the fractions of the four nucleotide bases equals 0 due to a deletion, the entropy H is set to 2.5.

3. Normalized base coverage: for each read group, base coverage within the ±30-bp flanking regions is determined and then normalized by the maximum coverage observed across all four groups. Deletions and insertions were not counted toward read coverage. This results in four lists of 61 normalized coverage values.

4. Normalized distance to read ends: for each read, the distances from the target SNP candidate to both ends are measured and normalized as follows: 0 if ≤30 bp, 1 if >30 bp and ≤100 bp, and 2 if >100 bp. The average normalized distance is then calculated for each read group, yielding eight values in total.

5. Normalized maximum coverage: this feature represents the maximum read coverage within ±30 bp of the target SNP candidate across the four read groups. If the coverage is below 50, the value is divided by 50; otherwise, it is capped at 1. This feature provides information about the actual read coverage, ensuring that coverage differences are accounted for while preventing excessively high values from dominating the model.

6. Base fraction of REF/ALT/other alleles for the 20 nearest SNP candidates: for each target SNP candidate, the base fractions of REF, ALT and other allele types for the 20 nearest SNP candidates (10 upstream and 10 downstream), each supported by at least five ALT reads, are calculated separately for the four read groups: FWD REF, REV REF, FWD ALT and REV ALT.

NanoTS model architecture

NanoTS employs a hybrid deep learning framework that integrates convolutional neural networks with fully connected (dense) layers. Base fractions from the ±30-bp flanking regions and from nearest SNP candidates are processed separately by two distinct convolutional neural network structures. Additional features, including entropy of base fraction, base coverage, distance to the read end, and maximum coverage, are processed through dense layers. The phased model uses the same architecture but incorporates an extra fully connected layer to integrate information from unphased, haplotype 1, and haplotype 2 datasets. Finally, a softmax function outputs probabilities for three genotype classes (0/0, 0/1, 1/1). In total, the unphased and phased models have 170,455 and 219,735 parameters, respectively. Additional details regarding layers, connectedness and parameters can be found in Supplementary Table 11.

After deep learning-based prediction, the genotype with the highest probability is selected as the predicted genotype. This probability is then converted into a Phred quality score (QUAL). To further correct potential misclassifications between homozygous alternative (1/1) and heterozygous (0/1) genotypes, we introduce an additional statistical refinement. Specifically, for genotypes predicted as 1/1 and a higher probability for 0/1 than for 0/0, we perform a Fisher’s exact test using the following read counts: REF from haplotype 1, REF from haplotype 2, ALT from haplotype 1 and ALT from haplotype 2. If the resulting P value is less than 1 × 10⁻⁵, the genotype is adjusted from 1/1 to 0/1.

Benchmarking and comparison with existing tools

NanoTS was evaluated and benchmarked against Clair3-RNA and LongcallR using both nanopore and PacBio datasets. Evaluation metrics (precision, recall and F₁ score) were assessed (1) across variant sets defined by varying ALT read support, (2) between phased and unphased NanoTS models, (3) when measuring contributions of individual NanoTS features and (4) across diverse genomic contexts. All analyses were restricted to variants within the GIAB high-confidence regions.

To evaluate the contributions of individual features to SNP calling, we permuted the order of one test feature while keeping the other features unchanged. The unphased model was then applied and evaluated.

Diverse genomic contexts, including homopolymer regions, repeat elements, duplicated sequences, low mappability regions, RefSeq CDS, RefSeq non-CDS and major histocompatibility complex regions, were obtained from the GIAB genome stratifications repository (v3.5) (https://github.com/genome-in-a-bottle/genome-stratifications). Performance was evaluated on regions for each genomic context.

Alignment data phasing

For training, the raw alignment file was phased into haplotype 1 and haplotype 2 alignments based on all high-confidence variants from GIAB. For testing, the raw alignment file was phased using the initial variant calls of NanoTS with a minimum QUAL score of 10. Phasing was performed with the ‘phase’, ‘haplotag’ and ‘split’ submodules of WhatsHap (v2.3), resulting in two new alignment files corresponding to haplotype 1 and haplotype 2.

Calling variants using Clair3-RNA and LongcallR

Variants were called using Clair3-RNA (v0.2.2) with the parameters ‘--snp_min_af 0.05 --min_coverage 2’. For nanopore dRNA data, ‘--platform ont_dorado_drna004 --enable_phasing_model’ was specified. For nanopore cDNA data, the ont_r10_dorado_cdna model was applied with ‘--platform ont_r10_dorado_cdna --enable_phasing_model’. For PacBio MAS-seq data, ‘--platform hifi_mas_minimap2 --enable_phasing_model’ was specified.

For LongcallR (v1.11.0), variants were called using its nextflow version longcallR-nf, in which results from ‘longcallR-nn + longcallR, phasing (whole chromosome) analysis’ were used in the study. The following parameters were used: ‘--min_af 0.05 --min_depth 2 --min_bp 10 --platform ont’ with ‘--datatype dRNA’ for nanopore dRNA data and ‘--datatype cDNA’ for nanopore cDNA data. For PacBio MAS-seq data, ‘--min_af 0.05 --min_depth 2 --min_bp 0 --platform pb --datatype masseq’ was specified.

Predicting variant effects on splicing

Splicing effects of pathogenic variants were predicted using SpliceAI via its web interface (https://spliceailookup.broadinstitute.org; accessed January 2026). Four Δscore categories, including acceptor loss, donor loss, acceptor gain and donor gain, were evaluated to assess whether a variant affects splicing at any position within a ±500-bp window.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Online content

Any methods, additional references, Nature Portfolio reporting summaries, source data, extended data, supplementary information, acknowledgements, peer review information; details of author contributions and competing interests; and statements of data and code availability are available at https://doi.org/10.1038/s41592-026-03225-4.

Author contributions

Z.L. and Y.X. conceived the study and designed the research. Z.L. developed the methodology. Z.L., F.W., J.J.-H.P. and M.T.S. performed experiments and generated the data. Z.L., R.W., D.W.W. and N.D. analyzed the data. K.K., E.M.M., A.C.E. and M.J.F. contributed patient samples and data. L.L. and Y.X. supervised the research. Z.L., F.W., R.W. and Y.X. wrote the manuscript with input from all authors.

Peer review

Peer review information

Nature Methods thanks Ruli Gao and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available. Primary Handling Editor: Lin Tang, in collaboration with the Nature Methods team.

Data availability

The whole-transcriptome cDNA sequencing and TEQUILA-IEI data generated in this study have been deposited in the NCBI Sequence Read Archive (SRA) under accession number PRJNA1247965. All generated SNP calls and evaluation results for the GIAB benchmark are available via Zenodo at https://doi.org/10.5281/zenodo.19900024 (ref. ²²). Due to privacy considerations and the risk of reidentification, the TEQUILA-CDG and TEQUILA-PMD data from patient fibroblast cell lines will be made available from the corresponding author (xingyi@chop.edu) upon reasonable request, under appropriate data-use agreements and in compliance with institutional and ethical guidelines. We will respond to data access requests promptly, no later than 2 weeks after receipt. Restrictions on data use include no secondary sharing and no attempts to reidentify patients or family members from the shared data.

Code availability

The NanoTS software, along with documentation and example datasets, is available on GitHub (https://github.com/Xinglab/NanoTS) and available via Zenodo at https://doi.org/10.5281/zenodo.19900024 (ref. ²²).

Competing interests

L.L., Y.X. and F.W. are inventors on a patent application filed by The Children’s Hospital of Philadelphia covering the TEQUILA method (US Patent 63/277,894). The other authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41592-026-03225-4.