Accurate prediction of oligonucleotide probe hybridization efficiency via a transformer-based model

Abstract

Gene detection chip design critically depends on oligonucleotide probe hybridization efficiency, yet conventional thermodynamic models for its evaluation diverge from experiments, and the lack of comprehensive universal datasets further stops models from reflecting real probe behaviors. To address these issues, a dataset with 8 subsets (over 3.8 million probes) was first constructed by integrating public microarray data. We then propose TransHybrid—a sequence-based deep learning framework that embeds complementary strand features into a Transformer architecture. The validation and test results show that the performance of TransHybrid, as measured by the Pearson correlation coefficient (PCC), is more than double that of traditional thermodynamic models, significantly outperforming conventional evaluation methods. The robustness of our model was validated via our independent experimental dataset, which was constructed through a series of designed experiments for fine-tuning TransHybrid. Following fine-tuning on this custom dataset, TransHybrid achieved a 25% improvement over the PCC of the thermodynamic models. Furthermore, mechanistic analyses revealed that TransHybrid captures biologically meaningful sequence features, such as the contribution of GC-rich regions, and provides interpretable attention maps. These findings demonstrate the superior ability of TransHybrid to accurately model hybridization efficiency, paving the way for more effective probe design in gene detection applications.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12864-026-12833-5.

Introduction

Oligonucleotide probes serve as the core component of gene detection chips, facilitating complementary hybridization with fluorescently labeled target nucleic acid sequences in solution. The hybridization status is determined by the fluorescence intensity [1]. These probes are widely used in high-throughput nucleic acid detection [2], methylation profiling [3], and single-nucleotide polymorphism (SNP) genotyping [4]. However, the accuracy of these applications depends critically on the probe hybridization efficiency. Current evaluation methods rely on time-consuming and resource-intensive experimental validation, in which numerous probes are tested individually. This trial-and-error approach impedes rapid probe design optimization [5], underscoring the need for computational methods to predict hybridization efficiency and enable efficient probe selection.

Early studies integrated thermodynamic and kinetic principles to develop hybridization models by fitting parameters to probe signal intensity and target concentration, as exemplified by Li’s competitive model [6] and Jakubek’s extended model [7]. Although these models provide theoretical support for microarray design, they are tailored to fit unique parameters for specific probes and cannot be applied to other probes, resulting in limited generalizability. Researchers subsequently used universal tools such as ViennaRNA [8] and NUPACK [9] to calculate the thermodynamic properties that affect the hybridization efficiency of nucleic acid sequences to screen probes. These tools, grounded in nucleic acid thermodynamics, predict the most stable secondary structure by assuming that sequences adopt the lowest free-energy conformation, such as NUPACK. However, they are unable to capture all sequence-specific interactions and complexities, relying heavily on thermodynamic properties that poorly reflect real experimental outcomes [10]. For example, while NUPACK predicts secondary structures, it cannot fully explain the sequence-dependent hybridization efficiency [11]. Moreover, DNA hybridization is closely tied to stability, the number of nucleating interactions [11], and the position and type of probe base mutations [12]. Collectively, these findings indicate that thermodynamic properties, such as free energy, are insufficient to predict oligonucleotide hybridization efficiency comprehensively [13].

With the development of deep learning, some studies have begun to use deep learning models to predict sequence hybridization. Zhu et al. employed boosting trees and random forests [14], whereas Buterez et al. applied convolutional and recurrent neural networks to predict RNA/DNA hybridization [15]. Wang et al. developed a quantum chemistry-based deep learning model that outperforms nearest neighbor models in predicting DNA hybridization free energy and strand displacement rate constants [16]. However, these studies have not deviated from traditional paradigms; the former relies on NUPACK-predicted results as gold standards due to the scarcity of real hybridization datasets, limiting its ability to capture complex sequence interactions. The latter focuses on thermodynamic properties and fails to directly reflect the actual probe hybridization efficiency. Thus, constructing large-scale, real-world hybridization efficiency datasets and models is essential for advancing the understanding of oligonucleotide probe interactions and optimizing chip probe detection systems. Building on this foundation, we constructed comprehensive datasets of real probe hybridization efficiencies from extensive microarray data—encompassing 8 subsets with over 3.8 million probes, integrating both public resources and in-house comparative data—and developed TransHybrid, a novel transformer-based model. By embedding double-stranded DNA molecular features into its transformer architecture, TransHybrid uniquely captures strand-specific properties. Requiring only two probe sequences or a probe‒target pair as input significantly lowers application barriers while enhancing generalizability. Through transfer learning, its pretrained parameters enable rapid adaptation to new datasets, reducing training costs across diverse experimental scenarios. These advantages establish TransHybrid as a high-precision tool for probe design, empowering researchers to screen probes efficiently with high specificity and binding efficiency. This capability enhances the accuracy and efficiency of chip probe hybridization and related experiments, with broad potential applications in precision medicine, in situ hybridization, and beyond.

Materials and methods

Data acquisition

For the hybrid data, we selected eight Affymetrix gene expression microarrays, namely, GPL570, GPL6244, GPL16043, GPL1355, GPL6247, GPL6246, GPL1261, and GPL2529, which cover four organisms: humans, mice, rats, and yeasts. For each array, we collected eight datasets from different tissues in GEO database, and five samples from each dataset were selected to yield 40 multitissue samples per array for subsequent processing (Supplementary Document 1).

Data processing

To facilitate understanding of the overall workflow, a clear data acquisition and processing pipeline is systematically described in this section, covering raw data collection, probe filtering, signal processing, thermodynamic feature calculation, and dataset partitioning for model training.

For the eight Affymetrix microarray datasets, only perfectly matched (PM) probes were selected, excluding duplicate probes and those targeting unknown transcripts. The final number of probes per microarray, along with associated details, is presented in Supplementary Table S1. For each microarray, expression data from 40 samples were extracted for the selected PM probes. The raw signal intensity for each probe was calculated according to the following formula:

1

S_{j, k,m} represents the signal values of probe k in probe set j of sample m, a_{j, m} represents the abundance of transcript j in sample m, e_{j, k} represents the hybridization efficiency of probe k in probe set j, λ represents a global scaling factor, and n represents noise. We follow the basic process of chip data processing to remove background, normalize quantiles, and perform logarithmic transformation on the raw data. The resulting signal value formula is as follows:

2

S’_{j, k,m} are the signal values of the processed signal values, log2 (a_{j, m}) is the abundance after log2 conversion, log2 (e_{j, k}) is the hybridization efficiency after log2 conversion, and ε_{j, k,m} is the error term. For probe set data targeting the same transcript, the relative hybridization effects of each probe were extracted via Medpolish [17]:

3

θ_{j, m} is the expression level of transcript j in sample m, which is related to transcript abundance and the average hybridization efficiency of the probe set. e_k is the effect of probe k, reflecting its relative hybridization efficiency. ε_{j, k,m} is the error term. The average hybridization effect for each probe set was calculated as follows:

4

C_j represents the average hybridization efficiency of the log2 probe set (j). S_{j, m} is a signal that approximately represents the abundance of transcripts. The overall hybridization effect of each probe is ultimately obtained, and the overall hybridization effect is used as the final hybridization evaluation index for each probe:

5

We employed the integrated Python program of NUPACK to compute thermodynamic properties for probe sequences and their target sequences, including p_unpair(the probability of a base being unpaired), pfunc(the partition function), CFE(the constrained free energy), MFE(Minimum Free Energy) pair, MFE probe, MFE target, and concentration(the amount of binding product), with the temperature set at 57 °C per the literature. Simultaneously, the GC content and TM values of the probe sequences were calculated via Vienna RNA, yielding a total of nine thermodynamic parameters for the probe‒target pairs.

To ensure species generalizability across training and external testing, GPL570, GPL6244, GPL1261, and GPL6246 were pooled and partitioned into an 8:2 training-test set. Four additional microarray datasets served as external independent validation sets, with all the models trained via identical data splits.

Transfer learning data generation

The Banff DNA chip was designed for hybridization experiments and was fabricated via light-directed synthesis technology via a custom high-resolution photolithography aligner and DNA oligonucleotide synthesizer controlled by DNAPrinter™ 2.0 software (Centrillion Tech, Palo Alto, CA, USA). The chip incorporates tiled 21-mer oligonucleotide probes, enabling rapid imaging via single field-of-view acquisition. To standardize probe hybridization efficiency, a sandwich hybridization strategy was employed, comprising three components: a 21-mer chip probe sequence, a 44-mer sandwich sequence (combining a chip-complementary single strand and a universal sequence), and a 23-mer single-stranded target sequence complementary to the universal segment of the sandwich sequence.

Single-stranded targets (5’-CY3 labeled) and sandwich probes were synthesized by Sangon Biotech (Shanghai), dissolved in the respective buffers, and hybridized on the Banff chip using a buffer containing 5 mM EDTA, 6.25 mM HEPES (pH 8.0), 312.5 mM NaCl, and 1.25% Ficoll. The protocol involved thermal denaturation of the targets and sandwich probes at 95 °C for 1 min, incubation at 5 °C for 30 min, addition of the chip to the hybridization mixture, incubation at 50 °C for 60 min, washing steps (3 × 5 min in 4×SSC/0.1% Tween-20 at 42 °C, followed by 3 × 5 min in 2×SSC/0.1% Tween at RT), scanning on a Centrillion Summit™ scanner (5 s per channel: brightfield and CY5), and image analysis via VirusHunter™ software. Following three replicate experiments, fluorescent signals were extracted and normalized against Affymetrix data, and median values were computed, yielding an independent set of experimental data comprising 3,357 21-mer probes.

Probe comparison dataset generation

For the eight Affymetrix chip datasets, to establish comparative samples for supervised learning, a combinatorial enumeration strategy was employed for each chip to generate sequence pairs, fully mining the competitive relationships between probes within the same transcript: for a transcript containing n probes, the first probe was sequentially paired with the 2nd to nth probes, the second probe with the 3rd to nth probes, and so forth, thus generating C(n,2) groups of nonrepeating sequence pairs (seq1, seq2). The Mann‒Whitney U rank-sum test was used to analyze distribution differences in two groups of several experimental data: when the significance level was p < 0.05 and the test statistic was > 0, the data were labeled “seq1 has stronger hybridization efficiency” (Label 1); when p < 0.05 and statistic < 0, the data were labeled “seq2 has stronger hybridization efficiency” (Label 2); otherwise, the data were labeled “similar hybridization efficiency between the two sequences” (Label 0). The processing of the 21-mer independent experimental data was the same as above, and the Mann‒Whitney U rank sum test was used to analyze the distribution differences of duplicate data between the two teams of probes for labeling.

Framework design of TransHybrid

For sequence encoding, we adopted a 1-mer strategy: mapping the DNA bases ‘A’, ‘C’, ‘G’, and ‘T’ to integers ‘1’, ‘2’, ‘3’, and ‘4’, concatenating the encoded sequences of both strands, and inserting special markers [SOS] (encoded as 0) and [EOS] (encoded as 5) at the start, end, and interstrand positions.

The TransHybrid model was implemented via the PyTorch framework with a DistilBERT-based transformer architecture from Hugging Face. The model converts discrete encodings into 768-dimensional dense vectors via a learnable embedding layer, extracts features through 3 layers of transformer encoders (8 attention heads per layer), and uses the global representation of the [CLS] token as input to a two-layer linear output layer.

The data workflow consists of two phases: Pre-training Phase: We utilized large-scale Public Affymetrix Data (comprising 8 subsets) to pre-train the TransHybrid model, establishing a robust baseline for general hybridization efficiency prediction. Transfer Learning Phase: We generated an Independent Experimental Dataset (comprising 3,357 probes) via in-house hybridization experiments. This dataset served as the target domain for fine-tuning. Specifically, we performed full fine-tuning on the pre-trained TransHybrid model, where all layers were set to trainable weights to ensure the model could fully adapt to the specific characteristics of the target experimental dataset. We employed a 5-fold cross-validation strategy on this dataset—fine-tuning the pre-trained parameters on the training folds and evaluating performance on the validation fold—to assess the model’s ability to adapt to specific experimental environments via transfer learning.

For the regression task, the linear layer outputs a single scalar representing the predicted log-transformed hybridization efficiency (log2ϵ), using the mean squared error (MSE) as the loss function and the RMSE, Pearson correlation coefficient (PCC), and Spearman’s rank correlation coefficient (SPCC) as performance metrics. The median hybridization efficiency was set as a threshold to generate binary classification labels for calculating the accuracy (ACC) and F1 score. For the classification task, the linear layer outputs a probability distribution over three classes (accounting for experimental variance: (1) Probe A has significantly higher efficiency than Probe B; (2) Probe B has significantly higher efficiency than Probe A; (3) The difference between the two probes is statistically insignificant), trained end-to-end with cross-entropy loss, and evaluated using the F1 score, area under the receiver operating characteristic curve (AUROC), and area under the precision-recall curve (AUPRC). Both the regression and classification tasks share the exact same TransHybrid encoder architecture and hyperparameters, differing only in the final prediction head and the loss function (MSE for regression, cross-entropy for classification). The hyperparameters of the TransHybrid model, including the number of epochs and batch size, were optimized via grid search and preliminary validation experiments. The model was optimized with the Adam optimizer (learning rate of 1e-4), using a batch size of 512 and trained for 30 epochs on a GeForce RTX 4090 GPU.

Attention scores and feature visualization

Within specific layers of the model, each attention head outputs a set of attention weight matrices that quantitatively describe the attention allocation weights from source token encodings to target token encodings. On this basis, a heatmap was constructed to visualize attention weights across specific layers, reflecting the intensity of attention resource allocation at specific layer–attention head combinations. This provides an intuitive quantitative basis for revealing the core patterns of the model’s attention allocation. To further dissect the model’s decision-making mechanisms at the token level, this study additionally leverages the Captum [18] library to compute gradient-based contribution values for each token. Specifically, by employing algorithms such as Integrated Gradients within Captum, we quantified the extent to which each input token influences the model’s final output or intermediate feature representations. By aggregating these token-level contributions across the dataset, we establish a statistical profile of token importance, which complements attention weight analysis by providing a gradient-based perspective on the model’s focus.​.

Simultaneously, to explore the model’s feature representation patterns and the intrinsic properties of probes in detail, this study performed dimensionality reduction and visualization analyses on two types of data separately: on the one hand, dimensionality reduction was applied to the thermodynamic properties of the probes; on the other hand, dimensionality reduction was conducted on the intermediate feature embedding extracted from the token of the last hidden state of the DistilBERT module in TransHybrid. Both types of data were mapped to a low-dimensional space via the uniform manifold approximation and projection (UMAP) algorithm [19] for visualization. Beyond visualization, the study further applied the k-means [20] clustering algorithm to the low-dimensional embeddings obtained from UMAP. After both the thermodynamic properties of the probe and the TransHybrid intermediate features were projected into a 2D space via UMAP, K-means was used to group the data points into two clusters. As the primary goal of the UMAP-based analysis is to intuitively illustrate the discriminative capacity of the learned feature representations, clustering into two clusters is sufficient and straightforward to reveal the separability of the embedded features. Together, these analyses offer a multidimensional perspective for dissecting model feature extraction patterns and understanding probe attribute characteristics.

Benchmark methods

Xgboost [21] was employed to predict probe hybridization efficiency on the basis of sequence thermodynamic properties, with distinct processing for different datasets. For the eight Affymetrix chip datasets, which contained only perfect match (PM) data, nine thermodynamic properties were used as model inputs. Furthermore, to ensure a fair comparison, we performed a grid search to optimize the Xgboost hyperparameters (learning rate: [0.05, 0.1, 0.3], max depth: [3, 5, 7], min_child_weight: [1, 3, 5], subsample: [0.6, 0.8, 1.0]). The results reported correspond to the best-performing configuration.

For the LSTM model, the architecture described in the literature was adopted [15], with a 3-layer bidirectional LSTM utilized to capture sequence context dependencies. Additionally, the ‘nucleotide-transformer-v2-250 m-multispecies’ version of the genomic large model nucleotide-transformer (NT) was employed to encode each probe sequence and target sequence [22]. The embeddings of the corresponding probe and target sequences were concatenated to form a total sequence embedding of 1536 dimensions. A five-layer multilayer perceptron (MLP) model was used for downstream regression prediction on the sequence embeddings from the pretrained genomic large model. The performance indicators of the above model are consistent with those of the TransHybrid model.

Results

TransHybrid enables stable and accurate prediction of probe hybridization efficiency

This study presents TransHybrid, a novel model developed to predict oligonucleotide probe hybridization efficiency with high accuracy via the use of sequence information alone (Fig. 1a). Unlike traditional methods reliant on preselected thermodynamic properties, TransHybrid employs an innovative deep learning architecture to extract complex structural patterns directly from sequence data. Comparative analyses demonstrate that deep learning models, including TransHybrid, significantly outperform thermodynamic property-based models across all evaluated metrics (Fig. 1b–f; Tables 1 and 2). Specifically, TransHybrid achieved superior predictive performance, with an average Pearson correlation coefficient (PCC) of 0.6235—approximately double that of the thermodynamic models—along with an average Spearman correlation coefficient (SPCC) of 0.6012, a root mean square error (RMSE) of 1.0462, an accuracy (ACC) of 0.7142, and an F1 score of 0.7122 (Table 2). In contrast, the Xgboost model, which relies on thermodynamic properties, exhibited inferior performance across the validation and test datasets, with average PCC, SPCC, RMSE, ACC, and F1 scores of 0.2993, 0.2723, 1.3175, 0.5771, and 0.5475, respectively (Fig. S1, Table 2). Notably, the multilayer perceptron (MLP) model utilizing nucleotide (NT) encodings underperformed other deep learning models, yielding average PCC, SPCC, RMSE, ACC, and F1 scores of 0.5003, 0.4763, 1.1387, 0.6662, and 0.6513, respectively (Table 2), indicating its limited efficacy compared with sequence-based prediction models.

Fig. 1.

TransHybrid model prediction performance. A Schematic diagram of the TransHybrid model structure, including encoding, input, model processing, and output. B–E Scatter density plots of the TransHybrid prediction results on four independent Affymetrix validation sets (GPL2529, GPL1355, GPL16043, and GPL6247). The x-axis represents the log2-transformed real hybridization efficiency, and the y-axis represents the predicted values. The yellow solid line is the fitted line, the red dashed line is the y = x symmetry line, and the values of the PCC, SPCC, and RMSE are annotated in the plots. F Bar chart of the performance metrics (PCC, SPCC, ACC, F1, and RMSE) of TransHybrid and its compared baselines on the Affymetrix dataset. Different models are represented by different colors. G Density plot of the PCC values of different models on multiple validation sets. The color shading represents the degree of concentration of the values. H Bar chart of the mean PCC values between thermodynamic properties and real hybridization efficiency across eight Affymetrix datasets, ordered from negative to positive. I 3D bar chart depicting the frequency distribution of PCC values between thermodynamic properties and real hybridization efficiency across eight Affymetrix datasets, illustrating numerical range distributions

Table 1.

Performance prediction of affymetrix data via TransHybrid

Model	Metrics	val	GPL2529	GPL1355	GPL6247	GPL16043
TransHybrid	PCC	0.6751	0.5156	0.6700	0.6534	0.6035
	SPCC	0.6512	0.4918	0.6430	0.6475	0.5726
	RMSE	1.0208	0.9726	0.9815	1.0269	1.2294
	ACC	0.7850	0.7210	0.7717	0.7807	0.7671
	F1	0.7741	0.7186	0.7710	0.7683	0.7562

Table 2.

Comparison of the predictive performance of affymetrix data from different models

Model	PCC	SPCC	RMSE	ACC	F1
Xgboost	0.2993 ± 0.1311	0.2723 ± 0.1311	1.3175 ± 0.1431	0.5771 ± 0.0601	0.5475 ± 0.0954
NT	0.5003 ± 0.0637	0.4762 ± 0.0619	1.1387 ± 0.1097	0.6662 ± 0.0245	0.6513 ± 0.0356
LSTM	0.6003 ± 0.0588	0.5761 ± 0.0610	1.0588 ± 0.1058	0.7026 ± 0.0253	0.7054 ± 0.0269
TransHybrid	0.6235 ± 0.0666	0.6012 ± 0.0692	1.0462 ± 0.1051	0.7142 ± 0.0300	0.7122 ± 0.0307

Bold values represent the best-performing metrics among all models

In addition, TransHybrid demonstrated greater consistency across datasets than did the thermodynamic property-based Xgboost model, as evidenced by more stable PCC and SPCC values (Fig. 1g, Fig. S2). For example, TransHybrid achieved a PCC of 0.5156 on the GPL2529 dataset, highlighting its robust generalizability, whereas the thermodynamic model yielded a PCC of only 0.1441 on the same dataset (Fig. S1, Table S2). This trend of superior performance by TransHybrid extends to independent thermodynamic properties, further confirming its ability to capture relevant patterns effectively. Correlation analyses between hybridization efficiency and thermodynamic properties across eight datasets revealed that the probe concentration had the highest average SPCC (0.2709) and PCC (0.2586) (Fig. 1h, Fig. S3a). However, parameter correlations varied substantially across datasets; for example, the SPCC for minimum free energy (MFE_pair) ranged from − 0.0293 to 0.4101, a 20-fold difference (Tables S3–S4). Most datasets presented PCC and SPCC values between − 0.2 and 0.2, with only a few reaching 0.4 (Fig. 1i, Fig. S3b). These findings underscore the limitations of relying solely on thermodynamic properties for predicting hybridization efficiency and highlight the advantages of TransHybrid’s integrated, sequence-based approach.

TransHybrid accurately predicts by capturing features of bases at different positions

To further elucidate the performance of TransHybrid, we conducted visualization analyses to examine the distributions of the probe thermodynamic properties and TransHybrid-derived features across four independent external validation datasets. Probes with similar features were clustered via the K-means algorithm, yielding two distinct clusters (Fig. 2a–b, Fig. S4–S6). Analysis of hybridization efficiency distributions across these clusters revealed significant differences between clusters defined by thermodynamic properties only in the GPL6247 dataset (0.01 < p < 0.05). In contrast, no statistically significant differences were detected in the GPL2529, GPL1355, and GPL16043 datasets (*p* > 0.05). However, clusters defined by TransHybrid features exhibited significant differences in hybridization efficiency across all datasets (p < 0.001) (Fig. 2c–d, Fig. S4–S6). These findings confirm that TransHybrid effectively captures critical sequence-derived information related to hybridization efficiency, demonstrating superior feature representation compared with traditional thermodynamic property indices.

Fig. 2.

Model interpretability. A-B UMAP dimensionality reduction visualization of TransHybrid features and thermodynamic properties of GPL6247 set probes. The horizontal and vertical axes represent simplified UMAP features, with each point representing a probe. The colors are distinguished by the clustering categories of k-means. C-D Box plots of hybridization efficiency distributions in different clusters were used to calculate significance via t tests, NS: p > 0.05, *: p < 0.05, **: p < 0.01. ***: p < 0.001. E The three probe sequences of the GPL6247 dataset were visualized via DrawDNA (https://www.petercollingridge.co.uk/tools/draw-dna/) for base visualization. MFE was the binding free energy calculated by NUPACK, true log2 (HE) was the true hybridization efficiency processed by log2, and Xgboost and TransHybrid were the hybridization efficiencies predicted by the Xgboost and TransHybrid models. The bar chart represents the gradient contribution value of each token calculated by Captum, which is the relative contribution of the input features to the model prediction. A positive value represents a positive contribution to the prediction, whereas a negative value represents a negative contribution to the prediction. F The gradient contribution heatmaps of all probes on the GPL6247 and GPL2529 datasets are derived from the calculated average contribution of different token types at each position. G Heatmap of the sum of attention for all layers and heads on each token. Corresponding to the probes in Figure e from top to bottom

To investigate the interpretability of TransHybrid’s predictions, we analyzed three probes targeting the Zfhx2 gene in the GPL6247 dataset (Fig. 2e). Compared with TransHybrid’s predictions, both the MFE-based predictions and Xgboost outputs exhibited substantial deviations from the actual hybridization efficiencies and failed to track the empirical trends. An examination of the gradient contribution values for individual tokens within each probe revealed that the probe segment (tokens 1–20) generally contributed more significantly than did the target segment (tokens 22–51), with the highest contributions concentrated in the middle of the probe segment. Notably, guanine and cytosine bases were associated with positive contributions, whereas thymine and adenine carried negative contributions—consistent with the known higher stability of GC base pairs. Further analysis of average token contributions across datasets confirmed that GC bases consistently maintained positive contributions, with a more pronounced effect on the probe segment (Fig. 2f, Fig. S7). These results corroborate the generalizable pattern captured by TransHybrid. Analysis of attention weights further supported these findings, revealing higher weights in the probe–probe region, detectable as early as the first network layer (Fig. 2g, Fig. S8).

Collectively, these results demonstrate that TransHybrid captures complex structural features and base interactions within sequences, significantly enhancing the predictive performance for hybridization efficiency compared with thermodynamic property-based models.

TransHybrid enables transfer learning with pretrained parameters across diverse datasets

To evaluate the ability of TransHybrid to predict performance across diverse microarray designs, pretrained parameters were fine-tuned and retrained via independent experimental hybridization efficiency data from 3,357 21-mer probes. The performance of these models was compared against that of thermodynamic property-based models, long short-term memory (LSTM)-based models, and large language models (Fig. 3a). Consistent with prior findings, correlation analyses revealed that the probe concentration had the strongest PCC and SPCC with respect to hybridization outcomes (Fig. 3b; Table 3). The prediction results corroborated earlier trends: the deep learning models outperformed the thermodynamic property-based models, with the fine-tuned TransHybrid model outperforming the thermodynamic models by more than 25% in terms of the PCC and SPCC (Fig. 3c). Compared with the retrained TransHybrid model, the fine-tuned TransHybrid model also outperformed the other deep learning models, achieving higher PCC, SPCC, ACC, and F1 scores and a lower RMSE. These results suggest that parameters pretrained on large-scale 25-mer probes enhance training and prediction for diverse probe lengths and types (Fig. 3d). The visualization of two probes from the same cross-validation fold revealed that TransHybrid accurately predicted the trend of probe hybridization efficiency that MFE and Xgboost could not reflect (Fig. 3e). The contribution value of each token remains consistent with that of the previous text, with most GC bases maintaining a positive contribution gradient. These findings further validate the efficiency and generalizability of TransHybrid, laying a foundation for future in-depth investigations.

Fig. 3.

Transfer learning performance of the TransHybrid model. A Schematic of the hybrid transfer learning workflow, including data generation and transfer training procedures. B Pearson correlation between thermodynamic properties and hybridization efficiency in the independent experimental 21 bp dataset. The color intensity indicates the correlation strength. C Radar chart of fivefold cross-validation performance metrics for different models, including the PCC, SPCC, ACC, F1, precision, recall, and RMSE. Metrics are normalized by the maximum value of each indicator, with different colors representing distinct models. D Bar chart of fivefold cross-validation performance metrics for different models. Different models are color coded. E Visualization of the predicted probes. The left panel shows the fluorescence signal map on the microarray; the right panel presents the DrawDNA–visualized base-pairing diagram of the probe sequence. The bar chart represents the gradient contribution value of each token calculated by Captum, which is the relative contribution of the input features to the model prediction. A positive value represents a positive contribution to the prediction, whereas a negative value represents a negative contribution to the prediction. MFE is the binding free energy calculated by NUPACK, Norm HE is the processed real hybridization efficiency, and Xgboost and TransHybrid are the hybridization efficiencies predicted by the Xgboost and TransHybrid models

Table 3.

Average performance prediction of independent experimental data data via TransHybrid

Model	PCC	SPCC	RMSE	ACC	F1
Xgboost	0.4950 ± 0.0236	0.4894 ± 0.0213	2.0312 ± 0.0561	0.6661 ± 0.0114	0.6643 ± 0.0122
NT-FT	0.5308 ± 0.0351	0.5347 ± 0.0358	1.9824 ± 0.0411	0.6965 ± 0.0229	0.6950 ± 0.0236
LSTM-FT	0.6462 ± 0.0090	0.6605 ± 0.0144	1.7881 ± 0.0623	0.7566 ± 0.0055	0.7589 ± 0.0068
TransHybrid-retrain	0.5603 ± 0.0234	0.5669 ± 0.0250	1.9539 ± 0.0549	0.7093 ± 0.0117	0.7085 ± 0.0126
TransHybrid-FT	0.7001 ± 0.0253	0.7096 ± 0.0311	1.7019 ± 0.0573	0.7864 ± 0.0112	0.7861 ± 0.0111

Bold values represent the best-performing metrics among all model

Ablation experiment of the TransHybrid component

To evaluate the contributions of key architectural components in TransHybrid, we performed an ablation study to assess the impact of transformer layer depth and the incorporation of complementary probe pair sequences. We tested models with 3, 6, and 12 transformer layers, configured to use either single probe sequence encoding (denoted “+”) or complementary probe pair sequences (denoted “−”). The results (Table 4) revealed distinct trends. When complementary pair sequences (probe only “−”) were employed, increasing the layer depth from 6 to 12 resulted in a modest improvement in the PCC from 0.6205 to 0.6227 and a slight reduction in the RMSE from 1.0456 to 1.0415, indicating marginal gains in linear correlation and prediction accuracy. Conversely, models relying solely on single probe sequences (Probe Only “+”) exhibited inferior performance across all the metrics. For example, at 6 layers, the PCC decreased from 0.6205 (with complementary pairs) to 0.5773 (with a single probe), a reduction of approximately 7%, with corresponding decreases in the SPCC. These findings highlight the critical role of complementary probe pair sequences in capturing structural relationships essential for accurate hybridization efficiency prediction.

Table 4.

Ablation analysis results of TransHybrid

Layers	Probe Only	PCC	SPCC	RMSE
6	-	0.6205 ± 0.0691	0.5987 ± 0.0723	1.0456 ± 0.1060
12	-	0.6227 ± 0.0633	0.6010 ± 0.0657	1.0415 ± 0.1063
3	+	0.5775 ± 0.0681	0.5547 ± 0.0718	1.0813 ± 0.1113
6	+	0.5773 ± 0.0683	0.5543 ± 0.0721	1.0804 ± 0.1122
12	+	0.5737 ± 0.0676	0.5518 ± 0.0704	1.0862 ± 0.1109

With respect to layer depth, under complementary pair conditions, the 12-layer model slightly outperformed the 6-layer model in terms of the PCC, but improvements in other metrics were minimal, and differences compared with the 3-layer model were negligible. This suggests that the inclusion of complementary pair sequences has a more substantial effect on model performance than increasing layer depth does. These results underscore the importance of strategic architectural design and sequence encoding in the TransHybrid framework, demonstrating that lightweight TransHybrid models can achieve robust predictive performance with high efficiency.

TransHybrid enables precise prediction of hybridization efficiency hierarchy between probe pairs

To validate the model’s ability to classify hybridization efficiency disparities between sequences, we select pairs of probe sequences from comparative datasets as inputs and evaluate TransHybrid against the LSTM-based model. The results (Table 5) indicate that TransHybrid achieved an average AUROC of 0.7556 across four independent external validation sets, with AUPRC and F1 scores reaching 0.6136 and 0.5585, respectively—outperforming the LSTM (AUROC: 0.7334, AUPRC: 0.5910, F1: 0.5526). In addition, we calculated the concentration difference between the two probes as the thermodynamic characterization of the probe pair, as it has the highest correlation with hybridization efficiency. The results revealed that the thermodynamic characterization prediction probe had extremely poor classification performance, with AUROC, AUPRC, and F1 values of only 0.5000, 0.3502 and 0.2909, respectively, which were significantly lower than those of TransHybrid. These findings demonstrate the efficiency of TransHybrid in predicting hybridization efficiency class disparities between sequences. Furthermore, a threefold cross-validation of independent experimental data mirrored the regression experiment outcomes: the fine-tuned TransHybrid model exhibited optimal performance, with mean AUROC, AUPRC, and F1 scores of 0.7161, 0.7520, and 0.7244, respectively, surpassing thermodynamic characterization, fine-tuned LSTM, and retrained TransHybrid. This confirms the generalizability of TransHybrid in efficiently classifying hybridization efficiency disparities between probe pairs.

Table 5.

Performance prediction of comparative datasets via TransHybrid

model	dataset	AUROC	AUPRC	F1
Concentration-diff	affymetrix	0.5000 ± 0.0001	0.3502 ± 0.0166	0.2909 ± 0.0298
LSTM		0.7267 ± 0.0475	0.5807 ± 0.0635	0.5415 ± 0.0527
TransHybrid		0.7556 ± 0.0307	0.6136 ± 0.0420	0.5585 ± 0.0384
Concentration-diff	Experimental hybridization dataset	0.5136 ± 0.0193	0.3589 ± 0.0052	0.2115 ± 0.0115
TransHybrid-retrain		0.6494 ± 0.0192	0.6881 ± 0.0208	0.6893 ± 0.0193
LSTM-ft		0.7018 ± 0.0176	0.7308 ± 0.0113	0.7168 ± 0.0166
TransHybrid-ft		0.7161 ± 0.0207	0.7520 ± 0.0122	0.7244 ± 0.0117

Bold values represent the best-performing metrics among all models

Discussion

Kinetics and thermodynamics are critical factors in hybridization processes, with traditional probe selection being guided primarily by simulated thermodynamic properties. However, these approaches have limited applicability in real-world scenarios. Using NUPACK, a widely adopted tool for predicting sequence interactions, we calculated thermodynamic parameters for probe–target binding, including the minimum free energy of the structure and product concentration. Consistent with prior studies, the product concentration was positively correlated with hybridization efficiency, whereas the structure free energy was negatively correlated with hybridization efficiency [23]. Additionally, the GC content and melting temperature (Tm), key parameters associated with hybridization kinetics and thermodynamics, were positively correlated with hybridization efficiency, similar to the product concentration [24]. These findings suggest that thermodynamic properties moderately reflect the nucleotide sequence composition for hybridization but lack sufficient predictive power. Notable variability in generalizability was observed across the microarray platforms, with PCC ranging from 0.1441 (GPL2529) to 0.2276 (GPL1355) and 0.4477 (GPL6247), underscoring the limitations of thermodynamic models in capturing hybridization efficiency and highlighting the need for a more effective predictive approach.

Previous probe design tools, such as Oli2go [25] and chipD [26], focused on optimizing probe specificity or enabling customizable parameters for specificity, thermodynamic properties, and coverage uniformity, particularly for bacterial and yeast genomes. However, these tools do not address inconsistent hybridization efficiency. Here, we introduce TransHybrid, the first end-to-end model trained on real-world data to predict sequence hybridization efficiency. TransHybrid enhances probe design by accurately predicting efficiency directly from probe sequences. When only sequences are input, TransHybrid effectively identifies hybridization patterns to filter low-efficiency probes, surpassing traditional thermodynamic models in both accuracy and computational efficiency. To investigate the potential impact of data leakage caused by correlations among probes targeting the same transcript, we performed an additional validation using a transcript-level splitting strategy. In this setup, probes were partitioned such that all sequences belonging to the same transcript appeared exclusively in either the training or test set. The TransHybrid model achieved a PCC of 0.6227 and RMSE of 1.0533 under this strict splitting regime(Table S5), which is statistically indistinguishable from the results obtained via random splitting. This consistency indicates that the model relies on sequence-specific hybridization features rather than transcript-level confounds. Furthermore, its predictive stability and generalizability were validated on multispecies Affymetrix test microarrays and independent experimental 21-mer probe datasets via transfer learning. Notably, encodings from the NT genomic large language model underperformed expectations, only marginally outperforming thermodynamic models, likely owing to its training on single-stranded nucleotide sequences, which may be better suited for sequence classification than interaction prediction.

Despite the aforementioned advantages of TransHybrid in predicting hybridization efficiency and generalizing across multiple microarray platforms, this study has several limitations that should be acknowledged. First, the model was trained and validated primarily on 25-mer and 21-mer probes; its performance on other probe lengths remains untested. Additionally, although our data preprocessing pipeline effectively linearizes the probe signals, raw microarray hybridization fundamentally follows nonlinear Langmuir adsorption kinetics. TransHybrid does not explicitly embed these physical equations into its architecture, which may restrict its predictive accuracy at extreme concentration boundaries. Third, although TransHybrid implicitly captures complex factors related to hybridization, it does not explicitly model the underlying molecular mechanisms (e.g., detailed base-pairing dynamics, steric effects at the single-molecule level), which limits its interpretability in elucidating the specific sequence features that drive hybridization efficiency. Finally, the number of independent external validation datasets remains limited; future studies with larger and more diverse external datasets are needed to further confirm the model’s robustness in practical applications.

With the advancement of large genomic models, more advanced models such as Evo2 [27] and AlphaGenome [28] have demonstrated stronger performance than NT in genomic variant prediction, and their performance in predicting double-strand hybridization can be further tested. Future research will focus on addressing the aforementioned limitations through data augmentation and feature extraction, including generating diverse real-world probe hybridization datasets, training large models on double-stranded genomes, and integrating structural and interaction features to further enhance model performance. Moreover, future iterations of the model could explore Physics-Informed Neural Networks (PINNs) to explicitly incorporate Langmuir-based nonlinear adsorption kinetics, potentially further improving prediction performance at extreme concentration boundaries. Practically, TransHybrid offers a distinct advantage over traditional thermodynamic models by directly learning from experimental signal data, thereby implicitly capturing complex factors like surface steric hindrance that MFE models miss. For practitioners, this translates to a more reliable probe design tool that can predict high-efficiency sequences for diagnostic applications without requiring manual parameter tuning or computationally expensive physical simulations.

Authors' contributions

Leiming Fang: Conceptualization, Methodology, Writing—Original draft preparation. Hanlin He: Methodology, Data Curation. Xingyun Liu and Yiqing Li: Formal analysis. Wenjun Song: Investigation and Writing—Reviewing and Editing. Taijiao Jiang and Yuedong Yang: Supervised and Coordinated the study, Writing—Reviewing and Editing.

Funding

This work was supported by the Major Talent Project of Dr. Taijiao Jiang for Guangzhou Laboratory, (Project Number: MP-GZNL2025C03010-01).

Data availability

The Affymetrix data used in this study are available from the GEO database, with specific accession numbers provided in Supplementary document 1. Related codes are deposited in Zenodo under DOI 10.5281/zenodo.18051481 and the independent experimental data in Zenodo under DOI 10.5281/zenodo.17430213.

Declarations

Competing interests

Centrillion provided the microarrays used in this study but had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.