BiToxNet: a deep learning framework integrating multimodal features for accurate identification of neurotoxic peptides and proteins

Abstract

Background

Accurate prediction of the neurotoxicity of peptides and proteins is critically important for the safety assessment of protein therapeutics and the development of protein-based drugs. Although experimental methods can reliably identify neurotoxic peptides and neurotoxins, they are labor-intensive, costly, and unsuitable for large-scale screening. Existing computational approaches are often limited by shallow feature engineering and suboptimal multimodal fusion strategies, which restrict their predictive accuracy and generalizability in real-world applications.

Results

In this study, we propose BiToxNet, a deep learning framework that integrates evolutionary embeddings derived from a protein large language model with ten handcrafted biochemical descriptors through a bilinear attention network (BAN). This design enables effective modeling of cross-modal interactions and residue-level dependencies critical for neurotoxicity prediction. BiToxNet was evaluated on three datasets of different sequence lengths, namely Protein, Peptide, and Combined datasets, achieving accuracies of 92.3%, 96.0%, and 92.7%, respectively, and consistently outperforming existing state-of-the-art methods. Ablation studies confirmed the importance of both evolutionary embeddings and handcrafted features, as well as the critical role of BAN in feature fusion. Visualization analyses using t-SNE and hierarchical clustering further demonstrated that BiToxNet learns highly discriminative representations without reliance on domain-specific prior knowledge. Additional evaluation on an external imbalanced dataset validated the robustness and strong generalization capability of the proposed framework.

Conclusions

Overall, BiToxNet provides a powerful and generalizable computational framework for the accurate identification of neurotoxic peptides and proteins. By effectively integrating evolutionary and biochemical information through bilinear attention, BiToxNet offers a valuable tool for neurotoxin screening and protein drug safety assessment, and presents a distinctive modeling strategy applicable to a wide range of biological sequence analysis tasks.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-026-02508-8.

Background

Neurotoxins, a class of bioactive proteins derived from venomous animals, exert potent effects on neuronal tissues by disrupting synaptic transmission [1, 2]. Based on their site of action, neurotoxins are categorized into presynaptic and postsynaptic subtypes, each with distinct mechanisms [3–6]. Presynaptic neurotoxins ($\beta$-neurotoxins) inhibit acetylcholine release or destabilize the presynaptic membrane, ultimately blocking neuromuscular transmission through enzymatic disruption of neurotransmitter exocytosis. In contrast, postsynaptic neurotoxins ($\alpha$-neurotoxins) selectively bind to nicotinic acetylcholine receptors (nAChRs) on the postsynaptic membrane, competitively antagonizing acetylcholine and paralyzing nerve signal propagation [7, 8]. These mechanisms of action involving presynaptic inhibition of release by $\beta$-neurotoxins and postsynaptic blockade of receptor binding by $\alpha$-neurotoxins are illustrated in Fig. 1.

Fig. 1.

Mechanistic illustration of synaptic blockade by neurotoxins. The diagram depicts $\beta$-neurotoxins inhibiting acetylcholine (ACh) release through disruption of exocytosis, while $\alpha$-neurotoxins competitively antagonize neuromuscular transmission by binding to nAChRs on the postsynaptic membrane

Given the increasing use of protein-based therapeutics, accurate identification of neurotoxins can effectively assess the safety of protein drugs. Moreover, some neurotoxins exhibit potential therapeutic value. For instance, cone snail-derived $\alpha$9$\alpha$10-selective conotoxin RgIA exhibits anti-nociceptive properties in neuropathic pain models [9], while spider venom peptides (e.g., cysteine knot toxins and m-TRTX-Ca2a) modulate voltage-gated Na⁺ channels (Nav1.7), offering promise for treating chronic pain and epilepsy [10]. Presynaptic neurotoxins have further been leveraged clinically for conditions such as migraines and cerebral palsy [11, 12].

However, achieving effective prediction faces significant barriers, exacerbated by their extreme structural and functional diversity. For example, as a representative source of diverse neurotoxins, spider venoms secreted from specialized glands contain small cysteine-rich peptides targeting ion channels alongside high-mass latrotoxins affecting neurotransmitter release [13]. Traditional biochemical approaches, while reliable, are time-intensive and costly, creating bottlenecks in neurotoxin discovery. To overcome these limitations, computational methods have emerged as powerful predictive alternatives. These include Saha and Raghava’s support vector machine (SVM)-based NTXpred model integrating amino acid composition (AAC), dipeptide composition (DPC), protein length, and Position-Specific Iterated BLAST (PSI-BLAST) evolutionary profiles to predict neurotoxins and classify them by biological source and function [14]. Similarly, Guang et al. developed an SVM classifier leveraging AAC, position-specific scoring matrices (PSSM), and biophysical properties for neurotoxin classification [15]. More recently, the field has seen the advent of even more sophisticated techniques, such as Rathore et al.’s NTxPred2 [16], a large language model (LLM)-based framework employing distinct specialized models for neurotoxic peptides and neurotoxins. This approach integrates compositional features, physicochemical properties, protein language model embeddings, and fine-tuned classifiers to significantly enhance neurotoxicity prediction capability. Although significant progress has been made, computational neurotoxin prediction still faces some limitations. Early SVM-based methods rely heavily on handcrafted feature engineering and shallow architectures, failing to capture complex sequence-function relationships and residue-level interactions. Moreover, even advanced LLM-based tools like NTxPred2, which incorporate valuable evolutionary context, lack explicit fusion with biochemical properties and cannot dynamically align multimodal features (e.g., physicochemical constraints with residue-wise evolutionary embeddings). Collectively, these gaps fundamentally impair the predictive accuracy and generalizability of existing methods for identifying neurotoxins.

To address these limitations, in this study, we present BiToxNet, a novel deep learning framework that leverages bilinear attention mechanisms for accurate neurotoxin prediction across peptides and proteins. Compared to existing methods, BiToxNet introduces three key innovations: the integration of evolutionary embeddings from the Evolutionary Scale Modeling 2 (ESM2) protein language model with handcrafted descriptors; a dynamic bilinear attention network (BAN) that captures residue-level interactions critical for neurotoxicity; and robust generalizability validated on rigorously curated Peptide, Protein, and Combined datasets. Extensive benchmarking demonstrates that BiToxNet significantly outperforms state-of-the-art tools like NTxPred2 across multiple evaluation metrics, providing an effective solution for accelerating neurotoxin discovery in drug development.

Results and discussion

Performance comparison with existing methods

To evaluate the predictive capability of our proposed model for neurotoxin identification, we conducted a comprehensive comparison against established neurotoxin prediction tools. Specifically, BiToxNet was compared with the machine learning method proposed by NTxPred, the deep learning model developed by Lee et al., and the recent protein language model based NTxPred2. The comparison was performed across five key performance metrics: accuracy, MCC, sensitivity, specificity, and AUC. BiToxNet and NTxPred2 were trained on the same datasets proposed in the NTxPred2, ensuring a consistent data foundation. In contrast, NTxPred and the model of Lee et al. were developed using different datasets. To ensure fair and reproducible comparison, we evaluated Lee et al.’s publicly available code and the official NTxPred web server on the same independent test sets used in our study. This unified evaluation strategy eliminates potential biases caused by dataset discrepancies and allows direct performance comparison under identical testing conditions.

As detailed in Table 1, BiToxNet consistently achieved the highest performance across all metrics on three rigorously curated datasets. Notably, for Protein dataset, BiToxNet attained a remarkable accuracy of 92.3%, surpassing NTxPred2 by 5.8% and Lee et al by 28.8%, with an MCC of 0.846 indicating strong class separation. On the Peptide dataset, BiToxNet maintains exceptional performance with 96.0% accuracy and 0.921 MCC, outperforming NTxPred2 by 0.9% and 0.023 respectively. Most critically, on the Combined dataset integrating both peptide and protein data, BiToxNet achieved 92.7% accuracy and 0.855 MCC, representing improvements of 3.7% in accuracy and 0.075 in MCC over NTxPred2. We further validated the robustness of BiToxNet through five-fold CV, as summarized in Additional file 1: Table S1 and illustrated in Additional file 1: Fig. S1. The narrow 95% confidence intervals and compact interquartile ranges of the boxplots indicate small performance variability across folds and confirm the model’s stability and generalization capability.

Table 1.

Comprehensive performance comparison of BiToxNet with existing neurotoxin identification methods on testing sets

Dataset	Method	Accuracy (%)	Sensitivity (%)	Specificity (%)	MCC	AUC (%)
Protein	NTxPred (SVM, DPC+Length)	58.7	41.2	76.1	0.185	58.9
	NTxPred (SVM, DPC)	62.9	63.9	61.9	0.258	71.5
	NTxPred (SVM, AAC+Length)	59.3	52.9	65.8	0.188	64.8
	NTxPred (SVM, AAC)	61.2	57.4	65.1	0.226	66.9
	Lee et al. (2021)	63.5	27.7	99.3	0.383	63.5
	NTxPred2	86.5	82.6	90.3	0.731	93.7
	BiToxNet	92.3	89.7	94.8	0.846	98.2
Peptide	NTxPred (SVM, DPC+Length)	79.2	79.0	79.4	0.584	82.7
	NTxPred (SVM, DPC)	85.5	89.8	81.1	0.712	91.6
	NTxPred (SVM, AAC+Length)	76.1	81.3	70.9	0.524	83.2
	NTxPred (SVM, AAC)	85.5	93.2	77.7	0.718	92.2
	Lee et al. (2021)	49.8	0	100	0	50.0
	NTxPred2	95.1	92.6	97.0	0.898	98.4
	BiToxNet	96.0	94.9	97.1	0.921	99.4
Combined	NTxPred (SVM, DPC+Length)	59.6	58.9	60.3	0.192	65.6
	NTxPred (SVM, DPC)	71.2	65.2	77.2	0.428	78.3
	NTxPred (SVM, AAC+Length)	68.0	60.7	75.5	0.365	69.2
	NTxPred (SVM, AAC)	71.4	66.1	76.6	0.430	77.0
	Lee et al. (2021)	56.3	13.0	99.6	0.245	56.3
	NTxPred2	89.0	87.0	90.9	0.780	95.4
	BiToxNet	92.7	94.6	90.9	0.855	97.5

The exceptional performance of BiToxNet stems from three key synergistic advancements. First, BiToxNet integrates a comprehensive set of ten handcrafted descriptors, explicitly capturing fundamental physicochemical and structural constraints governing neurotoxin function. Second, it leverages powerful evolutionary sequence embeddings derived from the ESM2 protein language model, providing deep contextual understanding of residue-level conservation and long-range dependencies. Third, a dedicated BAN dynamically fuses these complementary feature streams, enabling precise modeling of critical cross-modal interactions essential for neurotoxicity prediction. Collectively, this integrated framework overcomes the limitations of existing methods, such as NTxPred2’s lack of explicit biochemical-evolutionary fusion, establishing BiToxNet’s superior predictive capability.

Investigating the effectiveness of different feature representations

To evaluate the contributions of ESM2 evolutionary embeddings and protein descriptors in BiToxNet, we conducted ablation experiments across the Peptide, Protein, and Combined datasets. Specifically, we systematically compared the following three configurations. (i) w/o ESM2 configuration was generated by removing the ESM2 module while retaining only the handcrafted biochemical descriptors. (ii) A variant designated w/o protein descriptors was created to utilize solely representations derived from the ESM2 module, involving the removal of the handcrafted feature module. (iii) The complete proposed model, BiToxNet, integrates both ESM2 embeddings and handcrafted protein descriptors. We evaluated the performance of BiToxNet and its variants across four key metrics: accuracy, MCC, sensitivity, specificity. The visualization results are presented in Fig. 2, and the detailed quantitative results are provided in Additional file 1: Table S2.

Fig. 2.

Ablation study of feature contributions in BiToxNet. a, d, g Radar charts comparing four metrics (accuracy, sensitivity, specificity, MCC) for handcrafted features, ESM2 embeddings, and integrated features on Peptide dataset, Protein dataset, and Combined dataset. b, e, h ROC curves with AUC values. c, f, i PR curves with AP values

As shown in the radar charts, BiToxNet consistently occupies the outermost region, demonstrating superiority across all four metrics. Specifically, for the Peptide dataset in Fig. 2a, the strategy combining ESM2 and protein descriptors achieves exceptional performance with an MCC of 0.921, significantly surpassing the model using only ESM2 by 0.08 and the model using only protein descriptors by 0.052. A similar trend is observed for the Protein dataset in Fig. 2d, where BiToxNet achieved an MCC of 0.846, outperforming the model employing only ESM2 by 0.084 and the model based solely on protein descriptors by 0.019. Critically, on the Combined dataset (Fig. 2g), BiToxNet maintains a high MCC of 0.855, while the model variants drop to 0.805 (w/o protein descriptors) and 0.837 (w/o ESM2).

ROC curves further validate the superiority of BiToxNet’s feature fusion strategy: our proposed model generates the leftmost trajectories with high AUC values. In contrast, the curves from both the protein descriptor–only and the ESM2-only models show a certain degree of performance degradation. Precision–recall (PR) curves reinforce this pattern, as BiToxNet produces curves that closely approach the upper-right corner, while the model variants show reduced performance.

These findings demonstrate that the evolutionary features extracted by the pretrained ESM2 model play a crucial role in enabling accurate and reliable neurotoxin prediction, likely by capturing long-range residue dependencies and evolutionary constraints inherent in toxin sequences. Similarly, the handcrafted protein descriptors are critical for enhancing the model’s ability to discriminate neurotoxins from non-toxins, as they explicitly encode essential biophysical and biochemical properties. Comparing the performance degradation across the model variants reveals that removing the protein descriptors leads to a slightly greater decline in performance than removing ESM2. This finding suggests that the fundamental physicochemical properties captured by the protein descriptors are crucial for identifying core toxic functionalities, while the evolutionary embeddings remain indispensable by providing complementary contextual and evolutionary information. Neurotoxic peptides and proteins, particularly cysteine-rich toxins, exhibit distinctive residue compositions and local physicochemical constraints [17, 18], and these characteristics are more explicitly represented through handcrafted descriptors that capture amino acid composition, charge distribution, polarity, and hydrophobicity. Consequently, ablation of either feature component impairs overall performance, affirming that a comprehensive feature representation integrating both evolutionary context and biophysical properties is indispensable for robust neurotoxin identification across diverse sequence lengths.

The effectiveness analysis of the bilinear attention network

To investigate the effectiveness of the BAN, we conducted an additional ablation study. Specifically, we compared two settings, w/o BAN module utilized direct concatenation of ESM2 embeddings and protein descriptors without BAN fusion processing, and BiToxNet employed the BAN module for feature integration. Visualization results are illustrated in Fig. 3 for the Combined dataset, with results for the Peptide and Protein datasets provided in Additional file 1: Fig. S2 and Additional file 1: Fig. S3, respectively. Detailed numerical comparisons across all datasets are provided in Additional file 1: Table S2.

Fig. 3.

Performance comparison between models without BAN and BiToxNet across four key metrics on the Combined dataset

As shown in the bar charts, removal of the BAN module significantly impairs performance across all evaluation metrics. Furthermore, ROC curves without BAN fusion consistently shift downward and rightward, while PR curves deviate from the optimal upper-right corner. These observed degradations confirm BAN’s critical role in BiToxNet’s performance. Unlike direct feature concatenation, which simply stacks ESM2 embeddings and handcrafted descriptors without modeling their interactions, the BAN explicitly captures second-order dependencies between modalities. By dynamically weighting cross-modal feature interactions between ESM2’s evolutionary embeddings and handcrafted descriptors, BAN enables BiToxNet to highlight informative and complementary relationships that are essential for precise neurotoxin prediction. For instance, it dynamically resolves couplings between key features, such as hydrophobic cores (captured by Z-scale descriptors) and conserved $\alpha$-helical motifs—critical for neurotoxins like $\kappa$-hefutoxin targeting potassium channels [19, 20]. By computing pairwise attention weights, BAN captures dynamic dependencies that simple concatenation and other fusion schemes miss [21]. Consequently, the bilinear attention mechanism provides a more expressive yet computationally efficient approach for integrating evolutionary and biophysical representations through attention-based feature interaction.

Visualization of positives and negatives in different layers of BiToxNet

To further investigate the effectiveness of the BiToxNet architecture, we employed the t-SNE [22] technique to visualize the distribution of positive and negative samples in training sets across different layers of BiToxNet. Specifically, we extracted latent features from four distinct layers of BiToxNet: features encoded by ESM2, features obtained from ten protein descriptors, features formed by direct concatenation of ESM2 and protein descriptor features, and features fused by the BAN module from ESM2 and protein descriptors. Figure 4 illustrates the results for the Combined dataset, with analogous visualizations for the Peptide and Protein datasets provided in Additional file 1: Fig. S4 and Additional file 1: Fig. S5.

Fig. 4.

t-SNE visualization of positive and negative samples from the training set of the Combined dataset across different layers of BiToxNet

As visualized in Fig. 4a, initial ESM2 embeddings exhibit disordered and mixed distributions, where neurotoxic (orange) and non-toxic (blue) samples appear entangled with no clear separation. Figure 4b demonstrates that Protein descriptor encoding yields increased clustering but substantial overlap between classes. These patterns demonstrate that neither ESM2 alone nor protein descriptors alone can effectively extract features that robustly distinguish neurotoxins from non-toxins. After simply concatenating ESM2 and protein descriptor features in Fig. 4c, separation improves compared to isolated representations, however, class boundaries remain ambiguous, indicating that static fusion fails to resolve complex feature interactions essential for accurate neurotoxicity prediction. When applying BAN to dynamically align features (Fig. 4d), the distinction becomes dramatically evident: neurotoxins coalesce into dense, isolated clusters while non-toxic samples form discrete groupings separated by clear margins. This demonstrates BAN’s capacity to learn salient biological representations critical for neurotoxin identification, enabling precise discrimination of neurotoxins.

Feature analysis

One of the key factors behind the success of deep learning lies in its ability to extract discriminative features through hierarchical representations. Here, we extracted the latent features of BiToxNet from the test set along with five commonly used handcrafted features, and visualized them in two-dimensional space using t-SNE. The results for the Combined, Peptide, and Protein datasets are shown in Fig. 5, Additional file 1: Fig. S6, and Additional file 1: Fig. S7, respectively. As shown in the figures, traditional handcrafted features struggled to separate positive and negative samples. In contrast, BiToxNet’s latent features formed compact, well-separated clusters, with neurotoxic and non-toxic samples occupying distinct regions. This further demonstrates that BiToxNet can automatically extract neurotoxin-related discriminative features through its hierarchical architecture, thereby validating the rationality and effectiveness of its architectural design.

Fig. 5.

t-SNE visualization of feature discriminability on the Combined testing dataset. a-e Traditional handcrafted descriptors exhibit chaotic class overlap. f Latent features extracted by BiToxNet form distinct clusters of neurotoxic and non-toxic samples

To further validate the discriminative power of latent features extracted by BiToxNet, we performed hierarchical clustering [23] on 100 randomly selected samples (50 neurotoxic and 50 non-toxic) from the test sets of the Peptide, Protein, and Combined datasets, as illustrated in Additional file 1: Fig. S8, Additional file 1: Fig. S9, and Fig. 6, respectively. Two important observations can be derived: first, neurotoxic sequences form a distinct subtree that is clearly separated from non-toxic sequences; second, neurotoxic and non-toxic samples exhibit fundamentally divergent feature patterns. This clear class separation across all 100 randomly selected samples strongly confirms that BiToxNet consistently extracts high-fidelity features capturing intrinsic functional distinctions. These results underscore the robustness of BiToxNet in capturing neurotoxin-related discriminative feature representations rather than simply overfitting to the training data, thereby further validating the effectiveness of BiToxNet. Moreover, these discriminative features may provide valuable insights for neurotoxin-related downstream tasks, such as therapeutic peptide design.

Fig. 6.

Hierarchical clustering of latent features extracted by BiToxNet on the Combined testing dataset. Neurotoxic and non-toxic sequences form separated subtrees

Evaluation on external and imbalanced dataset

Real-world neurotoxin data are typically imbalanced and temporally dynamic, posing additional challenges for predictive modeling [24, 25]. To further assess the robustness and generalization ability of BiToxNet, we collected an external and temporally independent combined dataset containing both peptide and protein sequences. The data were curated following the same procedure as the original combined dataset to ensure data quality and consistent preprocessing, including sequence retrieval from Swiss-Prot, redundancy removal by CD-HIT, and all sequences were filtered using CD-HIT to remove homologous overlap with the training data. The resulting dataset comprises 4476 samples, including 746 positive and 3730 negative sequences, yielding a positive to negative ratio of 1 to 5, which is closer to real-world distributions. This temporal holdout design enables evaluation on newly annotated sequences unseen during model training or validation, providing a realistic estimation of BiToxNet’s generalization capability.

As shown in Additional file 1: Fig. S10, BiToxNet achieved strong performance on the independent imbalanced dataset, with an AUC of 0.970, average precision (AP) of 0.870, accuracy of 90.2%, MCC of 0.730, sensitivity of 0.956, and specificity of 0.892. Compared with the balanced combined test set, only a slight decline in AUC and accuracy was observed, demonstrating that BiToxNet maintains strong predictive power under class imbalance. The high AP and MCC values further confirm that BiToxNet effectively handles imbalance while preserving discrimination between neurotoxic and non-toxic sequences, supporting its applicability to realistic neurotoxin screening and temporal generalization scenarios.

Conclusions

In this study, we developed BiToxNet, a deep learning framework that integrates evolutionary embeddings from ESM2 with handcrafted biochemical descriptors via a bilinear attention mechanism. This design enables precise cross-modal interaction modeling, leading to consistent and significant performance improvements across Peptide, Protein, and Combined datasets. Extensive benchmarking, ablation studies, and visualization analyses demonstrate the model’s robustness and superior generalization compared with state-of-the-art methods. BiToxNet demonstrates strong performance for neurotoxin identification and represents a promising multimodal fusion framework. It also holds strong potential for broader applications in biological sequence analysis. To more fully establish its generalizability, future work will evaluate the transferability of BiToxNet to additional bioinformatics prediction problems, such as the prediction of sumoylation sites [26, 27] and RNA 5-methyluridine modification prediction [28], as well as other protein function prediction tasks.

Nevertheless, additional experiments revealed that cross-type transfer performance remained limited: training on Peptide dataset and testing on Protein dataset achieved an AUC of 0.641, while the reverse setup yielded 0.736. This suggests that BiToxNet primarily learns type-specific rather than type-invariant representations, likely due to inherent differences in amino acid composition and physicochemical properties between peptides and proteins. For instance, cysteine content and sulfur-containing residue composition (e.g., PCP_SC) show strong discriminative power in the peptide dataset but weaker signals in proteins [16, 29], impeding transferable feature learning. These findings delineate the current model’s applicability scope and motivate future efforts to achieve true cross-type transfer, potentially through domain adaptation, contrastive feature alignment, or hierarchical shared-specific representation modeling [30–32].

Methods

Dataset preparation

The development of a robust and generalizable model for neurotoxicity prediction critically depends on a high-quality dataset. In this study, we adopted the same datasets proposed by the NTxPred2 to ensure comparability. Experimentally validated neurotoxic and non-toxic sequences were systematically retrieved from the Swiss-Prot database [33]. For the peptide dataset, sequences $<$51 amino acids were selected, rigorously filtered to remove non-canonical amino acids and redundancy, resulting in 877 unique neurotoxic peptides. To achieve class balance and length distribution matching, non-toxic peptides were clustered using CD-HIT [34] at a 60% sequence identity threshold, yielding an equivalent set of 877 non-neurotoxic peptides. Similarly, for the protein dataset (sequences $\ge$51 amino acids), stringent filtering and application of CD-HIT at a 40% sequence identity threshold yielded 775 non-redundant neurotoxic proteins, paired with 775 carefully selected non-toxic proteins of comparable lengths. Additionally, a combined dataset was formed by strategically merging the Peptide and Protein datasets to evaluate cross-molecular type generalization. To guarantee unbiased evaluation and prevent data leakage, all three datasets were rigorously partitioned using an 8:2 split, allocating 80% for model training and reserving 20% as a strictly independent test set. The precise sizes of the training and test sets for each dataset configuration are summarized in Table 2.

Table 2.

Overview of the datasets used in this study

Dataset	Training data			Independent testing data
	Neurotoxic	Non-toxic	Total	Neurotoxic	Non-toxic	Total
Peptide	701	702	1403	176	175	351
Protein	620	620	1240	155	155	310
Combined	1321	1322	2643	331	330	661

Overall architecture of BiToxNet

As illustrated in Fig. 7, the BiToxNet framework employs a dual-stream feature extraction approach followed by a BAN for neurotoxin prediction. In the feature extraction module, two parallel pathways process input sequences: the first stream utilizes ESM2 to generate evolutionary embeddings, which are transformed through a fully connected (FC) layer into a 256-dimensional feature vector. The second stream computes ten handcrafted biochemical descriptors including ACC, DPC, pseudo-amino acid composition (PAAC), composition of k-spaced amino acid pairs (CKSAAGP), BLOSUM62 encoding, Z-scale properties, physicochemical properties (PHYC), dipeptide deviation from expected mean (DDE), quasi-sequence-order descriptors (QSOrder), and the PseAAC of distance-pair and reduced alphabet (DistancePair). These descriptors are combined into a comprehensive feature set, which undergoes dimensionality reduction via recursive feature elimination (RFE) [35]. The reduced features are then projected through an FC layer into a 256-dimensional vector.

Fig. 7.

Schematic overview of the proposed framework for neurotoxin prediction. The workflow comprises four core modules: A Dataset construction; B Feature extraction; C Bilinear attention network; D Output layer

These aligned 256-dimensional representations are fed into the BAN, the core module of BiToxNet. The BAN dynamically models cross-modal interactions through bilinear pooling, computing outer products between evolutionary and biochemical feature streams to capture residue-level spatial dependencies [36]. A multi-head mechanism further enriches this process by learning diverse interaction patterns across feature subspaces, generating a high-order joint representation. Finally, the output layer processes this BAN-derived joint representation through a multilayer perceptron (MLP) [37] with Gaussian error linear unit (GELU) [38] activation in the hidden layer, followed by a sigmoid function to produce the final probability of neurotoxicity. This integrated design enables BiToxNet to holistically encode sequence-function relationships.

Sequence representation based on a protein language model

In this study, we employed ESM2 [39] to extract sequence features, a transformer-based protein language model pre-trained using masked language modeling objectives on the UniRef50 dataset. Built on the BERT framework, ESM2 captures long-range dependencies between amino acid residues through its self-attention mechanism, which is the core component of transformer architectures. This self-attention mechanism enables the model to dynamically weigh the importance of different residues within a sequence when constructing contextual representations. For an input sequence of length L, ESM2 generates a residue embedding matrix $\mathbf{X}\in {\mathbb{R}}^{L\times d_{\text{esm2}}}$, where $d_{\text{esm2}}$ denotes the embedding dimension. The self-attention mechanism computes context-aware representations as follows:

$$\begin{array}{c}\hfill \text{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\text{softmax}\left(\frac{\mathbf{Q}{\mathbf{K}}^T}{\sqrt{d_k}}\right)\mathbf{V}.\end{array}$$ 1

Here, $\mathbf{Q}=\mathbf{X}{\mathbf{W}}_Q$, $\mathbf{K}=\mathbf{X}{\mathbf{W}}_K$, and $\mathbf{V}=\mathbf{X}{\mathbf{W}}_V$ are linearly projected queries, keys, and values, respectively, with learnable weight matrices ${\mathbf{W}}_Q,{\mathbf{W}}_K,{\mathbf{W}}_V\in {\mathbb{R}}^{d_{\text{esm2}}\times d_k}$. The scaling factor $\sqrt{d_k}$ stabilizes gradient dynamics during training.

The model provides multiple architectural variants with different depths and parameter sizes. In this study, we utilized the esm2_t30_150M_UR50D variant with 30 transformer layers, $d_{\text{esm2}}=640$, and approximately 150 million parameters. The transformer architecture outputs residue-level embeddings ${\mathbf{H}}_{\text{esm2}}\in {\mathbb{R}}^{L\times 640}$. To ensure dimensional compatibility with the biochemical feature stream for subsequent bilinear attention fusion, we then apply a FC layer for dimensionality reduction:

$$\begin{array}{c}\hfill {\mathbf{H}}_{\text{evo}}=\text{GELU}({\mathbf{H}}_{\text{esm2}}{\mathbf{W}}_{\text{fc}}+{\mathbf{b}}_{\text{fc}}),\end{array}$$ 2

where ${\mathbf{W}}_{\text{fc}}$ and ${\mathbf{b}}_{\text{fc}}$ are learnable parameters. This projects embeddings into a unified 256-dimensional space, yielding ${\mathbf{H}}_{\text{evo}}\in {\mathbb{R}}^{L\times 256}$. The reduced dimensionality enables efficient cross-modal interaction modeling while preserving critical evolutionary information.

Sequence representation based on several protein descriptors

To complement the evolutionary information captured by ESM2, we computed ten handcrafted descriptors that encode fundamental compositional, structural, and physicochemical properties of peptide and protein sequences. These traditional descriptors provide orthogonal information to the deep learning embeddings, capturing explicit biochemical constraints that govern neurotoxin function. Table 3 summarizes the ten biochemical descriptors implemented in BiToxNet. Detailed descriptions of these protein descriptors can be found in the Additional file 1. Each descriptor quantifies distinct protein attributes, collectively providing a multidimensional representation of sequence characteristics critical for neurotoxin activity.

Table 3.

Summary of handcrafted protein descriptors used in this study

Descriptors	Description	Dimension	References
AAC	Quantifies relative frequency of 20 standard amino acids	20	[40]
DPC	Computes occurrence frequency of 400 possible dipeptides	400	[41]
PAAC	Incorporates sequence-order information through pseudo-composition	24	[42]
CKSAAGP	Calculates composition of amino acid pairs separated by k residues	100	[43]
BLOSUM62	Encodes evolutionary conservation using substitution matrix scores	$L\times 20$	[44]
Z-scale	Represents normalized biophysical attributes as 5-dimensional vectors per residue	$L\times 5$	[45]
PHYC	Describes physicochemical characteristics including isoelectric point, net charge, hydrophobicity, etc	8	[46]
DDE	Measures deviation of dipeptide frequency from expected mean	400	[47]
QSOrder	Integrates compositional statistics with positional information for local and global features	46	[48]
DistancePair	Quantifies pairwise distances between amino acids grouped via a reduced alphabet	20	[49]

L: Sequence length of the input protein or peptide

The initial concatenation of these ten descriptors resulted in a high-dimensional feature vector. To mitigate potential overfitting and eliminate redundant or low-importance features while retaining the most predictive biochemical attributes relevant to neurotoxicity, we applied RFE. RFE is a backward selection algorithm that iteratively removes the least important features, optimizing for maximum classification accuracy through cross-validation (CV). This process yielded a refined, discriminative set of biochemical features for subsequent processing. The refined biochemical features were then projected into a 256-dimensional space matching the evolutionary embedding dimensions through a FC layer with GELU activation:

$$\begin{array}{c}\hfill {\mathbf{H}}_{\text{bio}}=\text{GELU}({\mathbf{H}}_{\text{rfe}}{\mathbf{W}}_{\text{fc}}+{\mathbf{b}}_{\text{fc}}),\end{array}$$ 3

where ${\mathbf{H}}_{\text{rfe}}$ denotes the RFE-selected features, ${\mathbf{W}}_{\text{fc}}$ and ${\mathbf{b}}_{\text{fc}}$ are learnable parameters. This uniform dimensionality between biochemical and evolutionary representations allows BAN to effectively model their synergistic relationships, preserving toxin-specific biochemical characteristics essential for accurate prediction and classification.

Bilinear attention network for cross-modal feature interaction

To model residue-level dependencies between evolutionary constraints and biochemical properties essential for neurotoxicity, BiToxNet employs a BAN [50]. This module dynamically captures interactions between evolutionary embeddings derived from ESM2 (${\mathbf{H}}_{\mathit{evo}}\in {\mathbb{R}}^{L\times 256}$) and handcrafted biochemical descriptors (${\mathbf{H}}_{\mathit{bio}}\in {\mathbb{R}}^{L\times 256}$), where L denotes the sequence length. The core mechanism constructs a joint representation $\mathbf{J}$ through bilinear feature fusion:

$$\begin{array}{c}\hfill \mathbf{J}=\text{Norm}\left({\mathbf{W}}_I,\left(\text{GELU},({\mathbf{U}}^T{\mathbf{H}}_{\mathit{evo}}),\otimes ,\text{GELU},({\mathbf{V}}^T{\mathbf{H}}_{\mathit{bio}})\right),+,{\mathbf{b}}_I\right).\end{array}$$ 4

Here, $\mathbf{U}\in {\mathbb{R}}^{256\times k}$ and $\mathbf{V}\in {\mathbb{R}}^{256\times k}$ are learnable projection matrices mapping both modalities into a shared k-dimensional latent subspace. The GELU activation introduces nonlinear transformations, while the outer product $\otimes$ computes pairwise interactions between projected residue features, generating high-order interaction tensors. The weight matrix ${\mathbf{W}}_I\in {\mathbb{R}}^{k\times o}$ and bias ${\mathbf{b}}_I$ linearly project this tensor into an output feature space of dimension o, with weight normalization (Norm) applied for optimization stability.

The BAN operates through a multi-head architecture where feature projections are first computed across parallel heads. Subsequently, bilinear pooling calculates outer products between aligned residue representations to explicitly capture spatial dependencies such as those between conserved cysteine residues and physicochemical properties. Finally, the outputs from all heads are aggregated into a joint representation $\mathbf{J}$. This design elucidates how biochemical constraints (e.g., disulfide-bonded framework [51]) modulate evolutionarily conserved motifs critical for neurotoxins targeting ion channels [9], while maintaining computational efficiency comparable to standard attention mechanisms. The learned interaction weights further provide interpretable insights into residue-specific determinants of neurotoxicity.

The resulting joint representation $\mathbf{J}$ is then processed by MLP for the final neurotoxin prediction. The MLP computes the probability $\widehat{y}$ of a sequence being neurotoxic as follows:

$$\begin{array}{cc}\hfill {\mathbf{h}}_1& =\text{GELU}({\mathbf{W}}_1\mathbf{J}+{\mathbf{b}}_1)\hfill \end{array},$$ 5

$$\begin{array}{cc}\hfill \widehat{y}& =\sigma ({\mathbf{W}}_2{\mathbf{h}}_1+{\mathbf{b}}_2)\hfill \end{array},$$ 6

where $\sigma (·)$ denotes the sigmoid activation function that maps the output to the range [0, 1], and ${\mathbf{W}}_1$, ${\mathbf{b}}_1$, ${\mathbf{W}}_2$, and ${\mathbf{b}}_2$ are learnable parameters, with ${\mathbf{h}}_1$ representing the hidden layer output.

Performance evaluation

To comprehensively evaluate the predictive performance of proposed models, we employed five widely accepted metrics: Accuracy, Sensitivity, Specificity, Matthews correlation coefficient (MCC) [52], and the area under the receiver operating characteristic (ROC) Curve (AUC) [53]. These metrics were calculated based on the fundamental classification outcomes: true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). The formulations of these metrics are defined as follows:

$$\begin{array}{cc}\hfill \text{Accuracy}& =\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}\hfill \end{array},$$ 7

$$\begin{array}{cc}\hfill \text{Sensitivity}& =\frac{\text{TP}}{\text{TP+FN}}\hfill \end{array},$$ 8

$$\begin{array}{cc}\hfill \text{Specificity}& =\frac{\text{TN}}{\text{TN+FP}}\hfill \end{array},$$ 9

$$\begin{array}{cc}\hfill \text{MCC}& =\frac{\text{TP}\times \text{TN}-\text{FP}\times \text{FN}}{\sqrt{\text{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}}\hfill \end{array}.$$ 10

AUC represents the area under the ROC curve plotting true positive rate (TPR) against false positive rate (FPR) across classification thresholds. Among these metrics, accuracy, sensitivity, and specificity range from 0 to 1, quantifying the overall correctness of predictions, the model’s capability to detect positive instances, and its ability to recognize negative instances, respectively. MCC provides a balanced assessment particularly valuable for imbalanced datasets, ranging from −1 to 1 where 1 indicates perfect prediction, 0 random guessing, and −1 inverse prediction. AUC offers a comprehensive threshold-independent evaluation of class discrimination capability, where 1 represents perfect separability and 0.5 indicates no discriminative power. Higher values across all five metrics signify superior model performance in their respective measured aspects. Notably, all threshold–based metrics are computed at a prespecified constant decision threshold $\tau$ applied to the predicted probability $\widehat{y}$. A sample is predicted as positive if $\widehat{y}>\tau$ and negative otherwise. The threshold is fixed within a dataset to enable fair comparisons with other classifiers and to avoid any influence from the test set during model development. We use $\tau =0.55$ for the Protein dataset, $\tau =0.45$ for the Peptide dataset, and $\tau =0.50$ for the Combined dataset.

Implementation details

BiToxNet is implemented in PyTorch [54], using the Adam optimizer [55] with an initial learning rate of $4\times {10}^{-4}$ and an early-stopping [56] schedule that monitors validation AUC and halts after 20 consecutive non-improving epochs, with a maximum of 100 epochs.

The loss function for binary classification is binary cross entropy (BCELoss). For a dataset of size N with ground–truth labels $y_i\in \{0,1\}$ and model–predicted probability ${\widehat{y}}_i\in [0,1]$ of the positive class, BCELoss is defined as

$$\begin{array}{c}\hfill \text{BCELoss}=-\frac{1}{N}\sum _{i=1}^N\left[y_i,·,log,({\widehat{y}}_i),+,(1-y_i),·,log,\left(1,-,{\widehat{y}}_i\right)\right]\end{array},$$ 11

where $log$ denotes the natural logarithm.

Model development adopts a nested five-fold CV scheme: five outer folds provide performance estimation, and, within the training portion of each outer fold, a fivefold inner CV is used exclusively for estimator and feature selection via hyperparameter tuning and RFE. In every fold, all preprocessing steps (e.g., standardization and feature selection) are fit on the training split only and then applied to the matched validation split; neither outer-validation nor test samples are ever used for fitting, thereby avoiding information leakage and “double dipping.” The model is trained on the fold-specific training data, and the epoch with the best validation AUC within that fold is selected without any access to the test set. After CV, we report outer-CV performance and evaluate once on the independent held-out test set.

Abbreviations

AAC

Amino acid composition

AP

Average precision

AUC

Area under the receiver operating characteristic curve

BAN

Bilinear attention network

BCELoss

Binary cross entropy loss

CKSAAGP

Composition of K-spaced amino acid pairs

CV

Cross-validation

DDE

Dipeptide deviation from expected mean

DistancePair

The PseAAC of distance-pair and reduced alphabet

DPC

Dipeptide composition

ESM2

Evolutionary Scale Modeling 2

FC

Fully connected

FN

False negative

FP

False positive

FPR

False positive rate

GELU

Gaussian error linear unit

LLM

Large language model

MCC

Matthews correlation coefficient

MLP

Multilayer perceptron

nAChRs

Nicotinic acetylcholine receptors

PAAC

Pseudo-amino acid composition

PHYC

Physicochemical properties

PSI-BLAST

Position-Specific Iterated BLAST

PR Curve

Precision–recall curve

PSSM

Position-Specific Scoring Matrix

QSOrder

Quasi-Sequence Order Descriptors

RFE

Recursive feature elimination

ROC

Receiver operating characteristic

SVM

Support vector machine

TN

True negative

TP

True positive

TPR

True positive rate

t-SNE

t-distributed Stochastic Neighbor Embedding

Authors’ contributions

L.T.Y. and F.W. presented the idea. F.W., P.L.X., X.Q.L., and J.H.G. implemented the framework. T.-Y.L. collected the data. F.W., J.H.G., C.L., and X.X.H. analyzed the results. L.T.Y., X.R.L., and L.Y.W. supervised the research project. All authors reviewed the manuscript.

Funding

This work was supported by Shenzhen Science and Technology Innovation Commission (JCYJ20230807114206014), and the Kobilka Institute of Innovative Drug Discovery, The Chinese University of Hong Kong, Shenzhen, China. This work was also financially supported by the Center for Intelligent Drug Systems and Smart Biodevices (IDS2B) from The Featured Areas Research Center Program within the framework of the Higher Education Sprout Project and Yushan Young Fellow Program (113C51N055) by the Ministry of Education (MOE) and National Science and Technology Council (NSTC 113-2321-B-A49-025, 113-2634-F-039-001, 113-2221-E-A49-160-MY3 and 112-2740-B-400-005) in Taiwan and The National Health Research Institutes (NHRI-EX114-11320BI) in Taiwan. This work was also supported by the Fundamental Research Funds for the Central Universities (20720250172).

Data availability

All data generated or analyzed during this study are included in this published article, its supplementary information files and publicly available repositories: https://github.com/Feng106-w/BiToxNet and 10.5281/zenodo.17973979 [57].

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.