Improving hERG Cardiotoxicity Prediction via SMILES Transformer, Molecular Fingerprints, and Layer-Wise Self-Adaptive Graph Attention Network

Abstract

Blocking of hERG (human ether-à-go-go-related gene) potassium channels by certain drug-like molecules remains a major obstacle in pharmaceutical development, as it can induce QT interval prolongation and serious cardiotoxic effects. Experimental assessment of hERG liability is accurate but costly and time-consuming, which motivates the development of reliable computational screening tools. In this work, we propose TDMFLSGAT (transformer-enhanced deep learning model with molecular fingerprints and layer-wise self-adaptive graph attention network), a multimodal architecture that integrates three complementary molecular representations. Sequential information from SMILES strings is encoded with a Transformer, graph-based structural features are extracted using an adaptive graph attention mechanism, and physicochemical patterns are captured through a diverse set of molecular fingerprints. These modalities are fused into a unified representation for robust classification. To enhance model transparency, we further incorporate a multimodal interpretability framework that combines attention-based analyses and fingerprint-level explanations to highlight structural motifs associated with hERG blockade. Under 5-fold cross-validation, TDMFLSGAT achieves strong and well-balanced predictive performance, with an accuracy of 0.823, an AUC of 0.901, an average precision (AP) of 0.915, a sensitivity of 0.850, a specificity of 0.792, an NPV of 0.810, a PPV of 0.834, and an MCC of 0.641. The inclusion of AP confirms that these performance gains remain robust under class imbalance, providing a more realistic assessment of model reliability. Overall, these results indicate that TDMFLSGAT offers both accurate predictions and meaningful mechanistic insights, making it a promising tool for early stage cardiotoxicity screening in drug discovery.

1. Introduction

Cardiotoxicity is a critical adverse effect in pharmacology, referring to structural or functional damage to the heart muscle caused by therapeutic agents, environmental toxins, or other external factors. It can manifest as alterations in cardiac electrophysiology, contractility, or morphology, and may lead to severe outcomes such as arrhythmias, heart failure, or sudden cardiac death. Among the various mechanisms underlying drug-induced cardiotoxicity, the inhibition of ion channels responsible for maintaining cardiac action potential stability is one of the most clinically relevant.

A key molecular player in this context is the human ether-à-go-go-related gene (hERG), which encodes a specific class of voltage-gated potassium channels that play an essential role in maintaining the electrical stability of cardiac activity. These channels, part of the ether-à-go-go potassium channel subfamily, possess unique structural properties and are primarily involved in mediating repolarization currents during cardiac action potentials. hERG channels are predominantly found in the heart and brain, where they are critical to proper action potential repolarization processes. When these channels are inhibited or impaired by certain pharmacological compounds, they cause a delay in ventricular repolarization, clinically observed as QT interval prolongation on the electrocardiogram (ECG). This electrical disturbance increases the risk of malignant ventricular arrhythmias, such as torsades de pointes, and can ultimately result in sudden cardiac deatha phenomenon collectively referred to as hERG-induced cardiotoxicity. ,

Because of this, the unintended blockage of hERG channels is considered a critical challenge in the pharmaceutical industry. Many drug candidates have been abandoned in advanced clinical stages for this reason. , Moreover, several marketed drugs such as astemizole, terfenadine, and cisapride have been withdrawn due to their proven links to hERG-associated cardiotoxic effects. This context has made the evaluation of hERG channel blocking potential a crucial phase in the drug discovery pipeline. To mitigate the risk of introducing cardiotoxic drugs into the market, regulatory bodies such as the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) mandate early stage assessments. , These evaluations typically involve a combination of in vitro assayssuch as binding affinity tests and patch-clamp electrophysiologyand in vivo measurements of the QTc interval. ,

Given the growing number of candidate moleculesover 10,000 small compounds under evaluation in the early phases of drug developmentconducting such biological evaluations for every compound remains both time-consuming and resource-intensive. As a response, artificial intelligence, especially deep learning, has become a promising tool for accelerating predictive toxicology. These data-driven approaches have already achieved notable success in multiple bioinformatics tasks, such as predicting miRNA–lncRNA interactions, miRNA–disease relationships, associations between metabolites and diseases, and circRNA–disease interactions. , These methodological advancements have paved the way for the development of machine learning-based tools that aim to detect hERG blockers more effectively during the early screening phase. Most existing models rely on ligand-based learning algorithms, such as random forest (RF), support vector machines (SVM), deep neural networks (DNN), and graph convolutional networks (GCNs). In parallel, structure-based strategies like pharmacophore modeling, quantitative structure–activity relationship (QSAR) analysis, and molecular docking are also widely mployed. For example, Wang et al. (2012) constructed a series of classification models that integrated recursive partitioning and Bayesian classifiers, using both molecular properties and fingerprints. Their findings pinpointed certain substructures that significantly contribute to hERG inhibition risk. Building upon this, Zhang et al. (2016) applied machine learning techniques on molecular descriptors to detect cardiotoxic structural motifs. Later work by Wang incorporated pharmacophore techniques alongside ML to enhance prediction performance. Cai et al. proposed deephERG, a deep learning model fusing molecular vectors and physicochemical descriptors, although it showed lower NPV scores in internal validation. In 2020, Ryu et al. introduced DeepHIT, which combined molecular fingerprints and graph structures within a deep neural framework to predict hERG blockers accurately. Liu et al. curated a data set from ChEMBL (1865 compounds) and Zhang et al.’s training data, using it to evaluate their classifiers on multiple external benchmarks. They applied three distinct ML algorithms based on nine molecular fingerprint types and descriptor categories. However, many of these methods relied on a single type of molecular representation, which limited their robustness. Some later works attempted feature fusion, but often lacked sophisticated integration strategies to fully exploit multimodal molecular data. Typically, hERG classification data sets encode molecular structures using SMILES (Simplified Molecular Input Line Entry System) stringsa linear notation using ASCII characters to describe atoms and bonds. While interpretable, SMILES are low-level representations that require preprocessing. One common strategy involves transforming them into binary feature vectors called molecular fingerprints. More complex representationssuch as 2D and 3D descriptorscan also be extracted and used in machine learning workflows,. , Another avenue is to convert molecules into graphs, where atoms are nodes and bonds are edges. Such representations are well-suited for graph-based models like GCNs, which can integrate various molecular features into a unified format that better captures chemical semantics and interatomic relationships. ,

In 2023, Wang, Sun, and Zhao introduced a deep learning framework known as DMFGAM (Deep learning model with molecular fingerprints and graph attention mechanism). This model combines molecular fingerprints with graph-based molecular representations and employs a multihead graph attention network (GAT) to classify compounds as hERG blockers or nonblockers. DMFGAM reports strong performance in both cross-validation and external evaluation, marking an important step toward more effective cardiotoxicity prediction.

However, several limitations remain. The model primarily relies on molecular fingerprints and basic descriptors, which restrict the diversity and depth of the chemical information captured. In addition, DMFGAM does not incorporate the sequential and contextual information embedded in SMILES strings, which is crucial for modeling internal structural dependencies. Another limitation lies in the use of the classical GAT architecture for graph processing, which is known to suffer from oversmoothing as more layers are stacked. As node representations gradually become indistinguishable across layers, the model may lose its ability to preserve important local structural differences, reducing the expressiveness of graph-based chemical features.

To address these shortcomings, we developed a new architecture called TDMFLSGAT (transformer-enhanced deep learning model with molecular fingerprints and graph attention mechanism), which introduces several key innovations. First, we replaced the traditional GAT with a Layer-wise Self-Adaptive Graph Attention Network (LSGAT). This improved architecture adjusts node contributions dynamically at each layer. When certain nodes become overly connected and risk dominating feature aggregation, LSGAT automatically reduces their influence. This mechanism preserves local structural information across layers and prevents oversmoothing, maintaining distinctive node representations even in deep networks. In addition, TDMFLSGAT incorporates a Transformer encoder designed specifically to process SMILES sequences. Unlike static encodings used in DMFGAM, this encoder extracts context-rich sequential features, capturing complex dependencies within molecular structures. The model also benefits from a broader range of molecular fingerprints, including five distinct typesamong them topological torsion and Klekota–Roth (KR) fingerprints. These more interpretable descriptors replace the hashed fingerprints used in DMFGAM, providing clearer insights into the structural motifs linked to hERG channel inhibition. Finally, TDMFLSGAT’s hyperparameters are optimized automatically using the Hyperopt framework, ensuring efficient exploration of the parameter space and improved generalization. Together, these improvementsLSGAT, the Transformer encoder, diversified fingerprints, and advanced optimizationenable TDMFLSGAT to outperform DMFGAM in terms of both prediction accuracy and model stability. This architecture thus emerges as a reliable and effective tool for early cardiotoxicity screening, contributing to safer and faster pharmaceutical development.

The remainder of this paper is organized as follows. The Materials and methods section describes the data set, the architecture of the proposed TDMFLSGAT model, and the molecular featurization strategies. The Results section presents the experimental evaluation, including performance assessment, comparisons with baseline methods, external validation, and ablation studies. Finally, the discussion and conclusion section interprets the main findings, discusses the limitations of the study, and outlines potential directions for future research.

2. Materials and Methods

2.1. Data Preparation

In this study, we used the same data set of 10,355 compounds with hERG channel blocking activity, originally compiled and used by Wang et al. These data come from multiple sources, including the ChEMBL v29 database (https://www.ebi.ac.uk/chembl/), as well as literature-derived data from Konda et al., Liu et al., Munawar et al., and Negami et al.

Following the methodology described in, the chemical structures of the compounds were standardized using Python libraries RDKit (http://www.rdkit.org) and MolVS (https://github.com/mcs07/MolVS), ensuring consistency in molecular representations. The standardization process includes selecting the largest molecular fragment, removing explicit hydrogens, ionization, and stereochemistry calculation.

For classification, we applied the same pIC50-based criteria as in, where compounds with pIC₅₀ < 5 μM are considered hERG blockers, while those with pIC₅₀ ≥ 5 μM are classified as hERG nonblockers.

The final data set consisted of 4479 blocker compounds and 5876 nonblockers, and was divided into 80% for training and 20% for testing, in accordance with the procedure outlined in.

2.2. Model Architecture

In this work, we designed a model named TDMFLSGAT, which integrates three complementary molecular representations: SMILES sequences, molecular fingerprints, and molecular graphs. The overall structure of the model is illustrated in Figure .

1.

Architecture of the TDMFLSGAT model integrating a SMILES transformer, molecular fingerprints, and a Layer-wise Self-Adaptive Graph Attention Network (LSGAT). The model combines sequential, structural, and physicochemical molecular representations for cardiotoxicity prediction.

The first branch of the model processes the SMILES sequences. Each molecule is first tokenized using a Byte-Level BPE tokenizer, which segments the chemical sequence into substructure-level tokens. These tokens are then transformed into dense vectors, enriched with positional encodings to preserve sequential information. We use a multilayer Transformer encoder to process these embeddings, as it can capture complex sequential dependencies inherent in the molecular structure. Finally, the sequence output is summarized via mean pooling to produce a compact representation of the molecule.

In parallel, a second branch extracts molecular fingerprints. We computed several types of fingerprints (ECFP, PubChem, KRFP, APC2D, and Topological Torsion), each encoding different structural and physicochemical features of the molecule. These descriptors complement the sequence-based features and help improve the model’s ability to distinguish between hERG blockers and nonblockers.

For the third branch, we represent each molecule as a graph, where atoms are treated as nodes and chemical bonds as edges. To process these molecular graphs, we deliberately replaced the traditional graph attention network (GAT) with a more advanced approach: the deep Layer-wise Self-Adaptive Graph Attention Network (LSGAT). This architecture applies several successive LSGAT layers, each operating with multihead attention mechanisms. At every layer, node messages are aggregated while applying an adaptive scaling mechanism: nodes that become overly connected have their influence automatically reduced, based on their overlap degree. This prevents the oversmoothing effect and preserves local structural information, even in deep architectures. After processing through all layers and attention heads, node features are concatenated, flattened, and projected through a dense layer to obtain a comprehensive graph-level representation.

Finally, we fused the outputs from the three branchesSMILES Transformer, molecular fingerprints, and graph featuresinto a single unified feature vector. This combined representation is then passed through a fully connected neural network, responsible for producing the final prediction: classifying each molecule as either a potential hERG blocker or a nonblocker.

2.3. Featurization Stage

In our data set, chemical structures are encoded using SMILES (Simplified Molecular Input Line Entry System) strings. To ensure our model can effectively recognize molecular structures and predict hERG inhibition, it is essential to transform these molecular representations into rich feature vectors. At this stage, we extract three distinct types of features for compound molecule, namely Transformer-derived features, molecular fingerprint features, and molecular graph features. Below, we provide a detailed explanation of how these features are generated and integrated into our model.

2.3.1. Molecular Fingerprint Features

In this study, five types of molecular fingerprints are computed to represent the structural properties of each compound. These fingerprints include Extended Connectivity Fingerprints (ECFP), PubChem fingerprints, AtomPairs2DFingerprintCount (APC2D), Klekota–Roth Fingerprints (KRFP), and Topological Torsion fingerprints. ECFP, also known as circular fingerprints, are specifically designed to capture the molecular substructure by iteratively expanding the atomic neighborhoods, which is particularly useful for structure–activity relationship modeling. PubChem fingerprints, APC2D fingerprints, and KRFP focus on encoding structural patterns and functional groups relevant for molecular similarity analysis, while Topological Torsion fingerprints encode sequential connectivity patterns associated with torsional angles in molecular conformations. We generated all fingerprints using PaDEL-Descriptor (version 2.21) and RDKit toolkits. In total, 1024 ECFP fingerprints, 881 PubChem fingerprints, 780 APC2D fingerprints, 4860 KRFP fingerprints, and 1024 Topological Torsion fingerprints were calculated for each compound. The key characteristics of the fingerprints used in this study are summarized in Table .To optimize fingerprint representation and reduce dimensionality, a multilayer perceptron (MLP) is applied to each fingerprint vector. Each fingerprint is projected into a 128-dimensional latent space, ensuring efficient information retention while reducing computational complexity. As defined in eq , this transformation applies two fully connected layers with ReLU activations to extract the most informative molecular features.

$${\mathbf{Z}}_i=\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}(W_2·\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}(W_1·{\mathbf{X}}_i+{\mathbf{b}}_1)+{\mathbf{b}}_2)$$ 1

where X _i represents the fingerprint vector of molecule i; W ₁ and W ₂ are weight matrices; b ₁ and b ₂ are bias terms; and ReLU denotes the rectified linear unit activation function. This nonlinear transformation enhances the model’s capacity to extract the most informative molecular features.

1. Name, Length, and Type of Molecular Fingerprints.

molecular fingerprint	bits	type
ECFP	1024	circular fingerprints
PubChem	881	structural features
APC2D	780	structural features
KRFP	4860	functional group-based fingerprints
topological torsion	1024	torsional connectivity

2.3.2. Molecular Graph Features

To accurately represent the structural properties and interactions of molecules, we employ molecular graph representations. Each molecule is depicted as a graph, where atoms are represented as nodes and chemical bonds This method draws inspiration from the approach described by Wang et al. (2023).!N This representation relies on two primary components:

Adjacency matrix (A _N×N ): encodes the connectivity between atom pairs. If a bond exists between atoms i and j, then A _ij = 1; otherwise, A _ij = 0.

Node feature matrix (X _N×F ): describes the atomic properties, including atomic number, degree, number of bonded hydrogens, aromaticity, and stereochemistry. Here, N denotes the number of atoms in the molecule, and F represents the dimensionality of each atom’s feature vector.

2.3.3. SMILES Transformer Features

The SMILES Transformer architecture is designed to process molecular structures expressed as SMILES strings, aiming to extract deep, sequence-dependent molecular features. The procedure involves the following key phases.

2.3.3.1. Tokenization via Byte-Level BPE

Initially, each SMILES string is segmented into tokens using a Byte-Level Byte Pair Encoding (BPE) strategy. This method decomposes the chemical sequence into smaller, recurrent substructures. Every identified token is then associated with a specific integer identifier, which serves as the input for subsequent processing layers.

2.3.3.2. Construction of Input Representations

Each token identifier i _t is projected into a continuous vector space using a learned embedding matrix, as defined in eq

$${\mathbf{e}}_t=\mathbf{E}(i_t)$$ 2

where ${\mathbf{e}}_t\in {\mathbb{R}}^d$ represents the embedding vector for token t.

To incorporate positional context, sinusoidal positional encodings are applied according to eqs and

$$\mathrm{P}\mathrm{E}(t,2i)=\sin \left(\frac{t}{10000^{2i/d}}\right)$$ 3

$$\mathrm{P}\mathrm{E}(t,2i+1)=\cos \left(\frac{t}{10000^{2i/d}}\right)$$ 4

where t denotes the token position and d the dimensionality of the embedding space.

The final token input combines both embedding and positional information, as expressed in eq

$${\mathbf{x}}_t={\mathbf{e}}_t+\mathrm{P}\mathrm{E}(t)$$ 5

2.3.3.3. Sequence Encoding via Transformer Layers

The sequence of token vectors {x _t } is processed through a stack of Transformer encoder layers. Within each layer, the self-attention mechanism computes contextual relationships between all token pairs, as formulated in eq

$$\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(\frac{\mathbf{Q}{\mathbf{K}}^T}{\sqrt{d_k}}\right)\mathbf{V}$$ 6

where the query, key, and value matrices are defined in eq

$$\mathbf{Q}=\mathbf{X}{\mathbf{W}}^Q,\mathbf{K}=\mathbf{X}{\mathbf{W}}^K,\mathbf{V}=\mathbf{X}{\mathbf{W}}^V$$ 7

This mechanism enables the model to capture both short- and long-range dependencies inherent in molecular sequences.

2.3.3.4. Aggregation of Sequence Features

After processing through all Transformer layers, a mean pooling operation condenses the sequence information into a single vector, as defined in eq

$$\mathbf{h}=\frac{1}{T}\sum _{t=1}^T{\mathbf{x}}_t^{(L)}$$ 8

where x _t denotes the output representation at token position t after the final (L ^th) Transformer layer, and T is the sequence length.

The resulting vector h provides a dense, global representation of the molecule, effectively summarizing both its sequential and structural properties. These Transformer-derived features are subsequently combined with graph-based and fingerprint-based representations to enrich the molecular characterization used in the classification tasks.

It is important to emphasize that the SMILES module in TDMFLSGAT uses only the encoder part of the Transformer architecture. The encoder is employed as a contextual embedding module for SMILES sequences, and no decoder or sequence-to-sequence component is involved. We also deliberately chose not to rely on large-scale pretrained chemical language models such as SMILES-BERT, ChemBERTa, or MolBERT. These models use tokenization schemes and vocabulary definitions that are not directly compatible with our Byte-Level BPE tokenizer, and preliminary experiments with pretrained checkpoints led to unstable training and no consistent performance improvement on the present hERG data set. Training the Transformer encoder from scratch with a tokenizer and sequence-length configuration tailored to this data set provided more stable convergence and competitive performance.

2.4. Layer-Wise Self-Adaptive Graph Attention Network (LSGAT)

The Layer-wise Self-Adaptive Graph Attention Network is a neural network architecture designed to process molecular representations in the form of graphs. Inspired by the work of Su et al., this approach introduces a layer-wise adaptive scaling mechanism that dynamically regulates the contribution of nodes based on their connectivity, effectively mitigating the oversmoothing effect often encountered in deep graph networks.

In the context of molecular analysis, graphs are constructed from SMILES sequences where:

• each atom is represented as a node,

• each chemical bond is represented as an edge,

• and the graph’s connectivity is encoded via the adjacency matrix A, augmented with self-loops as defined in eq .

$$Â=A+I$$ 9

where I denotes the identity matrix to incorporate self-connections.

The input to LSGAT consists of atomic node features, as shown in eq

$$s=\{{\mathbf{s}}_1,{\mathbf{s}}_2,...,{\mathbf{s}}_N\},{\mathbf{s}}_i\in {\mathbb{R}}^F$$ 10

where ${\mathbb{R}}^F$ is the feature space of atomic nodes, and N is the total number of nodes. Each node feature is transformed using a shared linear transformation, as given in eq

$${\mathbf{h}}_i=\mathbf{W}{\mathbf{s}}_i$$ 11

where $\mathbf{W}\in {\mathbb{R}}^{F^{\prime }\times F}$ is a learnable weight matrix.

For each layer l, the coverage degree of a node j is computed to quantify how connected node j becomes after l layers of message propagation, as described in eq

$$d_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{(l)}(j)=\sum _{i^{\prime }=1}^N{({Â}^l)}_{i^{\prime }j}$$ 12

A dynamic threshold τ is then computed to regulate the influence of overconnected nodes, following eqs and

$${\eta }^{(\mathrm{l})}=\beta N+(1-\beta )N{e}^{-l}$$ 13

$$\tau =D_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{(l)}({\eta }^{(l)})$$ 14

where β ∈ [0, 1] is a hyperparameter.

Next, the normalized coverage degree is computed as shown in eq

$$D_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{*(l)}(j)=\frac{d_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{(l)}(j)}{\tau }$$ 15

The Layer-wise SCALING Score (LSS) for each node is defined in eq

$$\mathrm{l}\mathrm{s}{\mathrm{s}}_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{(l)}(j)=\{\begin{array}{ll}1& \text{if}{D}_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{*(l)}(j)\le 1\\ \frac{1}{D_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{*(l)}(j)}& \text{if}{D}_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{*(l)}(j)>1\end{array}$$ 16

This factor adaptively scales the attention contribution of nodes.

The attention score between node i and its neighbor j is computed according to eq

$$e_{ij}^{*}=\mathrm{l}\mathrm{s}{\mathrm{s}}_{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}^{(l)}(j)\times \mathrm{L}\mathrm{e}\mathrm{a}\mathrm{k}\mathrm{y}\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}({\mathrm{\delta ⃗}}^T[\mathbf{W}{\mathbf{h}}_i\parallel \mathbf{W}{\mathbf{h}}_j])$$ 17

where.

• $\mathrm{\delta ⃗}\in {\mathbb{R}}^{2F^{\prime }}$ is a learnable weight vector,

• ∥ denotes the concatenation operator.

To make attention scores comparable, a SoftMax normalization is applied as shown in eq

$${\alpha }_{ij}^{*}=\frac{\exp (e_{ij}^{*})}{\sum _{k\in \mathcal{N}(i)}\exp (e_{ik}^{*})}$$ 18

Using the normalized attention coefficients, LSGAT aggregates features according to eq

$${\mathbf{s}}_i^{\prime }=\sigma (\sum _{j\in \mathcal{N}(i)}{\alpha }_{ij}^{*}\mathbf{W}{\mathbf{s}}_j)$$ 19

where σ denotes a nonlinear activation function.

To stabilize learning, a multihead attention mechanism is introduced. Specifically, m independent attention heads perform the above computations in parallel, and their outputs are concatenated, as given in eq

$${\mathrm{s⃗}}_i^{\prime }={\parallel }_{k=1}^m\sigma (\sum _{j\in \mathcal{N}(i)}{({\alpha }_{ij}^{*})}^k{\mathbf{W}}^k{\mathbf{s}}_j)$$ 20

where W ^k and ${({\alpha }_{ij}^{*})}^k$ are specific to each attention head.

In standard graph attention networks (GAT), each layer updates node representations by computing a weighted average of the features of neighboring nodes. When several layers are stacked, this repeated neighborhood averaging is applied over progressively larger l-hop neighborhoods, causing node embeddings to become increasingly similar and eventually converge toward a low-frequency subspace of the graph Laplacian, a phenomenon commonly referred to as oversmoothing. In practice, this effect is particularly pronounced for high-degree “hub” nodes, which appear in many neighborhoods and often receive large attention scores; their features can dominate the message passing process and pull many surrounding nodes toward nearly identical representations.

The layer-wise scaling score lss_over (j) in LSGAT is specifically designed to mitigate this behavior. Instead of uniformly shrinking all attention scores, LSGAT monitors how frequently each node is visited across layers through the coverage degree d _over (j) and compares it with the layer-dependent threshold τ. Nodes whose coverage remains within a typical range (D _over (j) ≤ 1) keep a scaling factor of 1 and are treated as in a classical GAT. Only nodes whose coverage exceeds the threshold (i.e., overvisited hubs with D _over (j) > 1) receive a down-scaling factor lss_over (j) < 1. In other words, LSGAT selectively reduces the influence of a small subset of overconnected nodes at each layer, acting as a layer-wise regularizer on the message passing process.

By preventing a few hubs from dominating the aggregation over many layers, LSGAT slows down the speed at which information diffuses across the graph and helps preserve local structural differences between nodes, thereby alleviating oversmoothing. At the same time, the risk of information loss is controlled in two ways. First, the scaling factor is lower-bounded by 1/D _over (j), so even highly connected nodes can still contribute, albeit with reduced strength. Second, the use of multihead attention and self-loops allows the model to retain important self-information and complementary views of the neighborhood, ensuring that informative signals are not completely suppressed while the impact of overvisited nodes is moderated. In our experiments, this mechanism leads to more discriminative and better balanced molecular graph representations compared to a classical GAT, as reflected by the ablation results reported in Table .

6. Ablation Study Highlighting the Contribution of Each Architectural Component to the Performance of TDMFLSGAT.

variant	ACC	AUC	AP	SPE	SEN	NPV	PPV	MCC
without transformer	0.807	0.888	0.906	0.796	0.817	0.779	0.831	0.612
classic GAT (no LSGAT)	0.811	0.886	0.910	0.807	0.815	0.780	0.839	0.620
TDMFLSGAT (full)	0.823	0.901	0.915	0.792	0.850	0.810	0.834	0.641

2.5. Feature Fusion Mechanism

In the TDMFLSGAT framework, the three molecular information streamsthe SMILES Transformer embedding, the graph representation produced by the LSGAT module, and the five molecular fingerprint vectorsare first mapped onto a common 128-dimensional space. This preliminary transformation is carried out through modality-specific fully connected layers, each followed by Batch Normalization and a ReLU activation, ensuring that all representations share a comparable numerical scale before integration.

Let h _tr denote the 128-dimensional SMILES embedding, h _g the graph-derived vector, and h _fp = [h _fp1, ..., h _fp5] the processed fingerprint features. These components are assembled into a single multimodal descriptor through concatenation, as shown in eq

$$h_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}}=[h_{\mathrm{t}\mathrm{r}}\parallel h_{\mathrm{g}}\parallel h_{\mathrm{f}\mathrm{p}1}\parallel ...\parallel h_{\mathrm{f}\mathrm{p}5}]$$ 21

The merged vector is then refined through a learnable projection layer that applies a linear mapping, followed by Batch Normalization and a nonlinear activation, as expressed in eq

$$h_{\mathrm{f}\mathrm{u}\mathrm{s}\mathrm{e}\mathrm{d}}=\sigma (\mathrm{B}\mathrm{N}(W{h}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}}+b))$$ 22

where W and b are trainable parameters. Rather than simply reducing the dimensionality, this projection stage serves as an implicit adaptive weighting mechanism. During training, the model gradually learns which modalities carry the most informative signals and adjusts their contributions accordingly, without requiring manually predefined weights. This adaptive and data-driven integration strategy is widely used in multimodal molecular modeling, as it enables different sources of information to interact while maintaining a transparent fusion process.

The resulting fused representation h _fused serves as the input to the predictive module described below.

2.6. Fully Connected Neural Network

The fused molecular representation is then processed by a fully connected neural network (FCNN), which transforms it into a compact and discriminative feature vector. The layers of this network progressively reduce the dimensionality while preserving the essential structure of the fused information. Each layer applies the ReLU activation function, defined in eq , which introduces nonlinearity and allows the model to capture complex interactions between features

$$y=\max (0,Wx+b)$$ 23

where W is the weight matrix, x the input feature vector, b the bias term, and y the activated output.

The final layer applies a sigmoid activation to produce a probability between 0 and 1, corresponding to the likelihood that a compound is predicted as a hERG blocker. Model training is performed using the binary cross-entropy loss (BCELoss), expressed in eq

$$\begin{array}{ll}\mathrm{B}\mathrm{C}\mathrm{E}\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}=-\frac{1}{Z}\sum _{i=1}^Z& [y_i\log (p_i)\\ & +(1-y_i)\log (1-p_i)]\end{array}$$ 24

where Z is the number of samples, y _i the true class of the ith instance, and p _i the predicted probability.

To ensure efficient and stable optimization, the model uses the Adam algorithm, which dynamically adjusts learning rates based on gradient statistics. A cosine annealing scheduler is also applied to modulate the learning rate during training, helping the model generalize more effectively.

3. Results

3.1. Performance Evaluation

The predictive performance of the proposed TDMFLSGAT model was assessed using a 5-fold cross-validation protocol. This evaluation strategy allows the model to be trained and tested on multiple data partitions, providing a robust estimate of its generalization ability while reducing the risk of bias associated with a single train–test split.

Several complementary metrics were employed to characterize different aspects of the classification performance. The area under the receiver operating characteristic curve (AUC) was first considered, as it reflects the model’s ability to discriminate between hERG blockers and nonblockers across a wide range of decision thresholds. Higher AUC values indicate stronger separability between the two classes.

To further analyze classification behavior, sensitivity (SEN) and specificity (SPE) were computed. Sensitivity, defined in eq , quantifies the proportion of true hERG blockers that are correctly identified, whereas specificity, given in eq , measures the ability of the model to correctly recognize nonblockers. These two metrics provide complementary insights into the trade-off between false negatives and false positives.

In addition, positive predictive value (PPV) and negative predictive value (NPV) were included to evaluate the reliability of the model’s predictions for each class. PPV, defined in eq , represents the probability that a compound predicted as a blocker is truly active, while NPV, given in eq , reflects the likelihood that a predicted nonblocker is indeed inactive.

Overall classification performance was summarized using accuracy (ACC), defined in eq . However, since hERG data sets are typically imbalanced, accuracy alone may provide a misleading picture. For this reason, the Matthews correlation coefficient (MCC), shown in eq , was also reported. MCC takes into account all four entries of the confusion matrix and provides a more balanced and informative measure, with values ranging from −1 (complete disagreement) to +1 (perfect prediction).

Beyond ROC-based evaluation, model performance was also examined using precision–recall (PR) analysis, which is particularly informative for imbalanced classification problems such as hERG prediction. Under skewed class distributions, ROC curves may appear overly optimiztic, whereas PR curves directly characterize the trade-off between precision and recall and therefore provide a more realistic assessment of a model’s ability to correctly identify hERG blockers. In this framework, precision corresponds to the positive predictive value (PPV), while recall is equivalent to sensitivity (SEN), both of which are defined above.

Figure presents the precision–recall (PR) curves obtained on the test set for the strongest baseline model, hERG-MFFGNN, and the proposed TDMFLSGAT framework. The side-by-side comparison highlights the advantage of the proposed approach under imbalance-aware evaluation, as TDMFLSGAT consistently maintains higher precision over a broad range of recall values.

2.

Precision–recall curves on the test set for (a) the strongest baseline hERG-MFFGNN and (b) the proposed TDMFLSGAT model. Average precision (AP) corresponds to the area under the precision–recall curve (AUPRC).

To summarize PR performance in a single scalar value, we report the average precision (AP), which corresponds to the area under the precision–recall curve (AUPRC) computed from prediction scores. Consistent with standard practice, AP reflects how well the model preserves precision as recall increases and provides a concise indicator of performance under class imbalance (as opposed to thresholded metrics at a single operating point). In our experiments, TDMFLSGAT achieves a higher AP than hERG-MFFGNN (AP = 0.915 vs 0.890), indicating more reliable identification of hERG blockers under imbalance-aware evaluation.

The equations used to compute these evaluation metrics are reported below

$$\mathrm{S}\mathrm{E}\mathrm{N}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}$$ 25

$$\mathrm{S}\mathrm{P}\mathrm{E}=\frac{\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}}$$ 26

$$\mathrm{P}\mathrm{P}\mathrm{V}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}$$ 27

$$\mathrm{N}\mathrm{P}\mathrm{V}=\frac{\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N}}$$ 28

$$A\mathrm{C}\mathrm{C}=\frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}$$ 29

$$\mathrm{M}\mathrm{C}\mathrm{C}=\frac{\mathrm{T}\mathrm{P}·\mathrm{T}\mathrm{N}-\mathrm{F}\mathrm{P}·\mathrm{F}\mathrm{N}}{\sqrt{(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P})(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N})(\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P})(\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N})}}$$ 30

$$\mathrm{A}\mathrm{P}=\sum _{n=1}^N(R_n-R_{n-1})P_n$$ 31

In eq , P _n and R _n denote the precision (PPV) and recall (SEN) values computed at the nth operating point along the Precision–Recall curve.

3.2. Experimental Detail

In the TDMFLSGAT framework, each molecule is first encoded using its SMILES representation, and three complementary sources of information are extracted in parallel. Molecular fingerprints (ECFP, PubChem, APC2D, Klekota–Roth, and Topological Torsion) are computed using PaDEL-Descriptor and RDKit, then individually processed through small multilayer perceptrons equipped with batch normalization and ReLU activations, producing one 128-dimensional vector per fingerprint. These five vectors are subsequently concatenated into a single 640-dimensional descriptor.

In parallel, the molecular graph is constructed by treating atoms as nodes and chemical bonds as edges. Each atom is encoded using a 65-dimensional feature vector that captures its structural and physicochemical properties. A detailed description of all atomic descriptors, together with the possible values considered for each feature, is provided in Supplementary Table S1. The graph is then processed by a Layer-wise Self-Adaptive Graph Attention Network (LSGAT) comprising five layers with four attention heads each. At every layer, node contributions are adjusted according to their coverage degree, modulated by the scaling parameter β = 0.6. This adaptive mechanism prevents highly connected atoms from dominating the message-passing process and helps preserve local structural distinctions that would otherwise be smoothed out in conventional GAT architectures.

To support the choice of β, we carried out a one-dimensional grid search over β ∈ {0.2, 0.4, 0.6, 0.8, 1.0}, retraining the entire model from scratch under an identical protocol for each value. As summarized in Supplementary Table S2 (Supporting Information), model performance remained stable within this interval, with β = 0.6 providing the most balanced combination of AUC (0.896) and MCC (0.641). For this reason, β = 0.6 was adopted in all final experiments. The outputs of the attention heads are concatenated and flattened to obtain a 128-dimensional graph-level representation.

Simultaneously, the SMILES sequence is tokenized using a Byte-Level BPE tokenizer and converted into 128-dimensional embeddings, supplemented with positional encodings. These sequences are processed through a Transformer encoder composed of six layers, each using eight attention heads and a 256-unit feedforward block. A mean-pooling operation produces a compact 128-dimensional summary vector capturing both local substructures and long-range dependencies within the SMILES string. To maintain consistent input length, all sequences are padded or truncated to a maximum of 79 tokens, corresponding to the 95th percentile of the token-length distribution.

The outputs of the three information streamsfingerprints, graph representation, and SMILES Transformer embeddingare concatenated into an 896-dimensional feature vector and passed through a fully connected network that reduces it to 128 dimensions. A final sigmoid-activated layer produces the probability of a molecule acting as a hERG blocker, and model optimization is performed using binary cross-entropy loss.

Training was conducted with the AdamW optimizer (initial learning rate 0.0001; weight decay 10^–4). A cosine-annealing schedule (CosineAnnealingLR) with T _max = 50 was used to modulate the learning rate. Each fold of the 5-fold cross-validation was trained for up to 50 epochs with a batch size of 100, while Early Stopping based on validation AUC (patience = 7 epochs) was applied to select the best model. Hyperparameter refinement combined an initial random search with subsequent optimization using the Hyperopt Tree-structured Parzen Estimator (TPE). The final hyperparameter configuration is listed in Table .

2. Hyperparameters Used in the TDMFLSGAT Model.

parameter	description	range/value
batch_size	input batch size	100
lr	initial learning rate	[10–5, 10–3] (final: 10–4)
train_epoch	maximum number of epochs per fold	50 (with early stopping)
dropout	dropout probability	[0.0, 0.5]
embed_dim	transformer embedding dimension	128
ff_dim	hidden size in transformer FFN	256
attn_layers	transformer encoder layers	6
attn_heads	attention heads per transformer	[2, 8] (final: 8)
num_lsgat_layers	number of LSGAT layers	5
num_lsgat_heads	attention heads per LSGAT layer	4
beta	overlap scaling hyperparameter	[0.2, 0.8] (final: 0.6)
T_max	cosine scheduler period	50
weight_decay	L2 regularization coefficient	10–4
patience	early stopping patience (AUC criterion)	7

3.3. Comparison with Other Methods

To evaluate the effectiveness of the proposed TDMFLSGAT model, we conducted a systematic comparison with both conventional machine-learning (ML) baselines and recent deep-learning (DL) approaches developed for hERG cardiotoxicity prediction. This distinction makes it possible to separately assess the added value of the proposed architecture over classical classifiers and to position its performance relative to current state-of-the-art models.

All methods were evaluated using the same data partitioning strategy and a consistent 5-fold cross-validation protocol. For conventional ML baselines, all models were trained on the same concatenated molecular fingerprint representation used by TDMFLSGAT, ensuring a fair and controlled experimental setup. Model performance was assessed using multiple complementary metrics, including accuracy (ACC), area under the ROC curve (AUC), average precision (AP), specificity (SPE), sensitivity (SEN), negative predictive value (NPV), positive predictive value (PPV), and the Matthews correlation coefficient (MCC). In particular, AP was included to better capture model behavior under class imbalance, which is common in hERG-related data sets.

3.3.1. Comparison with Conventional Machine-Learning Baselines

We first compared TDMFLSGAT with several widely used conventional ML classifiers, namely k-nearest neighbors (kNN), support vector machine (SVM), random forest (RF), and XGBoost. These models are commonly employed in ligand-based toxicity prediction and therefore serve as strong reference baselines.

As reported in Table , TDMFLSGAT consistently outperforms all conventional ML approaches across most evaluation metrics. Among the classical models, XGBoost achieves the strongest performance, with an accuracy of 0.792, an AUC of 0.871, and an AP of 0.897. Nevertheless, TDMFLSGAT surpasses this baseline by a clear margin, achieving an accuracy of 0.823, an AUC of 0.901, and an AP of 0.915. These results indicate that the observed performance gains cannot be attributed solely to the use of molecular fingerprints, but rather stem from the effective integration of sequential, structural, and physicochemical molecular representations.

3. Performance Comparison between TDMFLSGAT and Conventional Machine-Learning Baselines under 5-fold Cross-Validation.

method	ACC	AUC	AP	SPE	SEN	NPV	PPV	MCC
kNN	0.780	0.867	0.880	0.709	0.836	0.776	0.782	0.552
SVM	0.780	0.866	0.891	0.693	0.849	0.786	0.776	0.552
random forest	0.789	0.782	0.890	0.720	0.843	0.786	0.790	0.570
XGBoost	0.792	0.871	0.897	0.750	0.826	0.775	0.805	0.578
TDMFLSGAT	0.823	0.901	0.915	0.792	0.850	0.810	0.834	0.641

3.3.2. Comparison with Deep-Learning and State-of-The-Art Methods

We further compared TDMFLSGAT with several representative deep-learning models proposed for hERG cardiotoxicity prediction, including DMFGAM, hERG-MFFGNN, Smiles2vec, hERG-Att, and a standard graph convolutional network (GCN). For hERG-MFFGNN, the publicly available implementation was retrained on our data set using the same 5-fold cross-validation protocol as TDMFLSGAT to ensure a fair comparison.

As summarized in Table , TDMFLSGAT achieves the best overall performance among all evaluated deep-learning methods, obtaining the highest values for accuracy, AUC, AP, and MCC. Although DMFGAM and Smiles2vec report competitive ROC-based metrics in their original studies, Average Precision values are not provided and could not be reliably reproduced due to the lack of accessible prediction scores. Accordingly, AP is reported only for models whose outputs allow a direct precision–recall analysis.

4. Performance Comparison between TDMFLSGAT and Deep-Learning–Based hERG Prediction Models under 5-Fold Cross-Validation.

method	ACC	AUC	AP	SPE	SEN	NPV	PPV	MCC
DMFGAM	0.817	0.894		0.780	0.847	0.805	0.827	0.630
Smiles2vec	0.718	0.803		0.548	0.853	0.749	0.705	0.426
hERG-Att	0.788	0.855	0.876	0.764	0.795	0.750	0.807	0.558
GCN	0.737	0.819	0.844	0.686	0.778	0.714	0.755	0.467
hERG-MFFGNN	0.799	0.872	0.890	0.754	0.833	0.778	0.814	0.590
TDMFLSGAT	0.823	0.901	0.915	0.792	0.850	0.810	0.834	0.641

When compared with hERG-MFFGNN, which represents a recent and strong multimodal baseline, TDMFLSGAT shows consistent improvements across all reported metrics. These results highlight the benefit of integrating a Transformer-based SMILES encoder together with a layer-wise self-adaptive graph attention mechanism, which jointly enhance the model’s capacity to capture relevant molecular patterns associated with hERG blockade.

Overall, these results demonstrate that TDMFLSGAT delivers consistent and robust improvements over both conventional machine-learning baselines and recent deep-learning approaches. The inclusion of Average Precision further confirms that these gains remain meaningful in the presence of class imbalance, reinforcing the relevance of the proposed framework for realistic hERG cardiotoxicity screening.

3.4. External Validation

To further assess the robustness of TDMFLSGAT, we evaluated the model on three external data sets released by Karim et al. These data sets form part of the CardioToxNet benchmark and are commonly used to examine the stability of hERG-blocker prediction methods. Test Set I contains 44 compounds (30 blockers and 14 nonblockers), Test Set II is considerably larger with 740 molecules (34 blockers and 706 nonblockers), and Test Set III is a small collection of 41 samples (11 blockers and 30 nonblockers).

CardioToxNet also includes a separate training subset composed of 12,620 molecules, of which 6643 are annotated as hERG blockers and 5977 as nonblockers. Although this external training set was not used for developing our model, its distribution closely resembles that of the three test partitions and provides useful context regarding the chemical domain represented in the external evaluation. All molecules in the external sets are independent from our internal training and testing data, ensuring that the assessment remains fully unbiased.

The predictive results obtained on the three external data sets are summarized in Table . On Test Set I, TDMFLSGAT achieves the highest ACC, AUC, SEN, and MCC among all evaluated models, reflecting a clear advantage in distinguishing blockers under imbalanced conditions. For Test Set II, the model again obtains the best ACC, AUC, and SPE, demonstrating strong control over false positives in a large data set dominated by nonblockers. On the low-data Test Set III, TDMFLSGAT remains competitive and reaches the top ACC, AUC, SPE, PPV, and MCC, indicating that its multimodal feature integration results in more stable predictions than approaches relying solely on SMILES- or graph-based representations.

5. External Validation Performance of TDMFLSGAT Compared with Existing Methods on Three Independent hERG Datasets from CardioToxNet.

evaluation data	method	ACC	AUC	SPE	SEN	NPV	PPV	MCC
test set I	TDMFLSGAT	0.812	0.884	0.865	0.775	0.928	0.652	0.604
	DMFGAM	0.795	0.871	0.857	0.767	0.920	0.631	0.586
	Smiles2vec	0.736	0.846	0.874	0.617	0.663	0.847	0.500
	hERG-Att	0.705	0.805	0.714	0.700	0.526	0.840	0.389
	GCN	0.726	0.842	0.714	0.733	0.555	0.846	0.424
	RF	0.750	0.779	0.857	0.700	0.571	0.913	0.519
test set II	TDMFLSGAT	0.768	0.836	0.763	0.748	0.985	0.142	0.251
	DMFGAM	0.748	0.819	0.749	0.735	0.983	0.123	0.227
	Smiles2vec	0.707	0.802	0.548	0.849	0.762	0.683	0.420
	hERG-Att	0.700	0.774	0.695	0.794	0.985	0.112	0.218
	GCN	0.632	0.766	0.624	0.794	0.984	0.092	0.179
	RF	0.675	0.676	0.675	0.676	0.977	0.091	0.155
test set III	TDMFLSGAT	0.762	0.869	0.733	0.827	0.921	0.538	0.497
	DMFGAM	0.731	0.848	0.700	0.818	0.913	0.500	0.463
	Smiles2vec	0.712	0.784	0.677	0.743	0.691	0.728	0.422
	hERG-Att	0.683	0.846	0.600	0.909	0.947	0.454	0.452
	GCN	0.609	0.655	0.533	0.818	0.889	0.391	0.313
	RF	0.658	0.737	0.567	0.907	0.944	0.434	0.424

Taken together, these results confirm that TDMFLSGAT generalizes effectively to previously unseen chemical structures and maintains consistent performance across data sets that differ in scale, class distribution, and data sparsity.

3.5. Impact of Architectural Contributions

To better understand the role played by each architectural component within the TDMFLSGAT framework, three configurations were evaluated: (i) the complete model combining the SMILES Transformer, the Layer-wise Self-Adaptive Graph Attention Network (LSGAT), and molecular fingerprints; (ii) a version excluding the Transformer branch (retaining only the graph and fingerprint modules); and (iii) a version in which the LSGAT was replaced by a conventional graph attention network (GAT) while keeping the other two branches unchanged. The quantitative comparison is summarized in Table , and the corresponding diagnostic plots are presented in Figures –.

3.

Performance of the complete TDMFLSGAT model on the test set.

5.

Ablation results when LSGAT is replaced by a conventional GAT.

As illustrated in Figure , the full TDMFLSGAT model achieves the most balanced performance across all evaluated metrics. The ROC curve (Figure a) shows strong separability between blockers and nonblockers, while the confusion matrix (Figure b) confirms a coherent distribution of true positives and true negatives.

When the Transformer branch is removed (Figure ), the model exhibits a noticeable drop in both AUC and sensitivity. This is reflected in a higher rate of false negatives in the confusion matrix (Figure b), suggesting that sequential dependencies encoded in SMILES strings contribute meaningfully to the detection of hERG blockers.

4.

Ablation results for the model without the transformer component.

Replacing LSGAT with a classical GAT (Figure ) also leads to a degradation in predictive performance. The narrower separation observed in Figure a and the increase in misclassifications in Figure b indicate that the adaptive scaling mechanism of LSGAT effectively mitigates oversmoothing and preserves chemically informative relationships during graph propagation. To verify the ability of LSGAT to mitigate oversmoothing, we compared the attention allocated to high-degree atoms across layers for both LSGAT and classical GAT. As shown in Figure S1 (Supporting Information), LSGAT consistently assigns less attention to hub atoms, confirming the effectiveness of the adaptive scaling mechanism.

3.5.1. SMILES Transformer Branch

Excluding the Transformer reduces accuracy from 0.823 to 0.807, AUC from 0.901 to 0.888, and sensitivity from 0.850 to 0.817. This confirms that the Transformer helps capture long-range chemical dependencies encoded in SMILES sequencesparticularly those linking distant but functionally related fragments.

3.5.2. Layer-Wise Self-Adaptive Graph Attention Network Branch

Replacing LSGAT with a conventional GAT lowers accuracy to 0.811, AUC to 0.886, and MCC to 0.620. Unlike standard GAT layers, LSGAT incorporates an adaptive mechanism that prevents high-degree atoms from dominating the aggregation process and mitigates oversmoothing. This leads to richer and more discriminative graph representations, especially for molecules with multiple aromatic rings or complex local environments.

3.5.3. Molecular Fingerprint Branch

The fingerprint branch is included in all three variants and is therefore not removed as a standalone ablation. This choice reflects two considerations. First, fingerprints constitute the core representation used in several established hERG prediction modelsincluding DMFGAMand removing them would shift the comparison away from this widely adopted baseline. Second, preliminary experiments showed that models relying only on the transformer and graph branches exhibited reduced predictive performance and unstable training dynamics. Fingerprints thus provide complementary and chemically meaningful cues that cannot be entirely recovered from the other modalities. Their contribution is analyzed in more detail in the multimodal interpretability analysis subsection, where a multimodal interpretability study employs LIME to quantify their feature-level impact.

3.5.4. Combined Effect

Integrating all three branches yields the most balanced and robust results: a sensitivity of 0.850 ensures that potential blockers are reliably detected, a specificity of 0.792 limits false positives, and the MCC reaches 0.641, underscoring the overall consistency of the model.

3.6. Multimodal Interpretability Analysis

To complement the global evaluation metrics, we examined in more detail how the different components of TDMFLSGAT contribute to its predictions. For this purpose, we selected one representative compound from the internal test set that the model classifies as a hERG blocker with high confidence (SMILES: Cc1cc2nc­(CCC3CCCN3C)­n­(Cc3ccc­(Cl)­cc3)­c2cc1C). Using the same molecule in all panels of Figure allows a direct comparison of the explanations provided by the three modalities of the model.

6.

Multimodal interpretability analysis for a representative hERG blocker correctly classified by TDMFLSGAT. (a) SMILES transformer token–token attention map. (b) Atom–atom attention matrix of the last LSGAT layer. (c) Projection of the most influential atoms (red circles) onto the molecular graph. (d) LIME bar plot showing the most influential fingerprint features contributing to the blocker prediction; several topological torsion (FP2) and Klekota–Roth (FP4) features appear among the strongest positive contributors. Representative SMILES: Cc1cc2nc­(CCC3CCCN3C)­n­(Cc3ccc­(Cl)­cc3)­c2cc1C.

Figure a shows the token–token attention pattern generated by the SMILES Transformer. Once the attention scores are averaged across heads, clear vertical and horizontal structures appear in the heat map. These patterns reveal that the encoder gives particular emphasis to chemically meaningful fragmentsaromatic systems, heterocycles, and nitrogen-containing groupsrather than processing the SMILES merely as a linear sequence of characters.

In parallel, Figure b illustrates the atom–atom attention obtained from the last LSGAT layer. Strong off-diagonal values highlight atom pairs that exchange substantial message-passing information. After averaging these interactions across local neighborhoods, each atom receives an importance score that can be projected back onto the two-dimensional molecular structure (Figure c). The atoms identified as most influential concentrate around a tertiary amine and adjacent aromatic carbons, two motifs frequently associated with hERG activity in medicinal chemistry reports.

The contribution of the fingerprint branch was examined separately using a LIME analysis (Figure d). Several features belonging to the Topological Torsion (FP2) and Klekota–Roth (FP4) descriptorsfingerprint families added in this work to extend the original DMFGAM feature setemerge as dominant positive contributors to the blocker prediction. At the same time, smaller but non-negligible contributions from Morgan and PubChem fingerprints are also observed. This pattern indicates that the model does not rely exclusively on a single fingerprint type; instead, it integrates complementary information across all families (Morgan, PubChem, FP2, and FP4) to refine its decision boundary between blockers and nonblockers.

Taken together, these three layers of interpretationsequence-level attention, graph-level attention, and fingerprint-level LIME analysisshow that TDMFLSGAT bases its predictions on chemically meaningful features rather than on incidental correlations. The prominent role of FP2 and FP4 in the fingerprint explanation further supports their inclusion, as these descriptors capture structural motifs that are highly relevant for distinguishing hERG-active compounds.

4. Discussion and Conclusion

This study introduced TDMFLSGAT, a multimodal deep-learning model for predicting hERG channel blockade by combining three complementary sources of information: SMILES sequences, molecular graphs, and multiple fingerprint families. The main motivation was to overcome the limitations of approaches that rely on a single representation or on simple fusion strategies.

Overall, the results support the effectiveness of this design. Under the same 5-fold cross-validation protocol used throughout the manuscript, TDMFLSGAT achieves strong and well-balanced performance (ACC = 0.823, AUC = 0.901, MCC = 0.641), and it remains consistently better than conventional machine-learning baselines trained on the same fingerprint inputs (Table ). This point is important, because it shows that the improvement is not only due to fingerprints, but to the benefit of combining sequential, structural, and physicochemical signals within a single framework.

In addition to ROC-based evaluation, we also assessed the model using Precision–Recall analysis, which is more informative when class distributions are skewed. As shown in Figure , TDMFLSGAT maintains higher precision across a wide range of recall values than the strongest baseline (hERG-MFFGNN). This is reflected by a higher Average Precision score (AP = 0.915 vs 0.890), indicating that the proposed model remains reliable when evaluated with imbalance-aware criteria.

The ablation study further clarifies where the gains come from. Removing the SMILES transformer branch reduces AUC and sensitivity, suggesting that contextual SMILES information helps detect blockers more accurately. Likewise, replacing LSGAT with a classical GAT leads to weaker overall performance (Table ), supporting the role of the adaptive scaling mechanism in preserving discriminative graph information. This interpretation is consistent with Figure S1 (Supporting Information), where LSGAT assigns less attention to high-degree atoms and therefore helps limit oversmoothing effects across layers.

We also evaluated generalization on three external CardioToxNet benchmarks. Across data sets that differ in size and class balance, TDMFLSGAT remains competitive and often achieves the best or near-best results compared with existing methods (Table ). These outcomes suggest that the multimodal representation is not only effective in internal testing but also transfers well to independent chemical sets.

At the same time, the conclusions should be interpreted within the scope of this work. The study remains a retrospective computational evaluation, and the available labeled data are limited relative to the complexity of the problem. In addition, the current feature set mainly captures 2D structure; incorporating 3D conformational information or channel-related interaction descriptors could be a useful direction. Future work will therefore focus on larger and more diverse data sets, as well as on prospective validation settings.

In summary, the data presented in this manuscript provide quantitative support that TDMFLSGAT offers consistent improvements over both conventional ML baselines and recent deep-learning methods. The gains are confirmed not only by ROC-based metrics but also by PR-based evaluation, reinforcing the practical relevance of the proposed framework for early stage cardiotoxicity screening.

The data sets and source code used in this study are available at https://github.com/BeidjaCheikh/TDMFLSGAT_data_and_code.

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acsomega.5c10745.

• Table S1detailed description of the 65 atomic features used for molecular graph construction; Table S2sensitivity analysis of the LSGAT scaling hyperparameter β; Figure S1comparison of the average fraction of attention received by high-degree atoms across graph layers for LSGAT and a classical GAT, illustrating the ability of the proposed LSGAT to mitigate oversmoothing through adaptive attention scaling (PDF)

The authors declare that no external funding was received for this research.

The authors declare no competing financial interest.