Bridging predictive reliability and explainability: a multi-representation deep learning framework for chemical space analysis of immune bioassays

Abstract

Convergence of chemical and biological space into a unified decision space requires coordinated interactions across the triad- molecular representations, learning algorithms, and explainability. This study establishes a benchmarking framework for Virtual Screening (VS) that integrates this triad to enable consistent, predictive, and interpretable modelling of complex immune-related bioassays. The study employed a wide range of Machine Learning (ML) and Deep Learning (DL) models across three molecular input modalities—descriptors, images, and graphs—while prioritizing predictive reliability and model interpretability. Among classical ML models, Support Vector Machines (SVM) demonstrated the strongest overall performance, while the AttentiveFP architecture outperformed other DL models. Beyond identifying context-specific optimal VS networks, this study derives latent learned concepts that support subsequent hypothesis generation and experimental validation of the interaction between immune targets and ligands.

Scientific contribution

This study establishes a benchmarking framework for virtual screening of immune targets by curating representative bioassay datasets, systematically evaluating multiple molecular representations and learning algorithms, and integrating explainability into the best-performing models. By combining Concept Whitening with the AttentiveFP architecture, we introduce an integrated explainable DL framework that maintains predictive reliability while enabling alignment and separation of domain-specific concepts for immune-target virtual screening. In addition, the study highlights the potential for decision- and logical-level model ensembling, identifies target-specific concept relevance in decision-making, and demonstrates the relationship between data characteristics and model behaviour.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13321-026-01206-5.

Introduction

The immune system is a highly complex, multilayered and context-dependent network, which limits the effectiveness of standardized datasets, general-purpose algorithms and direct analytical pipelines for elucidating ligand–target interactions. Owing to their inherently dynamic and nonlinear behavior, single-scale datasets and generic modeling approaches rarely capture the full spectrum of functional states, thereby masking mechanistically relevant patterns in traditional data-mining workflows. Therefore, immune-focused modeling necessitates target-specific datasets, adaptive molecular representations and explainability-driven algorithms capable of capturing complex, non-linear relationships and biologically conditioned variability.

The effectiveness of virtual classification critically depends on data quality, structural diversity, and class imbalance [28], and the crucial factor influencing model performance is the choice of molecular representation [41]. Descriptor-based representations encode molecules as structured numerical descriptors, emphasizing computational efficiency [12, 22]. Image-based encodings represent rich structural and chemical features in visual formats, enabling the extraction of spatial patterns from molecular structures, whereas graph-based representations preserve stereochemical and topological relationships [22, 26, 30].

The predictive reliability of a model is critically influenced by selecting an architecture whose inductive biases align with the intrinsic structure of the chosen molecular representation [9, 24]. Descriptor-based vectors can be processed using Feed Forward Neural Networks (FFNNs), which can capture non-linear relationships in low-dimensional structured data [48]. Image-based molecular representations are encoded using Convolutional Neural Networks (CNN) for their ability to extract spatially coherent motifs and localized structural features [16]. Graph Neural Networks (GNNs) can be used to model graph-based molecular representations that effectively capture atomic connectivity and bonding topology, thereby capturing stereochemical features necessary for reliably predicting bioactivity [14, 47, 51]. Despite the rise of DL, classical ML models, such as SVMs, remain highly effective for virtual classification tasks in settings with limited sample sizes of molecular data [43]. The prevalence of ensemble learning in classification tasks necessitates a systematic investigation of the optimal ensemble strategies [11, 13, 33, 37].

Although DL has advanced rapidly and is widely adopted, in safety–critical domains like Drug Discovery, it is frequently regarded as a black-box system, limiting mechanistic interpretability and decision transparency [17, 19, 23, 31, 34]. Consequently, Explainable AI (XAI) has emerged as an essential architectural component for enabling trust-centric and regulatory-compliant deployment of AI systems within the drug discovery pipeline [6, 39]. The existing XAI methodologies can be systematically classified into three categories based on their underlying logic: post-hoc attribution methods, attention-based interpretability approaches, and concept-based interpretability frameworks that aim to bridge the gap between model outputs and their biological relevance [2, 45]. The post-hoc attribution methods, such as Shapley Additive Explanations (SHAP) [27], Local Interpretable Model-Agnostic Explanations (LIME) [36], and Integrated Gradients (IG) [44] are widely adopted for their feature-level or atom-level contributions in model predictions. However, these approaches can exhibit stochastic instability, high sensitivity to input perturbations and limited alignment with chemically meaningful concepts, as the resulting explanations are derived from latent representations and are not explicitly optimized for interpretability, which restricts the derivation of meaningful insights [3, 38, 42]. Attention-based interpretability, prevalent in GNNs and transformer-based molecular models, enables structural transparency by highlighting salient atoms, bonds, or molecular substructures, and offers intuitive visualisations of focus regions in molecular graphs and their target interactions [21]. However, the empirical evidence suggests that attention weights do not correlate with causal feature importance, which cannot account for mechanistic decision pathways [21, 40]. Concept-aligned interpretability approaches address the limitations of post-hoc methods, which integrate transparency into training by aligning latent axes with predefined, domain-relevant concepts via orthogonal rotations, enabling disentangled feature learning, stable concept-level traceability and improved mechanistic alignment [1, 7, 8, 25, 34].

In drug discovery, in silico Virtual Screening (VS) enables systematic interrogation of large chemical libraries, thereby prioritizing potential lead candidates by computationally predicting the interactions, binding affinity, biological activity, and pharmacophoric profiles, however lack mechanistic understanding and regulatory acceptance [10, 52]. By elucidating descriptors, pharmacophoric features, and spatial interaction motifs, XAI mitigates this issue, thereby shifting VS from purely statistical correlation to mechanism-informed drug discovery. Recent advancements in XAI-enabled VS include GNN explainability methods, which identify the molecular subgraphs with concept-level explainability and a unified benchmarking platform facilitating visualization, quantitative evaluation and comparison of multiple XAI methods for drug screening [4, 50].

This study investigates how different molecular representation paradigms- descriptor-based, image-based and graph-based approaches- affect the learning pipeline using bioassay outcomes as the benchmark. The study also evaluates how different algorithms, ranging from classical ML models to DL architectures, perform with varying choices of molecular representation. The study further investigates whether ensemble-based integration of multiple models can enhance predictive reliability, particularly in bioactivity prediction, in which maintaining a trade-off between sensitivity and specificity is crucial. To strengthen interpretability across state-of-the-art deep classification frameworks, we integrate CW into the best-performing DL model AttentiveFP, providing transparent attribution of predictions to chemically relevant concepts while retaining the predictive power of latent graph embeddings, thereby bridging reliability and interpretability. The graphical abstract summarizing the conceptual framework of the study is presented in Fig. 1.

Fig. 1.

Graphical abstract depicting the proposed representation-learning-explainability triad and optimization for virtual screening of bioassay molecules

In essence, the study makes the following key contributions to the field:

1. The study developed a biologically relevant immune-target benchmark integrating both activation and inhibition bioassays, making context-aware evaluation of models under complex immunological signaling.

2. By systematically benchmarking classical ML models, DL models and hybrid ensemble architectures against descriptor-based, image-based and graph-based molecular representations, the study identified graphs as the principal direction of immune-target virtual screening. Following systematic experimentation, benchmarking, and comparative analysis, the study identified SVM and AttentiveFP as the best-performing ML and DL architectures, respectively, for virtual classification of immune-related targets.

3. The study developed an AttentiveFP–Concept Whitening framework that provides domain-aligned, concept-level interpretability and enables the identification of target-specific subsets of the most relevant chemical concepts.

4. For virtual classification of bioassays, the study identified Majority Voting as the most effective ensembling approach for DL architectures.

5. Identified target-specific and cross-target molecular descriptors, reinforcing model learning and interpretation, are relevant to immune-target virtual screening.

Methods

Datasets

The datasets employed in the study comprise four bioassays, namely AID932, AID1239, AID1578, and AID1259354, sourced from the publicly available PubChem database, which modulate key immune response pathways to classify molecules by their molecular bioactivity. The target variables examined in the study include four proteins: STAT1-alpha/beta isoform alpha (activation), NF-Kappa-B p105 subunit isoform1 (activation), NOD1isoform 1 (inhibition), and ST2 (IL1RL1) (inhibition), which play a vital role in signalling pathways that regulate immune and inflammatory response, as shown in Table 1.

Table 1.

Overview of the immune targets selected for the study

Dataset	Immune target	Activity type	Actives (selected/total)	Inactives (selected/total)	Total compounds (selected/total)
AID932	STAT1-alpha/beta isoform alpha	Activation	2000/5134	2000/190,878	4000/196,012
AID1239	NF-Kappa-B p105 subunit isoform1	Activation	2000/3101	2000/190,195	4000/193,297
AID1578	NOD1 isoform 1	Inhibition	2000/3000	2000/286,584	4000/289,484
AID1259354	Chain B, Interleukin-1 receptor-like 1 ST2 (IL1-RL1)	Inhibition	1823/1823	1825/54,555	3648/77,700

The dataset composition of the selected bioassays is provided in Table 1. The dataset AID932, focused on the immune target STAT1-alpha/beta isoform alpha of activity type ‘activation’, contained 196012 compounds, of which 5134 were active and 190878 were inactive. The datasets were selectively designed to obtain a balanced distribution of active and inactive molecules, enabling robust algorithm evaluation to mitigate class imbalance. For each molecule in the balanced datasets, the corresponding SMILES strings were retrieved from the PubChem database, and RDKit-based molecular descriptors were extracted through the Online Chemical Modeling Environment (OChem) database.

The data is preprocessed to prepare the datasets for subsequent analysis, ensuring compatibility with the input representations required by descriptor-based, image-based and graph-based modeling techniques. Descriptor columns containing more than 50% missing or zero values were discarded, and remaining missing entries were assigned with zero. Details of data preprocessing are provided in Additional File 1: Table 1. Descriptor-based and image-based models were standardised using z-score normalization, whereas graph-based models (Vanilla GNN, GCN, AttentiveFP) were constructed from SMILES using PyTorch Geometric, with atoms encoded as node features and bond types represented as edge attributes.

Across all datasets, a stratified 80:20 train–test split was applied; descriptor-based models used scikit-learn utilities for splitting and scaling; and graph-based data were mini-batched using the PyTorch Geometric DataLoader. Activity labels were binarized (active = 1, inactive = 0), and residual NaN or infinite values were sanitised to prevent computational instability.

Algorithmic framework selection aligned with molecular representation in chemical space

To systematically decode immune-relevant chemical space, this study derives an optimal triad comprising the most informative molecular representation, the best-performing learning framework, and an enhanced concept-level interpretability approach. As a result, we propose a unified analytical framework for exploring chemical space and classifying virtual bioassays, as illustrated in Fig. 2.

Fig. 2.

Framework for analyzing the chemical space of molecules by classification

The chemical space analysis incorporates four distinct bioassay datasets, evaluated using a range of algorithms spanning both conventional ML and advanced DL approaches. The processing was initiated with classical ML models, including SVM and Gaussian Naive Bayes classifiers, which were used as baseline methods for predictive analysis. To qualitatively assess the feature-space topology and facilitate interpretation of class separability and the underlying structural organization, t-SNE visualisation was implemented. The DL analysis was based on three types of molecular input representations: descriptor-based, image-based and graph-based. Molecular features were first represented as descriptor vectors, and used FFNN to capture the non-linear relationships among the molecules. Then, as two-dimensional images, which allows CNN to extract spatial and topological features. Finally, molecules were represented as graphs. As a base model, VanillaGNN was adopted to define a benchmark for graph learning. Next, GCN was applied to learn structure-informed representations that capture atomic connectivity and bonding patterns in molecular graphs. AttentiveFP was employed to embed molecular properties using attention mechanisms, allowing the model to selectively focus on substructures most relevant to predicting target properties. Model performance strongly depends on hyperparameters, making optimization essential for interpretable benchmarking across models and datasets.

Table 2 lists the specific setup for all ML and DL models used in the study.

Table 2.

Summary of machine learning and deep learning model configuration

Model name	Input representation	Core architecture/learning strategy	Loss and optimizer
Support Vector Machine (SVM)	High-dimensional descriptors	Nonlinear margin-based classifier with RBF kernel; probability calibration enabled	(Scikit-learn default), probability estimates enabled
Gaussian Naïve Bayes (NB)	Descriptor-based feature vectors	Probabilistic classifier assuming Gaussian likelihood per feature	(NB closed-form), no gradient-based optimization
Feed-Forward Neural Network (FFNN)	Molecular descriptors	Fully connected MLP with progressively reduced layers; dropout + L2 regularisation	Binary Cross-Entropy and Adam
Convolutional Neural Network (CNN)	1D encoded descriptor vectors	Stacked 1D Conv + MaxPooling blocks → Dense layers with dropout	Binary Cross-Entropy and Adam
VanillaGNN	SMILES-derived molecular graphs	Node/edge feature embedding → dense transformations → global pooling	Binary Cross-Entropy and Adam
Graph Convolutional Network (GCN)	SMILES-derived Atom–bond graph representation	Multi-layer GCNConv aggregation → global readout → FC output	Binary Cross-Entropy and Adam
AttentiveFP	SMILES-derived Atom-wise graph embeddings	Attention-based message passing + global attentive readout	Binary Cross-Entropy, Adam and weight decay

Optimization of hyperparameters using Optuna

Optuna is a framework for optimizing hyperparameters designed to identify the set of parameters that deliver the best performance [5]. The main aim of the optimization process is to maximize the ROC-AUC score or to improve the test accuracy on the test set [5]. The hyperparameter tuning process defined a search space consisting of the number of hidden layers, the number of neurons, the learning rate on a logarithmic scale from 10^–5 to 10^–1, dropout ranging from 0.0 to 0.6, and weight decay ranging from 10^–7 to 10^–3.

The parameter set that produced the best performance after 20 iterations of hyperparameter tuning was selected for the final model training. Hyperparameter optimization was employed on every DL model trained on each dataset. The specific configurations used for hyperparameter tuning of each model are described in Table 3.

Table 3.

Hyperparameter tuning using Optuna on different models

Hyperparameters	FFNN	CNN	VanillaGNN	GCN	AttentiveFP
Hidden units	32–256	32–256	32–128	32–128	64–256
Layers	2–4	2–4	3 Fully connected layers	2–5	2–8
Dropout rate	0.1–0.5	0.1–0.5	0.1–0.5	0.1–0.5	0.0–0.6
Learning rate	0.00001–0.01 (log scale)	0.00001–0.01 (log scale)	0.0001–0.01 (log scale)	0.0001–0.01 (log scale)	0.00001–0.1 (log scale)
Weight decay/L2	0.00001–0.01 (log scale)	0.00001–0.01 (log scale)	0.000001–0.0001 (log scale)	0.000001–0.0001 (log scale)	0.0000001–0.001 (log scale)
Batch size	32,64,128	32,64,128	32	32	32

Model training after Optuna optimization

All five models were trained for 75 epochs, employing binary cross-entropy loss as the objective function for the binary classification problem. Adam optimizer was used by incorporating weight decay for regularisation to train the models. To address overfitting, early stopping was employed by monitoring the validation loss and halting training when performance levelled off.

Model evaluation metrics

To assess the performance of each model, a set of evaluation metrics was used, including accuracy, Receiver Operating Characteristic (ROC) Area Under Curve (AUC), F1- score, Cohen’s Kappa (κ), precision and recall [35, 46].

As a methodological extension beyond the previously described representation-specific models, an ensemble-based framework was devised to integrate the aforementioned predictive DL models and synthesize their outputs through logic-based networks, including Logical AND, Logical OR, and Majority Voting [32]. Ensemble learning is well-established for improving predictive accuracy, robustness, and generalization in molecular classification tasks [15, 18, 20]. In this study, predictions from FFNN, CNN, VanillaGNN, GCN, and AttentiveFP models were combined, and the efficacy of the ensemble-combination: Logical AND, Logical OR, and Majority Voting was systematically evaluated, as detailed in Fig. 3.

Fig. 3.

Ensemble learning for virtual classification of molecules

Only upon unanimous agreement among all constituent models did the Logical AND ensemble approach classify a molecule as ‘active’, thus prioritising high precision at the cost of recall. Therefore, the approach minimizes false positives by compromising the identification of true positives, thereby reducing type I errors. Logical AND voting is most effective during late-stage validation, where high-confidence predictions are crucial to minimizing experimental costs [15]. Conversely, the Logical OR ensemble strategy classified a molecule as Active if ‘any’ constituent model predicted it as such, prioritizing high recall, potentially at the expense of precision. The Logical OR approach is ideal when the goal is to identify the maximum number of true positives. Though the approach prioritizes sensitivity (true positive rate), which maximizes the identification of active compounds, it may lead to a higher occurrence of false positives. The third technique, Majority Voting, determines the final classification label based on the most common prediction among the ensemble of five constituent models. In the study, a given compound is assigned the 'Active' classification if a minimum of three of the five individual models yield a positive prediction. In contrast to the Logical OR and Logical AND techniques that employed row-wise filtering based on label agreement across models, the Majority Voting technique utilized the entire test dataset, assuming consistent ground truth labels across datasets [29].

AttentiveFP with concept whitening

To support interpretable molecular activity prediction, the AttentiveFP architecture [49] was extended with CW, enabling latent dimensions to align with predefined physicochemical descriptors. A high-level schematic of the integrated framework is presented in Fig. 4.

Fig. 4.

Graphical abstract of Concept Whitening integrated in AttentiveFP framework

Molecular graphs were generated from SMILES using RDKit. Atom-level features encoded element type, valency, hybridization, formal charge, aromaticity, total hydrogen count and chirality. Bond-level features captured bond order (single, double, triple, aromatic), conjugation, ring membership and stereo configuration. Graphs were processed with an AttentiveFP backbone comprised of four graph attention layers (hidden dimension = 64, diffusion steps = 4, dropout = 0.2), followed by attentive pooling and a feed-forward classifier (64 → 32 → 2). The network was trained for 50 epochs with Adam (learning rate 3 × 10⁻⁴, weight decay 1 × 10⁻⁵, batch size 64) using cross-entropy loss.

To obtain interpretable latent dimensions, we applied CW to the pooled AttentiveFP embeddings. A curated set of fifty RDKit-derived descriptors was used as target concepts, grouped into six chemically distinct families: (A) constitutional size and composition, (B) polarity, lipophilicity and surface area, (C) hydrogen-bonding and ionisation capacity, (D) aromaticity, conjugation and ring topology, (E) charge distribution and electrotopological state, and (F) topological shape and molecular complexity. For each molecule in the training set, we computed a 64-dimensional pooled embedding $z$ and a 50-dimensional descriptor vector $y_c$. Let $Z$ denote the matrix of pooled embeddings. ZCA whitening was applied by centring with the empirical mean $\mu$ and applying the inverse square root of the covariance as shown in Eq. 1.

$$Z_W=\left(Z-\mu \right){\mathrm{\Sigma }}^{-1/2}$$ 1

Each standardized concept $y_{c,i}$ are then regressed on $Z_w$ using linear regression, and the resulting coefficient vectors are orthonormalised (QR decomposition) and extended to a full orthonormal rotation matrix $R\in {\mathbb{R}}^{64\times 64}$. The Concept Whitening transform is defined as in Eq. 2.

$$Z_{\mathit{CW}}=\left(Z-\mu \right){\mathrm{\Sigma }}^{-1/2}R$$ 2

where the first $k=50$ coordinates are constrained to be maximally correlated with the predefined descriptors, and the remaining coordinates span an orthogonal residual subspace. A Hungarian assignment on the absolute correlation between CW axes and descriptors was used to obtain a one-to-one axis–concept mapping, followed by sign flips to ensure that the correlation was positive. After fitting CW on the warm-up model, the AttentiveFP backbone and classifier were fine-tuned end-to-end with CW integrated into the forward pass (learning rate $1\times {10}^{-4}$, up to 200 epochs, early stopping with patience 20). At inference time, each pooled embedding is centred, whitened and rotated using the learned $\mu$, ${\mathrm{\Sigma }}^{-1/2}$ and $R$, yielding a representation $[z_{\mathit{concept}},z_{\mathit{residual}}]$ in which the leading dimensions are aligned to specific physicochemical descriptors and the remaining dimensions capture task-relevant residual variation.

Results

This section analyses the performance of each model trained on four bioassay datasets.

T-SNE analysis

Figures 5, 6, 7, 8 show t-SNE embeddings of the training and test sets for each bioassay dataset, derived from the SVM classifier outputs, providing a visual complement to the chemical space analysis.

Fig. 5.

SVM model trained on the AID932 dataset

Fig. 6.

SVM model trained on the AID1239 dataset

Fig. 7.

SVM model trained on the AID1578 dataset

Fig. 8.

SVM model trained on the AID1259354 dataset

The T-SNE projections provide qualitative evidence that the topology of chemical space differs substantially across datasets. It is evident from the Figures that, by employing SVM, the T-SNE projections of dataset AID932 reveal two highly distinct, compact clusters corresponding to the active and inactive classes, indicating that the underlying chemical space is governed by well-defined structural or physicochemical features that strongly discriminate biological activity. The t-SNE visualizations for the dataset AID1239 are given in Fig. 6. AID1239 exhibits a clear class separation, with a limited but noticeable margin overlap. That means, while the actives and inactives largely occupy different areas of chemical space, specific subsets with shared features overlap at the margins. The t-SNE embeddings for AID1578, as shown in Fig. 7, indicate diverse features with overlapping physicochemical properties between active and inactive compounds. This suggests heterogeneous chemotypes with subtle or context-dependent activity signals. AID1259354 exhibits the highest overlap between active and inactive classes, with unclear cluster boundaries among the analyzed datasets. As shown in Fig. 8, the dataset exhibits widespread intermixing of molecules, suggesting weak distinguishing attributes, and their high inter-class variability influences the chemical space. Hence, it can be concluded that while AID932 and AID1239 show clear class separations, indicating strong structure–activity relationships, the greater overlap in datasets AID1578 and AID1259354 implies weaker relationships and increased chemical diversity. These insights reveal that classification performance depends on the underlying structure of chemical space, necessitating dataset-specific model selection and evaluation.

Tables 4 and 5 report how SVM and NB models performed across the four datasets.

Table 4.

Performance evaluation of the SVM model

Dataset	Train accuracy (%)	Test accuracy (%)	Precision	Recall	ROC-AUC	F1-score	Cohen’s kappa
AID932	100	99.75	1.00	0.9950	0.9988	0.9975	0.9950
AID1239	94.41	83.25	0.8079	0.8725	0.9136	0.8389	0.6650
AID1578	90.25	71.88	0.7297	0.6950	0.8060	0.7119	0.4375
AID1259354	91.36	78.22	0.7994	0.7534	0.8638	0.7757	0.5644

Table 5.

Performance metrics of Bayesian classification

Dataset	Train accuracy (in %)	Test accuracy (in %)	Precision	Recall	ROC-AUC	F1-score	Cohen’s kappa
AID932	53.94	54.25	0.52	0.94	0.5427	0.67	0.0850
AID1239	65.75	65.87	0.60	0.93	0.6611	0.73	0.3175
AID1578	58.44	53.37	0.52	0.87	0.5314	0.65	0.0675
AID1259354	65.15	59.04	0.68	0.34	0.6938	0.46	0.1808

Traditional ML model analysis

SVM outperformed the NB model across all datasets, indicating that bioactivity depends on correlated, partially non-linear interactions among descriptors that violate the feature-independence assumption. Table 4 shows that AID932 exhibited exceptional class discrimination (test accuracy = 99.75%, ROC-AUC = 0.9988, κ = 0.9950), whereas it showed only moderate performance on AID1239 and AID1259354, which are more complex or noisy. These findings indicate that the SVM outperformed the NB classifier and is therefore well-suited to capturing the intrinsic structure–activity relationships across the datasets.

DL model analysis

A major drawback of the SVM model is its inability to capture intrinsic structural relationships in chemically heterogeneous datasets. This shortcoming necessitates exploring computational methodologies, such as DL architectures and GNNs, to achieve better classification performance. The following sections present the performance of the DL models classified into descriptor-based (FFNN), image-based (CNN) and graph-based (VanillaGNN, GCN, and AttentiveFP).

Descriptor-based model analysis

The FFNN model employed for the descriptor-based analysis exhibited strong performance on AID932 and AID1239, indicating clear structure-activity relationships as shown in Table 6. Conversely, the performance declined, with reduced accuracy and a moderate Cohen’s Kappa value indicating weaker descriptor-activity associations on AID1578 and AID1259354. These results suggest that descriptor-based models are highly dataset-dependent, act as benchmarks for structured bioassays, and emphasize the need for advanced hybrid methods in noisy chemical datasets.

Table 6.

Performance metrics of the FFNN model

Dataset	Train accuracy (in %)	Test accuracy (in %)	Precision	Recall	ROC-AUC	F1-score	Cohen’s kappa
AID932	95.34	81.25	0.7969	0.8095	0.8728	0.8031	0.6242
AID1239	96.56	81.12	0.7789	0.8386	0.8797	0.8076	0.6229
AID1578	89.19	72.12	0.6933	0.7354	0.7964	0.7137	0.4426
AID1259354	90.99	76.71	0.7690	0.7147	0.8502	0.7409	0.5299

Image-based model analysis

The image-based CNN showed only moderate generalisation on AID932 and AID1239 (test accuracy of 68.50–70%, ROC-AUC ranging from 0.76 to 0.88, F1≈0.68) with limited reliability as evident from Table 7. For AID1578 and AID1259354, pronounced overfitting was observed, with high training accuracy (> 89%) collapsing to 56–57% test accuracy, low ROC-AUC (0.59–0.62), and minimal agreement beyond chance with Cohen’s Kappa less than 0.15.

Table 7.

Performance metrics of the CNN model

Dataset	Train accuracy (in %)	Test accuracy (in %)	Precision	Recall	ROC-AUC	F1-score	Cohen’s kappa
AID932	82.47	70.00	0.7235	0.6475	0.7645	0.6834	0.4000
AID1239	86.00	68.50	0.6832	0.6900	0.8752	0.6866	0.3700
AID1578	95.69	56.00	0.5541	0.6561	0.6150	0.5829	0.1200
AID1259354	89.62	57.26	0.5596	0.6822	0.5911	0.6148	0.1452

These results indicate that while CNNs are capable of capturing discriminative image-derived features in relatively less heterogeneous datasets. However, their robustness and reliability deteriorate significantly when applied to complex molecular data, highlighting the need for more rigorous regularisation strategies and domain-guided design optimization.

Graph-based model analysis

The VanillaGNN model showed moderate, yet dataset-dependent, predictive performance as detailed in Table 8. AID932 showed high recall (0.965) but reduced precision, reflecting sensitivity bias, whereas AID1239 achieved a more balanced profile with Cohen’s Kappa of 0.495. In AID1578, the degraded performance, with an accuracy of 69.63% and a Cohen's Kappa of 0.3925, can be attributed to noisy structure–activity relationships. However, the most consistent performance of AID1259354 achieved an accuracy of 76.16% and a Cohen's Kappa of 0.5233, indicating stronger graph-activity correlations. These results indicate that while VanillaGNN learns key graph-level representations, its recall bias and moderate discrimination limit its effectiveness as a base model. Therefore, to improve performance on chemical heterogeneous datasets underscores the need for more advanced GNN variants.

Table 8.

Performance metrics of the Vanilla GNN model

Dataset	Train accuracy (in %)	Test accuracy (in %)	Precision	Recall	ROC-AUC	F1-score	Cohen’s kappa
AID932	74	73	0.7908	0.9650	0.73	0.7572	0.46
AID1239	75.31	74.75	0.7829	0.875	0.7475	0.7748	0.4950
AID1578	66.16	69.63	0.7923	0.9925	0.6963	0.7163	0.3925
AID1259354	75.88	76.16	0.8366	0.9973	0.7616	0.7568	0.5233

The GCN model exhibited moderate predictive performance, which depends on the dataset used, with test accuracies ranging from 66.75% (AID1578) to 74.62% (AID1239), as shown in Table 9. For datasets AID932 and AID1259354, the model achieved recall rates of 0.9225 and 0.9260, but lower precision rates of 0.77 and 0.78, indicating a recall-oriented bias with more false positives (Table 9). In contrast, AID1239 achieved balanced outcomes with a precision of 0.7729 and a recall of 0.89, resulting in a stable F1-score of 0.7577 and a Cohen’s Kappa of 0.4925, indicating that the structural–activity relationships in the dataset aligned well with the GCN’s feature aggregation. In contrast, AID1578 displayed the lowest performance, with an accuracy of 66.75%, a Cohen’s kappa of 0.34, moderate recall of 0.79, but lower precision of 0.71 and an ROC-AUC of 0.70, indicating limited generalisability to noisy or diffuse graph-activity relationships. From these results, it can be concluded that, although GCN captures relevant graph-level representations, predictive accuracy is constrained by a recall-oriented bias and reduced robustness across datasets. This variability emphasizes need for attention-based or hierarchical GNN variants to improve generalizability and reliability in virtual classification and activity prediction.

Table 9.

Performance metrics of the GCN model

Dataset	Train accuracy (in %)	Test accuracy (in %)	Precision	Recall	ROC-AUC	F1-score	Cohen’s kappa
AID932	72.94	74.50	0.7739	0.9225	0.8083	0.7630	0.49
AID1239	75.06	74.62	0.7729	0.89	0.8134	0.7577	0.4925
AID1578	62.78	66.75	0.7094	0.7850	0.7017	0.6781	0.3350
AID1259354	72.29	74.11	0.7811	0.9260	0.7834	0.7458	0.4822

The AttentiveFP model exhibited strong predictive capability on datasets with well-defined structure–activity relationships, as shown in Table 10. The balanced precision-recall, substantial agreement and test accuracies exceeding 81% on datasets AID932 and AID1239 indicate that AttentiveFP effectively captures structural dependencies resulting in robust classification. AID1578 showed the weakest performance, indicating a sensitivity bias and over-prediction of actives in noisy or weakly correlated patterns. With an accuracy of 78.2% and a Cohen’s Kappa of 0.56, AID1259354 showed moderate performance, characterised by strong recall but reduced discriminative ability with chemically diverse molecules.

Table 10.

Performance evaluation of the AttentiveFP model

Dataset	Train accuracy (in %)	Test accuracy (in %)	Precision	Recall	ROC-AUC	F1-score	Cohen’s kappa
AID932	84.19	81.13	0.8349	0.8995	0.8109	0.817	0.6222
AID1239	86	81.25	0.8851	0.9167	0.8134	0.8108	0.6256
AID1578	79.94	70.5	0.7433	0.8431	0.7053	0.7319	0.4102
AID1259354	80.71	78.22	0.8639	0.9378	0.7819	0.7888	0.5641

Comparative performance analysis of DL-based models

The comparative evaluation of DL architectures, including FFNN, CNN, Vanilla GNN, GCN, and AttentiveFP, across the datasets reveals distinct patterns in predictive accuracy, error tendencies, and dataset-specific sensitivity. FFNN and AttentiveFP demonstrated stable performance with higher test accuracy and ROC-AUC on AID932 and AID1239, indicating effective feature learning and generalisation on moderately complex datasets. CNN exhibited severe overfitting, with high training accuracy and sharp drops in test accuracy, along with very low Cohen’s Kappa. This indicates their limited suitability for chemically heterogeneous datasets. Considering the graph-based models, VanillaGNN and GCN achieved test accuracies between 70 and 76% across all datasets, outperforming CNN in recall and reliability, though slightly lower than FFNN and AttentiveFP on less complex datasets. These findings suggest that graph-based models maintain stability across heterogeneous molecular spaces; FFNN remains competitive when descriptors are chemically informative, and AttentiveFP achieves the most generalizable performance across diverse molecular spaces.

To rigorously evaluate model generalization under chemically realistic conditions, all models were assessed using Murcko scaffold-based splitting, enforcing strict structural disjointness between training, validation, and test sets. Results across all models and datasets are presented in Tables 2–5 of Additional File 1, (see Additional File 1), enabling direct comparison with baseline methods. Under this setting, most models exhibit marginally reduced performance relative to random splits, indicating the influence of structural overlap on prior estimates.

Despite this, AttentiveFP consistently demonstrates superior or competitive performance across datasets, consistent with its performance under random-split evaluation; achieving the highest or near-highest ROC–AUC, PR–AUC, and F1-score, indicating robust generalization and effective learning of transferable structure–activity relationships beyond scaffold-specific patterns. Among baseline methods, FFNN exhibits relatively strong and stable performance under scaffold-based constraints, underscoring the continued relevance of descriptor-based representations. Notably, GCN achieves the highest F1-score in the AID1259354 dataset, likely reflecting dataset-specific structural characteristics rather than consistent generalization.

Overall, AttentiveFP exhibits superior generalization and predictive stability under scaffold-based validation, identifying it as the most reliable model and highlighting the effectiveness of attention-driven graph representations in capturing transferable structure–activity relationships.

Model selection and Robustness validation

Given the results of comparative performance analysis and scaffold-based evaluation, in which the AttentiveFP model exhibited the most reliable performance across diverse datasets, it is necessary to verify whether this performance arises from true structure–activity learning rather than from dataset-specific bias or split-dependent effects. Therefore, the AttentiveFP model was selected for rigorous robustness evaluation, using all datasets with adequate active and inactive molecules, employing four orthogonal validation strategies namely, scaffold generalization, multi-seed stability, y-randomization and imbalance sensitivity analysis, to ensure reliability in realistic screening and deployment settings. The results are provided in Table 6-Additional File 1.

Scaffold generalization

Murcko scaffold-based splitting was employed, and the model performance was evaluated using ROC-AUC, PR-AUC, F1-score, and Cohen’s Kappa to quantify discriminative ability, early retrieval capability, and predictive agreement under strict structural separation. The model maintained strong predictive performance across all datasets, achieving ROC-AUC values of approximately 0.79–0.86 and PR-AUC consistently exceeding 0.90, with Cohen’s Kappa indicating moderate to substantial agreement. These results confirm that the predictive signal arises from learned structure–activity relationships rather than scaffold-level memorization, thereby demonstrating robust generalization under chemically realistic validation conditions.

Multi-seed stability

All experiments were repeated across five fixed random seeds (0–4), and performance metrics were reported as mean ± standard deviation (see Additional File 1). The consistently low dispersion (ROC-AUC σ typically < 0.04; accuracy σ typically < 0.03) indicates stable convergence and high reproducibility across data permutations, confirming that model performance is not driven by stochastic variability.

Y-randomization (leakage control)

To verify that predictive performance was not driven by data leakage, scaffold overlap, or spurious dataset correlations, Y-randomization was performed. Training labels were randomly permuted while preserving the Murcko scaffold-based train/validation/test splits, model architecture, and fixed hyperparameters; validation and test labels were kept unchanged to maintain a valid external evaluation reference. Collapse of ROC-AUC to near-random levels (≈0.48–0.57) and κ ≈ 0 confirmed the absence of data leakage, scaffold memorization, or hyperparameter-induced artifacts.

Imbalance sensitivity analysis

To simulate realistic high-throughput screening conditions, datasets were restructured to a negative-dominant distribution (1:5 active: inactive) by retaining all active compounds and randomly subsampling inactive compounds. Class imbalance was addressed using class-weighted Binary Cross-Entropy loss. Hyperparameters were fixed without retuning to ensure that performance differences reflected data distribution shifts rather than optimization effects, and all experiments were repeated across five independent random seeds. Under this imbalance regime, the model maintained stable discriminatory performance (ROC-AUC ≈ 0.80–0.86; PR-AUC ≈ 0.90–0.93), substantially exceeding the random baseline PR-AUC (≈ 0.17, equal to positive class prevalence). These results confirm that predictive performance is not dependent on artificially balanced datasets and remains robust under screening-relevant negative-dominant conditions.

Ensemble modeling performance analysis

Additionally, AttentiveFP, the best-performing DL model, was combined with the top classical ML model, SVM, and compared to the baseline end-to-end DL configuration. That means, an SVM classifier trained on AttentiveFP-derived molecular embeddings was tested against a fully end-to-end AttentiveFP-Multi-Layer Perceptron (AttentiveFP-MLP) model. The AttentiveFP-SVM approach achieved the best ROC-AUC results across the four datasets, but it showed lower F1 score, accuracy, precision, recall, and Cohen’s kappa than the end-to-end AttentiveFP-MLP model. This shows a clear trade-off between ranking performance and reliable classification accuracy.

To evaluate whether ensembling enhances predictive performance, outputs from five DL models- FFNN, CNN, VanillaGNN, GCN and AttentiveFP- were integrated using three ensemble schemes: Logical AND, Logical OR and Majority Voting. The findings are outlined in Table 11. With a highest precision of 0.9462, the Logical AND approach indicates that the ‘active’ predictions were highly reliable. But the recall dropped to 0.4681, resulting in an F1 score of 0.6263 and a Cohen’s kappa of 0.4563. This approach is well-suited when minimizing false positives is the priority. The Logical OR ensemble achieved high recall (0.9840), identifying almost all active compounds, but with lower precision (0.6446), leading to a higher false-positive rate. While maintaining high ROC-AUC (0.7491) and F1-score (0.7789), the ensemble prioritized sensitivity over balance, making it ideal for screening tasks, where avoiding false negatives is the priority. The Majority Voting ensemble attained the most balanced classification performance with the highest accuracy of 74.88%, and Cohen’s Kappa of 0.4935, supported by precision (0.7584) and recall (0.73), yielded strong F1-score (0.7439) as shown in Table 11. The ROC-AUC (0.7488) of Majority Voting was comparable to the Logical OR scheme, but this ensemble approach delivered more stable and reliable discrimination. By leveraging complementary error patterns across the five models, Majority Voting mitigated the extreme conservatism of Logical AND and the permissiveness of Logical OR, making it the most adaptable ensemble method for chemically diverse datasets.

Table 11.

Performance analysis of the Ensemble model

Evaluation metric	Logical AND approach	Logical OR approach	Majority voting
Test accuracy (in %)	73.62	73.62	74.88
Precision	0.9462	0.6446	0.7584
Recall	0.4681	0.9840	0.73
ROC-AUC score	0.7221	0.7491	0.7488
F1 score	0.6263	0.7789	0.7439
Cohen’s Kappa score	0.4563	0.4849	0.4935

Concept whitening performance analysis

Post-training analysis focused on the CW-aligned latent space to evaluate whether the resulting axes captured underlying physicochemical determinants of biological activity. For each bioassay dataset, the influence of individual physicochemical descriptors was quantified from the first 50 concept-aligned coordinates using three metrics: axis–descriptor correlation, permutation-induced accuracy drop (∆Acc), and classifier weight-based importance. A composite importance index was subsequently computed by combining the ΔAcc and weight-based importance scores.

Descriptor-axis correlations

Figures 9, 10, 11, 12 summarise the ten most strongly correlated axes for each dataset. Across all four datasets, FractionCSP3 consistently emerged as the dominant aligned concept, while ExactMolWt and MolWt recurred among the top axes in three datasets, and LabuteASA, Chi1n, NumHeteroatoms, HeavyAtomCount and AromaticAtomProportion did so in two. These patterns indicate that CW aligns the latent space with chemically interpretable dimensions rather than arbitrary embeddings, and the consistency of descriptor alignment across assays supports the robustness of the approach.

Fig. 9.

Descriptor-axis correlation-AID932

Fig. 10.

Descriptor-axis correlation-AID1239

Fig. 11.

Descriptor-axis correlation-AID1578

Fig. 12.

Descriptor-axis correlation-AID1259354

Permutation ∆Accuracy

Permutation results across the four datasets (Figs. 13, 14, 15, 16) showed no descriptor with consistently high importance, indicating dataset-specific predictive relevance.

Fig. 13.

Permutation ∆Accuracy-AID932

Fig. 14.

Permutation ∆Accuracy-AID1239

Fig. 15.

Permutation ∆Accuracy-AID1578

Fig. 16.

Permutation ∆Accuracy-AID1259354

HallKierAlpha uniquely recurred among the top-ranked features across three datasets, whereas NumValenceElectrons, EState_VSA3, NumHBA, HBD_VSA, PEOE_VSA3 and AromaticAtomProportion did so in two. Near-zero and negative ∆Acc values across several axes indicate compensatory behaviour, in which correlated features preserve performance when a concept-aligned dimension is disrupted.

Weight-based importance

Figures 17, 18, 19, 20 summarize the weight-based descriptor importances. In contrast to permutation ΔAcc, weight rankings showed greater cross-dataset consistency: MinEStateIndex ranked among the top ten features across all datasets, indicating a systematic orientation of the decision boundary along this axis. MolMR, Aromaticity_VSA, AromaticAtomProportion and MinAbsPartialCharge also recurred in at least three datasets, reflecting charge-distribution and aromatic scaffold effects commonly implicated in small-molecule bioactivity.

Fig. 17.

Weight-based importance-AID932

Fig. 18.

Weight-based importance-AID1239

Fig. 19.

Weight-based importance-AID1578

Fig. 20.

Weight-based importance-AID1259354

Composite scoring

A composite importance score was computed as the mean of permutation ΔAcc and weight-based importance; correlation scores were omitted because they are scale-dominant and probe alignment rather than predictive contribution. The composite rankings specific to each dataset are presented in Figs. 21, 22, 23, 24.

Fig. 21.

Composite score-AID932

Fig. 22.

Composite score-AID1239

Fig. 23.

Composite score-AID1578

Fig. 24.

Composite score-AID1259354

A global ranking across all datasets was computed, with the top 25 descriptors summarised in Fig. 25. The top-ranked descriptors resolved into three chemical domains—(i) electronic properties (MinEStateIndex, MinAbsPartialCharge, NumValenceElectrons), (ii) aromaticity and scaffold complexity (AromaticAtomProportion, Aromaticity_VSA, HallKierAlpha), and (iii) steric/topological factors (MolMR, Chi indices, HeavyAtomMolWt), delineating chemically interpretable structure–activity axes rather than opaque latent features.

Fig. 25.

Top-25 most important concepts

Set theoretic stability analysis

To assess the stability of the CW alignment, an intersection analysis was performed to evaluate how frequently descriptors appeared across datasets and evaluation metrics. Results are summarized in Table 12. The top 10 descriptors were identified for each metric category, and the descriptors AromaticAtomProportion, MolMR, HallKierAlpha, and Aromaticity_VSA were found to recur across three datasets.

Table 12.

Intersection analysis

Metric	Descriptor present in all datasets
Correlation	FractionCSP3
∆Accuracy	None
Weight-importance	MinEStateIndex
Composite score	MinEStateIndex

These findings collectively indicate that descriptor–axis alignment is globally stable under correlation analysis, whereas predictive importance is strongly dataset-dependent, as reflected by ΔAcc. At the same time, the weight and composite metrics reveal a conserved chemical subspace that the model exploits consistently across datasets.

Composite-stability feature selection

To derive a stable and biologically meaningful feature subset, a two-stage Composite–Stability Feature Selection method was employed. Initially, features were ranked using a composite score computed as the mean of the permutation-based accuracy drop and the weight-based importance across datasets, and only descriptors above the 75th percentile were retained as high-impact candidates. Second, robustness was assessed via cross-dataset recurrence, retaining only features that appeared in the top 10-ranked set across at least three of the four activity datasets, thereby reducing dataset-specific bias and enforcing cross-domain generalizability.

The process yielded a compact, consistently relevant set of features across all experiments. MinEStateIndex and AromaticAtomProportion were identified as the most prominent descriptors. The selected descriptors align strongly with the model's decision boundaries and generalise across chemical space, validating their role as reliable and interpretable factors for bioactivity classification. Therefore, these results indicate that CW serves both as an interpretability mechanism and as a data-driven framework for feature selection, while also enabling model refinement, making it well-suited for downstream applications such as interpretable generative modeling.

Conclusion

The application of the triads-algorithm, representation, and explainability-together with their downstream variants, enables their effective convergence of chemical and biological spaces into a unified decision space for meaningful results. The study established graphs as the principal direction for the virtual screening pipeline, with molecules represented as graphs, deep graph associations for learning, with the intricacies of ligand-target interactions, attention applied both to edge-atom bonds and to atoms themselves during the final pooling stage, and aligning graph latent embeddings to explainable concepts. Across different stages of the experimental workflow, datasets such as AID932 were identified as exhibiting predominantly linear behaviour with clear class separability, making them analytically simpler. In contrast, more challenging datasets could be effectively distinguished only through subtle substructural patterns captured within both chemical and algorithmic spaces. Among traditional ML approaches, SVM consistently demonstrated strong performance, reflecting its effectiveness in modeling structural-activity relationships. Although the best-performing model varied across individual datasets, the AttentiveFP network achieved the highest overall predictive accuracy, establishing itself as the most effective DL architecture in this study. Among the ensemble strategies evaluated, Majority Voting achieved superior performance with high precision, recall and ROC-AUC values, confirming its suitability for hybrid DL frameworks. In the best-performing AttentiveFP model, the application of Concept Whitening enabled the identification of chemically meaningful concepts driving classification and revealed clear target-specific differences. Notably, MinEStateIndex and AromaticAtomProportion consistently emerged as influential features across immune-related targets.

The study is limited by the evaluation on four immunity-related bioassays, including datasets with a relatively small number of active compounds, and by restricting the Concept Whitening framework to fifty predefined domain-specific descriptors. Despite these limitations, integrating AttentiveFP embeddings with SVM classifiers shows that deep feature learning and classical machine learning can together provide reliable, interpretable, and generalizable predictive performance. These results indicate the potential for future ensemble approaches at both decision and architectural levels.

Future studies should focus on improving dataset diversity using generative AI methods to address data scarcity in DL models. Incorporating protein structural information from experimental studies or predictive models such as AlphaFold, along with binding affinity measurements and molecular dynamics data, into graph-based virtual screening frameworks represents an important research direction. In addition, systematic evaluation of XAI methods beyond CW and investigation of hybrid approaches combining multiple explainability techniques are required to further improve model interpretability and reliability. The development of target-specific virtual screening architectures guided by concept-aligned representations is expected to enhance predictive reliability and support dependable deployment of AI-driven drug discovery systems.

Author contributions

Jyothy V A—conceptualised the study, implemented the computational framework, performed experimental analysis, interpreted the results and prepared the original manuscript. Dr Maya L Pai—supervised the overall study, provided methodological guidance and reviewed the manuscript. Dr Sandesh E PA—contributed to the conceptualisation, result analysis, interpretation, and edited the manuscript. Dr Jaleel U C A—contributed to the data analysis and interpretation, critically reviewed and edited the manuscript.

Funding

Open access funding provided by Amrita Vishwa Vidyapeetham. Open access funding provided by Amrita Vishwa Vidyapeetham.

Data availability

The datasets supporting the conclusions of this article are available in the PubChem repository. AID932 available in https://pubchem.ncbi.nlm.nih.gov/bioassay/932. AID1239 available in https://pubchem.ncbi.nlm.nih.gov/bioassay/1239. AID1578 available in https://pubchem.ncbi.nlm.nih.gov/bioassay/1578. AID1259354 available in https://pubchem.ncbi.nlm.nih.gov/bioassay/1259354.

Declarations

Competing interests

The authors declare no competing interests.