I‑GAT: Interpretable Graph Attention Networks for Ligand Optimization

Abstract

Designing selective and potent ligands for target receptors remains a significant challenge in drug discovery. Computational approaches, particularly advancements in machine learning (ML), offer transformative potential in addressing this challenge. In this study, our goal was to develop a composite ML model capable of predicting ligand selectivity and potency with high accuracy while also providing interpretable insights to guide ligand optimization. To achieve this goal, we first compiled a data set of 757 ligands, including metabotropic glutamate receptor subtype 2 (mGlu2) negative allosteric modulators (NAMs), metabotropic glutamate receptor subtype 3 (mGlu3) NAMs, and nonselective dual mGlu2/3 NAMs from patent filings. In three phases, we developed an ML model with Interpretable Graph Attention (I-GAT) networks for drug optimization. In phase 1, we created a composite model that can accurately predict selectivity and potency metrics by integrating graph architecture with transfer learning methodologies. Our model demonstrated over 97% accuracy in predicting ligand NAM selectivity and upward of 78% accuracy in potency prediction. Phase 2 used attention mechanisms to enhance model interpretability, effectively illuminating the “black box” of ML decision-making. Finally, in phase 3, we utilized attention gradients to intelligently modify known ligands, leading to the design of a novel ligand with predicted superior properties compared to the original. Our approach demonstrates the dual benefits of predictive accuracy and atom-level interpretability, offering a powerful framework for ligand design. When applied to external data, our model matched and, in some cases, exceeded the performance of current state-of-the-art chemistry-focused ML models across multiple data sets. Ultimately, our model has the potential to be adapted to other receptors and molecular properties, paving the way for a more efficient and targeted drug discovery process.

1. Introduction

Selectivity and potency are critical factors in the drug discovery process. Designing ligands that effectively bind to specific target receptors without affecting others is essential for therapeutic efficacy and safety. However, this task is particularly challenging when there is a high structural similarity between the receptors. Selectivity is especially critical for G-protein-coupled receptors (GPCRs), the largest family of signal transduction proteins, − due to their similar structures. Considering that over 30% of Food and Drug Administration (FDA)-approved drugs target exert biological activity through GPCR targeting, selectivity considerations have widespread scientific and clinical implications. , Multiple factors, ranging from the surrounding water networks to differences in binding poses, affect a ligand’s selectivity for GPCRs. While the rules for ligand selectivity can be determined on a case-by-case basis, ,, conventional approaches face limitations when dealing with a large number of candidate ligands. Additionally, in cases where sparser quantities of ligands are available, conserved rules for optimizing target properties may not be visible with manual analysis. These challenges highlight the need for alternative methods that can systematically assess selectivity, paving the way for more efficient and effective drug discovery processes.

This need for improved selectivity assessment is especially relevant in the context of GPCRs, where closely related subtypes play divergent roles but remain difficult to differentiate using conventional ligand design. Among the GPCR family, metabotropic glutamate receptor subtype 2 (mGlu2) and metabotropic glutamate receptor subtype 3 (mGlu3) are promising targets for neurological diseases. In general, mGlu2 activation contributes to neuronal toxicity. Its genetic deletion can improve outcomes after toxic insults. , On the other hand, mGlu3 activation played a neuroprotective role, encouraging antiamyloidogenic functions within astrocytes, which was evidenced through in vivo and in vitro studies in the context of Alzheimer’s disease. ,− Interestingly, simultaneous activation of mGlu2 and mGlu3 signal transduction can be neuroprotective. , Considering the different functions that these receptors modulate, the identification of selective ligands for either the mGlu2 or mGlu3 receptor could be beneficial in the treatment of various neurological conditions.

The orthosteric binding pockets are highly conserved between the mGlu2 and mGlu3 receptors, making the design of selective ligands specific to the orthosteric site extremely challenging. , Indeed, the structural similarity of mGlu2 and mGlu3 extends to functional similarity, as studies have shown that identical mutations in the binding sites of these receptors lead to analogous effects on receptor function upon metabotropic group II ligand binding. These traits create difficulty when designing receptor-specific ligands, for researchers who want to design ligands that cater to the specific attributes of mGlu2 or mGlu3 signal transduction. However, both mGlu2 and mGlu3 receptors possess distinct allosteric sites in addition to their orthosteric sites. This distinction has enabled the design of negative allosteric modulators (NAMs) that are selective for either mGlu2 or mGlu3, representing an improvement over the previously nonselective dual mGlu2/3 NAMs. − Therefore, drug design targeting differences in the mGlu2 and mGlu3 allosteric sites has facilitated advancements in this GPCR family by enabling the selective modulation of two receptors with distinct physiological roles. Despite recent advances in ligand development, the existing library of compounds remains limited. At this time, there are no efficient, scalable methods that can allow researchers to screen large volumes of mGlu2 or mGlu3 candidate ligands for both potency and selectivity. This gap underscores the need for computational approaches to perform this task with greater precision, at scale. To address this challenge, we sought to leverage advances in machine learning (ML) and artificial intelligence (AI).

One promising direction lies in the use of Graph Neural Networks (GNNs), a class of deep learning architectures that has rapidly gained traction since their introduction in 2005. Derivatives of this architecture have been implemented to process a wide range of biological stimuli, ranging from brain networks and medical images to medical text. − Due to their capacity to represent connectivity, GNNs may more natively represent molecules through the perspective of chemistry, when compared to other ML representations. Conversely, GNN-based models often require more training time than descriptor-based models, dispelling the thought that GNN implementation is a “silver bullet” for biological tasks. However, the recent development of certain GNN architectures, such as Graph Attention Networks (GATs), often exceeded descriptor-based model performance, suggesting the promising application of GNN in the field of drug discovery with task-specific implementation and optimal architecture.

The adaptation of GNNs to chemistry represents a natural progression of this subfamily of ML architecture, supported by the abundance of successful GNN applications in chemistry. Like other ML architectures, graph-based models can be used for classification and regression. For instance, GNNs have been used to predict toxicity profiles across a variety of data sets. , They can also be used to accurately predict molecular properties, serving as prime candidates for use in regression tasks such as the prediction of water solubility, hydration free energy in water, lipophilicity, and glass-transition temperature. − Additionally, GNNs can be combined with other architectures, like Convolutional Neural Networks (CNNs) to predict complex phenomena such as compound–protein interactions. Ultimately, the application of GNNs to chemistry is a promising scientific subfield, especially when considering the unique qualities, such as interpretability.

As interest in interpretability grows, several models within the GNN subfamily have recently been developed. For example, GNNExplainer leverages neighborhood aggregation to identify important graph features. Another methodology seeks to pair Graph Convolutional Network (GCN) architecture with established methods of evaluating feature salience, such as Local Interpretable Model-Agnostic Explanations (LIME), Gradient-Based Saliency Maps, and Gradient-Weighted Class Activation Mapping (Grad-CAM). Although these methods were originally suited to CNN architectures, the authors indicate that their approach can apply to convolutions with graph data.

Most currently available ML models seek to evaluate the importance of certain atoms on target properties but encounter multiple limitations, including a lack of atomic-level resolution and reliable interpretability. For example, a GNN model used Grad-CAM to correlate atomic structure to X-ray absorption spectra. However, only four node features could be incorporated, and the approach had to work within the limitations of Grad-CAM integration. An interpretable GCN derivative sought to evaluate the structural contribution to chromatographic retention using saliency maps; however, interpretability was limited to substructure-level resolution. Structural relation to activity was elucidated with another GCN model, which assessed the defined substructures without atomic-level resolution. A similar lack of atomic level resolution limitation is encountered in another work corroborating substructure importance for graph architecture. Another approach used Shapley approximation with the Monte Carlo sampling; however, the approach demonstrated inconsistency between runs. There are also indirect methods for attaining interpretability. For example, substructure masking can be used to identify the most important substructures to a particular prediction, only affording substructure-level resolution. In summary, most of these prior models do not offer atomic-level resolution. They work with various types of GCNs with older GNN architectures. Few leverage GAT architecture for interpretable models. In this work, we sought to build a GAT model while offering attention-based interpretability methods to deliver atomic-resolution explanations, a feature rarely seen in molecular GNNs.

Our study has three phases. In phase 1, we developed a multifunctional prediction GAT model with three tasks. The first task aimed to accurately predict selectivity, as a classification problem. The second task functioned to predict potency (IC50) values, which was a regression output. The third task also predicted potency but classified each ligand as “high” or “low” potency. In phase 2, we further developed the multifunctional model by integrating interpretability, allowing us to examine ligands to identify strengths and weaknesses. In phase 3, we leveraged this interpretable model for drug optimization. This approach allowed for the design of a novel ligand with enhanced mGlu2 NAM selectivity and improved potency, as evaluated by computational methods. To assess the broader applicability of our framework, we conducted comprehensive comparisons against state-of-the-art machine learning models across multiple benchmark data sets. We were able to adapt our model to various data sets with different features, and we anticipate that future researchers will be able to build upon this approach for their particular data sets. An overview of this study is depicted in Figure .

1.

General overview of three phases of this study. Phase 1 focused on training the model to accurately predict ligand properties, including selectivity and potency. After the model was successfully trained, it was interrogated during phase 2 to identify the atoms or functional groups that negatively affect selectivity/potency. Subsequently, in phase 3, these regions were modified to optimize the ligand. The new ligand was evaluated to determine if the modifications improved the selectivity and/or potency of the ligand.

2. Materials and Methods

2.1. Node Feature Selection

To layer information about chemical structures, a specialized encoding scheme was adopted and node features were layered as inputs into graph architecture. To tailor inputs to GNN adaptation, most input features were one-hot encoded, which converts categorical data into a numerical vector where each category is represented by a “1” (indicating the presence of the category) and 0’s elsewhere. One-hot encoding allows for ease of computation for aggregation functions that are typical of GNN architectures, as features are represented in a fashion that is favorable to mathematical operations. Of note, this encoding can be paired with scaled vector values, allowing for the transmission of categorical values, which can then be paired with other vector values. For each constituent atom, information about atomic identity was encoded, from the list of {Carbon (C), Nitrogen (N), Oxygen (O), Phosphorus (P), Sulfur (S), Fluorine (F), Chlorine (Cl), Bromine (Br), and Iodine (I)}. The scaled atomic weight, the total number of hydrogens bound to a target atom, implicit and explicit valence were also included. Covalent radius, van der Waals radius, formal charge, and partial charges were calculated and appended as node features. Finally, degree, aromaticity, and hybridization were also encoded as features. Ultimately, the encoding of the aforementioned molecular attributes allowed for the stacking of 54 node features, which were used to create a distinct data object for each input ligand. More information about the rationale for these features is hosted in the discussion section.

2.2. Data Set Creation

As a list of mGlu2 and/or mGlu3 NAMs, along with the Simplified Molecular Input Line Entry System (SMILES), is unavailable, we sought to generate this data for ML applications. The two targets designated for selectivity classification were mGlu2 or mGlu3, meaning that an input ligand could be deemed as displaying selectivity as a NAM for one receptor. Thus, data was compiled from a Merck patent and five Vanderbilt patents to generate a data set of 351 mGlu2 NAMs and 309 mGlu3 NAMs. − As potency values were available for this data, the IC50 value distribution of the data set is visualized in Figure S1. All IC50 values were then transformed using a log base 10 (log10) scale and subsequently normalized within their respective classes (mGlu2 or mGlu3) based on their class-specific minimum and maximum values.

One additional consideration was that the algorithm may encounter ligands that function as NAMs while displaying nonselectivity for mGlu2 and mGlu3 receptors at the allosteric site. Therefore, these dual mGlu2/3 NAMs must be considered, and patents were referenced for this purpose. Unfortunately, as some patents report differing methodologies and measurements, they could not be included. However, there was one particular patent that was included, US10597367B2. EC50 value reporting was incomplete in some cases; many samples used varying or missing time points for the assays, precluding a head-to-head comparison. Due to the lack of data, we eliminated samples with missing data points. Additionally, some values were out of the measured range, eliminating the possibility of including potency metrics. After these samples were excluded, selectivity could be extracted from the remaining 97 compounds. Compounds that demonstrated a 10-fold potency difference for one receptor versus the other were considered “selective”. This led to the addition of 13 mGlu2 selective NAM compounds and 1 mGlu3 selective NAM compound. Compounds that did not achieve this cutoff were considered “non-selective” and labeled as nonselective dual mGlu2/3 NAMs (n = 83). With all patents combined, the final count amounted to 364 mGlu2 NAMs, 310 mGlu3 NAMs, and 83 dual mGlu2/3 NAMs. The compiled data set of 757 samples will be available in the supplementary section within Table S13.

2.3. Data Fractioning

An independent validation set was created using 10% of the data designated for testing, with 85% for training and 5% for model validation. A second set was created after excluding 10% of the data designated for testing, for the purpose of performing cross-validation. The StratifiedKFold module from scikit-learn was used to split this data into 75% training, 5% validation, and 20% testing over five folds. This ensured randomization while allowing future researchers to replicate this work.

The cross-validation process is specifically included to assess the model’s robustness and provides a reliable measure of performance across multiple runs. In contrast, the independent test set is reserved for final model evaluation, ensuring an unbiased assessment of generalization. Importantly, the test set was never used during training or cross-validation, which prevents the risk of data leakage during performance quantification.

2.4. Loss Functions

For all classification tasks, the loss function was BCEWithLogitsLoss, as detailed in eq .

$$\mathrm{BCEWithLogitsLoss}=-\frac{1}{\mathit{n}}\sum _{\mathit{i}=1}^{\mathit{n}}{\mathit{y}}_{\mathit{i}}·\log (\sigma ({\mathit{x}}_{\mathit{i}}))+(1-{\mathit{y}}_{\mathit{i}})·\log (1-\sigma ({\mathit{x}}_{\mathit{i}}))$$ 1

For the regression task, Root Mean Squared Error (RMSE) was used, with a slight modification detailed in eq .

$$\mathrm{Modified\; RMSE\; loss}=\sqrt{\frac{\sum _{\mathit{i}=1}^{\mathit{n}}{({\mathit{y}}_{\mathrm{true}}-{\mathit{y}}_{\mathrm{pred}})}^2}{{\mathit{y}}_{\mathrm{pred}}+0.05}}$$ 2

The root mean squared error between y _true and y _pred was divided by y _pred, which served as a scaling mechanism. Additionally, to prevent division by zero, a small bias value of 0.05 was added to the denominator.

2.5. Brief Model Outline

The I-GAT framework was implemented as a sequentially informed GAT-based architecture across three phases. In phase 1, the model was trained to classify ligand selectivity. This selectivity model consisted of multiple graph attention layers whose trained weights were saved for later transfer. In phase 2, a regression model was developed to predict IC50 values (potency). To inform this model with prior knowledge, we initialized the regression architecture using the trained weights from the previously trained selectivity model. This form of transfer learning allowed the regression model to build upon learned selectivity patterns. In phase 3, the goal was to classify ligands into high-potency or low-potency categories using both selectivity and potency information. To achieve this, we constructed a dual-branch architecture that accepted the weights from the selectivity network (phase 1), and from the regression network (phase 2). Both branches passed through GAT layers and global mean pooling before their outputs were concatenated and fed into a multilayer perceptron (MLP). This allowed the model to jointly consider both receptor selectivity and potency regression outputs for the final classification task. This modular approach ultimately allowed for the transmission of learned chemical insights across multiple prediction tasks. More detail regarding the precise architecture is included in the Results section, along with streamlined diagrams.

2.6. Attention Gradient Extraction

After phase 1 was completed, in order to prepare the model for attention visualization in subsequent phases, 100% of the mGlu2, mGlu3, and dual mGlu2/3 NAM data set was designated for training; after training, the mode state was saved. Thereafter, one ligand at a time was selected for attention visualization, and node features were generated in a fashion identical to the preprocessing workflow. For each particular ligand, the output was predicted using the weights of the trained model, and the output node was examined to confirm that correct classification occurred. Thereafter, a backward pass was made of the loaded model with respect to the correctly indexed target node of interest. This allowed for the weighing of each individual node feature, for each particular node. Note, as many of the node features were one-hot encoded in a manner that is incompatible with human comprehension, the average values of these features were appended and mapped to the respective nodes. Thus, for each node, the average gradient indicated the importance of that particular atom to the final prediction.

2.7. Redesign Process

After identifying problem areas, the SMILES string of the selected molecule was inputted onto JSME Molecule Editor hosted on https://www.cheminfo.org/. A host of synthetically feasible changes to the structure could then be made. For each change, the SMILES string of the edited molecule was recorded and thereafter evaluated using the machine learning approach.

2.8. Model Comparisons for Regression

To allow for direct comparisons of regression performance, we replicated the molecular solubility prediction framework from Ahmad et al. More specifically, we used the 9943 compounds derived from the Cui et al. data set as the training set. Then, we evaluated the trained model’s capacity to generalize by performing predictions on an independent test set (also from Cui et al.), which consisted of 62 experimentally validated anticancer compounds. For each molecule, we used its canonical SMILES representation along with its aqueous solubility value (log S, which is measured in log mol/L). Molecules were parsed using RDKit and converted into molecular graphs, using the methods previously described in the Node Feature Selection section. However, we applied one-hot encoding to capture the 31 unique atoms that are present in this data set. Compared to the mGlu2/3 data set, one fewer formal charge category was required, leading to 75 node features per ligand. The I-GAT model architecture previously used for potency (IC50) regression was then trained on the Cui et al. data set and evaluated on the independent test set of anticancer compounds to evaluate the model’s capacity to generalize. A total of three independent runs with random initialization were conducted to ensure that performance was replicable.

2.9. Model Comparisons for Classification

In order to compare the classification performance of our model with current state-of-the-art chemistry models, we adopted the data sets used in Xu et al. We loaded four MoleculeNet benchmark databases, namely the Blood-Brain Barrier Penetration (BBBP), β-secretase enzyme (BACE), Human Immunodeficiency Virus (HIV), and Clinical toxicity (ClinTox) data sets. Additionally, we applied the same methodology for data loading as reported in the Xu et al. paper, with an 80:10:10 ratio for training, validation, and testing sets. The same preprocessing from the regression task was used to generate canonical SMILES representations along with the corresponding classification output. Hyperparameter tuning was conducted using the Optuna module, allowing for the identification of optimal hyperparameter values such as hidden neuron count, number of attention heads, number of GAT layers, dropout rate, learning rate, and weight decay. Once optimal configurations were identified, we retrained the best-performing model for 500 epochs. Final performance was assessed using the ROC-AUC metric, allowing for a direct comparison against the state-of-the-art models reported by Xu et al. As with the regression task, three independent runs with random initialization were used to ensure that classification performance was reproducible and to evaluate model stability.

3. Results

3.1. Data Preparation

Model inputs utilized a compiled data set of mGlu2 NAMs, mGlu3 NAMs, and dual mGlu2/3 NAMs. Details of the molecular representation used to convert molecular structures to model-interpretable encodings are included in the methods section. For selectivity classification (task 1), 757 total ligands were used. However, as not all compounds had IC50 values associated with them within the original patents from which they were derived, only 660 ligands could be included for both potency regression (task 2) and potency classification (task 3). More details about the origin of the structures and data fractioning are included in the Materials and Methods section.

3.2. Phase 1: Prediction Model for Ligand Selectivity and Potency Metrics

3.2.1. Phase 1, Task 1: Selectivity Classification

To assess a particular drug candidate, it is critical to know if it binds to off-target receptor(s). Similarly, to develop a targeted neurotherapeutic for use in specific pathologies, such as Alzheimer’s disease treatment, it is important to identify selective NAM ligands for the mGlu2 receptor. In task 1, our goal was to develop a model to accurately classify the receptor selectivity of the input ligands.

From our available data, we assigned output categories depending on the selectivity of ligands, which were mGlu2 NAMs, mGlu3 NAMs, or dual mGlu2/3 NAMs. We designed the first segment of the model with three sequential graph attention layers. A simplified representation of model architecture is depicted in Figure . After each distinct layer, 20% of the nodes were subject to dropout as a form of regularization. This was complemented by adding three output nodes corresponding to each possible output category. As the model predictions are outputted as logits, this method allows for correspondence with input labels.

2.

Simplified three-layer GAT model architecture tailored to task 1.

From the available data, an independent validation set was created, with 10% of the ligands (n = 75) designated for testing, 85% (n = 644) for training, and 5% (n = 38) for model validation. A second cross-validation set was created after excluding the 10% of data designated for testing. Using the StratifiedKFold, this remaining data (n = 682 ligands) was split into 75% training, 5% validation, and 20% testing over five folds for cross-validation. More detail pertaining to the data fractioning is included in the methods section. After training for 250 epochs, evaluation of the training loss and validation loss curves confirmed that overfitting did not occur. The graph of training loss and validation loss, for the independent validation set, is depicted in Figure S2. Average selectivity accuracy across all folds reached 97.65 ± 1.21%, with 98.81% accuracy for the independent validation data set, indicating that our model demonstrated high accuracy in the selectivity classification task. The selectivity classification results for the five folds and the independent validation data set are compiled in Table S1, indicating that results were reproducible across folds.

Confusion Matrices, depicted in Figure , along with the classification reports (Tables S2 and S3) were used to further evaluate the model for the 5-fold cross-validation, and for the independent validation set, respectively. This analysis serves to indicate the number of misclassifications for each target class. We intentionally included dual mGlu2/3 NAMs to “confuse” the model, however, the performance was still satisfactory for the cross-validation data set (Figure A,B). Similar accuracy was approached by the independent validation set as well, and only two ligands were misclassified out of 75 tested ligands (Figure C,D). Ultimately, these findings demonstrate that this model segment was capable of accurately identifying input ligand NAM selectivity for mGlu2/mGlu3 receptors.

3.

Confusion matrices are depicted for the selectivity classification task. (A) Numerical values for the 5-fold cross-validation set, (B) with values normalized across the “true” category for the 5-fold cross-validation set. (C) Numerical values for the independent validation set, (D) with values normalized across the “true” category for the independent validation set.

3.2.2. Phase 1, Task 2: IC50 Prediction

IC50, or half maximal inhibitory concentration, is a measure to indicate how much of a particular ligand is needed to inhibit a given biological process or biological component by 50%. It is often used to assess the potency of ligands. The IC50 value is inversely related to the potency of the ligand; thus, ligands with lower IC50 values are considered to be more potent. Measuring the IC50 values for a series of compounds in experiments is expensive and time-consuming. Therefore, it is crucial to develop a model to predict the IC50 values of ligands, enabling comparisons of promising candidates more efficiently.

To perform this task, only mGlu2 NAMs and mGlu3 NAMs with IC50 values were included. As some ligands were not documented with their corresponding IC50 values, they were excluded in this task, resulting in only n = 660 ligands for analysis. From this corpus, an independent validation set was created, with 10% of the ligands (n = 66) designated for testing, 85% (n = 561) for training, and 5% (n = 33) for model validation. A second cross-validation set was created after excluding the 10% of data designated for testing. Using the StratifiedKFold, these remaining ligands (n = 594 ligands) were split into 75% training, 5% validation, and 20% testing over five folds. More detail pertaining to the data fractioning is included in the methods section. Two main regression models were created. One “non-informed” model functioned solely to predict IC50 values, using mGlu2 NAMs and mGlu3 NAM data as inputs. Another “informed” model transferred the lessons learned from the selectivity task to the regression task. This was achieved through transfer learning methodology, by loading the weights of the three-layer selectivity model onto a new four-layer model tasked with IC50 prediction. For both informed and noninformed models, stable gradient descent was confirmed to have occurred over the course of 400 epochs. Training and validation loss curves for the independent validation set are represented by Figure 3, indicating that overfitting did not occur. The simplified architecture of the informed model is depicted in Figure . This arrangement effectively allows for the execution of a potency regression prediction via an IC50 value using prior knowledge from the selectivity task.

4.

Simplified GAT model architecture tailored to task 2. The weights of the selectivity classification network were transferred onto the four-layer model subcomponents to “inform” the model.

To evaluate the predictive capacity of our model about IC50 values, we used mean absolute percentage error (MAPE) to measure the prediction accuracy. This allowed for the measurement of the average percentage difference between predicted IC50 values and actual IC50 values, for the available data set. Here we denoted the measured in vitro experimental IC50 values recorded in the original patent filings as the “actual” values. For the 5-fold cross-validation data set, models that were informed with selectivity weights yielded an average MAPE of 76.33 ± 11.73%. Meanwhile, MAPE for regression prediction in the cases where the models were not informed of selectivity averaged 103.15 ± 37.14%, yielding a greater percentage error than the informed prediction. This trend was also seen after the evaluation of the independent validation data set, where the regression prediction informed with selectivity weights yielded a MAPE of 75.44%. Meanwhile, the noninformed model yielded a higher MAPE of 91.12%. A side-by-side comparison is summarized Table S4, further detailing the superiority of the model that received more information, through the loading of selectivity weights.

Regression of the predicted vs actual log-transformed values is depicted in Figure , allowing for visualization of the value distributions. Figure S4 details regression graphs of the predicted vs actual IC50 values for the compiled five folds, and for the independent validation data set. When examining the data, the normalized arrays displayed significant correlation for the 5-fold cross-validation data set. Additionally, the non-normalized IC50 values demonstrated significant correlation for the independent validation data set when assessed with Pearson’s correlation (Table S5). These results indicate that the informed model can accurately predict the potency of our input ligands as quantifiable IC50 values through the regression task.

5.

Top graph depicts the regression output of the GAT model (task 2) comparing actual versus predicted normalized IC50 values after log transform, compiled across all five folds. The bottom graph depicts the regression output of the GAT model (task 2) comparing actual versus predicted normalized IC50 values after log transform for the independent validation data set.

3.2.3. Phase 1, Task 3: Potency Classification

The function of task 3 is to classify ligands based on potency, with the intention of outputting “high” or “low” as a binary measure of potency. This would allow for easier filtering out low potency ligands when applied to large-volume drug discovery. Additionally, accuracy values are easier to interpret, when compared to the error metrics of regression values.

To cater to this goal, the data was divided into high-potency or low-potency compounds based on median cutoffs. For the mGlu2 ligands, any molecule with an actual IC50 less than the median value of the data set, which was 74 nM, was designated as high potency (n = 175); values at or above this cutoff were designated as low potency ligands (n = 176). A separate median value of 1050 nM was the defining cutoff for the mGlu3 data set, yielding n = 154 high potency mGlu3 ligands and n = 155 low potency mGlu3 ligands. To leverage information learned from selectivity classification and potency regression, we designated two segments, the selectivity segment and the regression segment. The selectivity segment appended weights from the selectivity tasks, retaining three GAT-based layers, whereas the regression task received knowledge from the regression task and was characterized by four layers, aligning with the original model cores. At the terminus of each segment, global mean pool was used to return graph-level outputs in a batch-wise format. To incorporate and resolve information transferred from tasks 1 and 2, a multilayer perceptron (MLP) block was designated. The final model architecture is simplified in Figure .

6.

Simplified GAT model architecture tailored to task 3. The weights of the selectivity classification network (left side) and of the potency regression network (right side) were transferred onto the model core. The MLP subcomponent (gray nodes) was used to process both data streams.

After training, the model was evaluated to determine the percentage of correct predictions, for high or low potency classes. The model that was informed with selectivity and potency weights reached 78.95 ± 3.51% overall accuracy over five folds. This approach yielded superior accuracy when compared with the model that simply classified compounds as high or low potency without any selectivity or potency information (noninformed model), which resulted in 75.93 ± 1.66% classification accuracy. For the independent validation data set, the informed model again yielded a higher potency classification accuracy (81.82%), when compared to the accuracy of the noninformed model (75.76% accuracy). This potency classification data is summarized Table S6.

Confusion matrices are also used to summarize the data in Figure , along with the classification reports Tables S7 and S8. Across five folds, the model yielded 78.95 ± 3.51% accuracy in correctly distinguishing high-potency versus low-potency ligands using the informed model. For the independent validation data set, the model yielded 81.82% accuracy in correctly distinguishing high-potency versus low-potency ligands for the informed model. However, if the same architecture was used to perform a classification prediction using the same cutoff values for the same data set, accuracy was reduced for the naïve model. For the cross-validation and independent validation data sets, potency classification accuracies only reached 75.93 ± 1.66% and 75.76%, respectively. Thus, the transfer of selectivity and potency regression information was necessary to increase potency classification (task 3) accuracy. Overall, these results indicate that the aforementioned transfer learning approach allowed for accurate prediction of potency as discrete “high” or “low” values, which will be helpful in the rapid screening of large volumes of compounds regardless of differing IC50 distributions.

7.

Confusion matrices are depicted for the potency classification task. (A) Numerical values for the 5-fold cross-validation set, (B) with values normalized across the “true” category for the 5-fold cross-validation set. (C) Numerical values for the independent validation set, (D) with values normalized across the “true” category for the independent validation set.

In summary, during phase 1, the model was designed to perform three tasks. Task 1 aimed to predict selectivity classification, and the model yielded excellent performance. In task 2, the model leveraged selectivity information to perform an IC50 regression prediction with low measured error. Finally, task 3 utilized information from selectivity classification and potency regression to classify compounds as high or low potency, with acceptable accuracy.

3.3. Phase 2: Attention Visualization

In phase 1, we developed the model to accurately predict the properties including selectivity and potency. In phase 2, further analysis was conducted to understand what the machine learning model had learned. The goal was to design an algorithm to determine the contribution of each atom or functional group to the prediction results. The network was constructed to enable the operator to train the model and examine the composite gradients across the entire network. Proceeding onward, all available data was used for training, with all three tasks performed sequentially as previously described. Model states were then saved, and evaluation was performed for task 1 output nodes, which correlated to selectivity prediction (Table S9). Additionally, analysis was performed on task 3 output nodes which represented the potency classification prediction, as summarized in Table S10. For both task 1 and task 3, a backward pass was performed with respect to the target node of interest. This allowed the resulting gradients to be collected with respect to these target nodes for their respective selectivity (task 1) or potency (task 3) functions. This allowed for weighting of each individual node feature, for each particular node. More details on gradient extraction are provided in the Materials and Methods section.

As an example, the mGlu2 ligand A041 was analyzed to understand the contribution of each atom or functional group to selectivity, as shown in Figure . There are some positive gradients superimposed on some atom locations, indicating that those specific atoms contribute favorably to the classification of the molecule as a mGlu2 NAM ligand. Additionally, within this ligand, A041, the negative gradients indicate atoms that negatively contribute to this structure’s classification as a mGlu2 ligand. As an example, the carbon atom with a negative gradient of −0.14 (circled in purple) denotes the atom that most negatively contributes to the prediction of mGlu2 NAM selectivity. However, another nearby carbon encircled in red is also a detrimental contributor to selectivity according to the model’s knowledge base. In addition, a distant nitrogen atom (circled in orange) also contributes negatively to the selectivity prediction.

8.

MGlu2 ligand A041 is analyzed for selectivity interpretability. (A) The original structure is depicted. (B) The resulting selectivity gradient evaluation reveals the three most “problematic” atoms, designated in purple, red, and orange highlights.

In a similar manner to attention visualization for selectivity, an examination of the relevant gradients for potency was performed, with atom-level resolution. The potency classification (task 3) could be adapted in a straightforward manner to offer interpretability. The example analysis of A041 is depicted in Figure . As indicated in the blue highlight, the same carbon identified in the selectivity task also appears to contribute negatively to high potency. While this multitask approach separately allows for the identification of atoms that negatively contribute to target selectivity and high potency, it also enables the identification of common “problem areas” for both properties. If common “problem areas” could be identified in selected molecules, perhaps they could be replaced. In summary, we leveraged interpretability mechanisms of our graph attention network in order to further explore atomic contributions to potency.

9.

MGlu2 ligand A041 is analyzed for potency interpretability. (A) The original structure is depicted. (B) The resulting potency gradient evaluation reveals the most “problematic” atom, circumscribed in blue highlight.

3.4. Phase 3: Intelligent Optimization of Existing Ligands

With successful visualization of both selectivity and potency gradients in phase 2, we sought to optimize the existing ligands to design more potent and selective compounds in Phase 3. Here we hypothesize that improving the “worst” performing area of the initial molecule, which displayed the most negative gradient, would improve the target property if the area displayed a less negative gradient after the modification. To test this hypothesis, we analyzed pairs of ligands from our data set, leveraging the already existing in vitro data.

To investigate if selectivity could be improved by improving negative gradient, we analyzed two ligands from our data set, a nonselective ligand (BOTH112) and a mGlu2 selective ligand (BOTH116). For BOTH112, the mGlu2 EC₅₀/mGlu3 EC₅₀ ratio was 0.466, indicating a relatively nonselective NAM function. For BOTH116, the mGlu2 EC₅₀/mGlu3 EC₅₀ ratio was less than 0.027, indicative selectivity for mGlu2. When analyzing their gradients, we found that a nitrogen atom presented with the most negative gradient (−0.31, red highlight in Figure ) in BOTH 112. When the nitrogen atom presenting with the most negative gradient in BOTH 112 is substituted with carbon in BOTH116, the gradient became less negative at that site (−0.10, blue highlight in Figure ), suggesting attenuating the negativity of gradient may improve mGlu2 selectivity. We further analyzed BOTH116 with another nonselective ligand BOTH114 (mGlu2 EC₅₀/mGlu3 EC₅₀ ratio of 0.29). Again, the improvement of the negative gradient surrounding the nitrogen atom of BOTH114, into the less negative carbon atom of BOTH116, is correlated with improved mGlu2 selectivity, as illustrated Figure S5. Ultimately, these findings imply that improving the most negative gradient site might be used as a strategy to improve selectivity.

10.

Top panel depicts the effect of the structural modification without superimposed selectivity gradients; the bottom panel depicts the structures with superimposed potency gradients. The negatively performing region of the starting molecule, BOTH112, is highlighted in red. After the substitution of the nitrogen with the carbon atom (blue highlight), the target areas show a favorable gradient change. This correlated to a favorable change in predicted selectivity for the target area in ligand BOTH116.

Next, we sought to evaluate the capacity of this model to perform potency optimization. Here, we analyzed two ligands with similar structures but significantly different IC50 values, A273 and A275 (Figure ). The primary structural difference between these two compounds lies in the functional group at a specific location: A273 contains a nitrile group, whereas A275 features an amide group at the same position. The nitrile group in A273 (highlighted in red in Figure ) exhibits a negative gradient, indicating a detrimental contribution to the prediction of high ligand potency. In contrast, the amide group in A275 (highlighted in blue in Figure ) displays a less negative gradient, suggesting it reduces the adverse effect on potency. As depicted in Figure , substituting the nitrile group in A273 with the amide group of A275 attenuates the most negative gradient in the molecule. Indeed, this replacement drastically improved the IC50 value from 173 nM (A273) to 7 nM (A275), implying that modifying the region with the most negative gradient might improve potency. Again, it is important to note that the only difference between both molecules is the replacement of the nitrile group with the amide group. We further tested this hypothesis with another ligand pair with similar structures but with different IC50 values, A026 and A041. A026 is a low-potency compound with an IC50 of 416 nM, whereas A041 has a much higher potency with an IC50 of 12 nM. As depicted in Figure S6, the carbon atom in a nitrile group of A026 appears to have the most negative gradient (highlighted in red), contributing negatively to the high-potency prediction. When this “poor-performing” group of A026 was replaced with an amide group in the ligand A041, the gradient on the carbon atom became positive (highlighted in blue in Figure S6), again suggesting that improving the most negative gradient might be an effective strategy to improve potency.

11.

Potency optimization for the ligand A273. The top panel depicts the effect of the structural modification without superimposed potency gradients; the bottom panel depicts the structures with superimposed potency gradients. The negatively performing region of the starting molecule, A273, is highlighted in red. After the substitution of the nitrile group with the amide group (blue highlight), the target areas show a favorable gradient change. This correlated to a favorable change in predicted potency for the target area in ligand A275.

As seen by these examples, there does appear to be an improvement in potency when nitrile groups are replaced with amide groups, for mGlu2 NAMs. It is also interesting to note that for the 14 mGlu2 NAMs with nitrile groups, the mean IC50 value is 413.57 nM, with a median value of 157.50 nM. These values are much higher than the mean IC50 value for the mGlu2 data set (78.65 nM), and the median value of 74 nM. This data is summarized in Table S2.

This correlation of underlying gradients with potency was not restricted to mGlu2 ligands. We randomly selected a low-potency mGlu3 NAM ligand Be53 (IC50 of 4270 nM) and identified its higher-potency counterpart Be38 (IC50 of 980 nM). A similar trend was observed, and the improvement of mGlu3 IC50 correlated with target gradient improvement. As illustrated in Figure S7, the “worst” performing area was a nitrogen atom (with a gradient value of −0.19) in Be53. Substitution to a carbon atom changed the gradient to 0.00, which is correlated with a drastic potency improvement. In another example, the low potency mGlu3 ligand, BEM21, with an IC50 of 4070 nM, was assessed. If the trifluoromethoxy group in this molecule had been replaced with the nitrile group seen in BEM18, the actual IC50 would have improved to 784 nM. Concordantly, the changed regions show favorable gradient change (Figure S8), becoming less negative with this process. In summary, for both mGlu2 and mGlu3 ligands, leveraging underlying gradients allows for localization of the “worst performing” atoms or groups. Additionally, making favorable changes according to underlying gradients allows for the improvement of IC50.

Furthermore, we investigated the reproducible interpretation of the model. We conducted three separate model runs with random initialization and examined the mGlu2 NAM A041. While the selectivity gradients showed slight variance when the model was run in triplicates, the values were still very similar (Figure S9). Additionally, the gradients with the most negative values were localized to the same regions for each run. Furthermore, the potency gradients were similar between the triplicate runs, indicating the stability of the model (Figure S10). This indicates a reliable and replicable interpretation has been reached, highlighting the strength of this model through its reproducible interpretability.

To further expand the applicability of the interpretable GAT model, we sought to redesign an existing ligand into a novel one to enhance its selectivity and potency. We examined the list of mGlu2 ligands with nitrile groups (Table S12) and picked ligand A053 as our example. A053 displayed the highest mGlu2 IC50 value of 3059 nM, indicating that this was the least potent of the group. Selectivity interpretability was examined using the methods described previously. We identified a carbon atom in a nitrile group hosting the most negative gradient. In a similar manner, potency interpretability was also examined. Yet again, this carbon atom in the nitrile group yielding the most negative gradient. Indeed, for both selectivity and potency, this carbon atom that was part of the nitrile group was the “problem area”. Thereafter, we substituted the nitrile group with the amide group, a substitution that had previously yielded more potent compounds, as indicated by the compounds within the existing database. When analyzed for selectivity, the substitution appeared to be more favorable, according to the ML model’s knowledge base, as indicated by the less negative gradients (Figure ). Additionally, this change yielded gradients of less negative magnitude for the potency prediction, which implicates the generation of a higher-potency compound according to the ML model’s knowledge base (Figure ).

12.

Top panel depicts the effect of the structural modification without selectivity gradients; the bottom panel depicts the structures with superimposed selectivity gradients. The problem region of the ligand A053 is highlighted in red. After the substitution of the nitrile group with an amide group, the subsequent novel structure is analyzed for selectivity interpretability. The target area shows a favorable gradient change, indicating a favorable change in predicted selectivity for the “problem area”, now highlighted in green within the novel ligand, KE_01.

13.

Top panel depicts the effect of the structural modification without potency gradients; the bottom panel depicts the structures with superimposed potency gradients. The problem region of the ligand A053 is highlighted in red. After the substitution of the nitrile group with an amide group, the subsequent novel structure is analyzed for potency interpretability. The target area shows a favorable gradient change, indicating a favorable change in predicted potency for the “problem area”, now highlighted in green within the novel ligand KE_01.

After this substitution, we generated a novel compound, named KE_01, and then predicted its properties using our prediction models. Evaluation by the model yields predictions that this ligand is a selective mGlu2 ligand (task 1 prediction), with a predicted IC50 value of 55.72 nM (task 2 prediction), which is less than the median value (74 nM) of the mGlu2 data set. Separate evaluation by the task 3 segment of the model further demarcates this molecule as a high-potency ligand, predicted to be much more potent than the predecessor, A053, which had an IC50 value of 3059 nM. At this time, this molecule does not exist in the literature, as determined by SMILES search on PubChem and in general Google search. While biological mGlu2 NAM selectivity and potency need to be confirmed, further studies can be undertaken to evaluate this novel ligand structure in vitro. The SMILES of this novel ligand, (KE_01), is as follows: NC­(O)­c4cc­(c1ccc­(F)­cc1)­c3ccc­(CN2CCCC2O)­cc3n4.

In brief, our work consists of three phases. The first phase centered around designing an ML model that accurately predicted the target properties of our data set, namely selectivity and potency. After achieving this goal, phase two focused on the interrogation of the model to identify the “problem” areas of select molecules. In phase 3, we resolved these problem sites with a targeted optimization process and then used the ML model to reevaluate the optimized ligand to see if modifications were favorable. Leveraging the interpretability of this model led to the generation of one novel ligand, which will be optimized further in the near future.

3.5. I-GAT Comparisons to Current Models

While our model demonstrated excellent performance in predicting properties for our specific data set, the question remains whether it will work for other data sets as well, and for other chemical properties. To investigate this further, we employed a regression data set gathered by Cui et al. We compared our model against other state-of-the-art models, using the data already reported by Ahmad et al. Additionally, we used RMSE, along with R ² values, to make a fair comparison. Using the methods employed by Ahmad et al., we noted that the performance of our I-GAT model was marginally better in terms of the R ² metric, yielding a value of 0.543 ± 0.065 over three runs, when compared to 0.52 from the prior best-performing model. Additionally, our model delivered a lower RMSE value than the other state-of-the-art models, as detailed in Table . The previous best-performing model, AttentiveFP, yielded an RMSE of 0.61, which was surpassed by the RMSE of our model, which yielded a value of 0.563 ± 0.040 over three runs. Ultimately, these findings strongly support the capability of our regression arm to match or exceed the performance of state-of-the-art models for predicting other properties of unique data sets with minimal modifications.

1. Comparative Performance of Multiple ML Models for Solubility Prediction .

Model	R2	RMSE
GIN-based GNN	0.21	0.78
SGConv-based GNN	0.09	1.98
GAT-based GNN	0.38	0.69
14 layers ResNet	0.13	0.82
20 layers ResNet	0.41	0.68
26 layers ResNet	0.07	0.85
AttentiveFP-based GNN	0.52	0.61
I-GAT	0.543 ± 0.065	0.563 ± 0.040

^aExcept for the I-GAT data, all other metrics have already been reported in the manuscript from Ahmad et al. I-GAT metrics are reported as mean ± standard deviation from three independent runs.

We also investigated the potential for our model to apply to classification data sets as well. To benchmark the classification capabilities of our I-GAT model, we compared our performance against the highest-performing models reported for four MoleculeNet classification data sets, namely BBBP, BACE, HIV, and ClinTox. ROC-AUC scores (mean ± standard deviation over three runs) are presented in Table . For the BACE data set, I-GAT achieved the highest ROC-AUC of 86.6 ± 0.6, marginally outperforming ChemXTree (86.2 ± 0.5), the previous best performing model. In the HIV classification task, I-GAT again achieved the top performance, with a ROC-AUC of 81.6 ± 0.8, exceeding the scores of Uni-Mol (80.8 ± 0.3), which was previously the highest performing model. For the ClinTox data set, ChemXTree remained the top performer with a ROC-AUC of 92.3 ± 0.8. While I-GAT did not exceed this result, it still yielded exceptional performance with a ROC-AUC score of 88.0 ± 2.9, which outperformed the majority of other reported models. In the BBBP task, I-GAT achieved a ROC-AUC of 70.6 ± 1.9, placing it in the middle of the range relative to other state-of-the-art chemistry models. In summary, these results demonstrate that I-GAT consistently matches performance on multiple molecular classification benchmarks. However, it exceeded state-of-the-art models with regard to BACE and HIV benchmarks.

2. Comparative Performance of Multiple ML Models for Classification Tasks .

Model	BBBP	BACE	HIV	ClinTox
D-MPNN	71.0 ± 0.3	80.9 ± 0.6	77.1 ± 0.5	90.6 ± 0.6
Attentive FP	64.3 ± 1.8	78.4 ± 0.02	75.7 ± 1.4	84.7 ± 0.3
N-GramRF	69.7 ± 0.6	77.9 ± 1.5	77.2 ± 0.1	77.5 ± 4.0
N-GramXGB	69.1 ± 0.8	79.1 ± 1.3	78.7 ± 0.4	87.5 ± 2.7
PretrainGNN	68.7 ± 1.3	84.5 ± 0.7	79.9 ± 0.7	72.6 ± 1.5
GROVERbase	70.0 ± 0.1	82.6 ± 0.7	62.5 ± 0.9	81.2 ± 3.0
GROVERlarge	69.5 ± 0.1	81.0 ± 1.4	68.2 ± 1.1	76.2 ± 3.7
GraphMVP	72.4 ± 1.6	81.2 ± 0.9	77.0 ± 1.2	79.1 ± 2.8
MolCLR	72.2 ± 2.1	82.4 ± 0.9	78.1 ± 0.5	91.2 ± 3.5
GEM	72.4 ± 0.4	85.6 ± 1.1	80.6 ± 0.9	90.1 ± 1.3
Uni-Mol	72.9 ± 0.6	85.7 ± 0.2	80.8 ± 0.3	91.9 ± 1.8
ChemXTree	75.6 ± 0.6	86.2 ± 0.5	80.6 ± 0.5	92.3 ± 0.8
I-GAT	70.6 ± 1.9	86.6 ± 0.6	81.6 ± 0.8	88.0 ± 2.9

^aExcept for the new I-GAT data, all other metrics have already been reported in the manuscript from Xu et al. I-GAT metrics are reported as mean ± standard deviation from three independent runs.

4. Discussion

The design of the model required careful alignment with an appropriate molecular input representation. For atom identity, instead of using atomic numbers, we encoded the chemical symbol based on a permitted list of atoms. In addition to this atom-name encoding, we also included the atomic mass. While this seems redundant, including this information could allow for prioritization of rarer elements, especially considering that lower-weight elements such as carbon are more common in chemical structures. We encoded information pertaining to hydrogen atoms by assigning a numerical value equivalent to the total number of hydrogens bound to a single atom, which we superimposed as a node feature at that atom. Other data inputs focused on transmitting data about the three-dimensional structure of a molecule. These inputs included Gasteiger (partial) charges and hybridization, which were originally explored by Goh et al. As a whole, this data preparation serves to transmit information about the arrangement and structure of the molecule.

To increase the predictive capacity of our ML model, we included the following values correlating with the physicochemical properties of molecules. We included implicit valence, as this quantification of reaction capacity could correlate with ligand stability. Formal charge is a computed property that serves as a proxy for charge transfer, which can be useful in predicting reactivity. − The number of directly bonded neighbors, called degree, is also chemically relevant. Considering that isolated atoms display the tendency to display more reactivity when compared to an atom with more bonds, degree may potentially correlate with ligand–receptor interaction. Explicit valence includes bonds that are connected to hydrogen atoms, in addition to all the bonds that a particular atom has to its surrounding atoms. In the realm of chemical simulations, explicit valence allowed for the description of the polarizability, flexibility, and dissociation capacity of water in the context of proton transfer. The scale covalent radius was incorporated; as shorter bond lengths imply stronger bonds, and vice versa, this information could transmit data about the physical properties of chemical structures. , When considering features of a single molecule, van der Waals radius can lend information about molecular geometry, which has implications for factors such as chemical reactivity and even selectivity in some contexts. , The inclusion of these values could allow the model to recognize similar computational “landscapes” between molecules of interest, which may be corollaries of receptor selectivity or potency in the context of metabotropic glutamate receptors. While this work offers atom-level resolution, some node features, such as hybridization, consider neighboring atoms and bonds. Thus, with our particular encoding, the atom-level resolution is paired with metrics that consider macro-level features.

When combined with this style of molecular representation, we designed our network with the capacity to perform three tasks and leveraged transfer learning to inform each subsequent task. For task 1, simple classification was performed, as the model predicted ligand selectivity with high accuracy. For task 2, regression was performed, as the model estimated the IC50 values of individual input ligands, yielding an acceptable error. Improved performance was noted when the classification weights from task 1 were broadcasted into the attention layers of the model for task 2, serving as a starting point. Initially, the model yielded adequate performance for task 2 but appeared highly unstable. Performance and stability both increased when we added another GAT layer. This fourth layer could be surmised to add “flexibility”, allowing the model to resolve previous knowledge of selectivity with new knowledge of potency, allowing for stable gradient descent. Adding a fifth layer decreased performance, which is succinctly explained by the oversmoothing phenomenon that arises with excessively deep GNN architecture, which leads to reduced predictive power. It is intriguing to note that this same oversmoothing effect existed in the selectivity task as well, and thus we tuned layer depth with this consideration.

The advantages of transfer learning were evidenced in task 2 by the drastic differences between the MAPE values of the informed regression model when compared with the noninformed regression model. The improved performance resulting from the transfer of selectivity weights could be explained by multiple factors. There could be a nonlinear relationship between selectivity and potency, although concrete evidence of this relationship does not exist in the literature. Additionally, the IC50 value distributions of the mGlu2 and mGlu3 ligands were fundamentally different. Thus, broadcasting the highly accurate selectivity information could allow the machine learning model to learn about this inherent distribution inequality. Ultimately, using the informed model is more representative of real-life use cases, especially in instances where currently existing ligands display varying potency for target receptors. In the drug discovery process, researchers would need to identify the target receptor of a ligand before selecting the most potent ones.

To cater to large-volume data handling, potency classification was also integrated with task 3. With this classification output, the model sought to determine if a ligand displayed low or high potency for the target class. Having a concise method to exclude low-potency compounds would be ideal, especially as data volume increases. To perform this function, the network would have to be designed in a manner such that the high selectivity accuracy and the low MAPE of the repression prediction could be leveraged to inform the potency classification. While more GAT-based layers were added, the network again appeared to experience the phenomenon of oversmoothing yet again. Thus, we designated a shallow MLP to resolve the outputs of the two segments. There was still instability noted, theorized to be due to difficulty resolving both selectivity and potency regression weights.

To address this issue, for the first 100 epochs, the weights of the selectivity segment were frozen. Notably, when we froze the regression segment, training was unstable. Ultimately, these results lead to the assumption that allowing for task 3 to be “biased” in favor of the potency prediction inputs, allows for stability of the loss landscape. After 100 epochs, the weights of the regression segment were unfrozen. Although there was a noticeable spike in both training and validation loss, stable training ensued thereafter. The comparison between partially frozen loss curves (Figure S11) and unfrozen loss curves (Figure S12) demonstrates the superiority of the partially frozen method, which is paralleled by increased accuracy. It is important to note that unfortunately, some overfitting did occur, even with partial layer freezing. This could be due to the data complexity, as resolving two separate molecular characteristics is a difficult task for the model. Nonetheless, similar to the selectivity prediction, the potency classification prediction also achieved high accuracy.

Building on the model’s strong performance, we next sought to investigate why certain ligands were predicted to exhibit ideal properties by exploring the interpretability of the model’s outputs. Before discussing the interpretable output, it is important to consider that computational inputs are difficult to understand from a human perspective. However, identifying the relative contributions of each atom to a target property can allow chemists to visualize problem areas. To achieve this task, after completion of training tailored to selectivity and potency prediction, we examined the model’s hidden state with gradient evaluation. While the gradients are relative and are only comparable intramolecularly, the decreased magnitude of the gradients for potency in comparison to selectivity could be explained in a few ways. The process of multitask learning through three discrete tasks, along with the weight mechanism intrinsic to attention networks, most likely induced a scaling effect. As the network trained and incorporated information from multiple tasks, the gradients that were propagated reduced in magnitude to permit stable gradient descent. Additionally, the added layers of the GATs tailored to task 2 and task 3 may have contributed to some degree of oversmoothing. Random initialization and subsequent training could yield discrepancies due to the stochastic nature of ML models, especially with such a low sample size. However, this was not the case; ultimately, the relative reproducibility of the predictions and their underlying gradients across multiple runs is very interesting. Taken as a composite, this yields evidence of an extremely stable, multifunction selectivity and potency interpretability model using graph architecture.

We further leveraged this robust interpretability evaluation to identify trends associated with target properties. For the two assessed mGlu2 ligands, amide substitution of the original nitrile group appeared to correlate with increased potency, as confirmed by ground truth values. For the available data set, the average median and mean IC50 values for the mGlu2 NAMs with nitrile groups were higher than the corresponding values of the entire mGlu2 NAM data set. While it is difficult to conclusively state that the presence of nitrile groups is detrimental to mGlu2 potency, the data suggests this possibility. At this time, there is no literature indicating whether nitrile groups correlate with reduced mGlu2 potency or influence selectivity. As this has not been previously reported in the literature, this could be a valuable direction for future study, and could allow us to fully validate the interpretability mechanism.

One of the limitations of leveraging interpretability to optimize target characteristics stems from the data corpus. Due to the limitation of our data set, our model was only evaluated for the ligands with similar scaffolds. However, the possibility that our ligand could go from displaying an IC50 of 3059 nM to a theoretical IC50 of 55.72 nM with one targeted modification inspires optimism. Of note, our computational methods still need to be validated with in vitro experiments in future studies. In the meantime, further optimization of this novel ligand could take place. From the structure-based approach, docking studies could be used to optimize ligand characteristics. Docking maps can be generated to find favorable interactions with the receptor. Structural modification and transplantation of functional groups can be combined with our approach to evaluate if favorable binding can be increased to the target receptor.

While recent advances in lead compound optimization have favored end-to-end approaches, our choice to implement a modular, three-phase design for I-GAT was intentional and strategic. We designed our pipeline reflecting how a trained chemist would undertake ligand optimization. Unlike end-to-end models, which often obscure the internal reasoning behind predictions, our staged architecture enables the extraction of intermediate outputs, such as attention gradients, which can be visualized and validated by experts. Our architecture isolates key tasks, allowing for more granular insight into model behavior and decision-making at each step. The model’s interpretability facilitates rational, structure-guided modifications by highlighting detrimental regions, such as a nitrile group, that can be strategically substituted to improve both potency and selectivity. While the I-GAT model utilizes components grounded in established graph architectures, its staged, multitask design paired with gradient-based interpretability provides a novel integration tailored to ligand optimization. In particular, we designed custom training flows and attention-based interpretability methods to deliver atomic-resolution explanations. As seen by multiple examples, the model performs at an atom-level resolution with the capacity to identify individual poor-performing atoms. Simultaneously, this model can also consider granular, macro-level details of input chemical ligands. This lends further differentiation of the capability of our model when compared to current chemistry interpretability models.

These methods expand beyond traditional black-box modeling and provide actionable insights for medicinal chemistry. Moreover, this modular design allows for flexibility and adaptability. Each task can be removed, retrained, or replaced, allowing the framework to accommodate different types of molecular inputs or output properties without requiring a full model redesign. This plug-and-play structure is particularly valuable in iterative experimental workflows, where model interpretability can inform compound synthesis and subsequent rounds of design. To further demonstrate the generalizability of our approach, we also leveraged this modularity to conduct model comparisons across a diverse range of data sets.

While searching for comparative models, we found that existing literature primarily focuses on one output prediction. While there are GNN-based models that employ transfer learning, this is in the context of integrating low-fidelity and high-fidelity data with respect to a single output property prediction. To the best of our knowledge, no existing GNN models perform multifunctional property prediction while incorporating transfer learning from multiple tasks to yield better informed predictions. Thus, we were unable to find a benchmark model in the existing literature to compare to our model while exploring all properties of the data set in the manner detailed in the paper. However, we sought to compare our model performance to state-of-the-art models using existing databases for single-property prediction.

In this work, we initially trained I-GAT on a relatively small data set focused on mGlu2 and mGlu3 receptors, particularly for potency prediction via regression in the second task. While our model demonstrated excellent performance on the original data set, its performance may vary depending on the input data sets. To address this issue, we trained our I-GAT on a large solubility regression data set and predicted on an independent data set, following the methodology of Ahmad et al. This allowed us to incorporate a regression task and a data set for which the model was not explicitly tuned. Despite this shift in task objective, I-GAT outperformed AttentiveFP, which was the previously best-performing model reported by Ahmad et al. Our model yielded a lower RMSE of 0.563 ± 0.040 compared to the 0.61 RMSE value from AttentiveFP. Additionally, we were able to attain a higher R ² score of 0.543 ± 0.065 compared to 0.52 from the prior state-of-the-art model. These results suggest that I-GAT has the capacity to yield exceptional performance for predicting other physicochemical properties, and that its regression prediction capacity was not restricted to potency.

Similarly, our original classification experiments from Task 1 were limited to ligand selectivity prediction for mGlu2 and mGlu3 receptors. To test I-GAT’s broader applicability, we examined four classification data sets derived from MoleculeNet. The BBBP data set evaluates blood–brain barrier permeability, which is an important feature for designing CNS drugs. The BACE focuses on ligands that are human beta-secretase 1 (BACE-1) inhibitors; we used the qualitative binary label to form a fair comparison to Xu et al. The HIV database classifies compounds for their ability to inhibit viral replication, and ClinTox classifies ligands based on toxicity. These data sets are diverse in terms of molecular characteristics, data set counts, chemical structure variability, and the available properties. However, I-GAT achieved the highest performance on two particular benchmarks, namely BACE (with ROC-AUC of 86.6 ± 0.6) and HIV (with ROC-AUC of 81.6 ± 0.8), outperforming state-of-the-art models. For the ClinTox data set, I-GAT maintained a strong performance (88.0 ± 2.9), exceeding the performance of most comparison models. On the BBBP data set, however, I-GAT yielded a midrange result (70.6 ± 1.9), while remaining competitive with several established models. While investigating this further, we noted a paper referencing duplicates, mischarged atoms, and conflicting labels in the BBBP data set. In some instances, identical molecules were labeled both as BBB-penetrant and nonpenetrant, which can degrade model performance. Additionally, the presence of chemically invalid atoms, such as uncharged tetravalent nitrogen atoms, can also degrade model performance. Ultimately, the aforementioned errors could impair learned structure–property relationships during training.

Of note, while we employed Optuna to tailor hyperparameters to each task, we did not tailor the approach to these data sets. Additionally, the strength of our model comes from its transfer learning capacity. With these data sets, we were not able to leverage the full reach of our model. Despite these constraints, I-GAT still delivered performance comparable to leading models. These findings indicate that I-GAT demonstrates strong generalization and adaptability across diverse prediction tasks, including solubility, selectivity, toxicity, permeability, and enzyme inhibition. While the purpose of the paper was to focus on property prediction for mGlu2 and mGlu3 ligands, future investigation is needed to apply I-GAT to additional data sets, particularly for properties that are priorities for lead optimization.

When considering the strong predictive performance of our approach across multiple data sets, one perspective should be considered. Given that our results were generated using ligands with similar receptors, our model may demonstrate even greater predictive power when applied to data sets containing more structurally diverse ligands. The application of our approach is not limited to group II metabotropic glutamate receptor ligands. Further testing with additional ligand data sets and target properties would be highly beneficial. Not only would this help assess the generalizability of the model, but it could also assist researchers in identifying specific molecular regions for optimization. Furthermore, interdisciplinary collaboration could enhance this approach. Research groups with computational expertise but limited biological validation capacity, and vice versa, could share insights to maximize research impact.

While in vitro validation remains a crucial next step to fully substantiate our findings, future improvements from a computational perspective could involve incorporating additional input features and refining the model architecture. Specifically, integrating mesh graph architecture with multilayer perceptron models while maintaining multioutput prediction capability could enhance predictive accuracy. Moreover, the model’s adaptability could be tested by training on more diverse data sets. If various ligand classes are incorporated, transfer learning techniques could be leveraged to extract shared insights across different chemical spaces. Finally, the inclusion of additional chemical properties would enable a more comprehensive multitask learning approach, further strengthening model robustness across diverse drug discovery applications. While we explored other relevant data sets, we anticipate that properties such as chemical absorption, distribution, metabolism, excretion, and toxicity (ADMET) values can also be predicted and optimized with our approach.

5. Conclusions

In this work, we developed a tiered multifunction model that can perform three tasks: 1) predict compound selectivity for mGlu2 and/or mGlu3; 2) estimate IC50 values, and 3) classify compounds as high-potency or low-potency ligands. Transfer learning was used to transmit the lessons learned from the selectivity tasks to inform downstream potency tasks in a sequential manner. We further developed a visualization approach to highlight “problem areas” within molecules, which is a novel strategy to facilitate the optimization of the selectivity and/or potency of the existing ligands. We demonstrated the utility of our strategy by designing a novel mGlu2 ligand and evaluating its selectivity and potency using our trained model. We used mGlu2/mGlu3 as an example to develop our approach. However, our approach can be applied to other ligand data sets and other chemical properties beyond potency and selectivity. With more data points and higher data volume, multiple properties could be assessed simultaneously and further scrutinized through gradient evaluation to optimize the drug candidates. Our work lays the foundation for artificially intelligent redesign of existing ligands into ones with potentially superior properties. Once validated in vitro, the successful implementation of this paradigm could lead to the development of precise therapeutics for various use cases.

Glossary

Abbreviations

GPCRs

G-protein-coupled receptors

FDA

Food and Drug Administration

mGlu2

metabotropic glutamate receptor subtype 2

mGlu3

metabotropic glutamate receptor subtype 3

NAMs

negative allosteric modulators

I-GAT

Interpretable Graph Attention Networks

ML

machine learning

AI

artificial intelligence

GNNs

Graph Neural Networks

XGBoost

eXtreme Gradient Boosting

SVM

support vector machine

GAT

Graph Attention Networks

GCN

Graph Convolutional Network

LIME

Local Interpretable Model-Agnostic Explanations

Grad-CAM

Gradient-Weighted Class Activation Mapping

CNNs

Convolutional Neural Networks

SMILES

Simplified Molecular Input Line Entry System

RMSE

Root Mean Squared Error

BBBP

Blood-Brain Barrier Penetration

BACE

β-secretase enzyme

HIV

Human Immunodeficiency Virus

ClinTox

Clinical Toxicity

MAPE

mean absolute percentage error

MLP

multilayer perceptron

ADMET

absorption, distribution, metabolism, excretion, and toxicity

The data used for this work are included in the Supporting Information: compiled data set. The code used to generate the composite models is available from the GitHub repository: https://github.com/EzekMathew/MultiFunction-CHEML_GAT.

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acsomega.5c02173.

• Figure S1: IC50 distribution of the n = 351 mGlu2 and n = 309 mGlu3 ligands derived from patent database; Figure S2: training and validation loss curves for the independent validation set; Table S1: selectivity accuracy summaries for five folds, along with the values for the independent validation set; Table S2: classification report for the selectivity model for the 5-fold cross-validation data set; Table S3: classification report for the selectivity model for the independent validation data set; Figure S3: training and validation loss curves for the independent validation set for the potency regression task; Table S4: comparison of MAPE between models; Figure S4: potency regression output of the GAT model; Table S5: Pearson’s correlation examination of target arrays; Table S6: potency accuracy summaries for five folds along with the values for the independent validation set; Table S7: potency classification report for the dual potency and selectivity-informed model, pertaining to the 5-fold cross-validation data set; Table S8: potency classification report for the dual potency and selectivity-informed model, pertaining to the independent validation data set; Table S9: cumulative evaluation of selectivity output nodes for all available ligands; Table S10: cumulative evaluation of potency output nodes for all available ligands; Figure S5: selectivity optimization for the ligand BOTH114; Figure S6: potency optimization for the ligand A026; Figure S7: potency optimization for the ligand Be53; Figure S8: potency optimization for the ligand BEM21; Figure S9: selectivity analysis for the ligand A041 in triplicate; Figure S10: potency analysis for the ligand A041 in triplicate; Figure S11: training and validation loss curves confirm that severe overfitting did not occur over the course of 400 epochs for the independent validation set; Figure S12: training and validation loss curves confirm that severe overfitting did occur over the course of 400 epochs for the independent validation set when freezing was not incorporated; Table S11: optimal training hyperparameters identified by Optuna for each data set; Table S12: nitrile-containing mGlu2 compounds; Table S13: patent-derived data set of compounds (PDF)

The manuscript was written through the contributions of all authors. All authors have given approval to the final version of the manuscript.

This work is partially supported by a grant (#RP210046) from the Cancer Prevention and Research Institute of Texas (CPRIT).

The authors declare the following competing financial interest(s): J.L. is a co-founder of Neoclease, Inc.