Calibrated geometric deep learning improves kinase–drug binding predictions

Abstract

Protein kinases regulate various cellular functions and hold significant pharmacological promise in cancer and other diseases. Although kinase inhibitors are one of the largest groups of approved drugs, much of the human kinome remains unexplored but potentially druggable. Computational approaches, such as machine learning, offer efficient solutions for exploring kinase–compound interactions and uncovering novel binding activities. Despite the increasing availability of three-dimensional (3D) protein and compound structures, existing methods predominantly focus on exploiting local features from one-dimensional protein sequences and two-dimensional molecular graphs to predict binding affinities, overlooking the 3D nature of the binding process. Here we present KDBNet, a deep learning algorithm that incorporates 3D protein and molecule structure data to predict binding affinities. KDBNet uses graph neural networks to learn structure representations of protein binding pockets and drug molecules, capturing the geometric and spatial characteristics of binding activity. In addition, we introduce an algorithm to quantify and calibrate the uncertainties of KDBNet’s predictions, enhancing its utility in model-guided discovery in chemical or protein space. Experiments demonstrated that KDBNet outperforms existing deep learning models in predicting kinase–drug binding affinities. The uncertainties estimated by KDBNet are informative and well-calibrated with respect to prediction errors. When integrated with a Bayesian optimization framework, KDBNet enables data-efficient active learning and accelerates the exploration and exploitation of diverse high-binding kinase–drug pairs.

Proteins are vital drug targets for therapeutic purposes, but at present only 11% of human proteome can be targeted by drugs or small molecules, leaving a large proportion to be explored for therapeutic opportunities¹. A group of proteins called kinase is of particular interest as drug targets because of their tractability in drug development and diverse pharmacological implications in various diseases^2,3. Protein kinases present high evolutionary conservation in sequence and structure. Most of the kinase inhibitors bind to conserved adenosine triphosphate (ATP)-binding pockets of kinases, leading to extensive target promiscuity⁴. Chemical compounds that inhibit a single kinase are still rare despite significant research efforts devoted to target-based drug discovery⁵.It is therefore crucial to map out target binding profiles of kinase inhibitors to uncover new therapeutic effects and better predict and manage possible adverse effects. Unfortunately, even with automated high-throughput profiling assays, it is still infeasible to exhaustively measure compound-target binding activities because of the vast chemical space.

Machine learning (ML) methods have emerged as alternative solutions to efficiently map compound–protein interaction profiles⁶. Early studies included bipartite graph-based methods that framed the prediction problem as a recommendation system-like task^7–11. These methods computed the similarity between compounds or proteins on the basis of simple features like molecule fingerprints or sequence-alignment scores, enabling the prediction of new protein–drug interactions on the basis of known, similar proteins and drugs. With recent advancements in deep learning, a series of studies^12–16 leveraged deep neural networks to automatically learn features from raw compounds and protein representations in a fully data-driven way, also known as end-to-end learning. Commonly used data representations include one-dimensional (1D) features such as protein sequences and molecule simplified molecular-input line-entry system (SMILES) strings^12,13. Recent approaches indicated that incorporating two-dimensional (2D) features, including molecular graphs and protein contact maps, enhanced prediction accuracy^14–17. Although the compound–protein binding is, in essence, a physicochemical process in the three-dimensional (3D) space, there remains a paucity of studies that incorporate 3D structure information to enhance protein–drug binding prediction, in part because of the scarcity of protein structure data and the absence of predictive models that effectively use 3D structure data. Fortunately, for kinases, the data bottleneck is less pronounced owing to their biological significance. Kinases are one of the best-represented protein families in the Protein Data Bank (PDB) database¹⁸, with a rapidly growing number of solved kinase structures^19,20. In parallel, recent progress in graph deep learning offers promising avenues for effectively modelling 3D protein structures^21–23. Jointly, there are great opportunities and pressing needs to develop new methods that integrate 3D structure information to improve predictions of kinase–drug binding affinity.

The primary importance of ML approaches for compound–protein binding prediction is to accelerate the discovery of compounds or targets. With an accurate ML predictive model, virtual screening can be performed by applying the model to generate hypotheses about binding activities, allowing the selection of candidates with the highest predicted activities for further validation. However, these data-driven methods are susceptible to inherent noise and bias in the training data, rendering them vulnerable to failures when applied to out-of-distribution scenarios. To mitigate this issue, one solution is to quantify the uncertainty of model predictions, providing a confidence assessment to support human decision-making, as higher novelty often comes with a higher risk of failure. Although the importance of uncertainty estimation in ML algorithms has been recognized^24–26, most existing methods of compound–protein binding prediction only provide point-estimate predictions without quantifying uncertainty^12–16. In the context of compound or target discovery, relying solely on point-estimate predictions to select top candidates for validation may result in false positives. Ref. 17 introduced uncertainty estimation using Gaussian processes (GPs) to prioritize strong-binding compound–protein pairs, but quantifying uncertainty with more expressive deep neural networks has not been explored for kinase–drug binding prediction.

Here, we develop the kinase–drug binding prediction neural network (KDBNet), a deep learning algorithm that integrates 3D structure information to predict the binding affinity of kinase–drug binding while also estimating prediction uncertainties. KDBNet represents the 3D protein and molecule structure data as graphs and uses graph neural networks (GNNs) to learn structure representations from binding pocket structures of proteins and atom coordinates of molecules. We built KDBNet as an ensemble model of several replicates of individual neural networks, which not only improves prediction accuracy and robustness but also allows us to estimate the uncertainty of model predictions. We further applied an uncertainty recalibration technique to refine the uncertainty estimates, enhancing KDBNet’s utility in the ML-guided discovery of proteins and targets. Benchmarking on public datasets of kinase–drug binding-affinity measurements, we found that KDBNet achieved more accurate predictions than existing models that used only 1D or 2D representations of proteins and drugs. Our experiments also indicated that KDBNet’s uncertainty estimates were largely consistent with respect to prediction errors, meaning predictions with lower uncertainty are often more accurate. Furthermore, we found the uncertainty estimates were also well-calibrated, providing interpretable confidence intervals for individual predictions. Finally, we extended KDBNet into a Bayesian optimization (BO) framework, showcasing its capability for data-efficient active learning and accelerated exploration of strong-binding kinase–drug pairs.

Results

Overview of KDBNet

KDBNet is a deep learning model that integrates 3D structures to predict binding affinities between kinases and small-molecule compounds (Fig.1). KDBNet receives 3D structures of proteins and compounds and represents them as two graphs: the graphs’ nodes are protein residues or molecule atoms, and the edges encode residue contacts or atom distance. A set of features, which collectively describe the structural, evolutionary, biophysical and chemical properties of protein residues or chemical atoms, are also derived for each node and edge in the protein and molecule graphs. Next, KDBNet uses GNNs to learn structure representations of the input kinase and compound, reflecting the spatial organization and topological neighbourhood of the 3D protein and molecule structures. The learned representations are then combined to predict the binding affinity through another fully connected (FC) neural network. In addition to the binding-affinity prediction, KDBNet also associates each of its predictions with an uncertainty estimate, quantifying its confidence about the prediction. KDBNets achieves this by training an ensemble of models and estimating the uncertainty using the variance of individual models’ predictions.

Fig. 1 |. Overview of KDBNet.

KDBNet is a neural network that integrates protein 3D structure and compound 3D structure to predict compound–protein binding affinity. KDBNet derives a set of features, including sequence (seq), evolutionary representations and 3D-invariant geometric features, on the basis of the input 3D structure and uses a GNN to learn structure-aware representations of a protein For the input compound, KDBNet uses a 3D-equivariant GNN to directly learn structure representations from the compound’s coordinates in the 3D space. The representations of the input protein and are then used to predict the binding affinity as well as the uncertainty of the prediction.

Accurate prediction of kinase–drug binding affinity

We first assessed KDBNet’s performance in predicting kinase–drug binding affinity using two public datasets of experimental measurements of kinase–compound binding affinity, Davis²⁷ and KIBA²⁸, which were widely used to benchmark previous methods^{10,12,15–17}. We created three evaluation settings to simulate out-of-distribution scenarios in which the training and test sets do not share any drugs or proteins (Fig. 2a and Supplementary Notes 1.1 and 1.2). We compared KDBNet with several state-of-the-art methods for predicting kinase–drug binding affinity, including three deep-learning-based methods^12,15,16, a GP-based method¹⁷ and a kernel-based method²⁹. These baseline methods rely solely on 1D and 2D representations or pairwise similarities of compounds and proteins, without incorporating 3D structural information (Supplementary Note 1.3).

Fig. 2 |. KDBNet achieves accurate prediction of kinase–drug binding affinity.

a, Four train–test split evaluation settings in which the model is evaluated on data of unseen drugs (‘new-drug split’), unseen proteins (‘new-protein split’) or both (‘both-new split’) and unseen proteins with low (<50%) sequence identity (‘seq-id split’). b, Comparison of KDBNet prediction performance with KronRLS, DeepDTA, GraphDTA, DGraphDTA and GP on the four train–test split settings. c, Comparisons between KDBNet variants that use or do not use 3D structure data on the both-new split. When 3D drug structure is not used, the 2D molecule graph parsed from a SMILES string is used as the representation of the input drug, and no 3D geometric features are used in the molecule GNN. When 3D protein structure is not used, the sequence is used as the representation of the input protein, and the protein GNN is replaced by a convolutional neural networ. The full model use both 3D drug and protein structures. d, Comparisons between KDBNet and three baseline methods that receive 3D binding complex structure as input (GNN3D, CNN3D and SIGN) on the PDBbind dataset. KDBNet differs from them in that it only uses separate 3D drug and protein structures as input: the baseline methods thus have an advantage as they are aware of the protein–compound docking structure through the input complex. Results of methods that receive non-3D input (GraphDTA and DeepDTA) are also shown for comparison. Asterisks indicate the statistical significance (one-sided Mann–Whitney U rank test $P=0.00397$ for both GraphDTA and DeepDTA) that KDNBet’s performance is higher than the baseline’s performance over $n=5$ random train/test splits. Bar plots in b-d represent the mean ±s.d. of the evaluation results on five random train/test splits. Pearson correlation and MSE were computed using the predicted and true p_d values.

The evaluation results (Fig. 2 and Supplementary Fig. 1) indicated that KDBNet consistently outperformed other methods across several metrics, including Pearson correlation, Spearman correlation and mean squared error (MSE; one-sided rank test $P<{10}^{-3}$). These improvements held across various split settings. The enhancements achieved by KDBNet also underscored the efficacy of end-to-end feature learning in comparison to methods (for example, GP) that rely on precomputed, fixed feature embeddings. Similar observations were made when applying KDBNet to the larger KIBA dataset, where it surpassed the baseline methods and even outperformed two recently developed methods that used protein language model embeddings³⁰ or contrastive learning strategies³¹ (Extended Data Fig. 1).

The improvements of KDBNet primarily stem from its direct modelling of 3D structures of proteins and molecules in the neural network. This was confirmed by our ablation study, in which 3D structure data of either the input protein or drug were dropped (Fig. 2c). Compared to baselines that consider only 1D or 2D representations of proteins and compounds, the 3D structure data and structure-derived geometric features in KDBNet (Supplementary Fig. 2) provided more explicit information related to the binding activity, which better respects the 3D physical symmetries of binding activities that might not be fully reflected by the 1D or 2D features. Even compared to recent methods that use the 3D protein–compound binding complex structure as input (CNN3D^32,33, GNN3D³⁴ and SIGN³⁵) on the PDBbind database³⁶, KDBNet achieved performance comparable to these complex-based baselines (Fig. 2d and Supplementary Fig. 3) and substantially higher than baselines that used non-3D input (DeepDTA and GraphDTA; one-sided rank test $P<{10}^{-3}$). It is noteworthy that complex-based methods had an advantage in this comparison, as they can capture the interaction features from the complex structure. Although slightly superior in prediction performance, those complex-based methods^{32–35,37–41} are constrained by the availability of binding complex structures. In contrast, KDBNet achieved comparable prediction performance using separate 3D structures, which are more readily accessible, making it suitable for numerous tasks such as virtual drug screening for which complex structures between novel targets and compounds are rarely available.

Overall, these results demonstrated that by incorporating 3D structure data and leveraging geometry-aware deep learning, KDBNet made clear performance improvements in kinase–drug binding prediction compared to several existing methods and was able to generalize to predictions for unseen proteins, unseen drugs or both.

Informative and calibrated uncertainty estimation

One immediate application of an accurate ML model for protein–compound binding-affinity prediction is using it to generate new hypotheses, such as prioritizing promising compounds, to assist drug discovery and drug repurposing. From a practical perspective, in addition to predicting affinity, it is also desirable that the model can provide associated uncertainty estimates, allowing researchers to assess the likelihood of hypothesis success and allocate experimental efforts more effectively. Unlike many previous deep learning methods that only predict a point estimate of binding affinity while overlooking uncertainties in the data or model^12,15,16, KDBNet goes a step further by providing an uncertainty estimate for each affinity prediction (Methods).

First we aimed to investigate whether KDBNet’s uncertainty estimate is indicative of prediction accuracy. Ideally, the model’s uncertainty would be correlated with its prediction error, and predictions with lower uncertainty would have lower prediction errors. We assessed KDBNet’s uncertainty quantification on the Davis dataset. We ranked all of KDBNet’s predictions by their associated uncertainty estimates (Supplementary Note 1.4) and observed that there was a consistent trend for KDBNet’s predictions with lower uncertainty to exhibit lower prediction errors across various split settings (Fig. 3a; average Spearman’s correlation $\bar{\rho }=0.98$). Compared to the two GP-based methods, GP and GP-multilayer perceptron (GP-MLP)¹⁷, KDBNet achieved much lower mean absolute errors (MAEs) across different uncertainty percentiles (Fig. 3a) and higher correlations between the estimated uncertainty and prediction errors (Fig. 3b and Supplementary Fig. 4a). These indicated that KDBNet’s uncertainty estimates were correctly ranked with respect to prediction errors, and its predictions were highly accurate when it had a low level of uncertainty.

Fig. 3 |. KDBNet provides accurate and calibrated uncertainty estimation.

a, Prediction errors of KDBNet, GP and GP-MLP, measured as MAE, at different cutoffs of uncertainty percentiles. The $x$ axis represents the sorted uncertainty such that the 100% percentile is the lowest uncertainty (highest confidence). b, Spearman correlation between the estimated uncertainty and the prediction error measured in MAE on the both-new test set. c, Calibration curve. For a confidence interval of confidence level $e(0\le e\le 1$), the curve shows the expected fraction and observed fraction of test points that fall within that interval. The diagonal line corresponds to the calibration curve of a perfectly calibrated model. The miscalibration area, defined as the area between a curve and the diagonal line, is used to quantify the uncertainty calibration, and lower values indicate better calibration. d, Calibration curves of KDBNet, KDBNet without recalibration, GP and GP-MLP on test sets. e, Miscalibration area of KDBNet, GP and GP-MLP on the new-protein test set. Solid lines in curve plots represent the mean value of five independent trials, and error bands indicate the s.d. MAE values were measured in $\mathrm{p}{K}_{\mathrm{d}}$ values. Bar plots in b and e represent the mean ±s.d. of the evaluation results on five random train/test splits. Error bands in a and c depict mean ±s.d. calculated over five random train/test splits. Recalib., recalibration.

The previous evaluation confirmed that KDBNet’s uncertainty estimates provided indicative ranking. We now examined whether the magnitude of these uncertainty estimates was statistically meaningful. Models that are over-confident or under-confident usually produce uncertainty estimates that are either too small or large, rendering them challenging to interpret as credible intervals with statistical meaning. This issue is known as miscalibration in uncertainty quantification⁴². Ideally, we desire well-calibrated uncertainty estimation from the model, meaning, for instance, that if the model predicts a 95% confidence interval, we anticipate the true values to fall within the interval 95% of the time. We computed the miscalibration area^26,42,43 to quantify the degree of uncertainty calibration, which is defined as the area between the model’s calibration curve (Supplementary Note 1.5) and the diagonal line representing a perfectly calibrated model (Fig. 3c). A lower miscalibration area signifies superior calibration. We noted that KDBNet’s calibration curves closely resembled the ideal diagonal curve (Fig. 3d), resulting in substantially lower miscalibration areas than those observed with GP-based methods (Fig. 3e and Supplementary Fig. 4b). Additionally, KDBNet’s recalibration algorithm (Methods) effectively pushed the calibration curves closer to the diagonal and reduced the miscalibration area (Fig. 3d,e and Supplementary Fig. 4b; one-sided rank test $P<{10}^{-3}$). These results indicated that KDBNet’s uncertainty estimates were calibrated and scaled with errors.

Together, these two sets of experiments demonstrated that the uncertainty estimates of KDBNet were both accurate with respect to prediction errors and well-calibrated. The accurate quantification of uncertainty holds important implications for iterative ML-guided experiment design for which the uncertainty estimates can guide data acquisition and candidate prioritization, as we illustrate in the next section.

Uncertainty-guided, data-efficient active learning

Having validated that KDBNet provides informative and calibrated uncertainty estimation, we set out to assess the utility of uncertainty in ML-guided discovery. The first application is active learning, for which the objective is to strategically select training samples to achieve improved prediction performance with fewer training data. Analogous to human experts who rely on intuitive confidence to acquire and test new samples, KDBNet used its estimated uncertainty for iterative training and selection (Fig. 4a). We initiated the training of KDBNet using a random 1% subset of the KIBA training data. In each subsequent round, KDBNet predicted binding affinities and uncertainties for the remaining training data and then ranked these samples (drug–protein pairs) on the basis of the predicted uncertainty from highest to lowest (referred to as the ‘explorative’ strategy). Two other ranking strategies (Methods) were considered for comparison:(1) ‘greedy’, which prioritizes samples with higher predicted affinity, and (2) ‘random’, which ranks all samples uniformly at random.

Fig. 4 |. Leveraging uncertainty for active learning, exploration and exploitation.

a, Schematic visualization of the active learning process, which consists of several rounds of model training, data acquisition and model evaluation. b, Active learning performance in Pearson correlation on the KIBA both-new test set at different rounds. The explorative sampling is compared to the greedy and random sampling strategies. c, Efficiency gain of the explorative and greedy samplings over the random sampling, defined as the relative improvement in Pearson correlation. d, Schematic illustration of data acquisition on the basis of KDBNet’s prediction and uncertainty. One can exploit regions with high-confidence, high-desirability samples or explore potentially high-desirability regions with less model confidence. e, Exploration using KDBNet and UCB acquisition function with a BO framework. Curves represent the performance trajectory, measured by the percentage of top-500 binding affinities found as a function of the number of kinase–compound pairs explored in the Davis dataset. f, Exploitation using KDBNet and LCB acquisition function with BO. True $K_{\mathrm{d}}$ values of the top 10 kinase–drug pairs prioritized by each model are shown. A lower $K_{\mathrm{d}}$ value means a stronger binding affinity. Curve plots in b, c and e depict the mean values over five independent trials in solid lines with s.d. in error bands. Bar plots in f represent mean ± s.d. of the results for five independent trials of top-10 acquisition $(n=50)$.

We found that KDBNet achieved efficient active learning by using its estimated uncertainty to acquire new training samples, reaching performance on par with full data training by using only 50% of the data (Fig. 4b). Noticeably, KDBNet’s performance was improved by a large margin in the initial rounds compared to the random strategy, highlighting the efficiency of uncertainty-based active learning compared to brute-force random searches. Furthermore, in contrast to the greedy strategy that continually seeks samples with the highest affinities, KDBNet’s explorative strategy focused on samples that could diversify the training set and best address the model’s uncertainties, thereby exhibiting faster rates of performance improvement and higher efficiency gains (performance improvement over random selection) across all active learning stages (Fig. 4b,c). These indicated that KDBNet, enabled by uncertainty quantification, achieved sample-efficient active learning for data acquisition and model training, a valuable capability in model-guided experimental design where an exhaustive search is costly or infeasible.

Bayesian optimization for rapid exploration and exploitation

As another application of uncertainty estimation, we integrated KDBNet with BO for the exploration and exploitation of strong-binding kinase–drug pairs. Although the previous active learning experiments acquired new samples solely on the basis of uncertainty for diversifying the training set, BO provides a principled framework to combine both predicted values and estimated uncertainties to guide data acquisition more effectively, enabling us to prioritize candidates in high-confidence, high-desirability regions (‘exploitation’) or probe potentially high-desirability regions, although with less confidence (‘exploration’), as illustrated in Fig. 4d. In BO, a common way to combine predicted scores and uncertainties is through an acquisition function called the upper confidence bound (UCB) with the form $\mathrm{U}\mathrm{C}\mathrm{B}\left(x\right)=\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}\left(x\right)+\beta \times \text{uncertainty}\left(x\right)$, where $x$ represents a kinase–compound pair and constant $\beta$ controls the trade-off between exploitation and exploration.

High-recall exploration.

We first evaluated KDBNet’s exploration capability using the Davis dataset. The objective was to identify kinase–compound pairs with the strongest binding affinity by observing the ground-truth binding affinities of only a small subset of pairs. We started the data acquisition by training KDBNet on 1% of the kinase–compound pairs (~100 pairs). Subsequent steps followed the active learning framework but incorporated UCB as the score function, defined as $\mathrm{U}\mathrm{C}\mathrm{B}\left(x\right)=\mu \left(x\right)+\beta \sigma \left(x\right)$, where $\beta =1$, and $\mu \left(x\right)$ and $\sigma \left(x\right)$ are the binding affinity and associated uncertainty predicted by KDBNet, respectively (Methods). Intuitively, this score function promotes samples with high binding affinity and high uncertainties. Because our goal was to identify strong-binding pairs as comprehensively as possible, we aimed to explore ‘good’ regions that had some variability (uncertainty), as this increased the chances of discovering even better samples. We observed that KDBNet yielded clear improvements compared to the random exploration and GP baselines, as quantified by the recall of top-500 (about the top 1%) kinase–compound pairs as a function of the number of pairs explored. Specifically, KDBNet retrieved 50% of the top-500 pairs from the pool of 10,000 pairs after exploring only 1,000 pairs (Fig. 4e). This experiment highlighted KDBNet’s effectiveness in accelerating the exploration and discovery of strong-binding kinase–compound pairs in the BO framework.

High-confidence exploitation.

Next we performed an analysis to evaluate KDBNet’s exploitation capability: that is, how effectively it prioritized top kinase–compound pairs with strong binding affinities. This mirrored real-world biological discovery processes in which researchers typically focus on only a handful of top compounds or proteins for further validation instead of testing the entire unexplored space. We simulated an experiment on the Davis dataset where the model was tasked with prioritizing kinase–compound pairs with the strongest binding affinity from the test data. For KDBNet, we defined the score function as the lower confidence bound (LCB): $\mathrm{L}\mathrm{C}\mathrm{B}\left(x\right)=\mu \left(x\right)-\beta \sigma \left(x\right)$ with $\beta =1$. Compared to UCB, LCB introduces a negation sign before the uncertainty term, prompting KDBNet to prioritize pairs with strong binding affinity and low uncertainty. Figure 4f presents the binding affinities, measured in $K_{\mathrm{d}}$ (dissociation constant) values, of the top 10 kinase–drug pairs acquired by different methods, where lower $K_{\mathrm{d}}$ values indicate stronger binding affinities. We found that, on average, KDBNet retrieved pairs with stronger binding affinity, outperforming other baseline methods across all three split settings. KDBNet successfully prioritized kinase–drug pairs with a mean $K_{\mathrm{d}}$ value lower than $0.5\mu \mathrm{M}$ for both the new-protein and new-drug split settings and a mean $K_{\mathrm{d}}$ of $3.5\mu \mathrm{M}$ for the most challenging both-new split setting. For reference, a $K_{\mathrm{d}}$ value lower than $3\mu \mathrm{M}$ was considered a very strong binding by the original study of this dataset²⁷. As the binding-affinity datasets often contain inherent noise or technical errors, ML models can be greatly affected and generate uncertain predictions. Consequently, top predictions from uncertainty-agnostic models often include false positives. In contrast, KDBNet’s estimated uncertainty offers a further dimension of information, complementing mean-affinity predictions and facilitating the prioritization of lead candidates with high confidence.

Discussion

We have presented KDBNet, a geometric deep learning algorithm for predicting kinase–drug binding affinity. KDBNet integrates the 3D structure data of both kinases and drugs and models them using structure-aware GNNs to predict binding affinity. Although we focused on the binding activities of kinases in this work primarily because of their important pharmacological implications^1,44, KDBNet’s principles can be extended to other proteins as well. Our experiments have demonstrated that KDBNet outperformed several existing deep learning methods in predicting kinase–drug binding affinity. Additionally, KDBNet provides well-calibrated uncertainties that scale with prediction errors, offering statistically indicative confidence intervals. We further showcased KDBNet’s practical utility in both active learning and BO frameworks for prioritizing kinase–drug pairs with strong binding affinities. In future work, KDBNet’s performance for binding prediction and uncertainty quantification can be enhanced by integrating recent techniques such as contrastive learning on low-coverage data³¹ (Supplementary Note 1.6), computation-efficient uncertainty-quantification algorithms²⁶ (for example, conformal predictions^45,46) and new calibration techniques^42,45,47,48

We note that the availability of kinase and compound structures is not a critical limitation of KDBNet, as the compound structures are largely available in PubChem⁴⁹, the number of kinase structures is increasing in PDB^20,50, and high-quality predicted structures from AlphaFold²¹ can be used as reasonable proxies for understudied kinases with no solved structures (Supplementary Note 1.7). We also found that replacing the PDB structures input to KDBNet with AlphaFold-predicted structures resulted in comparable prediction accuracy (Supplementary Fig.5). Another line of recent studies examined binding-affinity prediction on the basis of 3D protein–compound binding complex^{32–35,37–41}. The problem setup considered in this work, which was also used in several concurrent studies^51,52, is less restricted by data availability than those works, as KDBNet only requires separate structures as input rather than the binding complex.

KDBNet’s integration of uncertainty estimation is particularly valuable for biological discovery processes when data are limited and uncertainty is prevalent. ML-guided discovery can be affected by biases in the ML model stemming from data noise, small sample size and the model’s intrinsic uncertainty. KDBNet’s uncertainty quantification and recalibration serve as crucial safeguards against biased or over-confident model predictions, which are particularly useful for guiding both exploitation and exploration in virtual drug screening, allowing the prioritization of lead predictions for further validation and proposing data samples to explore previously uncharted regions or address model uncertainties. KDBNet creates an interactive cycle between computation and experiment to improve the ML model’s sample efficiency and success rate in drug screening⁵³. We anticipate that KDBNet can facilitate the reliable and robust deployment of ML-guided drug discovery and lead to the identification of promising therapeutic candidates with higher precision and efficiency.

Methods

Structure and sequence data

Most protein kinases share a common structural fold with two lobes connected by a flexible region that forms the adenosine triphosphate and substrate binding site. The activation loop in this region, typically in a length of 20–30 residues, is crucial for binding activity. We use the pocket structure, rather than the entire structure of the kinase, as the structure input of a kinase to KDBNet because (1) residues in the pocket directly interact with the drug molecule, largely determining the binding activity; and (2) structure elements outside the pocket–that is, the N- and C-terminal lobes-are relatively conserved across different kinases and may not directly coordinate the binding as they are relatively far from the binding sites. Several structure conformations may exist for the same protein kinase in the RCSB PDB database⁵⁴ because the loop can fold into catalytically active and inactive states; we thus first selected the representative PDB structure for a kinase following a recently developed nomenclature for the active and inactive states of protein kinases^50,55. To extract the binding pocket, we used the KLIFS database²⁰ that defines a pocket formed by 85 residues that cover the binding sites in a wide range of kinase inhibitors by analysing around 1,200 kinase-ligand binding crystal structures (Supplementary Note 1.1). In total, we obtained the pocket structure of 283 kinases.

The sequence of amino acids (AAs) of the 85 residues in the binding pocket of a kinase was obtained from its reference protein sequence in UniProt⁵⁶. We did not use the associated sequence in the PDB file because it may contain missing or inaccurate residues. To do this, we mapped the PDB pocket sequence to the full UniProt sequence using pairwise local alignment (score matrix:BLOSUM62, gap open penalty: 10, gap extend penalty: 0.5). We successfully mapped 281 of 283 structures to UniProt sequences.

The 3D structure data in the structure-data format (SDF) of compounds in the kinase–drug binding datasets (described below) were downloaded from the PubChem database⁵⁷.

Kinase–drug binding datasets

For evaluation, we used two public datasets of experimental measurements of binding affinity, Davis²⁷ and KIBA²⁸ (Supplementary Note 1.2), which were widely used to benchmark previous methods^{10,12,15–17}. The Davis study contains binding-affinity measurements of kinase–compound pairs, represented by $K_{\mathrm{d}}$ values ranging from 0.1 to 10,000 nM, where a lower $K_{\mathrm{d}}$ value indicates stronger binding affinity. The KIBA dataset derived a score to integrate three bioactivity values: $K_i$ (inhibitory constant), $K_{\mathrm{d}}$ and IC₅₀ (half maximal inhibitory concentration). We removed from both datasets compounds and kinases that do not have 3D structures in the PDB and PubChem databases. We successfully retrieved the 3D structures for nearly all compounds and 50%−70% of proteins in Davis and KIBA (Supplementary Tables 1,2). We expect the availability of kinase structure data will keep increasing (see discussions in Supplementary Note 1.7). Raw $K_{\mathrm{d}}$ values in the Davis dataset were transformed to $\mathrm{p}{K}_{\mathrm{d}}$ (binding affinity) values, defined as $\mathrm{p}{K}_{\mathrm{d}}=-{\mathrm{l}\mathrm{o}\mathrm{g}}_{10}\left(K_{\mathrm{d}}/{10}^9\right)$, to facilitate numerical stability during model training^12,41. We created four train–test split settings (Supplementary Note 1.2) to evaluate prediction performance on data of unseen drugs (‘new-drug split’), unseen proteins (‘new-protein split’) or both (‘both-new split’) and unseen proteins with low (<50%) sequence identity (‘seq-id split’). To compare KDBNet with baseline methods that predict binding affinity from binding complex data, we additionally created another benchmark task using the PDBBind dataset³⁶. Further details about the creation of benchmark datasets are provided in Supplementary Note 1.2.

Representation of protein structure

The 3D PDB structure of a protein is given as 3D coordinates of the backbone $𝒞={\left\{{\mathbf{c}}_i\in {\mathbb{R}}^3\right\}}_{i=1}^N$, where $N$ is the number of residues, ${\mathbf{c}}_i$ is the coordinate of the $C_{\alpha }$ atom of the $i$ th residue, and $\mathbb{R}$ is the set of real numbers. We represent the protein structure as a graph ${𝒢}_{\mathrm{p}}=\left({𝒱}_{\mathrm{p}},{ℰ}_{\mathrm{p}}\right)$, where nodes ${𝒱}_{\mathrm{p}}$ are residues and edges ${\epsilon }_{\mathrm{p}}$ indicate residue contacts. In this work, we define a pair of residues as being in contact if the Euclidean distance between their $C_{\alpha }$ atoms is within 8 Å (ref. 58).

To make the structure graph representation more informative, we associate every node or edge in the graph with a feature vector. Intuitively, we want our node and edge features to be (1) invariant to rotation and translation so the features not depend on the placement, orientation and centring of the PDB structure inputs; and (2) informative about the local structure, as unique structural motifs may lead to distinct binding affinities. Here, we derive a set of invariant spatial features following a previous study⁵⁹. We further extend their approach to include other features that encode sequence and evolutionary properties of residues. The constructions of node and edge features are described below.

Node features: For every residue, we build three types of features: (1) sequence feature, (2) geometric feature and (3) evolutionary feature. The sequence feature is a one-hot representation to indicate the AA type (of the total 20 possible AAs) of the residue. For geometric features, we compute the three dihedral angles (${\varphi }_i,{\psi }_i,{\omega }_i$) on the basis of the backbone coordinates of residue $i$. These angles are encoded as a vector of cosine and sine values: ${\mathbf{v}}_i=\left(\mathrm{s}\mathrm{i}\mathrm{n}{\varphi }_i,\mathrm{s}\mathrm{i}\mathrm{n}{\psi }_i,\mathrm{s}\mathrm{i}\mathrm{n}{\omega }_i,\mathrm{c}\mathrm{o}\mathrm{s}{\varphi }_i,\mathrm{c}\mathrm{o}\mathrm{s}{\psi }_i,\mathrm{c}\mathrm{o}\mathrm{s}{\omega }_i\right)$. Lastly, for evolutionary features, we ran ESM⁶⁰, a recent protein language model trained on 250 million sequences, to generate the embedding for each residue. The ESM embeddings have been shown to encode structural, functional and evolutionary properties of the protein and can improve a wide range of protein-related prediction tasks, such as function and structure prediction^60,61. Those three features are concatenated together as the node feature for a residue.

Edge features: To characterize the local structure surrounding residue $i$, we create edge features that describe the spatial relationships between residue $i$ and its neighbours (residuesj’s). In particular, we compute an orientation matrix $O_i$ that defines the local coordinate frame for residue $i$⁵⁹:

$$\begin{gathered} O_i=\left[b_i,n_i,b_i\times n_i\right],\text{where}{\mathit{u}}_i=\frac{{\mathit{c}}_i-{\mathit{c}}_{i-1}}{\parallel {\mathit{c}}_i-{\mathit{c}}_{i-1}\parallel }, \\ {b}_i=\frac{u_i-u_{i+1}}{\parallel u_i-u_{i+1}\parallel },n_i=\frac{u_i\times u_{i+1}}{\parallel u_i\times u_{i+1}\parallel }, \end{gathered}$$ (1)

where ${\mathbf{c}}_i\in {\mathbb{R}}^3$ is the coordinates of residue $i$. For an edge ($i,j$), we consider an edge representation that reflects the local distance, direction, orientation and relative positions⁵⁹:

$$e_{ij}=\left(\text{RBF}\left(‖c_j-c_i‖\right),O_i^T\frac{c_j-c_i}{‖c_j-c_i‖},q\left(O_i^T{O}_j\right),E_{\text{pos}}\left(c_j-c_i\right)\right),$$ (2)

The edge feature ${\mathit{e}}_{ij}$ has four components: (1) The first part, $\mathrm{R}\mathrm{B}\mathrm{F}\left(∥{\mathbf{c}}_j-{\mathbf{c}}_i∥\right)$, is the distance encoding embedded into radial basis functions (RBFs). We use 16 RBFs with centres evenly spaced between 0 and 8 Å. (2) The second term is the direction encoding that corresponds to the relative direction of $c_j$ in the local frame of residue $i$. (3) The third term is the orientation encoding of the quaternion representation $\mathbf{q}(\cdot )$ of the spatial rotation matrix $O_i^T{O}_j$. (4) The last term, $E_{\text{pos}}\left({\mathbf{c}}_j-{\mathbf{c}}_i\right)$, encodes the relative distance and direction between residues $i$ and $j$. We used the relative positional encoding⁶², an extension of the positional encoding introduced in the Transformer model⁶³. The relative positional embedding represents the vector pointing to ${\mathbf{c}}_j$ from ${\mathbf{c}}_i$ through a sinusoidal function. We keep the sign of the distance vector ${\mathbf{c}}_j-{\mathbf{c}}_i$ because protein sequence structures are generally asymmetric.

Representation of molecule structure

KDBNet also incorporates the 3D molecular structure of compounds to predict binding affinities. Similarly, given the 3D coordinates of atoms in the molecule, we represent the molecule structure as a graph ${𝒢}_{\mathrm{d}}=\left({𝒱}_{\mathrm{d}},{ℰ}_{\mathrm{d}}\right)$ where nodes ${𝒱}_{\mathrm{d}}$ are atoms of the molecule and edges ${\epsilon }_{\mathrm{d}}$ are defined for a pair of atoms if their distance is less than 4.5 Å, following ref.34. As molecules do not have a natural backbone as in proteins, we do not derive the angle, orientation and direction features for atoms as we did in the protein graph. Instead, we directly use the 3D coordinates of atoms or edge vectors as node features and edge features, allowing the GNN in KDBNet to learn meaningful geometric representations of the molecule in a data-driven way. The node and edge features of the molecule structure are detailed below.

Node features: For every atom, we include a vector-valued feature and a scalar-valued feature as its node feature. The vector feature is the atom coordinates ${\mathbf{c}}_i\in {\mathbb{R}}^3$. The scalar feature is a list of 66 descriptors of chemical properties^15,16,41, including the atom type, bond degree, number of hydrogen bonds, number of implicit hydrogen bonds and whether the atom is aromatic (Supplementary Table 3).

Edge features: For an edge between atoms $i$ and $j$, we also create a vector feature and a scalar feature. The vector feature is the unit vector in the direction of ${\mathbf{c}}_j-{\mathbf{c}}_i$, and the scalar feature RBF $\left(∥{\mathbf{c}}_j-{\mathbf{c}}_i∥\right)$ is the pairwise distance embedded into 16 Gaussian RBFs with centres evenly spaced between 0 and 4.5 Å.

KDBNet model architecture

The primary components of KDBNet are two GNNs to learn structure representations from the input protein and compound, respectively. The representations produced by the two GNNs are then passed to a FC neural network to predict the binding affinity between the input protein and compound.

Protein GNN.

For the protein GNN, we use Graph Transformer⁶⁴, an effective GNN architecture adapted from the vanilla Transformer model for text data⁶³, to model the kinase structure. Given the protein structure graph ${𝒢}_{\mathrm{p}}=\left\{v_{\mathrm{p}},{ℰ}_{\mathrm{p}}\right\}$, a Graph Transformer model builds $L$ graph convolution layers. The $i$th layer is a non-linear transformation function that maps node $i^{’}$s embedding ${\mathbf{h}}_i^{(\ell -1)}\in {\mathbb{R}}^{{\mathrm{d}}_{\ell -1}}$ to ${\mathbf{h}}_i^{(\ell )}\in {\mathbb{R}}^{{\mathrm{d}}_{\ell }}$ for $i\in \left[N\right]$, $\ell \in \left[L\right]$, where $d_{\ell }$ is the embedding’s dimension at layer $\ell ,N$ is the number of nodes in ${𝒢}_{\mathrm{p}}$ and $L$ is the total number of layers in the GNN. In particular, when $\ell =0$, the embedding ${\mathbf{h}}_i^{\left(0\right)}\in {\mathbb{R}}^{{\mathrm{d}}_0}$ is just the node feature of residue $i$. In addition, we have edge features of each edge ($i,j$) denoted as ${\mathbf{e}}_{ij}\in {\mathbb{R}}^{{\mathrm{d}}_e}$, where ${\mathrm{d}}_e$ in the dimension of input edge features.

Formally, in the $\ell$-th Graph Transformer layer of the GNN, the hidden representation ${\mathbf{h}}_i^{(\ell )}$ is updated by performing a message passing between node $i$ and its neighbours

$$h_i^{\left(\ell \right)}=W_1^{\left(\ell \right)}{h}_i^{\left(\ell -1\right)}+\sum _{j\in 𝒩\left(i\right)}{\alpha }_{i,j}\left(W_2^{\left(\ell \right)}{h}_j^{\left(\ell -1\right)}+W_3^{\left(\ell \right)}{e}_{ij}\right),$$ (3)

where $𝒩\left(i\right)$ is the set of neighbour nodes of node $i$ in the graph, ${\mathbf{\text{W}}}_1^{\left(\ell \right)}\in {ℝ}^{{\text{d}}_{\ell -1}\times {\text{d}}_{\ell }}$, ${\mathbf{\text{W}}}_2^{\left(\ell \right)}\in {ℝ}^{d_{\ell -1}\times d_{\ell }}$ and ${\mathbf{\text{W}}}_3^{\left(\ell \right)}\in {ℝ}^{d_e\times d_{\ell }}$ are learnable parameters of the GNN, and ${\alpha }_{i,j}$ is the attention weight used to aggregate messages. The weights ${\alpha }_{ij}$ are computed using self-attention:

$${\alpha }_{i,j}=\text{softmax}\left(\left[{\left(W_4^{\left(\ell \right)}{h}_i^{\left(\ell -1\right)}\right)}^{\top }\left(W_5^{\left(\ell \right)}{h}_j^{\left(\ell -1\right)}+W_3^{\left(\ell \right)}{e}_{ij}\right)\right]/\sqrt{d_{\ell }}\right),$$ (4)

where ${\mathbf{W}}_4^{(\ell )}\in {\mathbb{R}}^{d_{\ell -1}\times d_{\ell }}$ and ${\mathbf{W}}_5^{(\ell )}\in {\mathbb{R}}^{d_{\ell -1}\times d_{\ell }}$ are learnable parameters and $d_{\ell }$ is the length of vector ${\mathbf{h}}_i^{\left(e\right)}$.

We stack three Graph Transformer layers and use the Leaky ReLU activation function⁶⁵ between two adjacent layers. After the final layer, we use the global add pooling operation as the readout function to aggregate all node representations into a summary representation ${\mathbf{h}}^{\mathrm{p}}\in {\mathbb{R}}^{256}$ of the input protein: ${\mathbf{h}}^{\mathrm{p}}=\mathrm{A}\mathrm{D}\mathrm{D}\left(\left\{{\mathbf{h}}_i^{\left(L\right)}\mid i=1,\dots ,N\right\}\right)$.

Molecule graph neural network.

Given the molecule structure graph ${𝒢}_{\mathrm{d}}=\left\{{𝒱}_{\mathrm{d}},{ℰ}_{\mathrm{d}}\right\}$, we also use a GNN to learn the representation for the input molecule. Recall that in graph ${𝒢}_{\mathrm{d}}$, we associate each node and edge with both geometric vector features (for example, 3D coordinates) and scalar features (for example, descriptors of chemical properties). We thus use a specialized layer, geometric vector perceptrons (GVPs)²², to build the molecule GNN. The key advantage of GVP is that it has special consideration for 3D data in design (Supplementary Note 1.8) and allows KDBNet to learn structure representations directly from the raw atom coordinates in ${\mathbb{R}}^3$ without requiring the construction of features invariant to rotations and translation, such as relative direction embeddings. In the GNN, the GVP layer can be used as a drop-in replacement for MLPs, such as ${\mathbf{W}}_1^{(\ell )},{\mathbf{W}}_2^{(\ell )},{\mathbf{W}}_3^{(\ell )}$ in the protein GNN (equation (3)).

Formally, we use the tuple ${\mathbf{v}}_i=\left({\mathbf{v}}_i^{\nu },{\mathbf{v}}_i^s\right)$ to denote the node feature of atom $i$ (the superscripts $v$ and $s$ stand for vector and scalar, respectively), where ${\mathbf{v}}_i^{\nu }\in {\mathbb{R}}^{\mu \times 3}$ is a list of vector features in ${\mathbb{R}}^3$ and ${\mathbf{v}}_i^s\in {\mathbb{R}}^{\nu }$ is a list of scalar features ($\mu$ and $v$ are the number of vector features and scalar features, respectively). The edge feature ${\mathbf{e}}_{ij}=\left({\mathbf{e}}_{ij}^v,{\mathbf{e}}_{ij}^s\right)$ of edge $(i,j)$ has similar meaning. The molecule GNN transforms the node and edge features through $L$ graph convolution layers to obtain the representation of the input molecule. Specifically, in the $i$th layer, each node aggregates ‘messages’ (embeddings) from neighbouring nodes and edges and then updates its own representations:

$$h_i^{\left(\ell \right)}=h_i^{\left(\ell -1\right)}+g\left(h_i^{\left(\ell -1\right)}+\frac{1}{\left|𝒩\left(i\right)\right|}\sum _{j\in 𝒩\left(i\right)}{m}_{ji}^{\left(\ell \right)}\right),$$ (5)

where $g(\cdot )$ is a sequence of three GVP layers, $𝒩\left(i\right)$ is the set of neighbour nodes of node $i$ in ${𝒢}_{\mathrm{d}},{\mathbf{h}}_i^{(\ell )}$ is the embedding of node $i$ in layer $\ell$ (in particular, ${\mathbf{h}}_i^{\left(0\right)}={\mathbf{v}}_i$ is the node feature), and ${\mathbf{m}}_{ji}^{(\ell )}$ is the ‘message’ passed from node $j$ to node $i$, computed using another sequence of GVP layers: ${\mathbf{m}}_{ji}^{(\ell )}=g\left(\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\left({\mathbf{h}}_j^{(\ell -1)},{\mathbf{e}}_{ji}\right)\right)$, where $\text{concat}(\cdot )$ is the concatenation operation of two embeddings. Similar to the protein GNN, after the final layer of the molecule GNN, we also apply the global add pooling operation to aggregate all node representations into a scalar representation ${\mathbf{h}}^{\mathrm{d}}\in {\mathbb{R}}^{128}$ of the input drug.

In addition to the molecule GNN, the GVP layers can also be used to build the protein GNN. As introduced above, the default architecture of the protein GNN was on the basis of the Graph Transformer layers as we found in our local tests that GVP- and Transformer-based protein GNNs lead to comparable prediction accuracy (Supplementary Fig. 5), but the latter took 50% less training time and GPU memory use. We thus chose Transformer layers as the default building blocks for the protein GNN. Nevertheless, for larger training sets that cover diverse protein families rather than only kinases, we expect GVP layers to be more effective for learning structure representations, as they are able to learn many other implicit geometric features that go beyond the manually defined features used in the Transformer-based GNN.

Prediction module, hyperparameter tuning and model training.

We tuned the hyperparameters of KDBNet by performing a small-scale grid search using the training data, such that seven-eighths of the training data were used to train a model with a specific set of hyperparameters, and the remaining one-eighth of the data were used as the validation set to select the hyperparameters. The test split was not used for hyperparameter selection. We tested combinations of GNN layer dimensions from {64, 128, 256, 512, 1, 024}, combinations of FC layer dimensions from {64, 128, 256, 512, 1, 024}, the number of FC layers in {1, 2, 3} and the dropout rate in {0.1, 0.25, 0.5}.

By performing nested cross-validation on the training data, we decided to use three layers with sizes 128, 256 and 256 for the protein GNN and three layers with uniform size 128 for the molecule GNN, which were robust across different settings in our experiments. The two representations of protein and drug structures generated by the GNNs, ${\mathbf{h}}^{\mathrm{p}}$ and ${\mathbf{h}}^{\mathrm{d}}$, are then projected to dimension 128 using two FC layers with sizes 1,024 and 128 and a dropout rate of 0.25. The two projected embeddings are then concatenated and passed to a two-layer FC neural network with sizes 1,024 and 512 and a dropout rate of 0.25, followed by a single scalar output as the predicted binding affinity between the input protein and drug.

The training objective of KDBNet is to minimize the MSE between the predicted binding affinity and the true affinity value. The model is trained using the Adam optimizer with a learning rate of 0.0005. We trained all models for 500 epochs.

Uncertainty quantification

We equipped KDBNet with an uncertainty-quantification module. This was achieved by training an ensemble of $M$ independent model replicates²⁴, which has been widely demonstrated as an effective way to estimate uncertainty⁶⁶. The $M$ model replicates had the same neural network architectures and hyperparameters, but the learnable parameters were initialized with different random seeds. We set $M=8$ in this work unless otherwise specified. Specifically, let ${\hat{y}}_k\left(x_i\right)$ be a prediction given by the $k$th individual model, where $x_i$ represents the input kinase–drug pair. KDBNet’s final prediction of binding affinity $\mu \left(x_i\right)$ and its estimated uncertainty $\sigma \left(x_i\right)$ are given by the mean and standard deviation (s.d.) of the individual model’s predictions:

$$\mu \left(x_i\right)=\frac{1}{M}\sum _{k=1}^M{\widehat{y}}_k\left(x_i\right),\sigma {\left(x_i\right)}^2=\frac{1}{M}\sum _{k=1}^M{\left({\widehat{y}}_k\left(x_i\right)-\mu \left(x_i\right)\right)}^2$$ (6)

The uncertainty $\sigma \left(x\right)$ estimated by KDBNet above is known as epistemic uncertainty. In the literature, uncertainties are often categorized into aleatoric uncertainty (data uncertainty due to inherent noise in observations) and epistemic uncertainty (model uncertainty due to uncertainty in parameters or predictions; Supplementary Note 1.9). In this work, we focus on estimating epistemic uncertainty, as many recent studies have demonstrated the utility of epistemic uncertainty for discovery in various domains, including biology¹⁷, chemistry⁶⁷ and healthcare⁶⁸. Nevertheless, KDBNet can be extended to estimate aleatoric uncertainty by modifying the objective function from an MSE minimization to a maximum likelihood estimation^24,69.

Active learning

We started training KDBNet on a random 1% subset of KIBA training data. At each subsequent round, KDBNet predicted binding affinities and uncertainties for the remainder of the training data and then ranked them on the basis of the score function $s\left(x\right)=\sigma \left(x\right)$, where $\sigma \left(x\right)$ is the predicted uncertainty for sample $x$ (hereinafter referred to as the ‘explorative’ strategy) and where a sample $x$ represents an input kinasedrug pair. We then added the top $T$ samples with the greatest uncertainties to the training set and retrained KDBNet from scratch with the expanded training set. In our experiments, we performed seven rounds of active learning. The number of samples to acquire $\left(T\right)$ for each round was determined such that 10%, 20%, 30%, 40%, 50%, 75% and 100% of the training samples were used to retrain the model in each of the seven rounds, respectively. Two other types of score function $s\left(x\right)$ were considered for comparison: (1) ‘greedy’, where samples with higher predicted affinity receive higher scores, $s\left(x\right)=\mu \left(\mathbf{x}\right)$; and (2) ‘random’, where samples receive random scores, $s\left(x\right)\approx 𝒰(0,1)$, that is, the continuous uniform distribution between zero and one. The performance was evaluated on the ‘both-new’ test set.

Uncertainty recalibration

There are two widely used definitions of regression calibration in the literature: confidence-interval-based calibration⁴² and error-based calibration⁴⁷. Under confidence-based calibration, a model is said to be well-calibrated if $e\%$ of its predictions fall in the $e\%$ predicted confidence interval $(0\le e\le 1)$⁴², whereas error-based calibration defines a well-calibrated model as one for which the uncertainty estimate of a prediction, in expectation, equals the prediction errors⁴⁷. Several approaches have been proposed to recalibrate regression models^42,47,48. The general idea is to learn a post hoc transformation function, which receives the model’s predicted uncertainties as input and outputs the transformed uncertainty estimates that would be better calibrated. In our method, we use a simple yet effective scaling approach^47,70 to recalibrate the uncertainty estimates. Specifically, we transform the model’s output $\left(\mu \left(x_i\right),\sigma \left(x_i\right)\right)$ to $\left(\mu \left(x_i\right),r\sigma \left(x_i\right)\right)$, where $r$ is the scaling factor to be learned. Note that the model’s prediction of binding affinity $\mu \left(x_i\right)$ does not change. To learn the scaling factor $r$, we introduce an optimization problem in which the objective is to minimize the miscalibration area (Supplementary Note 1.5). The recalibration is a post hoc process, meaning the model’s predicted uncertainties are fixed and only $r$ is optimized. As indicated previously⁴², the recalibration is performed on a held-out validation set that has not been used for model training. We use Brent’s method⁷¹ implemented in the SciPy package⁷² to solve this single-variable optimization.

Extended Data

Extended Data Fig. 1 ∣. Prediction performance evaluation on KIBA dataset.

(a) Four train-test split settings of evaluation, where the model is evaluated on data of unseen drugs (‘new-drug split’), unseen proteins (‘new-protein split’) or both (‘both-new split’), and unseen proteins with low (<50%) sequence identity (‘seq-id split’). (b) Comparisons of prediction performance of KDBNet with KronRLS, DeepDTA, GraphDTA, DGraphDTA, EnzPred, and ConPLex on the KIBA dataset using four train-test split settings. The performances of GP were not shown as it was not evaluated in the original study¹⁷ and it is computationally costly to run GP at the scale of KIBA dataset because of the high memory footprint of kernel computation. Performances were evaluated using three metrics, including Pearson correlation, Spearman correlation, and mean squared error (MSE) between predicted and true KIBA scores²⁸. All bar plots represented the mean ± SD of evaluation results on five random train/test splits. Abbreviations: seq. id.: sequence identity.

Data availability

The two kinase–drug binding-affinity datasets, Davis²⁷ and KIBA²⁸, were curated by and available in the Therapeutics Data Commons benchmark⁷³. The PDBbind dataset (v.2020) was downloaded from http://www.pdbbind.org.cn/. The PDB codes of representative structures of kinases were obtained from the Kincore database⁵⁰ (http://dunbrack.fccc.edu/kincore/home). The binding pocket structure of each kinase was downloaded from the KLIFS database²⁰ (https://klifs.net/). Full AA sequences of kinases were obtained from UniProt⁵⁶ (https://www.uniprot.org/). The 3D molecular structures were downloaded from PubChem⁵⁷ (https://pubchem.ncbi.nlm.nih.gov/). Our processed version of the binding-affinity datasets and the identifier list of kinases and drugs are available on our GitHub repository (https://github.com/luoyunan/KDBNet).