Combined physics- and machine-learning-based method to identify druggable binding sites using SILCS-Hotspots

Abstract

Identifying druggable binding sites on proteins is an important and challenging problem, particularly for cryptic, allosteric binding sites that may not be obvious from X-ray, cryo-EM, or predicted structures. The Site-Identification by Ligand Competitive Saturation (SILCS) method accounts for the flexibility of the target protein using all-atom molecular simulations that include various small molecule solutes in aqueous solution. During the simulations the combination of protein flexibility and comprehensive sampling of the water and solute spatial distributions can identify buried binding pockets absent in experimentally-determined structures. Previously, we reported a method for leveraging the information in the SILCS sampling to identify binding sites (termed Hotspots) of small mono- or bicyclic compounds, a subset of which coincide with known binding sites of drug-like molecules. Here we build on that physics-based approach and present a ML model for ranking the Hotspots according to the likelihood they can accommodate drug-like molecules (e.g. molecular weight > 200 daltons). In the independent validation set, which includes various enzymes and receptors, our model recalls 67% and 89% of experimentally-validated ligand binding sites in the top 10 and 20 ranked Hotspots, respectively. Furthermore, we show that the model’s output Decision Function is a useful metric to predict binding sites and their potential druggability in new targets. Given the utility the SILCS method for ligand discovery and optimization the tools presented represent an important advancement in the identification of orthosteric and allosteric binding sites and the discovery of drug-like molecules targeting those sites.

Graphical Abstract

Introduction

There has been no time like the present for structure-based drug design (SBDD) given the number of protein structures solved at or near atomic resolution currently available in the Protein Data Bank,¹ with >200,000 experimental structures and >1,000,000 computed structure models,² and the >200,000,000 computed structures in the AlphaFold Database.³ These structural models cover a plethora of potential drug targets.⁴ Furthermore, just as GPUs have revolutionized deep-learning models for protein structure prediction,^3,5,6 they have also brought all-atom molecular dynamics (MD) simulations of large proteins at meaningful timescales into routine reach.^7,8 This combination, along with advances in our understanding of the molecular nature of disease and the associated growth of personalized medicine, has the potential to produce many new therapeutic agents.

After target identification, the critical first step in the SBDD process is either to identify binding sites of known ligands or identifying candidate sites for virtual screening. Historically, computational binding pocket identification was first carried out using the protein molecular surface defined with the LJ potential and a grid of lattice points sampling the space around that surface.⁹ Standard methods still often use geometric analysis,^10–12 in addition to molecular docking, and/or machine-learning.¹³ When a representative structure is available and the binding pocket is relatively well-defined, methods including FTMap^14–16 and Fpocket¹⁷ are effective, as well as the widely-used methods related to common CADD software packages, such as SiteMap^18,19 (Glide/Schrödinger),²⁰ SiteFinder²¹ (MOE/Chemical Computing Group), or AutoLigand²² (AutoDock).²³ Some methods employ template based modeling to predict binding sites when only a sequence is known.^24–27 PepSite uses 3D grids of position-specific scoring matrices to efficiently identify linear peptide binding sites across the proteome, an interesting approach for a highly-specialized class of ligand-protein interactions.²⁸ There are many machine-/deep-learning models^13,29 that incorporate geometry, sequence-homology, structural features, molecular docking, and/or consensus to predict ligand binding sites.^30–36 The recently published AlphaFold 3 model claims to predict protein-ligand interactions with higher fidelity than standard docking methods,³⁷ although the web server available for non-commercial researchers only predicts sites for nineteen common cofactors like ATP and citric acid. To remain highly computationally efficient, methods reliant on static structures necessarily neglect protein backbone flexibility, thus cannot capture protein allostery or cryptic binding sites.^38–42 In addition, the traditional molecular docking approaches used in available methods,^{43,20,23,44,45} while efficiently sampling known ligand-protein interactions,^16,34 rely on continuum electrostatic models and/or statistical potentials to estimate the energetics of binding. Such methods are limited in their ability to accurately account for the complex balance of enthalpic and entropic costs and desolvation contributions that contribute to ligand binding.

A powerful way to overcome these limitations is through the use of MD simulations, and of particular interest, all-atom cosolute MD simulations.^46,47 Alternatively, a key example of a natural, non-cosolute approach to incorporating dynamics into site prediction is to utilize enhanced sampling or coarse grained simulations to sample pocket openings, and include the resulting dynamics in the inputs to a ML model, such as the method CryptoSite.⁴² On the other hand, cosolute methods are conceptually similar to experimental fragment-based drug design^48,49 wherein proteins are co-crystallized with various small solutes to determine their binding sites.⁵⁰ In general, cosolute methods involve solvating the target biomolecule with various small molecules and performing molecular simulations to analyze the distribution of the molecules over the course of the simulation. This approach is widely-employed^51–56 including by MDmix,^46,57 pMD-Membrane,^58,59 Mix-MD,^60–62 SWISH and SWISH-X,^63,64 Cosolvent Analysis Toolkit (CAT),⁶⁵ and SILCS.^47,66,67 The coarse grain MD cosolute method Colabind was recently released,⁶⁸ which allows substantially faster sampling than all-atom MD, but with corresponding accuracy sacrifices. The success of the all-atom cosolute MD methods is due to advances in efficient, GPU-enabled molecular dynamics software packages,^69–72 combined with consistent improvements in the accuracy of all-atom force fields,^73–77 such that accurate sampling of the interactions of solutes with flexible proteins in the presence of explicit atomistic water is readily achievable.

Specifically, the present study is based on the SILCS methodology. SILCS samples the protein conformational ensemble in the presence of multiple solutes and water while alternating between an oscillating chemical potential Grand Canonical Monte Carlo (GCMC) sampling scheme and conventional MD^78,79 that dramatically accelerates the rates of penetration of solutes and water into hydrophobic pockets and other buried cavities. After extensive sampling, the occupancies of the solute molecules and water are converted to functional group-type specific free energy maps, or FragMaps. An example of the FragMaps surrounding the protein TEM-1 β-lactamase is depicted in Figure 1A, and Figure 1B shows molecular renderings of the 8 solutes used in the standard SILCS simulations. These FragMaps form the basis for all subsequent analysis in SILCS, such as performing molecular docking of small molecules in the field of the maps.^80,81 In a previous paper, a method was presented for identifying a comprehensive set of fragment binding sites, or Hotspots, on proteins,⁸² and subsequently applied to RNA.⁸³ Although some Hotspots correspond with the known binding sites of small molecules (Figure 1C), it was unclear which Hotspots were really ‘druggable’ using only the previous method. Here we define druggable as being suitable for binding drug-like molecules, such as those with molecular weight (MW) > 200 Da.

Figure 1: Example SILCS FragMap and Hotspots and depiction of the SILCS solutes.

A) TEM-1 β-lactamase is rendered in NewCartoon style (PDB: 1JWP), with the various FragMaps contoured at −1.2 kcal/mol. The green map corresponds to generic apolar carbons (propane and benzene carbon), the red corresponds to hydrogen-bond acceptors, the blue corresponds to hydrogen-bond donors, the cyan corresponds to positive charges (methylammonium nitrogen), the orange corresponds to negative charges (acetate oxygen), gold corresponds to alcohols (methanol oxygen), and the solid tan surface is the Exclusion map. B) Depiction of the 8 solutes used in the SILCS GCMC/MD simulations, namely: benzene, propane, methylammonium, acetate, imidazole, formamide, dimethyl ether, and methanol. The molecules are rendered in CPK style, where cyan atoms are carbons, red atoms are oxygen, blue atoms are nitrogen, and white atoms are hydrogen. C) Depiction of TEM-1 in NewCartoon style, with the Hotspots rendered as pink spheres, and with the crystallographic ligands from PDBs 1ERO and 1PZO. The ligands are colored as in panel B).

In this study we present a new set of tools to identify Hotspots that contribute to binding sites for drug-like molecules. The method first calculates a range of properties characterizing each Hotspot, which are then used as features in a machine learning (ML) algorithm that predicts the likelihood of each Hotspot participating in a drug-like binding site. For model training Hotspots identified as being in a druggable site were 1) within 12 Å of at least one adjacent Hotspot, 2) within 5 Å of the non-hydrogen atoms of a crystal location of a drug-like ligand, and 3) partially buried. The first criteria assumes that a drug-like molecule is comprised of a minimum of two linked fragments. The second criteria is experimental validation of Hotspots being located in a site which binds a drug-like molecule through X-ray crystallography. The third criteria is based on the assumption that binding sites are pockets in which the ligands are partially buried^84–86 as determined by an empirical relative buried surface area cutoff described below. For the training set, the developed ML model identifies 76% and 80%, of druggable sites in the top 10 and 20 Hotspots, respectively. In the validation set it recovers 67% and 89% of druggable sites in the top 10 and 20 total Hotspots, respectively.

Methods

SILCS workflow

The overall workflow was to run standard SILCS GCMC/MD simulations of the target proteins solvated in water with a variety of solute molecules (Figure 1B) at 0.25 M for a total of 1 μs as previously described.^47,67 Analysis of the occupancies, and therefore free energy affinities, of each solute gives an atom-type specific 3D affinity map (FragMap) over the entire 3D space of the protein, as well as an Exclusion map containing all the voxels with zero solute or water occupancy (Figure 1A). The PDB identifiers of the protein structures used for the SILCS simulations are provided in Table S1. Note that wherever possible, an apo structure was used for the SILCS simulations; else, a structure with minimal ligand size was used. Any ligands were removed from the structure prior to the simulations. For transmembrane proteins, the membrane orientation was determined using the PPM (Positioning of Proteins in Membranes) webserver,^87,88 after which a bilayer composed of 1-palmitoyl-2-oleoyl-sn-glycero-3-phosphocholine (POPC) and cholesterol (9:1 ratio) was constructed using the CHARMM-GUI webserver.^89,90 The CHARMM-GUI webserver was also used to generate small missing loops (<12 amino acids) and to adjust the protonation state of titratable residues.^89,90 The protonation state of titratable residues at pH 7.0 was determined using PropKa3.⁹¹ The FragMaps were obtained from our previous study⁸² that were performed using SILCS software version 2019 (SilcsBio LLC) and Gromacs version 2019, except for ANGPTL4, TEM-1, NKG2D, and GABA_BR, for which SILCS software version 2023⁹² and Gromacs version 2022 were used.^69,70 The SILCS simulations are based on a published GCMC/MD approach⁷⁸ that has not been changed beyond porting the GCMC code to GPUs⁷⁹ that is implemented in version 2023. The computations for each set of SILCS FragMap using version 2023, were carried out in parallel on ten compute nodes each with 1 GPU (e.g. GTX 980, GTX 1080Ti, RTX 2080Ti) and eight CPU threads (e.g. AMD Ryzen 7 1700, AMD EPYC 7551P), and require between ~1–7 days to complete depending on the system size. The full simulation boxes in this study contain between ~35,000 and ~190,000 atoms.

After calculating the FragMaps, we performed the SILCS-Hotspots calculation as described in our previous work.⁸² The Hotspots calculation consists of comprehensively docking a library 90 mono- and bicyclic fragments⁹³ with MW < 190 Da into the FragMaps and Exclusion map. Then two rounds of clustering are performed to identify binding sites that include one or more of the fragments (Figure 1C). Each original Hotspot is then defined by the number of fragments in that site and the LGFE scores of those fragments from which features such as the minimum (e.g. most favorable) LGFE or mean LGFE over all the fragments in that Hotspot are calculated and used for ranking. The SILCS-Hotspots calculations were run using version 2019, except for all proteins in the validation set, where version 2023 was used.⁹² The SILCS-Hotspots docking performed for this study utilized a GPU implementation of SILCS-MC docking.⁹⁴ The SILCS-Hotspots calculations generated ~6,000 to ~65,000 independent SILCS-MC jobs that each run for ~15 sec total and can be scheduled to run in parallel on a given cluster.

Additional characterization of Hotspots as potential druggable binding sites was performed by screening a database of 348 FDA-approved compounds at selected Hotspots. The docking was carried out in a 5 Å radius sphere centered on the Hotspot. After docking, each Hotspot was characterized by the average LGFE and relative buried surface area (rBSA) for the top twenty molecules, ranked by the LGFE. rBSA is defined as the ratio of the solvent accessible surface area of the ligand alone relative to that of the ligand in the presence of the protein, such that 100% rBSA indicates a fully buried ligand with no solvent accessible surface area (SASA). The SASA of the ligand in both the presence and absence of the protein was based on the conformation of the ligand from the SILCS-MC docking. The 348 compound FDA database was extracted from an initial set of FDA-approved molecules derived from the online databases DrugBank⁹⁵ and Drugs@FDA.⁹⁶ An initial filter was applied to select only molecules with MW between 250 and 500 Da. To reduce the dimensionality while maintaining the diversity of the molecules in the FDA set, we clustered the dataset with Morgan fingerprints using a radius of 2 and Tanimoto similarity index of 0.3, then selected a representative molecule from each cluster, yielding a total of 380 molecules. The final set of 348 molecules was arrived at by manually removing outliers in the number of rotatable bonds or hydrophobic groups. The FDA database is available in sdf and pdf formats on GitHub at https://github.com/mackerell-lab/FDA-compounds-SILCS-Hotspots-SI. The FDA dataset curation and generation of the pdf table of 2D molecular images was done with the python API for RDKit.⁹⁷

Calculation of new analysis features

The Hotspot analysis workflow to calculate features for ML model development consists of three keys steps: cluster adjacent Hotspots within some user-tunable cutoff distance, collect various properties of the individual Hotspots and Hotspot clusters, and then use those features to develop the ML model to identify Hotspots at the binding sites of drug-like molecules. Here we define a Hotspot cluster as containing all the Hotspots within 12 Å of each Hotspot (centroid), because the maximum distance between two neighboring Hotspots in the training set is 11.6 Å. Based on this definition, each individual Hotspot can be a member of multiple Hotspot clusters, though each Hotspot is the centroid of just one Hotspot cluster with the features based on that cluster assigned to the centroid Hotspot.

The new features include the number of protein non-hydrogen atoms in the input PDB file within a user-defined radius of each Hotspot (default 3 Å), the SASA and volume of each Hotspot in the presence of the protein (using a 3 Å radius for the Hotspots), the SASA and volume of the Hotspot clusters, the distances between Hotspots in the cluster, as well as various statistical measures (e.g. mean, minimum, and maximum values) of the distribution of these properties over the Hotspot cluster (Table 1). The protein-derived features are similar to those used in previous ML models.^98,99 As a feature we wanted the calculation of the SASA of a Hotspot in the presence of the protein to account for the protein flexibility that is included in the SILCS simulations. Accordingly, in addition to using the original crystal structure used for the SILCS simulations for the SASA calculation, an “Exclusion-map HS SASA” was calculated where the solvent-accessibility of the Hotspot (default radius 5 Å) was relative to voxels that were included in the SILCS Exclusion map rather than the standard use of the positions of the protein atoms. The different Hotspot radii (3 Å for use with protein PDB file and 5 Å for use with Exclusion map) adjusts for the smaller size of an Exclusion map relative to a corresponding protein. All SASA calculations used a solvent probe radius of 1.4 Å. Additional features using the Exclusion map were calculated as described in Table 1.

Table 1: Names and descriptions of the features calculated by the new SILCS-Hotspots workflow.

The radius of each Hotspot for the SASA calculations can be user-defined separately for the protein coordinates and Exclusion map calculations; defaults are 3 Å and 5 Å, respectively. LGFE stands for Ligand Grid Free Energy of the fragments located in each Hotspot and SASA stands for solvent-accessible surface area.

Name	Description
Orig	Mean LGFE of each Hotspot (Original ranking metric).
Min	Minimum LGFE of each Hotspot cluster.
Ave	Average LGFE of each Hotspot cluster.
NFrag	Number of drug-like fragments in each Hotspot.
N_Heavy_Atoms	Number of protein non-hydrogen atoms within 3 Å of each Hotspot.
N_BBone_Atoms	Number of protein backbone atoms within 3 Å of each Hotspot.
PDB_SASA	SASA of protein atoms occluded by each Hotspot.
Excl_SASA	SASA of protein Exclusion map occluded by each Hotspot.
PDB_HS_SASA	SASA of each Hotspot occluded by the protein.
Excl_HS_SASA	SASA of each Hotspot occluded by the Exclusion map.
Adj_PDB_SASA	SASA of protein atoms occluded by each Hotspot cluster.
Adj_PDB_HS_SASA	SASA of each Hotspot cluster occluded by the protein.
Relative_Adj_SASA	The relative SASA of each Hotspot cluster defined as the ratio of SASA of the Hotspot cluster in the presence of the protein PDB to total SASA of the Hotspot cluster without the protein.
Vol	Volume of each Hotspot excluding the volume overlapping with protein atoms.
Excl_Vol	Volume of each Hotspot, excluding the volume overlapping with the SILCS Exclusion map.
MinDist	Minimum distance between each Hotspot and the other Hotspots in the cluster.
MaxDist	Maximum distance between each Hotspot and the other Hotspots in the cluster.
MidDist	Median distance between each Hotspot and the other Hotspots in the cluster.
AvgDist	Average distance between each Hotspot and the other Hotspots in the cluster.
Sum_<feature>	Sum of <feature> over the Hotspot cluster.
Mean_<feature>	Mean of <feature> over the Hotspot cluster. This is sum divided by the number of Hotspots in the cluster.
Min_<feature>	Minimum of <feature> among Hotspots in the cluster. For example, the value of the most favorable LGFE of the Hotspots in the cluster.
Max_<feature>	Maximum of <feature> among Hotspots in the cluster. For example, the value of the Hotspot with largest Volume in the cluster.

The code to calculate the SASA of Hotspots with respect to the Exclusion map was built on the freeSASA¹⁰⁰ package in python. The freeSASA code was modified to allow for non-default input atomic radii for the Hotspots and Exclusion map voxels. In addition, the SASA of Hotspot clusters was calculated based on the SASA of all the Hotspots in the cluster (default radius 5 Å). The Exclusion map is represented as a set of spheres of radius 1 Å sitting on 1 ų grid voxels. To calculate the volume of the Hotspot clusters not within the protein or Exclusion map a Monte Carlo integration algorithm was implemented. The calculation of the SASA and volume of the Hotspot clusters requires substantial CPU time, and so the algorithms were parallelized with numba.¹⁰¹

Training and validation data set curation

The training set is constructed from the seven protein systems from the previous SILCS-Hotspots paper:⁸² Cyclin-dependent kinase 2 (CDK2) in both active and inactive states,^102,103 Extracellular-signal-regulated kinase 5 (ERK5),¹⁰⁴ Protein tyrosine phosphatase 1b (PTP1B),^105–108 Androgen receptor,^109,110 and three G-protein coupled receptors (GPCRs), namely G protein-coupled receptor 40 (GPR40),^111,112 M2 Muscarinic receptor,^113,114 and β2 Adrenergic receptor.^115,116 The validation set is comprised of eleven proteins, seven of which we recycle from previous SILCS-MC publications.^80,81 namely: P38 mitogen-activated protein kinase,^117,118 Farnesoid X bile acid receptor (FXR),¹¹⁹ β-Secretase 1 (BACE1),^120,121 tRNA methyl transferase (TrmD),¹²² Myeloid cell leukemia 1 (MCL1),^123,124 Heat-shock protein 90 kDa (Hsp90),⁴⁸ and Thrombin.¹²⁵ To those we added the C-terminal domain of the lipid-binding protein angiopoietin-like 4 (ANGPTL4),¹²⁶ TEM-1 β-lactamase,^127–129 Natural killer group 2D receptor (NKG2D),^130,131 and GPCR γ-aminobutyric acid receptor (GABA_BR) in both active and inactive states.^132–134

For each protein system, we identified relevant crystal structures where there is a drug-like ligand bound and aligned these structures to the structure used to generate the SILCS FragMaps. Hotspots within 5 Å of a ligand non-hydrogen atom are classified as a “true hit”. In addition, a Hotspot must be within 12 Å of at least one other Hotspot to be a true hit, and the 12 Å path must be unobstructed by any Exclusion map voxels. In the training set, if a Hotspot is within 5 Å of more than one ligand, it is counted for both ligands to reflect its importance in identifying more than one distinct ligand binding site. The PDB¹ and D3R¹³⁵ structures used are listed in Table S1, and the Hotspots considered true hits are listed in Table S2. In each system, there may be several ligands bound in similar positions available in different PDB files, but only one such ligand was selected to represent that binding site. In a few cases, there are Hotspots that are within 5 Å of the ligand but are located on the surface of the protein above the ligand binding site. Figure S1 depicts one such example, Hotspot 25 in the ERK5 system, which is within 5 Å of the ligand but largely solvent-exposed. As one of our criteria of druggable binding sites was that they are partially buried sites, we removed outlying Hotspots with greater than 300 Ų Exclusion-map HS SASA (Figure S2), as these sites were assumed to not be suitable for binding drug-like molecules. This empirical cutoff corresponds to ~42% rBSA.

Evaluation of model performance

To evaluate the developed models, we calculated precision, recall, weighted F₁, and binding site recall using the Hotspots identified as true hits. Evaluating a Hotspot classification model requires ranking the Hotspots, then selecting a cutoff, such as taking all Hotspots with LGFE < 0 or taking the top N Hotspots. For a given cutoff, precision is the ratio of true hits to the total number of Hotspots up to and including the cutoff, while recall is the ratio of true hits up to and including the cutoff to the total number of experimentally verified hits. For example, if a protein has four total experimentally verified hits, two of which are identified with a cutoff at ten Hotspots, the precision is 2/10 = 0.2 and the recall is 2/4 = 0.5. The weighted F₁ statistic is the population-weighted harmonic mean of precision and recall. This is important because it accounts for the low proportion of Hotspots which are true hits: only 7% of all the Hotspots in the training set are experimentally verified hits and only 2% in the test set. Accordingly, a random predictor would have a precision of ~0.02 for the validation set, which is a useful comparison when evaluating the precision of a model (e.g., 0.2 for the validation set example represents a ten-fold increase over a random predictor). In addition, binding site recall was calculated to compare the performance of the models on the practical problem of identifying at least one Hotspot per ligand. Binding site recall is defined as the ratio of identified ligand binding sites to the total number of experimentally identified ligand binding sites for that protein. A ligand binding site is identified once a single Hotspot within 5 Å of that ligand is identified above a given cutoff. Accordingly, the maximum number of ligand binding sites is equivalent to the total number of experimentally identified ligand binding sites although the total number of Hotspots defined as true hits may be greater than the total number of experimentally identified ligand binding sites. Below the total number of experimentally verified hits is indicated as “# Sites” in the tables.

We note that the calculated performance of the models may underestimate their true performance, since we base our true hits on crystallographically-identified ligand binding sites. It is possible that some of the Hotspots occupy sites for which a ligand indeed exists but has not yet been identified. Accordingly, the number of true hits may actually be higher than is calculated in the present study.

We used the proteins TEM-1 and NKG2D, both containing cryptic sites, to benchmark our method against three alternative methods, namely CryptoSite,⁴² SiteMap^18,19 and SiteFinder.²¹ Note that previously the SILCS-Hotspots approach was also benchmarked against FTMap and Fpocket. These proteins are in common between our validation set and a recent method employing SiteMap and SiteFinder to identify cryptic sites, which found that both SiteMap and SiteFinder struggled to identify the cryptic sites on these two proteins.¹³⁶ We used the free, online CryptoSite server at https://modbase.compbio.ucsf.edu/cryptosite to obtain the results of the predictions using the apo structures of each protein listed in Table S1. The results took ~ 7 hours, although the site and original publication notes that on average there can be a total time of 1–2 days depending on the server load.⁴²

Machine learning methods

Given the limited size of the dataset, we focused our efforts on Support Vector Machine (SVM) and Random Forest classifier models. Random forest models and SVM with nonlinear kernels resulted in over-training (Table S3). While all models generated reasonable average weighted F₁ statistics on the 5-fold cross-validation (CV), there is a significant degradation in performance between the average CV recall and the recall after fitting on the whole training dataset (single-fit) (Table S3). In comparison, the linear kernel SVM had similar recall between a single-fit and the average CV recall (Table S3), so we selected the linear kernel SVM model and fully trained its hyperparameters (Table 2). To optimize the performance of the SVM, we performed standardization ($(\stackrel{⃑}{X}-\mu )/\sigma$) of each feature, then performed principal component analysis (PCA) on these features and used the principal components as inputs for all subsequent models. This ensures the inputs are all mutually orthogonal. The hyperparameters were optimized using a grid search of the parameter space described in Table 2. Each round of grid search was performed using 5-fold cross-validation, and the selection of optimal parameters was made based on the weighted F₁ statistic. Subsequently we performed recursive feature elimination¹³⁸ to identify the optimal number of input principal components and reduce the risk of overfitting by reducing the dimensionality of the inputs (Figure S3A). The first 22 principal components were selected, corresponding to the maximum weighted F₁ in Figure S3A. The distribution of the data in the first two principal components is given in Figure S3B, indicating that the two classes are somewhat linearly separable. The final model hyperparameters are indicated in Table 2 with bold text. These were used to train the final model on the whole training dataset; all subsequent results in the paper are based on this model. A key output of an SVM model is the Decision Function, defined as the distance a Hotspot lies from the SVM’s decision boundary and can be interpreted as the confidence that a given Hotspot corresponds to a true hit and, therefore, likely located within 5 Å of a crystallographic ligand binding site.^139,140 The Decision Function is positive for higher confidence, and negative for confidence that the Hotspot is not a suitable binding site. The ML scripts were written using the scikit-learn version 1.3.0¹³⁷ and pandas 2.0.3¹⁴¹ python libraries. All 3D molecular renderings were generated using VMD version 1.9.3,¹⁴² and all plots were created with the python library matplotlib¹⁴³ using the accessible color sequences of Petroff.¹⁴⁴

Table 2: Linear SVM hyperparameters.

Descriptions of hyperparameters are adapted from the sci-kit learn library documentation.¹³⁷ Where multiple hyperparameter values were tested, the bolded parameter value was selected in the final model.

Hyperparameter	Values	Description
C	1e-4, 1e-3, 1e-2, 1e-1	Regularization strength, which is proportional to 1/C. Regularization provides a way to reduce the final model complexity.
intercept_scaling	1e1, 1e2, 1e3	Reduce impact of C on intercept fitting.
loss	hinge, squared_hinge	The loss function used in training the classification model. Hinge loss is the standard for SVM.
penalty	l2	Regularization penalty, the l2-norm.
fit_intercept	True	The input feature vector includes a scalar intercept term.
dual	auto	Automatically select optimization algorithm where the optimal choice depends on the relative numbers of features versus samples, and some choices of other parameters. Auto will be the default in scikit-learn version 1.5.
max_iter	1e8	Maximum number of iterations of the linear solver.
tol	1e-4	Tolerance criterion for convergence of the linear solver.
class_weight	balanced	A weight for the regularization parameter C, in this case inversely proportional to the class proportion.

Results

The present study involved the development of a ML model to predict the probabilities that SILCS Hotspots are located in druggable binding sites, based on those sites which are occupied by drug-like molecules (MW > 200 Da) as identified in crystallographic studies. The model builds on the previously reported SILCS Hotspots based on fragment docking into the SILCS FragMaps combined with additional features for each Hotspot used in ML model development targeting the known druggable sites. The training set included seven proteins while the validation set included eleven proteins. As presented, the developed ML model predicts those Hotspots with a high probability of defining druggable sites based on a quantitative ranking score that may be applied to new systems.

Of the eleven proteins in the validation set, seven were used in previous SILCS-MC benchmarking studies, and as such each contain a single orthosteric binding site.^80,81 In addition, allosteric ligands were identified for the validation set proteins where available. The full details of the structures and ligands used in both the training and validation sets is described in Table S1, but some additional details are given here. For P38 we selected the allosteric inhibitor ligand BIRB 796 bound in PDB 1KV2.¹¹⁸ Note that for the purposes of this study BIRB 796 may be only partially allosteric, as it also overlaps with orthosteric site defined by the ligand in PDB 3FLS.¹¹⁷ We collected five additional systems, ANGPTL4, TEM-1, NKG2D, and GABA_BR in both the active and inactive state. For ANGPTL4, we selected a structure with glycerol bound for the SILCS simulations (PDB: 6U0A) and used a Palmitic acid-bound structure for assessing which Hotspots are in a ligand binding pocket (PDB: 6U1U).¹²⁶ TEM-1 was selected because of its cryptic allosteric binding site,^38,128 which is absent in the apo structure we used for the SILCS simulation (PDB: 1JWP).¹²⁷ Similarly, NKG2D was selected for a cryptic allosteric site.^130,131 For the GABA_BR, as previously described for the CDK2 system,⁸² we collected two sets of FragMaps corresponding to the active (PDB: 7CA3, allosteric modulator BHFF) and inactive (PDB: 7CA5, apo) conformations. Each FragMap set was used to identify ligands from separate PDBs (6UO8 and 7C7Q). This allows us to assess if the individual FragMap sets allows the prediction of binding sites from either state of the protein. However, the large interdomain rearrangement of the transmembrane (TM) helices between active and inactive states¹³² disallows predicting the allosteric binding site present in the active conformation using the inactive conformation with the an equilibrium MD method such as SILCS.

New Hotspot properties improve the identification of druggable Hotspot clusters

To generate features of model development we calculated numerous properties of individual Hotspots including features based on the Hotspot clusters of which they are the centroid Hotspot. The previously published Hotspot ranking (Orig in Table 1) was based purely on the mean LGFE over all the specific fragments present in each Hotspot.⁸² As discussed above a single Hotspot represents a binding site for fragments (MW < 200 Da) which are generally smaller than most drugs. The ranking of all the Hotspots using the mean LGFE, as well as being within 12 Å of at least one other Hotspot, is shown in Figure S4, which highlights that for many proteins in the training set, the mean LGFE has limited predictive power. To evaluate the ability of the LGFE to predict the binding sites for drug-like molecules, the binding site recall was calculated with respect to the crystallographic ligand poses. The mean LGFE ranking captures 40%, 44%, and 80% experimental binding sites in the top 10, 20, and 40 Hotspots, respectively, over the training set protein systems (Table 3). While the mean LGFE score used to rank the original Hotspots is somewhat successful as a predictor of the Hotspot being a drug-like molecule binding site in some systems, significant improvements can be made by incorporating additional features in ML model development, as shown below.

Table 3: Training set binding site recall in the top 10, 20, and 40 Hotspots.

The recalls are reported for three models: Hotspot LGFE, Exclusion-map HS SASA, and the SVM model. Binding site recall is the ratio of unique ligands within 5 Å of an experimentally-validated ligand binding site over the total number of such sites for that protein.

Protein Name	# Sites	Top 10	Top 20	Top 40
LGFE (Original ranking metric)
CDK2 Active	6	0.67	0.67	0.67
CDK2 Inactive	6	0.33	0.33	0.83
ERK5	2	0.50	0.50	1.00
PTP1B	3	0.33	0.33	1.00
β2 Adrenergic	2	0.00	0.50	0.50
GPR40	2	0.00	0.00	0.00
M2 Muscarinic	2	0.50	0.50	1.00
Androgen	2	0.50	0.50	1.00
Total	25	0.40	0.44	0.80
Exclusion-map HS SASA
CDK2 Active	6	0.50	0.83	0.83
CDK2 Inactive	6	1.00	1.00	1.00
ERK5	2	1.00	1.00	1.00
PTP1B	3	0.33	0.33	1.00
β2 Adrenergic	2	0.50	1.00	1.00
GPR40	2	1.00	1.00	1.00
M2 Muscarinic	2	0.50	1.00	1.00
Androgen	2	1.00	1.00	1.00
Total	25	0.76	0.88	0.96
SVM model
CDK2 Active	6	0.50	0.50	0.83
CDK2 Inactive	6	1.00	1.00	1.00
ERK5	2	1.00	1.00	1.00
PTP1B	3	0.33	0.33	1.00
β2 Adrenergic	2	1.00	1.00	1.00
GPR40	2	0.50	1.00	1.00
M2 Muscarinic	2	1.00	1.00	1.00
Androgen	2	1.00	1.00	1.00
Total	25	0.76	0.80	0.96

When designing new features, we considered another limitation in the original ranking where the mean LGFE scores of Hotspots with high solvent exposure are often quite favorable. To account for the degree of solvent accessibility required to make a binding site more favorable for drug-like molecules as well as consider the size of drug-like molecules, we designed features related to the degree of solvent accessibility of the Hotspot, the volume of the Hotspot not occluded by the protein, the number of Hotspots in a cluster, and the totals of these in each Hotspot cluster. Figure 2 shows the ranking based on Exclusion-map HS SASA for all Hotspots also within 12 Å of at least one other Hotspot. Those Hotspots within 5 Å of a drug-like molecule from crystallographic structures are shown as large circles. The Exclusion-map HS SASA ranking greatly improves the selection of Hotspots close to drug-like molecules. Table 3 shows that the mean binding site recalls have increased over that of the original LGFE Hotspot ranking to 76%, 88%, and 96% for the top 10, 20, and 40 Hotspots, respectively. While accounting for the SASA and presence of at least one adjacent Hotspot greatly improves the identification of druggable Hotspots, there is variability over the training set proteins. For example, with PTP1B or the M2 Muscarinic receptor, these two criteria alone aren’t particularly effective. Accordingly, we reasoned that using a ML classifier method to combine the information from many features should provide a better ranking. If the model is trained with cross-validation, it could also lead to robust generalization across a range of protein systems.

Figure 2: Ranking based on Exclusion-map HS SASA of individual Hotspots with a minimum of one adjacent Hotspot within 12 Å.

The larger circles denote Hotspots within 5 Å of a non-hydrogen atom of a drug-like compound bound to the proteins.

Machine learning model improves identification of druggable Hotspots

While the individual feature of Exclusion-map HS SASA, and presence of adjacent Hotspots, contain substantial information about whether a Hotspot is located in a drug binding site, an appropriately selected and trained ML model should better integrate the information from a wider range of features and improve the model’s accuracy as well as generalizability. Accordingly, we trained several ML models using the features listed in Table 1, as shown in the supporting information (Table S3). From that analysis we selected the SVM classifier with a linear kernel as implemented in scikit-learn library.^137,139 The final model improves the predictive power over the untrained features alone, as shown in Figure 3. Figure 3A shows the model’s Hotspot ranking for each system and highlights the Hotspots which are within 5 Å of a ligand. Figure 3B presents a precision-recall curve for the training data and includes comparison to two untrained models, the original mean LGFE of all the molecules in the Hotspot, and Hotspot Exclusion-map HS SASA. Precision-recall curves show the change in precision over increasing recall, which corresponds to lowering the level of the cutoff above which a Hotspot is predicted to be a hit. Figure 3C shows the merged ranking of Hotspots from all proteins, for each of the three models, corresponding to Figure 3B. To facilitate easy comparison, the LGFE and Exclusion-map HS SASA were inverted, and then the LGFE, Exclusion-map HS SASA and SVM Decision Function were Min-Max normalized ($(\stackrel{⃑}{X}-min)/(max-min)$) so that they all predict maximal druggability at 1 and minimal druggability at 0 (Figure 3C). Figure 3C shows that generally, the SVM model has the greatest density of true hits in the lower rankings; we note that the relative ranking within each metric is important in Figure 3C, not the position of the curves with respect to one another (Figure 3C). Indeed, the SVM model has superior performance to the other models, demonstrated by the larger area under the precision-recall curve (AUC) for the SVM model (0.42) as compared to the LGFE (0.08), Exclusion-map HS SASA (0.29), and the random model (0.07) (Figure 3B). The SVM model’s AUC increased six-fold from that of the random model (0.07 to 0.42) (Figure 3B).

Figure 3: Performance of final model on the training set.

A) Ranking of each protein’s Hotspots by the final SVM model’s Decision Function with Hotspots within 5 Å of the non-hydrogen atoms of known drug-like molecules (true hits) shown as large circles. B) Precision-Recall curves of the original LGFE (blue), Exclusion-map HS SASA (yellow), and SVM Decision function (red) models. AUC stands for area under the curve, and the black dashed line reflects the ratio of hits to total Hotspots, or the expected AUC for a random model. C) Ranking of all training set Hotspots using the Min-Max normalized ranking metric in which the range for each metric is set from 0 to 1 using $(\stackrel{⃑}{X}-Min)/(Min-Max)$. Hotspots within 12 Å of at least one other Hotspot from all proteins are combined and plotted as a continuous curve. Prior to Min-Max normalization the Exclusion-map HS SASA and LGFE were inverted to allow direct comparison to the SVM Decision Function. The large markers denote hits, as in panel A).

In practical terms, the model identifies 80% of ligand binding sites in the top 20 Hotspots (Table 3). This is impressive performance given the challenging nature of the problem since the binding sites identified here include both allosteric and orthosteric sites based on ligands exclusively absent in the crystal structures used in the SILCS simulations.⁸² In the top 20 Hotspots the SVM model fails to identify three out of twenty-five ligand sites (Table 3). One is a relatively solvent-exposed site on the protein PTP1B, and so are unusual in our training set and challenging to the model. The remaining three missing ligands belong the CDK2 kinase in the active state. Two of these missing sites share the same Hotspot ranked 34^th by the SVM model (Table S2). The last missing site has no Hotspot within 5 Å (Table S2), as highlighted in the previous paper.⁸² Missing this binding site is therefore not a limitation of the ranking method itself but the sampling of that particular pocket using the CDK2 Active structure 3MY5 with the SILCS method. While the system PTP1B, which has largely surface-exposed binding sites, remains challenging even for the SVM model, the model prediction generally improves across all systems (Figure 3B), and may be more generalizable than a single feature such as the Exclusion-map HS SASA, which happens to perform well on this particular dataset. However, an unbiased assessment of the final model must rely on an independent dataset.

Validation of the final SVM model

To validate the final model, we gathered a set of proteins independent of the training set, as discussed in the Methods. The details of the ligands analyzed for each system are listed in Table S1 and Table S2. The results for predicting all Hotspots near crystal ligands using the SVM model are given in Figure 4A, and a comparison of the model’s performance to the untrained LGFE and Exclusion-map HS SASA models are given in Figure 4B and Figure 4C. The results for predicting individual binding sites is given in Table 4. There is a six-fold increase in precision-recall AUC between the random model and the SVM model in the validation set (0.02 to 0.12), the same as was in the training set (0.07 to 0.42), which suggests that the model was not overfit to the training data. More practically, the model recalls 67% of ligand binding sites in the top 10, and 89% of sites in the top 20 Hotspots, respectively (Table 4). The SVM model’s Decision Function outperforms the untrained models as demonstrated by the increased precision-recall AUC (Figure 4B). Notably, the Exclusion-map HS SASA ranking performs worse in the validation set than in the test set, suggesting that the trained SVM model is more generalizable than either individual feature alone (Figure 4B). Furthermore, although the Exclusion-map HS SASA ranking performed slightly better at binding site recall on the training set (Table 3, top 20), the SVM model performs better than either untrained model on the validation test (Table 4). Overall, the results argue that the model is not over-fitted to our limited training data, and that the model can predict druggable binding sites across a range of proteins with reasonable accuracy.

Figure 4: Performance of final model on the validation set.

A) Ranking of each protein’s Hotspots by the final SVM model’s Decision Function with Hotspots within 5 Å of the non-hydrogen atoms of known drug-like molecules (true hits) shown as large circles. B) Precision-Recall curves of the original LGFE (blue), Exclusion-map HS SASA (yellow), and SVM Decision Function (red) models. AUC stands for area under the curve, and the black dashed line reflects the ratio of hits to total Hotspots, or the expected AUC for a random model. C) Ranking of all training set Hotspots using the Min-Max normalized ranking metric in which the range for each metric is set from 0 to 1 using $(\stackrel{⃑}{X}-Min)/(Min-Max)$. Hotspots within 12 Å of at least one other Hotspot from all proteins are combined and plotted as a continuous curve. Prior to Min-Max normalization the Exclusion-map HS SASA and LGFE were inverted to allow direct comparison to the SVM Decision Function. The large markers denote hits, as in panel A).

Table 4: Validation set binding site recall in the top 10, 20, and 40 Hotspots.

The recalls are reported for three models, the LGFE, Exclusion-map HS SASA of the Hotspot, and SVM model’s Decision Function. Binding site recall is the ratio of the total number of ligand binding sites within 5 Å of a Hotspot in the top N Hotspots. A site is identified when at least one Hotspot corresponding to a ligand is selected in the top N.

Proteins Name	# Sites	Top 10	Top 20	Top 40
LGFE
P38	2	0.50	1.00	1.00
BACE1	1	1.00	1.00	1.00
Hsp90	1	1.00	1.00	1.00
TrmD	1	1.00	1.00	1.00
Thrombin	1	1.00	1.00	1.00
MCL1	1	1.00	1.00	1.00
FXR	3	0.67	0.67	1.00
ANGPTL4	1	1.00	1.00	1.00
TEM1	3	0.33	0.33	0.33
GABABR Active	2	0.00	0.50	1.00
GABABR Inactive	1	0.00	0.00	1.00
NKG2D	1	1.00	1.00	1.00
Total	18	0.61	0.72	0.83
Exclusion-map HS SASA
P38	2	1.00	1.00	1.00
BACE1	1	0.00	1.00	1.00
Hsp90	1	1.00	1.00	1.00
TrmD	1	1.00	1.00	1.00
Thrombin	1	0.00	1.00	1.00
MCL1	1	1.00	1.00	1.00
FXR	3	0.67	1.00	1.00
ANGPTL4	1	1.00	1.00	1.00
TEM1	3	0.33	0.33	0.67
GABABR Active	2	0.00	0.00	0.00
GABABR Inactive	1	0.00	0.00	0.00
NKG2D	1	1.00	1.00	1.00
Total	18	0.56	0.72	0.78
SVM model
P38	2	1.00	1.00	1.00
BACE1	1	1.00	1.00	1.00
Hsp90	1	1.00	1.00	1.00
TrmD	1	1.00	1.00	1.00
Thrombin	1	0.00	1.00	1.00
MCL1	1	1.00	1.00	1.00
FXR	3	1.00	1.00	1.00
ANGPTL4	1	1.00	1.00	1.00
TEM1	3	0.33	1.00	1.00
GABABR Active	2	0.00	0.50	0.50
GABABR Inactive	1	0.00	0.00	0.00
NKG2D	1	1.00	1.00	1.00
Total	18	0.67	0.89	0.89

While the model performs quite well across most of the validation set, it performs poorly on the heterodimer GABA_B Receptor in both active and inactive states. It captures one of nine true hit Hotspots in the active state and zero of three in the inactive, which corresponds to identifying only one of three ligand binding sites (Table 4). The orthosteric binding site (2C0, Baclofen) was not identified in GABA_BR Inactive, despite being identified in the GABA_BR Active simulations. In the simulations of the inactive state, the orthosteric binding site is highly solvent exposed, and the Hotspots’ Exclusion-map rBSA values range from 1% to 40%, less than the empirical 42% cutoff used to define the training set (see Methods). This makes this site an outlier compared to the data used to train the model. However, another challenge is that the GABA_BR heterodimer is much larger than the other proteins considered. A total of 416 Hotspots were identified or about four- to five-times the number in the training set systems. To account for this, we ranked the Hotspots near the extracellular part of the GABA_B1 subunit. From among these 118 Hotspots, a Hotspot near the ligand 2C0 is now ranked in 33^rd, or in the top 40 (Table S2). Finally, the missing site in the GABA_BR active state is an allosteric binding site between the two TM domains and directly interacts with lipids in the bilayer during the SILCS GCMC/MD simulations (Figure S5), making this site uniquely challenging to identify with our method. We ranked all the Hotspots in the TM region and found that the first two Hotspots near the ligand are only ranked 50^th and 57^th, respectively (Table S2). A future improvement of the model could explicitly account for lipid interactions at membrane-protein interfaces, since this burial is not explicitly accounted for in the highly-predictive Exclusion map surface area calculations.

Model’s Decision Function is a predictor of Hotspot druggability

While the SVM model highly ranks most Hotspots corresponding to known drug-like ligand binding sites in the top 20 (Table 4), there are a number of high-ranking Hotspots that do not correspond to known binding sites. Because some may be associated with true drug-like binding sites for which no ligand has yet experimentally been identified, we hypothesized that the most highly-ranked Hotspots should be more druggable than those ranked poorly. To test this hypothesis, we selected two proteins in the validation set, namely TEM-1 and GABA_BR Active, and docked the FDA database of 348 compounds at the Hotspots ranked 1–10, 91–100, and for GABA_BR 391–400. These Hotspots represent the most and least-druggable according to the SVM model’s ranking. For each Hotspot we report the mean LGFE and rBSA for the top twenty compounds ranked by LGFE (Table S4). The mean LGFE scaled by mean rBSA (mean LGFE × mean rBSA), where 100% rBSA is equivalent to 1.0, was used as a measure of Hotspot druggability. This assumes that druggable sites have favorable LGFE scores with high rBSA values, associated with high affinity and with buried sites, respectively. We plotted the final SVM model’s Decision Function against the mean LGFE × rBSA for these Hotspots in Figure 5. In general, it shows the expected anti-correlation between Hotspot predicted druggability, based on larger positive SVM Decision Function values and more negative LGFE × rBSA scores corresponding to druggable sites.

Figure 5: SVM model Decision Function and the Mean LGFE times rBSA for selected Hotspots.

For TEM-1 and GABA_BR, the Hotspots 1–10 and 91–100 were selected, and for GABA_BR Hotspots 391–400 were also selected. The trendlines show the linear line of best fit. For TEM-1 Hotspots 1–10 and 91–100 correspond to SVM Decision Function scores of ~1.0 and −1.5, respectively, while Hotspots 1–10, 91–100, and 391–400 correspond to SVM Decision Function scores of ~1.0, 0.2, and −1.5. The discrepancy in the relationship is due to the significantly higher number of Hotspots with GABA_BR versus TEM-1, which biases the overall distribution towards lower ranking SVM Decision Function scores.

The SVM Decision Function’s anti-correlation with the LGFE × rBSA druggability scores accounts for slightly different trends in LGFE and rBSA individually between GABABR and TEM-1. For the TEM-1 Hotspots, the top 10 Hotspots have substantially higher average rBSA and the average LGFE values of Hotspots 91–100 decrease only slightly, whereas in GABA_BR Active the average LGFE score decreases substantially while the average rBSA values decrease slightly (Table S4). The fact that GABA_BR Hotspots appear far more druggable, having more favorable average LGFE and lower rBSA, despite only considering Hotspots 91–100 is due to that system have significantly more Hotspots due to its larger size than the TEM-1 system. Importantly there are large differences between the SVM Decision Function scores between Hotspots 1–10 and 91–100 for both proteins, indicating the ability to discriminate between sites in difference proteins. In addition, it is notable that with both proteins the SVM Decision Function scores for the top Hotspots are similar, ~1.0, indicating that the SVM values may be applied directly to new proteins for the selection of potential druggable sites. Finally, the lack of a stronger anti-correlation between SVM Decision Function scores and the Mean LGFE × rBSA druggability scores may be associated with the concept of druggability being fairly imprecise. For example, some binding sites may have high affinity for just a few ligands, and low affinity for all other ligands, yielding lower druggability score despite the fact that the site is druggable in principle.

Comparison to existing methods of cryptic binding site prediction

In our previous work introducing the SILCS-Hotspots method, we compared the Hotspots generated against the fragment binding sites identified by FTMap¹⁶ and Fpocket,¹⁷ and found that SILCS-Hotspots identifies more Hotspots near the crystallographic sites than the other methods.⁸² To give a sense of the performance of the model against other available cryptic binding site identification methods, we selected two proteins in our validation set, TEM-1 and NKG2D, to compare with CryptoSite.⁴² These cryptic sites were selected because they were recently identified¹³⁶ as being particularly challenging to SiteMap (Schrödinger, Inc.)^18,19 and SiteFinder (Chemical Computing Group).²¹ CryptoSite successfully identified the cryptic site in NKG2D (Figure S6). As noted in the original CryptoSite paper, it identifies the residues involved in the disruption of a core region upon ligand binding to the cryptic site of TEM-1, although the scores of ~0.06–0.08 are below the typical CrytoSite cutoff score of 0.1 (Figure S6).⁴² These results suggest that both CryptoSite and SILCS-Hotspots perform better than either SiteMap or SiteFinder at identifying cryptic sites. It should be noted that CryptoSite requires more computation than SiteMap/SiteFinder, and similarly SILCS-Hotspots requires more than CryptoSite associated with the computational requirements of the initial SILCS Simulations. The SILCS-Hotspots method is not intended to be used as a standalone tool, but as part of the integrated SILCS workflow with methods for site identification, pharmacophore discovery and lead optimization.

Conclusions

We previously presented the SILCS-Hotspots method to leverage the information in SILCS FragMaps to identify a comprehensive set of fragment binding sites. Here we have built upon the previous work and developed a predictive algorithm which identifies the binding sites of larger, drug-like molecules. As a training set, we used the original set of proteins which included a list of Hotspots within 5 Å of a drug-like ligand in a crystal structure of the protein. We first demonstrated that the existing SILCS-Hotspot ranking, based solely on the mean LGFE of each Hotspot that is within 12 Å of at least one other Hotspot, was insufficient to efficiently identify druggable binding sites. Next, use of the Exclusion-map HS SASA of each Hotspot and presence of at least one adjacent Hotspots was shown to substantially improve the ranking. Building on this, a SVM classification model was developed using a wide array of Hotspot and Hotspot cluster properties as features. This led to improved predictions and the final model was validated on a separate set of 9 proteins, on which the model performs quite well. On the problem of identifying at least one Hotspot per ligand binding site, the final model achieves 80% recall in the top 20 Hotspots per protein (20 out of 25 total ligand binding sites total) in the training set, and 89% recall in the top 20 on the validation set (16 out of 18 total sites). By comparing the model’s ranking with the predicted affinity and solvent accessibility of members of a chemically-diverse set of FDA-approved compounds, we argue that the model predicts sites which are likely druggable even if they haven’t yet been identified through the presence of crystallographic ligands.

In practice, the presented workflow and SVM model offers the capability of identifying novel binding sites for drug-like molecules in proteins, including allosteric sites. This takes advantage of the high information content in the SILCS FragMaps that include contributions from protein flexibility, desolvation and protein-functional group interactions which, in a ligand discovery scenario can be used for database screening and ligand optimization. Notable is the high performance of the SVM model on the validation-set proteins. This is suggested to be due to the use of the physics-based SILCS FragMaps in the initial Hotspots calculation avoiding inherent overtraining effects that may occur with a ML model solely based on data fitting. However, the model may have limitations associated with sites adjacent to the lipid bilayer, such as the site observed in GABA_BR Active state. Future efforts will focus on addressing this issue, such as by directly accounting for burial in lipids and by constructing a training set of sites at protein-bilayer interfaces. Furthermore, while the model has been tested on a reasonably diverse test set of proteins including challenging cryptic sites, more extensive testing is necessary to conclude the model will generalize to exotic systems. We expect that this relatively simple classification model with the physical insights from SILCS sampling will tend to generalize well.

Supplementary Material

SI

Figure S1: Surface-exposed Hotspot 25 in ERK5.

Figure S2: Distribution of Hotspot SASA by protein system.

Figure S3. Analysis of the recursive feature elimination and the top two principal components (PCs) of the training set.

Figure S4: Ranking based on mean LGFE of each Hotspot.

Figure S5: Burial of allosteric binding site between GABA_BR Active TM domains.

Figure S6: CryptoSite predictions for NKG2D (A) and TEM-1 (B).

Table S1: List of proteins and ligands used for methods validation.

Table S2: Training and validation set Hotspots and ligand distances.

Table S3: Stratified 5-fold Cross-validation training of higher-order SVM Classifier with polynomial or radial basis functions kernels and a Random Forest model.

Table S4. FDA compound screening for selected Hotspots of TEM-1 and GABA_BR Active.

Data and Software Availability

Information about the training and validation set, including the crystallographic ligands and the adjacent Hotspots, is provided in Table S1 and Table S2. The compounds used to perform the FDA analysis in sdf and pdf file formats, as well as all the data in training and test data sets in csv format, are provided free on GitHub at https://github.com/mackerell-lab/FDA-compounds-SILCS-Hotspots-SI.