Shape Signatures: New Descriptors for Predicting Cardiotoxicity In Silico

Abstract

Shape Signatures is a new computational tool that is being evaluated for applications in computational toxicology and drug discovery. The method employs a customized ray-tracing algorithm to explore the volume enclosed by the surface of a molecule and then uses the output to construct compact histograms (i.e., signatures) that encode for molecular shape and polarity. In the present study, we extend the application of the Shape Signatures methodology to the domain of computational models for cardiotoxicity. The Shape Signatures method is used to generate molecular descriptors that are then utilized with widely used classification techniques such as k nearest neighbors (k-NN), support vector machines (SVM), and Kohonen self-organizing maps (SOM). The performances of these approaches were assessed by applying them to a data set of compounds with varying affinity toward the 5-HT_2B receptor as well as a set of human ether-a-go-go-related gene (hERG) potassium channel inhibitors. Our classification models for 5-HT_2B represented the first attempt at global computational models for this receptor and exhibited average accuracies in the range of 73−83%. This level of performance is comparable to using commercially available molecular descriptors. The overall accuracy of the hERG Shape Signatures–SVM models was 69−73%, in line with other computational models published to date. Our data indicate that Shape Signatures descriptors can be used with SVM and Kohonen SOM and perform better in classification problems related to the analysis of highly clustered and heterogeneous property spaces. Such models may have utility for predicting the potential for cardiotoxicity in drug discovery mediated by the 5-HT_2B receptor and hERG.

Introduction

The heart is a highly complex structure that ensures the survival of the organism. Consequently, xenobiotic-mediated interference with its role in homeostasis can have catastrophic effects manifesting in cardiotoxicity. For example, interference with ion homeostasis by channel (1) or exchanger blockade (2), altered coronary blood flow, oxidative stress, organellar dys-function, and apoptosis are all potential mechanisms of cardiotoxicity (3). Two different proteins, namely, the 5-HT_2B receptor and the human ether-a-go-go-related gene (hERG)¹ potassium channel, have raised particular concern primarily due to their association with cardiac valve disease or potassium channel blockade, respectively. Unintended activity at these two proteins independently by several drugs resulting in toxicity has prompted their withdrawal from the market by the FDA. An area of considerable interest in drug discovery research is the computational modeling of toxicity-related proteins to identify such potential problems as early as possible (4), especially as currently there are no X-ray structures for these proteins.

Serotonin is found in many physiological systems from the central nervous system to the intestinal wall and, in concert with its many receptors, plays a major regulatory function in cardiovascular morphogenesis. The 5-HT₂ receptor family of G-protein-coupled receptors including 5-HT_2B is expressed in cardiovascular, gut, and brain tissues as well as human carcinoid tumors (5). In recent years, this receptor has been implicated in valvular heart disease defects, caused by the now FDA-banned “fen-phen” and pergolide (6-9). The primary fenfluramine metabolite, norfenfluramine, potently stimulates 5-HT_2B (10, 11). Computational modeling of this receptor has been very limited to date but is urgently needed to proactively identify drugs that may bind this receptor.

Numerous classes of drugs have been shown to prolong the QT interval, which reflects a slowing of repolarization of the ventricular myocardium (12, 13), where excessive prolongation can lead to the potentially life-threatening ventricular tachyarrhythmia, torsade de pointes. In cardiac tissue, inhibition of potassium channels is associated with QT interval prolongation (14, 15). The most common potassium channel linked to drug-induced QT interval prolongation is also responsible for the rapid component of the delayed rectifier potassium current (I_Kr). The focus of considerable research is hERG, which is believed to encode the protein that underlies the delayed rectifier potassium current I_Kr (16, 17), and many drugs associated with QT interval prolongation have been found to block hERG (18-20). Several drugs have been withdrawn from the market in the past decade due to cardiovascular toxicity associated with undesirable blockade of this channel. Since 2002, there have been numerous studies that have described individual three-dimensional (3D) quantitative structure–activity relationship (QSAR) models, statistical models, or pharmacophores for hERG (21-34). These different studies and others have encompassed a wide set of data generation and modeling techniques as well as an array of molecules for model building and testing as recently reviewed (35). There are some gross similarities in the suggested requirements for hERG inhibitors, namely, the requirement for hydrophobic features surrounding a positive ionizable/basic nitrogen feature. However, depending on the molecules and techniques used for model building, the pharmacophore or descriptors suggested may vary widely.

The focus to date has been primarily on individual hERG models of a “global” nature consisting of structurally diverse molecules across therapeutic targets (antipsychotics, antihista-mines, antibiotics, etc.) although “local” models have also been generated around narrow structural series (24). These ligand-based computational models, along with a growing number of homology models (36, 37), have provided insights that complement experimental studies such as site-directed mutagenesis (38, 39). We recently illustrated for the first time the comparison of multiple modeling approaches including Kohonen maps, Sammon maps, and recursive partitioning with the same training set, to assess whether one or a combination of approaches is preferable (40). All hERG models were assessed with a sizable external data set of published molecules and exhibited good predictivity. In addition, it was found that a structural similarity measure provides a valuable means to limit extrapolations far beyond the training set of the quantitative recursive partitioning model. The descriptors selected for the qualitative mapping methods provide further insight into the structural features of hERG inhibitors when compared with those generated by other available methods, suggesting that molecular shape or topological characteristics are also important for hERG inhibitors. Drugs for noncardiovascular indications that interact with either 5-HT_2B or hERG are undesirable. Although there has been considerable modeling of hERG within pharmaceutical companies and less so for 5-HT_2B, there is an unmet need for a computational platform that focuses on identifying molecules that bind to either of these proteins with affinity that may be clinically significant.

A new approach has recently been proposed that utilizes molecular shape-dependent signatures as the basis for molecular recognition (41). The Shape Signatures method employs a customized ray-tracing algorithm to explore the volume enclosed by the surface of a molecule, then uses the output to construct compact histograms (“shape signatures”) that encode for molecular shape, polarity, and other biorelevant properties (Figures 1 and 2). The method has already proven successful for a number of drug discovery programs when used for database similarity searching (41-45) and has several advantages over other approaches (Table 1). The goals of the present study were to extend the Shape Signatures tool into the domain of toxicology modeling. More specifically, we demonstrate that Shape Signatures can be employed to generate ensembles of 3D molecular descriptors useful for classifying compounds with respect to their experimentally tested activity at the 5-HT_2B receptor and the hERG channel. These models were also tested against more traditional classification models with two-dimensional (2D) molecular descriptors. Our aim is to develop practical and accessible models that reliably predict whether a molecule is likely to exhibit cardiotoxicity mediated via these two proteins.

Figure 1.

One-dimensional (1D) and 2D shape signatures of pergolide, a 5-HT_2B active (strong binder, K_i = 14 nM). (a) Chemical structure. (b) 1D (shape only) signature histogram. (c) 2D (shape and polarity) signature plot.

Figure 2.

1D and 2D shape signatures of melatonin, a 5-HT_2B nonactive (weak binder, K_i = 12900 nM). (a) Chemical structure. (b) 1D (shape only) signature histogram. (c) 2D (shape and polarity) signature plot.

Table 1.

Comparison of Several Key Performance Measures for Traditional Descriptor-Based QSAR Approaches (1D, 2D, and 3D QSAR) vs Shape Signatures

asset	traditional descriptor-based QSAR approaches	Shape Signatures
speed	✓✓✓	✓✓✓
accuracy	✓✓	✓✓
scalability	✓	✓✓✓
	model requires reformulation as new data added	no reformulation needed as new data added
coverage	✓	✓✓✓
	descriptors must be available for chemical species	always works, i.e., organics, inorganics, organometallics, ions, etc.
sensitivity	✓	✓✓
	global model, lacks sensitivity (can also be used for local models)	local model, enhanced sensitivity
domain applicability	✓	✓✓✓
	model very sensitive to chemical (sub) structure of training set	much less sensitive to chemical (sub) structure of training set
interoperability	✓✓	✓✓✓
	integration with other QSAR models requires reformulation	fully compatible with other methods
ease of use	✓✓	✓✓✓
	preprocessing of queries requires time and know-how	no preprocessing, extremely simple to use

Experimental Procedures

Data Acquisition

A database of >130 unique molecules was assembled for which patch clamp data for the hERG channel were available (40). Following the analysis of Ekins and co-workers, we selected 39 strong binders (IC₅₀ < 1μM) and 44 weak binders (IC₅₀ > 10μM) (40). In the case of 5-HT_2B, the recent annotation of a database of binding information (K_i) for receptors (http://kidb.cwru.edu/) provided 182 molecules with documented binding properties (46). Among them, 116 compounds with K_i ≤ 100 nM were designated “active”, while 66 other compounds with low affinity (K_i ≥ 1μM) were “nonactive”. The full lists of compounds for each target class are available in the Supporting Information. The associated libraries of Shape Signatures required for classification were generated using the procedure outlined below.

Shape Signatures Method

In the Shape Signatures method, 3D molecular features, such as overall molecular shape and distribution of charges, are encoded in the form of the 1D and 2D dimensional histograms. The structure-related properties are regarded as key indicators of ligand–receptor molecular recognition and, thus, to the relative biological activity of the compound. These histogram-based fingerprints (“signatures”) have been used to compare the query molecule with other druglike compounds from precomputed databases. Shape similarity between the two molecules is then assessed by comparing their 1D signatures (Figures 1b and 2b). Matching the 2D signatures of the two compounds compares their overall molecular shapes and molecular electrostatic potentials (MEP) (Figures 1c and 2c). The closest matches are retrieved for further analysis. The process is fast and efficient, and the method benefits from its ability to capture the true 3D structure of the molecules without atom-based alignment of the molecules (43, 44).

A detailed description of the Shape Signatures technique can be found in the original publication (41), and we only briefly highlight the major steps of the algorithm. These descriptors are similar to the PEST shape/property descriptors described previously by Breneman et al. (47, 48). The procedure starts by generating a 3D structure of the molecule under investigation using CORINA (developed by J. Gasteiger et al., Molecular Networks GmbH, Nägelsbachstrasse 25, 91052 Erlangen, Germany; http://www.molnet.de), followed by computing partial charges for each atom using the Gasteiger–Marsili scheme (49). A solvent-accessible surface (SAS) is then constructed around the molecule, and the triangulated representation of this surface is subsequently generated by the SMART algorithm (50). Next, the ray-tracing process is initiated inside the cavity bound by the SAS, which encompasses the molecule. The ray is propagated from a randomly chosen point on the interior lining of the molecular compartment. As it strikes the opposite side, it is reflected and propagates in the direction determined by the law of optical reflection. As the ray bounces back and forth inside the enclosed molecular compartment, it generates a path composed of a number of straight line segments joined by the reflection points. For each reflection point, two quantities are calculated and stored in memory: the value of the truncated Coulomb potential at this geometric point created by the nearest atomic charges and the combined length of the incident and reflected ray segments. Given a sufficient number of reflections (100000 in this study), the trajectory of the ray will eventually explore the entire volume of the molecule. To prevent trapping of light inside some tight and unusually shaped parts of the molecular compartment, the ray-tracing procedure is periodically stopped and reinitiated from a different randomly selected point on the inner surface. At the end of the run, all recorded ray segments are binned by their length in a 1D histogram with the predefined bin width of 0.5 Å (Figures 1b and 2b). Simultaneously, a second histogram is also constructed; this one bins records by values of the MEP with a step of 0.05 e/Å and the associated total length of the two path segments joined by the reflection point, resulting in a 2D histogram (41) (Figures 1c and 2c). Both histograms are properly normalized. The first histogram encodes exclusively for the shape characteristics of the molecule (it represents the probability distribution of finding a ray segment of a particular length inside the SAS surrounding the molecule), whereas the second histogram reflects both the molecular shape and the 3D arrangement of atomic charges in this molecule (it expresses a joint probability distribution of finding a particular value of MEP with a certain length of the two ray segments connected by the reflection point). Once calculated, the resultant Shape Signatures fingerprints can be employed in a variety of problems in drug discovery and computational toxicology, which require matching chemical structures based on their shapes and polarities.

Shape Signatures Molecular Descriptors

For every molecule in this study, the heights of the corresponding normalized 1D and 2D shape signature bins comprise the sets of distinct molecular descriptors related to this particular structure. Consequently, each chemical has two sets of continuous descriptors: one based exclusively on molecular shape and the other reflecting both molecular shape and polarity. It is important to emphasize that although these features are pieces of 1D and 2D shape signatures, they are inherently 3D molecular descriptors since they encode for the 3D arrangements of atoms and atomic charges in a molecule. We also note that for all classification runs based solely on Shape Signatures, no additional descriptors were appended to the input Shape Signatures data vectors. As will be discussed later in the text, a mixed descriptor scheme with some combination of the traditional commercially available 2D descriptors [e.g., those in Molecular Operating Environment (MOE), Montreal, Canada: Chemical Computing Group Inc.] and the Shape Signatures-derived 3D descriptors seems an interesting continuation of the reported analysis in the future.

As a preliminary test, we used the suggested descriptor allocation scheme to cluster 22 bioactive compounds from our in-house libraries. The set included 10 nuclear receptor (five estrogen receptor and five androgen receptor) binders, five Pfmrk kinase inhibitors, and seven tubulin ligands. Clustering was based on molecular shape captured by the 1D Shape Signatures histograms, used simple Euclidean distances as similarity scores, and was performed by constructing and analyzing Kohonen self-organizing maps (SOM) (51). The results obtained (data not shown) indicated that structures from different target families occupied distinct regions of the SOM, and with only two clear misclassifications for the 22 molecules. These misclassifications were genistein [a frequent hitter (52-54)] and raloxifene, which was observed to lie closer to the androgen receptor molecules rather than the estrogen receptor compounds (55). Encouraged by these preliminary findings, we decided to build classification models for the 5-HT_2B receptor and hERG potassium channel data sets.

Classification Procedures

We have investigated descriptors derived from the Shape Signatures representations using different classifier algorithms. For a simple classifier, we chose the k-nearest neighbors (k-NN) algorithm (56), which is very easy to implement. Despite its simplicity, this method has been shown to produce acceptable results for many applications (57-59). In this method, each query molecule from a given test set is compared in turn with all compounds in the training data set with known class affiliations, and similarity scores are calculated for every pair. The comparison is made between the corresponding 1D and 2D shape signatures of the two molecules, and we utilize the χ-square measure (56), widely employed for comparing discrete distributions, to compute similarity scores. At the end of each run, the entire training set is rank ordered with those more similar (to the query) structures being placed at the top of the list. The decision on which class a given query structure shall belong to is made based on a majority vote of its k nearest, that is, k most similar, neighbors. For highly unbalanced data sets, the weights of the neighbors are adjusted according to their class prior probabilities. For an unbalanced data set, k-NN with a straightforward majority vote will favor assignment to the larger of the two classes as the size of the NN list grows. To avoid such a situation, we need to adjust the majority vote rule accordingly (or equivalently assign different weights to molecules on the NN list belonging to different classes). It is therefore customary to vary k within some range, depending on the size of the training set, in search of the value with maximum prediction accuracy.

The support vector machine (SVM) method, based on the principle of structural risk minimization (60, 61), is a relatively recent addition to the family of supervised classification methods [discussed in detail in a recent book chapter (62)]. This technique has already gained recognition as one of the most robust and efficient classifiers (21, 56-58, 63). It can tackle nontrivial problems by projecting the original descriptor vectors to a higher dimensional feature space where a clearer division between the two classes of data becomes feasible. In such a high-dimensional feature space, a linear SVM routine is applied next to optimally position the separating hyperplane between the instances from the two classes (62). Minimization of the expected generalization error for the test data sets is achieved by finding a separating hyperplane with the maximal margin. Computationally, the transformation into a higher dimensional feature space is implicit as only the distances between the pairs of the transformed data are needed for training and these are computed using the predefined kernel functions K, the associated parameters for K, and the original input descriptor vectors. As such, this approach is less likely to suffer from “data overfitting” and can successfully handle situations involving hundreds or many thousands of descriptors. We used a well-tested and freely available program LIBSVM (C-SVM) (64). We worked with the radial basis function kernel, whose parameter γ and the penalty term C were determined in each case via a grid search procedure utilizing 5-fold cross-validations.

The data sets ultimately used in this study included 83 chemicals for the hERG potassium channel and 182 chemicals for 5-HT_2B receptor. For each of these data sets, a pair of 1D and 2D shape signatures was constructed according to the procedure detailed above. There are on average about 20−60 nonzero bins/descriptors for the 1D (shape only) Shape Signatures histograms. For the 2D histograms (shape and polarity), this number is significantly higher, on the order of several hundred. Consequently, to avoid overfitting in the latter case, we applied the unsupervised forward selection (UFS) method of Livingstone and co-workers (65) to reduce the dimensionality of the problem. The UFS scheme, which was designed to eliminate redundancy and diminish multicollinearity of the input data, has been demonstrated to be fairly successful for a number of QSAR studies (65). The algorithm consists of two major steps. While processing the original descriptor data matrix (responses are not included), the routine first excludes descriptor columns with small standard deviations (ε < ε_min) as contributing no significant information. It then analyzes the reduced data matrix, selects two least correlated descriptor columns, and rejects those with high pairwise correlation coefficients (R² > R_max²). The list of the selected descriptors is augmented by the column that has the smallest squared multiple correlation coefficient. This step is performed repeatedly, producing a growing list of nominated descriptor columns, which survive the rejection filter based on the squared multiple correlation coefficient with the columns picked in the previous step. The procedure stops when the list of columns is fully exhausted. For our experiments, we used the code available from Whitley et al. (65), with the default parameter settings for ε_min and R_max² as 0.0005 and 0.99, respectively.

Neural Network Modeling Using Self-Organizing Kohonen Maps

As we have described previously (40, 66), the general idea behind Kohonen maps (51) is to map a set of vectorial samples onto a 2D lattice in a manner that preserves the topology of the original space. Kohonen maps belong to a class of neural networks known as competitive learning or self-organizing networks. The Kohonen map consists of artificial neurons that are characterized by weight vectors with the same dimensionality as the descriptor set. The neurons are connected by a distance-dependent function. In an unsupervised training algorithm, the neurons self-organize until their pairwise neighborhoods represent the correct topology of the original data set. Kohonen maps have recently been applied to successfully model cytochrome P450-mediated drug metabolism (67, 68) and hERG inhibition (40). The generation of the Kohonen SOMs (51) was conducted using the Smart Mining software v1.01 (ChemDiv, Inc., San Diego, CA, www.chemosoft.com). A 7 × 7 node architecture was chosen to provide the studied molecules with the optimal distribution space. The 5-HT_2B data set included 140 compounds (77% of the entire database) denoted as a “training” set (89 active and 51 nonactive) and 42 compounds (23% of the entire database) denoted as a “test” set (27 active and 15 nonactive). In total, 182 compounds (100%) were used for generation and validation of the SOMs. Similar to the SVM analysis, Kohonen networks were constructed using 102 2D Shape Signatures descriptors computed for each molecule in the set. The training parameters for the SOM were as follows: The classical algorithm based on the incremental learning method was applied for generation of the Kohonen maps, the neurons were studied using the normal distribution law encoded by the Gaussian probability function, the initial distribution of synaptic weights was randomly assigned, the number of interactions for the training runs was 3000, the starting adjustment radius for the training runs was 4, and the initial learning rate factor was 0.5. We have used 30 randomizations of the input training set for the Kohonen map generation. After the SOMs were generated, we studied the distribution of active and nonactive compounds within the best mapping. The resulting maps are shown in Figure 3.

Figure 3.

Separate distribution of 5-HT_2B active (a) and nonactive (b) compounds within the Kohonen network (the best randomization). Darker intensities relate to more molecules in that area. The data have been smoothed for presentation purposes.

Model Testing

To carefully evaluate the performance of each Shape Signatures/classifier combination applied to the hERG or 5-HT_2B data, three different types of statistical testing were undertaken. For the Shape Signatures paired with the SVM approach, we conducted straightforward 10-fold cross-validations on the entire data sets and subjected the systems to a series of leave-N-out runs. The leave-N-out tests were designed as follows. For either target, N compounds from the original data set were randomly picked to represent the hold-out test set, and the rest of the data constituted the training set for this particular data partition. The selection was carried out to approximately preserve the correct proportion of active and nonactive structures in both sets. In particular, for hERG, N = 20 (24% of the data set) with 10 active and 10 nonactive, and for 5-HT_2B, N = 42 (23% of the data set) including 27 active and 15 nonactive molecules. Each classification algorithm was then trained on the training set and applied to predict class attributes of the compounds in the test set. Next, a set of statistical indicators of prediction accuracy were computed and stored. To obtain better statistical estimates, the described procedure was repeated 30 times, each time with a different composition of the test and training sets. For each target, the reported final statistical measures were averaged over the indicated number of repetitions.

Model Statistics

A broad spectrum of statistical indicators is available for assessing the performance of a given classification model (56). In this study, we report the most commonly encountered measures for estimating prediction accuracy of a classifier: sensitivity (SE), specificity (SP), and overall accuracy (Q). These quantities are defined in terms of the numbers of true positives (tp) and false positives (fp), indicating strong binders to either hERG or 5-HT_2B in our case, and the numbers of true negatives (tn) and false negatives (fn), that is, presumably nonactive compounds. Sensitivity, SE = tp/(tp + fn), then expresses the prediction accuracy for molecules with high affinity to the considered targets, whereas specificity reflects the prediction accuracy for weak binders: SP = tn/(tn + fp). We also tabulate the overall prediction accuracy defined as Q = (tp + tn)/(tp + fp + tn + fn). In addition, following Ung et al. (58), we report the values of Matthew's correlation coefficient (69) C = [tp × tn - fp × fn]/[(tp + fn)(tp + fp)(tn + fp)(tn + fn)]^1/2, which is another measure of the overall prediction performance. This indicator has interesting properties: For a perfect classifier (fp = fn = 0), C ) 1.0, while for random performance (resulting in tp ≈ fp and tn ≈ fn on average), C ≈ 0. A negative value C would imply worse than random performance.

Results

hERG Models

An initial evaluation of the Shape Signatures descriptors was performed with the hERG data set. The results of various classification schemes applied to discriminate between strong and weak blockers of hERG are summarized in Table 2. All of the reported models perform substantially better than random. The UFS-SVM model with shape and charge descriptors appears to perform slightly better than the k-NN models. The average prediction accuracy for the external test sets varies from 66 to 74%, which is comparable to the 70−85% established by summarizing the results of other predictive modeling studies of hERG reported in the literature (35).

Table 2.

Classification of hERG Active and Nonactive Compounds Using Shape Signatures Descriptors with Different Methods

			leave-20-out testingd
classification methoda	descriptorsb	10-fold cross-validationc (%)	SE (%)	SP (%)	Q (%)	C
SVM	shape only	77	70	68	69	0.390
UFS-SVM	shape + charges	78	73	74	74	0.488
k-NN (k = 7)	shape only	68	79	53	66	0.343
k-NN (k = 3)	shape + charges	69	79	56	67	0.367

^aThe specified values of k for k-NN classifications are those that yield the maximum average overall prediction accuracies Q for the leave-20-out experiments.

^b“Shape only” label descriptor sets derived from the 1D Shape Signatures histograms, and “shape + charges” designates descriptors sets based on the 2D Shape Signatures histograms.

^cThis column lists prediction accuracies estimated from 10-fold cross-validations performed on the entire data set.

^dThe tabulated values of SE, SP, Q, and C are averaged over 30 different hold-out test sets.

5-HT_2B Models

Our evaluation of the 5-HT_2B data is shown in Table 3. As a direct comparison with Shape Signatures descriptors, we have used a set of 184 2D molecular descriptors available in MOE. The initial data matrix for these descriptors was also processed by the UFS algorithm (described above), and the resulting ensemble of 73 2D MOE descriptors was used for final calculations. Both of the UFS-SVM models with Shape Signatures or MOE descriptors perform similarly in terms of model statistics resulting in prediction accuracies of 87% after 10-fold cross-validation (Table 3). We have also used UFS with SOM and the Shape Signatures descriptors. Among 30 randomizations used in the SOM, the average percentage of correctly predicted compounds belonging to both classes included in the training set was 85% for active compounds and 86% for nonactive compounds. The average percentage of correctly predicted compounds from the test set was 78% for active compounds and 54% for nonactive compounds. On average, 5% of the tested compounds were assigned to the “unclassified” class. The ratio of correctly predicted compounds from both tested groups (active and nonactive) was in general well-balanced for the training sets (Figure 4a). At the same time, a clear bias toward better prediction accuracy for active structures was observed for the internal test sets (Figure 4b). One possible reason for such bias is an increased dissimilarity and smaller number of nonactive compounds in the training set as compared to active compounds. Therefore, the training results are statistically less significant for the nonactive subset.

Table 3.

Classification of 5-HT_2B Active and Nonactive Compounds Using Shape Signatures or MOE Descriptors with Different Methods

			leave-42-out testingd
classification methoda	descriptorsb	10-fold cross-validationc (%)	SE (%)	SP (%)	Q (%)	C
SVM	shape only	80	81	59	73	0.424
UFS-SVM	shape + charges	87	91	69	83	0.638
UFS-SVM	MOE (2D)	87	91	70	84	0.640
UFS-SOMe	shape + charges	86	78	54	70	0.345
k-NN (k = 3)	shape only	69	86	46	72	0.352
k-NN (k = 1)	shape + charges	74	93	53	79	0.527

^aThe specified values of k for k-NN classifications are those that yield the maximum average overall prediction accuracies Q for the leave-42-out experiments.

^b“Shape only” labels descriptor sets derived from the 1D Shape Signatures histograms, and “shape + charges” designates descriptors sets based on the 2D Shape Signatures histograms.

^cThis column lists prediction accuracies estimated from 10-fold cross-validations performed on the entire data set.

^dThe tabulated values of SE, SP, Q, and C are averages over 30 different hold-out test sets.

^eOn average, one active and one nonactive molecule were unclassified.

Figure 4.

Prediction accuracy for 5-HT_2B active and nonactive compounds, from the training (a) and test (b) sets used in the Kohonen network (SOM).

Discussion

Computational Methods

Several ligand-based and structure-based computational methods exist that implicitly or explicitly include some representation of molecular shape. The program UCSF DOCK (70, 71) packs spheres into a protein receptor site; candidate ligands can then be evaluated for shape compatibility with the site by checking for containment within the array of spheres. Comparative molecular field analysis (72) (CoMFA) represents the shape of molecules implicitly by mapping steric and electrostatic fields on a 3D grid that surrounds the molecule; biological activity is then correlated with variations of the fields at the grid points. Essentially all automated docking programs implicitly represent molecular shape via some form of energy calculations. Inasmuch as shape is directly related to molecular structure, tools that employ pharmacophore models (e.g., Catalyst, UNITY, etc.) represent molecular shape via interatomic distance constraints. The aforementioned shape-based methods, although invaluable, demand considerable computation involving either energy or distance–geometry calculations. Moreover, the matching of a compound to a receptor site or pharmacophore model typically involves some sort of simulation (genetic algorithm, Monte Carlo method, etc.) for generating orientations and configurations of the ligand. CoMFA requires manual alignment of the series of molecules, a highly subjective process that effectively limits the number of compounds to ≈150. When the goal is to screen large vendor or legacy databases of compounds, such methods may lead to prohibitive computational costs. What is needed is a method that can rapidly compare shapes of large databases of compounds to each other, or to a receptor site, with a minimum of computation, without requiring explicit 3D representation of shape and without actual ligand–receptor docking. This is just the sort of method embodied in Shape Signatures descriptors used for QSAR, which has several advantages over traditional molecular descriptor-based QSAR methods (Table 1).

hERG Models

While there have been many models for this potassium channel, our goal in this study was to use hERG as a test case to evaluate Shape Signatures descriptors with different classifier algorithms. A qualitative comparison of our hERG results with those extracted from other studies reported to date (35) suggests that the differences may be insignificant. Overall, in this study, the prediction accuracies of SVM based methods outperform the k-NN models. Although k-NN models yield better selectivity rates, this is achieved at the expense of much lower specificity values. In particular, k-NN classifiers suffer from a large number of false positive predictions and, thus, are less selective than the procedures that use SVM. We found that regardless of the classifier employed, the models based on the 2D Shape Signatures (shape and charge) are slightly more accurate than those derived from the 1D signatures (shape). This implies that for better selectivity, one may need to incorporate the polarity of the molecules into the model. This observation is consistent with the notion that the hERG channel can accommodate inhibitors of different size and shape. This may also relate to the position of the basic N atom found in many hERG inhibitors and suggested to be important in many of the published pharmacophores. Ekins et al. previously used a single external set of 21 compounds (seven active and 14 nonactive) to test their classification models based on Kohonen and Sammon mapping techniques (40). For Kohonen SOM, they obtained SE = 86%, SP = 79%, and Q = 81%, and for Sammon mapping, they reported SE = 86%, SP = 100%, and Q = 95%. These results are better than the average values obtained over the series of 30 leave-20-out experiments tabulated in Table 2. However, if we turn to some of the best-performing UFS-SVM models in our study, these models yield SE = 80−90%, SP = 80−100%, and Q = 85−95%, which is comparable to the Sammon mapping model previously described (40).

5-HT_2B Models

Our results with the 5-HT_2B classification models are the major focus of this study and are the first classification models for this receptor to be reported to date. Previous computational modeling of 5-HT_2B has encompassed a traditional QSAR study, which used a small number of tetrahydro-β-carboline derivatives as antagonists with the rat 5-HT_2B contractile receptor in the rat stomach fundus (73). A 3D QSAR with GRID-GOLPE using 38 (aminoalkyl)benzo and heterocycloalkanones as antagonists of the human receptor resulted in poor model statistics possibly due to the limited range of activity measured and the complexity of the functional response (74). Homology models based on the bacteriorhodopsin as well as rhodopsin X-ray structures have been used for the mouse and human 5-HT_2B receptor and combined with site-directed mutagenesis. The models based on bacteriorhodopsin proved more reliable and confirmed an aromatic box hypothesis for ligand interaction along transmembrane domains 3, 6, and 7 with serotonin (75). A more recent 5-HT_2B homology model with the rhodopsin-based model of the rat 5-HT_2A together with molecular dynamics simulations was used to determine the sites of interaction for norfenfluramine. Site-directed mutagenesis showed that Val 2.53 was implicated in high affinity binding through van der Waals interactions and the ligand methyl groups (76).

We have found in this study that, similar to the hERG modeling described previously, for 5-HT_2B, SVM generally outperforms k-NN methods (Table 3). Interestingly, the same observation has been documented in a number of classification studies across different classes of protein targets (57, 58). In comparison to the results with the hERG data set (Table 2), for 5-HT_2B, we were able to achieve generally better overall prediction accuracies for the test sets within the range of 72−84%. Among the models based on Shape Signatures descriptors, the UFS-SVM procedure is again the best. On average, these models compare well with SVM classifications paired with the traditional 2D molecular descriptors computed with MOE. This observation further validates the applicability of the Shape Signatures-derived molecular descriptors. The 2D Shape Signatures classifiers appear superior to the models based on the 1D histograms, indicating that molecular polarity is likely necessary for generating more accurate predictions for 5-HT_2B.

The results presented in Table 3 also demonstrate that the prediction accuracies achieved in the Kohonen modeling experiments (86% prediction accuracy estimated from 10-fold cross-validations), in general, were similar to those observed in the best UFS-SVM models (87%) and better than in k-NN models (74%), when the same 102 “shape + charges” descriptors were used. The iterative methods based on vector quantization algorithms, such as SVM and Kohonen SOM, perform better in classification tasks related to analysis of highly clustered and heterogeneous property spaces. Classification results certainly vary from one classifier to another, but if several different classification models using the same collection of molecular descriptors produce consistent results, it would certainly add to the credibility of the utilized descriptor set. We found that the average statistics produced for either hERG or 5-HT_2B by k-NN, SVM, and SOM models are generally consistent overall, indicating that the Shape Signatures histograms constitute a useful set of new molecular descriptors for these types of classification problems. Their utility likely also rests in their ability to reliably capture the shape and charge requirements for molecules to fit to these proteins.

To further investigate the utility of the shape signatures-based molecular descriptors, we evaluated the SVM and SOM models with an additional external test set (77). This set included 20 compounds with documented activities toward 5-HT_2B: six active and 14 nonactive molecules. We attempted to classify compounds in this data set using models built on the original data set of 182 5-HT_2B molecules, which are described in Table 3. As before, all calculations were performed using the set of 102 2D Shape Signatures descriptors. The best UFS-SVM and SOM models yielded SE = 33%, SP = 71%, and Q = 60%. This prediction accuracy is lower than the corresponding values reported in Table 3 for the original data set, especially for the SVM models. The reason for this may be 2-fold. First, with the current settings, Shape Signatures may not perform well in separating close structural analogues in an external test set. However, the use of alternative descriptors, such as the electrotopological indices (calculated using the SmartMining program), led to similar prediction accuracy on the same test set. Second, and probably the most important point to consider, is that the compounds in the external test set may lie outside the chemical space occupied by the structures from the original 182 molecules data set. Indeed, similarity measures in the form of Euclidean distances, calculated using the ChemoSoft software between pairs of molecules, suggest significant structural differences between the original data set (182 structures) and the external test set (data not shown).

A major objective of the reported study was to thoroughly examine the quality of a novel set of molecular descriptors derived from the associated molecular Shape Signatures previously used as a virtual screening tool for drug discovery (41-45). These descriptors are inherently 3D and fundamentally different from other 2D/3D descriptor collections normally used in predictive QSAR modeling (1, 78). We have therefore extended the Shape Signatures methodology in the form of molecular classifiers for computational toxicology. Practical classification models for the 5-HT_2B receptor and the hERG potassium channel have been constructed and validated. Our classification models for 5-HT_2B offer the potential to predict cardiotoxicity earlier in drug discovery. In the case of 5-HT_2B, we report the first Shape Signatures-SVM-based classification models, which exhibit average accuracies in the range of 73−83%. These findings are comparable with the results of the SVM classification using traditional 2D molecular descriptors available in the commercially available software MOE, which was also performed in this study. Further research is currently underway in our laboratories to examine the combination of Shape Signature histograms with traditional 2D descriptors (such as from MOE) to assess whether this improves the models. For hERG, the prediction accuracy is comparable with the results of alternative computational models published to date. Altogether, our study demonstrates that the reported classification models perform well in discriminating between hERG and 5-HT_2B active and nonactive molecules and could be applicable to other protein targets. We also note that, as with any molecular descriptors and algorithms used for QSAR to date, it is important to understand the chemical space covered in both the training and the test sets for optimal predictions (i.e., the applicable prediction space). Our results certainly attest to the notion that molecular shape and polarity are indeed key characteristics that regulate molecular activity toward specific protein targets. Given the simplicity, physical transparency, and applicability of the Shape Signatures representation, this method encodes these main features in a compact and practical form. Because the procedure obviates direct 3D molecular alignment or grid generation [as in CoMFA and comparative molecular similarity analysis (CoMSIA) etc.], the algorithm is also relatively fast and efficient. Models based on Shape Signatures histograms can therefore accommodate structurally diverse compounds; once generated, they can be used for a variety of tasks that require molecular recognition, and no model refitting is necessary in going from one problem to another (Table 1). We are currently using Shape Signatures to aid in drug discovery projects while also evaluating the Shape Signatures descriptors for other physicochemical properties, as we believe this approach is generally applicable. Overall, we conclude that the Shape Signatures method offers a novel practical approach to classifying compounds with respect to their potential for cardiotoxicity. Further future studies will use these 5-HT_2B models for mining databases to identify additional compounds for in vitro testing to prospectively validate them, a strategy that we have successfully undertaken for transporters (79, 80).