In Silico Pharmacoepidemiologic Evaluation of Drug-Induced Cardiovascular Complications Using Combined Classifiers

Abstract

Drug-induced cardiovascular complications are the most common adverse drug events and account for the withdrawal or severe restrictions on use of multitudinous post-marketed drugs. In this study, we developed new in silico models for systematic identification of drug-induced cardiovascular complications in drug discovery and post-marketing surveillance. Specifically, we collected drug-induced cardiovascular complications covering five most common types of cardiovascular outcomes (hypertension, heart block, arrhythmia, cardiac failure, and myocardial infarction) from four publicly available data resources: Comparative Toxicogenomics Database, SIDER, Offsides, and MetaADEDB. Using these databases, we developed a combined classifier framework through integration of five machine-learning algorithms: logistic regression, random forest, k-nearest neighbors, support vector machine, and neural network. The totality of models included 180 single classifiers with area under receiver operating characteristic curves (AUC) ranging from 0.647 to 0.809 on 5-fold cross validations. To develop the combined classifiers, we then utilized a neural network algorithm to integrate the best four single classifiers for each cardiovascular outcome. The combined classifiers had higher performance with an AUC range from 0.784 to 0.842 compared to single classifiers. Furthermore, we validated our predicted cardiovascular complications for 63 anticancer agents using experimental data from clinical studies, human pluripotent stem cell-derived cardiomyocyte assays, and literature. The success rate of our combined classifiers reached 87%. In conclusion, this study presents powerful in silico tools for systematic risk assessment of drug-induced cardiovascular complications. This tool is relevant not only in early stages of drug discovery, but throughout the life of a drug including clinical trials and post-marketing surveillance.

Graphical Abstract

INTRODUCTION

The systematic evaluation of drug cardiovascular (CV) safety profiles is essential for drug development and patient care. Cardiotoxicity is one of the most common severe and life-threatening adverse effects of drug treatments and thus is a major concern in drug discovery and post-marketing surveillance.¹ Acute and chronic cardiotoxicity induced by drug treatments has a relatively high incidence rate and is characterized by severe negative symptoms including high blood pressure, heart failure, and death.² According 1to a study of all safety-related withdrawals of prescription drugs from worldwide markets from 1960 to 1999, heart toxicity is one of the most common reasons for drug withdrawal.³ Numerous otherwise effective drugs, including terfenadine, astemizole, cisapride, vardenafil, and ziprasidone, have been withdrawn from the market owing to CV complications.³ Compounding the problem, cardiotoxicity has been reported for many anticancer drugs including chemotherapies, targeted therapies, and immunotherapies.^4–7 These reports likely represent the tip of the iceberg, given the explosion of molecular targeted therapies with few systematic evaluations of cardiotoxicity risk. One of the 10 recommendations for 2016 Cancer Moonshot initiative is to “Accelerate the development of guidelines for monitoring and management of patient symptoms to minimize side effects of therapy.”⁸ This statement emphasizes the driving imperative to accelerate drug development by systematically identifying drug-induced CV complications.

In the past several decades, tests including radio ligand binding assays, electrophysiology measurements, rubidium-flux assays, and fluorescence-based assays have been used to assess the propensity of compound cardiotoxicity.⁹ Such experimental methods are not suitable for evaluation of a large number of compounds in early stage drug discovery due to high expense, and poor throughput. Moreover, animal models are limited by significant functional disparities between animal and human cardiomyocytes.¹⁰ Recent advances of in silico approaches and tools have promise for systematic evaluation of drug-induced CV complications in both drug discovery and post-marketing surveillance.^11–16 For example, a recent study has integrated chemical, biological, and phenotypic properties of drugs to develop predictive and reasonably accurate machine-learning models for evaluation of adverse drug reaction.¹⁴ In 2010, Frid and co-workers developed in silico predictive models for prediction of cardiac adverse effects with good sensitivity.¹⁵ Building on this, Hitesh and co-workers built classifiers for assessment of drug cardiotoxicity with accuracies ranging from 0.675 to 0.95 by leave-one-out cross validation.¹⁶ Reported studies thus far are largely limited by use of only a single machine-learning algorithm with low or moderate accuracy. In order to advance the field of drug development, it is vital to develop robust and effective in silico models with high accuracy for evaluation of drug-induced cardiotoxicity.

In this study, we proposed a combined classifier framework for prediction of five common CV complications associated with drug treatments (Figure 1). In total, we built 180 single classifiers through integration of molecular fingerprint (FP) and physical descriptors of drugs with four machine-learning algorithms: logistic regression, random forest, k-nearest neighbors, and support vector machine. We then utilized a neural network to combine the four best single classifiers for each CV complication. We showed that the combined classifiers outperformed the single classifiers. Using our combined classifier, we computationally identified multiple CV complications induced by various anticancer agents. We validated the predicted CV complications for various anticancer agents with experimental data. Altogether, the combined classifiers presented here offer a useful computational framework for systematic evaluation of drug-induced CV complications in drug discovery and post-marketing surveillance.

Figure 1. Diagram illustrating a combined classifier framework for prediction of drug-induced cardiovascular (CV) complications.

Five types of drug-induced CV complications are collected from three public databases (CTD, SIDER and MetaADEDB). The single classifiers are built on the basis of molecular fingerprints and the selected physical descriptors using four machine-learning algorithms (logistic regression, random forest, k-nearest neighbors, and support vector machine). The four best single highest performance classifiers were picked for building the combined classifiers using a neural network algorithm. The performance of all models was evaluated by both 5-fold cross-validation and the external validation sets collected from Offsides database ²⁰. kNN: k-nearest neighbors; SVM: support vector machine; RF: random forest; LR: logistic regression.

MATERIALS AND METHODS

Data Preparation

We searched over 20 types of CV events defined by Medical Subject Headings (MeSH) and Unified Medical Language System (UMLS) vocabularies.¹⁷ All drug-induced CV complications were collected from four databases: Comparative Toxicogenomics Database (CTD),¹⁸ SIDER,¹⁹ Offsides,²⁰ and MetaADEDB.²¹

The protocol of data collection is implemented in five steps: (1) All CV complication terms were further screened based on the publication of “Common Terminology Criteria for Adverse Events” (CTCAE, version 4.03, 2010) released by the U.S. Department of Health & Human Services; (2) Each item was annotated with the most commonly used MeSH and UMLS; (3) The four databases were searched using the MeSH unique ID or UMLS ID to obtain drug-induced CV complication information and then the InChI Keys of according drugs were calculated via Open Babel GUI;²² (4) Drugs annotated in DrugBank²³ with unique ID were pinpointed by matching the InChI Keys; and (5) The items with well-annotated clinical report data were used and all the duplicated drugs in the same class of CV complications were excluded. To maintain a sufficient number of drugs with well-annotated CV complication information, we finally obtained five common drug-induced CV complications for building models: hypertension (MeSH ID: D006973), heart block (MeSH ID: D006327), arrhythmia (MeSH ID: D001145), cardiac failure (MeSH ID: D006333), and myocardial infarction (MeSH ID: D009203). The drug information of the five CV complications is provided in Supporting Information, Table S1. Detailed statistical results are listed in Table 1.

Table 1.

Statistical description of five drug-induced cardiovascular (CV) complications collected from four databases.

CV complication	Number of unique drugs
	CTD	SIDER	Offsides	MetaADEDB	Total
Hypertension	289	266	145	481	726
Arrhythmia	473	66	71	660	796
Heart block	159	106	201	237	473
Cardiac failure	135	164	270	267	585
Myocardial infarction	139	58	89	257	408

Chemical structure representation

In this study, two-dimensional (2D) descriptors of drugs were generated by MOE 2010 software²⁴ to represent molecular descriptors (MD) and structural information. All drug structures were processed in MOE 2010 software by protonating strong bases, deprotonating strong acids, removing inorganic counter ions, adding hydrogen atoms, generating stereo isomers, and validating single 3D conformers by molecular washing and energy minimizing using. Descriptors calculated by MOE consist of 186 2D descriptors, including physical property descriptors, atom count and bond count descriptors, adjacency and distance matrix descriptors, subdivided surface area descriptors, Kier and Hall connectivity and Kappa shape indices descriptors, pharmacophore feature descriptors, and partial charge descriptors. Moreover, four sets of molecular fingerprints were also generated by PaDEL-Descriptor,²⁵ including MACCS, EState, Pubchem, and Substructure fingerprint (SubFP). The more details of the descriptors can be found in recent studies.^25,26

Molecular descriptor selection

Pearson correlation coefficient analysis is a common measure for eliminating irrelevant and redundant features (descriptors). In this study, all descriptors were selected based on two criteria: (1) highly relevant label of CV complications (CV complication [1] vs. non-CV complications [−1]); (2) lack of inter-correlation or self-correlation to avoid the over-fitting issue.^27,28 Specifically, descriptors that have absolute Pearson correlation coefficient less than 0.1 (|PCC|<0.1) with the label “CV complications” were eliminated to reduce irrelevant descriptors. In addition, for any two descriptors having a pairwise correlation coefficient higher than 0.9 (|PCC|>0.9), the one with the lower correlation coefficient with the CV complication label was excluded. In total, 36, 36, 26, 32 and 37 molecular descriptors (Supporting Information, Table S2) were ultimately selected to build single classifiers for hypertension, heart block, arrhythmia, cardiac failure, and myocardial infarction, respectively.

Description of four single classifiers

Four machine-learning methods, including k-nearest neighbors (kNN), logistic regression (LR), random forest (RF), and support vector machine (SVM), were employed to build single classifiers. We then utilized a neural network (NN) algorithm to construct the combined classifiers based on the single classifiers with the best performance. All tools are freely available in Orange Canvas (version 2.7.6).²⁹

k-nearest neighbors (kNN)

kNN is a non-parametric algorithm to classify objects based on the closest training samples in the feature space.³⁰ It determines the category of a sample based on the categories of its k nearest neighboring samples. In this study, the k value was set to 5, and the distance d between samples x and y was measured by Euclidean distance that is calculated using equation (1) where n is the number of descriptors.

$$\mathit{d}(x,y)=\sqrt{\sum _{k=1}^n{(x_k-y_k)}^2}$$ (1)

Logistic Regression (LR)

LR is a classification algorithm developed by statistician David Cox in 1958.³¹ LR calculates the probabilities using a logistic function to measure the relationship between a multitude of independent variables and categorical dependent variables. It maps the result of a linear regression function to a value ranging from 0 to 1 by the sigmoid function, and this value can be modeled as a probability. The realization of LR can be summarized as equation (2) and (3), where a and b are the coefficients determined by LR, n is the number of independent variable x.

$$z=a+\sum _{i=1}^n{b}_i{x}_i$$ (2)

$$p=\frac{1}{1+e^{-z}}$$ (3)

Random Forest (RF)

RF is an ensemble algorithm for classification developed by Leo Breiman and Adele Cutler.³² It creates a large number of decision trees by bootstrapping training samples and randomly selecting subsets of original independent variables and predicts the category of new samples via integrating the outputs of the individual trees.³³ RF is appealing because it is relatively fast at training models and can be used directly for high-dimensional problems.³⁴ In this study, the number of decision trees in forest is set to 10, the minimal number of instances in a leaf is set to 5, and the considered number of attributes at each split is equal to the square root of the number of attributes in the input dataset.

Support Vector Machine (SVM)

SVM was first developed by Vapnik.³⁵ The core idea of this algorithm is to map data into a higher dimensional space on the basis of the frame of Vapnik-Chervonenkis (VC) theory. SVM defines a decision boundary which is expressed as separating hyperplane based on a linear combination of functions parameterized by support vectors. It seeks the support vectors by maximizing the margin between the instances of different classes. Each molecule is expressed in terms of an eigenvector t, and the chosen patterns t₁, t₂, … t_n are the components of t. The classification label y was introduced in SVM training. The i^th molecule in the data set is defined as M_i = (t_i, y_i), where y_i = 1 for the “CV complication” class and y_i = −1 for the “non-CV complication” class.³⁶ SVM gives a decision function (classifier) using equation (4)

$$f(\mathbf{t})=sgn\left(\frac{1}{2}\sum _{i=1}^n{a}_{i}K({\mathbf{t}}_i,\mathbf{t})+b\right)$$ (4)

where α_i is the coefficient to be trained, and K is a kernel function. Parameter α_i is trained via maximizing the Lagrangian expression using equations (5) and (6).

$$\underset{{\alpha }_i}{\text{maximize}}\sum _{i=1}^n{\alpha }_i-\frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n{\alpha }_i{\alpha }_j{y}_i{y}_{j}K({\mathbf{t}}_i,\mathbf{t})$$ (5)

$$\text{subject}\text{to}:\sum _{y_i=1}{y}_i{\alpha }_i=0,0\le {\alpha }_i\le C$$ (6)

In this study the commonly used kernel, Gaussian radial basis function (RBF), was utilized. In order to obtain the optimal performance model, the auto searching program “grid” in the LibSVM 3.2 package³⁷ was employed to determine the kernel parameter γ and penalty parameter C by utilizing a grid strategy based on 5-fold cross validation.

The SVM algorithm in this study is provided by an SVM learner in Orange Canvas 2.7, which can provide posterior predictive probability for each prediction. Probabilities are created by directly training an SVM and then training the parameters of an additional sigmoid function to map the SVM outputs into probabilities.^38,39 Given training examples x_i ∈ R_n, i = 1, … , l, labeled by y_i ∈ {+1, −1}, the conversion algorithm for the probability P r(y = 1|x) is described as equation (7)

$$Pr(y=1\mid x)\approx P_{A,B}(f)\equiv \frac{1}{1+exp(Af+B)}$$ (7)

where f_i is an estimate of decision function f(x_i) , and parameters A and B are determined by minimizing the negative log likelihood of the training data, which is a cross-entropy error function (with N⁺ of the y_i’s positive, and N⁻ negative):

$$\begin{gathered} z=\begin{array}{c}min\\ (A,B)\end{array}F(z)=-\sum _{i=1}^l(t_{i}log(p_i)+(1-t_i)log(1-p_i)), \\ \text{for}{p}_i=P_{A,B}(f_i),\text{and}{t}_i=\{\begin{array}{l}\frac{N^{+}+1}{N^{+}+2}if{y}_i=+1\hfill \\ \frac{1}{N^{-}+2}if{y}_i=-1\hfill \end{array},i=1,\dots ,l. \end{gathered}$$ (8)

The detailed description about probability generation in SVM approach are provided in original works.^38,39

Description of Combined Classifiers

Combined classifiers can improve prediction accuracy to some extent via comprehensively synergizing the complementary information provided by other methods. This approach is particularly suitable for the cases where single classifiers may not achieve satisfactory predictive accuracy. A combined classifier optimizes the performance of single classifiers by enhancing prediction reliability.^36,40,41 In this study, four best single classifiers were combined using the NN algorithm (Figure 1).

Neural Network (NN)

NN is an information processing paradigm that is inspired by biological neural networks.⁴² A collection of connected units called artificial neurons is the basis of NN. Each connection between neurons can transmit a unidirectional signal with various activating strength hinges on the weight of neurons. The combined incoming signals that exceed a given threshold will promote the propagation of a signal to the downstream neurons and activate them. The output of neuron i is given in Equations (9) and (10).

$${\mathit{net}}_i=\sum _{j=1}^n{w}_{ij}{x}_j-\theta$$ (9)

$$y_i=f({\mathit{net}}_i)$$ (10)

where w_ij the connect weighting value from neuron j to neuron i, θ is the threshold, and f is the activation function.

In Orange Canvas 2.7, the NN is implemented by a multilayer perceptron with a single hidden layer. It is performed by minimizing an L₂-regularized cost function with SciPy’s implementation of Limited-memory BFGS (L-BFGS), which is often used for parameter estimation in machine-learning.^43,44 In this study, the number of hidden layer neurons is set to 11, the regularization factor is set to 3.0, the max iteration is set to 200, and data was normalized prior to learning. Normalization was done by subtracting each column by the mean and dividing by the standard deviation.

Experimental design

Drugs with at least one of five types of CV complications (hypertension, arrhythmia, heart block, cardiac failure and myocardial infarction) were curated from CTD, SIDER, MetaADEDB, and Offsides. The same number of corresponding decoys were then randomly selected from remaining drugs in DrugBank. The drugs collected from CTD, SIDER and MetaADEDB were combined with their decoys as the training sets, while the drugs from Offsides and the corresponding decoys were used as the validation sets after eliminating the ones existing in the training sets. The drugs with CV complications were labeled as “+1” and the decoys (or negative) were labeled as “−1”. Eventually, the training sets of hypertension, arrhythmia, heart block, cardiac failure and myocardial infarction contained 1,162, 1,450, 544, 630 and 638 drugs, and the validation sets included 290, 142, 402, 540 and 178 drugs, respectively (Table 2).

Table 2.

Number of drugs in the training sets and the external validation sets.

CV complication	Training set			External validation set
	Positive	Negative	Total	Positive	Negative	Total
Hypertension	581	581	1,162	145	145	290
Arrhythmia	725	725	1,450	71	71	142
Heart block	272	272	544	201	201	402
Cardiac failure	315	315	630	270	270	540
Myocardial infarction	319	319	638	89	89	178

For each CV complication, single classifiers were firstly built using four algorithms (RF, kNN, SVM and LR) and the selected molecular descriptors. Subsequently, four types of fingerprints were introduced to enhance the predictive accuracy. For each algorithm and specific CV complication, the best single classifiers were constructed using the selected molecular descriptors combined with molecular fingerprints. Then the training sets were predicted with the four best single classifiers to obtain positive and negative output probabilities ( $P_i^{+1}$ and $P_i^{-1}$ i = 1,2,3,4). After that, the eight output probabilities were chosen as new descriptors to develop the NN classifiers, generating probabilities ( $P_C^{+1}$ and $P_C^{-1}$) for each drug as final predictions.

Model validation

All classification models were assessed by true positives (TP, drugs with known CV complication were predicted as cardiotoxic drugs), true negatives (TN, drugs with non-CV complication were predicted as non-cardiotoxic drugs), false positives (FP, drugs with non-CV complication were predicted as cardiotoxic drugs), and false negatives (FN, drugs with known CV complication were predicted as non-cardiotoxic drugs). In addition, four metrics (sensitivity [SE], specificity [SP], overall predictive accuracy [Q], and precision [P]) were calculated using equations (11–13) for further evaluation of model performance.

$$\text{SE}=\frac{\text{TP}}{\text{TP}+\text{FN}}$$ (11)

$$\text{SP}=\frac{\text{TN}}{\text{TN}+\text{FP}}$$ (12)

$$\mathrm{Q}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FN}+\text{FP}+\text{TN}}$$ (13)

$$\mathrm{P}=\frac{\text{TP}}{\text{TP}+\text{FP}}$$ (14)

Moreover, receiver operating characteristic (ROC) curves were used to evaluate the performance of the classifiers.⁴⁵ An ROC curve exhibits the behavior of a model by measuring the relationship between true positive rate and false positive rate. The area under ROC curve (AUC) was computed for each of the classifiers. A perfect classifier yields an AUC of 1, whereas a random model has an expected AUC of 0.5.

RESULTS

GPCRs are the most common target family in training sets

To explore target distribution of drugs in the training sets, we extracted the drug-target interactions for drugs in the training sets from DrugBank²³ and examined the drug target family distribution according to the IUPHAR/BPS Guide to PHARMACOLOGY in 2018.⁴⁶ Here, targets are divided into five categories: kinases, G-protein-coupled receptors (GPCRs), nuclear receptors, ion channels, or others (Figure S1). We found that GPCRs were most highly represented among the four types of target families across five types of CV complications, following by ion channels, kinases, and nuclear receptors.

Molecular descriptors together with molecular fingerprint were the best single classifiers

We built 36 single classifiers for each CV complication: a. 4 classifiers built by 4 machine-learning algorithms (kNN, LR, RF and SVM) with the selected molecular descriptors, b. 16 classifiers built by combining 4 machine-learning algorithms with 4 different types of fingerprints (EState, MACCS, Pubchem and SubFP), and c. 16 classifiers built by 4 machine-learning algorithms with 4 types of integrated features by the selected molecular descriptors and fingerprints. In total, we built 180 single classifiers across 5 types of drug-induced CV complications. The performance of each classifier was assessed by both 5-fold cross validation and external validation. The performances of the 180 single classifiers are provided in Supporting Information, Table S3.

To make a more intuitive comparison between the single classifiers, the average AUC scores of the single classifiers developed using the selected molecular descriptors (MD) only, molecular fingerprint (FP) only, and combination of MD and FP (MD & FP) respectively are shown in Figure 2. The average AUC scores ranged from 0.687 to 0.793 on the 5-fold cross validations and ranged from 0.613 to 0.710 on the external validation sets. The MD&FP classifiers showed better performance on both 5-fold cross validations and the external validation sets compared to classifiers built by the MD or FP alone (Supporting Information, Figure S2). Hence, single classifiers built on MD & FP together were selected as the best single classifiers for further studies. Table 3 illustrates the performance of all single classifiers based on MD & FP across four machine-learning algorithms.

Figure 2.

Comparison of the average area under the receiver operating characteristic curves (AUC) scores across the single classifiers built using three types of descriptors: (i) molecular descriptors only, (ii) molecular fingerprint only, and (iii) molecular descriptors combined with molecular fingerprints, for 5-fold cross validations (A) and the external validation sets (B). Note: MD: molecular descriptors; SubFP: Substructure fingerprint.

Table 3.

Detailed performance of single classifiers built using the selected molecular descriptors (MD) combined with four types of molecular fingerprints across four machine learning algorithms.

Cardiovascular Complications	Descriptors	5-fold cross validation				External validation set
		kNN	RF	LR	SVM	kNN	RF	LR	SVM
Hypertension	MD&MACCS	0.759	0.727	0.723	0.763	0.717	0.682	0.647	0.644
	MD&Estate	0.702	0.752	0.732	0.749	0.669	0.672	0.678	0.701
	MD&Pubchem	0.717	0.765	0.689	0.765	0.667	0.720	0.609	0.711
	MD&SubFP	0.736	0.754	0.714	0.743	0.706	0.698	0.610	0.675
Arrhythmia	MD&MACCS	0.746	0.763	0.682	0.756	0.727	0.685	0.650	0.681
	MD&Estate	0.702	0.750	0.721	0.747	0.683	0.692	0.702	0.694
	MD&Pubchem	0.718	0.753	0.663	0.743	0.633	0.743	0.602	0.733
	MD&SubFP	0.725	0.739	0.685	0.681	0.680	0.658	0.657	0.553
Heart block	MD&MACCS	0.809	0.799	0.734	0.804	0.691	0.701	0.622	0.666
	MD&Estate	0.787	0.802	0.752	0.774	0.682	0.700	0.695	0.667
	MD&Pubchem	0.769	0.801	0.737	0.797	0.667	0.690	0.686	0.704
	MD&SubFP	0.800	0.799	0.755	0.808	0.624	0.657	0.670	0.642
Myocardial infarction	MD&MACCS	0.765	0.746	0.647	0.766	0.718	0.716	0.667	0.700
	MD&Estate	0.708	0.714	0.712	0.731	0.664	0.702	0.709	0.718
	MD&Pubchem	0.732	0.720	0.654	0.754	0.645	0.680	0.599	0.645
	MD&SubFP	0.706	0.723	0.672	0.717	0.733	0.698	0.664	0.672
Cardiac failure	MD&MACCS	0.757	0.741	0.667	0.767	0.641	0.667	0.632	0.659
	MD&Estate	0.728	0.737	0.709	0.744	0.643	0.663	0.685	0.646
	MD&Pubchem	0.708	0.720	0.665	0.739	0.615	0.664	0.648	0.615
	MD&SubFP	0.741	0.727	0.665	0.720	0.609	0.660	0.652	0.624

Note: The value in table represents the area under the receiver operating characteristic curve. MD: molecular descriptors; SubFP: Substructure fingerprint; kNN: k-nearest neighbors; SVM: support vector machine; RF: random forest; LR: logistic regression.

Combined classifiers outperform single classifiers

For each type of CV complications, we selected the best single classifiers built by MD & FP and generated by the 4 machine-learning algorithms in order to construct the combined classifiers. From the comparison of AUC values as shown in Figure 3, the combined classifiers outperformed single classifiers in cross validation. For example, the AUC of the combined classifier (AUC = 0.842) is higher than all four single classifiers (kNN AUC = 0.809, LR AUC = 0.755, RF AUC = 0.802, SVM AUC = 0.808) for prediction of drug-induced heart block. Building on this observation, Figure 4 further shows that the combined classifiers outperform the corresponding four best single classifiers for all 5 types of CV complications on both cross validations and external validation sets. Table 4 provides detailed performance of the five combined classifiers. Overall, most of the combined classifiers achieved a satisfactory performance in both cross validation and external validation sets. For example, the AUC values of the combined classifiers for prediction of drug-induced hypertension are 0.800 in the 5-fold cross validation and 0.756 in the external validation set. High cross validation AUC values are also achieved by the combined classifiers on other CV complications including heart block (AUC = 0.842), arrhythmia (AUC = 0.784), myocardial infarction (AUC = 0.790), and cardiac failure (AUC = 0.785). Taken together, the combined classifiers offer potential tools for computational risk assessment of drug-induced CV complications with high accuracy compared to the single classifiers. We hence examined the predicted drug-induced CV complications for anticancer agents via combined classifiers.

Figure 3.

Receiver operating characteristic (ROC) curves of combined classifiers and the four best MD & FP classifiers built by combining the selected molecular descriptors and molecular fingerprints across five types of drug-induced cardiovascular complications on 5-fold cross validation. Note: AUC: the area under the receiver operating characteristic curves; MD: molecular descriptors; SubFP: Substructure fingerprint; kNN: k-nearest neighbors; SVM: support vector machine; RF: random forest; LR: logistic regression.

Figure 4.

Comparison of area under the receiver operating characteristic curves (AUC) of the combined classifiers with the average AUC for four best single classifiers and on 5-fold cross validation (A) and external validation sets (B).

Table 4.

Performance of the five combined classifiers across five types of drug-induced cardiovascular complications.

Combined classifier	5-fold cross validation					External validation set
	SE	SP	Q	P	AUC	SE	SP	Q	P	AUC
Hypertension	0.750	0.728	0.739	0.734	0.800	0.669	0.710	0.690	0.698	0.756
Arrhythmia	0.712	0.723	0.717	0.720	0.784	0.648	0.761	0.704	0.730	0.734
Heart block	0.721	0.813	0.767	0.794	0.842	0.507	0.746	0.627	0.667	0.699
Myocardial infarction	0.690	0.765	0.727	0.746	0.790	0.596	0.708	0.652	0.671	0.742
Cardiac failure	0.686	0.737	0.711	0.722	0.785	0.537	0.707	0.622	0.647	0.693

Note: SE: sensitivity; SP: specificity; Q: overall predictive accuracy; P: precision; AUC: the area under receiver operating characteristic curves.

Clinical studies and human pluripotent stem cell-derived cardiomyocyte assays validate combined classifiers

Despite advances in cancer treatments, the frequency of CV complications induced by anticancer agents (i.e., chemotherapy and targeted therapy) has been substantially increasing.⁴⁷ We applied the four best single classifiers as well as the combined classifiers to predict 5 types of CV complications for 63 anticancer small molecular agents, including 26 targeted therapeutic agents (kinase inhibitors in Figure 5A) and 37 chemotherapeutic agents (non-kinase inhibitors in Supporting Information, Figure S3). According to the known drug-induced CV complications labeled by Drugs@FDA database⁴⁸, we found a higher success rate of 87% (108/124) for the combined classifiers compared to the four best single classifiers (79%=392/496, Supporting Information, Table S4). For instance, pazopanib-induced cardiac failure and myocardial infarction are severe adverse reactions listed in the Drugs@FDA database and have been reported by several double-blind placebo-controlled trials in patients^49,50 and a variety of clinical reports.^51–54 Our combined classifiers successfully identified pazopanib-induced cardiac failure and myocardial infarction, while only one of the best single classifiers offered true prediction in this case. An additional example includes Arsenic trioxide (As₂O₃), which is approved for treating acute promyelocytic leukemia. Here itwas predicted to have potential cardiac failure by the combined classifiers, while only half of the best single classifiers generated the toxicity consistent prediction. Arsenic trioxide was reported to lead to dysfunction of myocardium and reduction of contractility,⁵⁵ suggesting 100% accuracy of the combined classifiers compared to 50% accuracy of the best single classifiers.

Figure 5.

Validation of the combined classifiers using reported experimental data from human pluripotent stem cell-derived cardiomyocyte assays and literature data. (A) Predicted cardiovascular complications for molecularly targeted cancer therapeutic agents (kinase inhibitors) by four best single classifiers and the combined classifiers respectively. Drugs existing in the training sets are underlined. (B) Comparison of cardiotoxic profiles from human pluripotent stem cell-derived cardiomyocyte assays (cardiac safety indexes described in Supporting Method S1) and the predicted probabilities from the combined classifiers for 16 kinase inhibitors. A lower cardiac safety index represents higher risk of cardiotoxicity. A predicted probability of more than 0.5 denotes an identified probable cardiovascular complication. A higher probability (e.g., 1.0) shows increased likelihood of cardiotoxicity. Red color indicates higher relative likelihood of drug-induced CV complications, while green or blue color indicates a lower relative likelihood of drug-induced CV complications. Note: HTN: hypertension; HB: heart block; Arrhy: arrhythmia; MI: myocardial infarction; CF: cardiac failure; MD: molecular descriptors; SubFP: Substructure fingerprint; kNN: k-nearest neighbors; SVM: support vector machine; RF: random forest; LR: logistic regression.

Human pluripotent stem cell-derived cardiomyocytes (PSC-CMs) are an effective way to assess drug cardiotoxicity in vitro.⁵⁶ We computationally evaluated CV complications using the combined classifiers for 16 tyrosine kinase inhibitors (TKIs) with known cardiotoxic profiles identified by PSC-CMs assays⁵⁶ and literature evidence. We calculated cardiac safety indexes (CSI) to provide a relative metric for cardiotoxicity by normalizing 4 contractility and viability parameters (cessation of beating, effective concentration, amplitude of effect, and median lethal dose) as described previously⁶ (See the details in Supporting Method S1). Then we compared CSI values with the corresponding true probabilities of having CV complications predicted by the combined classifiers. The combined classifiers successfully predicted the reported CV complications for all reported TKIs except vemurafenib (Figure 5B). Altogether, the combined classifiers show high accuracy for identification of drug-induced CV complications.

Combined classifiers predict oncology molecularly targeted therapeutic agent-induced cardiovascular complications

We next used our model to predict several novel CV complications in cancer molecularly targeted therapeutic agents. Figure 5A shows the predicted CV complications for 26 FDA approved kinase inhibitors covering multiple biological pathways, including epidermal growth factor receptor (EGFR), vascular endothelial growth factor receptor (VEGFR), platelet-derived growth factor receptor (PDGFR), cyclin-dependent kinases (CDKs), BRAF V600E kinases, mechanistic target of rapamycin (mTOR), mitogen-activated protein kinase (MAPK), janus kinase (JAK), Abelson murine leukemia viral oncogene homolog (Abl), and breakpoint cluster region-Abelson leukemia virus (Bcr-Abl). Erlotinib was predicted to have a high probability of inducing arrhythmia and myocardial infarction by the combined classifiers. In support of our model, recent studies reported that erlotinib induced QTc interval in patients⁵⁷ and CV damage in rat model.⁵⁸ In addition, an increasing number of clinical case reports of acute myocardial infarction following treatment by erlotinib in cancer patients also were reported.^59–61 Gefitinib, a multi-targeted tyrosine kinase inhibitor, was approved for treating lung cancer. Our combined classifiers predicted that gefitinib induced all five CV complications (hypertension, heart block, arrhythmia, cardiac failure and myocardial infarction), consistent with recent clinical and preclinical studies.^62,63 Sunitinib, a FDA-approved tyrosine kinase inhibitor for treatment of renal cell carcinoma and imatinib-resistant gastrointestinal stromal tumor in 2006, was predicted to induce all five CV complications by the combined classifiers. Among them, myocardial infarction and heart block have not yet been listed on its FDA label. Interestingly, a multicenter and randomized phase 3 trial reported that 1.33% (5/375) of sunitinib-treated patients had myocardial infarction, the most common reported CV complication (NCT00098657).⁶⁴ A multi-parameter in vitro toxicity screening approach based on a human cardiac cell model also reported that sunitinib significantly altered the cardiac beat pattern and selectively blocked the human Ether-a-go-go Related Gene (hERG) channel,⁶⁵ consistent with our predicted heart block by the combined classifiers. Tandutinib (MLN-518), a novel and selective inhibitor of PDGFR, has no reported cardiotoxic profiles in Drugs@FDA database.⁴⁸ In stark contrast, tandutinib was predicted to have a high likelihood of cardiotoxicity via the combined classifiers. An in vitro experiment reported that tandutinib potentially caused a progressive increase in rats’ ventricular myocyte damage,⁶⁶ confirming our prediction.

Integration of five machine-learning algorithms uncover cardiovascular complications induced across a diversity of anticancer drugs

It is plausible to hypothesize that drugs in the same class may have similar pharmacological characteristics, of which adverse reactions can be inferred.^67,68 Figure 6 presents the relationships among anticancer agents, their target families, and the predicted CV complications. Interestingly, anticancer agents covering Bcr-Abl, DNA topoisomerase, and microtubule inhibitors are clustered together (Figure 6 and Figure S4). Bcr-Abl inhibitors are the first-line treatment for chronic myelogenous leukemia. Recently, cardiovascular safety has been an emerging challenge in patients treated with second-generation Bcr-Abl inhibitors.⁷⁰ For example, dasatinib reportedly may induce potentially fatal pulmonary hypertension,⁷⁰ and ponatinib and nilotinib may induce CV disease.^71,72 Topoisomerases are ubiquitous enzymes involving in regulating the over- or underwinding of DNA strands.⁷³ Anthracyclines, typical topoisomerase II inhibitors, have demonstrated cardiotoxicity.⁷³ Microtubule inhibitors are anti-mitotic agents and known anticancer agents by inhibiting tubulin polymerization.⁷⁴ The CV complications induced by several known drugs in this category have been successfully identified by our combined classifiers. For example, paclitaxel and docetaxel, as members of taxanes, lead to dysfunctional microtubules and release massive histamine, resulting in arrhythmias, myocardial ischemia, and conduction disturbances.⁷⁵ In addition, cabazitaxel, as the fourth taxane, has the potential to induce cardiac-related deaths, including ventricular fibrillation, sudden cardiac death, and cardiac arrest.^76,77 Put together, the combined classifiers successfully identified multiple CV complications across multiple pathways targeted by various anticancer agents. Hence, our combined classifiers offer powerful tools for identifying potential cardiotoxicity across drug families

Figure 6.

Circos plot representing the predicted associations between 63 anticancer drugs and the five types of cardiovascular complications. The predicted associations with positively predicted probabilities higher than 0.8 from the combined classifiers are connected lines. Drugs are grouped based on their target families using annotation from DrugBank database.²³ Kinase inhibitors are highlighted in bold font. Circos plot was drawn using Circos (v0.69).⁶⁹

DISCUSSION

In this study, we developed combined classifiers for the systematic identification of 5 types of drug-induced CV complications. We demonstrated that the combined classifiers outperformed the single classifiers in both cross validation and the external validation sets. Moreover, the newly predicted CV complications by the combined classifiers were validated by clinical study experimental data, cardiomyocyte assays and literature. Building on previous studies,^15,16 this study adds to the field by: (1) collecting comprehensive CV complications for over 1,000 FDA-approved drugs by integrating clinically reported data from 4 databases, which are reliable and sufficient; (2) improving on the efficiency and accuracy of single classifier algorithms by leveraging a combined classifier infrastructure; (3) demonstrating that our combined classifiers outperformed several traditional ensemble approaches: maximum, minimum, and majority vote (Supporting Information, Method S2 and Table S5); (4) Utilizing our combined classifiers on 63 anticancer agents with high validated accuracy (especially for the top-10 predicted agents, Supporting Information, Table S6).

Several shortcomings should be recognized in the presented current study. First, high quality negative samples is quietly crucial for the accuracy of the machine-learning models. In this work, the decoy sets were randomly extracted from the rest drugs of DrugBank database²³ without the known cardiotoxicity, which may bring in potential noise and data bias. Second, data quality of the validation sets may affect the model performance evaluation.⁷⁸ In this study, the external validation sets were derived from Offsides, which contains computationally inferred adverse drug events from FDA's Adverse Event Reporting System (FAERS).²⁰ This may explain the lower AUC range (0.693 to 0.756) of the combined classifiers on the external validation sets compared to high AUC range in the 5-fold cross validations (0.784 to 0.842). Third, current models were built based on integration of molecular descriptors and fingerprints of drugs. Recent studies have shown that integration of biological descriptors from drug-target networks may further improve model performance.⁷⁹ In addition, an ideal drug target would be expressed only in disease tissue and sparsely anywhere else.⁸⁰ In the future, we plan to integrate more relevant biological descriptors and tissue-specific expression profiles of drug targets to further improve the model performance. Furthermore, replacement of the currently used NN algorithm by deep learning algorithms^81,82 could further improve accuracy. Finally, the predicted CV complications by the combined classifiers should be further validated by experimental assays or pharmacoepidemiologic analyses from the real-world data (e.g., electronic medical records or health insurance claims databases)⁸³ in the future.

CONCLUSIONS

In this study, four different classification algorithms were applied to develop 180 single classifiers for evaluation of 5 types of drug-induced cardiovascular complications. The best four single classifiers of each cardiovascular complications were used to together construct the combined classifiers with a neural network algorithm. The combined classifiers outperformed the single classifiers not only in 5-fold cross validation but also external validation. Lastly, the combined classifiers were employed to pinpoint anticancer agents with cardiovascular complications. We report novel drug-induced cardiovascular complications which have been discovered and further validated by reported experimental data from clinical studies, in vitro assays, and literature. In summary, the combined classifiers presented here offer powerful in silico tools for systematic evaluation of drug-induced cardiovascular complications throughout the life cycle of a drug.