Insights into Drug Cardiotoxicity from Biological and Chemical Data: The First Public Classifiers for FDA Drug-Induced Cardiotoxicity Rank

Abstract

Drug-induced cardiotoxicity (DICT) is a major concern in drug development, accounting for 10–14% of postmarket withdrawals. In this study, we explored the capabilities of chemical and biological data to predict cardiotoxicity, using the recently released DICTrank data set from the United States FDA. We found that such data, including protein targets, especially those related to ion channels (e.g., hERG), physicochemical properties (e.g., electrotopological state), and peak concentration in plasma offer strong predictive ability for DICT. Compounds annotated with mechanisms of action such as cyclooxygenase inhibition could distinguish between most-concern and no-concern DICT. Cell Painting features for ER stress discerned most-concern cardiotoxic from nontoxic compounds. Models based on physicochemical properties provided substantial predictive accuracy (AUCPR = 0.93). With the availability of omics data in the future, using biological data promises enhanced predictability and deeper mechanistic insights, paving the way for safer drug development. All models from this study are available at https://broad.io/DICTrank_Predictor.

Introduction

Drug-induced cardiotoxicity (DICT) is a leading cause of drug withdrawals during postmarket surveillance. One study showed that 10% of withdrawals in the last 4 decades were due to cardiovascular safety concerns, including previously successful therapeutics such as rofecoxib, tegaserod, sibutramine, and rosiglitazone.¹ Another study found that cardiotoxicity was the third most common reason for adverse drug reactions and accounted for 14% of withdrawals.² Worryingly, the rate of DICT-related withdrawals may even be increasing, accounting for 17 out of 38 cases among drugs approved between 1994 and 2006.^1,3

DICT is associated with both functional damage such as arrhythmia, which alters mechanical function, and structural damage such as morphological damage in cardiomyocytes; functional damage and structural damage in the heart can be interrelated, where one may precipitate the other.⁴ DICT can be attributed to several underlying mechanisms affecting myocardial functions and viabilities.⁵ Some drugs, such as anthracyclines, inflict direct myocyte injury via reactive oxygen species production and compromising DNA replication.⁶ Electrophysiological disruptions, for example, measured in the hERG potassium channel blockers, can lead to arrhythmias by causing QT interval prolongation.⁷ Cardiac energy demands can be affected by drugs that interfere with mitochondrial functionality.¹ Drugs may also adversely influence vascular supply, inducing ischemic conditions.⁸ Intracellular calcium regulation for cardiomyocyte activity can also disrupt its homeostasis, resulting in contractile and rhythm abnormalities.⁹ Furthermore, alterations in growth factors and cytokine balances can induce cardiac conditions like fibrosis, and immunologic drug reactions can also cause cardiotoxicity.^10,11 Several neurohormonal pathways also offer indirect routes for drug-induced cardiac stress.¹² Notably, a single drug might induce cardiotoxicity via multiple mechanisms, and individual patients’ responses (which can often manifest as side effects) can be modulated by genetics, concurrent health conditions, and other medications.¹³

To move beyond a limited focus on specific adverse reactions or related proxy assays for cardiotoxicity, the FDA recently released the drug-induced cardiotoxicity rank (DICTrank) that categorizes drugs based on their risk of causing cardiotoxicity.¹⁴ Similar to the DILIrank data for liver injury,^15,16 the DICTrank system uses FDA drug labeling to comprehensively categorize 1318 human drugs into four DICT Concern categories based on their potential risk for cardiotoxicity: (1) most-DICT-concern, (2) less-DICT-concern, (3) no-DICT-concern, and (4) ambiguous-DICT-concern. The DICTrank data set was generated with an expert review from the FDA, keyword searches, and manual curation of FDA labeling documents as well as data from clinical trials, postmarketing, and literature surveys.

Predictive models for DICT could save considerable time, resources, and human suffering, with the ultimate goal of preventing adverse events in clinical trials and the postmarket stage. However, predicting any in vivo effect is not a trivial classification task, and most predictive models are built on proxy end points (which are often reduced to binary end points) without taking into account in vivo parameters such as pharmacokinetic parameters.^17,18 While no models for DICTrank have been publicly available yet to the best of our knowledge, various studies have predicted proxy in vitro assays or side effect data from side effects resource (SIDER), some of which are related to cardiotoxicity.¹⁹ Studies focusing on side effects and proxy targets (such as hERG) are reasonable given that compounds that have cardiac-related indications are more likely to show related side effects as well or activity on ion channels.⁷

Previously it was shown that adverse events data and biological data can be used for identifying mechanism hypotheses leading to cardiotoxicity.²⁰ Wang et al. used LINCS L1000 gene expression features to predict a wide range of drug-induced adverse events from the SIDER data set.²¹ Particularly for acute myocardial infarction, the models developed achieved an AUC-ROC of 0.84 when using chemical structural data and 0.76 when using Gene Ontology annotations (compared to 0.5 for random models). Galeano et al. used a matrix decomposition algorithm to predict side effect frequencies for drugs and provide biologically interpretable insights.²² MoleculeNet predictions for SIDER side effects, trained on chemical structure data, range from 0.65 to 0.70 AUC-ROC when using a bypass network, a modified version of a multitask network.²³

Most predictive models mentioned above were built on chemical structure data as input features. Although certain structural motifs or patterns in a molecule can be indicative of toxic properties and analyzing the chemical structure can flag potential cardiotoxic compounds, such models are often limited in their applicability domain; that is, their accuracy is limited to the chemical space of the training data, and they fail to generalize to markedly different chemical structures. Novel chemical and biological data have been previously used to evaluate side effects in general from the SIDER data set.²⁴ Previous studies have shown that Random Forest models trained on a combination of biological, chemical, and phenotypic features achieved an AUCPR of 0.76 for cardiac disorders.²⁵

With the availability of the new DICTrank data set, we used a novel multifaceted approach using both chemical and biological data (that considers a multitude of possible mechanisms that can lead to DICT) intending to better understand and make mechanistic insights into a drug’s cardiac safety profile. We evaluated a wide range of chemical and biological information, as shown in Figure 1, to determine which feature space is most predictive of DICTrank and evaluated these feature spaces to build the first predictive models of DICTrank using machine learning. Biological data sources included Cell Painting, gene expression, and Gene Ontology,²⁶⁻³⁰ as well as bioactivity, and annotated mechanisms of action (MOA)³¹ and pharmacokinetic parameters for the peak unbound and total concentration of a drug molecule in plasma;³² these offer an alternate feature space to chemical space.³³ We aimed to glean insights into which chemical and biological data best capture the carefully curated manual annotations in the DICTrank data. Incorporating data from all these sources as feature spaces for predictive models allows for a multifaceted assessment of a drug’s potential cardiotoxicity, potentially enhancing the model’s accuracy and reliability. Overall, the use of biological data sources along with chemical data improved the detection and offered mechanistic insights into the cardiotoxicity of compounds. The models based on chemical structures and physicochemical characteristics are readily accessible for direct use on https://broad.io/DICTrank_Predictor (the other models are not implemted on the server due to the lack of public data for other feature types). All code and data for all models can be found on GitHub (https://github.com/srijitseal/DICTrank) for local implementation with further details on https://broad.io/DICTrank_Predictor.

Figure 1.

Chemical and biological data sources were used in this study to perform exploratory data analysis on DICTrank and training predictive models.

Methods

Data Sources

We obtained the DICTrank data set, as released by Qu et al. which includes comprehensive DICTConcern categories for a diverse set of over 1300 drugs.¹⁴ The SIDER database, a pharmacovigilance resource, contained associations for drugs with side effects.^23,34 We used data from cardiac disorders from the SIDER data set to compare concordance with DICTrank and enrich the data set as described later. To gain insights into the MOA of various drugs, we assessed relevant data from the Drug Repurposing Hub,³⁵ which contained information on 6777 drugs for 1130 MOAs and 2183 known targets. To explore the potential targets of drugs, we incorporated the CELLSCAPE target predictions on inhibition/antagonism for 2094 targets at four concentrations (0.1, 1, 10, and 100 μM).³⁶ We used morphological profiles from the Cell Painting assay²⁶ which considers the impact of drugs on cellular morphology and function. This data set contained a range of ca. 1700 morphological features for over 15,000 compound perturbations. We obtained gene expression data from LINCS L1000 data which contains over 19,000 drugs as described in Wang et al.²¹ This study utilized gene expression features derived from LINCS L1000²⁷ transcriptomic data, capturing changes in 978 landmark genes across diverse human cell lines in response to compound perturbations. Gene Ontology-transformed expression features,²⁸ which encode biological processes involved with gene expressions affected by the compound perturbations, were extracted from a data set containing 4438 annotated features linked to these compounds in the study.²¹ The analysis by Wang et al. prioritized the strongest signatures across cell line, concentration, and time point for each compound using characteristic direction and evaluated the enrichment across various gene set libraries via principal angle enrichment analysis.³⁷ Finally, we used pharmacokinetic data, specifically the maximum unbound and total concentrations (Cmax) of 758 drugs in the bloodstream, as compiled by Smith et al.³² This data set contains Cmax (unbound) for 534 compounds and Cmax (total) for 749 compounds.

Standardization of the SMILES

For each data set, we standardized chemical SMILES iteratively using RDKit³⁸ and MolVS³⁹ functionalities. This includes steps for the InChI transformation, molecular cleanup, charge neutralization, tautomer normalization, and final standardization. We carried out up to five iterations of the standardization until a standardized SMILES was finalized; otherwise, we chose the most common SMILES from the counter. Finally, the molecule was protonated at pH 7.4 using DimorphiteDL to reflect its likely state at physiological pH.⁴⁰ Hence, we obtained standardized SMILES and standardized InChI.

Preprocessing Data

For the DICTrank data set, we binarized the data set considering DICT no-concern as 0 and less- and most-concern as one as DICTrank labels for machine learning classifiers. We removed compounds that were ambiguous and treated a compound as toxic if there was at least one record of toxicity among duplicates. For the SIDER data set, we removed duplicate standardized smiles and, similar to the above, labeled a compound as toxic if there was at least one evidence of toxicity among the duplicates. Labels from both SIDER and DICTrank are described in Table 1.

Table 1. Distribution of Compound Toxicity Labels Related to Cardiotoxicity/Cardiac Disorders for All Unique Compounds from Each of the Datasets Used in This Study.

data set	label	number of toxic compounds	number of non-toxic compounds	description
SIDER	cardiac disorders (binary)	829	360 (absence of evidence)	recorded adverse drug reactions from marketed medicines
DICTrank	DICT concern category (categorical)	most: 299, less: 443	no: 278 (evidence of absence)	ranking system from DICTrank that categorizes drugs according to risk for cardiotoxicity
	DICTrank label (binary)	742	278 (evidence of absence)	binarized labels obtained from DICT concern categories used in this study

For the Cell Painting, gene expression, and Gene Ontology data sets, we use median cell profiles over standardized SMILES obtaining two data sets: 1783 Cell Painting features for 15,406 compounds, and gene expression features for 978 landmark genes and 4428 Gene Ontology annotations for 9132 compounds. For the MOA data set, we used one hot encoding of given annotations for compounds, which effectively gives us data for evidence of the presence of MOA/known targets and the absence of evidence. We used a variance threshold of 0.001 to identify and remove low-variance features, reducing the dimensionality to 264 MOA and 551 known target features with significant variability. All data sets are released publicly at figshare (10.6084/m9.figshare.24312274) and https://broad.io/DICTrank_Predictor.

Analyzing Chemical Space Overlap Between SIDER and DICTrank

We used standardized InChI to calculate the overlap between SIDER and DICTrank data sets. We assessed the physicochemical space using a t-distributed stochastic neighbor embedding (TSNE; as implemented in scikit-learn⁴¹) for six physicochemical properties, namely, molecular weight, topological polar surface area, number of rotatable bonds, hydrogen bond donors and acceptor, and the computed logarithm of the partition coefficient. To analyze the chemical space, we used a principal component analysis (PCA) of the FragFP fingerprints from DataWarrior,⁴² which in our experience works better with a higher explained variance in the plot of the PCA compared to Morgan fingerprints.

Structural and Physicochemical Features

For structural features, we used 2048 bit Morgan Fingerprints as implemented in RDKIT.³⁸ For chemical compounds, we computed 1579 descriptors using Mordred.⁴³ These physicochemical descriptors are derived from 2D representations of compounds; that is, we did not consider 3D descriptors. We removed the descriptors that failed to compute and finally obtained 1038 2-D physicochemical descriptors, and these were used for the machine learning models. For the analysis of feature distributions, we used the full set of 208 RDKit descriptors, (which are better interpretable compared to Mordred descriptors) as defined in the descriptors module.³⁸

Predicted Targets from CELLSCAPE

To derive predicted molecular targets for compounds, we utilized the commercially available CELLSCAPE target prediction package (Ignota Labs, 2023).³⁶ This package applies models trained on a mixture of publicly available and proprietary bioactivity data (primarily inhibitory/antagonistic mechanisms) at 0.1, 1, 10, and 100 μM with chemical structural features to output a probability score (between 0 and 1) of predicted activity for 2094 distinct human targets. Although not used in this study, publicly available target prediction alternatives are also available such as PIDGINv4^44,45 and SwissTargetPred.⁴⁶ We provide the computed CELLSCAPE features for compounds in the DICTrank data set publicly via figshare (10.6084/m9.figshare.24312274) and https://broad.io/DICTrank_Predictor.

Substructure Analysis and Retrospective Analysis of DrugBank

For substructure analysis, we used SARpy⁴⁷ on the DICTrank data set, in a method similar to the one applied by Hemmrich et al.⁴⁸ SARpy uses a recursive algorithm for fragmentation. We used two distinct settings for analysis: (1) using both toxic and nontoxic compounds and (2) using only toxic compounds to yield the desired substructures. For both settings, we confined the fragment size within a range of two-18 atoms, with a minimum occurrence of five times. Furthermore, the positive predictive value (PPV) was adjusted to minimize false negatives. We combined structural alerts from both settings and quantified the frequency of these fragments within the entirety of the DICTrank data set. We eliminated fragments with a PPV of below 0.5. We then manually assessed the remaining fragments, for example, removing those having four or fewer atoms, removing substructures like benzene to obtain 58 structural alerts.

We analyzed all compounds in DrugBank⁴⁹ for the presence of structural alerts from the above to evaluate the risk of the chemical space of drugs for cardiotoxicity. We only used the compounds that did not overlap with the DICTrank data set for this retrospective analysis to avoid information leaks. We annotated these compounds with labels for cardiac disorders from SIDER and disease area labels from the MOA data set. We then checked for the presence of structural alerts among the subset of compounds that are currently approved, investigational, experimental, or withdrawn drugs.

Analysis of Chemical and Biological Data for Differences in Feature Distribution for DICTrank Compounds

We detected features that are predictive of highly cardiotoxic compounds. In order to do this, we detected features for each chemical and biological data set that had a significant difference in the distribution for the DICT concern categories. For categorical features (SIDER, MOA annotations, and some of the 208 RDKit descriptors), we employed the chi-squared test (as implemented in SciPy⁵⁰) to evaluate the association between categorical variables. We used a contingency table, delineating the frequency distribution for each combination of category values. The chi-squared test yielded a statistical value alongside a corresponding p-value. For continuous features (as in the Cell Painting, Gene Expression, Gene Ontology data sets, and some of the 208 RDKit descriptors), we chose the Kruskal–Wallis test (as implemented in SciPy⁵⁰) for evaluating the DICT-Concern labels since it is suited for comparisons involving three or more independent groups. Conversely, when comparing two classes, pairwise, the Mann–Whitney U test (as implemented in SciPy⁵⁰) was used which is adept at discerning differences in distributions between two independent samples. Both tests yield a statistic value alongside its corresponding p-value. For both total unbound/plasma concentrations, as in the Cmax data set, we used the Mann–Whitney U test to compare the distribution of Cmax among each DICT concern class and the DICTrank label.

Enriching DICTrank Compounds with SIDER Compounds

We next determined the overlap of compounds (and the concordance in their labels) in the DICTrank data set with the compounds in SIDER labeled with “cardiac disorders” using the standardized InChI yielding 776 compounds in common. We next enriched DICTrank with SIDER giving a preference to the DICTrank label in the case of a conflict. In this manner, we obtained three data sets besides the DICTrank data set with the distribution of toxic/nontoxic compounds given in Supporting Information Table S1. These are (1) DICTrank, (2) DICTrank enriched with cardiotoxic compounds from SIDER, (3) DICTrank enriched with noncardiotoxic compounds from SIDER, and (4) DICTrank enriched with all compounds from SIDER.

Training Predictive Models for DICTrank

We trained 11 Random Forest models, each using the following features (as listed in Table 2): (1) Structural fingerprints, (2) Mordred descriptors, (3) MOA labels, (4) MOA labels along with total Cmax, (5) MOA labels along with unbound Cmax, (6) CELLSCAPE predicted protein targets, (7) CELLSCAPE predicted protein targets along with total Cmax, (8) CELLSCAPE predicted protein targets along with unbound Cmax, (9) Cell Painting features, (10) Gene Expression features, and (11) Gene Ontology features.

Table 2. Description of Various Feature Spaces Used in This Study.

feature space	dimensions after feature selection (where applicable)	description	signal expected	source
chemical structure	2048 bit vector	ECFP4 (Morgan) fingerprints representing chemical structures	distinctive patterns of chemical bonding and arrangement	(54)
physicochemical properties (Mordred descriptors)	1038 2-D descriptors	properties such as lipophilicity, solubility, molecular weight, ionizing potential, and so forth	properties that are associated with negative impacts on ion channels in the heart	(43)
MOA data set	264 binary encoded MOAs +551 known targets	annotations for mechanism of action and known targets based on knowledge	mechanism of action for drugs that inhibit certain ion channels	(31)
CELLSCAPE target prediction data set	1893 predictions for 817 unique targets and concentration combinations (0.1, 1, 10, and 100 μM)	predicted protein target for inhibition/antagonism; does not consider the functionality; prediction is based on chemical structure; updated algorithm from PIDGINv445	understanding how a drug interacts with various biological targets (not just its primary target) can provide insights into potential off-target effects	(36,45)
Cell Painting	1783 features	morphological changes in U2OS cells by a chemical perturbation, using a 5-channel fluorescence microscopy assay	morphological changes in cells that reflect basic biological processes	(26,55)
gene expression	978 features	transcriptomic changes in response to chemicals using the L1000 assay	upregulation or downregulation of genes associated with cardiac stress, apoptosis in cardiac cells, or ion channel function	(21,27)
Gene Ontology	4438 annotations	Gene Ontology manual annotations based on collective knowledge	understanding the biological processes, cellular components, and molecular functions affected, e.g., related to cardiac function, cardiac muscle tissue development, or ion homeostasis	(21,28)
Cmax	2 features	the maximum total and unbound concentration of a drug in plasma	High Cmax would indicate a high risk of cardiotoxicity	(32)

The training data available for these models depended on the number of compounds for which data was available and varied, as given in Supporting Information Table S1. As the external test set, we aimed to keep that fixed for a fair evaluation depending on available data, as shown in Supporting Information Table S2. For models not using Cmax data (where overlaps were larger and hence more data was available), we randomly selected 90 compounds (8.8% of the data set, 65 cardiotoxic and 21 nontoxic) for which all annotations of feature spaces were available (as described in Supporting Information Table S1). These 90 compounds struck a similar balance of DICT concern categories (most: 39, less: 26, and no: 25) as the original DICTrank data set. For models using total Cmax data, we used the same external test set comprising 90 compounds since total Cmax data were available for these compounds. However, for models using unbound Cmax data (which had smaller overlaps compared to the above), we used a subset of 78 compounds (57 cardiotoxic and 21 nontoxic) as the external test set as shown in Supporting Information Table S2.

Among the models that relied on omics data (Cell Painting, Gene Expression, and Gene Ontology), we checked for each training compound whether a profile (feature set) was available. If there was no profile available in the respective data sets, we calculated the median profile of all compounds in the original data set using a v-NN approach, which is different from a fixed k-nn approach; v-nn selects the neighbors based on a condition for each query compound. We used the median profile on the v training compounds that had a Tanimoto similarity greater than 0.70. We ignored any similar compound that appeared in the external test set to avoid information leaks. Subsequently, we further discarded any compounds for which no feature profile was found directly or using the above v-nn approach. Thus, while the test sets for the DICTrank and DICTrank enriched data sets are the same, it is important to note that the training data for them vary for the models (as described in Supporting Information Table S1) since we dropped compounds where no feature data could be found or matched.

For each of the 11 models, we used a Random Forest classifier, with hyperparameter optimization on the training data using a halving random search with a 5-fold stratified cross-validation with a random oversampling to account for class imbalance (as implemented in scikit-learn⁴¹). We used the best hyperparameter-optimized estimator and obtained out-of-fold predictions with a 5-fold stratified cross-validation. We used the out-of-fold predictions and the true labels to optimize the decision threshold for binary classification using the J statistic, calculated as the difference between the true positive rate and the false positive rate. This determines the threshold from ROC curve values, where the J statistic is maximized. The model was finally refitted on the entire training data set, and we used the optimized threshold to make final predictions based on the predicted probabilities of the external test set.

We trained two ensemble models to combine the models from the 11 feature spaces above. These were based on soft voting, which considered the mean of the scaled predicted probabilities of each mode (scaled according to the best threshold of each model). The first model considered only the six best-performing models (structural, physicochemical, MOA, CELLSCAPE, MOA with Cmax total, and CELLSCAPE with Cmax total) in the cross-validation (AUC > 0.65). The second ensemble model considers all 11 models and thus is evaluated on the reduced external test set of 78 compounds where data from all feature spaces were available.

Model Evaluation and Applicability Domain

We evaluated the classifiers using the balanced accuracy, sensitivity (or recall), specificity, F1 score, Matthews Correlation Coefficient (MCC), AUC-ROC, and the AUCPR, or precision–recall curve, which focuses on the positive class.

To evaluate the applicability domain of the models, for each compound in the external test set, we calculated the Tanimoto similarity of the nearest neighbor of the same DICTrank label (toxic/nontoxic) in the training data set. We grouped compounds in five equal bins from Tanimoto similarity of 0.0 to 1.0 and evaluated the balanced accuracy and AUCPR in this range for the models used in this study.

Statistics and Reproducibility

We have released the data sets used in this study which are publicly available at 10.6084/m9.figshare.24312274. We released the Python code for the models which are publicly available at https://github.com/srijitseal/DICTrank and further details are available on https://broad.io/DICTrank_Predictor.

Results and Discussion

In this study, we used various biological and chemical data sets to discern among the DICT concern categories, deriving insights into the carefully annotated FDA DICTrank data set. We also trained predictive models using these feature spaces. In particular, we used the Cell Painting data from Bray et al., which captures a wide array of cellular phenotypes after perturbation, e.g., drug treatment, and has been shown to have a signal for various in vitro toxicity.^26,51 We also used experimental (from the Repurposing Hub³¹) and predicted bioactivity data derived from models trained on a mixture of publicly available and proprietary data sets (Ignota Labs CELLSCAPE³⁶), mostly relating to inhibitory/antagonist mechanisms. For structure-derived feature spaces, we used Morgan fingerprints derived from chemical structures as well as physicochemical Mordred descriptors which are often related to pharmacokinetic properties (such as logD, molecular weight, solubility, permeability, and so forth) and implicitly encode the bias between bioactivity classes and chemical structures.⁵² Finally, we looked at pharmacokinetic parameters for the peak unbound and total concentration of a drug molecule in plasma (Cmax).⁵³ We organized and standardized various chemical and biological data, as shown in Table 2, to analyze their ability to predict DICTrank labels.

DICTrank Labels are Highly Concordant with SIDER Labels

Among the 776 compounds present in both DICTrank and SIDER cardiac disorders data sets (Figure 2a), we found an 87.24% concordance rate in the annotations (labels) between the two data sets (Supporting Information Table S3; SIDER labels have an F1 score of 0.91 when compared against DICTrank labels). This suggests that SIDER labels, which ascertain cardiac disorder events reported as associated with each drug and are often dependent on aggregated dispersed public information and package inserts, agree with DICTrank labels, which ascertain if a compound is classified as cardiotoxic by the FDA.

Figure 2.

Comparison of the SIDER data set with DICTrank: (a) the overlap and concordance (percentage of the total compounds with the same annotation, among the 776 compounds present in both sets) of DICTrank labels with SIDER cardiac disorder labels, (b) the overlay of SIDER and DICTrank chemical space in a TSNE of physicochemical properties, and (c) the postivie predictive value (PPV) of other side effects in SIDER for DICTrank labels (toxic/nontoxic).

The physicochemical space of SIDER and DICTrank generally overlap (Figure 2b), defined as a TSNE space for six physicochemical properties, namely, molecular weight, topological polar surface area, number of rotatable bonds, hydrogen bond donors and acceptors, and the computed logarithm of the partition coefficient. Still, compounds exclusively available in the SIDER data set could help enrich nontoxic compounds in areas of the chemical space where DICTrank only covers toxic compounds. We see a similar trend for a chemical space defined in fragment fingerprints space from DataWarrior⁴² (Supporting Information Figure S1). Therefore, we chose to assess whether adding SIDER compounds to DICTrank compounds improved the predictive ability. Interestingly, other categories of SIDER adverse effects were highly correlated to DICTrank (Figure 2c); the interrelationships of vascular disorders and nervous system disorders are well-known.^5,56 Overall, drug adverse events, as recorded in SIDER, have a high concordance with DICTrank labels from the FDA and there is a strong rationale to rely on both resources.

Maximum Total and Unbound Compound Concentration in Plasma Predict Cardiotoxicity

We next determined if a high Cmax indicated compounds more likely to be cardiotoxic as seen in the case of doxorubicin where cardiotoxicity was found to be Cmax dependent.⁵⁷ As a single parameter, Cmax was not sufficiently discerning to differentiate between compounds that fall under the ‘most-concern’ and ‘less-concern’ categories as per the DICT concern classification (Figure 3). However, for both peak total plasma levels and peak unbound (active) plasma levels’ Cmax, the median distributions were significantly distinguishable between cardiotoxic and nontoxic compounds (Figure 3) suggesting that Cmax can be a useful parameter in determining cardiotoxicity.

Figure 3.

Distribution of (a) peak total concentration in plasma and (b) peak unbound (active) concentration in plasma for each drug in the DICTrank data set across the three DICT concern categories.

Cyclooxygenase Inhibition is Predictive of Cardiotoxicity Concern

Turning to manual annotations of compound MOA and/or targets, we found that cyclooxygenase inhibitors⁵⁸ along with tyrosine kinase receptor inhibitors were the most significant annotations differentiating the various DICT concern categories (Table 3); this is plausible given cyclooxygenase inhibition, besides reducing inflammation, can also lead to increased blood pressure⁵⁹ while tyrosine kinase receptor inhibition can induce endoplasmic reticulum stress and inflammation in cardiomyocytes.⁶⁰ In agreement with this, known targets of prostaglandin endoperoxide synthases (PTGS1 and PTGS2 genes, which encode cyclooxygenases COX-1 and COX-2) could significantly distinguish among most-, less-, and no-DICT concern categories (Table 3).

Table 3. Features from Chemical and Biological Data Sources That Have a Particularly High Significance in the Difference of Distributions for the Three DICT Concern Categories (Most, less, No-Concern).

feature	P-value (statistical test)	test applied	feature space	description/biological interpretation	source
Cmax (total)	2.97 × 10–4	Mann–Whitney Wilcoxon test two-sided with Bonferroni correction (most vs no)	pharmacokinetic parameters32	the peak total concentration of a drug in plasma indicates how much of the drug reaches the bloodstream	(57)
Cmax (unbound)	7.58 × 10–4			the peak unbound (active) concentration of a drug in plasma, indicates how much of the drug is available for interaction with its target
cyclooxygenase inhibitor	2.98 × 10–6	chi-squared test	MOA (Drug Repurposing Hub31)	inhibits cyclooxygenase enzymes, often leading to reduced inflammation but also increased blood pressure	(59)
tyrosine kinase receptor inhibitor	3.24 × 10–5			inhibition of tyrosine kinase receptors can affect cell growth and proliferation and can also induce endoplasmic reticulum stress, hypertension, heart failure, myocardial infarction, and cardiac arrhythmias	(60)
PDGFR tyrosine kinase receptor inhibitor	3.24 × 10–5
PTGS2	9.67 × 10–7	chi-squared test	known targets (Drug Repurposing Hub31)	prostaglandin-endoperoxide synthase 2 (an enzyme) also known as COX-2	(59)
PTGS1	9.67 × 10–7			prostaglandin-endoperoxide synthase 1 (an enzyme) also known as COX-1
HTR1D	1.27 × 10–5			5-hydroxytryptamine receptor 1D, a serotonin receptor subtype. Previous enrichment analysis for methylation differences identified HTR1D among the genes with decreased promoter methylation, suggesting its involvement with serotonin receptors, which influences human cardiac function	(65,66)
Q12809 at 100 μM (KCNH2) (hERG)	1.21 × 10–18	Kruskal–Wallis test	CELLSCAPE predicted target36	the hERG gene encodes Kv11.1 channels crucial for heart function, linked to genetic and drug-induced arrhythmias	(61)
cytoplasm granularity 2 ER	4.64 × 10–4	chi-squared test	Cell Painting26	fine-grained smoothness of the ER staining. Disruptions in ER function can lead to ER stress, which is associated with various cardiovascular diseases	(67)
cells granularity 2 ER	1.05 × 10–3			contains information about the size, shape, number, or texture of nucleoli within the nucleus. This could encode signals for cellular stress
nuclei texture contrast RNA 3 0	1.19 × 10–3			(68)
222,103 at (ATF1)	1.01 × 10–4		gene expression21,27	ATF1 is essential for cardiomyocyte function	(69)
201,080 at (PIP4K2B)	6.29 × 10–4			the role of PIP4k2 in cardiac disorders remains uncertain. PIP4Ks regulate insulin production and immune response, with PIP4k2c impacting TGFβ1 signaling which is vital in heart disease and other fibrotic conditions	(70)
209,092 s at (GLOD4)	8.14 × 10–4			the physiological function of GLOD4 remains largely unexplored. The glyoxalase gene family, comprising six enzymes with roles in metabolism and disease prevention, is crucial for detoxifying reactive dicarbonyls and maintaining cellular homeostasis	(71)
transport vesicle (GO:0030,133)	5.53 × 10–4		Gene Ontology21,28	extracellular vesicles play important roles in cardiovascular communication, transporting bioactive molecules that both maintain heart health and contribute to cardiovascular diseases	(72)
negative regulation of potassium ion transmembrane transport (GO:1901380)	1.04 × 10–3			cardiac K+ channels play a crucial role in cardiac repolarization and their dysfunction can lead to arrhythmias	(73)
response to methylmercury (GO:0051,597)	1.56 × 10–3			exposure to mercury (Hg) is considered to be an increased risk of developing cardiovascular system	(74)
VSA EState6	6.67 × 10–9		physicochemical Descriptors from RDKit38	VSA estate descriptor 6 (6.00 < = x < 6.07) related to molecular surface area and electronic state	(75)
Qed	1.20 × 10–7			quantitative estimate of drug-likeness, a measure indicating how drug-like a molecule is	(76)
numHAcceptors	1.26 × 10–7			number of hydrogen bond acceptors in the molecule	(77)

CELLSCAPE-Predicted Protein Targets Such as hERG are Predictive of Cardiotoxicity

Among CELLSCAPE-predicted protein targets, the predicted activity of compounds against KCNH2 best differentiates among the three DICT concern categories. The KCNH2 gene, also known as the human ether-à-go-go-related gene (hERG), is well-known for its significance in the cardiac electrical cycle and hERG inhibition can lead to cardiac arrhythmias.⁶¹ We also found that the top three features to distinguish the two DICTrank labels (cardiotoxic versus nontoxic) were α-l-fucosidase I, P-selectin, and carbonic anhydrase IX. The activity of plasma α-l-fucosidase has been pinpointed as a potential biomarker for cardiac hypertrophy and complements the currently used marker, atrial natriuretic peptide.⁶² Elevated amounts of soluble P-selectin in the blood are evident in various heart-related conditions, like coronary artery disease, hypertension, and atrial fibrillation.⁶³ Carbonic anhydrase IX plays a role in managing the intracellular pH in the heart muscle, which is vital for the heart’s functionality.⁶⁴

Hypothesis-free Omics Data for Cardiotoxicity are Related to MOA

Omics data sources such as Cell Painting (imaging), gene expression, and Gene Ontology features cover a broad swath of biology, not specifically targeted to cardiac function. For Cell Painting, the fine-grained smoothness of the ER in the cytoplasm and RNA in the nucleus were the top features that differed significantly among toxicity classes. This is plausible given disruptions in ER function can lead to ER stress, which is associated with various cardiovascular diseases.⁶⁷ For the gene expression feature space, activating transcription factor 1 (ATF1), which is essential for cardiomyocyte function, was the top feature. The other two gene expression features that could distinguish DICT concern categories were phosphatidylinositol-5-phosphate 4-kinase type 2 beta (PIP4K2B) and glyoxalase domain containing 4 (GLOD4); both have indirect links to heart disease and other fibrotic conditions (Table 3). Among Gene Ontology annotations, we found that biological processes related to vesicle transport, potassium ion transmembrane transport, and response to methylmercury could best differentiate signals for the concern categories. This is plausible given that cardiomyocytes rely on vesicular transport for various functions, including the delivery of membrane proteins and lipids. The potassium ion channels play crucial roles in cardiac cell electrical activity and dysregulation can lead to arrhythmias and other heart complications.^72,73 Exposure to mercury (Hg) is also considered a risk for ischemic heart disease.⁷⁴

Physicochemical Properties can Differ Among DICT Concern Categories

Among the various molecular descriptors evaluated in our study, VSA_EState6 could significantly distinguish among the DICT-concern categories. This electrotopological state descriptor aggregates the differences in electronegativity between an atom and its neighboring atoms in a molecule, adjusted by their relative distances while focusing on atoms with specific van der Waals surface area.⁷⁵ This suggests that specific electronic and spatial properties are captured by the VSA_EState6 descriptor, although it is difficult to interpret directly. The second predictive feature, Qed, captures a quantitative estimation of the drug-likeness score that encapsulates the underlying distribution data for a range of drug properties.⁷⁶ The third predictive feature, NumHAcceptors refers to the number of hydrogen bond acceptors in the compound. Munawar et al. showed that the most potent hERG inhibitors typically possess two aromatic groups, one hydrophobic group, and one hydrogen bond acceptor, at specific relative distances from each other.⁷⁷

Structural Alerts from DICTrank can Detect Compounds Causing Cardiac Disorders from a Retrospective Analysis of DrugBank

We determined 59 structural alerts that distinguish cardiotoxic and nontoxic compounds in the DICTrank data set (Figure 4). Two structural alerts had a high PPV for the DICT most-concern category, including one with aromatic rings. Aromatic rings can lead to π-stacking or hydrophobic interactions with aromatic rings of amino acids within the hERG channel cavity increasing the potential for blocking and subsequent cardiotoxic effects.⁷⁸ Six structural alerts distinguished toxic versus nontoxic compounds with a PPV of one and more than ten occurrences in the data set (the PPV was used to filter the structural alerts, hence is not an evaluation metric here). Structural alerts with tertiary amines were consistently protonated at physiological pH in the DICTrank data set, suggesting their importance in biological activity and hERG channel binding.^79,80 It is also known that compounds with secondary amine (more hydrogen bond donor number) are likely to be less potent hERG inhibitors compared to tertiary amine (less hydrogen bond donor number).⁸⁰

Figure 4.

Structural alerts for (top) the most-concern DICT category and (bottom) DICTrank labels with more than ten occurrences and a PPV > 0.6 for compounds in the DICTrank data set.

We next analyzed compounds in DrugBank⁴⁹ for the presence of at least one of the two structural alerts above for the most-concern category. We annotated these hits with heart-related side effects from SIDER³⁴ and their current status (approved, withdrawn, and so forth) as indicated in DrugBank. We found six approved drugs, some experimental and some investigation, with reported cardiac disorders from SIDER (Table 4). These compounds spanned different classes of compounds, with the presence of a tertiary amine that remains protonated or aminopyridine rings as defined by the structural alerts. We found evidence in the literature for the risk of cardiovascular disorders for three of the six compounds, namely, ipratropium, tiotropium, and mivacurium.⁸¹⁻⁸³ Overall, our analysis shows that the DICTrank data set is a rich source of cardiotoxicity-causing compounds, with the potential to be used to build pharmacophore models and evaluate compounds with reported adverse events for their potential mechanisms of toxicity. Overall, we could detect multiple approved drugs that match the structural alerts for both the DICT most-concern category (as shown in Table 4) and DICTrank labels for cardiotoxicity (further details in Supporting Information Figure S2).

Table 4. Six Hits from SIDER with Structural Alerts for the DICT Most-Concern Category.

Predictive Models for DICTrank Labels

Finally, given the promising signals seen in each data type, as described above, we evaluated whether cardiotoxicity might be predicted using the data sources currently publicly available. Several data sources contained sufficient information to successfully train models to predict DICTrank labels (Table 2). We trained 11 models on four types of training data: the DICTrank compounds alone and DICTrank compounds enriched with cardiotoxic/nontoxic/all compounds in the SIDER data set (as shown in Supporting Information Table S1). A direct comparison of the predictive value of data sources is not possible due to the incomplete intersection of compounds with available data of each type. Still, we fixed the held-out test set of compounds to be those where data were available for all feature spaces such that only the training set of compounds varied among data sources. We trained two ensemble models, one using six models (structural, physicochemical, MOA, CELLSCAPE, MOA with Cmax total, and CELLSCAPE with Cmax total) that performed relatively well on the internal cross-validation (evaluation metrics from cross-validation for all feature space and data set combinations, are given in Supporting Information Table S4). This ensemble model was evaluated on an external test set of 90 compounds. Another ensemble model was built on all 11 models, which required testing on a smaller held-out test set due to the limited overlap of data. Evaluation metrics for all models are given in⁸⁴ Supporting Information Table S5.

Looking at each data source independently, we found that models using Mordred descriptors evaluated on the 90 compounds held-out test set (AUC: 0.84, AUCPR: 0.93; random AUC: 0.50, AUCPR: 0.72) performed better compared to models trained on predicted protein targets (AUC: 0.77, AUCPR: 0.89) and MOA annotations with Cmax (total) (AUC: 0.77, AUCPR: 0.90) (Figure 5a,b). In fact, models using Mordred descriptors were as good as the ensemble of six selected models (AUC: 0.83, AUCPR: 0.92; random AUCPR: 0.72) also evaluated on the 90 compounds held-out test set (Supporting Information Figure S3). Further, models across most data sets performed with high AUCPR and F1 scores, with top-performing models using Mordred descriptors (AUCPR: 0.93; random AUCPR: 0.72) and ensemble models (AUCPR: 0.93 for both ensemble models) when using the DICTrank data set directly (Supporting Information Figure S3a and b). Exceptions were models using the broad-based omics data, Cell Painting, Gene Expression, and Gene Ontology, where the performance was relatively poor and similar to random predictions according to the distribution of respective training data. This lack of predictive power may be inherent to the data sources but could also be due to the highly unbalanced and sparse training data available for these data sources (see Supporting Information Table S2). When comparing the models evaluated with the smaller test set (Supporting Information Figure S3), we found that models trained on the DICTrank data set enriched with all SIDER compounds and using MOA data with Cmax (unbound) (AUCPR: 0.93, random AUCPR: 0.73) performed equally as the ensemble models that used predictions from all 11 models trained on just the DICTrank data set (AUCPR: 0.93; random AUCPR: 0.73). Overall, a strong detection of cardiotoxicity was seen equally among the ensemble model and models using physicochemical descriptors.

Figure 5.

Comparison of evaluation metrics models built in this study with an external test set of 90 compounds evaluated by the (a) AUC-ROC and (b) AUCPR and (c) performance of each model across compounds that are similar (dissimilar) to the training data. The ensemble mode in (c) is based on the models built on six data sources listed in the main text, and evaluation is for 90 held-out compounds; results for the ensemble using 11 data sources and 78 held-out compounds are in Supporting Information Figure S3.

We next analyzed the applicability domain of these models based on evaluating the quality of prediction for groups of compounds that are structurally dissimilar to the training data. We found that ensemble models and models using MOA annotations perform consistently well across the similarity range (Figure 5c). Models using Mordred descriptors, on the other hand, perform with slightly lower AUCPR when compounds are structurally dissimilar to the training data.

Finally, we predicted the DICTrank labels for 82 unique compounds that were labeled ambiguous in the original DICTrank data set (Supporting Information Table S6). We used Mordred descriptors and retrained the model on all 1020 compounds (training and held-out compounds) in DICTrank, except for the ambiguous compounds. We found that 43 of the 82 ambiguous compounds were predicted to be cardiotoxic and 39 were predicted to be nontoxic and provided this list to the community for further study (Supporting Information Table S6).

Limitations of This Study

While we considered in this study various chemical and biological data sources, it is important to remember that conclusions are based on limited data. Certain feature spaces contain features that are computed based on chemical structure, such as CELLSCAPE target predictions and physicochemical properties, while data sets such as MOA and SIDER are manually gathered and have 'evidence-of-the-presence' and 'absence-of-evidence' annotations. To train models using feature spaces such as Cell Painting, Gene Expression, and Gene Ontology data sets, we dropped compounds where we could not find profiles (whether experimentally captured or imputed based on matching to highly similar compound profiles using the v-nn approach). The amount of training data (and also the class balance of SIDER/DICTrank labels) is lower for these models. Although we compare data sources using the same test compounds, the varying amounts of training data and the differing types of compounds represented therein can disadvantage some data sources versus others, such that we cannot with certainty compare the signal contained across the feature spaces. The poor performance of -omics data should therefore not yet be attributed to representing the signal in the feature space. Rather in this study, we aim to evaluate the signal present in the data that is available and build the best predictive models possible with public data. Recently, deep learning lgorithmms have been shown to learn feature representations from various -omics data. Transfer learning allows for leveraging pretrained models on large data sets (for example general image-based models), which can then be fine-tuned for specific tasks with limited data (such as Cell Painting data).^85,86 Similarly, one-shot learning shows potential in enabling models to make precise predictions with minimal data.⁸⁷ In the future, deep learning models, by learning and generalizing across feature representations, hold the promise of enhancing predictive accuracy and broadening the scope of data analysis in the study of cardiotoxicity. Further, a recurring challenge in using comprehensive -omics data is the sparsity of data, which limits prospective validation.⁸⁸ This necessitates the development of models that can make reliable predictions even with sparse or incomplete data sets. In this study, we observed that models based on computed physicochemical properties performed on par with other ensemble models. We recommend that this model, which we have made available for public use, be used for prospective validation. In the future, the availability of more data, for example, Cell Painting from JUMP-CP⁸⁹ and Recursion RxRx3⁹⁰ will significantly improve our ability to ascertain the presence of a signal for cardiotoxicity in -omics data.

Conclusions

In this work, we used biological and chemical data (Figure 1) to predict DICT. We determined the feature contained in each data source that most differed between the most-concern versus nontoxic category for DICTrank and found these could drive mechanistic insights. Features from data sources such as predicted protein targets and annotated MOAs that could distinguish the DICT concern categories resembled activity against targets (ion channels in particular) that are mechanistically most plausible. We further evaluated these feature spaces using machine learning to build the first predictive models of DICTrank. Our findings indicate that models relying on physicochemical properties trained on larger training data sets performed on par with the ensemble models based on diverse data sources. The exploratory data analysis in this study suggests that as more -omics data becomes accessible in the future, it will enhance our ability to predict cardiotoxicity. Therefore, for the present, when constructing models using public data sets, we advocate the use of Mordred descriptors and predicted targets (based on chemical structure), since these computed properties are readily available for compounds; they do not require experimental data and could be used to build models for cardiotoxicity. In the future, using biological data, we can look into the biological pathways and mechanisms of DICT leading to better drug design and safer therapeutic strategies.

Data Availability Statement

The models based on chemical structures and physicochemical characteristics are readily accessible for direct use online at https://broad.io/DICTrank_Predictor. All code and data for all models can be found on GitHub (https://github.com/srijitseal/DICTrank) for local implementation, with further details on https://broad.io/DICTrank_Predictor. All data sets are also available at 10.6084/m9.figshare.24312274.v1.

Supporting Information Available

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acs.jcim.3c01834.

• The chemical space of all compounds DICTrank and SIDER; DrugBank compounds containing structural alerts; and evaluation metrics for models developed in this study on an external test set (PDF)

• Number of compounds in Train data set; number of compounds in Test data set; evaluation metrics for SIDER labels for cardiac disorders considering the DICTrank data as true labels; evaluation metrics for all models and data set combinations on cross validation and on held out test sets; and predicted DICTrank for 82 unique compounds originally labeled ambiguous in DICTrank (XLSX)

Author Contributions

S. Seal designed and performed exploratory data analysis and implemented and trained the models. L.H.G. trained CELLSCAPE for the prediction of drug targets. S. Seal, A.E.C., and S. Singh analyzed the biological interpretation of morphological features. S. Seal, O.S., and A.B. analyzed the performance of machine learning models. S. Seal wrote the manuscript with extensive discussions with all authors. All of the authors (S. Seal, O.S., L.H.G., M.G.O., A.E.C., A.B., and S. Singh) reviewed, edited, and contributed to discussions on the manuscript and approved the final version of the manuscript.

S. Seal acknowledges funding from the Cambridge Centre for Data-Driven Discovery (C2D3) and Accelerate Programme for Scientific Discovery. A.E.C., S. Singh, and S. Seal acknowledge funding from the National Institutes of Health (R35 GM122547 to A.E.C.). O.S. acknowledges funding from the Swedish Research Council (Grants 2020-03731 and 2020-01865), FORMAS (Grant 2022-00940), Swedish Cancer Foundation (22 2412 Pj 03 H), and Horizon Europe (Grant Agreements 101057014 (PARC) and 101057442 (REMEDI4ALL)).

The authors declare the following competing financial interest(s): S. Singh and A.E.C. serve as scientific advisors for companies that use image-based profiling and Cell Painting (A.E.C.: Recursion, SyzOnc; S. Singh: Waypoint Bio, Dewpoint Therapeutics) and receive honoraria for occasional talks at pharmaceutical and biotechnology companies. O.S. declares shares in Phenaros Pharmaceuticals. L.H.G. is an employee at Ignota Labs, where CELLSCAPE is a proprietary software.