Deep Learning to Predict the Formation of Quinone Species in Drug Metabolism

Abstract

Many adverse drug reactions are thought to be caused by electrophilically reactive drug metabolites that conjugate to nucleophilic sites within DNA and proteins, causing cancer or toxic immune responses. Quinone species, including quinone-imines, quinone-methides, and imine-methides, are electrophilic Michael acceptors that are often highly reactive and comprise over 40% of all known reactive metabolites. Quinone metabolites are created by cytochromes P450 and peroxidases. For example, cytochromes P450 oxidize acetaminophen to N-acetyl-p-benzoquinone imine, which is electrophilically reactive and covalently binds to nucleophilic sites within proteins. This reactive quinone metabolite elicits a toxic immune response when acetaminophen exceeds a safe dose. Using a deep learning approach, this study reports the first published method for predicting quinone formation: the formation of a quinone species by metabolic oxidation. We model both one- and two-step quinone formation, enabling accurate quinone formation predictions in nonobvious cases. We predict atom pairs that form quinones with an AUC accuracy of 97.6%, and we identify molecules that form quinones with 88.2% AUC. By modeling the formation of quinones, one of the most common types of reactive metabolites, our method provides a rapid screening tool for a key drug toxicity risk. The XenoSite quinone formation model is available at http://swami.wustl.edu/xenosite/p/quinone.

Graphical abstract

INTRODUCTION

Unanticipated toxicity frequently causes drug candidates to be discontinued late in development.¹ Furthermore, idiosyncratic adverse drug reactions (IADRs) are often detected only after approval, causing significant morbidity and mortality.^2,3 Around 75% of IADR cases result in liver transplants or death.^4,5 There is considerable evidence that many varieties of drug toxicity, including IADRs, may be driven by a common mechanism: drug-metabolizing enzymes bioactivate drugs into electrophilically reactive metabolites, which covalently bind to sites within DNA or proteins.^6–9 Frequently, DNA adducts are mutagenic and may interfere with transcription, replication, or regulation, causing gene dysfunction or initiating cancer.^10–12 Likewise, protein adducts can disturb important biological activities or engender adverse immune responses.^12–14 This study focuses on quinones, a major class of Michael acceptors, which are soft electrophiles¹⁵ and the most common type of reactive metabolites.¹⁶ Quinone species, such as quinone-imines, quinone-methides, or imine-methides, are commonly highly electrophilically reactive.¹⁷ Over 40% of all known reactive metabolites are quinones.¹⁶ Quinone formation, the formation of quinones by metabolic oxidation, is performed by cytochromes P450^18–20 and peroxidases.^9,21–23 Quinone formation generally occurs in one or two metabolic steps, where the final step is usually the coreduction of a cytochrome P450’s catalytic iron by the substrate, thereby dehydrogenating and oxidizing the substrate into a quinone species.²⁴ For example, in a single step both cytochromes P450²⁵ and peroxidases²⁶ convert acetaminophen to the toxic metabolite N-acetyl-p-benzoquinone imine, which covalently binds to proteins (Figure 1). In contrast, lumiracoxib requires two steps to form a quinone. First, cytochromes P450 hydroxylate lumiracoxib, the product of which then forms a quinone after further metabolism by cytochromes P450 or peroxidases.^18,27,28

Figure 1.

Examples of one- and two-step quinone formation: the formation of a quinone by metabolic oxidation. The quinone (indicated by red ovals) metabolites of acetaminophen and lumiracoxib cause hepatotoxicity due to the formation of a hapten complex with off-target proteins that triggers toxic immune responses.^18,27,28 Acetaminophen’s conversion to the reactive quinone N-acetyl-p-benzoquinone imine (NAPQI) illustrates one-step quinone formation, and lumiracoxib’s conversion to lumiracoxib quinone-imine via the intermediate 4′-hydroxylumiracoxib demonstrates two-step quinone formation.^18,27,28 The quinone formation model encompasses both one- and two-step quinone formation.

Detecting or anticipating quinone formation is critical for avoiding drug candidates early in drug development that can form reactive metabolites. Experimental methods for detecting quinone formation, most commonly incubation with glutathione, are well developed.²⁹ However, these techniques require time and resources, which can be significant, especially when considering thousands of candidates during the initial screening phase of drug development. Moreover, some methods detect metabolic events that are inconclusive for quinone formation, such as aromatic hydroxylation, because the product may not form a subsequent quinone in the in vitro screens, which are often tuned to only generate one-step metabolites. For example, buspirone undergoes aromatic hydroxylation but does not form a quinone (Figure 2). In contrast, after hydroxylation at a structurally similar motif, the analogue nefazodone does proceed to form a hepatotoxic quinone.^30,31 An accurate computational method for predicting both one-and two-step quinone formation pathways would be complementary to experimental screens and might identify otherwise undetected quinone formation risk.

Figure 2.

Both quinone-forming and nonquinone-forming molecules can undergo comparable initial metabolic events. Nefazodone first undergoes aromatic hydroxylation to form hydroxy-nefazodone, which is then oxidized to form the reactive nefazodone quinone, thereby causing hepatotoxicity in some patients.^30,31 In contrast, the analogue buspirone does not form a quinone, despite undergoing a comparable initial hydroxylation event.³¹ A key goal of our model was to accurately identify molecules that form quinones.

Several studies demonstrated that computational methods can predict sites of metabolism.^32–38 However, these methods do not predict the metabolic structures produced by these reactions, and cytochromes P450 can alter the same site in different ways, each with different consequences to toxicity.^39,40 In contrast, we recently published a computational model that went beyond site of metabolism prediction to specifically predict the formation of epoxides, the second most common type of reactive metabolites after Michael acceptors such as quinones, which demonstrated the feasibility and utility of modeling the formation of specific reactive metabolites.⁴¹

In this study, we built the quinone formation model (Figure 3), which succeeded at two crucial tasks. First, the model accurately predicted the specific atom pairs within molecules that form quinones: their quinone-forming pairs (QPs). Knowledge of QPs can guide structural modifications to reduce the chances of reactive metabolite formation. Second, the model distinguished quinone-forming and nonquinone-forming molecules. Molecule quinone formation predictions can be used to quickly screen for molecules likely to form reactive metabolites, a key toxicity risk.

Figure 3.

Structure of the quinone formation model. The network on the left depicts how information propagated through the model, which contained one molecule layer, one input layer, three hidden layers, and three output layers. From the structure of an input molecule, several molecule-level, atom-level, and pair-level descriptors were calculated at the input layer. The molecule- and atom-level descriptors flowed into the first hidden layer, which outputted an atom quinone formation score (AQS) for each atom in the input molecule. In this study, we defined a quinone-forming pair (QP) as the exact pair of ring carbons that forms a quinone. To predict QP, the atom-level scores of both atoms and the pair-level descriptors were presented to the second hidden layer, which computed a pair-level quinone formation score (PQS). Finally, the top three atom-level scores, the top three pair-level scores, and all molecule-level descriptors were submitted to the third hidden layer, which calculated a molecule quinone formation score (MQS). On the right, atom-level data are illustrated on the bottom (with atoms of quinone formation circled), pair-level data are illustrated in the middle (with a pair of atoms circled), and molecule-level data are illustrated on top (with the quinone-forming molecule circled). The molecule input node is a chemical structure, and all other circles are predictions ranging from 0 to 1. Blocks are vectors of real numbers. Right, a flowchart visualizes the overall experimental method. First, quinone formation data were extracted from a literature-derived database. Next, chemical descriptors were generated from the structures of these molecules. The data were then split into a training set and a test set. The training data were modeled by three steps of sequential training, on the atom-, pair-, and molecule-levels. Finally, the trained model was evaluated on the test data. Ten-fold cross-validation was used, a standard procedure in machine learning for estimating generalization accuracy. For each cross-validation fold, one-tenth of the data was set aside as the test set and the remaining data used as the training set.

MATERIALS AND METHODS

Quinone Formation Training Data

From the literature-derived Accelrys Metabolite Database (AMD), we extracted an extensive data set of chemically diverse metabolic reactions that form quinones. Overall, 576 reactions were extracted, each observed in humans, human cells, or human microsomes. These reactions consisted of 377 single-step quinone formation reactions and 199 two-step quinone formation reactions. An automated algorithm used the structure of each quinone product to label the QP on its metabolic parent, for one-step quinone formation, and its metabolic grandparent, for two-step quinone formation. We use “metabolic parent” to refer to the starting molecule of a one-step quinone formation reaction and “metabolic grandparent” to refer to the starting molecule of a two-step quinone formation reaction. Metabolic parents form quinones directly, whereas metabolic grandparents form a quinone through an intermediate structure. Both metabolic parents and grandparents were included in the training data. The results of the labeling algorithm were manually inspected and corrected as necessary.

For this study, we defined a QP as the pair of ring carbons that forms a quinone. In depictions, the atoms of these QPs are indicated by circles. When a pair of atoms was topologically equivalent to an observed QP, it was itself labeled as a QP. Duplicate starting molecules were merged into a lone training molecule with all observed QPs marked. The final data set included 359 quinone-forming molecules, each with its QPs labeled. Primarily, these QPs were found in six-membered rings (Table S1). Excluding atoms shared between fused rings of different sizes, such as the fused six- and seven-member rings of nevirapine in Figure 4, 90.2% of the QP atoms were in six-membered rings, compared to 8.6% and 1.2% for five- and seven-membered rings, respectively.

Figure 4.

Pair-level model, computing pair quinone formation scores (PQS), accurately predicted quinone-forming pairs (QP). Top left, average pair area under the curve (AUC) was calculated for 359 molecules extracted from the Accelrys Metabolite Database and labeled with their QPs. Within these quinone-forming molecules, the average pair AUC metric quantifies how frequently QP were ranked higher than other atom pairs.^41,44,45 Bottom left, the top-two metric was calculated, which counts a molecule correctly predicted if any of its QP received the first- or second-highest prediction.^{32,34,41,52,53} By both metrics, the cross-validated pair-level scores outperformed atom quinone formation scores (AQS), computed by a control model that did not include training on the pair-level, instead mapping the atom-level to the pair-level by multiplying the scores of both atoms together. The pair-level model performance also exceeded that of quantum chemical descriptors, the five best of which are included in each panel. The baseline of each metric, indicated by the random model, was computed by calculating performance over randomized predictions. For each metric, asterisks indicate results significantly worse than the best-performing method, as determined by a paired t-test.⁵⁴ Examples from the data set are visualized with their scores (ranging from 0 to 0.98) indicated by the colored shading, assigning each atom the probabilistic OR of all pair-level quinone formation scores including the atom. Top center, nevirapine;^89,90 top right, mefenamic acid;⁹¹ bottom center, pyronaridine;⁹² and bottom right, mitoxantrone.⁹³ The atoms of experimentally observed QPs are circled.

Predicting which molecules will form quinones is a key goal of a useful model. Therefore, the training data set also included molecules that do not form quinones. To assemble these molecules, first all metabolically studied molecules in human-relevant experiments were extracted from the AMD. Second, all molecules that contained a six-membered aromatic ring were selected. Third, all molecules that formed quinones were filtered out, including quinones that formed in multiple steps. This procedure yielded a pool of 11884 molecules that have been studied in humans, human cells, or human microsomes but have not been reported to form quinones. From this pool, 359 (the same number of quinone-forming molecules) randomly selected nonquinone-forming molecules were added to the training set, for a total of 718 training molecules.

External Non-Quinone-Forming Test Sets

After removing the 359 nonquinone-forming molecules added to the training data, 11525 nonquinone-forming molecules remained. From this pool, external test sets were similarly extracted by randomly selecting 359 molecules for each test set. We repeated this process 20 times, thereby constructing 20 external test sets with 359 molecules apiece. All of these molecules were withheld from model training and were only evaluated by the final model.

Because of the terms of our AMD license, we were not able to share the structures of the training and test molecules. Instead, all reaction and molecule registry numbers were included in the Supporting Information, consistent with the June 2015 AMD release.

Topological Descriptors

To predict quinone formation, the first step of our method assigned each atom a vector of numbers, topological descriptors, which described chemical properties of that atom. Each atom was assigned 390 numbers, including 375 atom descriptors and 15 molecule descriptors. These descriptors were computed by in-house Python software using the structure of each molecule as input.⁴² We used topological descriptors, previous versions of which have been used for models of metabolism,^32,43 reactivity,^44,45 and epoxidation.⁴¹ For this study, we started with our most recent set of topological descriptors,⁴⁴ which we supplemented with several new atom descriptors for ortho, meta, and para substituents. A second descriptor generation step assigned two descriptors to each pair of atoms: the connectivity distance between the atoms, and whether this distance was odd or even. A comprehensive listing of descriptors is available in the Supporting Information (Tables S3–S7), which also specifies the new topological descriptors developed for this study.

Combined Pair- and Molecule-Level Quinone Formation Model

We built a model using a deep convolutional neural network with one molecule layer, one input layer, three hidden layers, and three output layers (Figure 3). The top output layer computed molecule quinone formation scores (MQS), the middle output layer computed pair quinone formation scores (PQS), and the bottom output layer computed atom quinone formation scores (AQS). Respectively, the molecule-, pair-, and atom-level scores represented the probability that a molecule, pair, or atom formed a quinone, each score ranging from zero to one. The network was trained in three stages.

First, the atom-level network was trained to produce accurate atom-level scores. In this training step, each ring carbon within a molecule was considered a possible atom of quinone formation. Each atom was assigned a vector of numbers that described chemical properties of that atom. The data set was a matrix, with one row per atom, and one column per descriptor. Experimentally observed atoms of quinone formation were labeled with a 1 in a final binary target vector. Using gradient descent on the cross-entropy error, we trained the network weights so that atoms of quinone formation received higher scores than other atoms.

Second, the pair-level network was trained to compute scores for each pair of atoms. In this training step, each row of the data matrix was a pair of ring carbons, and each column was a descriptor. Descriptors included the atom-level score of each atom and both pair-level descriptors. For each atom pair, the two associated atom-level scores were sorted by magnitude and transformed by the logit function into log-odds. The pair-level network was trained to assign experimentally observed QPs with higher scores than all other atom pairs.

Third, the molecule-level network was trained. Each row of this data matrix was a molecule, and each column was a descriptor. Descriptors included the top three atom-level scores, the top three pair-level scores, and all molecule-level descriptors. The weights of the network were trained to assign quinone-forming molecules with higher scores than those for nonquinone-forming molecules.

The final model included hidden layers at all three stages. Several alternative models were considered, including a logistic regressor at all three stages, an AND function that mapped the atom-level to the pair-level, and a max layer that used a molecule’s maximum PQS as its molecule-level score. For all three stages, adding a hidden layer offered either higher classification performance or a better scaled prediction than alternative methods.

Quantum Chemical Descriptors

Currently, there are no other methods for explicitly predicting quinone formation. However, there are several computational tools available for predicting sites of metabolism, which frequently use quantum chemical calculations that correlate to the activation energies of cytochromes P450.^{34,35,46–49} Because of the absence of other methods for predicting quinone formation, we computed several quantum chemical descriptors to establish baseline performances (Tables S8 and S9). These descriptors were calculated by MOPAC, a quantum chemistry package that performs self-consistent field computations, with the COSMO implicit solvent model and the semiempirical method PM7.^50,51 On both the pair- and molecule-level, we compared the performances of quantum chemical descriptors to those of the quinone formation predictions from the model, which only used topological descriptors. Atom-level quantum chemical descriptors were mapped to the pair-level by multiplying the descriptor values of both atoms together.

RESULTS AND DISCUSSION

The following sections analyze the performance and applications of the quinone formation model. First, we quantified the power of pair quinone formation scores (PQS) to predict quinone-forming atom pairs within quinone-forming molecules. Second, we evaluated the ability of molecule quinone formation scores (MQS) to separate quinone-forming and nonquinone-forming molecules. Third, we investigated whether MQS can separate drugs containing the same structural alert. Fourth, we screened for drugs that are predicted to form quinones but are not currently known to do so. Finally, we studied how the model can potentially direct rational drug modifications to prevent quinone formation.

Accuracy at Predicting Quinone-Forming Pairs

To minimize reaction metabolite formation during drug design, identifying a molecule’s QP is critical. Additionally, knowledge of the QP, the exact pair of carbons involved in quinone formation, provides a precise hypothesis about the source of a molecule’s toxicity. Information about QPs also suggests where a molecule could be modified to prevent quinone formation, thereby minimizing a key toxicity risk. The quinone formation model is the first published computational method that specifically predicts QPs across a wide range of molecules.

To predict QPs, the model was trained in two stages. First, the model computed an atom quinone formation score (AQS) for each ring carbon within a test molecule. The atom-level scores ranged from zero to one, reflecting the probability than an atom was involved in quinone formation. The weights of the network were trained such that experimentally known atoms of quinone formation receive high scores, and all other atoms received low scores.

The atom-level scores were intended to predict whether a single atom is part of a quinone formation reaction. However, because quinone formation always operates on a pair of atoms, the atom-level scores did not directly predict quinone formation. Therefore, a second training step maps the atom score to the pair level, producing a pair-level score. Within each molecule, this step considered all pairs of ring carbons as possible QPs. For each pair of atoms, this training step took as input two atom level scores (corresponding to each atom), as well as pair-level descriptors. The weights of this stage of the network were trained to assign QPs with high pair-level scores and all other pairs with low scores.

For both the atom-level and pair-level training steps, 10-fold cross-validation was used, a standard procedure in machine learning for estimating generalization accuracy. Within quinone-forming molecules, accurate predictions should differentiate QPs from all other pairs. We used two metrics to quantify the accuracy of cross-validated atom- and pair-level scores. First, we calculated the “average pair AUC” by computing the area under the receiver operating characteristic curve (AUC) for each molecule and averaging the AUCs across all molecules to measure performance for the entire data set.^41,44,45 Second, we calculated “top-two” performance, which considers a molecule correctly predicted if any of its QPs are predicted in the first or second rank positions, a standard metric for site of metabolism predictions.^{32,34,41,52,53}

To assess the performance of the atom-level model, we translated the atom-level scores to the pair-level by multiplying the scores of both atoms together, thereby calculating the probability that both atoms were involved in quinone formation. By this approach, the atom-level scores computed by a neural network yielded an average pair AUC performance of 97.1% and a top-two performance of 83.8% (Figure 4). In contrast, a logistic regressor computed significantly less accurate predictions, with performances of 94.4% (average pair AUC) and 70.2% (top-two). On the basis of this result, we selected the neural network for the atom-level training stage.

For the second training stage, on the pair-level, we similarly tested both a neural network and a logistic regressor. The neural network predicted QPs with performances of 97.6% and 86.9%, for average pair AUC and top-two, respectively. These did not exceed the performances of the logistic regressor: 97.8% (average pair AUC) and 88.3% (top-two). However, both the pair-level neural network and the pair-level logistic regressor significantly improved performance compared to only training on the atom-level, especially by the top-two metric. For example, comparing the top-two performances of the pair-level logistic regressor and the atom-level neural network by a paired t-test⁵⁴ yielded a p-value of 0.037. Consequently, we retained the pair-level training and model.

However, at the pair level, choosing between the neural network and the logistic regressor was more difficult because they had statistically indistinguishable classification performances. Paired t-tests⁵⁴ indicated that the neural network and the logistic regressor were statistically equivalent for both average pair AUC (p-value 0.339) and top-two (p-value 0.286). To break the tie, we constructed reliability plots, which quantify how well predictions correlate to probabilities (Figure 5).^41,55 These plots revealed that the neural network is a better scaled prediction than the logistic regressor, with the lowest root-mean-square error to a perfectly scaled prediction. Well-scaled predictions are interpretable as probabilities, so we decided to use the neural network. Nevertheless, the choice between the neural network and the logistic regressor is arguable because the logistic regressor is a simpler model structure. While we settled on the neural network for this component of the model, we believe a logistic regressor could likely be utilized with comparable results.

Figure 5.

Pair-level neural network computes the best scaled prediction of quinone-forming pairs and corresponds closely with a probability. Across 78341 atom pairs within 718 quinone-forming and nonquinone-forming molecules, the bar graphs plot the normalized distributions of the pair-level logistic regressor, the pair-level neural network, the atom-level logistic regressor, and the atom-level neural network. Using non-normalized frequencies, the solid lines in each bin plot the percentage of atom pairs that form quinones. Atom-level scores were mapped to the pair-level by multiplying the scores of both atoms together. On the pair-level, the logistic regressor and the neural network used the outputs of the atom-level neural network as inputs, along with pair-level descriptors. Each diagonal dashed line indicates a hypothetical perfectly scaled prediction. Of the four methods, the pair-level neural network provided the best scaled prediction, with the lowest root-mean-square error of a perfectly scaled prediction, and the highest correlation to the best-fit line. Therefore, the pair-level neural network most accurately reflected the probability that an atom pair will form a quinone.

The majority of the QPs in the data were found in six-membered rings (Table S1), which raised concerns that we would not accurately predict quinone formation on five-membered rings due to insufficient data. With this in mind, we investigated whether QPs in five-membered rings were also accurately predicted by calculating accuracy only on those molecules. Across the 36 molecules where a five-membered ring formed a quinone, pair-level scores predicted QPs with performances of 96.7% and 86.1%, for average pair AUC and top-two, respectively. These performances matched the accuracies across the whole data set of 97.6% (average pair AUC) and 86.9% (top-two). We accurately predicted QPs in five-membered rings despite limited training data, suggesting that the underlying principles that guide quinone formation on six-membered rings also applied to quinone formation on five-membered rings.

Currently, there are no other models published for predicting quinone formation to which this model can be compared. Instead, to provide a baseline we calculated several quantum chemical descriptors (Tables S8 and S9), which correlate to the activation energies of cytochromes P450 and are often used by sites of metabolism predictors.^{34,35,46–49} Each quantum chemical descriptor was considered as a model to predict QPs. To map each descriptor to the pair-level, the descriptor value of both atoms was multiplied together. None of the quantum chemical descriptors approached the accuracy of the pair-level score. For example, the best descriptor by the top-two metric was the density of the highest occupied molecular orbital (HOMO) on the test atom, which only had a top-two performance of 34.8%, far below the pair-level model’s performance of 86.9%. By considering many chemical attributes together using machine learning, the model predicted QPs more accurately than any quantum chemical descriptor.

Accuracy at Identifying Quinone-Forming Molecules

We also evaluated how well the model separated quinone-forming from nonquinone-forming molecules. A model that accurately predicts molecule quinone formation could be used to quickly screen drug candidates for a key toxicity risk.

In this assessment, we trained the model to differentiate molecules that formed quinones from those that did not. In addition to quinone-forming molecules, the training data contained molecules that could form quinones, defined as containing a six-membered aromatic ring, and that were metabolically studied in human-relevant experiments in the Accelrys Metabolite Database (AMD) but did not form quinones. Following pair-level training, we investigated several methods of discriminating between quinone-forming and nonquinone-forming molecules (Figure 6). To measure performance at this objective, we calculated the AUC across the full data set (molecule AUC).

Figure 6.

Molecule quinone formation scores (MQS) accurately identified quinone-forming molecules. Left, the ability of several methods to identify molecules that form quinones was computed, including taking each molecule’s maximum pair quinone formation score (PQS), training a logistic regressor or a neural network on the top three pair-level scores, the top three atom-level scores, and all molecule descriptors, and training a control model in the form of a neural network that only used as input molecule descriptors. The data set set included 359 quinone-forming molecules and 359 nonquinone-forming molecules. To quantify performance, the AUC was computed across quinone-forming and nonquinone-forming molecules (Molecule AUC), using cross-validated predictions. Asterisks indicate results significantly worse than the best-performing method, as determined by a false positive rate paired t-test.⁵⁴ Right, closely related example pairs of molecules that do and do not form quinones are visualized. Cross-validated scores (ranging from 0 to 0.78) are indicated by the colored shading, assigning each atom the probabilistic OR of all pair-level quinone formation scores including the atom. The number line denotes each molecule’s cross-validated score, computed by the neural network. From left to right, top to bottom: estradiol (MQS: 0.60),⁹⁴ nefazodone (MQS: 0.83),^30,31 buspirone (MQS: 0.09),^30,31 and a 2α-substituted androstenedione analogue (MQS: 0.42).⁹⁵ The atoms of experimentally observed quinone-forming pairs are circled.

First, we tried the most straightforward method of predicting molecule quinone formation: taking each molecule’s maximum cross-validated pair-level scores. This procedure distinguished quinone-forming and nonquinone-forming molecules with a molecule AUC of 88.5%. Second, we added a training step that submitted the top three pair-level scores, the top three atom-level scores, and all molecule descriptors to a neural network or a logistic regressor. Using 10-fold cross-validation, neither the neural network nor the logistic regressor outperformed the maximum pair-level score, with molecule AUCs of 88.2% (p-value 0.16) and 88.4% (p-value 0.31), respectively, with significance evaluated by a false positive rate paired t-test.⁵⁴

However, the neural network supplied a better scaled prediction than both the logistic regressor and the maximum pair quinone formation score, as indicated by the reliability plots of Figure 7. Furthermore, compared to both of the other methods, the neural network predicted more quinone-forming molecules with high scores and nonquinone-forming molecules with low scores. On the other hand, both the logistic regressor or the maximum PQS were simpler models with equivalent molecule AUC performance. Still, because of its well-scaled predictions, we decided to use the neural network, but we expect that the other two methods would provide similar results.

Figure 7.

Neural network produces a well-scaled probabilistic prediction of molecule quinone formation. Across 718 quinone-forming and nonquinone-forming molecules, the bar graphs plot the normalized distributions of the maximum pair-level score, the molecule-level logistic regressor, and the molecule-level neural network. Using non-normalized frequencies, the solid lines in each bin plot the percentage of molecules that form quinones. Each diagonal dashed line indicates a hypothetical perfectly scaled prediction. The neural network provided the best scaled prediction, with the lowest root-mean-square error (RMSE) of a perfectly scaled prediction, and the highest correlation to the best-fit line. Therefore, the neural network most accurately reflected the probability that a molecule will form a quinone.

Compared to the performance of the pair-level model in identifying QPs (97.6% AUC), the molecule-level model had a considerably lower AUC of 88.2%. This drop in accuracy might be because the molecule-level data contained more noise than the pair-level data. When selecting “nonquinone-forming” molecules, we inferred that a molecule did not form a quinone if it was not reported to do so in the AMD. This supposition, while obligatory for molecule-level training, was not a robust indication that molecules do not form quinones because not every study measures quinone products. As a result, our data set construction procedure imprecisely labeled some quinone-forming molecules as nonquinone-forming. In contrast, the pair-level quinone formation data were much cleaner because they were extracted from experiments capable of reporting quinone formation. Our previous studies with the AMD have observed comparable drops in site-level (equivalent to the pair-level in this study) to molecule-level accuracy.^41,44,45

Even so, the molecule-level scores classified quinone-forming molecules with 88.2% AUC, greatly exceeding that of any comparison quantum chemical descriptor and demonstrating that the model produces illuminating molecule-level predictions. The best quantum chemical descriptor was the energy of the highest occupied molecular orbital (E_HOMO), which only had an AUC of 64.2%. The relatively high performance of the molecule-level model supports our assumption that many of the molecules labeled as nonquinone-forming indeed do not form quinones. If the quinone-forming and nonquinone-forming molecules were actually extracted from identical chemical populations, separating them in our cross-validated experiments, which simulated performance on external data, would be highly unlikely.

To further confirm that molecule-level performance was not a consequence of overfitting, we evaluated the model on several external test sets of nonquinone-forming molecules. Each external test set was predicted by the final trained model. Separation between each external nonquinone-forming test set and the quinone-forming training molecules was measured by the molecule AUC. Over the 20 test sets, the AUC was 85.9% ± 1.2%, which is comparable to the previously reported cross-validated AUC of 88.2% computed by training on both quinone-forming and nonquinone-forming molecules. The model successfully generalized to new data, assigning nonquinone-forming molecules it had never seen before with much lower scores than quinone-forming training molecules.

Comparison to Structural Alerts

We compared the model to “structural alerts,” which are motifs known to commonly form reactive metabolites. A library of structural alerts can be easily used to flag potentially problematic molecules during drug development.^18,56 This is a widely used approach, but unfortunately structural alerts for quinone formation, such as anilines or phenols, are also found in many safe drugs because they are not bioactivated due to specific molecular context. We would hope that the model could identify which structural alerts are bioactivated to quinones and which are not.

To systematically evaluate whether the model distinguishes structural alerts, we focused on several motifs known to commonly form quinones. The exact patterns used are listed in the Supporting Information (Table S2). We included both motifs that form quinones in one step, such as hydroquinones and catechols, and motifs that form quinones in two steps, such as an oxygen or nitrogen ortho or para to a hydrogen. For each motif, we extracted all atom pairs from the training data set that matched the structure and recorded whether this pair formed a quinone. Next, we assigned each pair its cross-validated pair-level score and calculated the AUC across all pairs of that substructure, the pair AUC (Figure 8 and Table 1). For each structural alert, the model predicted quinone formation with performances significantly higher than random.

Figure 8.

Pair-level scores predicted whether several structural alerts form quinones. We tested several motifs that are structural alerts for both one-and two-step quinone formation. For each motif, across the 718 training molecules, the cross-validated scores of all atom pairs matching that motif were extracted. Left, the receiver operating characteristic (ROC) curve is displayed across all the atom pairs of three example motifs (visualized in the center). The graph displays example ROC curves for the second best, typical, and worst performing alerts. The diagonal dashed line indicates baseline performance. The ROC curves of all the motifs are available in the Supporting Information (Figure S1). In the depiction of the ortho to phenol motif, the “R” represents any atom, including hydrogens. Right, the area under the ROC curve (AUC) was calculated across all the atom pairs of each motif. The error bars represent 95% two-sided confidence intervals.⁵⁸

Table 1.

Pair-Level Scores Predicted Whether Structural Alerts Form Quinones

motif	category	quinone-forming pairs	nonquinone-forming pairs	pair AUC (%)
nitrogen ortho to oxygen	one step	3	31	100.0
oxygen para to hydrogen	two step	8	276	98.9
nitrogen ortho to carbon	one step	6	204	97.9
nitrogen para to nitrogen	one step	15	24	96.4
oxygen ortho to hydrogen	two step	57	981	95.2
oxygen ortho to carbon	one step	6	338	95.2
para to phenol	one and two step	108	325	94.6
oxygen para to carbon	one step	33	312	94.1
nitrogen para to carbon	one step	6	44	93.2
hydroquinone	one step	24	3	93.1
ortho to phenol	one and two step	79	813	92.8
nitrogen ortho to hydrogen	two step	12	483	91.4
nitrogen para to oxygen	one step	41	40	90.5
oxygen para to oxygen	one step	33	24	88.1
oxygen ortho to oxygen	one step	51	145	87.5
nitrogen ortho to nitrogen	one step	1	23	87.0
nitrogen para to hydrogen	two step	34	140	84.1
halogen para to carbon or nitrogen or oxygen	two step	4	105	83.6
catechol	one step	43	32	69.3
not matched		417	74398	97.7

For example, the catechol motif commonly forms an ortho-quinone in a single step (Figure 9).^18,56,57 Catechols might be considered a motif that “obviously” forms quinones. However, of the 75 catechols in our data set, only 43 actually are known to form quinones.

Figure 9.

Pair-level scores predicted whether catechols form quinones. Of all structural alerts tested, the model performed the worst on catechols (Figure 8). Nevertheless, the model distinguished catechols with a statistically significant performance. This motif (indicated by the red circles) requires a single step to form a quinone. Across the 718 training molecules, the cross-validated scores of all atom pairs matching this motif were extracted. Left, the receiver operating characteristic (ROC) curve is displayed across all these atom pairs. The diagonal dashed line indicates baseline performance: 50% area under the ROC curve (AUC). Four example molecules are visualized. Scores (ranging from 0 to 0.97) are indicated by the colored shading, assigning each atom the probabilistic OR of all pair-level quinone formation scores including the atom. The atoms of experimentally observed quinone-forming pairs are circled. Top center, a noscapine metabolite⁹⁶ and top right, a raloxifene metabolite,⁹⁷ both of which are correctly predicted to form quinones at the catechol. Bottom center, aspalathin⁹⁸ and bottom right, a methylenedioxypyrovalerone metabolite⁹⁹ neither of which forms quinones at the catechol-para-to-hydrogen motifs, which are correctly assigned low scores.

The quinone formation model’s pair-level scores separated quinone-forming and nonquinone-forming catechols with a pair AUC of 69.3%. This was the lowest performance of all the structural alerts we tested, and there is certainly room for improvement. Nevertheless, as indicated by the 95% confidence intervals⁵⁸ in Figure 9, we separated catechols with a performance significantly higher than baseline (AUC 50%). We find this an encouraging result because the method helps identify which catechols might not actually form quinones, unlike the structural alert approach that considers all catechols a bioactivation risk without evaluating specific molecular context.

We also found that pair-level scores are informative for two-step quinone formation structural alerts. For example, for the motif of a nitrogen para to a hydrogen to form a quinone, hydroxylation must first occur at the unsubstituted carbon, followed by quinone formation. The nitrogen-para-to-hydrogen motif can be found in some anilines or anilides, which are well-known structural alerts.^18,56,59 The entire training data set contains 174 total nitrogen-para-to-hydrogen atom pairs, of which 34 are actually known to form quinones. Encouragingly, the quinone formation model’s pair-level scores accurately separated quinone-forming and nonquinone-forming nitrogen-para-to-hydrogen groups with a pair AUC of 84.1% (Figure 10).

Figure 10.

Pair-level scores predicted whether nitrogen-bound carbons para to a carbon with a hydrogen form quinones. This motif (indicated by the red circles) requires aromatic hydroxylation to form a quinone. Across the 718 training molecules, the cross-validated scores of all atoms pairs matching this motif were extracted. Left, the receiver operating characteristic (ROC) curve is displayed across all of these atom pairs. The diagonal dashed line indicates baseline performance: 50% area under the ROC curve (AUC). Four example molecules are visualized. Scores (ranging from 0 to 0.96) are indicated by the colored shading, assigning each atom the probabilistic OR of all pair-level quinone formation scores including that atom. The atoms of experimentally observed quinone-forming pairs are circled. Top center, a dasatinib analogue¹⁰⁰ and top right, chlorpromazine,¹⁰¹ both of which are correctly predicted to form quinones at the nitrogen-para-to-hydrogen motif. Bottom center, saracatinib¹⁰⁰ and bottom right, a thioridazine metabolite,¹⁰² neither of which forms quinones at the nitrogen-para-to-hydrogen motifs, are correctly assigned low scores.

The phenol motif is a broad structural alert found in a wide variety of drugs.^18,56 In contrast to catechols (which require one step to form a quinone) and nitrogen-para-to-hydrogen (which require two steps), phenols can form quinones in one or two steps, depending on ring substituents. For example, phenols can form a quinone-methide in a single step if there is a para carbon or form a quinone in two steps via intermediate aromatic hydroxylation at the para position. Across our training data set, there are 433 carbons para to a phenol. For each atom, we extracted the atom-pair-level score of the carbon and the ipso-carbon of the corresponding phenol. Of these atom pairs, 108 form a para quinone.

The model accurately distinguished whether a quinone will form para to a phenol, with a pair AUC of 94.6% (Figure 11). Both one- and two-step quinone formation at the para position to a phenol were accurately predicted. For example, both oxymetazoline and an analogue of androstenedione contain a phenol-para-to-carbon motif, but only in oxymetazoline does the phenol-para-to-carbon motif form a quinone-methide.^60,61 The model’s pair-level scores cleanly distinguished the phenol-para-to-carbon in both molecules. Similarly, both propofol and doxycycline contain a phenol-para-to-hydrogen, which can form a quinone in two steps with an intermediate hydroxylation. Propofol’s phenol-para-to-hydrogen received a much higher pair-level score than doxycycline, consistent with only propofol being known to form a quinone.^62,63

Figure 11.

Pair-level scores predicted whether a quinone will form para to a phenol. This motif (indicated by the red circles) may require one or two metabolic steps to form a quinone. In the depiction of the phenol motif on the left, the “R” represents any atom, including hydrogens. Across the 718 training molecules, the cross-validated scores of all atom pairs matching this motif were extracted. Left, the receiver operating characteristic (ROC) curve is displayed across all these atom pairs. The diagonal dashed line indicates baseline performance: 50% area under the ROC curve (AUC). Four example molecules are visualized. Scores (ranging from 0 to 0.77) are indicated by the colored shading, assigning each atom the probabilistic OR of all pair-level quinone formation scores including that atom. The atoms of experimentally observed quinone-forming pairs are circled. Top center, oxymetazoline⁶⁰ and top right, propofol,⁶² both of which are correctly predicted to form quinones at the para-to-phenol motif. Bottom center, an androstenedione analogue⁶¹ and bottom right, doxycycline,⁶³ neither of which forms quinones at their para-to-phenol motifs, are correctly assigned low scores.

These results demonstrate that the model has better specificity than structural alerts. Specific substructures can determine the parts of molecules that have the capacity to form quinones, but they do not assess the likelihood of the quinone actually forming. The quinone formation model, however, can stratify molecules by their propensity of forming quinones, identifying which alerts are bioactivated and which are not.

Structural alerts are retrospective in nature. Consequently, they cannot detect quinone formation for motifs that have not previously been observed to form quinones. As a result, molecules that form quinones at atypical sites will likely be missed by structural alerts. In contrast, our model can potentially detect these unusual cases because it is built on a diverse data set that includes a wide variety of typical and atypical sites of quinone formation. To test this, we investigated the accuracy of the model at predicting quinone formation at sites that do not match structural alerts. After filtering out all atom pairs that matched any of the quinone structural alerts used (Table 1), we calculated the pair AUC over all remaining atom pairs. The model accurately predicted quinone formation at these nonobvious sites with a pair AUC of 97.7% (Figure 12). This indicates that the model is able to accurately predict quinone formation even for atom pairs that would not generally be considered to be at risk.

Figure 12.

Pair-level scores predicted whether quinones form at motifs that do not match structural alerts. Across the 718 training molecules, the cross-validated scores of all atom pairs that did not match any structural alert were extracted. Left, the ROC curve is displayed across all these atom pairs. The diagonal dashed line indicates baseline performance: 50% area under the ROC curve (AUC). Four example molecules are visualized. Top center, an antimalarial drug candidate.¹⁰³ Top right, a voltage-gated sodium channel inhibitor drug candidate.¹⁰⁴ Bottom center, an imiloxan metabolite.¹⁰⁵ Bottom right, a brimonidine metabolite.¹⁰⁶ The atoms of experimentally observed quinone forming pairs are circled. Scores (ranging from 0 to 0.92) are indicated by the colored shading, assigning each atom the probabilistic OR of all pair-level quinone formation scores including the atom.

Case Studies

We considered several case studies to investigate the model’s utility. One potential application is to screen for molecules that are likely to form quinones but that are currently not known to do so. To explore this application, we predicted the quinone formation of drugs that were not reported to form quinones in the AMD. After downloading a database of FDA-approved and withdrawn drugs, the molecules present in the training data set were removed.⁶⁴ Next, each drug was submitted to the trained quinone formation model, thereby assigning a molecule quinone formation score to each drug.

To define a cutoff above which a drug was considered predicted to form a quinone, we used the optimal point on the cross-validated molecule ROC curve from the training data (Figure S4). This optimal point, corresponding to a molecule score of 0.515, is the cutoff that offers the best trade-off between sensitivity and specificity. We selected all drugs that were assigned a higher score than the optimal cutoff. This revealed 288 drugs that are predicted to form quinones but that are currently thought to be nonquinone-forming across all the relevant literature in the AMD. All 288 drugs and their molecule quinone formation scores are available in the Supporting Information. Here, we highlight three example drugs (Figure 13). Each drug carries a risk of idiosyncratic toxicity, the causes of which are currently obscure.

Figure 13.

Examples of drugs associated with idiosyncratic toxicity that received high molecule quinone formation scores (MQS) but are not known to form quinones in our data set. Each drug predicted to form a quinone is visualized above its most similar quinone-forming molecule from the training data. Similarity was quantified using path based fingerprints and MinMax similarity, a modification of Tanimoto appropriate for count fingerprints.¹⁰⁷ Top, flecainide (MQS: 0.81), thenalidine (MQS: 0.79), and bunamiodyl (MQS: 0.96). Bottom, a remoxipride metabolite (MQS: 0.58),¹⁰⁸ thioridazine (MQS 0.36),¹⁰¹ and a flutamide metabolite (MQS 0.88).¹⁰⁹ The atoms of experimentally observed quinone forming pairs are circled. Scores (ranging from 0 to 0.99) are indicated by colored shading, assigning each atom the probabilistic OR of all pair-level quinone formation scores including the atom. For the training examples, cross-validated scores are reported.

First, the antiarrhythmic flecainide is associated with idiosyncratic hepatotoxicity, the origin of which is currently unknown.^18,59,65 There are no known reactive metabolites of flecainide, and previous studies have suggested that flecainide is harmlessly metabolized to a phenol via oxygen-dealkylation, followed by glucuronidation.^18,59,66 However, our model suggests that the phenol metabolite may actually form a quinone.

Second, the antihistamine thenalidine is associated with idiosyncratic cases of agranulocytosis, a potentially deadly condition that entails a severely lowered blood white cell count that leaves a patient vulnerable to critical infection.^67–70 Indeed, three years after thenalidine was introduced to the U.S. market, it was withdrawn in 1958 due to fatal cases of agranulocytosis.^71–74 The mechanism of thenalidine’s idiosyncratic toxicity was never uncovered, and it is has no known reactive metabolites. An obvious hypothesis is that thenalidine’s thiophene ring, a well-known structural alert, could undergo bioactivation through epoxidation or sulfur-oxidation.^18,75 However, the quinone model’s predictions yield a second, more surprising hypothesis: thenalidine may undergo hydroxylation at the para position on its aniline ring, followed by formation of a potentially highly reactive quinone-imine.

Third, bunamiodyl carries a risk of severe nephropathy.^71,76 It was introduced in 1958 as a cholecystographic contrast medium but was withdrawn in 1964 after around 100 patients died due to renal failure.^71,74,76,77 While bunamiodyl is not known to produce reactive metabolites, our model predicts that it undergoes dehalogenation followed by formation of an ortho or para quinone-imine.

As seen for flecainide, thenalidine, and bunamiodyl, the model can yield an explicit, testable hypothesis about the mechanisms of a molecule’s toxicity. Experimentally validating these quinone formation predictions is beyond the scope of this study but is planned for future work.

In another application, we see that the model’s scores accurately reflect the impact of rational drug modifications. Often, small modification to a molecule can prevent reactive metabolite formation while retaining drug efficacy. For example, the antimalarial drug amodiaquine forms a reactive quinone-imine metabolite that causes the hepatotoxicity and agranulocytosis sometimes associated with amodiaquine treatment, severely limiting its use.^18,78–80 Amodiaquine has an analogue that retains pharmacological efficacy while preventing reactive metabolite formation.^18,81,82 The quinone formation model detects the effect of this subtle change, cleanly distinguishing amodiaquine from its safe analogue (Figure 14). The quinone formation model also correctly predicts amodiaquine’s QP, demonstrating how the model could help guide rational drug modifications that require specific knowledge about where on a molecule a reactive metabolite forms.

Figure 14.

Model reflects the impact of structural modification to prevent quinone formation. The hepatotoxicity and agranulocytosis that have significantly curtailed the use of the antimalarial drug amodiaquine (left) have been traced to the formation of a reactive quinone-imine metabolite.^18,78–80 An amodiaquine analogue (right) has been identified that retains pharmacological efficacy while preventing reactive metabolite formation.^18,81,82 Molecule quinone formation scores (MQS) cleanly distinguished amodiaquine (MQS: 0.93) from this safe analogue (MQS: 0.18). Additionally, pair-level quinone formation scores (PQS) correctly predicted amodiaquine’s quinone formation pair, the atoms of which are circled. Scores (ranging from 0 to 0.98) are indicated by the colored shading, assigning each atom the probabilistic OR of all pair-level quinone formation scores including the atom. Cross-validated scores are reported for amodiaquine.

MODEL LIMITATIONS

Quinone formation is only one piece of the toxicity puzzle, and this model is only one step toward effective toxicity management. For example, while the model predicts quinone formation, it does not predict the reactivity of those quinones. Subtle changes in ring substituents can have a large impact on quinone reactivity and toxicity.⁸³ Consequently, in future work we plan to incorporate the quinone formation model with already-developed reactivity models.^44,45

Moreover, the quinone formation model does not correct for alternate metabolic pathways that may detoxify molecules before they can form quinones. Because of these metabolic alternatives, quinone formation is sometimes observed in vitro but does not occur in vivo.¹⁸ Furthermore, after quinones form, they can be further metabolized by reduction, which, for example, can produce hydroquinones that can then be conjugated and eliminated.⁸⁴ In the long run, integration of the quinone formation model with models of reduction and detoxification, such as uridine 5′-diphospho-glucuronosyltransferase conjugation,⁸⁵ could offer more nuanced predictions by weighing quinone formation against other metabolic routes.

Likewise, quinone formation that occurs in several metabolic steps may not be detected by the current model, which focused on one- and two-step quinone formation. As we continue to develop more complex systems of metabolite structure prediction, we plan to expand the quinone formation model by explicitly modeling intermediate structures, potentially extending its utility beyond two metabolic steps. Additionally, the current method only uses topological descriptors. As evidenced by the quinone formation model’s accuracy, these descriptors are tightly correlated with electronic structure, but performance on rare subclasses or new molecules could be affected by not explicitly including quantum chemical descriptors. In the future, we will consider expanding our method to include quantum chemistry, such as descriptors relating to the half-reactions between the substrate and Compound I within cytochromes P450.

Finally, there is no guarantee that the model’s applicability domain extends beyond its training domain of metabolically studied drug-like molecules. New areas of chemical space, possibly only currently explored in proprietary data within pharmaceutical companies, may not be well suited to the current topological-descriptor-based model trained on literature-derived data. However, this study’s methodology could be easily applied to new data, thereby expanding the utility of the model.

CONCLUSIONS

This study constructed a novel method that forecasts the formation of reactive quinone metabolites. The QP-trained quinone formation model predicted with 97.6% AUC the QPs within quinone-forming molecules. These pair-level predictions suggest where drug candidates could be modified to make them safer. The model also separated quinone-forming and non-quinone-forming molecules with 88.2% AUC. Furthermore, the model distinguished molecules containing the same quinone-formation structural alert and detected the impact of rational drug modification to prevent quinone formation. Molecule-level quinone formation predictions can be used to flag problematic molecules in the early stages of drug development. However, eventually both reactivity and metabolism must be modeled to accurately predict reactive metabolite formation. While we have recently made progress on modeling the reactivity of diverse chemicals,^44,45 previous studies of metabolism have primarily focused on predicting sites of metabolism, rather than actual metabolite structures.^{32,37,43,47,85–88} This study explicitly predicted the formation of quinones, one of the most common types of reactive metabolites, thereby supplying an essential piece of a unified model of reactivity and metabolism.

ABBREVIATIONS

AMD

Accelrys metabolite database

AQS

atom quinone formation scores

AUC

area under the receiver operating characteristic curve

HOMO

highest occupied molecular orbital

IADRs

idiosyncratic adverse drug reactions

MQS

molecule quinone formation scores

PQS

pair quinone formation scores

QPs

quinone-forming pairs