Predictive structure-based toxicology approaches to assess the androgenic potential of chemicals

Abstract

We present a practical and easy-to-run in silico workflow exploiting a structure-based strategy making use of docking simulations to derive highly predictive classification models of the androgenic potential of chemicals. Models were trained on a high-quality chemical collection comprising 1689 curated compounds made available within the CoMPARA consortium from the US Environmental Protection Agency and were integrated with a two-step applicability domain whose implementation had the effect of improving both the confidence in prediction and statistics by reducing the number of false negatives. Among the nine androgen receptor X-ray solved structures, the crystal 2PNU (entry code from the Protein Data Bank) was associated with the best performing structure-based classification model. Three validation sets comprising each 2590 compounds extracted by the DUD-E collection were used to challenge model performance and the effectiveness of Applicability Domain implementation. Next, the 2PNU model was applied to screen and prioritize two collections of chemicals. The first is a small pool of 12 representative androgenic compounds that were accurately classified based on outstanding rationale at the molecular level. The second is a large external blind set of 55450 chemicals with potential for human exposure. We show how the use of molecular docking provides highly interpretable models and can represent a real-life option as an alternative nontesting method for predictive toxicology.

Graphical Abstract

1. Introduction

In the past several decades, the exposure of people and environmental species to xenobiotics has dramatically increased (Egeghy et al., 2012; Judson et al., 2009) due to the use and manufacture of tens of thousands of natural and synthetic chemicals, derived from agricultural and industrial processes as well as from medical applications. It is suspected that some of these are responsible for a broad spectrum of adverse health effects seen in humans and wildlife (Diamanti-Kandarakis et al., 2009; Mrema et al., 2013). In particular, risks are posed by exposure to chemicals interfering with the physiological functioning of the endocrine system whose dysregulation can cause adverse effects in an intact organism or its progeny (Kavlock et al., 1996; Testai et al., 2013; Zoeller et al., 2012). It is widely acknowledged that the mode of action of well-known endocrine disrupting chemicals (EDCs) consists of either mimicking the interaction of natural hormones at the receptor level or of altering synthesis, transport, and metabolism pathways (Schug et al., 2011). A well-known case is diethylstilbestrol (DES), a synthetic estrogenic drug (Bibbo et al., 1975) widely prescribed until the early 1970s to pregnant women to prevent spontaneous abortion and to stimulate fetal growth. DES was banned upon discovery of its causative role in affecting the development of the reproductive system and in inducing vaginal cancer after puberty in women exposed in utero.(Hoover et al., 2011)

The need to protect human health and the environment has led to the development of scientific and regulatory approaches to detecting potential EDCs. A key milestone in this area is REACH (EC, 2006), the European regulation whose Annexes explicitly mention endocrine disruption as an important toxicological end point. In parallel, the assessment of EDCs is a focus of the Endocrine Disruptor Screening Program (EDSP) of the US Environmental Protection Agency (US-EPA). Two recent projects carried out under the EDSP have combined (quantitative) structure activity relationship ((Q)SAR) models from multiple research groups to predict estrogen receptor (ER) and androgen receptor (AR) activity of tens of thousands of chemicals in the environment (ER, Collaborative Estrogen Receptor Activity Prediction Project (CERAPP) (Mansouri et al., 2016); and AR, Collaborative Modeling Project for Androgen Receptor Activity (CoMPARA) (Mansouri et al., 2017).

These projects are examples of collaborations between national governments, the chemical industry, and academic scientific researchers to develop and apply in silico methods, such as QSAR and read-across (Jacobs, 2004; Low et al., 2013; Nicolotti et al., 2009, 2014; Patlewicz et al., 2014; Raunio, 2011), as alternatives to animal experimentation, with the aim of limiting the in vivo testing of chemicals that is time-consuming, costly, and ethically questionable (Devillers, 2009). (Q)SAR and read-across can work well when there is a high similarity between a data-poor target chemical whose activity is to be predicted and a panel of structural analogues having known toxicological profiles (Devillers et al., 2006). (Q)SAR and read-across can often fail when the target chemical and its analogues are built from significantly different scaffolds (Jagiello et al., 2016).

To overcome this intrinsic limitation (i.e., lack of data rich analogues with close structural similarity to a given target chemical), a strategy that is often adopted in drug discovery programs with protein receptor or enzyme targets is to use 3-dimensional information on the interaction of the ligand and its protein binding site. This can be especially useful when the known active compounds (ones that bind to the protein target) span a range of structural classes or backbones. Following this approach, we have used molecular docking to help predict whether chemicals could be EDCs through interaction with AR. Molecular docking is a very effective method to predict the preferred posing of a given molecule approaching a specific target macromolecule to form an energetically stable complex. Docking employs the wealth of physicochemical information contained in experimentally solved structures of target proteins, in order to infer interactions and other mechanistic knowledge about structurally heterogeneous small molecules (Ferreira et al., 2015; Meng et al., 2011). Docking benefits from the growing availability of solved crystallographic protein structures (https://www.wwpdb.org/stats/deposition). These structures represent a great high quality resource for improving the state of the art of predictive toxicology relative to endocrine disruption and other modes of action irrespective of their chemical structural class (Nicolotti et al., 2008; Vedani and Smiesko, 2009). A docking model was included in the CERAPP collaboration (Mansouri et al., 2016; Trisciuzzi et al., 2015) which combined results of a large number of in silico models to predict the ability of chemicals to interact with the ER. The results of this collaboration are being used to prioritize further testing of potential EDCs.

Comparable to estrogenic hormones, androgens are involved in the development and maintenance of the sexual reproductive system and secondary physical changes associated with puberty (Gao et al., 2005; Wilson, 1999). Recent studies (Emmelot-Vonk et al., 2008; Matsumoto et al., 2013) demonstrated that chemicals interfering with AR or dysregulating androgen-dependent signaling pathways can be classified as EDCs, and are responsible for the onset of several pathological conditions (e.g., decreased sperm counts, increased infertility) (Luccio-Camelo and Prins, 2011) or diseases (e.g., testicular dysgenesis syndrome, prostate cancer) (Fisher, 2004; Kim et al., 2005).

The integrated approach of the CERAPP project for evaluating candidate estrogenic compounds (Browne et al., 2015; Mansouri et al., 2016) was subsequently adapted to identify potential AR active chemicals through the CoMPARA initiative, a large-scale modeling collaboration between 35 research groups in the Americas, Europe, and Asia.

Following the CERAPP approach of Mansouri et al. (2016) each participating group in CoMPARA developed one or more in silico models using high quality experimental data derived from multiple in vitro AR high-throughput screening (HTS) assays (Kleinstreuer et al., 2017). The ultimate goal of CoMPARA consisted of deriving consensus scores in order to prioritize compounds for further analysis regarding their potential to interact with AR (Mansouri et al., 2017)

Building on recently published work (Trisciuzzi et al., 2015), we herein demonstrate a practical and easy-to-run protocol employing molecular docking to derive classification models able to identify compounds that interact with AR. The models were developed within the CoMPARA program using a 3D collection of 1689 chemicals with high-quality AR data provided by the US-EPA (Kleinstreuer et al., 2017). Importantly, we illustrate how docking-based classification models should be carefully evaluated in the context of predictive toxicology and how to interpret related statistical parameters. Special attention is given to the definition and implementation of the applicability domain (AD). Interestingly, restricting predictions to chemicals within the AD allowed us to improve robustness and reliability of our classification model in distinguishing AR active from inactive chemicals. From a methodological point of view, this work thus represents the first attempt to extend the concept of AD, normally used in (Q)SAR and 3D-(Q)SAR strategies (Gissi et al., 2014; Sahigara et al., 2012), to predictive models based on docking simulations.

2. Material and Methods

2.1. Starting training set EPA-ARDB

The starting training data set provided by the US-EPA (subsequently referred to as EPA-ARDB) consisted of a curated collection of 1689 chemicals having high quality experimental data. For each chemical, the AR bioactivity was quantified based on the results of 11 AR-related in vitro assays, conducted in HTS measuring the androgen-related activity at multiple points along the AR signaling pathway (Kleinstreuer et al., 2017). This data set is taken from the Tox21 and ToxCast programs (Kavlock et al., 2012; Tice et al., 2013). For classification modeling, a binary (0/1) value was assigned to each compound as a function of biological response. The training set contained 205 compounds (i.e., class = 1) acting as binders (i.e., approximately 12%), and 1484 compounds (i.e., class = 0) acting as nonbinders (i.e., approximately 88%). It is worth noting that the EPA-ARDB includes a relatively small fraction of binders, thus resulting in a strongly unbalanced training set. The full set of 1689 chemicals was supplied as SMILES strings, provided in the Supporting Information.

The training data additionally classified chemicals as being agonist or antagonists, but in the current model, we treat all actives as a single class of binders. For the sake of clarity, we hereafter refer to androgenic/nonandrogenic compounds as binders/nonbinders, respectively.

2.2. Molecular docking

Docking simulations were conducted using nine AR crystal structures retrieved from the Protein Data Bank (PDB) (Berman et al., 2000) selected as follows: six crystal structures previously selected by Kolšek et al. (2014) based on their capability to provide highly predictive docking-based classification models, and two of the most recent AR X-ray solved crystals available in the PDB (PDB IDs: 5CJ6 (Saeed et al., 2016) and 4QL8 (Ullrich et al., 2014)). In addition, a crystal structure (PDB ID: 2HVC (Wang et al., 2006)) showing a noncanonical binding mode of the cognate ligand compared to the other selected X-ray complexes was selected. For further details, the chemical structures of the X-ray cognate ligands and their interactions patterns are reported in Table S1. The receptor structures considered as targets for docking studies are summarized in Table 1.

Table 1.

Detailed information of the nine selected X-ray crystal structures.

PDB CODE	Cognate ligand	Ligand effect	Resolution(Å)	R- Value Free	References
2AM9	testosterone	agonist	1.64	0.230	(Pereira de Jésus-Tran et al., 2006)
2AX9	(R)-3-bromo-2-hydroxy-2-methyl-n-[4-nitro-3(trifluoromethyl)phenyl]propanamide]	agonist	1.65	0.249	(Bohl et al., 2005)
2PNU	(5S,8R,9S,10S,13R,14S,17S)-13-{2-[(3,5-difluorobenzyl)oxy]ethyl}-17-hydroxy-10-methylhexadecahydro-3h-cyclopenta[a]phenanthren-3-one	agonist	1.65	0.205	(Cantin et al., 2007)
3B66	4-{[(1R,2S)-1,2-dihydroxy-2-methyl-3-(4-nitrophenoxy)propyl]amino}-2-(trifluoromethyl)benzonitrile	agonist	1.65	0.242	(Bohl et al., 2008)
3L3X	5-alpha-dihydrotestosterone	agonist	1.55	0.206	(Zhou et al., 2010)
3RLJ	(2S)-3-(4-cyanophenoxy)-n-[4-cyano-3-(trifluoromethyl)phenyl]-2-hydroxy-2-methylpropanamide	antagonist	1.90	0.265	(Duke et al., 2011)
5CJ6	agonist 2-chloro-4-{[(1R,2R)-2-hydroxy-2-methylcyclopentyl]amino}-3-methylbenzonitrile	agonist	2.07	0.222	(Saeed et al., 2016)
4QL8	2-chloro-4-[(3S,3aS,4S)-4-hydroxy-3-methoxy-3a,4,5,6-tetrahydro-3H-pyrrolo[1,2-b]pyrazol-2-yl]-3-methylbenzonitrile	agonist	2.10	0.275	(Ullrich et al., 2014)
2HVC	6-[bis(2,2,2,-trifluoroethyl)amino]-4-(trifluoromethyl)quinolin-2(1h)-one	agonist	2.10	0.253	(Wang et al., 2006)

All protein structures were refined using Protein Preparation Wizard (Schrödinger Suite 2016–3) for correcting common problems such as missing hydrogen atoms, incomplete side chains and loops, ambiguous protonation states, and flipped residues. The 3D conformations of the EPA-ARDB were processed by LigPrep (Schrödinger Suite 2016–3) in order to properly generate all the possible tautomers and ionization states at a pH value of 7.0 ± 2.0. This procedure increased the database size to 2634 structures. Finally, all of the chemicals were docked into the nine pretreated AR protein structures using GOLD (Genetic Optimization for Ligand Docking) software (Jones et al., 1997). For all simulations, a spherical grid having a radius of 10 Å centered on the center of mass of the cognate ligands was used. All the default flexible ligand docking settings from Gold v.5.2 (Jones et al., 1997) and the fitness function ChemScore were used. Finally, all the compounds were ranked according to their docking scores.

In order to corroborate the robustness of our docking protocol, each original X-ray cognate ligand was redocked back into its corresponding protein binding site. Comparing the Cartesian coordinates of the corresponding heavy atoms of the obtained poses, the X-ray cognate ligands moved back to the original positions with a root mean square deviation (RMSD) < 2 Å (Wilantho et al., 2008) as reported in Table S2.

2.3. Applicability Domain (AD)

The AD represents the space of reliability of a model, where the predictions are the result of interpolations rather than extrapolations. Basically, the quality of the AD depends on the number of chemicals and, more importantly, on the set of molecular descriptors forming a model (Gadaleta et al., 2016; Sahigara et al., 2012). In our study, an initial pool of 237 2D physicochemical and topological molecular descriptors was generated for each chemical in the EPA-ARDB using CANVAS, a cheminformatics program available in the Schrödinger Suite (Canvas, Schrödinger Suite 2016–3). To compare the contribution of each descriptor to the model reliability, we calculated their statistical population variance. The molecular descriptors having population variance equal to zero were discarded, resulting in a final set of 162 descriptors. It is worth noting that the AD assessment is strongly influenced by the modeling approach and the characteristics of the starting training data set (Sahigara et al., 2012). In other words, the AD is determined case-by-case based on the pursued data modeling strategy (Nembri et al., 2016). In the present investigation, the AD was defined using a two-step approach. First, we applied a range-based method, known as the bounding-box (Gadaleta et al., 2016; Sahigara et al., 2012): a chemical is inside the AD if its set of molecular descriptors falls in the range bounded by the maximum and minimum values of the selected molecular descriptors calculated for the EPA-ARDB. In other words, a given predicted external compound showing even only one descriptor exceeding the range defined by the descriptors calculated over the training set chemicals is considered outside the AD (Jaworska et al., 2005). The maximum and minimum values for each molecular descriptor defining the bounding-box are listed in Table S3.

Chemicals within the initial AD were then further examined by using the convex-hull strategy (Sahigara et al., 2012). Following this geometric-based approach, an interpolation space based on the Cartesian coordinates of the top two principal components (PCs), obtained from the initial 162 descriptors, was defined.

The model AD is represented by the smallest convex area whose borders describe the perimeter of a polygon containing the training set compounds. In order to avoid the overestimation of underrepresented areas (Gissi et al., 2013), the polygon area was further restricted to an inner region containing the top 95% of EPA-ARDB chemicals on the basis of their closeness to the EPA-ARDB centroid in the PC space. In doing that, a given predicted external compound located outside the inner polygon is flagged as being outside of the AD.

2.4. Validation set

Three validation sets (VS) were obtained by extracting data from the DUD-E (Directory of Useful Decoys Enhanced) database (Mysinger et al., 2012). In particular, a total of 14619 chemicals relevant to AR interaction were available from the DUD-E web interface (the details of individual chemicals can be found at http://dude.docking.org/targets/andr). Three VS (hereafter referring to VS1, VS2, and VS3, respectively) comprised 2590 chemicals and were all equally sized. In particular, each benchmark database contained the same pool of 259 binders (10% of the set size), doped with three different groups of 2331 nonbinder compounds (90% of the set size) randomly chosen. Notably, duplicate compounds (same entry in both the VS and EPA-ARDB) were identified and removed from the validation set.

The 3D structures of all the compounds were processed by LigPrep (Schrödinger Suite 2016–3) in order to properly generate all the tautomers and ionization states at a pH value of 7.0 ± 2.0. This procedure increased the three VS sizes to 4323, 4332, and 4364 structures, respectively.

2.5. Performance evaluation

As elsewhere described (Trisciuzzi et al., 2015), a statistical confusion matrix was used to assess the goodness of fit of the molecular docking strategy to discern binders from nonbinders. As shown in Table S4, the confusion matrix contains the instances of experimental and predicted classes obtained from each considered classification model (Provost and Kohavi, 1998). More specifically, the confusion matrix reports the number of experimental positive and negative cases that were correctly predicted, i.e., true positives (TPs) and true negatives (TNs), as well as the number of experimental negative and positive cases that were incorrectly identified, otherwise called false positives (FPs) and false negatives (FNs), respectively. From a regulatory oriented conservative toxicological point of view, it is important to pay special attention to the FNs, whose number should be kept as low as possible to minimize the number of truly active compounds predicted to be inactive.

Sensitivity (SE) and specificity (SP) are used to measure the goodness of a classification model. They range from 0 to 1, represent the correctly classified proportion of binders and nonbinders, respectively, and are defined as parameters:

$$SE=\frac{TP}{TP+FN}$$

and

$$SP=\frac{TN}{TN+FP}$$

Furthermore, the performance of a classification model is assessed by the receiver operating curve (ROC) curve, which reports the variation of SE with respect to 1 – SP values, representing the TP and FP rates, respectively. In the present work, the area under the curve (AUC) and ROC-based enrichment factor (EF) were computed to evaluate and compare different docking-based classification models (Triballeau et al., 2005). The AUC curve provides an immediate idea of the goodness in the overall classification irrespective of any docking score used as cutoff. AUC values < 0.5 indicate that a given model is worse than a random classifier while AUC values > 0.5 indicate that a given model works better than a random classifier. The EF is defined as the percentage of binders found in a given early fraction of the ranked database, defined as follows:

$$\mathrm{EF}=\frac{{\mathrm{H}}_{\text{SCR}}}{{\mathrm{H}}_{\text{TOT}}}\times \frac{{\mathrm{DB}}_{\text{TOT}}}{{\mathrm{DB}}_{\text{SCR}}}$$

where HSCR is the number of binders recovered at a specific percentage level of the binder/nonbinder ratio of the ranked database, HTOT is the total number of binders for a given target, DBSCR is the number of compounds screened at a specific percentage level of the database, and DBTOT is the total number of compounds in the database. Notably, we considered the percentage of binders found in the top 1% of the ranked EPA-ARDB (EF1%). Remarkably, EF1% depends on the binder to nonbinder ratio, and its value should be compared to the ideal EF (EFmax) obtained by dividing the total number of compounds of database by the total number of binders. The smaller the gap between EF1% and EFmax is, the higher the ranks of binders and thus the lower the number of FPs (Li et al., 2009). For the EPA-ARDB, the ideal EFmax value is equal to 8.2.

In addition to be useful for drawing ROC curves, SE can be employed to set user-definable thresholds (Kolšek et al., 2014). In particular, SE values equal to 0.25, 0.50 and 0.75 were chosen to define four classes of binding, for each AR crystal receptor, summarized as follows:

• SE ≤ 0.25, the class with very high probability of binding (i.e., hazard molecules)

• 0.25 < SE ≤ 0.50, the class with high probability of binding (i.e., warning molecules)

• 0.50 < SE ≤ 0.75, the class with medium probability of binding (i.e., suspicious molecules)

• SE > 0.75, the class with low probability of binding (i.e., safe molecules)

Based on the obtained rankings, the energetic docking scores better approximating the SE-based thresholds equal to 0.25, 0.50 and 0.75 were identified.

The goodness of the classification can be computed using the following parameters:

$$\mathit{NPV}=\frac{\mathit{TN}}{\mathrm{TN}+\mathrm{FN}}$$

and

$$\mathit{PPV}=\frac{\mathit{TP}}{\mathrm{TP}+\mathrm{FP}}$$

At a given threshold, PPV is related to the probability that a chemical predicted as a binder (over-threshold) is actually a binder, whereas NPV is related to the probability that a chemical predicted as a nonbinder (under-threshold) is actually a nonbinder. In the present work, NPV and PPV values were calculated for each SE threshold (SE = 0.25, SE = 0.50 and SE = 0.75). The prediction classes were generated automatically by using an in-house Python script implementing the computation of PPV and NPV from desired SE.

Given that the EPA-ARDB is strongly unbalanced (i.e., very low number of binders), two additional parameters were computed at each set SE threshold, which are the positive/negative likelihood ratio (+/− LR) and the Balanced Classification Rate (BCR).

The positive (+LR) and negative (-LR) likelihood ratio were computed for each SE threshold as follows:

$$+\mathit{LR}=\frac{\mathit{SE}}{1-\mathrm{SP}}$$

and

$$-\mathit{LR}=\frac{1-\mathrm{SE}}{\mathit{SP}}$$

A +LR value equal to 4 would indicate a four-fold increase of the probability of a compound being a binder with respect to the initial condition before the classification; similarly, a -LR = 0.4, shows that, for an under-threshold compounds, the probability to be a binder is equal to 4/10 with respect to that at the initial condition. The larger the +LR value (or the lower the -LR) is, at a given SE threshold, the better the performance of the classification model. Usually applied in the medicine to assess the accuracy of diagnostic tests, the likelihood ratios were herein adapted to evaluate the performance of classification models in the toxicological field.

The BCR is a modification of the well-established correct classification rate. BCR awards with higher scores those models showing an optimal balance between SE and SP (Sokolova and Lapalme, 2009). It is defined as follows:

$$\mathit{BCR}=\frac{\mathit{SE}+\mathit{SP}}{2}\times (1-\mid \mathit{SE}-\mathit{SP}\mid )$$

For each given SE threshold (SE = 0.25, SE = 0.50 and SE = 0.75), the best model is selected as the one having the highest value of the BCR metric. (68).

3. Results & discussion

3.1. Models evaluation

The entire EPA-ARDB was docked into the binding sites of the nine selected X-ray crystal structures, and thereby nine docking-based classification models were derived. For the sake of clarity, each classification model is named using the same PDB code of the crystal structure used for docking simulations. Notice that the models were derived excluding a small fraction of compounds from the training set (i.e., percentage from 2.79% (2PNU) to 6.09% (2AM9) that are undocked or returning unrealistic (positive) values of docking score (see Table S5)). As reported in previous works (Kolšek et al., 2014; Trisciuzzi et al., 2015), the goodness of the docking protocol was preliminarily evaluated considering the AUC and ROC-based enrichment factor at 1% (EF1%) of the ranked database. ROC curves related to the nine classification models are reported in Figure 1 while the corresponding statistical parameters are summarized in Table 2.

Figure 1.

Receiver operating characteristic (ROC) curves derived from the nine selected AR structures (PDB IDs: 2AM9, 2AX9, 2PNU, 3B66, 3L3X, 3RLJ, 5CJ6, 4QL8 and 2HVC). Each classification model is named as the PDB code of the crystal structure employed to perform docking simulations. Red line: AUC = 0.50; black line: AUC = 1.0. All ROC-curves are colored as defined in the legend.

Table 2.

Area under the curve (AUC) of receiver operating characteristic (ROC) curves and enrichment factor at 1% (EF1%) relative to the Protein Data Bank entries (PDB IDs: 2AM9, 2AX9, 2PNU, 3B66, 3L3X, 3RLJ, 5CJ6, 4QL8 and 2HVC).

PDB CODE	AUC	EF1%
2AM9	0.75	8.2
2AX9	0.70	3.2
2PNU	0.76	6.7
3B66	0.72	4.7
3L3X	0.75	7.6
3RLJ	0.70	4.1
5CJ6	0.72	6.9
4QL8	0.74	6.4
2HVC	0.76	8.0

Notably, the AUC values range from 0.70 to 0.76 demonstrating the high probability of all the models to catch binders (or nonbinders) with good accuracy with respect to a random choice. Likewise, the EF at 1% displays eligible values comparable to the ideal EF_max (i.e., 8.2).

At first glimpse, this preliminary analysis shows a good predictive power of the whole docking procedure for all the considered models. Indeed, albeit ligand based models showing even better performances are available in the literature (Li et al., 2010; Bohl et al., 2004; Jensen et al., 2011; Waller et al., 1996; Vinggaard et al., 2008) they cannot capture the wealth of information on the protein binding sites, thus, making structure-based models more easily interpretable according to regulatory purposes.

In particular, 2PNU and 2HVC returned the highest AUC values in agreement with the ROC curves depicted in Figure 1. For each AR crystal, the docking scores values corresponding to the considered SE thresholds are reported in Table 3.

Table 3.

Docking scores related to the applied SE-based thresholds (docking scores are expressed as kJ/mol).

PDB CODE	SE=0.25	SE=0.50	SE=0.75
2AM9	−34.76	−29.21	−22.66
2AX9	−32.90	−28.13	−22.63
2PNU	−37.72	−31.34	−25.67
3B66	−35.22	−30.47	−24.18
3L3X	−33.94	−29.47	−23.01
3RLJ	−34.53	−30.61	−24.10
5CJ6	−34.59	−29.32	−22.55
4QL8	−34.99	−31.05	−24.68
2HVC	−36.22	−31.51	−24.69

For each AR crystal structure, the docking scores whose rank are closest to the SE thresholds will be designated as values for compound classification.

In this respect, compounds belonging to EPA-ARDB and ranked above the threshold SE = 0.25 were considered to have a high probability of binding; on the other hand, chemicals ranked below the threshold SE = 0.75 were predicted to have a low probability of binding. The remaining compounds were designated as substances difficult to classify and were placed into two diverse classes of prediction with two different degrees of potential toxicity, namely warning (0.25 < SE ≤ 0.50) and suspicious (0.50 < SE ≤ 0.75), according to their docking score values.

The PPVs and NPVs for each SE threshold were calculated (see Table 4). PPVs range from 29.1 to 64.4 for the first threshold (SE = 0.25) and from 16.8 to 22.3 for the third threshold (SE = 0.75). In drug discovery programs, PPVs play a key role to detect potential therapeutically active small molecules. Applied to computational toxicology, the focus shifts towards the latest fraction of the ranked database (that is below the threshold SE = 0.75). The ultimate goal is the prediction of potentially harmful chemicals while minimizing the number of FNs. In other words, we could consider acceptable a classification model even if it returns a high rate of FPs (low PPVs) but with a low number of FNs (high NPVs) at the SE threshold equals to 0.75. This is especially true in a screening and prioritization context where there will be follow-up studies to validate the initial positive calls (binders in this case). Keeping this in mind, we observe that the NPV values range from 94.0 in the case of 5CJ6 model to 95.1 in the case of 2PNU model, thus indicating a high general capability to minimize FNs. It can also be noted that PPVs displays a larger gap (computed considering the maximum and minimum value across all the models) with respect to NPVs. This apparent discrepancy is strictly connected to the already discussed unbalanced nature of the EPA-ARDB (only approximately 12% of the compounds are binders), which makes it difficult to estimate of the best performing classification model. To overcome this issue, we exploited more informative statistical parameters: the positive (+LR) and the negative (-LR) likelihood ratio and the Balanced Classification Rate (BCR). These parameters are indeed independent of the data distribution within the starting training set.

Table 4.

Negative and positive predictive values computed at three SE-thresholds (SE = 0.25, SE = 0.50 and SE = 0.75).

PDB CODE	SE=0.25		SE=0.50		SE=0.75
	PPV	NPV	PPV	NPV	PPV	NPV
2AM9	64.4	91.1	34.7	93.2	17.8	94.6
2AX9	48.0	90.8	27.7	92.7	16.8	94.1
2PNU	42.1	90.3	29.5	92.5	22.3	95.1
3B66	33.1	90.3	27.1	92.5	18.5	94.4
3L3X	58.8	91.0	36.5	93.2	19.3	94.9
3RLJ	29.1	90.0	27.0	92.3	18.7	94.3
5CJ6	60.0	90.8	31.7	92.9	16.9	94.0
4QL8	47.5	90.7	34.3	93.0	19.8	94.8
2HVC	60.0	90.9	39.3	93.3	20.2	95.0

As reported in Table 5, the 2AM9 crystal structure (+LR = 14.33 at SE = 0.25) is potentially able to catch binders with a good accuracy for further pharmacological investigations. However, as already mentioned, for our toxicological virtual screening application, the classification model should minimize the FNs. Consequently, model performance can be evaluated by computing –LR at SE = 0.75. Building on this evidence, one can see that 2PNU is the best performing model, returning the lowest –LR equal to 0.38 at SE = 0.75. The BCR values are similar for all the models at SE = 0.25 (ranging from 0.17 to 0.19) and SE = 0.50 (from 0.43 to 0.46). Conversely, BCR ranges from 0.47 (2AX9 and 5CJ6) to 0.62 (2PNU) at SE = 0.75, hence again confirming, among the different models, the best performance of 2PNU.

Table 5.

Positive (+LR) and the negative (−LR) likelihood ratio and the Balanced Classification Rate (BCR) computed at three SE-thresholds (SE = 0.25, SE = 0.50 and SE = 0.75).

PDB CODE	SE=0.25			SE=0.50			SE=0.75
	+LR	−LR	BCR	+LR	−LR	BCR	+LR	−LR	BCR
2AM9	14.33	0.75	0.17	4.11	0.56	0.43	1.68	0.44	0.52
2AX9	6.97	0.77	0.18	2.92	0.60	0.45	1.53	0.48	0.47
2PNU	5.38	0.77	0.18	3.06	0.59	0.45	2.10	0.38	0.62
3B66	3.69	0.79	0.19	2.76	0.61	0.45	1.69	0.44	0.52
3L3X	11.09	0.76	0.17	4.39	0.56	0.43	1.83	0.42	0.56
3RLJ	2.99	0.81	0.19	2.70	0.61	0.46	1.68	0.44	0.52
5CJ6	11.04	0.76	0.17	3.48	0.58	0.44	1.53	0.48	0.47
4QL8	6.86	0.77	0.17	3.94	0.56	0.43	1.86	0.41	0.56
2HVC	11.55	0.76	0.17	4.93	0.55	0.43	1.92	0.40	0.58

From a methodological point of view, all the models could be ranked according to the sum of their ranking positions taking into account three performance parameters (NPV, BCR and -LR metrics at SE = 0.75) in order to clarify the best performing classification model).

As clearly shown in Figure 2, the classification model based on the 2PNU crystal structure ranks at the first position for all metrics. Consequently, we can conclude that 2PNU produced the best performing model.

Figure 2.

Summary of the crystals ranking obtained by the three statistical metrics (NPV, -LR and BCR) computed at SE = 0.75. NPV: Negative Predictive Value; -LR: Negative likelihood ratios; BCR: Balanced Classification Rate.

3.2. AD assessment using three validation sets

A key stage for the development of a predictive and robust classification model involves the validation step (Dearden et al., 2009; Mangiatordi et al., 2016). In this regard, we built three VS by extracting data from the DUD-E database. Following the computational details described in the Material and Methods section 2.2, all the compounds of each considered VS were first docked into the binding site of the selected 2PNU X-ray structure and then the AD two-step approach was implemented to assess its effect (see Material and Methods section 2.3).

Our attention was focused on the class of non-binding molecules set at SE>0.75, given our goal of minimizing the number of FNs. Within the non-binding class predicted by 2PNU model we detected 7.59% (VS1), 8.00% (VS2) and 8.56% (VS3) of binders thus indicating an almost random distribution (i.e., 10%). This disappointing result was reinvestigated by implementing the AD. Interestingly, the AD two-step approach had the effect of improving statistics significantly. As shown in Table 6, we observed a drop in the occurrence of the percentage of true binders (i.e. FNs) within the class of predicted non-binders (i.e., from 7.59% to 4.42%, from 8.00% to 4.59% and from 8.56% to 5.09% for VS1, VS2 and VS3 respectively) after applying both AD filters (VS – Bounding box/Convex hull). In other words, AD implementation simultaneously increased model reliability and model performance and reduced the FN rate.

Table 6.

Percentage of binders found at SE >0.75 in the class of predicted non-binders. Based on 2PNU model only (total VS), after the application of the first AD filter (VS – Bounding box) and after the application of both AD filters (VS – Bounding box/Convex hull) on the three VS.

	VS1	VS2	VS3
total VS	7.59%	8.00%	8.56%
VS – Bounding box	7.09%	7.51%	8.09%
VS – Bounding box/Convex hull	4.42%	4.59%	5.09%

The interested reader can look at Table S6 of the Supporting Information to inspect the amount of excluded compounds after the application of the first AD filter (VS – Bounding box) and after the application of both filters (VS – Bounding box/Convex hull) for all the three docked VS.

Figure 3 shows the projection of both EPA-ARDB and VS1 into the top two PCs obtained from the 162 descriptors previously computed for each compound of the EPA-ARDB. The perimeter of the polygon that defines the smallest convex area containing the top 95% of the EPA-ARDB compounds based on their closeness to the centroid is depicted in solid line. More specifically, a total number of 253 chemicals (9.76%) falls outside this polygon and thereby was excluded from the VS1. Additional details about VS2 and VS3 are reported in Figure S1.

Figure 3.

The projection of both EPA-ARDB and VS1 into the top two PCs obtained from the 162 descriptors computed for each compound of the EPA-ARDB. The outer polygon (dashed line) takes into accounts all the chemicals in the EPA-ARDB (black circles), while the inner polygon (solid line) retains the 95% of them. Chemicals of the VS1 (red circles) outside the inner 95% polygon are flagged as outside AD.

The performance of the previously selected classification model (2PNU) was assessed by evaluating how the percentage of binders decreases when moving from random selection (i.e., 10%) to the AD selection (see Table 6). In particular, the AD filtering effectiveness was assessed considering before (total VS) and after the application of the first filter (VS – Bounding box) and then adding also the Convex hull filter (VS – Bounding box/Convex hull). This allowed us to obtain the best performance in validation among different strategies used to define the AD (see Table S7 and the section “Supplementary methodological details” in the Supporting Information).

Taken together, these encouraging results make us confident about the high predictive power of the developed model. To the best of our knowledge, this work represents the first methodological attempt to integrate docking experiments and AD.

3.3. External predictions of twelve representative androgenic substances and of the CoMPARA blind set

To prove its real-life predictive power, our model was challenged with two additional tests. The first challenge consisted of the prediction of 12 reference androgenic substances. In this respect, two groups of chemicals were chosen: six drugs (R1881, nandrolone, ketoprofen, diclofenac, naproxen, and ibuprofen) and six compounds widely employed for industrial and commercial uses (bisphenol A, dichlorodiphenyldichloroethylene better known as p,p′-DDE, 2,2′,4,4′,5-pentabromodiphenyl ether better known as PBDE-99, methylparaben, propylparaben, and butylparaben).

R1881 and nandrolone are synthetic anabolic steroid analogues of testosterone. R1881, also known as methyltrienolone, binds the AR with a high affinity and therefore is used as a marker in prostatic tumors (Roediger et al., 2014); nandrolone and its ester form are employed for testosterone replacement therapy in hypogonadotropic hypogonadism and in AIDS-associated cachexia (de Souza and Hallak, 2011). Ketoprofen, diclofenac, naproxen, and ibuprofen are four nonsteroidal anti-inflammatory drugs (NSAIDs) mainly used in treatment of acute pain and chronic arthritis (Ezechiáš et al., 2016). Bisphenol A,(79–81) commonly abbreviated as BPA, is a key constituent of epoxy resins and one of the most common forms of polycarbonate plastic largely used in food containers and consumer products (i.e., water bottles for infants, kitchen utensils, and medical consumables) (Geens et al., 2012). p,p′-DDE, the main metabolite of DDT, is an organochlorine pesticide that bioaccumulates in the environment and directly interacts with the AR (Gray et al., 2006; Kelce et al., 1995). PBDE-99 is a flame retardant added to manufactured materials such as plastics, textiles, and surface finishes and coatings (Gray et al., 2006). The parabens are a class of parahydroxybenzoates or esters of parahydroxybenzoic acid with a broad spectrum of commercial applications such as personal care products, pharmaceuticals, and bactericidal and fungicidal preservatives (Rastogi et al., 1995).

Interestingly, none of these 12 chemicals violated the AD two-step approach and are thus considered inside the AD of our model. We compared the docking score of each of these representative compounds with the values approximating the SE thresholds of the 2PNU classification model (see Table 3). Figure 4 reports the progression of the docking scores for each compound of EPA-ARDB docked on 2PNU as a function of ranking. Note that the graph is split according to the SE thresholds and translated into four color-coded regions corresponding to four binding classes of compounds with a different degree of binding probability: (i) high probability of binding (red class); (ii) warning molecules (orange class); (iii) suspicious molecules (yellow class); and (iv) nonbinding molecules (green class). Based on these criteria, we placed the 12 selected substances in the four binding classes as depicted in Figure 4.

Figure 4.

Progression of docking scores for EPA-ARDB compounds based on the 2PNU docking-based classification model. Twelve representative chemicals were assigned to a given binding class according to their docking scores. The curve is color-coded on the basis of binding class: the hazard, warning, suspicious and safe class is in red, orange, yellow and green, respectively.

For the sake of completeness, the values of docking scores obtained for each representative compound are reported in Table S8.

From a structural point of view, it is not surprising that R1881 and nandrolone are predicted to be binders as both are structurally similar to testosterone. Furthermore, it is also known that the wide abuse of the ester form of nandrolone (specifically nandrolone decanoate) by athletes in order to enhance their physical performance could be associated with infertility and testicular toxicity (Ahmed, 2015). Next, all four of the anti-inflammatory drugs were classified as suspicious molecules. In this regard, a recent study (Ezechiáš et al., 2016) warns of the intrinsic endocrine disrupting nature of these pharmaceuticals that contribute to explain a wide range of adverse health effects on human reproductive systems. For instance, ketoprofen is predicted with the best docking score value among the other NSAIDs in accordance with greatest ability to interact with the AR (Ezechiáš et al., 2016).

In the second group of industrial chemicals, three out of six are predicted to be warning compounds. The European Union and Canada have indeed banned the use of BPA in baby plastic bottles and infant formula packaging (Fischnaller et al., 2016; Kolšek et al., 2015). Additionally the use of p,p′-DDE and PBDE-99 are strictly restricted in several pieces of legislation, as they are persistent organic pollutants (POPs) that act as competitive inhibitors of AR binding (Gray et al., 2006; Kojima et al., 2013; Stoker et al., 2005). The parabens are chemicals with weak potential endocrine activity. It has been demonstrated that their endocrine activity increases with the size of the alkoxy ester group (Darbre and Harvey, 2008). It is worth noting that docking scores increase as a function of the length of the alkyl group (butyl > propyl > methyl). In particular, butylparaben has a docking score very close to the threshold of the class of suspicious molecules, whereas methylparaben and propylparaben are classified as nonbinding chemicals. Accordingly, a study (Kolšek et al., 2015) has recently documented that these two substances do not have androgenic activity.

Finally, encouraged by successful performance on the three VS and on 12 representative androgenic compounds, the 2PNU classification model was employed in the CoMPARA AR activity prediction challenge using a blind set consisting of 55450 chemicals provided by US-EPA.

First, the AD two-step approach was applied to check whether the external compounds were inside/outside AD. Specifically, 6912 chemicals (12.46%) were excluded by the application of both of the AD filters. Figure S2 depicts the projection of both EPA-ARDB and external data set compounds into the top two PCs obtained from the 162 descriptors.

Finally, the remaining 48538 chemicals were screened using the selected classification model. Considering the classification criteria, 3073 chemicals were classified as binders (SE ≤ 0.25) and 27730 substances as nonbinders (SE > 0.75). We also discerned the intermediate subclasses with different degree of binding affinity of 6725 and 10275 chemicals corresponding to warning (0.25 < SE ≤ 0.50) and suspicious (0.50 < SE ≤ 0.75) compounds, respectively.

4. Conclusion

In the present study, we proposed an innovative and practical in silico protocol which exploits the use of molecular docking to derive highly predictive classification models able to identify potential androgenic chemicals. The best performing model was derived from a starting training set of high-quality androgenic experimental data and was successfully challenged against a large blind external set. We also implemented the use of AD to define the areas of the chemical space where model predictions can be considered reliable. The AD implementation had the effect of significantly increasing the reliability of the docking model. To the best our knowledge, this is the first investigation aiming at integrating AD and docking-based toxicology studies.

The success of molecular docking in distinguishing binder from nonbinder chemicals with a high level of confidence can pave the road to its acceptance as a valuable nontesting method for regulatory purposes alongside QSAR and read-across approaches. More importantly, docking results can intrinsically provide an easier and more comprehensive mechanistic interpretation and thus add a valuable plus to the model beyond its mere statistical validity. This indeed matches the expectations of the OECD principles explicitly claiming “a mechanistic interpretation, if possible” for the validation of (Q)SAR (Gissi et al., in press; OECD, 2007).