Combining Computational Methods for Hit to Lead Optimization in Mycobacterium tuberculosis Drug Discovery

Abstract

Purpose

Tuberculosis treatments need to be shorter and overcome drug resistance. Our previous large scale phenotypic high-throughput screening against Mycobacterium tuberculosis (Mtb) has identified 737 active compounds and thousands that are inactive. We have used this data for building computational models as an approach to minimize the number of compounds tested.

Methods

A cheminformatics clustering approach followed by Bayesian machine learning models (based on publicly available Mtb screening data) was used to illustrate that application of these models for screening set selections can enrich the hit rate.

Results

In order to explore chemical diversity around active cluster scaffolds of the dose-response hits obtained from our previous Mtb screens a set of 1924 commercially available molecules have been selected and evaluated for antitubercular activity and cytotoxicity using Vero, THP-1 and HepG2 cell lines with 4.3%, 4.2% and 2.7% hit rates, respectively. We demonstrate that models incorporating antitubercular and cytotoxicity data in Vero cells can significantly enrich the selection of non-toxic actives compared to random selection. Across all cell lines, the Molecular Libraries Small Molecule Repository (MLSMR) and cytotoxicity model identified ~10% of the hits in the top 1% screened (>10 fold enrichment). We also showed that seven out of nine Mtb active compounds from different academic published studies and eight out of eleven Mtb active compounds from a pharmaceutical screen (GSK) would have been identified by these Bayesian models.

Conclusion

Combining clustering and Bayesian models represents a useful strategy for compound prioritization and hit-to lead optimization of antitubercular agents.

INTRODUCTION

Research targeted toward the identification of small molecule inhibitors of Mycobacterium tuberculosis (Mtb), the causative agent of tuberculosis (TB), has more recently focused on whole-cell phenotypic screening (1–6). Even though effective treatments have been approved for drug-sensitive infections, an urgent need exists for next generation drugs (7, 8) to address rising drug resistance for a disease that infects approximately one-third of the world’s population and kills 1.7–1.8 million people annually (9). Key to this effort has been research on new drugs that would significantly decrease treatment time of drug-sensitive TB from its current 6–9 month regimen (7, 8). Although many laboratories have screened libraries numbering 10³ – 10⁶ compounds (4, 5), the hit rate is usually below 1% (2, 3) as typically seen in many other high-throughput screening (HTS) (2, 3, 10, 11) campaigns for TB as well as other therapeutic indications (10–12). Occasionally the hit rate can reach the low single digits (~1.7–5%) (4–6). These TB HTS efforts are delivering interesting and potentially promising hits (albeit at great cost, Table 1), and in excess of a thousand actives may be deserving of follow-up. Hit-to-lead optimization must be properly balanced with continuing efforts to screen even larger compound libraries to more thoroughly cover chemical space and/or sample different experimental conditions to better mimic human TB infection (13).

Table 1.

Scoring recent Mtb hits from high-throughput screens with dual-event Bayesian Models (bold text = predicted active)

Name	Mtb Activity in vitro	Notes	Reference	MLSMR dose response and cytotoxicity Bayesian score	TAACF-CB2 dose response and cytotoxicity Bayesian score	TAACF Kinase dose response and cytotoxicity Bayesian score	Maximum Tanimoto similarity to MLSMR set using MDL Keys
Oxyphenbutazone	MIC 10μg/ml	Active against non-replicating TB Metabolite active against replicating TB-screened ~5600 compounds	(10)	−0.88	0.51	1.34	0.76
GNF-NITD 101	IC50 3.4μM	Active in hypoxic model, used to screen of 600,000 compounds	(2)	0.51	1.55	−4.93	0.68
GNF-NITD 82	IC50 9.3μM	Active in hypoxic model, used to screen of 600,000 compounds	(2)	−18.96	4.02	−4.63	0.79
GNF-NITD 46	IC50 7.4μM	Active in hypoxic model, used to screen of 600,000 compounds	(2)	−12.49	−1.32	−3.92	0.75
377790	IC50 0.5μM	From a screen of 20,000 compounds – Target DprE1	(3)	−3.69	2.77	3.87	0.81
A039	IC90 1.5μM	From a screen of 20,000 compounds	(3)	−6.02	1.72	−3.91	0.82
Gliotoxin	IC90 0.14μM	From a screen of 20,000 compounds	(3)	4.40	−2.20	−2.25	0.64
C215	IC90 16μM	From a screen of 20,000 compounds – target MmpL3	(3)	−17.44	1.48	−3.86	0.74
AU1235	MIC 0.1μg/ml	From a screen of 12,000 compounds – Target MmpL3	(53)	−8.24	−3.16	−8.07	0.76

The large number of hits for hit-to-lead optimization coupled with limited resources can benefit from established and highly efficient computational methods to expedite evolution of novel antitubercular lead compounds for clinical development. We and others (14–22) have suggested that computational approaches can assist in identifying compounds with activity against Mtb (20) and, in particular, Bayesian classification models are valuable (16–19, 23). More recently, we have described dual-event models that combine Mtb growth inhibition activity and cytotoxicity data to improve selection of actives with antitubercular activity (measured by IC₉₀ – the concentration of compound inhibiting bacterial growth by 90%) less than 10 μg/mL (or 10 μM depending on the original chemical library format and dataset) and a selectivity index (SI = CC₅₀/IC₉₀ where CC₅₀ = concentration of compound inhibiting growth of a cultured mammalian cell line, Vero cells, by 50%) greater than ten (22). We demonstrated using data from multiple laboratories that there are clear benefits of this approach: computational screening of 82,403 commercially available small molecules predicted 550 actives, which were assayed to identify 124 hits (22.5% hit rate) in one study (24), while another study computationally screened >13,000 molecules, assayed seven predicted actives and found five hits empirically (71% hit rate) (22).

We hypothesized that the Bayesian model technology could also positively impact hit-to-lead optimization. This phase of drug discovery is a significant driver of both process time and cost, typically entailing the design, synthesis, and biological evaluation of hundreds to thousands of compounds (25). A computationally-enhanced approach would expand the chemical space explored given the potentially unrestricted querying of commercial libraries for follow-up compounds and/or hit substituents derived from reactive building blocks. At the same time, this in silico approach could enable the efficient selection of a significantly smaller set of compounds for testing through the prioritization of analogs by their Bayesian score, which, in general, scales with the likelihood of activity. The following study describes the benefits of implementing Bayesian dual-event models in conjunction with the commonly used approach of hit structure clustering followed by the expansion of chemical space around core cluster scaffolds through commercial analog selections (26, 27) (Fig. 1). We also note how different mammalian cell types utilized in cytotoxicity determination can impact the rate at which active analogs are found.

Figure 1.

Schematic illustrating the integrated in vitro and computational processes described in this study

MATERIALS AND METHODS

Small Molecules

Small molecules for biological assay were purchased from Life Chemicals (Ontario, Canada) and ChemBridge (San Diego, CA).

CDD Database and SRI datasets

The development of the CDD TB database (Collaborative Drug Discovery Inc. Burlingame, CA) has been previously described (17). The Tuberculosis Antimicrobial Acquisition and Coordinating Facility (TAACF) and Molecular Libraries Small Molecule Repository (MLSMR) screening datasets (4–6) were collected and uploaded in CDD TB from sdf files and mapped to custom protocols (28). All of these Mtb datasets used in model building are available for free public read-only access and mining upon registration in the CDD database (18, 28–30), making them a valuable molecule resource for researchers along with available contextual data on these samples from other non Mtb assays. These datasets used previously for modeling are also publically available in PubChem (31). All data generated in this study (TB: ARRA) is available in the CDD TB database (Collaborative Drug Discovery, Burlingame, CA) (28).

Compound Selection and Clustering

Active compounds from previous H37Rv screens of the MLSMR, TAACF datasets and the kinase library from LifeChemicals (totaling ~4000 dose-response hits) have been clustered to identify common core scaffolds and analog series present among actives, as described previously (4–6). For cluster analyses a hierarchical clustering method implemented in LeadScope (LeadScope, Inc. Columbus OH.) was used applying default parameters. Clusters were separated using the ‘Complete Linkage (Furthest Neighbor)’ method with the cluster threshold distance set to 0.7. Each cluster may be characterized by a cluster scaffold that is a common core structure shared by all of its members. Clusters were also prioritized based on an enrichment ratio computed for each cluster, defined as the ratio of the percentage of compounds containing the cluster scaffold within the active (clustered) set and the percentage of such compounds within the entire library. High enrichment ratios are associated with structural motifs preferred among actives compared to primary screened compounds. Clusters with enrichment ratios below a specified threshold were excluded from further consideration; for the MLSMR and Chembridge datasets we used the threshold of ten and for the kinase library the more permissive value of five. We obtained 22, 29 and 26 conformational clusters corresponding to MLSMR, Chembridge and kinase library dose-response datasets, respectively. Out of these clusters, 30 cluster scaffolds were selected for follow-up by expanding variation around the core scaffold structures and potential SAR versus Mtb through the selection of analog series from commercial sources. Out of cluster scaffolds obtained from clustering dose-response hits from all three screens (MLSMR, TAACF datasets and the kinase library) totaling close to 4000 compounds, we selected 30 cluster scaffolds that were well represented among commercially available compounds. Analog compounds exploring chemical diversity around these 30 cluster scaffolds were selected from the Chembridge and Life Chemicals commercial libraries. Commercial compounds identical to any primary screened compounds were excluded except those that show inhibition of > 80% in primary screens but were not tested in dose-response. The final selection included 1847 compounds is described by the selected 30 major clusters out of which large clusters may be further grouped into sub-clusters, totaling 55 sub-clusters/clusters (or 30 major clusters). We also chose to add an additional cluster of 12 analogs (based on a designed ‘hybrid’ cluster scaffold) and 65 diverse compounds, totaling 1924 purchased from the aforementioned vendors.

Bacterial strain, growth conditions and media

Mtb H37Rv (ATCC 27294) was obtained from the American Type Culture Collection (Manassas, VA). To prepare permanent frozen stocks, H37Rv was grown as five mL subcultures (50 mL conical tubes, 36–37°C) in Middlebrook 7H9 broth (Becton Dickinson) supplemented with 0.2% glycerol (Becton Dickinson), 0.05% Tween 80 (Becton Dickinson), and 10% ADC enrichment (albumin, dextrose, catalase; Becton Dickinson). The subculture was mixed periodically and used to inoculate (5% inoculum) a second subculture (30 mL in 250 mL screw cap flask) when the turbidity reached a density similar to a #1 McFarland turbidity standard (A_{600 nm} ~0.2). The subcultures were incubated with periodic mixing for 18–21 days until the turbidity reached a #3–#4 McFarland turbidity standard (A_600nm ~0.6–0.8, 4–8×10⁷ CFU/mL). The caps on both the conical tubes and flasks were loosened and wrapped in parafilm to allow for adequate gas exchange and to prevent evaporation during incubation. Prior to harvest, samples from all cultures were spotted onto Trypticase Soy Agar (TSA) plates and incubated for 3–4 days to check for contamination. Mtb grows poorly on TSA which supports the growth of most potential contaminating microorganisms. Each culture was then transferred to a 50 mL tube and allowed to settle at ambient temperature for one h. The upper half of each culture was aspirated and pooled in a flask. Aliquots of one mL were then transferred to two mL cryovials and frozen at −80°C. At least three frozen stocks were thawed and used to determine the viable count by plating dilutions, prepared in supplemented 7H9 broth, onto Middlebrook 7H11 Agar followed by incubation for up to 21 days. A contamination check on the thawed cultures was also performed as described above.

Mtb Assay

Primary screening against replicating cultures of Mtb were determined using modifications to the microplate Alamar Blue assay (MABA (32, 33)) as previously published (4). This assay is widely used for HTS screening by many laboratories as recently documented (34). Antitubercular activity was determined against Mtb H₃₇Rv ATCC 27294 following 7 days incubation with test compounds. Compounds were evaluated initially in a stacked-plate dose response and final test concentrations for the compounds ranged from 100 μM to 0.0195 μM in two-fold dilutions with a final DMSO concentration of 1.0%.

Cytotoxicity in Vero Cells of compounds that inhibit Mtb

Cytotoxicity for Vero cells (ATCC CCL-81) was determined following 72 hours exposure (33). Cell viability was assessed using CellTiter-Glo reagent (Promega) according to the manufacturer’s protocol.

Cytotoxicity in THP-1 cells of compounds that Inhibit Mtb

This functional assay was developed for detection of compounds inhibiting THP-1 cells viability as a secondary screen to the Mtb bactericidal assay. The THP-1 cell line was chosen as a representative peripheral blood monocyte. In this assay, we treated THP-1 cells with compounds selected as “hits” in the Mtb assay over a 10 point 2-fold dilution series, ranging from 40 μM to 0.078 μM. Following 72 h of treatment, relative viable cell number was determined using Cell Titer Glo from Promega. Each plate contained 64 replicates of vehicle treated cells which served as controls. THP-1 cells were sub-cultured every 7 days in RPMI 1640 with 10% fetal bovine serum, incubated at 37 °C in 5% carbon dioxide. Cells were passaged as needed, harvested from flasks using 0.25% trypsin-EDTA and maintained for no more than 20 passages. Compounds and carrier controls were diluted in complete growth medium to prepare a 6× concentrated dosing solution which was dispensed into 384-well black clear-bottom tissue culture treated plates (5 μL volume). The final DMSO concentration for this assay was 0.4%. Cells were harvested as previously described. Twenty microliters of complete growth medium containing 3000 cells were dispensed per well. Plates were incubated at 37 °C, 5% CO₂ for 72 h prior to endpoint detection. At the end of the treatment period, assay plates were removed from the incubator and equilibrated to room temperature for 10 min. Twenty-five μL of Cell Titer Glo reagent was added and plates were incubated for an additional 10 min in the dark. At the end of the incubation, assay plates were analyzed using a PerkinElmer Envision microplate reader in luminescence mode with an integration time of 0.1 s.

Cytotoxicity in HepG2 Cells of compounds that inhibit Mtb

This functional assay was developed for detection of compounds inhibiting HepG2 cells viability as a secondary screen to the Mtb bactericidal assay. In this assay, HepG2 cells were treated with compounds selected as “hits” in the Mtb assay for 72 h over a 10 point 2-fold dilution series, ranging from 20 μM to 0.39 μM. Following the incubation, the relative viable cell number was determined using Cell Titer Glo (Promega). Each plate contained 32 replicates of vehicle treated cells which served as negative controls and 32 wells of 100 μM hyamine-treated cells that represent a positive control. The maintenance of the HepG2 cells followed the recommendations of the ATCC. Cells were passaged as needed, harvested from flasks using 0.25% trypsin-EDTA and maintained for no more than 20 passages. On the day of the assay, compounds or carrier control (DMSO) were diluted to 6× in complete growth medium supplemented with 1% Penicillin/Streptomycin and 5 μL was dispensed into 384-well black clear-bottom tissue culture treated plates using a Biomek FX. The DMSO concentration was maintained at 0.2% final concentration. A 10 point 2-fold serial dilution was generated in the “stacked plate” method previously published by this group (4). The HepG2 cells were harvested as previously indicated and the concentration was adjusted to 1.5×10⁵ cells/mL in complete growth medium supplemented with 1% Penicillin/Streptomycin. Using a Matrix WellMate in a certified biosafety cabinet, twenty microliters or approximately 3000 cells were dispensed to each well in the 384-well plate. The plates were then incubated at 37 °C, 5% CO₂ for 72 h prior to endpoint detection. Following the 72 h incubation period, the assay plates were equilibrated to room temperature for 10 min and twenty-five microliters of Cell Titer Glo reagent (Promega) was added to each well using a WellMate (Matrix, Hudson, NH). The plates were then incubated for an additional 10 min at room temperature. At the end of the incubation, luminescence was measured using a Perkin Elmer Envision microplate reader with an integration time of 0.1 s.

Biological Data Analysis

All data were imported into ActivityBase (IDBS) data management system for analyses and calculation of IC₅₀ and IC₉₀ values. Percent Inhibition was calculated as: 100× (1-(Median of Test Compound – Median of Positive control)/Median of Negative control – Median of Positive control)). Selectivity Index (SI) was calculated as SI = CC₅₀/IC₉₀, where CC₅₀ = concentration of compound inhibiting growth of cultured cells by 50%.

Using Dual Event Machine Learning Models with novel bioactivity and cytotoxicity data

We have previously described the generation and validation of the Laplacian-corrected Bayesian classifier models developed with cytotoxicity data to create Mtb dual-event models (22, 24) using Discovery Studio (16, 35–38). These models (22, 24) were developed based on: a. MLSMR dose response and cytotoxicity; b. CB2 dose response and cytotoxicity; and c. TAACF Kinase dose response and cytotoxicity, where cytotoxicity was determined in Vero cells for each set. All three models were generated using standard protocols using the following molecular descriptors: molecular function class fingerprints of maximum diameter 6 (FCFP_6) (39), AlogP, molecular weight, number of rotatable bonds, number of rings, number of aromatic rings, number of hydrogen bond acceptors, number of hydrogen bond donors, and molecular fractional polar surface area were calculated from input sdf files. Models were validated using leave-one-out cross-validation in which each sample was left out one at a time, a model was built using the remaining samples, and that model utilized to predict the left-out sample. Each model was internally validated and receiver operator characteristic (ROC) plots generated, and the cross-validated ROC area under the curve (XV ROC AUC) calculated. All models generated were additionally evaluated by leaving out 50% of the data and rebuilding the model 100 times using a custom protocol for validation, to generate the ROC AUC, concordance, specificity and selectivity as described previously (22, 24). The three models were used in this study to score a set of 1924 commercial analogs that expand the selected 30 major clusters (or 55 sub-clusters/clusters) obtained from cluster analyses of these screens. The set of 1924 compounds have been evaluated in dose-response in whole cell Mtb assay and Vero, THP-1 and HepG2 cytotoxicity assays. Defining non-toxic actives those that possess IC₉₀ < 10 μg/ml and SI > 10, we obtained 82, 81 and 52 non-toxic Mtb actives based on Vero, THP-1 and HepG2 cytotoxicity data sets, respectively. The prediction data were evaluated using a ROC plot and also with standard statistics (sensitivity, specificity, prediction accuracy and Matthews correlation).

Further retrospective evaluation of Dual Event Machine Learning Models

The previously developed dual-event Mtb and cytotoxicity models (22, 24) were further evaluated using a set of nine molecules collated from recent academic Mtb HTS studies (Table 1) as well as eleven hit molecules from GSK (Table 2) (40). These molecules were sketched using the mobile application Mobile Molecular DataSheet (Molecular Materials Informatics, Montreal, CA) (41, 42) to create sdf files which were used in Discovery Studio for prediction with the Bayesian models.

Table 2.

Scoring Mtb hits from GSK high throughput screen with dual event Bayesian Models (bold text = predicted active)

Molecule	Name	H37Rv MIC (μM)	MLSMR dose response and cytotoxicity Bayesian score	CB2 dose response and cytotoxicity Bayesian score	TAACF Kinase dose response and cytotoxicity Bayesian score
	GSK153890A	0.47	−10.79	−1.60	−1.99
	GSK163574A	0.76	−10.00	−2.27	−2.93
	GSK1589673A	0.25	26.39	3.33	−10.81
	GSK1829820A	0.19	12.03	3.24	4.94
	GSK1829731A	0.19	3.51	−0.46	−3.85
	GSK2200150A	0.38	−13.49	0.61	−10.36
	GSK353069A	0.13	−7.24	2.47	−9.18
	GSK358607A	0.7	−14.10	3.76	−10.96
	GSK749336A	0.25	−13.58	0.85	−10.21
	GSK888636A	0.94	0.10	1.07	−7.12
	GW623128	0.47	−9.78	−2.56	−2.66

RESULTS

Hierarchical clustering of actives from three previous antitubercular screens

Dose response hit compounds from three previous antitubercular screens (4–6) of the MLPCN (MLSMR), TAACF (Chembridge) and the Life Chemicals kinase libraries were pooled and clustered to identify common core scaffolds and analog series present among actives, as described previously (Fig. 2) (4–6). The pool of clustered compounds consisted of dose-response hits from the Chembridge and kinase libraries with IC₉₀ < 10 μg/mL and additionally we chose to include all dose-response hits from the MLSMR screen, totaling close to 4000 hits that were included for cluster analysis (see Materials and Methods). Within these 4000 hits 737 compounds satisfy the criteria of IC₉₀ < 10 μg/mL and SI (Vero cells) > 10, however we chose to use this larger, more inclusive set for the purposes of conformational clustering. The enrichment of cluster scaffolds among actives was assessed through the computation of cluster enrichment ratios, as described in detail under the ‘Materials and Methods’. Briefly, only cluster scaffolds represented as ‘enriched’ among dose-response hits compared to primary screening libraries were considered for the selection of commercial analogs. Based on the scaffolds shown in Fig. 2 a total of 1924 commercially available compounds were selected to explore the chemical space and Mtb SAR around these cluster scaffolds. As also shown in Fig. 2, large clusters have been grouped into sub-clusters, totaling 55 sub-clusters/clusters (or 30 major clusters, Table 3) and an additional cluster (Cluster 28) was also added based on a designed ‘hybrid’ substructure.

Figure 2.

Cluster scaffolds of MDR TB DR hits following hierarchical clustering as described in the Materials and Methods.

Table 3.

Number of cluster members listed for each cluster that contain at least one (or more) non-toxic active cluster member, defined as showing activity against Mtb. with IC₉₀ (averaged over three runs) < 10 μg/mL while possessing a selectivity index (SI) > 10 in any one out of the three cytotoxicity assays (Vero, THP-1, HepG2 cell lines). For each cluster the numbers of compounds are listed as follows: total number per cluster (‘Total’), dose-response actives with IC₉₀ < 10 μg/mL (‘DR-A’), and non-toxic actives as defined above (‘NT-A’). Cluster scaffold structures of the listed clusters are shown in Figure 2 Note, to avoid duplicate listings of the same compound, compounds that belong to multiple clusters are counted once, in the first listed cluster only.

Cluster	Number of compounds
	Total	DR-A	NT-A
1e	204	69	1
1i	98	17	1
2a	53	13	1
2d	20	5	1
2g	49	5	1
4a	9	2	1
4b	51	25	13
4c	34	13	9
4d	9	3	2
5a	16	9	4
5b	13	9	5
5d	9	3	3
5e	52	11	6
7b	17	2	2
7c	9	1	1
7d	55	8	7
8a	41	7	1
9b	61	9	2
10b	17	2	1
10c	42	12	2
11a	15	3	1
15	45	2	2
17	43	14	7
20	33	15	5
21	30	7	1
22	27	3	1
23	25	16	13
24	24	11	1
25	32	5	2
26	10	2	1

In vitro Screens for growth inhibition of Mtb and cytotoxicity

The selected 1924 commercial compounds were tested in vitro for growth inhibition of Mtb and cytotoxicity versus three distinct mammalian cell lines: THP-1, Vero and HepG2. For each cell line used for cytotoxicity assessment, SI values have been calculated as the ratio of CC₅₀ determined in each cytotoxicity assay and the antitubercular IC₉₀ activity (averaged over three runs). Defining non-cytotoxic Mtb actives as possessing IC₉₀ < 10 μg/ml and SI > 10, out of the 1924 commercial compounds 82, 81 and 52 molecules satisfy the criteria of non-cytotoxic actives in Vero, THP-1 and HepG2 cells, respectively (Table S1). These numbers represent ‘hit rates’ of 4.3%, 4.2% and 2.7% for the three cytotoxicity cell lines, retrospectively. The use of a more stringent efficacy and/or SI cutoff would naturally decrease the hit rate and be useful for exploring in hit evolution.

Bayesian Machine learning for hit-to-lead optimization

An alternate approach to expanding on the diversity of these ~4000 screening actives, while also seeking to enhance their antitubercular growth inhibition and SI values, relies on our recently published dual-event Bayesian machine learning. These models have been educated through learning which compound physiochemical and structural features are consistent with activity and promising SI. Importantly, the models have been validated through retrospective enrichment studies with published screening datasets as well as prospective prediction of actives from a GlaxoSmithKline antimalarial library (22). We have also utilized the dual-event models to score nine recently published hits from Mtb whole-cell screening campaigns (Table 1) that were derived after our dual-event models were built. Using the panel of three Bayesian models, we would have identified seven of nine of the molecules as actives. Interestingly, the maximal Tanimoto similarity using MDL keys and the MLSMR dose response and cytotoxicity model dataset was quite high (range 0.64–0.82) and yet the model only correctly identified two molecules. The TAACF-CB2 dose response and cytotoxicity model alone performed better with these molecules in this case and alone would have selected five of the nine compounds. Similarly, a second test dataset of 11 active molecules that were tabulated in a paper describing the HTS of two million compounds performed by GSK (40) against Mtb was analyzed, and at least one of the three models predicted eight of the compounds as hits (Table 2). These results highlight the need for further studies to comprehend what model factors influence predictive value, or whether the utilization of a consensus scoring approach with our dual-event Bayesian models could further enhance their ability to pick actives.

The dataset of 1924 molecules selected by clustering was virtually screened with our current three previously generated dual-event Mtb and cytotoxicity models. The molecules were ranked using the classification from all three models, and the receiver operator curve plot was generated (Fig. 3). The MLSMR dose response and cytotoxicity model appeared to perform the best at identifying the active compounds and scoring them highly. This is exemplified by the MLSMR and cytotoxicity model identifying ten active molecules (~12%) in the top ranked 20 molecules (~1% of the entire dataset) when using the Vero cell cytotoxicity dataset. With random screening of the molecules, we would have expected less than one active (at the 4.3% hit rate empirical hit rate with the Mtb and Vero cell screens) (11.8 fold enrichment). For THP cells nine actives were in the top 20 molecules (10.7 fold enrichment). For HepG2 cells six actives were in the top 20 molecules (11.1 fold enrichment), however there were also fewer actives in this cell line. The TAACF Kinase dose response and cytotoxicity model has shown enrichments from 6.7 to 11.1 fold in the top 1% while the CB2 dose response and cytotoxicity model consistently performed poorly in all cell lines (Figure 3, Tables 4–6). Based on earlier studies (17, 18) we have focused on how the models enrich the top ranked molecules (top 1%) as this would suggest that we could screen a much smaller fraction of a library. In other applications it is advantageous to consider a larger percentage e.g. the top 10%, as well as use multiple models for compound selection. We would still observe a considerable enrichment over random as approximately 50% of the actives are identified by the MLSMR and cytotoxicity and TAACF Kinase dose response and cytotoxicity models, while random screening would have delivered only 10% of the hits (5 fold enrichment, Figure 3). Retrospectively, out of the three Bayesian models developed based on the MLSMR, CB2 and kinase library dose-response screens and cytotoxicity counter screens we found that the MLSMR-based model predicted the identity of actives out of the 1924 compound set most accurately, outperforming the kinase library based model, while the CB2 dataset-based model did not perform much better than random selection. These findings are illustrated in Figure 3 and also reflected in the number of true positives and prediction accuracy (Tables 4–6). These findings also underline the usefulness of applying multiple computational models to predict activity/toxicity since in typical applications it is not known a priori which model may perform better.

Figure 3.

Results for the 1924 compounds tested from the ChemBridge and Life Chemicals libraries screened for whole-cell TB activity and predicted with dual-event Bayesian models shown as receiver operator characteristic curves. The random rate is based on the empirical HTS hit rate; MLSMR+cytotox is based on the MLSMR dose response and cytotoxicity model; CB2+cytotox is based on the CB2 dose response and cytotoxicity model. Kinase+cytotox is based on the MLSMR dose response and cytotoxicity model. The best curve is based on a 100% hit rate. A. Vero cells, B. THP cells, HepG2 cells.

Table 4.

Model statistics for predictions made with the set of 1924 follow up compounds using Vero cells for cytotoxicity.

Model	True Positive	True Negative	False Positive	False Negative	Sensitivity	Specificity	Prediction Accuracy	Matthews Correlation
MLSMR dose response and cytotoxicity	66	1294	548	16	80	70	70.7	0.2
CB2 dose response and cytotoxicity	56	695	1147	26	68	38	38.7	0.02
TAACF Kinase dose response and cytotoxicity	59	1208	636	21	74	66	65.8	0.16

Table 6.

Model statistics for predictions made with the set of 1924 follow up compounds using HepG2 cells for cytotoxicity.

Model	True Positive	True Negative	False Positive	False Negative	Sensitivity	Specificity	Prediction Accuracy	Matthews Correlation
MLSMR dose response and cytotoxicity	39	1297	575	13	75	69	69.4	0.15
CB2 dose response and cytotoxicity	36	705	1167	16	69	38	38.5	0.02
TAACF Kinase dose response and cytotoxicity	35	1211	661	17	67	65	64.76	0.11

DISCUSSION

With the advent of antibacterial screening and chemotherapy in the early twentieth century, rapid advances led to a variety of new antibacterial agents. Research from the 1940’s – 1960’s led to current tuberculosis treatments (e.g., streptomycin, isoniazid and rifampicin) via the design and synthesis of small numbers of compounds (100’s per program) and their assessment in in vitro and in vivo models. Unfortunately, disease-focused research often occurs in spurts depending on perceived public health threat, pharmaceutical market size, and available funding. This phenomenon is best exemplified by the hunt for a cure for tuberculosis, caused by one of humankind’s oldest pathogens – Mtb. With the rapid development of effective antitubercular agents, the notion developed that tuberculosis would be eradicated worldwide, and this perception led to a reduction in efforts to maintain the tuberculosis research infrastructure, particularly the capabilities needed to drive new drug discovery. Beyond the incredible magnitude of eradicating this disease worldwide, the realities of treating latent disease and more recently resistant forms of tuberculosis have strained the public health infrastructure and led to the realization that new sources of drugs will continually be needed in order to simply contain the disease. The acute need for new faster acting therapies not subject to current drug-resistant strains is being partially addressed through large-scale renewed screening efforts much like those established in the 1940’s. Due to the advent of modern technology and HTS, millions of compounds can and have been screened for antitubercular efficacy under different metabolic conditions representing models of various states of human infection. Specific target-based screening of large synthetic libraries was found to be a relatively ineffective approach to antibacterial drug discovery due to a variety of reasons including bacterial permeability (43). On the other hand, whole-cell phenotypic screens suffer from the disadvantage of being target agnostic making compound optimization and selectivity problematic. In spite of these issues, recent large-scale phenotypic (4–6) HTS and computational- assisted HTS (22, 24) screens against Mtb have identified thousands of potential hit compounds. Our challenge is now to follow up on these data in a timely and efficient manner as described herein.

Our prior studies have demonstrated that dual-event Bayesian machine learning models can enrich hit discovery (22, 24). Using public Mtb screening data as a whole (actives and inactives) can enable us to make more effective decisions to identify active compounds. Our Bayesian models also indirectly take into account both uptake and activity against a growth-relevant target, making use of positive and negative information. This empirical, activity-based approach derived from large sets of screening results may be a useful and rapid alternative to other methods for predicting bacterial permeability, such as MycPermCheck which requires five molecular descriptors to be calculated (44).

As we see an increase in academic-industry collaborations around HTS such as the TB Drug Accelerator (45) screening for compounds active against Mtb, it is likely that the number of hits in the literature will only increase. Efforts to follow up on these compounds will create a bottleneck, perhaps similar to what we have seen with the wealth of antimalarial screening data (46). Therefore, the approach (Fig. 1) we propose of using the dual-event Bayesian classifiers to assist in selection of follow-up compounds would seem a natural progression, learning from all the data generated previously. Considering the tight research budgets and likely reductions in government supported tuberculosis drug discovery, it may be in the best interests of the academic research community to more widely employ these proven computational methods that are used in pharmaceutical company drug discovery programs, in order to accelerate progress. The potential for sharing the Mtb models derived from published literature (as used in this and previous studies (22, 24)) could quickly impact these efforts.

As an example we have demonstrated that assessment of compounds suggested by four academic groups and GSK from the literature as Mtb hits represents (in the absence of their entire screening libraries) one way to determine whether the three Mtb Bayesian dual-event models would have classified them as actives (Table 1, Table 2). Seven out of the nine (78%) academic screening derived compounds were identified by at least one model (Table 1) and eight out of eleven (73%) compounds in the dataset from GSK (Table 2). While we do not have access to the complete screening libraries used by these groups (ranging from tens of thousands to two million compounds) to do a complete assessment, predictions on their published hits may be instructive. Extension of such retrospective analysis is likely optimistic but it does suggest the benefits of using multiple models likely to cover a broader chemical space. Frequently, we have seen multiple Bayesian models perform differently with different datasets (17–19, 22, 24) and the current study using nearly 2000 compounds selected by clustering, is no exception. The CB2 and cytotoxicity model performed better with the literature compounds (Table 1 and 2) than with the 1924 compounds derived from clustering (Table 4–6, Fig. 3). This result may be a reflection of the diversity of the respective training sets for each model (compared to the test compounds), and, as we have seen previously, one of the models performed well in selecting Mtb active compounds from a library of antimalarial compounds (22). At the very least this result suggests that large libraries of compounds screened against Mtb can be used to generate Bayesian models (that incorporate activity and cytotoxicity information obtained in previous screens) to improve the selection of compounds for subsequent screening sets that are enriched in non-toxic actives. Full release of the large GSK dataset of two million compounds should allow significant improvement of these computational models, but results for the currently available 11 compounds prominently described in the paper suggest the models are performing well for hit to lead optimization and in line with our own data from previous studies (22, 24).

Bayesian classification models have been applied for identifying antibacterials in retrospective testing with 1–2 fold enrichments (23) and thus could have broader applicability than just finding compounds active against Mtb. In addition, Bayesian classification methods have also been used for ADME/Tox models (36, 47–49). Thus, using Bayesian models for hit follow-up outside of Mtb is worthy of further exploration. Limitations of using such models based on whole cell data are that there is of course no information on a target or SAR for a target, although this may not be necessary for further pursuit of a lead.

Interestingly, data derived with different cell types for cytotoxicity does suggest the benefits of using more than one cell line, as different cells appear to have different sensitivities based on the variation in hit rates observed. Fewer actives were present in HepG2 cells than Vero and THP-1, respectively. We are not aware of any discussion of such differences with or without mechanistic underpinning, although others have used many cell types to derive general cytotoxicity models (49). It could reflect expression of metabolizing enzymes involved in molecule activation versus detoxification, transport differences (import/export) or other possibilities. Some have compared the use of cardiac, hepatic and kidney derived cell lines at predicting compounds specific to each organ and found similar cytotoxicity across all cell types (50). It should also be noted in this study that we have used models incorporating only Mtb activity and cytotoxicity and have not tried to directly account for absorption, distribution, metabolism and excretion properties. A panel of models for different bacteria using different cell types for cytotoxicity could also be helpful for scoring potential compounds for follow up, to understand selectivity versus broad spectrum action and multi-targeting. As illustrated in Figure 3 and also reflected in model statistics (Tables 4–6) the MLSMR and Vero cytotoxicity based model performs well in the selection of non-toxic actives out of the set of 1924 compounds, with a prediction accuracy close to 70% using each of the three cytotoxicity datasets. The model developed based on the kinase library performed close to 65% with each of the three cytotoxicity datasets. The CB2 model performed relatively poorly as applied to the set of 1924 compounds. Out of clusters represented in the 1924 compound set cluster scaffolds containing at least one or more non-cytotoxic active hits are listed in Table 3. While the CB2 compound set utilized in the development of the CB2 model was lacking cluster 1 members (and its sub-clusters), all other clusters were represented equivalently or better compared to the MLSMR or the kinase set. Out of the three sets, the kinase library set was most under-represented in non-cytotoxic active scaffolds listed in Table 3 and yet performed well for the prediction of non-cytotoxic active compounds out of the set of 1924. In the case of a new (unknown) set of compounds, it is likely the best results may be achieved through the application of all three models followed by pooling top scoring compounds from each model. Non-cytotoxic actives identified in this study are distributed over a number of clusters as shown in Table 1. Core scaffolds shared among cluster members are related among sub-clusters. Among the less desirable are clusters that contain many evaluated or active compounds but only one (or few) non-cytotoxic active(s) such as sub-clusters 1e, 1i or 2a, d, g.

From the screen of 1924 compounds there were 33 compounds that met the bioactivity and selectivity criteria for all three cell lines (Table 7). Twenty seven of these compounds had been predicted as active with the MLSMR dose response and cytotoxicity Bayesian model. Twenty two were predicted active with the CB2 dose response and cytotoxicity Bayesian model while 23 were predicted active with the TAACF kinase dose response and cytotoxicity Bayesian score. AB00952642 is the most active compound out of these based on the IC₉₀ (0.63 μM and 0.2 μg/ml). Noteworthy is the observation that AB00953420 and AB00953487 share the tetrahydropyrazolopyrimidine carboxamide common to potent antitubercular agents recently disclosed by both GlaxoSmithKline (51) and the Novartis Institute for Tropical Diseases (52) as well as resembling previous active compounds identified in our own laboratories (22, 24). Many of the other compounds in Table 7 also represent promising starting points for drug discovery optimization.

Table 7.

Compounds that were active and with selectivity index (SI) > 10 across all three cell lines along with dual-event Bayesian Model predictions (bold text = predicted active)

Molecule	Structure	H37Rv IC90 (μg/ml)	H37Rv IC90 (μM)	SI Vero	SI HepG2	SI THP-1	MLSM R dose response and cytotoxicity Bayesian score	CB2 dose response and cytotoxicity Bayesian score	TAACF Kinase dose response and cytotoxicity Bayesian score
AB00077575		0.58	1.54	68.85	34.42	65.63	27.55	−12.19	−9.52
AB00110214		0.32	1.07	126.05	63.09	113.31	−16.67	4.61	−9.40
AB00143367		1.77	4.14	22.62	11.31	22.62	6.18	−1.20	1.96
AB00145154		1.65	3.99	24.29	12.14	24.29	42.50	4.87	8.47
AB00285526		0.37	0.85	41.41	54.20	80.80	6.72	−3.88	8.86
AB00297140		1.46	3.85	27.46	13.73	11.83	28.99	6.43	6.64
AB00305092		0.58	1.35	15.11	34.78	45.34	7.08	−0.63	9.07
AB00310530		0.86	2.33	46.51	23.26	46.51	3.47	−1.17	2.53
AB00315662		0.87	2.07	18.51	22.94	40.09	11.92	1.96	13.48
AB00316014		0.20	0.60	27.02	102.56	35.83	−5.19	−5.51	−6.85
AB00370800		1.41	3.40	28.47	14.23	28.47	−10.68	−1.36	−17.21
AB00372473		1.36	3.42	15.13	10.24	29.33	−12.32	2.59	−18.66
AB00434956		1.40	3.46	28.59	14.30	28.59	36.74	1.97	3.17
AB00952522		1.66	4.01	13.36	11.43	14.07	25.22	3.18	23.26
AB00952642		0.20	0.63	20.69	22.86	15.64	−5.25	3.54	−2.56
AB00952825		0.56	1.47	70.96	35.46	70.92	22.70	4.06	16.70
AB00952828		0.91	2.34	11.45	11.07	31.47	24.98	5.21	15.53
AB00952834		0.93	2.52	13.37	21.55	34.55	29.02	5.43	17.76
AB00952835		1.18	2.73	33.82	16.91	33.81	22.72	3.01	16.36
AB00952911		1.88	5.52	21.27	10.64	21.28	24.79	0.21	18.01
AB00953015		1.73	4.24	23.14	11.57	23.14	35.05	−9.14	−10.05
AB00953018		1.10	2.55	36.53	18.26	25.26	−10.48	1.89	−10.40
AB00953098		1.62	3.90	24.77	12.38	24.77	10.87	0.55	6.15
AB00953127		1.77	4.90	22.62	11.31	19.12	24.04	−0.62	13.46
AB00953133		1.03	3.53	38.70	19.34	21.89	33.48	3.39	9.90
AB00953146		1.48	4.80	27.11	13.55	13.65	35.82	4.27	12.49
AB00953181		1.06	2.83	18.98	16.95	15.10	16.80	5.69	8.26
AB00953202		1.90	5.54	21.08	10.54	19.59	27.53	2.24	8.73
AB00953319		1.29	2.87	31.13	15.56	31.13	4.14	−0.49	6.27
AB00953326		1.26	3.89	31.67	15.84	13.96	43.38	3.47	11.73
AB00953420		0.40	0.91	99.34	49.63	95.50	28.98	3.16	−14.71
AB00953487		0.45	1.00	21.26	19.48	22.86	26.51	3.35	−6.36
AB00953603		0.75	2.46	50.90	26.53	28.24	35.19	1.18	15.42

In summary, we have shown how computational approaches such as hierarchical clustering and Bayesian models could be used to assist human decision making in hit follow up for Mtb. Three Bayesian models have been developed based on Mtb dose-response activity and cytotoxicity datasets obtained for three previously screened libraries. We applied these models retrospectively for the prediction of actives out of a set of 1924 commercial compounds. The latter set consists of commercial analogs exploring chemical diversity around cluster scaffolds obtained from conformational clustering of the three previously screened libraries. The set of 1924 compounds was evaluated for antitubercular activity and cytotoxicity in three cell lines resulting in the identification of 82, 81 and 52 non-cytotoxic active compounds (IC₉₀ < 10 μg/ml and SI > 10) using Vero, THP-1 and HepG2 cytotoxicity results, respectively. The selection of the 1924 commercial compounds was based on cluster scaffolds of dose-response hits from previous screens, followed by chemical diversity selection for clusters with large numbers of commercially available compounds. Compared to this strategy the current study demonstrates that the selection of such new sets of compounds may be achieved more effectively through the application of Bayesian models incorporating available antitubercular activity and cytotoxicity datasets. Multiple dual-event Bayesian models can increase the enrichment of non-cytotoxic actives in the top 1% of compounds to greater than tenfold and thus decrease the number of compounds purchased and tested. For example the application of our MLSMR model onto the 1924 compounds chosen by standard clustering (using the Vero cytotoxicity dataset) achieved an 11.8-fold enrichment of non-cytotoxic actives in the top 1% compared to random selection. Ideally the Bayesian models should be used prior to purchasing and testing of compounds to maximize the number of active compounds selected. Considering the likely limited budgets for purchasing follow up active samples for screening, this approach also allows virtual screening of even larger commercial databases and the purchase of a small, select set of compounds for follow-up that will be enriched in active compounds potentially leading to larger numbers of active compounds for mechanism of action studies than chemical diversity selection following conformational clustering alone. As further examples, the computational models can also be used to score compounds already identified by others and may be useful to triage the overwhelming number of hits and follow up screening set samples which themselves would consume valuable testing resources if they were all to be followed up.

Table 5.

Model statistics for predictions made with the set of 1924 follow up compounds using THP cells for cytotoxicity.

Model	True Positive	True Negative	False Positive	False Negative	Sensitivity	Specificity	Prediction Accuracy	Matthews Correlation
MLSMR dose response and cytotoxicity	62	1291	552	19	77	70	70.3	0.20
CB2 dose response and cytotoxicity	52	692	1151	29	64	38	38.7	0
TAACF Kinase dose response and cytotoxicity	53	1201	642	28	65	65	65.2	0.13