The Next Era: Deep Learning in Pharmaceutical Research

Abstract

Over the past decade we have witnessed the increasing sophistication of machine learning algorithms applied in daily use from internet searches, voice recognition, social network software to machine vision software in cameras, phones, robots and self-driving cars. Pharmaceutical research has also seen its fair share of machine learning developments. For example, applying such methods to mine the growing datasets that are created in drug discovery not only enables us to learn from the past but to predict a molecule’s properties and behavior in future. The latest machine learning algorithm garnering significant attention is deep learning, which is an artificial neural network with multiple hidden layers. Publications over the last 3 years suggest that this algorithm may have advantages over previous machine learning methods and offer a slight but discernable edge in predictive performance. The time has come for a balanced review of this technique but also to apply machine learning methods such as deep learning across a wider array of endpoints relevant to pharmaceutical research for which the datasets are growing such as physicochemical property prediction, formulation prediction, absorption, distribution, metabolism, excretion and toxicity (ADME/Tox), target prediction and skin permeation, etc. We also show that there are many potential applications of deep learning beyond cheminformatics. It will be important to perform prospective testing (which has been carried out rarely to date) in order to convince skeptics that there will be benefits from investing in this technique.

Introduction

We have previously suggested that cheminformatics should look to other industries that use high performance computing approaches for inspiration (1). Six years on what is surprising is that we may not have to look to industries but instead we already have used the algorithms which we should be considering for cheminformatics (and other areas) in our everyday transactions, work or social-life online. Every time we use our smartphones with voice recognition software like Siri or read the news on them, make a purchase on the internet via Amazon or use our social network software, we take for granted that we are confronted with suggestions of other products we might like to buy or friends to connect too. Large companies such as Baidu, Google, Facebook etc. all use deep learning in facial recognition algorithms alone. We also live in an age in which self-driving cars and robot assistants are emerging rapidly after decades of research. Therefore we are literally surrounded by artificial intelligence software that use machine learning (2) that can in many ways now predict our needs before we know what they are. Perhaps for the first time we may also be directly seeing a preview of how such software tools could assist us in healthcare related research and development. We just did not know it until now.

In the area of pharmaceutical research and development and specifically that of cheminformatics there are many machine learning methods such as support vector machines (SVM), k-Nearest Neighbors, Naïve Bayesian, Decision Trees etc. (3) which have seen increasing use as our datasets have grown to become ‘big data’ (4-7). These methods are equally applicable in other areas and come with their own pros and cons (3) that can be used for binary classification, multiple classes or continuous data. The application of different computational approaches and machine learning algorithms to problems tends to follow the growth of datasets (8). As pharmaceutical datasets started out quite small, the methods initially used would focus on local models like pharmacophores and quantitative structure activity relationships (QSAR). In more recent years the biological data amassed from high throughput screening and high content screens has called for different tools to be used that can account for some of the issues with this bigger data (6). The power of computer processing has also increased so that more complex non-linear problems can be solved in real time with relatively inexpensive compute resources. Many of these resulting machine learning models can also be implemented on a mobile phone (9, 10). In recent years, there has been increasing use of one approach called deep learning (DL), (which builds on many years of artificial neural network research)(11), that has shown powerful advantages in learning from images and languages (12). This may represent the next era of cheminformatics and pharmaceutical research in general that is focused on mining the heterogeneous big data which is accumulating using more sophisticated algorithms such as DL.

Delving into deep learning

Standard artificial neural networks (ANN) approaches use an input layer, hidden layer and output layer where each connection has a weight and these vary during training in order to connect input to output data. This method has been used widely but suffers from overfitting of data, and a poor ability to generalize with an external dataset (3), although more recent versions such as Bayesian regularized artificial neural networks are more difficult to overtrain (13). DL or deep neural networks (3) in many ways is similar to ANN in that it mimics how the brain works and takes information in an input layer but unlike ANN has many hidden layers (14) to combine signals with different weights, passes the results successively deeper in the network until an output layer (Figure 1). The DL model is trained with a dataset by adjusting the weights to give the response expected for a certain input (e.g. if a compound is active or inactive or the level of activity/inactivity). The ability to have multiple learnable stages makes this approach more useful for tackling more complex problems. Deep learning can be used for unsupervised learning and appears to work well with noisy data. However it still suffers from the potential to over fit data, the black box problem, as well as higher computational cost than ANN or other methods (15). There has been relatively limited application of DL to pharmaceutical problems to date and very few in the area of cheminformatics compared with other machine learning methods (11). DL tools are available in popular open source statistical software such as R (16). In addition there is TensorFlow (17), Deeplearning4j (18) while Facebook made their deep learning software (Torch) open source (19, 20) followed a year later by Microsoft (CNTK) (21). Some of these methods have been summarized in a recent review (22). It should be noted that these deep learning toolkits are likely far from ‘plug and play’ type software tools for the average scientist which they can input their molecules and data to train a model (or for that matter any training or test datasets) and then generate predictions. It is likely that expertise in using these software toolkits is needed as well as integrating with molecular descriptor software. It is more likely that a specialized programmer / statistician / cheminformatician will be needed with knowledge of the software tools in order to generate the models which can then be made available for others to use. Existing cheminformatics and other software companies could facilitate making deep learning more accessible to non-expert users by developing accessible fully integrated tools which they can use for any dataset.

Figure 1.

A schematic of a deep learning neural network applied to cheminformatics and a single property with output as a quantitative or qualitative prediction.

Applications of deep learning in bioinformatics

DL has seen a rapid increase in the number of publications associated with bioinformatics (23) and computational biology (22) and has been used in diverse applications (15) such as protein disorder prediction (24). This resulted in a fast approach which was comparable to other machine learning approaches based on area under the curve (AUC) and recall statistics for test datasets. While it did not out-perform the other disorder methods it had the advantage of being fast (24) (Table 1). DL has also been used to refine docked protein complexes (25) based on 35,000 unbound docking complexes generated by RosettaDock and tested on 25 docking complexes not in the training set. Although this model was not compared to other methods it resulted in RMSD of 1.40Å indicating accurate predictions (25). DL has also been used to model structural features of RNA-binding protein targets using CLIP-seq datasets and testing on 24 datasets using the area under the receiver operator characteristic to evaluate the performance which was found to be close to the state of the art (Table 1)(26). DL has been applied and compared with other methods for mechanism of action prediction from high content image analysis data (27) and it was found to be superior to SVM with (87.62 vs 20.95% accuracy). Although it should be noted that the processing time was ten times longer for DL which is likely a major limitation for learning versus other methods (27). The preprocessing time can likely be offset by parallelization of calculation processes.

Table 1.

Examples of deep learning studies. Deep learning (DL), Support Vector Machine (SVM) and artificial neural network (ANN)

Example end point modelled	Dataset size	Model statistics	Summary	References
Solubility	1144, 1026, 74, 125	0.92 RMSE 0.58, 0.91 RMSE 0.60, 0.81 RMSE 0.72, 0.67 RMSE 0.90 All 10 fold cross validation	Did not compare to other machine learning methods themselves. Addition of log P in some cases improved models.	(28)
Drug Induced Liver Injury	190, 475, 1065	80.5, 88.4, 70.1, Accuracies from internal cross validation	DL Models also assessed with external test sets. DL outperformed ANN with different PaDEL and Mold2 descriptors used.	(33)
ADME and target activity	Multiple models from 2092 (microsomes) to 318,795 (hERG)	Compared DL to random forest (RF) models on external datasets (11 of 15 DL models outperform RF models from Kaggle test and 13 out of 15 additional models)	15 Kaggle datasets available consisting of activities and descriptors.	(32)
Biomarkers	62,419 records and 46 blood markers	R2 = 0.80 and 82% prediction accuracy in predicting chronological age within a 10 year window.	Ensemble model performs better R2 = 0.82 accuracy 83.5%	(61)
Protein contact maps	ASTRAL database release 1.75	CASP 8 and 9 data is used for test sets. CMAPpro more accurate on the CASP 8 dataset than other methods. Improvement 10% or higher.	Predicts contact maps with accuracy of 30%	(62)
Cancer diagnosis	13 published cancer gene expression datasets	10 fold cross validation average classification accuracy 46.33 – 100%	Principal Component Analysis was used to preprocess data	(63)
Gene expression patterns	2000 genes from Allen Developing Mouse Brain Atlas	Average AUC 0.894 versus 0.82 for bag of words for annotating gene expression pattern. Gene ontology functional annotation average AUC = 0.59 vs 0.57 for bag of words	Used deep convolutional neural network. Relative performance of different methods differs across different developing stages.	(64)
Protein disorder prediction	1111 proteins	AUC 86.8 on training set AUC 80.9 on CASP10 dataset	More sophisticated methods such as DISOPRED3 and DNdisorder have test AUC 87.2 and 82.3 respectively	(24)
RNA binding protein features	24 datasets	10 fold cross validation used AUROC values 0.71 – 0.99	Outperformed the GraphProt method when using base sequence, secondary and tertiary structural profiles.	(26)
High content analysis of breast cancer	148,649 rows	Leave one out cross validation Accuracy 87%	Linear SVM Accuracy 20.95, SVM using radial basis function Accuracy 21.04 Uses a public dataset for training.	(27)
Epoxidation	702	Site of epoxidation AUC 94.9% and separation of epoxidation vs non epoxidation molecules with AUC 79.3% after leave one out cross validation	The DL was compared with a logistic regression model which gave Site of epoxidation AUC 93.7% and epoxidation vs non epoxidation molecules AUC 78.9%	(29)
Tox21	11,764 training set, 296 Leaderboard set, 647 test set	DL average AUC = 0.837, SVM average AUC = 0.832, RF average AUC = 0.803	DL with multitask learning out performed single task learning on 10 of 12 assays and DL won 9 of 15 challenges with nuclear receptor and stress response panels	(47)
Tox21	11,764 training set, 296 Leaderboard set, 647 test set	Consensus model Balanced accuracies from 0.599-0.903	Analyzed 12 targets using associative neural networks. Built consensus models. Training and leaderboard sets were combined. Reported to have the best balanced accuracy	(48)
Refinement of docked proteins	35,000 samples of 35 unbound dimer proteins	Tested on 25 test cases across 5 proteins - RMSD 1.40Å	No information on dataset availability
Drug repurposing	977 Landmark genes 271 signaling pathways	10 fold cross validation deep learning outperforms SVM for both datasets. DL F1 score of 0.70 and SVM 0.53 for pathway 3 class problem.	Deep learning models trained just on gene data did not perform well at classifying 12 groups of drugs. Pathways performed better.	(30)
Label free cell classification	Not defined	5 fold Cross validation shows 16 multivariate features can provide a balanced accuracy of 96.4%	Classification of blood cells (OT-II) and cancer cells (SW-480). Deep learning outperforms logistic regression, SVM and Bayesian but all accuracies are greater than 85%	(65)
Classifying microscopy images	103 treatments – 25 images per class for training	Test set accuracy 0.96	Used an available MFC-7 breast cancer imageset BBBC021v1	(66)
Lung tumors	130 patients	Improved mean square error by 29.98% and prediction overshoot by 70.93%	Computation time of 1.54ms might achieve real-time estimation of intra-fractional variation and better tracking for radiotherapy.	(67)
Target-ligand interaction prediction	sc-PDB, 836 targets and 2710 ligands	5 fold cross validation AUC 0.959. Out performs other published methods such as BLM-NII (AUC 0.858) and CS- PD (AUC 0.799)	Used a pairwise input neural network. % fold accuracy = 0.887	(68)

Pharmaceutical applications of deep learning

One of the earliest applications of DL to a pharmaceutically relevant problem was to predict aqueous solubility using four published datasets and was shown to compare favorably to other machine learning methods using 10 fold cross validation (28) (Table 1). DL has been used to predict the site of epoxidation in molecules with average AUC > 94.0% for cross validation, although this method was not compared to additional machine learning methods (29) or used for prospective prediction. DL has been put to use with gene expression data to learn from drugs and therapeutic categories using pathway level or landmark gene level as data reduction methods. In both cases after 10 fold cross validation deep neural networks surpassed SVM used internal testing, suggesting this as a drug repurposing approach (Table 1)(30). In the area of drug formulation, predicting drug release from poly-lactide-co-glycolide (PLGA) microspheres showed deep learning to be comparable to random forest, single tree and genetic algorithms after 10 fold cross validation (31). At Merck, deep neural networks have been compared to random forests for use with large QSAR datasets and out-performed random forests for 11 out of 15 datasets (used in a Kaggle crowdsourcing competition) and 13 of 15 datasets in a second evaluation using time-split test sets (32). This utilization by a major pharmaceutical company suggests there is serious interest in the approach and it is likely other companies may have already performed similar evaluations. More frequently DL is applied to a single dataset such as drug induced liver injury. In this case multiple training and testing sets were used and comparison with normal neural networks was performed showing slight improvement with DL (33). In addition, one model was tested with 6 datasets as a form of external validation. The accuracy of DL with DILI data was around 60% for these test sets which is comparable to what has been seen elsewhere using other algorithms (34). Based on a non-exhaustive assessment of several different end points relevant to pharmaceutical research, while it appears that most have seen utilization of Bayesian or SVM approaches to develop predictive models, few have so far utilized DL (Table 2). Recent examples of computational models appearing in this journal alone over the past 18 months include: modeling thermodynamic proxies (35), predicting mouse liver microsomal stability (36), predicting autooxidation (37), drug solubility in human intestinal fluid (38), site of metabolism prediction in CYP2C9 (39), human skin permeability prediction (40, 41), blood brain barrier penetration modeling (42), predicting clearance mechanism (41) and skin concentration due to dermal exposure (43). Many of these datasets could likely utilize and benefit from DL and it would be of interest to see for how many an improvement in predictions could be obtained.

Table 2.

Representative examples of machine learning models applied to pharmaceutically relevant end points to indicate areas used for machine learning that could be useful datasets for potential future use of deep learning. Other publically accessible datasets are available in OCHEM (69), Chembench (70), CDD (71) etc.

Example end point modelled	Naïve Bayesian	Support Vector Machine	Deep Learning
Solubility	(72)	(73)	(28)
Drug Induced Liver Injury	(34)	(74)	(33)
hERG	(75, 76)	(77, 78)	(32)
ADME	(36, 72)	(79)	(29, 32)
Blood Brain Barrier penetration	(75)	(77)
Biological Targets	(5, 80)	(81)	(32, 82)
Skin Permeability		(40, 41)
Transporters	(10, 83-85)	(86-88)	(32)
Mutagenicity	(75, 89)	(89, 90)
Formulation		(91)
Adverse event prediction	(92)	(93, 94)
Counterfeit drug detection		(95)
Docking		(96)
Small molecule pKa		(3)

The future of Deep Learning

While there has been recent exhaustive analysis of artificial intelligence and its impact on jobs, ethical considerations and geopolitical impact (44) there have been very few discussions of the potential for using DL in pharmaceutical research (14, 15). Based on the results obtained to date which admittedly have focused on internal validation with little prospective testing as seen with other machine learning methods (Table 1)(45, 46), DL appears promising and will likely see greater application in the years ahead. Perhaps the largest example of validation of DL models alongside other machine learning approaches is in the case of the Tox21 Challenge. DL with multitask learning (47) slightly outperformed the closest consensus artificial neural network method (48) across nuclear receptor and stress response datasets (Table 1).

It is yet to be seen if DL could facilitate the ultimate robot scientist (49), as we see application to different datasets, it or other machine learning methods may become an invisible research assistant, a tool that we take for granted to perform the predictions we need before performing the experiments. This may come with challenges such as how much power do we provide to the software to make the decisions for us. Which raises the question of whether the experiments need human involvement at all? The rapid development of DL outside science suggests it is far from a new fad, and the impact is already being felt in numerous areas from fraud detection to internet search engines. So how long before DL is widespread in pharmaceutical research (14) and what can we expect? It is possible that DL could be the source of more predictive models but hurdles remain on the implementation and accessibility of models. What is clearly needed is software that is tightly integrated with the data to be modeled. This data would most frequently reside in private or public databases and could represent many different endpoints both quantitative and qualitative (Figure 2). Therefore any efforts to bring the molecules, sources of data and DL algorithms together would greatly streamline model generation and make it more accessible to other scientists. But as with other computational modeling approaches we may also want to consider the applicability domain (50) and various factors such as the quality of the underlying data (51, 52) which may determine the utility and relevance of a DL model for making a prediction (53).

Figure 2.

Using public and private data for generating deep learning models for application across vast numbers of endpoints whether quantitative or qualitative (classification models).

A major concern would be how could DL shape research and the future of science and biomedical research in particular? It is possible that DL or any machine learning method might be able to assist by increasing the efficiency of research and perhaps rule out likely less successful avenues of research. This is especially important in areas where research funding is tight like rare (54) and neglected diseases (55). Any advantage that DL could provide would be welcome in these and other resource constrained areas as a side effect. Clearly we should be educating the next generation of pharmaceutical researchers to use a wide array of machine learning approaches as well as assessing the likely impact and application of DL. Developing scientists that can generate predictions in silico and test them in vitro or in vivo would also be welcomed. It may be only a short time before we have vice presidents of machine learning or DL in pharmaceutical and biotechnology companies. There are of course still many criticisms of these black box machine learning approaches but it is probably now accepted that with greater accuracy in prediction will come limitations in transparency.

While we have only just seen the beginning of the era of DL, we should be prepared for how it will be used and its potential impact on a wide array of potential pharmaceutically relevant endpoints (Table 2). Already comparisons of DL with additional machine learning algorithms have shown that it frequently improves upon the state of the art using predominantly cross validation as the form of evaluation. We will likely see improvements in DL and maybe even alternative approaches that are superior by combining with other machine learning or other methods or data as consensus approaches. This in particular is an area we have yet to see developed for pharmaceutical applications. If we are to imagine machine learning models being ubiquitous in the pharmaceutical industry, DL may facilitate that. We predict the near future will likely see an increase in studies published in this and other journals applying DL and comparing it prospectively and retrospectively to other machine learning methods for predicting various molecule properties important for pharmaceutical research.

While at the time of writing there are over 100 DL startup companies globally, few are focused on pharmaceutical applications (56, 57). We anticipate that this will be an active area as the DL connection will be one way to attract technology investors who would normally steer clear of drug discovery and pharmaceuticals as an investment. It is likely that researchers in academia and industry could immediately apply DL if it was more accessible in software and they could plug their data into it. In the interim it may require some collaboration with those more experienced data scientists using R and the various available DL toolkits. There are several areas that could perhaps see an immediate benefit, for example in the areas of predicting metabolism, interactions with P450s and transporters as well as other ADME/Tox properties there has been an increasing number of large datasets and computational machine learning models published (Table 2) some of which are non-proprietary. Big datasets like melting point with over 300,000 compounds (58), DMSO solubility with over 163,000 compounds (59), all of the ChEMBL (5) datasets and of course data in PubChem (60), would represent “Big Data” (14) that could be used with DL. The data from PubChem and to a lesser extent ChEMBL, may need curation and organization prior to use in DL models. While the focus of this perspective has been predominantly molecule centric, clearly the application of DL outside the pharmaceutical industry attests to the broad array of applications and potential for impact, there is likely overlap in these domains. For example could we learn from social media data using DL methods what might be potential side effects of drugs or even new uses identified by the population. DL could be applied not only to adverse event prediction but also formulation properties, pharmacokinetic simulation, cost effectiveness and even clinical trial simulation and design. How far DL will take us and how quickly it will have an impact on research and the industry remains to be seen, but there is considerable opportunity to develop user accessible tools and apply them now to the accumulating public datasets. DL is already off to a good start in many areas and pharmaceutical researchers would do well to take a closer look and embrace it faster than they have other computational technologies in the past.

Abbreviations

ADME/Tox

Absorption, Distribution, Metabolism, Excretion/Toxicology

AUC

area under the curve

DILI

Drug induced liver injury

hERG

human ether a-go-go related gene

PLGA

poly-lactide-co-glycolide

QSAR

quantitative structure activity relationships

SVM

support vector machines