Comprehensive strategies of machine-learning-based quantitative structure-activity relationship models

Summary

Early quantitative structure-activity relationship (QSAR) technologies have unsatisfactory versatility and accuracy in fields such as drug discovery because they are based on traditional machine learning and interpretive expert features. The development of Big Data and deep learning technologies significantly improve the processing of unstructured data and unleash the great potential of QSAR. Here we discuss the integration of wet experiments (which provide experimental data and reliable verification), molecular dynamics simulation (which provides mechanistic interpretation at the atomic/molecular levels), and machine learning (including deep learning) techniques to improve QSAR models. We first review the history of traditional QSAR and point out its problems. We then propose a better QSAR model characterized by a new iterative framework to integrate machine learning with disparate data input. Finally, we discuss the application of QSAR and machine learning to many practical research fields, including drug development and clinical trials.

Graphical abstract

Data analysis in structural biology; Machine learning; Structural biology

Introduction

Machine learning (ML), with historical breakthroughs being made, has been widely used in Big Data and parallel computation applications such as image recognition, knowledge representation, robotics, autonomous driving, and drug development (Freitag, 2000; Krizhevsky et al., 2017; Kukar et al., 1999; Lecun et al., 2015; Cho et al., 2005). The latter is notoriously inefficient mainly owing to the high development cost (approximately US$2.6 billion for each newly approved drug), low clinical trial success rate (less than 12%), and low return on investment (Wenz, 1982). Computer-aided drug development (design, screening, and testing) may reduce the costs and increase the success rate and investment return (Cheng et al., 2012; Lu et al., 2006; Jain, 2017; Lill and Danielson, 2011; Mcinnes, 2007). Since the 1990s, techniques such as homology modeling, molecular docking, quantitative structure-activity relationship (QSAR) modeling, and molecular dynamics (MD) simulation have been used to research on drug activity mechanisms (Capener et al., 2000; Cavasotto and Phatak, 2009; Edwards et al., 2016; Ewing et al., 2001; Gombar et al., 2004; Hartmman et al., 2013; Kitchen et al., 2004; Krieger et al., 2003; Kwon et al., 2007; Lampi et al., 2010; Morris and Lim-Wilby, 2008; Ohashi and Tanaka, 2010; Rapaport, 2004; Rapaport et al., 2002; Thangapandian et al., 2013; Zheng et al., 2013). Artificial-intelligence (AI)-based big data analyses and high-performance computations have greatly improved the efficiency of drug development, especially in the drug discovery stage. More and more pharmaceutical companies are investing in AI technology. Currently, the value of the medical AI market is approximately US$700 million and is expected to grow at a compound annual rate of 53%, reaching US$8 billion by 2022. More than 35% of this AI market share is taken by drug discovery (Clancey and Shortliffe, 1985; Sondak, 1990). To improve the efficiency of drug discovery and increase the success rate of drug synthesis, many ML companies become specialized in serving pharmaceutical companies; their services include disease target identification, compound screening, de novo drug design, clinical image recognition (Secco et al., 2016), toxicity prediction, and the prediction of absorption, distribution, metabolism, and excretion (ADME) (Hou and Xu, 2002; Kassel, 2004; Li, 2001; Yang et al., 2004). A drug repurposed by the AI company Benevolent (Stephenson et al., 2019), for example, is now in the second phase of clinical trials and is being tested by Johnson & Johnson. However, it is important to be aware of the pros and cons of different AI systems, as they are optimized for specific purposes. By investing in different AI systems, pharmaceutical companies not only can engage multiple areas ranging from drug discovery to clinical trials but also may discover breakthrough treatments for complex diseases.

AI technologies are driven by new ML algorithms, advances in computational power, and ever-increasing experimental data (Bazoon et al., 2002; Pasquier and Hamodrakas, 2009). Advanced biotechnologies such as next-generation sequencing (Dijk et al., 2014; Kim, 2019), cryo-electron microscopy, high-throughput screening (Shin et al., 2016), medical digitization, and internet-of-things infrastructure (Adrian et al., 1984; Atzori et al., 2010; Cai et al., 2019; Dubochet et al., 1988; Feng et al., 2012; Gupta et al., 2009; Li et al., 2013; Song et al., 2019) provide high-quality Big Data for the application of ML-based drug development. ML algorithms can be roughly divided into three kinds: unsupervised, supervised, and reinforced (Ertel, 2017; Figueiredo and Jain, 2002; Kaelbling et al., 1996; Le, 2013; Piotr et al., 2006; Radford et al., 2015; Sahami, 1997; Sasakawa et al., 2010; Turian et al., 2010; Wang et al., 2019; Zhu and Goldberg, 2009). Unsupervised ML can be used to find hidden patterns in medical and biological data and to identify new disease targets (Bailey and Elkan, 1995; Wiskott and Sejnowski, 2002). Supervised ML can be used to predict drug activity, toxicity, and ADME from the existing data of drugs and clinical trials (Carneiro et al., 2007; Conneau et al., 2017; Igual and Seguí, 2017; Møller, 1993). Deep learning (DL), as advanced ML, has unprecedented power to scale up the capabilities of ML. AI is expected to become a major cost-effective, low-risk method in drug virtual screening.

QSAR, first established by Corwin Hansch (Hansch et al., 1962), was a natural extension of physical chemistry into the field of virtual drug screening. After more than 50 years of development with interdisciplinary breakthroughs and community promotion, QSAR has transformed from simple regression analysis (which can only handle similar compounds) to multiple statistics ML technique (which can analyze a very large data set of molecular structures). QSAR models have been widely used to model the biophysical properties of many chemicals (Hopfinger et al., 1997; Jaworska et al., 2005; Sabljić et al., 1995; Wold, 2010) and to assess the potential impacts of medicines, chemicals, and nanomaterials on human health and ecosystems (Alves et al., 2016; In et al., 2012; Karelson et al., 2010; Kim et al., 2015; Shin et al., 2017, 2018; Svetnik et al., 2003). For example, Lee et al. established a QSAR system called MS-HEMs to manage high-energy molecules (Lee et al., 2012). In the field of computer-aided drug design (CADD), QSAR has long been recognized as an effective method of structure- and knowledge-based drug design and optimization (Cramer, 2012). In this article, we review the merits, reliability, and limitations of QSAR to generate new insights into ML-based QSAR models, their integration with experimental or simulation data, and their potential applications (Zhao, 2003).

Structure-based drug design

The key to molecular targeted therapy is the discovery of lead drugs that can inhibit the targeted proteins, which usually entails the screening of a large number of small molecule compounds. Drug screening can be achieved by complex experiments (Larios et al., 2012; Lutz et al., 1996; Pan et al., 2005; Strasser et al., 2003), but their high cost and slow speed prevent high-throughput realization. CADD, through software such as GOLD, SYBYL, DiscoveryStudio, Autodock (Ash et al., 1997; Gao and Huang, 2011; Hashmi, 2007; Trott and Olson, 2010; Verdonk et al., 2010; Wang et al., 2015a; Yang et al., 2011), can rapidly search common libraries (e.g., proprietary libraries, Maybridge commercial library, and Food and Drug Administration drug library) and perform virtual screening (molecular docking and binding free energy evaluation) (Liu, 2016), which are characterized by high speed and high throughput. Compounds with high priority scores are selected for further screening by cellular experiments, animal experiments, or clinical trials, which may finally give rise to lead compounds.

CADD is a structure-based drug design method. Biological/chemical properties, no matter static properties (e.g., chemical composition) or dynamical changes (e.g., the conformational changes of DNA with different ethanol concentrations (Fang et al., 1999)), are fundamentally determined by molecular structure, which underscores the importance of studying structure-property relationship. Molecular mechanics (MM) (Humphrey et al., 1996; Phillips et al., 2010) and quantum mechanics (QM) (Csányi et al., 2004; Habasaki and Okada, 2006; Kwangho Nam and York, 2008; Mochizuki et al., 2007; Noel et al., 2010) are two approaches of MD aiming at the accurate computation of the moving trajectory of every atom in a biomolecule until the movement is stabilized, reaching the stable (low energy) conformation. Owing to the large number of atoms in a biomolecule, an MD simulation is usually computational costly. To expedite drug screening, less accurate but computationally efficient MM force field, such as Merck Molecular Force Fields (MMFF), are generally used (Bret et al., 2000; Cheng et al., 2000; Cieplak et al., 2009; Giese, 2005; Halgren, 2015; Tu and Laaksonen, 2001). Structure-based drug design mainly includes receptor-based methods (molecular docking, de novo design, MD simulation, homology modeling) (Cho et al., 2015; Nam et al., 2003, Nam et al., 2011; Semper et al., 2021) and ligand-based methods (QSAR, pharmacophore, substructure search) (Anderson, 2003; Bohacek et al., 1996; Chang et al., 1992; Greer et al., 1994; Shim et al., 2014; Ma et al., 2012; Verlinde and Hol, 1994). They all involve the interaction between the receptor and ligand, but their focuses are different (Figure 1).

Figure 1.

Classification of drug design methods

Receptor-based methods are mainly based on the three-dimensional structure of the receptor to find a matching ligand. To treat a disease by targeting some human receptor protein, a large number of small-molecule compounds need to be screened by molecular docking or de novo design to find the ones that fit well with the crystal structure of the protein (Butterfoss and Kuhlman, 2006; Degrado, 1997; Lichtenstein et al., 2012). If the crystal structure is unavailable, the structure can usually be estimated by homology modeling, namely, by constructing the human protein structure based on the corresponding structure of another species and the amino acid sequences of both (which only have small differences). Although receptor-based methods are dominant in drug screening, they still have great limitations. First, these methods rarely consider factors such as protein flexibility, the influence of water molecules, solvation effects, and the conformational limitations of the ligand. At present, most molecular docking procedures have certain defects such as the inability to correctly deal with the induced coincidence effect, the solvation effect, and the poor ranking ability of the scoring function. Second, although the efficiency of molecular docking is relatively high, its speed is far from enough for a million-level compound library. Finally, the crystal structure of some proteins has not been solved for any species, making ligand-based methods infeasible. For example, the membrane proteins are difficult to purify and crystallize owing to hydrophobicity.

Ligand-based methods can be used to circumvent the aforementioned problems because they are in principle independent of the receptor protein. They start from a known effective ligand to discover the substructures or structural characteristics that are genuinely responsible for the drug efficacy, which can then be used to guide the selection or design of the analogs of the ligand. In brief, the methods predict new ligands based on the features extracted from a known active ligand. These new ligands are candidate drugs and need to be tested further. Even if the crystal structure of the corresponding receptor is unknown for all the species, the ligands can still be tested by in vitro experiments (e.g., the measurement of binding free energy, pk(a), logP, logD, and other indicators (Bash et al., 2002; Culler et al., 1993; Laitinen et al., 2010; Xing and Glen, 2010)), making the ligand-based methods always useful. Ligand-based methods mainly include pharmacophore-based methods (Kim et al., 2017; Kurogi and Guner, 2001; Lee et al., 2013; Yang, 2010) and QSAR model prediction.

Pharmacophore

Pharmacophore refers to the common characteristics of all the small-molecule compounds actively interacting with the target protein. There must be specific sites in the receptor for small molecules to bind; thus, the ligands binding to the same receptor must have similar chemical substructures. The International Union of Pure and Applied Chemistry (IUPAC) defines pharmacophore as “an ensemble of steric and electronic features that is necessary to ensure the optimal supramolecular interactions with a specific biological target and to trigger (or block) its biological response.” The pharmacophore model uses not only the topology similarity but also the functional similarity of the molecule, allowing for the use of the concept of bioisosterism to make the model more reliable (Wolber and Sippl, 2008). In CADD, the pharmacophore model is mainly applied in three areas:

• a)Structure-activity relationship, which is established through the discovery and definition of key pharmacodynamical characteristics of the drug molecules.
• b)Scaffold hopping, which is established through the discovery of compounds with novel core structures by modifying the central core structure of a known active compound (Jang et al., 2016).
• c)Target fishing, which is established through the prediction of targets of a given compound according to the compound’s pharmacophore characteristics. This also allows for the identification of the potential off-targets (side effects) of the compound so that its candidacy as a lead compound may be eliminated early in time.

To obtain an accurate pharmacophore model, the correct three-dimensional (3D) structure of the compound must first be identified, which necessitates the careful inspection of valence, bond order, protonation state, tautomerism, and stereoisomerism. Although pharmacophore deserves further development, their practical application still faces many difficulties. A pharmacophore is receptor specific and may become useless when the drug target is different from the receptor, even if the difference is small. A pharmacophore is also sensitive to the change of its parental chemical structures. That is, a slightly different chemical may not match the pharmacophore. The matching difficulty is determined by the number of features and the tolerance of the pharmacophore.

Quantitative structure-activity relationship

QSAR is an empirical mathematical model in which regression (Doo Ho Cho and Bum Tae Kim, 2001) and classification (Choi et al., 2010; Choi et al., 2009; Kim et al., 2006a, 2006b; Kim et al., 2008; Lee et al., 2017; You et al., 2015) is performed on many structure-property data to reveal statistically significant correlations between chemical structures and biological properties (Figure 2). A QSAR model can thus predict a new chemical's biological/toxicological properties based solely on the chemical's structure, i.e., without resorting to the time-consuming molecular docking, which makes both the training and application of QSAR highly efficient. This does not necessarily mean that the receptor information cannot be incorporated. Because the structure of a protein is determined by its amino acid (AA) sequence, the structure and activity of the receptor can be taken into account by introducing the AA sequence information into the QSAR model, which may markedly increase the accuracy of model prediction. In summary, QSAR is promising in drug development because it can process a large amount of compounds with high speed and without losing much precision.

Figure 2.

Quantitative structure-activity relationship

(A) The conception of QSAR.

(B) Understanding of QSAR from the perspective of machine learning.

QSAR has received an increasing number of applications in recent years, including drug design, drug toxicity prediction (Wu and Wang, 2018), the study of enzymes' chemical-biological interaction mechanisms, and the prediction of compounds' biological activity. QSAR is generally based on the following three methods: molecular description, chemical similarity search, and ML. They are described in the following.

QSAR: methods based on molecular descriptors

The core of QSAR lies in the acquisition of molecular descriptors (also called chemical descriptors), that is, the extraction of information-rich numerical features from chemical structures (Joung et al., 2012; Kim et al., 2009; Nilakantan et al., 1987; Park et al., 2013; Randić, 1993). The past 50 years has witnessed different types of molecular descriptors, which can be divided into quantitative descriptors (molecular field, molecular shape) and qualitative descriptors (daylight fingerprints, MACCS keys, MDL, Public keys). According to the data type, molecular descriptors can be divided into Boolean (e.g., chiral or not), integer (e.g., the number of rings), real (e.g., molecular weight), vector (e.g., dipole moment), tensor (e.g., electronic susceptibility), scalar field (e.g., electrostatic potential), and vector field (e.g., electrostatic potential gradient). According to the physical meaning, molecular descriptors can be divided into composition descriptors, molecular property descriptors, topological descriptors, and geometric descriptors. According to the structural dimension, molecular descriptors can range from 1D to 6D. Molecular descriptors can further be divided into experimental descriptors (logP, molar refractivity, dipole moment, polarizability, etc.) and theoretical descriptors (e.g., atom number, molecular weight, atom-type count, etc.). These different types of molecular descriptors are widely used in molecular data mining, compound diversity analysis, and compound activity prediction.

2D-QSAR

It is intuitive and concise to distinguish molecular descriptors according to the structural dimension (Azari and Iranmanesh, 2015; Morell et al., 2005; Ivanciuc and Braun, 2006; Karelson et al., 1996; Sandberg et al., 1998; Sheridan et al., 1996). One-dimensional descriptors are scalars that describe aggregated information, such as atom count, bond count, molecular weight, atomic properties, and fragment count (Shimamoto, 1999). Although simple, 1D descriptors are rather degenerative—disparate compounds may have the same value. Therefore, a 1D descriptor is usually used in addition to other high-dimensional descriptors, or combined with other 1D descriptors to form a vector.

Most molecular descriptors in the literature belong to 2D, which defines the connectivity of atoms in a molecule according to the properties of chemical bonds, including topological index, molecular profile, 2D autocorrelation, and chemical fingerprints (Hetényi et al., 2006). Two-dimensional descriptors generally deliver simple and useful structural information, which is rotation and translation invariant and is also invariant when the local structure is optimized. Another important invariance is graphic: the descriptor value does not change with the renumbering of the graphic nodes (vertices), which is very useful for structural differentiation. To facilitate the analysis of the large amount of 2D descriptors, Hong et al. reported the Mold² system, which can quickly generate 200 types of 2D descriptors for large-scale composite data sets (Hong et al., 2008). Other commonly used commercial software for the generation of 2D descriptors includes the dragon system, which can generate up to 5000 2D molecular descriptors (Mauri et al., 2006). More detailed information about 2D descriptors can be found in the studies by Durant et al., 2002; Duvenaud et al., 2015; Lo et al., 2018; Polanski, 2009; and Rogers and Hahn, 2010.

3D-QSAR

3D-QSAR facilitates the study of QSAR by introducing the 3D structural information of the drug molecule (Cho et al., 2012; Choi et al., 2006). Compared with 2D-QSAR, it has clearer physical meaning and richer information and can better reflect the nonbonding interaction characteristics between the drug and target molecules. Since 1980, 3D-QSAR has gradually replaced 2D-QSAR and become one of the main methods of mechanism-based rational drug design.

The 3D-QSAR chemical descriptors include autocorrelation descriptors, substituent constants, surface and volume descriptors, quantum chemical descriptors, and unique molecular scaffold (Todeschini and Consonni, 2009). These 3D chemical descriptors are useful to derive new drugs from an existing one through scaffold hopping. That is, replacing the existing drug's scaffold with others without significantly affecting the drug's binding activity. The chemical descriptors are obtained by extracting chemical features of the molecules from their conformations (Lipkowitz and Boyd, 2007), which can generally be obtained from crystal structures or from quantum chemistry computation by programs such as CORINA (Gasteiger et al., 1990). The criterion for the determination of molecular conformations is usually the lowest free energy. However, the actual conformation of the molecule corresponds to low energy but not necessarily the lowest one, and a molecule can switch between multiple conformations. Relevant experiments have confirmed that the in vivo active conformation of a drug molecule is generally in a low energy state but not the lowest. As such, it is not guaranteed that the predicted bioactivity can be verified by experiments. These complexities represent the key limitations of generating 3D chemical descriptors.

Commonly used 3D fingerprints include chemical characteristics based on pharmacological patterns, surface properties, molecular volume, and molecular interaction fields, of which the most famous one is based on molecular interaction field (MIF) (Hayakawa et al., 2020), which was realized by Goodford in the GRID program (Goodford, 1985). The MIF fingerprint is obtained by placing the ligand in a rectangular grid with fixed intervals to calculate the fingerprint characteristics of each grid point. In other words, the electrostatic force, 3D coordinates, hydrophilic (hydrophilic) and other dimensions are calculated independently for each grid point. The obtained MIF fingerprint can then be used in a specific 3D-QSAR model to predict the activity relationship between the molecules comprising the complex. More detailed information about 3D-QSAR models can be found in the studies by Baskin and Zhokhova, 2013; Cramer et al., 1988; Datar et al., 2006; Dixon et al., 2006; Gerhard Klebe and Mietzner, 1994; Gohlke and Klebe, 2002; Hopfinger, 1980; Low and Vinter, 2008; Simon et al., 1977; Varela et al., 2012; Veselovsky et al., 2001; and Walters and Hinds, 1994.

Higher dimensional QSAR

Three-dimensional QSAR is computational costly, and its performance is sensitive to the changes in the conformation/orientation of the ligand. To overcome these drawbacks, higher dimensional QSAR models have been developed. Four-dimensional QSAR has solved the conformation/orientation problem by simultaneously considering multiple structural conformations (Andrade et al., 2010). For example, Ash and Fourches calculated a 3D descriptor based on a 20-ns MD simulation trajectory of the atoms of Erk2 (Ash and Fourches, 2017). This 4D (3D space + 1D time) chemical descriptor can effectively distinguish the most active Erk2 inhibitors from those inactive and highly enriched Erk2 inhibitors. Five-dimensional QSAR takes into account factors such as receptor flexibility and inducible fit (Vedani and Dobler, 2002). Six-dimensional QSAR takes into account an additional factor, namely, the effect of solvation on the main receptor-ligand interaction (Vedani et al., 2005).

Search based on compound structure similarity

Chemical similarity search is one of the most popular techniques for drug discovery based on ligands (Willett et al., 1998). It aims to query similar compounds with known active molecules in terms of structure from a database. The basic assumption is that compounds that are similar in function have similar chemical structures. However, this assumption is not always accurate. For example, the "active cliff" effect (Hu and Bajorath, 2020), a situation in which a small modification to a functional group leads to a sudden change in activity, does not meet the aforementioned assumptions and may lead to failure. To evaluate the structural similarity between two molecules, the Tanimoto coefficient (Tc, known as the Jaccard index), a measure of the similarity between two compounds, was proposed (Lipkus, 1999). A higher Tc score indicates that two compounds are more similar, but Tc does not provide details regarding what kind of chemical group they share. More detailed information about 3D chemical similarity can be found in the studies by Bero et al., 2018; Cheeseright et al., 2008; Dossetter et al., 2013; Ferreira and Couto, 2010; Lo et al., 2016; and Rush et al., 2005.

Further considerations

The success of QSAR lies in the fact that the difference in the representation of the ligand structure truly represents the essential difference in the scope of the ligand. For example, the difference in bioactivity of two compounds is solely determined by their structural difference, no matter how complex the subsequent physicochemical and biological interactions are. The ultimate goal of ligand comparison is to maximize the space of positive and negative samples. If the developed QSAR models are expected to be useful for drug synthesis in addition to drug screening, properties need to be considered such as synthesizability, hydrophobicity, drug-likeness, Lipinski rule, and false-positive issues (Alam and Khan, 2017).

Results

QSAR models based on ML

The ML-based QSAR model currently plays a notable role in drug design and screening, property prediction, category prediction, etc. Since the 1990s, ML techniques such as support vector machines (SVMs) and random forests (RFs) have been widely used to discover or design new medicines. For example, its application to a scoring function based on molecular docking has been reported (Lima et al., 2016). Data mining based on chemical graphs can derive a set of two-dimensional or three-dimensional chemical descriptors, which are packaged into chemical fingerprints in various ML models and prediction tasks. A key innovation in this field is the combination of big data and ML, including the mining of gene sequences and protein structure, single-cell sequences, multi-omics interaction data, and the interaction between proteins and genes, which enables the QSAR model to learn and predict a wider range of biological laws.

In the past, QSAR models focused only on the predictive ability of candidate compounds; thus, the commonly used ML models are discriminative prediction models. ML methods are mainly divided into discriminant prediction models and generative models (Jebara and Pentland, 2001). The essence of the discriminative model is to learn the mapping relationship between the characteristics of the characterizing sample and the prediction target from the data, and it is related to pattern recognition, which means that the model can only discriminate and predict. The generative model directly learns the distribution of data from the sample. In addition to being combined with the discriminant model, the generative model can also directly generate new samples based on the learned distribution and is thus used to generate candidate compounds. The methods to learn patterns hidden in data are mainly divided into supervised learning, semisupervised learning, unsupervised learning, and self-supervised learning. Because the QSAR model focuses only on the predictive ability of a model for candidate compounds, we introduce only the discriminant models commonly used under supervised learning. The key to this type of model is the selection of features, the selection of the algorithm itself, and the quality of the training set. Most aspects of feature construction have been included in the chemical descriptor section of the previous chapter. Examples of the extracted features can be found in the literature (e.g., Tomal et al., 2016).

Conformation-related indicators have been taken into account in the QSAR, and conformational calculations for isomers can be calculated using quantum chemistry, semiempirical methods, and molecular dynamics simulations. Because the relative stability of conformational isomerism is related to toxicity detection, the conformation of the compound may change as the solvent environment changes, which affects the three-dimensional correlation characteristics and then impacts the final screening result. Therefore, the methods of calculating the isomer conformations can be used to predict whether the relative stability of the conformation will affect the experimental results. In addition, the ML-based QSAR method can be applied to examine the energy changes between conformational changes, to determine how the conformation is affected by the solvent environment, and to predict the relationship between conformational changes and molecular energy (Lokuge et al., 2010; Singh et al., 2016).

The importance of these features varies in different structure-activity or attributes prediction tasks, but ML can learn from the data which features are more important. Therefore, we need only to exhaust all the features related to the nature of QSAR tasks as much as possible and apply the traditional ML algorithm to learn from the training set.

ML-based QSAR methods are effective under the usual condition that the system under study is so complex that it cannot be physically modeled. Indeed, many physical and chemical properties are difficult to calculate by theoretical methods such as density functional theory (DFT) or MD, but they can usually be computed by cheminformatics models, particularly the water solubility and logP (the logarithm of octanol: water partition coefficient), which are directly related to drug activity. The properties that are indirectly related to drug activity, such as the melting point, solubility, and sublimation energy, can also be calculated by cheminformatics. Of course, the QSAR methods may not be suitable when the underlying mechanisms are sufficiently clear. For example, quantities such as dipole moment, polarizability, and vibration frequency can be directly computed by quantum chemistry.

The characteristics of a compound in the traditional QSAR model are produced in an ideal environment. In reality, however, the solvent environment is not ideal. For example, we know that acidic chemicals are easily absorbed in an acidic environment. Therefore, to achieve the desired effect for a drug in an actual environment, the hydrophilicity and lipophilicity of the drug need to be considered (Ditzinger et al., 2019). The main environmental factors of a solvent can be found in the study by Bruno et al., 2021.

With the aforementioned preparation, the next stage is to build a QSAR model based on ML, which mainly consists of the following steps:

• (1)“Molecular coding,” which uses a vector of various characteristics to represent compound molecules, where the chemical characteristics and properties can be learned from the chemical structure or experimental data. The molecular characteristics should be independent as much as possible to avoid excessive correlation. The biological activity data and solvent environmental coefficients should be as accurate and sufficient as possible, and they are best obtained by conventional experiments.
• (2)An appropriate number of compounds must be chosen to construct the training/testing sets. Ideally, conformation optimization through MM/QM is carried out first to improve the data quality. Unsupervised learning is used to identify the most relevant attributes and reduce the dimensionality of the feature vector. Supervised ML model is applied to discover an empirical function (explicit or implicit) that can achieve the optimal mapping between the input feature vector and the biological activity.
• (3)Internal/external verification of the model is performed to determine its applicability and predictability. After generating exhaustive feature sets, the algorithm determines which feature combinations can be used for new predictions by finding the largest difference between positive and negative samples in the training set; thus, similar data distributions between the training and testing sets should exist. The techniques to detect out of distribution (OOD) can be found in the study by Hendrycks and Gimpel, 2016. The key to constructing a ML model is to constantly add and try more feature combinations, change the divisions of the training set, and adjust the algorithm parameters until the best model is obtained.

ML models trained on specific target sets are not universal, but the latest ML methods are improving versatility. Advances in computing power and software performance have also been used to improve QSAR models. By continuously integrating new algorithms and descriptive features, the model can be continuously optimized. Although the general idea of the ML-based QSAR is the same, the input characteristics may be different for different targets. The latest progress is the ability to learn general representations, such as self-learning, multitask learning, AutoEncoder, Generative Adversarial Networks (GAN), and Bidirectional Encoder Representations from Transformers (BERT) (Caruana, 1997; Devlin et al., 2019; Goodfellow et al., 2014; Liou et al., 2014; Xu et al., 2020). The QSAR learning models still face several problems beyond their construction (Dearden et al., 2009), which are detailed in the study by Artem et al., 2014.

DL invigorates QSAR

With excellent performance in imaging, speech, machine translation, etc (Minar and Naher, 2018), DL has entered into many biological fields, including genes, proteins, metabolites, microbiomes, and population-wide genetic variation, synthetic biology, drug discovery, and diseases (Alkawaa et al., 2018; Golkov et al., 2020; Hill et al., 2018; Zeng et al., 2021). The promising DL methods include capsule networks (Inokuma et al., 2010; Xi et al., 2017), multitask learning (Antropova et al., 2017; Wang et al., 2015b; Zhu et al., 2016), GANs, self-encoding decoders (Marchi et al., 2015; Wang et al., 2018; Xu et al., 2014; Yao et al., 2017; Zhao et al., 2016), Variational AutoEncoders (VAEs) (Panych et al., 2015), Long Short Term Memory Networks (LSTMs) (Baytas et al., 2017; Gers and Schmidhuber, 2001; Graves et al., 2005; Yildirim, 2018), transfer learning (Fernandes et al., 2017; Pan and Yang, 2010; Paul et al., 2016; Zoph et al., 2016), deep neural networks (DNNs) (Yoshioka et al., 2014), and CNNs (Horváth et al., 2017; Luo, 2015; Parashar et al., 2017; Wang, 2013; Xue et al., 2016). Because the biological information is generally rich, multitask learning can be performed to gain a better structural representation. For example, the microbial metabolic network and the loop network formed by the parasitic relationship are very scarce; they can be generated by a GAN. When using DL to predict the protein structure from a gene sequence, VAE can be used to encode compound structures and then be combined with the GAN for learning (Lin, 2009; Widera, 2010; Yu et al., 2015). With switch networks formed by atoms, strategy networks and Monte Carlo trees can be used to predict chemical reactions (Anderson and Long, 2003). CNN can be used to perform learning based on force fields and the protein-ligand complex (Clore and Gronenborn, 2010). A matching network, which is a variant of one-hot learning (Alaya et al., 2019), can be used to evaluate whether a new compound matches the receptor (Deka and Quddus, 2014; Tan et al., 2013). The cheminformatics-based LSTM methods are also frequently reported (Shu et al., 2018). The advantages, notwithstanding, DL entails a large amount of data and computation and is essentially a “black-box”; thus, it cannot replace all traditional ML methods, which are currently irreplaceable owing to better interpretability and lower data requirements.

Biological applications of DL

Since the late 1990s, DL has become a useful tool for drug discovery and can help researchers understand and construct the relationship between a chemical structure and its biological activity (Jing et al., 2018) (Figure 3). In the early days, structure-based methods such as cheminformatics are most successful in drug discovery (Feng et al., 2013; Hwang et al., 2020; Kim et al., 2010; Nam et al., 2014). They were used to conduct chemical structure searches, generate descriptors, construct fingerprints, and analyze chemical similarity. Expert experiences, combined with manually generated explanatory rules, were used to screen compounds. The later DL can capture the nonlinearity and complex relationships in the available data and generate more compact features than those manually generated by experts. Through multilayer networks, DL can directly process raw, unstructured data such as sequences and three-dimensional structures and has better nonlinear fitting capabilities. These multilayer networks are proven to have infinite VC dimensions (Vapnik et al., 1994) and thus can divide any data space. DL has been successfully applied to CADD (Chen, 2014; Quang et al., 2015). Although DL is more suitable for ligand-based drug discovery, there exist several interesting structure-based DL applications. In the AtomNet method (Wallach et al., 2015), for example, the input molecule is discretized into a three-dimensional grid and then sent to a convolutional neural network (CNN) and a fully connected layer to predict the binding affinity. This model needs not to preprocess but uses a learnable representation to identify the pair data of interacting atoms. AtomNet uses the same atom space to describe both the receptor and ligand, which is more natural, requires no preprocessing, and allows for the characterization and regularization of the interaction between the receptor and ligand atoms. To apply AtomNet, the 3D protein-ligand complex needs some conversion and coding (Klebe, 2000). First, the complex is formalized as a cube with 20 angstrom side length, and the cube is placed at the geometric center of the ligand. Then, a 3D grid with 1-angstrom resolution is used to discretize the positions of heavy atoms. This method allows for the representation of an atom as a 4D vector (three coordinates and one feature value) and the whole input as a 4D tensor. As another example, the protein-ligand complexes in the PDBbind database were used to train and test the neural network (Evers et al., 2003). The complexes were protonated and charged using UCSF Chimera with Amber ff14SB for standard residues and AM1-BCC for nonstandard residues and ligands. This study can determine which atom deletion causes the largest decrease in the predicted value. It also found novel interactions between the receptor residues and the ligand: e.g., Tyr693 forms a hydrogen bond with the ligand; Met713 forms a hydrophobic interaction with the ligand.

Figure 3.

Main steps and scenarios of ML-based drug discovery

Another important biological application of DL is the clarification of the molecular mechanisms of how protein structural changes and amino acid mutations cause the change of protein-protein interaction (PPI) networks. In a DL application, instead of using the traditional MM/QM (Vesely, 2001), the protein structure is first divided into fragments and partitions, and then the most effective fragments are predicted. DL can, on the one hand, seek interpretability from the successfully trained model, i.e., the interpretation of the feature contribution of a single sample; the combination of existing field experiences then helps us gain insights into complex relationships. On the other hand, DL can combine raw data to generate specific functional characterization and fragment representation; thus, it can facilitate the study of the mechanism of a protein structure.

Comparing DL with traditional ML

DL and traditional ML have their respective advantages and application suitability (Camacho et al., 2018; Chan et al., 2019; Pei et al., 2019; Yuan et al., 2019). The traditional ML relies more on the domain knowledge (theories and mathematical models), while DL directly extracts patterns, which are not always explainable, from the raw data. For tasks such as image analysis, DL is certainly better; it is particularly useful in de novo molecular design and reaction prediction (Button et al., 2017; Hartenfeller et al., 2012; Kang and Liu, 2021; Schneider and Schneider, 2018). For tasks with structured descriptors as input, DL seems to be at least as effective as traditional ML methods. Although DL can achieve better performance in biological activity prediction through multitasking, ML-based QSAR can also achieve better performance through continuous improvement of the feature engineering, which can be achieved by dividing the protein into fragments and functional regions to remove the influence of the orientation, flexibility, and solvent of a protein molecule. Although DL is better for automatic feature engineering, it entails a large amount of data. In contrast, ML can use a small amount of training data (Altae-Tran et al., 2016).

One common ground of DL and traditional ML is the learning algorithms, including supervised and unsupervised learning. Supervised learning is related to a specific task and thus can obtain task-related feature representations. Its input data must be of a similar structure. If a new task is related to the original one, one can generally use the representations obtained by supervised learning to join the new task to improve the effectiveness of the model. Unsupervised learning can obtain a data-distributed representation of its own existence, which is irrelevant to a specific task. It yields the patterns themselves, that is, the general characteristics of the data set. The use of unsupervised learning is promising to create new synthetic biology samples and then to determine the legibility of the new samples based on the structural knowledge. For example, one can use the general common-sense model learned by unsupervised learning to check whether a synthesized new compound is reasonable. Furthermore, unsupervised training can obtain some general common sense. If this common sense is a prerequisite for certain tasks, then the representations learned can be used to determine whether a sample belongs to this pattern. Multitask learning is a kind of supervised learning with some unsupervised features. It has great advantages when the data are scarce, which is typical in the study of the parasitic relationship between the host and microorganisms in a microbiome.

Combining DL with traditional ML

Although traditional ML and DL have their own advantages and disadvantages, they can be used in combination. For example, the DL-based face recognition is apparently better, but the learned features may be meaningless and the interpretability is thus lacking. This shortcoming can be compensated by the ML-based recognition. Therefore, their combination can achieve a better performance. In the following, such a combined model is outlined (Figure 4).

Figure 4.

ML-DL combined QSAR model

Data-oriented feature extraction

Different models have different requirements of the input data. To implement traditional ML models, the input data should be well prepared to reduce feature dimension and promote feature grouping; the interactions between the features need to be constructed manually, which relies heavily on experience. To implement DL models, raw data are welcome because DL algorithms can automatically extract features. In image recognition, for example, DL can extract hierarchical features from the pixel-level raw data through its convolution kernel. Feature extraction has two types: explicit and implicit. The explicit features, such as chemical descriptors, chemical fingerprints, physical and chemical properties, chargeability, substituent groups, fragments, geometric properties, are usually generated by experts. Being overlap with each other or redundant, these features may need to be processed by applying algorithms such as restricted Boltzmann machines (Pinaya et al., 2016; Zhang et al., 2015) and AutoEncoder to generate new, more compact, distributed features. The implicit feature extraction is very useful in processing sequences such as amino acid sequences, the atomic sequence of a compound, the 3D structure of a protein, and atomic grid structures, where the features can be learned by directly applying sequence processing algorithms including CNN, LSTM, and automatic codecs. For other applications, algorithms such as word2vec (Altszyler et al., 2018; Wang et al., 2016), BERT (Sun et al., 2019), GANs (Yi et al., 2019), and AutoEncoder can be used for unsupervised training.

The dimension of features and the number of data samples must be well balanced. A large feature dimension necessitates the acquisition of more data, which may be costly or even impossible. To solve this problem, the usual method is to filter the features, group them, or apply AutoEncoder or VAE for dimensionality reduction. However, this method usually cannot reduce the amount of data because both unsupervised feature expressions and supervised learning are involved. We therefore propose the following ML-DL combined strategy. By using the traditional ML, various explanatory features related to the training target are first constructed and grouped based on common underlying information; dimensionality reduction is then performed by applying feature selection or AutoEncoder, and the compact and effective explicit features are obtained. By using DL, the other types of data (e.g., sequence and structure data) can be used for unsupervised training to obtain compact and implicitly distributed features. Finally, both the implicit features (which reflect more information about the low-level data) and explicit features (which are more relevant to the task) are sent to the DNN to train the model.

Iterative training to generate more effective features

The explicit features of chemicals, usually in the form of molecular descriptor and fingerprint, are extracted from the structural chemical data and are grouped according to their correlation values. The redundancy of features is reduced by filtering out invalid features by using traditional ML algorithms such as xgboost and gbdt. These steps are iterated several times to determine the features to be input to a module called stacking (Palangi et al., 2014; Zhan et al., 2018; Zhang et al., 2017). The input samples to stacking include both chemical data and the features (sample = data + feature). The samples are divided into N parts, N − 1 of which are used to train a classifier and the remaining one is used to test the prediction made by the trained classifier. In total, N such classifiers are used to obtain N new sets of features, which replace the old sets of features in the next round of iteration.

On the other hand, the implicit features of chemicals are extracted from the nonstructural chemical data such as the chemical sequence, molecular bond, 3D coordinates by using DL methods such as transformer, CNN, and GNN. The samples are then trained by a DNN, whose last layer output is saved and then, together with the above explicit features, is sent to a stacking module consisting of DNN and ML. Through training by the stacking module, a more effective model is obtained.

Iterative integration of different models

QSAR is based on both experience-driven and data-driven research models, including wet experiments, MD simulation, ML, and DL. Their cooperation makes QSAR increasingly powerful. The experiments usually do not reveal the reaction mechanisms at the molecular and atomic levels, which can be revealed to a certain extent through MD simulation. ML-based QSAR is data driven (good at processing 2D and 3D data) and is oriented by feature extraction. The limitations of traditional ML are that the data are usually low-dimensional and the features must be interpretable and manually constructed. To overcome these limitations, DL is developed to automatically extract features from both unstructured and structured data (which could be high-dimensional). The integration of the four models would make QSAR powerful, for which a framework is provided (Figure 5). In the framework, the four models communicate with each other and adjust themselves iteratively to optimize the synergism of our field experience with ML/DL predictions.

Figure 5.

Iterative integration of wet experiments, MD simulation, traditional ML, and DL

There are two research studies that have implemented part of the framework in Figure 5. Yasushi Okuno's group at Kyoto University proposed an interdisciplinary framework called DEFMap, which combines experimental data, MD simulation, and DL to extract protein dynamic information from cryo-EM density maps (Matsumoto et al., 2021). Shuguang Yuan's group used part of the framework to prove that the frozen structure analyzed in the traditional artificial cell membrane environment is very different from the real cell membrane environment (Zhang et al., 2021), subverting the traditional understanding that the 3D structure of the membrane protein is the same as its physiological state in the artificial cell membrane or precipitant environment.

Conclusions

As a part of Big Data science, ML and DL are based on information theory and data fitting theories, where predictions depend mainly on a large amount of effective and reliable data, high-speed computation (for MM/QM simulations to process experiment data such as those from NMR, X-ray, and cryo-EM), and the number of compounds. These requirements cannot be met in many areas of computational biology; thus, ML/DL does not always work well. This problem can be partly solved in some fields by using powerful public tools such as AlphaFold (Senior et al., 2019). The wide application of ML in many fields depends on its human-like problem-solving capacity, including classification and clustering, explanation and verification, relevance and factor importance. But now the integration of iterative thinking is ever-increasingly important. Only through the cyclical upward process of prediction-verification-update can we form a systematic way of thinking and solve the problem from an overarching perspective.

In this article, we primarily reviewed ML-/DL-based QSAR methods. In addition to traditional descriptors, we emphasized the integration of MD with experimental solvent-related data. We also proposed a better framework that takes advantage of the respective merits of ML and DL to process different forms of data. The framework, through iteratively adjust itself, can potentially reveal the importance of some manual features, the correlation between features, and explain the contribution of different features in a single sample. The obtained embedded features or distributed vectors can help to understand the relationship between the constituent elements and the importance of local structures. The framework would ultimately achieve high efficiency and low cost in the fields of biomedicine and materials, especially in terms of QSAR.