Deep learning tools to accelerate antibiotic discovery

Abstract

Introduction:

As machine learning (ML) and artificial intelligence (AI) expand to many segments of our society, they are increasingly being used for drug discovery. Recent deep learning models offer an efficient way to explore high-dimensional data and design compounds with desired properties, including those with antibacterial activity.

Areas covered:

This review covers key frameworks in antibiotic discovery, highlighting physicochemical features and addressing dataset limitations. The deep learning approaches here described include discriminative models such as convolutional neural networks, recurrent neural networks, graph neural networks, and generative models like neural language models, variational autoencoders, generative adversarial networks, normalizing flow, and diffusion models. As the integration of these approaches in drug discovery continues to evolve, this review aims to provide insights into promising prospects and challenges that lie ahead in harnessing such technologies for the development of antibiotics.

Expert opinion:

Accurate antimicrobial prediction using deep learning faces challenges such as imbalanced data, limited datasets, experimental validation, target strains, and structure. The integration of deep generative models with bioinformatics, molecular dynamics, and data augmentation holds the potential to overcome these challenges, enhance model performance, and utlimately accelerate antimicrobial discovery.

1. Introduction

The introduction of antibiotics significantly improved the quality of health in modern society by extending human lifespan by as much as one to two decades [1]. Although antibiotics have provided countless benefits in healthcare, agriculture, zootechnics, and other areas, their unlimited and improper use has led to the development of pan-drug resistant microbes, which were responsible for more than 4 million deaths globally in 2019 alone [2,3]. The number of deaths caused by SARS-CoV-2 prior to vaccination campaigns in 2020, approximately 3 million, is comparable to the current number of deaths caused by drug-resistant infections [4]. The World Health Organization (WHO) has elaborated a global action plan that aims to improve the surveillance and understanding of antimicrobial resistance, optimize the use of antimicrobial drugs, and, particularly, ensure sustained funding in countering antibiotic resistance by supporting the Member States to develop national action strategies [5]. Besides the rapid evolution of resistance, traditional methods for the discovery and development of new antibiotics are no longer cost- or time-effective. In fact, it takes an average of 15 years to bring a new drug to the market, a timeline that includes drug discovery and development, preclinical and clinical trials, drug review, and post-market safety monitoring by the U.S. Food and Drug Administration (FDA) [6]. Shortening this timeline represents an orthogonal challenge that laboratories, companies, and political institutions need to address. Investments in both research and development, along with innovation and a viable marketplace, are key to tackling the antibiotic crisis.

Historically, new antibiotics were identified by screening microbes from the soil and looking for secondary metabolites. Starting from these molecules, many derivatives have been synthesized; these derivatives have improved bioactivities and are more likely than the original molecules to meet clinical standards [7]. More recent discovery approaches involve screening large libraries by using high-throughput methods. However, the unexplored chemical space for finding drug-like molecules is immense, and screening innumerable probable combinations is costly and time-consuming [8]. Ultimately, the discovery of a new antibiotic has always involved a large degree of chance and serendipity.

In this context, artificial intelligence (AI) and machine learning (ML) represent an innovative, emerging approach for speeding up the identification of new antimicrobial drugs [9–13]. AI is defined as the capability of computers to perform tasks that usually require human intelligence, while ML is a subcategory of AI that uses models to learn from datasets. Thus, the combination of computer science and large data sets is used to train ML algorithms, generating a system applicable to several functions, such as the design of new drugs, the prediction of their activity, the optimization of previously known molecules, and the identification of antimicrobial resistance (AMR) markers. Machine-learning algorithms could eventually be used for the diagnosis of diseases and the prescription of antibiotics [9,14].

Computer-aided drug discovery techniques are becoming essential in the preliminary stages of drug development. Recently, over 2,603 encrypted peptide antibiotics were discovered through a computational method able to search the human proteome for antimicrobial sequences [15–19]. Fifty-six of these sequences showed potent anti-infective activity in two different mouse models. Several of these encrypted peptides also synergized with each other at nanomolar concentrations to target pathogens. Other studies based on genetic algorithms demonstrated the ability of machines to design effective antibiotics [20]. Moreover, quantitative structure-activity relationship (QSAR) methods have been extensively used to predict the activity of new antimicrobial molecules by correlating biological properties and molecular descriptors [21–23]. Such methods present multiple advantages over non-quantitative methods, such as the wide or random exploration of sequence space, because quantitative explorations yields results that can be tracked and learnt heuristically and computationally, serving as the basis for guided design [24–26]. The predictions rely on the computational representation of antibiotic candidates, which can be used as input for the ML drug discovery pipeline. Topological properties and descriptors derived from quantum mechanics are usually employed. Hundreds of measurements have been extracted and combined in online platforms to describe amino acid residues and other small molecules (Table 1) [23].

Table 1.

Online tools for the extraction of molecular descriptors.

Antibiotic compounds	Molecular descriptor tools	References
Peptides/proteins	Z-scales	[83,84]
	VHSE	[83,85]
	T-scales	[83,86]
	ST-scales	[83,87]
	MS-WHIM	[83,88]
	FASGAI	[83,89]
	BLOSUM	[83,90]
	ProtFP	[83,91]
	Aaaindex	[92,93]
Small molecules	alvaDesc	[94]
	ChemAxon	[95]
	ISIDA/QSPR	[96]
	MOLD2	[97]
	QuBiLS-MIDAS	[98]
	Rdkit	[99,100]

Based on the adopted methodologies, the computational tools developed so far for antibiotic discovery can be grouped into two main categories. The first category includes traditional ML-based predictors, while the second category comprises deep-learning based methods, which draw upon larger sets of data and longer training compared to the former and can automatically learn high-level features from raw data. Here, we provide an overview of the main features applied as descriptors for peptides and small molecules, describe the primary deep-learning (DL) approaches used to develop new antibiotics and conclude with a section highlighting the main limitations encountered so far in this emerging field. This review offers valuable insights into how AI’s transformative role in antibiotic discovery by expediting new drug development, optimizing molecule design, and combating drug-resistant infections.

2. Deep learning models for antimicrobial drug discovery

2.1. Input representation

Before selecting which DL model to employ, it is important, even more than the algorithm itself, to choose the right representation for the input data [27]. Indeed, the input representation drives how the model learns about the molecules, thus playing an important role in the algorithm coding. Translating a molecule into vectors of numbers is called “molecular featurization” [27]. There are three main categories that describe the representation of the input data: one-dimensional [(1D), e.g., string-based], two-dimensional [(2D), e.g., graphical-based], and three-dimensional [(3D), e.g., coordinate-based]. The most common string-based representation is SMILES (Simplified Molecular Input LineEntry System), a sequence of characters based on symbols of atoms or, more specifically, amino acids, which are easily represented by computers [28–31]. Other 1D-like annotations such as Hierarchical Editing Language for macromolecules (HELM) have been also introduced to describe a broad range of macromolecules (e.g., peptides, antibodies, chemical modifiers)[32]. Although string-based sequences such as SMILES annotation are the easiest way to represent molecules, they cannot directly describe the molecular structures of compounds. Alternatively, it is possible to represent molecules in 2D by using neural network-based graphs where each node of the network describes an atom and the neural edges correspond to the bonds. Lastly, it is possible to represent molecules in 3D using point clouds, which capture not only atomic bonds, but also details about conformational preferences. More complex biophysical information is contained in 2D and 3D representations than in 1D representations. Although in the past few years, 2D and 3D representations have been applied to generative models, there are disadvantages associated with these types of input representations. For example, they cannot easily be applied to large molecules because they are computationally expensive. Also, graphs and point clouds used in autoregressive generative models, which impose a canonical ordering among atoms, can introduce bias into the algorithm, potentially limiting the model’s performance. This bias arises from the discrepancy between constitutional isomers. As a result, accurately representing diverse molecular structures becomes challenging, impacting the overall effectiveness of the model [28]. In several prediction tasks, peptides are extensively represented by their amino acid composition and physicochemical properties [33] (Table 1). Another common way to represent peptide molecules is through their primary structure, which refers to their amino acid sequence. A peptide with length L can be naturally encoded by using a string of characters or integers that is L units long. In this scenario, a peptide can be represented by a matrix with dimensions of L x n, where each amino acid corresponds to a unique n-dimensional vector able to capture peptide properties. Alternatively, studies based on model architectures such as variational autoencoders, generative adversarial networks, or attention networks, used the latent representation learned from deep neural networks as peptide representation [33].

2.2. Deep-learning approaches

Computers can use different algorithmic approaches to learn subjects from data and to provide outputs. We can distinguish two main types of models: discriminative and generative models. Whereas the first literally separates data points into two different classes that it then uses for classification or regression, the latter generates a probability based on the distribution of a dataset [34]. In the following sections, we describe the structure underlying DL-based discriminative and generative frameworks, and list examples of their application for antibiotic design.

2.2.1. Discriminative DL-models

2.2.1.1. Architecture and applications of NNs in predicting antibiotic activity

Historically, the architecture of NNs is composed of a fully-connected input layer, several hidden layers, and an output layer (Figure 2A). For antibiotic discovery, these layers are used to capture complex relationships between the structure and activity of antibiotics using nonlinear transformations [33,35,36]. The hidden layers are constructed from a set of nodes that serve to process input data (e.g., usually vectors) and combine and transmit the signals to the output layer. The model predictions are evaluated by the comparison of predicted and expected values and the iterative backpropagation of errors from the output to hidden layers to minimize the loss, i.e., the difference between predicted and expected values, through the optimization of network parameters, such as weight and bias values. Despite computational costs, the capacity (modeling power) of NNs usually grows with their sizes (i.e., number of hidden layers and hidden nodes) when the amount of data is large. The so-called deep-neural network (DNN) is trained on big datasets and used for predictions over the vast chemical spaces of small molecules and macromolecules, such as peptides [35].

Figure 2. Overview of neural network models.

(A) Fully-connected, (B) recurrent, (C) convolutional, and (D) graph neural networks. Models are frequently used for the prediction and design of antimicrobial peptides (AMPs) and small molecules. This figure was created in BioRender.com.

Currently implemented deep learning (DL) models to predict antimicrobial peptides (AMPs) and small molecules mainly rely on the convolution, recurrent, and graphical NN approaches [37,38]. Depending on their algorithms, DL models learn appropriate feature encoding methods to most accurately predict which peptides and small molecules have antimicrobial activity [33,36].

2.2.1.2. Convolutional neural networks (CNNs)

For convolutional neural networks (CNNs) [39], the raw input dataset is encoded in the form of multidimensional arrays and passed through convolutional layers composed of multiple 2D-, 3D- and N-dimensional kernel matrices of tunable width and height (Figure 2C). Convolution operations between kernels and inputs in each layer are applied to extract features sequentially; thus, the hidden features learned from the last few CNN layers can expressively represent the given input with respect to the modeling task of interest [17,18]. Additionally, the CNNs may contain optional pooling layers used to merge and reduce the size or resolution of feature maps. Finally, the feature representations of the last convolutional layer may be flattened and taken as input to fully connected layers to yield predicted values [33,36]. By adopting small kernels instead of fully-connected networks, CNNs exploit the local structures of inputs and are suitable for structural data whose local elements are highly correlated with each other, including images, sequences, and languages.

In the case of AMPs, each amino acid residue is represented by a single character or one-hot encoding feature in the context of 20-bit binary vectors to encode amino acid compositional information. To train the CNN models, the width and height of the matrices can represent amino acids and their physico-chemical properties as numerical values that are tunable hyper-parameters. In this respect, Deep-AmPEP30, a two-layered CNN model with additional pooling layers, was developed based on PseKRAAC reduced amino acid composition: experimentally validated by screening the genome of Candida glabrata, short-length peptides were predicted to have antimicrobial activity with an accuracy of 77% [40]. Another proposed CNN model, constructed from an embedding layer for encoding peptide sequences and 64 multi-scale convolutional layers followed by maximum pooling layers, resulted in an accuracy of 92% on the APD3 benchmark dataset [41]. In this approach, the embedding layer vector dimension was 128 and the peptide sequences were encoded as numerical vectors of real numbers (0–20) with a length of 200 amino acid residues. Also, the multiple kernel layers had various filter lengths to ensure that all latent features were learned. Moreover, introducing attention layers in the architecture of NN models has resulted in the identification of antimicrobial peptides from the human gut microbiome with significant efficacy in a mouse model of bacterial lung infection [42]. Moreover, a combined model of CNNs, regression, and decision tree was employed to optimize small molecules for inhibiting RNA translation in Mycobacterium tuberculosis, revealing important features for molecule-RNA binding and demonstrating the reliability of CNN as a predictor. Two representations, simplified molecular-input line-entry system (SMILES) and low-resolution images, were used showing that CNNs can reliably predict small molecule inhibitors for RNA translation in M. tuberculosis using either input type, with accurate results obtained from low-resolution images [43].

2.2.1.2. Recurrent neural networks (RNNs)

Recurrent neural networks (RNNs) have been developed to process inputs having sequential structures (e.g., time-series or biological sequences), [44]. In RNNs, sequential information is captured by enabling the nodes in a hidden layer to receive the information from both the current inputs and their previous nodes (Figure 2B) [44]. RNN models have resulted in excellent performance in sequence data analysis of AMPs of various sizes. In the RNN algorithms, each amino acid is considered as a word in a sentence and demonstrated by one-dimensional vectors. In addition, one-hot encoding may be used to construct numerical information. For instance, in 2D-convolution modules of AMP sequences with binary 2-D matrices, all variables are filled up with zeros, except the position of the corresponding amino acid [33,36]. Also, an RNN model with a stack-augmented GRU was used to discover new metronidazole drugs by generating precise SMILES strings [45]. The model was trained on ~ 1.6 million small molecules, utilized transfer learning to generate structurally similar molecules and finally generated generated 20 metronidazole-like compounds, with 19 potentially novel ones, that demonstrated activity in vitro against E. coli, P. aeruginosa, B. subtilis, and S. aureus strains [45].

2.2.1.3. Graph neural networks (GNNs)

Compared with these architectures, graph neural networks (GNNs) have been frequently used to extract complex feature vectors and predict the biochemical functions of antimicrobial peptides and small molecules from their 2D or structures [8,46]. In the GNNs, biomolecule can be interpreted as consisting of nodes (e.g., atoms and amino acids) and edges, (corresponding to their connectivities, e.g., bonds and geometric distances) (Figure 2D). Nodes and edges are associated with feature descriptors summarizing the types of atoms, amino acids, or bond types, as well as physicochemical and geometric information. For each node and edge, NNs are used to gather information from their neighbor edges and nodes and update the feature descriptors (Figure 2D). This procedure is known as message passing and enables the nodes and edges to capture their 2D/3D structural information in the graph. By combining all feature descriptors updated via several rounds of message passing, the structural fingerprint of the biomolecules is extracted and mapped to their properties using an end-to-end classification/regression neural network. Stokes et al. trained a GNN with 2,335 antimicrobial molecules selected from natural compounds and a library of FDA-approved drugs [8]. The algorithm was applied to predict antibiotic compounds from the Drug Repurposing Hub. Among the highly ranked molecules, halicin was experimentally identified and validated to demonstrate potent bactericidal activity.

2.2.1.4. Attention function

When predicting a certain property given an input x (e.g., an antibiotic), only a fraction of that input (e.g., important amino acid sites in an AMP or substructures in a small molecule antibiotic) contributes to the property formation, while the rest is irrelevant and can even confuse prediction models. The attention mechanism is employed to automatically identify important parts of the data, addressing the issue of irrelevant information in predictions [47].

To compare deep learning models with and without an attention mechanism we use an example, which involves an antibiotic, denoted as x, consisting of N elements such as amino acids or atoms. The deep learning model without attention is characterized by M layers of nonlinear transformations, which process the antibiotic x to generate a corresponding prediction y. The model is represented as follows:

$$\begin{array}{l}{\text{Hidden\_output}}_1{\text{=G}}_1(\text{x}),\hfill \\ {\text{Hidden\_output}}_2{\text{=G}}_2\left({\text{Hildden\_output}}_1\right),\hfill \\ \dots \hfill \\ {\text{Hidden\_output}}_{\text{M}}{\text{=G}}_{\text{M}}\left({\text{Hidden\_output}}_{\text{M-1}}\right),\hfill \\ y=\text{Out}\left({\text{Hidden\_output}}_{\text{M}}\right).\hfill \end{array}$$

Here, G1(), G2(), …, and GM() correspond to the M nonlinear transformations (i.e., each layer itself can be fully-connected, CNN and RNN), and Out() is a linear function to map the extracted hidden features to final output prediction y.

To introduce attention mechanism into above deep learning model, we can rewrite the above process as:

$$\begin{array}{l}{\text{Hidden\_output}}_1{\text{=G}}_1\left({\text{Att}}_1(\text{x}),\text{x}\right),\hfill \\ {\text{Hidden\_output}}_2{\text{=G}}_2{\left({\mathrm{Att}}_2\left(\text{Hidden}\_{\text{output}}_1\right),\text{Hidden}\_\text{output}\right)}_1),\hfill \\ \dots \hfill \\ {\text{Hidden\_output}}_{\text{M}}={\text{G}}_{\text{M}}\left({\mathrm{Att}}_{\text{M}}\left({\text{Hidden\_output}}_{\text{M-1}}\right),{\text{Hidden\_output}}_{\text{M-1}}\right),\hfill \\ y=\text{Out}\left({\text{Hidden\_output}}_{\text{M}}\right).\hfill \end{array}$$

Out() is defined the same as mentioned above. Att1(), Att2(),..., and AttM() are neural networks that accept the antibiotic or extracted features from the antibiotic as inputs. The output comprises an N-dimensional vector, which signifies the importance of each element in the antibiotic, considering that the antibiotic contains N elements in our specific example. Functions G1(), G2(), …, and GM() (implemented as neural networks) will take not only the antibiotic or its extracted features, but also the N-dimensional vectors highlighting the significant region of the antibiotic as input to make predictions. Altogether, utilizing the “attention” function in NNs, a large dataset of sequences is readily memorized by ignoring less important parts of the input data; thus, more focus is devoted to small but important part of the data [47].

The deep NNs for the prediction of antimicrobial compounds differ from other ML predictors in that, for the deep NNs, the high-level description vectors can automatically be generated and learned while less prior understanding of biology is required [33,48–50]. In fact, the model performance of the ML algorithms depends on how exactly the feature vectors are selected and extracted. In comparison, extracting the feature vectors is a less hand-coded and complicated part in DL techniques compared to other ML algorithms, which is less time-consuming in developing feature extractors. Nonetheless, the model performance and benchmarking in the DL tools to predict antibacterial molecules may be affected by the choice of datasets used for training [51]. Therefore, providing researchers with explicit information about the performance of the predictor, including code sharing, is suggested for fair model benchmarking [51].

2.2.2. Generative DL-models and their applications in antibiotic design

2.2.2.1. Neural language models (NLMs)

Given sequences of words, language models estimate their probability distribution. In the context of computational biology, primary sequences of DNAs, RNAs, proteins and peptides, as well as small molecules in SMILES representations, can all be represented in sequence forms. In these cases, the corresponding nucleic acids, amino acids, and SMILES characters are considered as words. Neural language models, equipped with deep neural networks, are widely used to take partial sequence inputs and estimate the likelihood of the missing input parts (Figure 3A). Data generation is achieved by completing the missing parts with the most likely words predicted by the DNN (Figure 3A). RNNs, graph neural networks, and attention models are commonly applied to build NLMs for biomolecules. NLMs have been extensively used to generate potentially antimicrobial compounds. Indeed, several studies have employed RNNs and public datasets of antimicrobial compounds to predict drug candidates. Moreover, additional filters (e.g., classifiers or prediction servers) have been applied to adapt the model to antibiotic discovery and remove undesired molecules.

Figure 3. Schematic of the main deep-learning architectures applied in antibiotic discovery.

(A) Neural language models (NLMs) use neural networks (NNs) to model the probability distribution of molecules predicted to have antimicrobial activity based on previously generated outputs. (B) Variational autoencoders (VAEs) encode and decode information to create a latent space, whose variables are used to generate new drug candidates. (C) In generative adversarial networks (GANs), a generator creates synthetic samples while a discriminator distinguishes real generated samples from fake. (D) Normalizing flows (NF) transform simple probability distribution into complex tractable distributions (i.e., input data) through a series of invertible mappings and vice versa. (E) Diffusion probabilistic models (DPMs) gradually add noise to the input data (x0) through a series of T steps, and then a trained neural network recovers the data by reversing the process. The model can generate new data through the denoising course. All elements present within the figure have been created in Biorender.com and assembled using Adobe Illustrator version 27.7.

Capecchi et al. identified eight short non-hemolytic AMPs by training a RNN with data from the DBAASP (Database of Antimicrobial Activity and Structure of Peptides) [52]. Besides an AMP activity classifier, these authors also applied a hemolysis classifier to select peptide sequences predicted to be non-hemolytic. As a result, they obtained peptide antibiotics with low cytotoxicity profiles that were active against drug-resistant bacterial strains. Additionally, Nagarajan et al. designed new AMPs by using an RNN language model [53]. The RNN was based on the correlation of frequencies and distribution of amino acid residues with antimicrobial sequences, as an underlying grammar of antimicrobial function. In their study, ten AMPs, identified and tested for activity against bacterial pathogens, targeted the bacterial membranes of methicillin-resistant Staphylococcus aureus and β-lactamase-resistant and carbapenem-resistant strains.

2.2.2.2. Variational autoencoders

Variational autoencoders (VAEs) present a model architecture similar to that of the classic autoencoders typically trained to compress and decompress inputs through an encoder and a decoder neural network. Unlike classic autoencoders, however, a VAE is a probabilistic modeling framework that interprets the latent representations (i.e., the compressed inputs) as random variables following some distribution (e.g., Gaussian distribution) (Figure 2B). Data generation is achieved by sampling latent representations followed by decoding and/or decompression.

Dean et al. demonstrated the potential utility of a VAE in designing de novo AMP sequences [54]. The latent space (i.e., the low-dimensional space that the latent representations lie in) extrapolated by the VAE learned from thousands of known and scrambled AMP sequences from APD3 highlighted the amphipathicity (mainly represented by hydrophobic moment) as the main difference between the active sequences generated and the scrambled random sequences. The obtained computational tool allowed the authors to generate new peptide sequences, whose antimicrobial activity has been experimentally confirmed in vitro. Indeed, in 2021 Dean et al. published a new study in which the improved model, named PepVAE, was demonstrated to generate new peptide sequences with low antimicrobial activity against Escherichia coli, S. aureus, and Pseudomonas aeruginosa strains [54,55].

Another interesting application of VAE has been shown by Das et al. [50,56] . The authors demonstrated that a VAE trained on a large number of peptide sequences, combined with ML-based predictions for properties such as antimicrobial activity or toxicity and molecular dynamics simulations, could be used to quickly design novel antimicrobial therapeutic molecules: 20 peptide candidates were identified and experimentally validated in 48 days. Among the tested compounds, two AMPs highly were active against both Gram-positive and Gram-negative bacterial species and had little likelihood of selecting for drug-resistant phenotypes [50].

Recently, Szymczak et al. proposed HydrAMP, a generative model based on a multitasking conditional VAE framework. HydrAMP generated not only effective analogues of existing peptide antibiotics but also de novo antimicrobial structures. To train the model, the authors used a curated data set from UniProt of non-redundant peptide sequences known to be antimicrobial. Besides standard encoder and decoder network-based models, HydrAMP also uses a Classifier, which is an extra pre-trained neural network used to determine whether a given sequence is or is not an AMP [57].

2.2.2.3. Generative adversarial networks (GANs)

GANs, which are built of two generative and discriminative neural networks, are trained in an adversarial way to predict antibacterial peptides [33,36,58,59]. Specifically, the generative module captures the input data but is trained to generate fake instances to fool the discriminative network, whereas the discriminator module obtains both real training data and the fake instances from the generator, which then determines whether the input data is real or not (Figure 2C). In training GANs for the AMPs prediction, the generator and discriminator accomplish adversarial functions against each other, resulting in the network modules [33,36,58,59]. PepGAN uses active and non-active datasets from various sources and controls the probability distribution of generated sequences to distinguish real antibacterial sequences from non-active ones. In this regard, a peptide activity-aware generative model, PepGAN, was designed to produce a sequence with antibacterial activity higher than that of ampicillin in vitro [60].. Also, AMPGAN models were utilized as bidirectional conditional networks for the design of AMPs with improved activity profiles [61]. These encoder networks can be used to structurally diversify and manipulate the generated peptide sequences for selective bacterial cell targeting [36,58]. The utilization of GANs accelerated the discovery of A-222, an a-helical peptide with low homology to well-established AMPs, that was strongly effective against the emerging global opportunistic strains Stenotrophomonas maltophilia WH 006 and P. aeruginosa PAO1 [59].

2.2.2.4. Normalizing flow and diffusion models

In addition to the three types of generative models described above, normalizing flow (NF) and diffusion models are also gaining increasing attention [28]. While resembling VAE in terms of framework (Figure 3D), NF utilizes invertible transformation functions (implemented as some neural networks) to convert complex data to random variables drawn from a simple distribution (i.e., encoding process) and vice versa (i.e., decoding process). While VAEs are trained by optimizing the lower bound of data likelihood, the use of invertible transformations enables NF to be optimized by directly maximizing the likelihood of data. For instance, in 2019 Madhawa et al. proposed GraphNVP, the first invertible NF-based model able to efficiently generate molecular graphs [62]. Interestingly, these authors empirically demonstrated that the learned latent space can be used to search for molecules with maximized chemical properties with respect to the parent compounds. In addition, Shi et al. proposed and experimentally validated GraphAF, a flow-based model for generating molecular graphs, which was found to generate chemical-valid and optimized molecules both with and without chemical knowledge rules [63].

Diffusion-based models have also been explored and applied to address molecular design tasks. Diffusion models describe a sequence of possible events, whose probability only depends on the previous events, by adding random noise to data (Figure 3E) until the inputs become a random noise (from some random distribution). Those models built the desired samples by learning how to reverse the diffusion process from the noise. Unlike the VAE and NF, the latent variable (the random noise) will have the same dimensionality as the original input. Corso et al. recently developed DiffDock, a diffusion generative model able to address molecular docking tasks [64]. After training the model with a variety of ligand and protein structures, DiffDock identifies new binding sites within the protein pocket, highlighting new 3D coordinates for the ligands. Molecular docking, a critical task in drug design, focuses on predicting the binding structure between a small molecule ligand and a protein. Although recent deep learning approaches have successfully reduced runtime when compared to traditional search-based methods, they have not yielded notable improvements in accuracy when treating docking as a regression problem. In contrast, DiffDock stands out from previous methods by showcasing significantly higher precision, offering fast inference times, and providing confidence estimates with a high level of selective accuracy [64]. This tool demonstrated to accelerate drug discovery compared to traditional methods. Also, Watson et al. introduced RoseTTAFold (RF) Diffusion, a new diffusion model able to generate proteins with minimal knowledge using simple molecular specifications [65]. They simulated the noise (for instance data - images or text - corrupted with Gaussian noise) on structure of protein from Protein Data Bank (PDB) to generate training inputs. To reverse the noising process, RF Diffusion was then trained by approximating the loss between the predicted shape and the true protein structure. According to Watson et al., the proteins generated by RF Diffusion can potentially treat infection, fight cancer, reprogram autoimmune disorders, or serve as building blocks for a new generation of materials [65].

5. Expert opinion

5.1. Physicochemical features associated with deep learning (DL) models

The use of physicochemical properties as quantitative descriptors for the guided design of antimicrobial agents has been significantly improved by the application of computational methods. The usual types of features used for drug design are classified by their dimensionality. 1D descriptors are scalar properties, such as molecular weight, net charge, hydrophobicity, number of hydrogen bond donors and acceptors, and aromaticity, which are by far the most frequently used physicochemical descriptors because they can be easily assessed computationally or empirically. 2D descriptors are related to properties that describe molecular topology, for example, topological polar surface area, rotatable bonds, molecular connectivity indices, and autocorrelation functions. Similarly to 1D descriptors, the 2D descriptors are easy to obtain computationally and can represent various molecules ranging from small molecules to polymeric or complex supramolecular structures. 3D descriptors, on the other hand, are much more difficult to obtain and are not usually related to small molecules or planar medium-sized molecules. The calculation of thse physicochemical descriptors requires a considerable computational cost. These descriptors represent most structural and electronic features of the molecules, such as molecular surface area, dipole moment, electrostatic potential, and solvent-accessible surface area.

DL and other computational methods have allowed the calculation of more complex and accurate features like 3D descriptors and other properties that are combinations of them, which are not related to structure or empirical characterization of molecules or their parts. The main disadvantage is that it is also costly and time demanding to include such complex features [66]. Some of the existing DL-based methods do not consider physicochemical features directly for the learning process and generation of bioactive molecules. Alternatively, deep learning models can be integrated with encoding techniques that utilize sequence-based characteristics to generate feature vectors based on the molecule’s composition. Among the most widely used feature-encoding methods, we highlight one-hot encoding [67], general and pseudo amino acid composition [68], and reduced amino acid composition [69].

AI4AMP, for instance, is a feature-encoding method based on general amino acid composition. The model was trained on sequences encoded with PC6 [70], a deep learning model for predicting AMP sequence based on a combined physicochemical property matrix. The features were selected from a hierarchical clustering and included, volume of side chains, polarity, hydrophobicity, pH at the isoelectric point, pKa, and the net charge index of the side chains, i.e., properties that play a role in the interaction of the compound with the target microbe. The PC6 encoding method combined with a DL architecture consists of three layers (convolutional, LSTM, and dense layers) and performed similarly to the word embedding-based model [71].

Other examples are Deep-ABPpred [72] and sAMP-PFPDeep [73]. Deep-ABPpred uses bidirectional long short-term memory [72] and the word embedding-based model coupled with amino acid level features and support vector machine and random forest models with peptide-level features, including physicochemical properties, to identify AMPs (accuracy of 97%). sAMP-PFPDeep consists of a DL method with three feature-encoding aspects: (1) position, (2) frequency, and (3) one or more of the following 12 physicochemical features: hydropathy index, volume, polarity, pKa side chain, percentage of exposed residues, hydrophilicity, refractivity, local flexibility, accessible surface area folded, mass, solvent exposed area, and accessible surface area. sAMP-PFPDeep was able to identify AMPs up to 30 residues in length with an accuracy of 84%.

In the future, we envision that the accurate embedding of multiple physicochemical features will lead to models that can generate highly active compounds. Some success in identifying anticancer peptides has been achieved with DNN models such as ENNAACT [74] which has eight dense layers with physicochemical features [75] as input. ENNAACT presented a 98.3% accuracy at 10-fold cross-validation and performed better than random forest and support vector machines with linear and radial basis function (RBF) kernels in the same dataset. Another DNN model that considers physicochemical properties for predicting the activity of antiviral peptides is ENNAVIA [76] (accuracy of 95.7%), which uses three dense layers with augmentative and alternative communication, data predictive control, AAindex [77], and physicochemical properties as input features. Deep-AntiFP [78] is a DL model with three dense layers: composite physicochemical properties [79], quasi-sequence order [80], and reduced amino acid alphabet [81], combined with physicochemical features. Recently, Deep-AntiFP predicted antifungal peptides with an accuracy of 94.23%. These models show that it is possible to combine physicochemical features with DL architecture for the accurate prediction and generation of active anticancer or antimicrobial agents.

5.2. Limitations related to datasets and experimental validation of deep learning models

There have been several major achievements in DL-based recognition and the prediction of antibacterial peptides and other small molecules; nevertheless, this field still requires development [37]. Indeed, the performance of DL models is significantly influenced by the quality rather than the size of the training dataset, which should consist of both active and inactive compounds [82,83]. In fact, the sources of the antibacterial datasets often suffer from inconsistency and heterogeneity in the activity evaluation of collected data, and lack of sufficient information on their mechanismsof action.

Besides data curation, public access to the negative datasets (i.e., peptides lacking activity against pathogens) is limited, as this information is rarely released [84]. Nevertheless, the negative data for the training of DL models to predict antibacterial peptides are often extracted as random sequences from available collections of proteins (e.g., Uniprot), which can be larger in data size than a few thousand true positive data. Most DL models have difficulty handling highly imbalanced antibacterial peptidesor small molecule datasets to prevent biased predictions [33]. Furthermore, the activities of AMPs are very sensitive to changes in peptide composition and length, and their activity varies greatly depending on the target bacterial strain [85,86]. These limitations pose problems especially for the prediction and recognition of antibacterial sequences. Additionally, depending on the selected DL algorithms, appropriate feature-encoding methods need to be selected over the training course of classifiers [36,37]. It is still a challenging task to consider expensive feature vectors relevant to the 3D-structure of peptides and small molecules to improve the prediction of new sequences or chemical scaffolds. Owing to the distinct selectivity of various antibiotic and antibacterial sequences for Gram-negative and Gram-positive bacteria, as well as the diversity of the activity determination protocols used under various laboratory conditions, sorting, and prioritization of predicted compounds for wet-lab activity determination and validation are still unsatisfactory, even though they are often used by binary classifiers [87]. Moreover, it is still unclear how to design classifiers that predict peptides that are resistant to protease degradation or that can model enzymatic cleavage sites in the peptide, to design more stable peptides [88]. Alternatively, there are models that leverage D-amino acid substitution, non-canonical amino acid residues, and cyclization to increase peptide stability [89]. Thus, the bioavailability of antimicrobial sequences remains an open area of research. Finally, the undesired concentration-dependent side-effects of antibacterial peptides need to be considered when designing DL predictors [37,87].

5.3. Data availability for DL models

The biggest hurdle to developing DL models that are accurate in generating peptides with biological activity is to have enough data for training the models. Whereas small organic compounds have databases such as ZINC [90], SciFinder, Reaxys, and PubChem, which count with millions to hundreds of millions of molecules but mostly with no information on bioactive activities, bioactive peptides databases, such as APD3 [91], DBAASP [92], CAMP [93], Hemolytik [94], CPPsite [95], DRAMP [96], and dbAMP [97], count no more than a couple of tens of thousands of data points. It is worth mentioning that databases such as UniProt or GenBank house extensive collections of peptides and proteins, yet it should be noted that these repositories do not exhibit antimicrobial activities and are still untapped sources for antimicrobially active molecules.

Overcoming the data availability issue relies on combining deep generative models with bioinformatic tools [98] or molecular dynamics simulations [50], or doing data augmentation [99] to increase the performance of the models. Bioinformatic tools and molecular dynamics simulations provide an enhanced sampling of sought-after properties for the biological activity desired. Among the most important features retrieved to enrich the datasets are the physicochemical features that can be extracted from high throughput simulations with a high level of detail. On the other hand, data augmentation will not generate new information on the dataset, but it will increase the number of examples in the training set while also introducing more variety in what the model sees and learns from. Both bioinformatic tools and molecular dynamics simulations are valid strategies for different biological purposes. We envision that coupling these strategies to high performing computing will improve quality sampling with less experimental input needed and may accelerate the discovery of potent antimicrobial agents.

5.4. Summary

In spite of advances in drug design and pharmacology [100] (Table 2), the development of new antibiotics is still time-consuming and costly, while the need for them is becoming ever more urgent. DL approaches have revealed promising results in accelerating antimicrobial discovery, thus attracting the attention of many researchers and companies in the field. In fact, the combination of generative models and DNNs demonstrates the ability to generate and predict molecules with desired bioactivities. Furthermore, these models can potentially reduce the time and expense of antibiotic discovery and increase the chance of success. Although there is no unique and standardized DL framework that yields the best results in terms of antibiotic prediction, all the machine architectures described here set the stage for the development of efficient platforms for discovering small molecules and peptides with antimicrobial properties. Particularly, well-trained DL models have been demonstrated to generate outputs with properties similar to those of training data, assign a probability of new data points within data distribution, and define important features. We envision that DL tools combined with larger datasets and more refined molecular descriptors will change the way that synthetic antibiotics will be designed in the near future.

Table 2.

Models and their application in antimicrobial peptides discovery.

Model	Application	References
ABP-Finder	Screen peptidomes and identify antibacterial peptides	[111]
rAMPage	To mine amphibian and insect transciptomes for antimicrobial peptides	[111,112]
AI4AMP	To predict antimicrobial potential of a given seqeunce and perform proteome screening	[62]
Deep-ABPpred	Identify antibacterial peptides within protein seqeunces	[65]
sAMP-PFPDeep	To predict short (<30 residues long) antimicrobial peptide sequences	[113]
Deep-AntiFP	To identify antifungal peptide seqeunces	[70]

Figure 1. Application of AI methods in antibiotic discovery.

(1) Datasets containing experimental information are selected for model training. (2) The data are translated into machine-readable representations. (3) Subsequently, the algorithms are trained and then used to predict new active molecules. (4) Wet-lab experiments are performed to validate the antimicrobial activity of computationally predicted candidate drugs. All elements present within the figure have been created in BioRender.com and assembled using Adobe Illustrator version 27.7. The peptide molecule shown in the figure was rendered using the PyMOL Molecular Graphics System, Version 2.5.2 Schrödinger, LLC.

Article highlights box.

• AI and ML provide innovative ways to expedite antibiotic discovery by optimizing molecular design.

• The successful use of discriminative and generative deep learning models is directly impacted by the algorithm and influences the modeĺs ability to represent diverse molecular structures.

• Discriminative deep learning models are computational frameworks for predicting antibiotic activity, leveraging their specific architectures and approaches.

• Generative deep learning models are applied in antibiotic design utilizing the composition of molecules to generate potent drug candidates.

• Deep learning models yet face challenges due to data quality and availability limitations.