A survey of generative AI for de novo drug design: new frontiers in molecule and protein generation

Abstract

Artificial intelligence (AI)-driven methods can vastly improve the historically costly drug design process, with various generative models already in widespread use. Generative models for de novo drug design, in particular, focus on the creation of novel biological compounds entirely from scratch, representing a promising future direction. Rapid development in the field, combined with the inherent complexity of the drug design process, creates a difficult landscape for new researchers to enter. In this survey, we organize de novo drug design into two overarching themes: small molecule and protein generation. Within each theme, we identify a variety of subtasks and applications, highlighting important datasets, benchmarks, and model architectures and comparing the performance of top models. We take a broad approach to AI-driven drug design, allowing for both micro-level comparisons of various methods within each subtask and macro-level observations across different fields. We discuss parallel challenges and approaches between the two applications and highlight future directions for AI-driven de novo drug design as a whole. An organized repository of all covered sources is available at https://github.com/gersteinlab/GenAI4Drug.

Introduction

During the drug design process, ligands must be created, selected, and tested for their interactions and chemical effects conditioned on specific targets [1]. These ligands range from small molecules with tens of atoms to large proteins such as monoclonal antibodies [2, 3]. While methods exist to optimize the selection and testing of probable molecules, traditional discovery methods across all fields are computationally expensive [4]. Recent artificial intelligence (AI) models have demonstrated competitive performance [5–7] in improving various tasks in the drug design process. Methods such as machine learning (ML)-driven quantitative structure-activity relationship approaches [8, 9] have significantly improved virtual screening (VS) in molecule design [10, 11], while ML-assisted directed evolution techniques for protein engineering [12, 13] have proven to be reliable and widely used tools. However, an emerging and even more powerful task for ML is the generation of entirely new biological compounds in de novo drug design [14–16].

In contrast to applications in VS and directed evolution, which seek to expedite and optimize tasks within an existing framework, de novo drug design focuses on generating entirely new biological entities not found in nature [14]. While other ML-driven methods search within existing chemical libraries for drug-like candidates, thereby facing inherent constraints, de novo design circumvents this limitation by exploring unknown chemical space and generating drug-like candidates from scratch [17–19].

In this paper, we explore the impacts and developments of ML-driven de novo drug design in two primary areas of research: molecule and protein generation. Within protein generation, we additionally explore antibody and peptide generation, given their high research activity and relevance. Although the types of pharmaceuticals and the associated chemical nuances differ across fields, the overarching goal of exploring chemical space through de novo design remains constant. Both fields are rapidly growing industries with traditionally high research and development costs [20–22], and current improvements are driven by active developments in ML-based de novo methods.

Molecule design specifically refers to the development of novel molecular compounds, often with the aim of small-molecule drug design. The generated molecules must satisfy a complex and often abstract array of chemical constraints that determine both their validity and “drug-likeness” [23, 24]. This, combined with the vast space of potential drug-like compounds (up to – [25]), renders traditional small-drug design time-consuming and expensive. Using traditional methods, preclinical trials can cost hundreds of millions of dollars [26] and take between 3 and 6 years [4]. In recent years, AI-driven methods have gained traction in drug design. AI-focused biotechnology companies have initiated over 150 small-molecule drugs in the discovery phase and 15 in clinical trials, with the usage of this AI-fueled process expanding by almost 40% each year [27].

An equally promising field, protein design, refers to the artificial generation or modification of proteins (protein engineering) for various biological uses. Native proteins have adapted and evolved over millions of years, so the rapid progression of human society in recent years poses challenges for naturally occurring proteins [28]. Protein design has an even more versatile range of applications, finding utility in immune signaling, targeted therapeutics, and various other fields. When executed efficiently, protein design has the potential to transform synthetic biology [21, 29, 30]. Like molecule generation, proteins must adhere to abstract biological constraints, yet the inherently more complex structure of a protein presents a nuanced generative objective, requiring a more direct application of chemical knowledge in the process [31–33]. Traditional methods such as directed evolution are confined to specific evolutionary trajectories for existing proteins; the de novo generation of proteins would add an entirely new dimension for researchers to explore [30, 34, 35].

Structurally, we aim to provide an organized introduction to the two fields mentioned above. We begin with a technical overview of relevant deep learning architectures employed in both small molecule and protein design. We then explore their applications in molecule and protein design, dividing our analysis into a variety of subfields highlighted in Fig. 1. For each subfield, we provide (1) a general background/task definition, (2) common datasets used for training and testing, (3) common evaluation metrics, (4) an overview of past and current ML approaches, and (5) a comparative analysis of the performance of state-of-the-art (SOTA) models. A detailed overview of this structure is shown in Fig. 2. Finally, we integrate concepts within each subfield into a broad analysis of de novo drug design as a whole, providing a comprehensive summary of the field in terms of current trends, top-performing models, and future directions. Our overall objective is to provide a systematic overview of ML in drug design, capturing recent advancements in this rapidly evolving area of research.

Figure 1.

An overview of the topics covered in this survey; in particular, we explore the intersection between generative AI model architectures and real-world applications, organized into two main categories: small molecule and protein generation tasks; note that diffusion and flow-based models are often paired with GNNs for processing 2D/3D-based input, while VAEs and GANs are typically used for 1D input.

Figure 2.

A structured layout for all terms and papers covered in our survey, including datasets, models, and metrics for each task; sections contained in the main text are highlighted in blue, and sections expanded upon in the appendix are highlighted in purple.

Related surveys

Several survey papers delve into specific aspects of generative AI in drug design, with some focusing on molecule generation [17, 36, 37], protein generation [28, 29], or antibody generation [38–41]. Other survey papers are organized based on model architecture rather than application, with recent papers by Zhang et al. [42] and Guo et al. [43] reviewing diffusion models in particular. While each of the above surveys provides an in-depth analysis of a specific application or type of model, the level of specialization may limit their scope. Our approach is a macro-level analysis of small molecule and protein generation, tailored for those seeking a high-level introduction to the emerging field of generative AI in chemical innovation. This broad perspective enables us to highlight relationships across fields, such as parallel shifts in methods of input representation, the common emergence of architectures like equivariant graph neural networks (EGNNs), and similar challenges faced in both molecule and protein design.

Preliminary: generative AI models

Generative AI uses various statistical modeling, iterative training, and random sampling to generate new data samples resembling the input data. Historically, prominent approaches include generative adversarial networks (GANs) [44], variational autoencoders (VAEs) [45], and flow-based models [46]. More recently, diffusion models [47] have emerged as promising alternatives. In our survey, we begin by providing a concise mathematical and computational overview of these architectures.

Variational autoencoders

A VAE [45] is a type of generative model that extends upon the typical encoder–decoder framework by representing each latent attribute using a distribution rather than a single value. This approach creates a more dynamic representation of the underlying properties of the training data and enables the sampling of new data points from scratch.

Formally, we can express the encoder as

Intuitively, each will be mapped to some mean and variance , which describe a corresponding normal distribution. We express the decoder as , where is a randomly sampled point from the latent distribution and is mapped into a point in the decoding process.

VAE loss is computed using two balancing ideas: reconstruction loss and Kullback–Leibler (KL) divergence loss. Reconstruction loss measures the difference between the ground truth and the reconstructed decoder output, often expressed using cross-entropy loss:

KL divergence measures the difference between two probability distributions [48]. For VAEs, KL divergence is computed between the encoded distribution and the standard normal distribution. This can be seen as “regularization,” as it encourages the encoder to map elements to a more central region with overlapping distributions, thus improving continuity across the latent space. Formally, the KL loss can be expressed as follows, with representing the th dimension in the latent space and representing the dimension of z:

Here, and represent the mean and variance of the kth component of the latent space, respectively, for datapoint . Then, the overall loss function can be expressed as follows, where can be adjusted to balance the reconstruction loss and KL loss:

Generative adversarial networks

GANs [44] utilize “competing” neural networks for mutual improvement. The two neural networks—the generator and the discriminator—compete in a zero-sum game. The generator () creates instances (e.g. chemical structures of potential drugs) from random noise () sampled from a prior distribution to mimic the training samples, while the discriminator () aims to distinguish between the synthetic data and the training samples.

The learning process involves the optimization of the following loss function:

Here, represents the likelihood applied by the discriminator to a correct sample, while represents the negative likelihood applied by the discriminator to an incorrect sample. This function returns a higher value when the discriminator accurately categorizes samples; thus, the discriminator aims to maximize this function, while the generator aims to minimize it.

Flow-based models

Flow-based generative models [46] generate data according to a target distribution by applying a chain of transformations to a simple latent distribution, often Gaussian, denoted . This transformation applies an invertible function , such that

where the trained model learns parameters . Since is invertible and thus the learned map is bijective, has the same dimensionality as . Often, is a composite function where ; this allows for more complex probability distributions to be modeled. Because each function is invertible, the posterior can be easily computed—the log-likelihood of a single point can be written in terms of its latent variable :

This function is used to train parameters to maximize the probability of observing the data. Various models build upon this premise to represent complex data distributions and capture relationships within sequential data.

Diffusion models

Diffusion models [47] perform a fixed learning procedure, gradually adding Gaussian noise to data over a series of time steps. We define two stages of the model: the noise-adding (forward) and the noise-removing (reverse) process.

In the forward process, each step can be represented as a Markov chain and is comprised of and a small amount of Gaussian noise. We represent this mathematically as follows:

Here, is the data at time t, and denotes the noise schedule. The variance decreases in the forward process so that after many steps, we have .

In the reverse process, we aim to reconstruct the data from the noise. In this process, a denoising function is learned, often modeled by a neural network:

where is the denoising function parameterized by .

To train a diffusion model, we approximate the added noise at each step; the loss function minimizes the difference between the true noise and the model-predicted noise:

Here, means that the time step is drawn uniformly at random from the set , and represents the noise predicted by the model parameterized by . represents the last time step of the model. Once the neural network has been trained, we can sample from the noise distribution and iterate through the reverse process to generate new data.

While the mathematical framework for the diffusion process is generally based on continuous data, adaptations made by Austin et al. [49] allow for smoother implementation with discrete data forms like molecular graphs.

Other models

While we cover the main generative methods used in these fields, a variety of other models also appear in specific applications and tasks in our paper, such as transformers, energy-based models, BERT, and more [50–56]. While not generative models on their own, graph neural networks (GNNs) [57] are often paired with the above generative methods to capture the graph-like structure of molecules. A wide variety of GNN variations exist, including EGNNs [58], message-passing neural networks (MPNNs) [59], graph convolutional networks [60], graph isomorphism networks [61], and convolutional neural networks (CNNs) [62–64]. We discuss GNNs and EGNNs in more detail in the appendix on page 23.

Applications

Molecule

Task background

Molecule generation focuses on the creation of novel molecular compounds for drug design. These generated molecules are intended to be (1) valid, (2) stable, and (3) unique, with an overall goal of pharmaceutical applicability. “Pharmaceutical applicability” is a broad term for a molecule’s binding affinity to various biological targets. While the first three tasks may seem trivial, there are a variety of challenges with simply generating valid and stable molecules. Thus, the field of target-agnostic molecule generation is focused on generating valid sets of molecules without consideration for any biological target. Target-aware molecule generation (or ligand generation) focuses on the generation of molecules for specific protein structures and therefore focuses more on the pharmaceutical component. Finally, 3D conformation generation involves generating various 3D conformations given 2D connectivity graphs.

For training and testing, molecule inputs can be formatted in a variety of ways, depending on the available information or desired output. Molecules can be expressed in 1D format through the simplified molecular-input line-entry system (SMILES), in 2D using connectivity graphs to represent bonds, or in 3D using point cloud embeddings on graph nodes [65].

Target-agnostic molecule design

Overview 

While the task of target-agnostic molecule design may seem simplistically open-ended, there is a vast array of chemical properties and rules that generated molecules must align with to be considered “valid” and “stable.” The determination of “validity” includes a complex combination of considerations, such as electromagnetic forces, energy levels, and geometric constraints, and a well-defined “formula” does not yet exist for predicting the feasibility of molecular compounds. This, combined with the vast space of potential drug-like compounds (up to –), makes brute-force experimentation quite time-consuming [25]. Deep learning can assist in learning abstract features for existing valid compounds and efficiently generate new molecules with a higher likelihood of validity.

Task 

The target-agnostic molecule design task is as follows: given no input, generate a set of novel, valid, and stable molecules.

Datasets 

To learn these abstract constraints, models must learn from large sets of existing valid, stable molecules. The following datasets are most commonly used for this task:

• QM9 [66]—Quantum Machines 9 contains small stable molecules pulled from the larger chemical universe database GDB-17

• GEOM-Drug [67]—Geometric Ensemble of Molecules contains more complex, drug-like molecules, often used to test scalability beyond the simpler molecules of QM9

Metrics 

The most general task within this field is unconditional molecule generation, where models aim to generate a new set of valid, stable molecules with no input. All of these metrics can be evaluated using either QM9 or GEOM-Drug as testing sets.

• Atom Stability—The percentage of atoms with the correct valency

• Molecule Stability—The percentage of molecules whose atoms are all stable

• Validity—The percentage of stable molecules that are considered valid, often evaluated by RDKit

• Uniqueness—The percentage of valid molecules that are unique (not duplicates)

• Novelty—The percentage of molecules not contained within the training dataset

• QED [23]—Quantitative Estimate of Drug-Likeness, a formulaic combination of a variety of molecular properties that collectively estimate how likely a molecule is to be used for pharmaceutical purposes

Note that the novelty metric is sometimes omitted, as argued by Vignac et al. [68], who contend that QM9 is an exhaustive set of all molecules with up to nine heavy atoms following a predefined set of constraints. Therefore, any “novel” molecule would have to break one of these constraints, making novelty a poor indicator of performance. While QED is a well-established metric and may see expanded usage in the future, many current models have focused solely on generating valid molecules and do not report performance on QED.

Models are often evaluated on conditional molecule generation, aiming to generate models that fit desired chemical properties. For evaluation, a property classifier network is trained on half of the QM9 dataset, while the model is trained on the other half. is then evaluated on the model’s generated molecules, and the mean absolute error between the target property value and the evaluated property value is calculated. Below are the six molecular properties considered:

• — Polarizability, or the tendency of a molecule to acquire an electric dipole moment when subjected to an external electric field, measured in cubic Bohr radius ()

• — Highest Occupied Molecular Orbit Energy, measured in millielectron volts ()

• — Lowest Unoccupied Molecular Orbit Energy, measured in millielectron volts ()

• —Difference between and , measured in millielectron volts ()

• — Dipole moment, measured in debyes ()

• — Molar heat capacity at 298.15 , measured in calories per Kelvin per mole

Models 

Approaches to the molecular generation task have seen significant shifts over the past few years, transitioning from 1D SMILES strings to 2D connectivity graphs, then to 3D geometric structures, and finally to incorporating both 2D and 3D information.

Early methods like Character VAE (CVAE) [77], Grammar VAE (GVAE) [78], and Syntax-Directed VAE (SD-VAE) [79] apply VAEs to 1D SMILES string representations of molecule graphs. While 1D SMILES strings can be deterministically mapped to molecular graphs, SMILES falls short in quality of representation—two graphs with similar chemical structures may end up with very different SMILES strings, making it harder for models to learn these similarities and patterns [80].

Junction Tree VAE (JTVAE) [80] was the first model to address this issue by generating 2D graph structures directly. JTVAE generates a tree-structured scaffold and then converges this scaffold into a molecule using a graph message-passing network. This approach allows JTVAE to iteratively expand its molecule and check for validity at each step, resulting in considerable performance improvement over previous SMILES-based methods.

2D graph methods like JTVAE still fall short due to the lack of 3D input; because binding and interaction with other molecules/proteins rely heavily on 3D conformations, models that do not consider 3D information cannot properly represent and optimize properties like binding affinity. Thus, more recent developments include models that incorporate 3D information. Earlier 3D-based methods like E-NF [71] and G-SchNet [69] approached the molecular generation problem with flow-based methods or autoregressive methods (in particular, G-SchNet uses the SchNet architecture developed by Schutt et al. [54]). More recently, a wave of diffusion-based models operate on 3D point clouds, taking advantage of E(3) equivariance and demonstrating superior performance.

EDM [76] provided an initial baseline for the application of diffusion, applying a standard diffusion process to an equivariant GNN with atoms represented as nodes with variables for both scalar features and 3D coordinates. While autoregressive models require an arbitrary ordering of the atoms, diffusion-based methods like EDM are not sequential and do not need such ordering, reducing a dimension of complexity and thus improving efficiency.

Many subsequent models compared themselves with EDM as a baseline on the molecule generation task, seeking to improve upon its performance by adding additional considerations and adjustments. GCDM [72] implements a crossover between geometric deep learning and diffusion, using a geometry-complete perceptron network to introduce attention-based geometric message-passing. While both EDM and GCDM have already demonstrated massive performance improvements, both models still struggle with both large-molecule scalability and diversity in the generated molecules. MDM [73] addressed the scalability issue by pointing out the lack of consideration for interatomic relations in EDM and GCDM. MDM separately defines graph edges for covalent bonds and for Van der Waals forces (dependent on a physical distance threshold ) to allow for thorough consideration of interatomic forces and local constraints. In addition, MDM addressed the diversity issue by introducing an additional distribution-controlling noise variable in each diffusion step. While previous diffusion models operated directly in the complex atomic feature space, GeoLDM [70] applies VAEs to map molecule structures to a lower dimensional latent space for its diffusion. This latent space has a smoother distribution and lower dimensionality, leading to higher efficiency and scalability for large molecules. In addition, conditional generation is improved, as specified chemical properties are more clearly defined within latent spaces than they are in raw format.

While previous models learned exclusively from either 2D or 3D representations, a new wave of models recognize the need for both: a molecule’s 2D connectivity structure is necessary to determine bond types and gather information about chemical properties and synthesis, while the 3D conformation is crucial for its interaction and binding affinity with other molecules. By jointly learning and generating both representations, models can maximize the amount of chemically relevant information and produce higher quality molecular samples. The Joint 2D and 3D Diffusion Model (JODO) [74] uses a geometric graph representation to capture both 3D spatial information and connectivity information, applying score SDEs to this joint representation while proposing a diffusion graph transformer to parameterize the data prediction model and avoid the loss of correlation after noise is independently added to each separate channel. MiDi [75] uses a similar graph representation but instead applies a DDPM. It proposes a “relaxed” EGNN, which improves upon the classical EGNN architecture by exploiting the observation that translational invariance is not needed in the zero center-of-mass subspace. A full overview of the developments described in this section can be seen in Fig. 3.

Figure 3.

An overview of the progress in target-agnostic molecule design over time; shortcomings of previous models are shown in the corresponding pink boxes, with subsequent models solving these shortcomings through novel design choices [70, 73–80].

As shown in Table 1, diffusion-based methods demonstrate significant improvements over previous methods, all achieving over 98.5% in atom stability. However, some models fall behind when extended to the larger GEOM-Drugs dataset, as shown in Table 2, where MiDi distinguishes itself for its capability to generate more stable complex molecules, albeit at the expense of validity. Table 3 illustrates that MDM and GCDM excel at conditional generation tasks, with the former model achieving the best performance in four out of six tasks and the latter outperforming the remaining two. Overall, current models demonstrate high performance on the QM9 dataset, but there is room for improvement when dealing with the more complex molecules found in the GEOM-Drugs dataset.

Table 1.

An overview of relevant molecular generation models; all benchmarking metrics are self-reported unless otherwise noted; all metrics are evaluated with the QM9 dataset; for models with multiple variations, the highest performing version was selected; [^**] represents the current SOTA; as MiDi and MDM use slightly different evaluation conditions, their results are not fully comparable

Model	Type of model	Dataset	At Stb. (%, )	Mol Stb. (%, )	Valid (%, )	Val/Uniq. (%, )
G-SchNet [69]	SchNet	QM9	95.7 [70]	68.1 [70]	85.5 [70]	80.3 [70]
E-NF [71]	EGNN, Flow	QM9	85 [70]	4.9 [70]	40.2 [70]	39.4 [70]
EDM [70]	EGNN, Diffusion	QM9, GEOM-Drugs	98.7	82.0	91.9	90.7
GCDM [72]	EGNN, Diffusion	QM9	98.7	85.7	94.8	93.3
MDM [73]	EGNN, VAE, Diffusion	QM9, GEOM-Drugs	99.2 [74]	89.6 [74]	98.6	94.6
JODO [74]	EGNN, Diffusion	QM9, GEOM-Drugs	99.2	93.4	99.0	96.0
MiDi** [75]	EGNN, Diffusion	QM9, GEOM-Drugs	99.8	97.5	97.9	97.6
GeoLDM** [70]	VAE, Diffusion	QM9, GEOM-Drugs	98.9	89.4	93.8	92.7

Table 2.

Molecular generation molecules evaluated on the larger GEOM-Drugs dataset; all metrics are self-reported; as MiDi uses slightly different evaluation conditions, its results are not fully comparable

Model	Atom Stb. (%, )	Mol Stb. (%, )	Valid (%, )	Val/Uniq. (%, )
EDM [76]	81.30	/	/	/
MDM [73]	/	62.20	99.50	99.00
MiDi [75]	99.80	91.60	77.80	77.80
GeoLDM [70]	84.40	/	99.30	/

Table 3.

Molecular generation models evaluated on the conditional molecule generation task; all metrics are self-reported; all metrics are evaluated with the QM9 dataset

Task ()
Units
EDM [76]	2.76	655	356	584	1.111	1.101
GCDM [72]	1.97	602	344	479	0.844	0.689
MDM [73]	1.591	44	19	40	1.177	1.647
GeoLDM [70]	2.37	587	340	522	1.108	1.025

Target-aware molecule design

Overview 

Contrasting target-agnostic molecule design, target-aware design involves generating molecules based on specific biological targets. Within target-aware design, two primary approaches exist: ligand-based drug design (LBDD) and structure-based drug design (SBDD). LBDD models often utilize the amino acid sequences of target proteins, leveraging the characteristics and features of known ligands to build new molecules with similar properties. By contrast, SBDD models use the 3D structure of the target protein to design a corresponding molecular structure. LBDD models are most useful when the 3D structure is not experimentally available, but are limited in novelty because they only learn from existing bindings [37]. When the 3D structure of the target protein is available, SBDD models are generally preferred, as they consider crucial 3D information.

Task 

Given input target information, typically in the form of protein amino acid sequences in LBDD and protein 3D structure in SBDD, these approaches generate molecules that exhibit high binding affinity and potential interactions with this target.

Datasets 

The following datasets are used for target-aware molecule design. CrossDocked2020 [81] is currently the most heavily used, as the cross-docking technique allows for the generation of combinatorially large quantities of data by “mixing and matching” similar ligand–protein pairs (22.5M compared with 40K in Binding MOAD [82]).

• CrossDocked2020 [81]—Contains ligand–protein complexes generated by cross-docking within clusters of similar binding sites called ”pockets”

• ZINC20 [83]—Fully enumerated dataset of possible ligands

• Binding MOAD [82]—Binding Mother of All Databases, a subset of PDB [84] containing experimentally determined protein–ligand pairings

Metrics 

Target-aware molecule design utilizes the following metrics. Beyond affinity/applicability metrics, an additional consideration for diversity is considered, as a diverse array of potential options for a given target can provide more flexibility in the drug development process.

• Vina Score [85]—Scoring function supported by the Vina platform that returns a weighted sum of atomic interactions and is useful for docking

• Vina Energy [85]—Energy prediction by the Vina platform that is used to measure binding affinity

• High Affinity Percentage—Percentage of molecules with lower Vina energy than the reference (ground-truth) molecule when binding to the target protein

• QED [23]—Quantitative Estimate of Drug-Likeness, which is also used in target-agnostic generation (see page 5)

• SAscore [86]—Synthetic Accessibility Score, a formulaic combination of molecular properties that determine how easy a molecule is to create in a real lab setting

• Diversity—The diversity of generated molecules for each specific binding site, measured by Tanimoto similarities [87] between pairs of Morgan fingerprints

Models 

LBDD models incorporate transformer architectures to generalize properties of learned ligands. For example, DrugGPT [88] is a recent autoregressive model that uses transformers to train numerous protein–ligand pairs. In their model, the ligand SMILES and protein amino acid sequence are tokenized for training, and the model produces viable SMILES ligand outputs. Generally, with improving protein structure prediction methods (see page 9) and increasing access to structural information as a whole, SBDD methods have become more prevalent than LBDD methods; thus, we explore SBDD methods below in more detail.

LiGAN [89] introduces the idea of a 3D target-aware molecule output, fitting molecules into grid formats for learning with a CNN and training the model under a VAE framework. Pocket2Mol [90] places more emphasis on the specific “pockets” on the target protein to which the molecules bind, using an EGNN and geometric vector MLP layers to train on graph-structured input. Luo et al. [91] directly model the probability of atoms occurring at a certain position in the binding site, taking advantage of invariance through the SchNet [54] architecture.

Recent SBDD models have also popularized the use of diffusion models.

TargetDiff [92] performs diffusion on an EGNN (with many similarities to EDM [76]) to learn the conditional distribution. To optimize the binding affinity, Guan et al. [92] note that the flexibility of atom types should be low, which is reflected by the entropy of the atom embedding. Schneuing et al. [93] propose DiffSBDD, which includes two sub-models: DiffSBDD-cond and DiffSBDD-inpaint. DiffSBDD-cond is a conditional DDPM that learns a conditional distribution in a similar way as TargetDiff. In our benchmarking, we focus on the higher performing model, DiffSBDD-inpaint, which applies the inpainting approach (traditionally applied to filling parts of images) by masking and replacing segments of the ligand–protein complex.

As shown in Table 4, DiffSBDD leads in Vina score and diversity, while TargetDiff leads in high affinity. Interestingly, diffusion-based methods seem to be outperformed by the MLP used in Pocket2Mol when it comes to drug-like metrics like QED and SA. However, Guan et al. [92] note that adjustments to TargetDiff, such as switching to fragment-based generation or predicting atom bonds, could improve performance on QED and SA.

Table 4.

An overview of relevant target-aware molecular generation models; all benchmarking metrics are self-reported unless otherwise noted; [^**] represents the current SOTA; as each paper uses slightly different benchmarking methods, their results may not be fully comparable; all metrics are evaluated with the CrossDocked2020 dataset

Model	Type of model	Dataset	Vina ()	Affinity (%, )	QED ()	SA ()	Diversity ()
Luo et al. [91]	SchNet	CrossDocked2020	-6.344	29.09	0.525	0.657	0.720
LiGAN [89]	CNN, VAE	CrossDocked2020	-6.144 [91]	21.1 [92]	0.39 [92]	0.59 [92]	0.66 [92]
Pocket2Mol** [90]	EGNN, MLP	CrossDocked2020	-5.14 [92]	48.4 [92]	0.56 [92]	0.74 [92]	0.69 [92]
TargetDiff** [92]	EGNN, Diffusion	CrossDocked2020	-6.3	58.1	0.48	0.58	0.72
DiffSBDD** [93]	EGNN, Diffusion	CrossDocked, MOAD	-7.333	∖	0.467	0.554	0.758

Protein

Task background

Proteins are large biomolecules that contain one or more long chains of amino acids. Each amino acid is a molecular compound that contains both an amino (-NH2) and a carboxylic acid (-COOH) [94]. While there are over 500 naturally occurring amino acids, only 22 are proteinogenic (protein-building) and thus relevant to the protein generation task [95]. Thus, in addition to 3D structural representations, proteins can be represented through their amino acid sequences, assigning each amino acid a letter label and representing each long chain as a string of labels. This sequential representation for amino acids mirrors the sequence structure of human language, allowing for natural language models to be applied in ways that would not be possible for the previously discussed molecule generation task. Within protein generation, several generative subtasks can be defined. Representation learning involves creating meaningful embeddings for protein inputs, which improves the data space for other models to train on. Structure prediction involves the generation of a protein structure for its corresponding amino acid sequence, historically a challenging task due to the vast conformational space. Sequence generation describes the inverse, creating a protein sequence for its corresponding structure. Finally, backbone design refers to creating protein structures from scratch, which forms the core of the de novo design task.

We also briefly discuss antibody generation due to its high relevance within the protein generation field. In particular, antibodies are Y-shaped proteins used by the immune system to identify and bind to bacteria known as antigens. While many protein design models use multiple sequence alignment (MSA) to map evolutionary relationships between related sequences, MSAs are not always available for antibody sequences, and antibody-specific models cannot rely on this input. Additionally, binding regions are specifically defined for antibodies, contained within six complementarity-determining regions (CDRs). The CDR-H3 region is particularly diverse and complex, leading to a specialized task for the reconstruction of this region, known as CDR-H3 generation. We discuss antibody-specific methods within each corresponding subtask, and we include a more detailed discussion of antibody generation in the appendix on page 26.

Finally, we provide an additional section on peptide generation due to its high relevance and more specialized applications. Advances in drug delivery and synthesis technology have broadened its therapeutic potential, and recent innovations in peptide drug discovery have significantly improved treatments for type 2 diabetes with the creation of glucagon-like peptide-1 receptor agonists (e.g. liraglutide and semaglutide). Protein generation, while related, differs from peptide generation in length and complexity. Peptides are shorter (often no more than 50 amino acids) and have greater flexibility; thus, distinct computational models are needed to capture this differentiation.

Protein representation learning

Overview 

Protein representation learning involves learning embeddings to convert raw protein data into latent space representations, thereby extracting meaningful features and chemical attributes. Specifically, given a protein , where each represents an amino acid (sequence-based) or atom coordinate (structure-based), the goal is to learn an embedding , where each represents a -dimensional token representation for amino acid . Although representation learning is not a generative task on its own, these embeddings create “richer” data spaces for other generative models to train on. Hence, we briefly discuss them here. For a more in-depth analysis of protein representation learning models, please refer to the appendix on page 23.

Structure prediction

Overview 

Generating 3D structures of proteins from their amino acid sequence is a challenging and important task in drug design. Historically, techniques like protein threading [96] and homology modeling [97] has been used to predict structures; however, these methods have fallen short due to a lack of computational power and the difficulty of finding structures in the vast conformational space. New research in computational modeling has used various deep learning architectures to discover information from the amino acid sequences to generate accurate 3D structures. Current models have achieved impressive accuracy in structure prediction, but there is room for improvement in terms of speed and scale.

Task 

Given a protein amino acid sequence, generate a set of 3D point coordinates for each amino acid residue, aiming to replicate a target ground-truth structure as closely as possible.

Datasets 

Unlike many of the other fields mentioned previously, the field of protein structure prediction benefits from a widely standardized benchmarking task through the Critical Assessment of Protein Structure Prediction (CASP). CASP conducts biennial testing on models using solved protein structures that have not been released to PDB.

• PDB [84]—Protein Data Bank, a central archive for all experimentally determined protein structures, widely used in almost all protein structure-related tasks

• CASP14 [98]—14th Critical Assessment of Protein Structure Prediction, a set of unreleased PDB structures used to create a standardized blind testing environment

• CAMEO [99]—Continuous Automated Model Evaluation, a complement to CASP that conducts weekly blind tests using around 20 pre-released PDB targets (to provide more continuous feedback, as CASP is biennial)

Metrics 

Models are evaluated by comparing each protein’s ground-truth structure with the generated structure. Three different approaches are taken to evaluate structural similarity:

• RMSD—Root-Mean-Square Deviation directly compares ground-truth positions with generated positions for each amino acid. If each denotes the distance between each ground truth and generated amino acid position, with total amino acids, we have

  

• GDT-TS [100]—Global Distance Test-Total Score finds the most optimal superposition between two structures, searching for the highest number of corresponding residues that are within a distance threshold from each other. The GDT-TS aims to represent global fold similarities.

• TM-score [101]—Template Modeling score a similarity scoring formula that adjusts the GDT metric, normalizing for protein sequence length to avoid dependency on protein size, evaluating all residues beyond just those within the proposed cutoff for a more cohesive score. The TM-score aims to represent both global fold and local structural similarities.

• LDDT [102]—Local-Distance Difference Test, a superpos.-free metric based on local distances between atoms. For each atom pair, its local distance is “preserved” if the generated local distance is within a given threshold of the ground-truth distance, and the proportion of preserved distances is calculated. The LDDT can protect against artificially unfavorable scores when considering flexible proteins with multiple domains.

Models 

AlphaFold2 [103] is a landmark model that uses deep learning techniques to compete with experimental methods. AlphaFold2 integrates numerous layers of transformers in an end-to-end approach. The transformers incorporate information from the MSA and pair representations to explore the folding space, potential orientations of amino acids, and overall structure based on pairwise distances. The MSA aligns multiple related protein sequences to create a 2D representation that informs the transformer architecture. Additionally, AlphaFold2 employs invariant point attention (IPA) for spatial attention, while the transformer captures interactions along the chain structure. Notably, AlphaFold2 introduces novel constraints from experimental data, which record probable distances between residues, preferred orientations between residues, and likely dihedral angles for the covalent bonds in the backbone.

Proposed in 2020, trRosetta [107], i.e. transform-restrained Rosetta, is another model that uses a deep residual network with attention mechanisms. Upon inputting MSA information, trRosetta predicts distances and orientations for residue pairs, which are then utilized to construct the 3D structure using a Rosetta protocol. Despite their advancements, both trRosetta and AlphaFold2 face several challenges, including their reliance on the MSA representation, limitations to natural proteins, and high computational requirements. Recently, RoseTTAFold [106], which replaced trRosetta [107], demonstrated performance comparable with AlphaFold2 based on CASP14 test data. Importantly, RoseTTAFold can generate samples within 10 min, which is around 100 times faster than AlphaFold2. RoseTTAFold employs a three-track neural network that simultaneously learns from 1D sequence-level, 2D distance map-level, and 3D backbone coordinate-level information with attention mechanisms integrated throughout. RoseTTAFold exhibits robust performance in predicting protein complexes, whereas AlphaFold2 excels primarily for single protein structure prediction.

Building on these techniques, ESMFold [105] is a language model that predicts protein structure by leveraging ESM-2 representations. The output embeddings from ESM-2 are passed to a series of self-attending “folding blocks.” A structure module, featuring an SE(3) transformer architecture, generates the final structure predictions. Since ESMFold uses ESM-2 representations instead of MSA representations, this model offers faster processing with competitive scores on CAMEO and CASP14. EigenFold [104] is another model that applies diffusion models to generate protein structures. The model represents the protein as a system of harmonic oscillators. Then, the structure can be projected onto the eigenmodes of that system during the forward process. In the reverse process, the rough global structure is sampled before refining local details. As a score-based model, EigenFold is not as computationally intensive but still underperforms in accuracy and range compared with other models. Protein structure prediction models described before are summarized in Table 5.

Table 5.

An overview of relevant protein structure prediction models; all metrics are self-reported unless otherwise noted; scores are provided as mean performance on single-structure tasks; lDDT represents the lDDT metric computed specifically using coordinates for the alpha carbon of each residue; [^**] denotes the current SOTA

Model	Type of model	Dataset	CAMEO				CASP14
			RMSD (Å, )	TMScore ()	GDT-TS ()	lDDT ()	TMScore ()
AlphaFold2** [103]	Transformer	CASP14	3.30 [104]	0.87 [104]	0.86 [104]	0.90 [104]	0.38 [105]
RoseTTAFold [106]	Transformer	CAMEO, CASP14	5.72 [104]	0.77 [104]	0.71 [104]	0.79 [104]	0.37 [105]
ESMFold [105]	Transformer	CAMEO, CASP14	3.99 [104]	0.85 [104]	0.83 [104]	0.87 [104]	0.68
EigenFold [104]	Diffusion	CAMEO	7.37	0.75	0.71	0.78	∖

Antibody structure prediction 

A class of models has also been developed specifically catering to antibody structure prediction. As discussed previously, MSA alignment cannot be used for antibody input, rendering general models like AlphaFold highly inefficient and slow in the context of antibody prediction. IgFold [108] uses sequence embeddings from AntiBERTy [109] and IPA to predict antibody structures, achieving SOTA generation speed with comparable accuracy with other models in the field. tFold-Ab [110] performs comparably with IgFold, generating full-atom structures more efficiently by reducing reliance on external tools like Rosetta energy functions. For a more in-depth analysis of the datasets, task definition, metrics, and performance, refer to the appendix on page 27.

Sequence generation

Overview 

Sequence generation, also known as inverse folding or fixed-backbone design, entails the inverse task of structure prediction. Generating amino acid sequences that can fold into target structures is crucial for designing proteins with desired structural and functional properties. As with molecules and protein structures, the space of valid sequences is vast; this figure has been estimated to lie between and [111]. In addition, the process of protein folding is naturally complex and difficult to predict.

To address these challenges, a variety of deep learning methods have been applied to represent the distribution of sequences with respect to structural information. While we briefly discuss some preliminary methods that are structure-agnostic, we place the highest focus on models that target specific protein structures.

Task 

Given a fixed protein backbone structure, generate a corresponding amino acid sequence that will fold into the given structure.

Datasets 

Models in this field utilize the following datasets. Models primarily use CATH for training, with some using various augmentation methods utilizing UniRef and UniParc. CATH and TS500 are most frequently used for evaluation. To produce a standardized benchmark, Yu et al. [112] created a set of 14 known de novo protein structures that do not exist in CATH to avoid data contamination.

• PDB [84]—Protein Data Bank, a comprehensive protein structure dataset (see page 9)

• UniRef [113]—A clustered version of the Unified Protein KnowledgeBase (UniProtKB), part of the central resource UniProt, which is a curated and labeled set of protein sequences and their functions

• UniParc [113]—A larger dataset of protein sequences, part of the central resource Uniprot, which includes UniProtKB and adds proteins from a variety of other sources

• CATH [114]—A classification of protein domains (subsequences that can fold independently) into a hierarchical scheme with four levels: class (C), architecture (A), topology (T), and homologous superfamily (H). Using their classification, the authors also provide a diverse set of proteins that have minimal overlap and sequence similarity.

• TS500 [115]—A subset of 500 proteins from PDB [116] filtered by sequence identity using the PISCES network. Li et al. also created a smaller subset, TS50, with additional filters to control for sequence length and fraction of surface residue.

Metrics 

While many models perform their testing, we use results from the independent benchmark created by Yu et al. for fair comparison. We list the evaluated metrics below. Note that while Yu et al. did not evaluate perplexity, we include it here due to its frequent use in sequence design method evaluations.

• AAR—Amino Acid Recovery, the proportion of matching amino acids between the generated and native sequences

• Diversity—The average difference between pairs of generated sequences, measured using Clustalw2 [117]

• RMSD—Root-Mean-Square Deviation, a structural comparison between two structures (see page 9). In the context of sequence generation, the proposed sequences are folded into structures before comparison with native backbone structures.

• Nonpolar Loss—A metric measuring the rationality of polar amino acid types within the folded structure, where higher presence of nonpolar amino acids on the surface results in higher loss

• PPL— Perplexity, an exponentiation of cross-entropy loss, representing the inverse likelihood of a native sequence appearing in the predicted sequence distribution. For a series of amino acids , we can express perplexity as

  

  Perplexity is calculated individually for each protein in the test set and averaged to produce a final PPL value.

Models 

A preliminary class of models generates novel protein sequences without considering a fixed backbone target, aiming to capture the unconditional distribution of amino acid sequence space. ProteinVAE [118] utilizes ProtBERT [119] to reduce raw input sequences into latent representations, employing an encoder–decoder framework with position-wise multi-head self-attention to capture long-range dependencies in these sequences. ProT-VAE [120] uses a different pre-trained language model ProtT5NV (https://www.nvidia.com/en-us/gpu-cloud/bionemo/). It includes an inner family-specific encoder–decoder layer to learn parameters relevant to specific protein families. Conversely, ProteinGAN [121] uses a GAN architecture to generate protein sequences. The model’s efficacy is exemplified through the example of malate dehydrogenase, demonstrating its potential to generate fully functional enzyme proteins. While these approaches demonstrate relative success in generating valid and diverse sets of protein sequences, models that operate entirely in sequence space cannot consider crucial structural information. This limitation restricts their ability to capture the full range of constraints and dependencies between amino acid residues.

The primary class of models in this field receives fixed backbone targets as input, generating corresponding amino acid sequences. ProteinSolver [122] draws connections between generating backbone structures and solving Sudoku puzzles, arguing that both are forms of constraint satisfaction problems where positional information imposes constraints on the labels that can be assigned to each “node.” After finding a GNN architecture that can effectively solve Sudoku puzzles, Strokach et al. [122] apply a similar architecture to the task of protein sequence design. In this design, node attributes encode amino acid types and edge attributes encode relative distances between pairs of residues. PiFold [123] extends this approach by introducing more comprehensive feature representations, including explicit distance, angle, and direction information in its node and edge features. Anand et al. [124] design a 3D CNN that directly learns conditional distributions for each residue given previous amino acid types, relative distances for heavy atoms, and torsional angles for side chains. Using these learned distributions, the 3D CNN autoregressively generates potential sequences. ABACUS-R [125] incorporates a pre-trained transformer to infer a residue’s amino acid types from nearby residues. To generate valid sequences, the model iteratively updates subsets of residues based on their environments, gradually constructing self-consistent sequences. ProRefiner [126] improves upon this design by introducing entropy scores for each prediction. While ABACUS-R uses every residue in the neighborhood for refinement, ProRefiner masks out high-entropy (low-confidence) predictions. By filtering out low-quality predictions, ProRefiner mitigates error accumulation from incorrect predictions.

To better model input protein structures, GPD [130] uses the Graphormer [131] architecture, which is a modified transformer for graph-structured data. GPD also uses Gaussian noise and random masks to improve diversity and recovery. GVP-GNN [127] uses a simple yet novel geometric representation for all nodes in the system. Rather than individually encoding vector features (like relative node orientations) for each node, these features are directly represented as geometric vectors that transform accordingly alongside global transformations. This approach defines a global geometric orientation rather than independent features of each node. ESM-IF1 [128] extends upon the representations in GVP-GNN by attaching a generic transformer block and training on an expanded dataset. To generate additional training examples, ESM-IF1 uses MSA Transformer [132], a representation learning model, to rank the sequences in the UniRef50 dataset by predicted LDDT scores. The top 12 million of these sequences are assigned corresponding structures through predictions made by AlphaFold2, producing a collection of sequence-structure pairs much larger than that of experimentally determined pairs. While previous methods require a fixed decoding order, ProteinMPNN [129] implements an order-agnostic autoregressive approach, which allows for a flexible choice of decoding order based on each specific task.

We report the benchmark results measured by Yu et al. in Table 6. ProteinMPNN generates the most accurate sequences, leading all methods in sequence recovery, RMSD, and nonpolar loss. GPD remains the most time-efficient method, generating sequences around three times faster than ProteinMPNN. Performance on diversity varies, but this can often be artificially controlled by adjusting a noise hyperparameter during testing to increase variation. Note that ProRefiner is not listed; ProRefiner primarily acts as an add-on module for existing methods and reports 2–7 percentage points of improvement on AAR when used to refine sequences for GVP-GNN, ESM-IF1, and ProteinMPNN.

Table 6.

An overview of relevant sequence design methods; Div. and Non. refer to diversity and nonpolar loss, respectively; results are reported by Yu et al. [112]; for GVP-GNN, we report self-evaluated AAR on a CATH test split; [^**] denotes the current SOTA

Model	Type of model	Dataset	AAR (%, )	Div. ()	RMSD (Å, )	Non. ()	Time (s, )
ProteinSolver [122]	GNN	UniParc	24.6	0.186	5.354	1.389	180
3D CNN [124]	CNN	CATH	44.5	0.272	1.62	1.027	536544
ABACUS-R [125]	Transformer	CATH	45.7	0.124	1.482	0.968	233280
PiFold [123]	GNN	CATH, TS50/TS500	42.8	0.141	1.592	1.464	221
GVP-GNN [127]	GNN	CATH, TS50	44.9* [127]	∖	∖	∖	∖
ESM-IF1** [128]	Transformer	CATH, UniRef+	47.7	0.184	1.265	1.201	1980
ProteinMPNN** [129]	MPNN	CATH	48.7	0.168	1.019	1.061	112
GPD [130]	Transformer	CATH	46.2	0.219	1.758	1.333	35

In general, sequence generation remains a challenging field, as current SOTA models only recover fewer than half of the target amino acid residues.

Backbone design

Overview 

Like molecule generation, generating novel proteins from scratch can directly expand the library of available proteins capable of performing highly complex and versatile functions. While other areas such as structure prediction and sequence generation contribute to the overall drug design process, backbone design lies at the core of de novo design, where new protein structures can be created entirely from scratch.

As seen in molecule design, protein design contains a similar distinction between structure and sequence. Some models generate 1D amino acid sequences, while others directly generate 3D structures, with some co-designing both 1D sequences and 3D structures.

Datasets 

Models in this field utilize the following datasets:

• PDB [84]—Protein Data Bank, a comprehensive protein structure dataset (see page 9)

• AlphaFoldDB [133]—AlphaFold Database, an expanded protein structural dataset created by using AlphaFold2 to predict structures for corresponding sequences in the UniRef dataset

• SCOP [134]—Structural Classification of Proteins, a classification of proteins by homology and structural similarity. SCOP has been updated several times to include additional categorizations and features, with many recent models using the extended SCOPe [135] database.

• CATH [114]—A classification of protein domains into a hierarchical scheme with four levels, also used in sequence generation (see page 10)

Tasks 

The backbone design task involves designing a protein backbone structure either from scratch or based on existing context. This involves generating coordinates for the backbone atoms for each amino acid (nitrogen, alpha-carbon, carbonyl, and oxygen atom). External tools like Rosetta (https://rosettacommons.org/software.) can be used for side-chain packing, generating the remaining atoms.

• Context-Free Generation—Given no input, the goal is to generate a diverse set of protein structures. This task is evaluated using the self-consistency TM (scTM) score.

• Context-Given Generation—This is an inpainting task for proteins. Given a motif (a set of existing amino acid residues for a native protein), the goal is to accurately fill in the missing residues according to the native protein, which is evaluated using a variety of similarity metrics like AAR, PPL, and RMSD.

Metrics 

Generated backbones should be highly designable. High designability is generally determined by the ability of a structure to be generated by a corresponding amino acid sequence. While lab testing is optimal, this is not always feasible, so the folding process is simulated by other generative models. Thus, Trippe et al. [136] proposed the scTM approach, which includes the following steps: a proposed structure is fed into a sequence-prediction model (typically ProteinMPNN [129]) to produce a corresponding amino acid sequence, and then fed back into a structure prediction model (typically AlphaFold2 [103]) to produce a sample structure. The TM-score (see page 9) between the generated structure and this sample structure is calculated.

• scTM [136]—Self-consistency TM-score, an approach proposed by Trippe et al. to simulate the folding process, described in detail above. Scores of scTM are typically considered designable, so the percentage of generated structures with scTM is often reported.

• scRMSD—Self-consistency RMSD, identical to the scTM but uses RMSD instead of the TM-score for evaluation. A score of scRMSD is typically used as a cutoff.

• AAR— Amino acid recovery, a comparison between the ground truth and the generated amino acid sequences. The AAR is also measured in antibody representation learning (see page 26).

• RMSD—Root-mean-square deviation, measures distances between the ground truth and the generated residue coordinates. The RMSD is also used in protein structure prediction (see page 9).

Models 

ProtDiff [136] represents each residue with 3D Cartesian coordinates and uses a particle filtering diffusion approach. However, 3D Cartesian point clouds do not mirror the folding process to create protein structures—FoldingDiff [138] instead uses an angular representation for the protein structure, which more closely mirrors the rotational energy-optimizing protein folding process. FoldingDiff treats the protein backbone structure as a sequence of six angles representing orientations of consecutive residues. It denoises from a random, unfolded state to a folded structure using a DDPM and a BERT architecture. LatentDiff [137] initially uses an equivariant protein autoencoder with GNNs to embed proteins into a latent space. Subsequently, it uses an equivariant diffusion model to learn the latent distribution. This process is analogous to GeoLDM [70] for molecule design. Notably, LatentDiff’s sampling on its latent space is ten times faster than sampling on raw protein space.

The above models have shown relatively high performance in generating shorter proteins (up to 128 residues in length) but struggle with larger and more complex proteins [140]. To address longer protein structures, other methods use frame-based construction methods. This representation was initially demonstrated in the architecture of AlphaFold2 [103] in structure prediction, known as IPA. In this paradigm, each residue is represented by orientation-preserving rigid body transformations (reference frames), which can be consistently defined regardless of global orientation. This allows for a more generalized representation than a series of 3D point clouds. Genie [140] performs discrete-time diffusion using a cloud of frames determined by a translation and rotation element to generate backbone structures. During each diffusion step, Genie computes the Frenet–Senet frames and uses paired residue representations and IPA for noise prediction. FrameDiff [139] also parameterizes the backbone structures based on the frame manifold, using a score-based generative model. This approach establishes a diffusion process on , the manifold of frames, invariant to translations and rotations. Then, the neural network predicts the denoised frame and torsion angle using IPA and a transformer model. Finally, RFDiffusion [141] combines the powerful structure prediction methods from RoseTTAFold with diffusion models to generate protein structures. RFDiffusion fine-tunes the RoseTTAFold weights and inputs a masked input sequence and random noise coordinates to iteratively generate the backbone structure. RFDiffusion also “self-conditions” on the predicted final structure, leading to improved performance. RFDiffusion is a large pre-trained model with significantly more parameters than the other two frame-based models, enabling it to outperform the other frame-based models. GPDL [145] utilizes a similar technique to RFDiffusion, using ESMFold instead of RoseTTAFold as its base structure prediction model. Additionally, it incorporates the ESM2 language model to extract important evolutionary information from input sequences to ESMFold. Due to ESMFold’s superior efficiency in structure prediction, GPDL generates backbone structures 10–20 times faster than RFDiffusion.

Another class of models aims to co-design both protein sequence and structure simultaneously. GeoPro [142] uses an EGNN to encode and predict 3D protein structures, designing a separate decoder to decode protein sequences from these learned representations. Protpardelle [144] creates a “superposition” over the possible sidechain states and collapses them during each iterative update step in the reverse diffusion process. The backbone is updated in each iterative step, while the sidechains are chosen probabilistically by another network to update. ProtSeed [143] uses a trigonometry-aware encoder that computes constraints and interactions from the context features and uses an equivariant decoder to translate proteins into their desired state, updating the sequence and structure in a one-shot manner. Anand et al. [124] use IPA as mentioned above, performing diffusion in frame space to efficiently generate protein sequences and structures.

For a comparison of performance between various models, see Table 7. For an overview of developments described in this section, see Fig. 4. Note that as seen in molecule generation, we observe a progression from 1D-based (amino acid) models to 3D structure-based models to 1D/3D co-design; in addition, the field of protein design faces analogous questions of complexity scaling and latent space regularization.

Table 7.

An overview of relevant backbone design methods; “AFDB” refers to AlphaFoldDB; “Design” refers to designability, defined by Lin et al. [140], where any proteins with scRMSD and pLDDT (a local distance metric used in AlphaFold2 [103]) are considered designable; all metrics are evaluated with the PDB dataset, while [^*] denotes results tested only on -lactamase metalloproteins extracted from PDB; [^**] denotes the current SOTA

Model	Type of model	Dataset	Context-Free		Context-Given
			scTM (%, )	Design. (%, )	PPL ()	AAR (%, )	RMSD (Å, )
LatentDiff [137]	EGNN, Diffusion	PDB, AFDB	31.6	∖	∖	∖	∖
FoldingDiff [138]	Diffusion	CATH	14.2 [137]	∖	∖	∖	∖
FrameDiff [139]	Diffusion	PDB	84	48.3 [140]	∖	∖	∖
Genie [140]	Diffusion	SCOPe, AFDB	81.5	79.0	∖	∖	∖
RFDiffusion** [141]	Diffusion	PDB	∖	95.1 [140]	∖	∖	∖
ProtDiff [136]	EGNN, Diffusion	PDB	11.8 [137]	∖	∖	12.47* [142]	8.01* [142]
GeoPro [142]	EGNN	PDB	∖	∖	∖	43.41*	2.98*
ProtSeed [143]	MLP	CATH	∖	∖	5.6	43.8	∖
Protpardelle [144]	Diffusion	CATH	85	∖	∖	∖	∖

Figure 4.

An overview of the progress in protein generation over time; shortcomings of previous models are shown in the corresponding pink boxes, with subsequent models solving these shortcomings through novel design choices; for consistency, only methods that generate proteins from scratch (without fixed backbone or sequence input) are depicted [118, 120, 121, 127, 136–143].

Antibody CDR-H3 Generation 

As mentioned previously, antibody generation primarily focuses on the generation of a particular region known as the CDR-H3 region. Similar to protein generation, models in CDR-H3 generation have transitioned from sequence-based methods like the LSTM used by Akbar et al. [146] to sequence-structure co-design pioneered by RefineGNN [147] through iterative refinement. Notably, some models extend beyond the CDR-H3 generation task, aiming to tackle multiple parts of the antibody pipeline at once. dyMEAN [148] is an end-to-end method incorporating structure prediction, docking, and CDR-H3 generation into a singular model. For a more in-depth analysis of datasets, task definition, metrics, and performance, refer to the appendix on page 28.

Peptide design

Overview 

While we have discussed the monumental and robust models developed for protein generation, it is necessary to have models tailored for peptide-specific needs due to the inherently intricate and context-dependent nature of peptide structure, as well as the highly diverse array of downstream applications [149]. This section briefly explores four different applications for AI in peptide generation, focusing on four SOTA models: MMCD (Multi-Modal Contrastive Diffusion Model), PepGB (Peptide–protein interaction via Graph neural network for Binary prediction), PepHarmony, and AdaNovo.

Peptide generation 

In peptide generation, like protein backbone design, models aim to generate novel peptides from scratch. MMCD [150] is a diffusion-based model for therapeutic peptide generation that co-designs peptide sequences and structures (backbone coordinates). It employs a transformer encoder for sequences and an EGNN for structures, along with contrastive learning strategies to align sequence and structure embeddings and differentiate therapeutic and non-therapeutic peptide embeddings. MMCD outperforms baselines in both sequence and structure generation tasks, as demonstrated by testing on datasets of antimicrobial peptides and anticancer peptides.

Peptide–protein interaction 

For peptide–protein interactions, models aim to predict the physical binding site for a proposed peptide–protein pair. PepGB [151] is a GNN-based model for facilitating peptide drug discovery. It predicts peptide–protein interactions and leverages graph attention neural networks to learn interactions between peptides and proteins. PepGB was trained on a binary interaction benchmark dataset of protein–peptide and protein–protein interactions. A mutation dataset of peptide analogs targeting MDM2 was used for validating PepGB, and a large-scale peptide sequence dataset from UniProt was used for pre-training. PepGB consistently outperforms baselines in predicting peptide-protein interactions for novel peptides and proteins, with an increase of at least 9%, 9%, and 27% in AUC-precision scores and 19%, 6%, and 4% in AUC-recall scores under novel protein, peptide, and pair settings, respectively.

Peptide representation learning 

As with protein representation learning, models in peptide representation learning aim to convert raw peptide sequences into latent representations that capture valuable information. PepHarmony [152] is a multi-view contrastive learning model that integrates both sequence and structural information for enhanced peptide representation learning. It employs a sequence encoder (ESM) and a structure encoder (GearNet), which are trained together using contrastive or generative learning. PepHarmony utilizes data from conventional datasets like AlphaFoldDB and PDB while also employing a cell-penetrating peptide dataset (compiled from a variety of existing datasets), a solubility dataset (PROSOS-II [153]), and an affinity dataset (DrugBank [154]). Zhang et al. report that PepHarmony demonstrates superior performance in downstream tasks such as cell-penetrating peptide prediction, peptide solubility prediction, peptide–protein affinity prediction, and self-contact map prediction. When compared with general protein representation learning methods like ESM2 and GearNet, PepHarmony outperforms baseline and fine-tuned versions of these models in most evaluation metrics, including accuracy, F1 score, area under the receiver operating characteristic curve, and correlation coefficients.

Peptide sequencing 

Mass spectrometry has played a crucial role in analyzing protein compositions from physical samples, but various forms of noise have posed challenges in extracting information from these reports. In peptide sequencing, models aim to address this challenge by predicting amino acid sequences given mass spectra data. AdaNovo [155] is a SOTA model for de novo peptide sequencing, composed of a mass spectrum encoder (MS Encoder) and two peptide decoders inspired by the transformer architecture. AdaNovo significantly improves upon previous models like DeepNovo [156], PointNovo [157], and Casanovo [158] in terms of peptide-level and amino acid-level precision across various species. For example, in a human dataset, AdaNovo achieves a peptide-level precision of 0.373 and an amino acid-level precision of 0.618, outperforming DeepNovo (0.293 and 0.610), PointNovo (0.351 and 0.606), and Casanovo (0.343 and 0.585). AdaNovo’s success is attributed to its innovative use of conditional mutual information and adaptive training strategies, which enhance its ability to identify post-translational modifications and handle noisy data typically associated with mass spectrometry.

Current trends

The drug design process, marked by a history of high complexity and cost, is poised for a transformative shift fueled by generative AI. AI-based methods are driving faster development and reducing costs, resulting in more effective and accessible pharmaceuticals for the public. Within the realm of generative AI, a notable shift has occurred; the emergence of GNNs and graph-based methods has fueled the transition from sequence-based approaches to structure-based approaches, ultimately leading to the integration of both sequence and structure in generation tasks.

Within the field of molecular generation, we are witnessing the recent dominance of graph-based diffusion models. These models take advantage of E(3) equivariance to achieve SOTA performance, with leaders like GeoLDM and MiDi excelling in target-agnostic design, and TargetDiff, Pocket2Mol, and DiffSBDD excelling in target-aware design. Finally, Torsional Diffusion outperforms all counterparts in molecular conformation generation. Additionally, we observe a shift from sequence to structural approaches in target-aware molecule design, where SBDD approaches demonstrate clear advantages over LBDD approaches, which operate with amino acid sequences.

Within protein generation, a shift from sequence to structure is evident, as exemplified by the emergence of structure-based representation learning models like GearNET. These models leverage established sequence-based representation models such as ESM-1B and UniRep, recognizing the importance of 3D structural information in the protein generation process. AlphaFold2 remains a clear SOTA model for structure prediction. Similar to molecule generation, a wave of diffusion models are now tackling the protein scaffolding task, with RFDiffusion emerging as the top-performing model.

Challenges and future directions

While the prospects for generative AI in drug design are promising, several issues must be addressed before we can embrace ML-driven de novo drug design. The main areas for improvement include increasing performance in existing tasks, defining more applicable tasks within molecule, protein, and antibody generation, and exploring entirely new areas of research.

Current generative models still struggle with a variety of tasks and benchmarks. Within molecule generation, we face the following challenges:

• Complexity—Models generate high frequencies of valid and stable molecules when trained on the simple QM9 dataset but struggle when trained on the more complex GEOM-Drugs dataset.

• Applicability—More applicable tasks like protein-molecule binding are especially challenging, and current models still struggle with generating molecules with high binding affinity for targets.

• Explainability—All methods discussed are fairly black-box and abstract; existing models do not reveal aspects like “important” atoms or structures, and explainable AI in molecule generation is undeveloped as a whole.

Within protein generation, we encounter the following challenges:

• Benchmarking—While most models in molecule generation use standardized benchmarking procedures, generative tasks in protein design lack a standard evaluative procedure, with variance between each model’s metrics and testing conditions, making it hard to objectively evaluate the quality of designed proteins.

• Performance—As the tasks in protein generation are generally more complex than those in molecule generation, SOTA models still struggle in several key areas like fold classification, gene ontology, and antibody CDR H3 generation, leaving room for future improvement.

While our paper focuses on generative models and applications, it is important to note that many current tasks are evaluated with predictive models, such as the affinity optimization task in antibody generation or the conditional generation task for molecules. In these cases, classifier networks are used to predict binding affinity or molecular properties, and improvements to these classification methods would naturally lead to more precise alignment with real-world biological applications.

Conclusion

The survey has provided an overview of the current landscape of generative AI in de novo drug design, focusing on molecule and protein generation. It has discussed important advancements in these fields, detailing the key datasets, model architectures, and evaluation metrics used. The paper also highlights key challenges and future directions, including improvements to benchmarking methods, improving explainability, and further alignment with real-world tasks to increase applicability. Overall, generative AI has shown great promise in the field of drug design, and continued research in this field can lead to exciting advancements in the future.

Key Points

• Our survey examines the advancements and applications of Generative AI within de novo drug design, particularly focusing on the generation of novel small molecules and proteins.

• We explore the intricacies of generating biologically plausible and pharmaceutically potential compounds from scratch, providing a comprehensive yet approachable digest of formal task definitions, datasets, benchmarks, and model types in each field.

• The work captures the progression of AI model architectures in drug design, highlighting the emergence of EGNNs and diffusion models as key drivers in recent work.

• We highlight remaining challenges in applicability, performance, and scalability, delineating future research trajectories.

• Through our organized repository, we aim to facilitate further collaboration in the rapidly evolving intersection of computational biology and AI.

Funding

Xiangru Tang and Mark Gerstein are supported by Schmidt Futures.