Diffusion models in bioinformatics and computational biology

Abstract

Denoising diffusion models embody a type of generative artificial intelligence that can be applied in computer vision, natural language processing and bioinformatics. In this Review, we introduce the key concepts and theoretical foundations of three diffusion modelling frameworks (denoising diffusion probabilistic models, noise-conditioned scoring networks and score stochastic differential equations). We then explore their applications in bioinformatics and computational biology, including protein design and generation, drug and small-molecule design, protein–ligand interaction modelling, cryo-electron microscopy image data analysis and single-cell data analysis. Finally, we highlight open-source diffusion model tools and consider the future applications of diffusion models in bioinformatics.

Introduction

Deep learning¹ was introduced to the field of bioinformatics and computational biology in 2012 (ref. 2) (Box 1) and has been applied to many bioinformatics problems, such as protein structure prediction³, protein function prediction^4–9, protein–ligand interaction prediction^10–14, gene-expression prediction^15–20 and gene regulatory network modelling^21–25. Various deep learning architectures, including convolutional neural networks²⁶, long short-term memory networks²⁷, residual networks²⁸, generative adversarial networks (GAN)²⁹, graph neural networks (GNN)³⁰ (Box 2) and transformers³¹ have been developed for bioinformatics data analysis.

Box 1. Deep learning.

Deep learning is a machine learning technology that applies artificial neural networks with many layers of neurons (hence, ‘deep’) to model and extract complex patterns in data. Deep learning can then learn patterns and features from complex data to perform intelligent tasks, such as speech and image recognition, natural language processing and protein structure prediction. The artificial neurons in each layer receive input from the neurons in the previous layers until the final output layer produces a prediction (for example, classifying an image into a category or generating a sentence of text). During training of a deep learning model, the weights associated with the connections between neurons are adjusted to fit the training data. A major advantage of deep learning models over other machine learning methods is their ability to automatically learn hierarchical representations from raw data through multiple layers of abstraction. This enables deep learning models to achieve high prediction accuracy in many domains, such as precision medicine and healthcare (for example, medical image segmentation^{237,258–261} and disease diagnosis^262–265), finance (for example, algorithmic trading^266,267 and risk management²⁶⁸) and agriculture (for example, crop monitoring^269,270 and pest detection²⁷¹). Some notable applications of deep learning are ChatGPT²⁷² for natural language processing, DALL-E-2 (ref. 83) and GLIDE²⁷³ for image generation, and AlphaFold2 (ref. 163) for protein structure prediction.

Box 2. Key concepts relevant to diffusion models.

Diffusion: the movement of molecules, atoms, ions or energy from a region of higher concentration to a region of lower concentration along a concentration gradient until the concentration becomes equal in both regions. Diffusion, which is driven by a gradient in Gibbs free energy or chemical potential, is a stochastic process owing to the inherent randomness in the movement of the diffusing entities.

Generative model: a type of machine learning model that aims at learning the underlying distribution of data to generate new, similar data. These models can approximate the joint probability distribution of input features and labels, if available, and generate new data points by sampling from the learned distribution.

Markov chain: a stochastic model that describes a sequence of possible states, in which the probability of a state depends (or is conditioned) only on its previous state.

Markov chain Monte Carlo: a statistical or computational simulation method that constructs a Markov chain to iteratively generate a sequence of samples according to a conditional probability distribution between two consecutive states. After running the Markov chain for enough iterations, the generated samples converge to the desired posterior distribution.

Graph neural network (GNN)³⁰: a type of deep learning model for processing graph-structured data (for example, molecular graphs and biological networks). Each node in a GNN receives messages from its neighbouring nodes, which are used to update its hidden representation. By iteratively updating node representations, the GNN can aggregate information from both the local neighbourhood and remotely connected nodes in the graph.

Equivariant GNN¹⁶¹: a special type of GNN that is equivariant to a transformation (for example, translation and rotation) in the input data (for example, of a three-dimensional object, such as a protein structure). For example, the translation of an object in the input space leads to the translation of the same output of the object generated by the equivariant GNN in the output space without changing the value of the output.

SE(3)-equivariant networks¹⁶²: a special equivariant GNN model that preserves the symmetry of the special Euclidean group SE(3). If a SE(3) transformation is applied to the input, the output generated by the networks undergoes an equivalent transformation. Achieving SE(3) equivariance allows the model to capture the inherent symmetries and geometric properties of the input 3D data.

SE(3)-transformer¹⁹⁸: a specific implementation of SE(3)-equivariant networks using the transformer’s self-attention mechanism to achieve SE(3) symmetry, including three-dimensional rotations and translations. The SE(3)-transformer is particularly useful for tasks involving three-dimensional structures, such as protein structure prediction and protein design, where different (x, y, z) coordinates of the same protein structure appearing in different orientations and positions can be treated as the exact same object.

Diffusion models leverage deep learning technology^32–35; however, they outperform other deep learning methods in many domains, including in image generation^36–42, image inpainting^43,44 and speech synthesis⁴⁵. Diffusion models are deep learning-based generative models^32–35 (Box 2) that aim to generate artificial yet realistic data (for example, a computer-generated Picasso painting or an answer to a user’s question) from input parameters. Compared to other generative models, such as autoregressive models⁴⁶, normalizing flows⁴⁷, energy-based models⁴⁸, variational auto-encoders (VAEs)⁴⁹ or GANs²⁹, diffusion-based generative models have the ability to learn complex distributions, handle high-dimensional data and generate diverse data^50–55. In particular, diffusion models can surpass GANs²⁹, which consist of a generator that generates data and a discriminator that can differentiate the generated data, in the challenging task of image synthesis^33,50. In addition, diffusion models can be applied for computer vision^{43,51,56–71}, natural language processing^55,72–75, temporal data modelling^76–81, multi-modal modelling^36,37,82,83, and in medical image reconstruction^84–93.

Diffusion models were originally introduced³² to address a central problem in machine learning, that of modelling complex datasets using highly flexible families of probability distributions while ensuring that learning, sampling, inference and evaluation remain analytically or computationally tractable (Fig. 1). Inspired by non-equilibrium statistical physics, this approach systematically and slowly destroys the structure of data through an iterative forward diffusion process. Then, a reverse diffusion process is applied to restore the structure in the data, yielding a highly flexible and tractable generative model of the data, thereby enabling rapid learning, data sampling and evaluating probabilities through deep generative models with up to thousands of layers or time steps as well as the computing of conditional and posterior probabilities under the learned model. Based on this concept, denoising diffusion probabilistic models (DDPMs)³³ can achieve performance comparable to or better than other generative models (for example, decoder, energy-based models and GANs)^46,94–96 in image generation tasks. The diffusion network structure and training strategy can further be improved to boost performance⁵⁰, surpassing GANs in image synthesis. For example, a multi-head attention mechanism and the BigGAN’s residual module⁹⁵ can be applied for up-sampling and down-sampling of data to improve the resolution and quality of generated images. In addition, a denoising diffusion implicit model (DDIM)⁹⁷ can be used to increase sampling rate.

Fig. 1 |. Timeline of advances in diffusion models and their applications in bioinformatics.

Data are taken from refs. 32–36,54,61,72,84,91,97,99, 101–118,130,133,136,138–140,150,151,256,257,273. Cryo-EM, cryogenic electron microscopy; SDE, stochastic differential equations; DPM, diffusion probabilistic model; ODE, ordinary differential equations.

Importantly, diffusion models can be applied in bioinformatics, for example, for denoising cryo-electron microscopy (cryo-EM) data⁹⁸, single-cell gene-expression analysis^99,100, protein design and generation^{84,91,101–107}, drug and small-molecule design^54,108–113 and protein–ligand interaction modelling^114–118. Diffusion models have the advantage of being able to handle high-dimensional data with high diversity and scalability.

In this Review, we provide a detailed survey of diffusion models, including denoising diffusion models, noise-conditioned score networks (NCSNs) and stochastic differential equations (SDEs), and discuss their applications in bioinformatics. We further highlight possible future developments of diffusion models, aiming to propose some challenging bioinformatics problems that may be tackled by creative diffusion models.

The concept of diffusion models

Diffusion models learn to reverse the process of data destruction or corruption (for example, introduced by noise), allowing the generation of realistic, clean data samples (for example, restoration of uncorrupted data). Thus, diffusion models can learn from data that has been progressively destroyed or degraded to generate new samples from a given distribution or to estimate the distribution from which a given sample is drawn (Box 2).

Diffusion models are based mainly on three frameworks, each with a different formulation of the forward and reverse processes (Fig. 2), that is, DDPMs^32,33, NCSNs^34,119 and score SDEs^35,120.

Fig. 2 |. Forward and reverse processes of diffusion models.

Forward and reverse processes are shown for denoising diffusion probabilistic models (DDPMs), noise-conditioned score networks (NCSNs) and score stochastic differential equations (score SDEs). The forward process progressively (from time 0 to T) adds noise to data (for example, to an image of a cat or a three-dimensional protein structure). The reverse process generates cleaner data from noisier data (that is, it denoises data) from time T to 0. x₀ denotes uncorrupted data, t ∈{1,…,T} where T represents the number of diffusion steps; β_t ∈[0,1) is the hyperparameter denoting the variance schedule across the diffusion steps; I is the identity matrix; N(x;μ, σ) is the normal distribution of x with mean μ and standard deviation σ; α_t = 1 – β_t and ${\overline{\alpha }}_t=\prod _{s=0}^t{\alpha }_s;$w represents the standard Wiener process known as Brownian motion; f (x, t) and g (t) are the drift and diffusion coefficients of SDE, respectively; and p(x) is the probability density function.

Denoising diffusion probabilistic models

DDPMs, which were the first diffusion models able to generate high-resolution data, typically contain two Markov chains (Box 2): the forward chain gradually adds noise to scramble the original data, followed by a reverse chain that removes the noise from the data to recover the original data. If q(x₀) denotes the distribution of the original data, in which x₀ denotes uncorrupted data, the transition kernel q(x_t |x_t−1) of the forward Markov process adding Gaussian perturbation at time t is denoted$𝒩\left(x_t;\sqrt{1-{\beta }_t}{x}_{t-1},{\beta }_t\mathbf{I}\right)$, in which tϵ{1,…,T}. Here T represents the number of diffusion steps; ϵ_t∈[0,1) is the hyperparameter denoting the variance schedule across diffusion steps; I is the identity matrix; and N(x;μ,σ) is the normal distribution of x with mean μ and covariance σ. If α_t =1 − β_t and ${\overline{\alpha }}_t=\prod _{s=0}^t{\alpha }_s$, a noisy sample x_t can be obtained directly from the distribution conditioned on the original input x₀:

$$q\left(x_t|x_0\right)=𝒩\left(x_t;\sqrt{{\overline{\alpha }}_t}{x}_0,\left(1-{\overline{\alpha }}_t\right)\mathbf{I}\right)$$ (1)

$$x_t=\sqrt{{\overline{\alpha }}_t}{x}_0+\sqrt{1-\overline{{\alpha }_t}}ϵ,ϵ~𝒩(0,\mathbf{I})$$ (2)

The forward process gradually introduces noise into the original data until it is completely replaced by noise. The reverse process is the opposite operation, resulting in the generation of new samples. This process typically starts with unstructured noise obeying the prior distribution, and then, by applying a model — typically a trainable neural network — that has learning ability, noise is removed step by step to restore the original data. The neural network N can be formulated as:

$$p_{\theta }\left(x_{t-1}|x_t\right)=𝒩\left(x_{t-1};{\mu }_{\theta }\left(x_t,t\right),{\sigma }_{\theta }\left(x_t,t\right)\right)$$ (3)

Given the starting point data of the reverse process as p(X_T) =𝒩(X_T ;0, I), the distribution of X₀ conditioned on X_T is given by:

$$p_{\theta }\left(X_{0:T}\right)=p\left(X_T\right)\prod _{t=1}^T{p}_{\theta }\left(X_{t-1}|X_t\right)$$ (4)

Eventually, a marginal distribution of X₀ close to the original data x₀ can be obtained by $p_{\theta }\left(x_0\right)=\int p_{\theta }\left(x_{0:T}\right)\text{d}{x}_{1:T}$.

To train the model parameterized with θ so that it can learn the pattern of the original data and make p(x₀) close to the true data distribution q(x₀), the loss function to be minimized is set as the negative log-likelihood (equation (5)). We note that the process of minimizing the negative log-likelihood of the observed data under the model is equivalent to minimizing the Kullback–Leibler (KL) divergence between the empirical distribution defined by the original data q(x₀, x₁, …, x_T) and the model distribution p_θ(x₀, x₁, …, x_T):

$$\begin{gathered} \mathbb{E}\left[-\text{log}{p}_{\theta }\left(x_0\right)\right]\le \text{KL}\left(q\left(x_0,x_1,\cdots ,x_T\right)\parallel p_{\theta }\left(x_0,x_1,\cdots ,x_T\right)\right) \\ ={\mathbb{E}}_q\left[-\text{log}\frac{p_{\theta }\left(x_{0:T}\right)}{q\left(x_{1:T}|x_T\right)}\right] \\ ={\mathbb{E}}_q\left[-\text{log}p\left(x_T\right)-\sum _{t\ge 1}\text{log}\frac{p_{\theta }\left(x_{t-1}|x_t\right)}{q\left(x_t|x_{t-1}\right)}\right] \\ =-L_{\text{VLB}} \end{gathered}$$ (5)

The objective of DDPM training is to minimize L_VLB, also known as the variational lower bound of the log-likelihood. L_VLB can also be parameterized to increase the quality of sample generation³³.

Noise-conditioned score networks

In NCSNs, the score function of a probability density function p(x) is represented by the gradient of the log density with respect to the input as ∇_xlogp(x). To learn and estimate the score function, a score-matching neural network s_θ is trained. The goal of this neural network is to make s_θ(x) ≈ ∇_xlogp(x) . Therefore, the objective function of the scoring network can be defined as:

$${\mathbb{E}}_{x~p(x)}\parallel s_{\theta }(x)-{\nabla }_x\text{log}p{(x)\parallel }_2^2$$ (6)

Even though the problem is well defined, optimizing equation (6) is numerically impossible because the value of ∇_xlogp(x) cannot be known. However, score functions can be learned from data by applying score matching¹²¹, denoising score matching^122–124 or sliced score matching¹²⁵.

Moreover, training remains difficult because the trained score functions are unreliable in low-dimensional manifold, because low-dimensional data is typically embedded in a high-dimensional space (the manifold hypothesis)³⁴. This challenge can be addressed by introducing Gaussian noise to the data at various scales, which improves the data distribution’s suitability for score-based generative modelling. Thus, a single NCSN can be applied to estimate the score corresponding to each noise level. If 0< σ₁<σ₂<…< σ_t <…< σ_T is a sequence of Gaussian noise levels, $p_{{\sigma }_t}\left(x_t|x\right)=𝒩\left(x_t;x,{\sigma }_t^2\mathbf{I}\right)$, $p_{{\sigma }_1}(x)\approx p\left(x_0\right)$, and $p_{{\sigma }_T}(x)\approx 𝒩(0,\mathbf{I})$. The NCSN s_θ(x, σ_t) with the denoising score matching can then approximate the gradient log density function, making$s_{\theta }\left(x,{\sigma }_t\right)\approx {\nabla }_x\text{log}\left(p_{{\sigma }_t}(x)\right),\forall t\in \left\{1,\dots ,T\right\}$. And for $x_t,{\nabla }_x\text{log}\left(p_{{\sigma }_t}(x)\right)$ is derived as:

$${\nabla }_{x_t}\text{log}{p}_{{\sigma }_t}\left(x_t|x\right)=-\frac{x_t-x}{{\sigma }_t}$$ (7)

Consequently, the optimization objective function in equation (6) can be transformed into:

$$\frac{1}{T}\sum _{t=1}^T\lambda \left({\sigma }_t\right){\mathbb{E}}_{p(x)}{\mathbb{E}}_{x_t~p_t\left(x_t|x\right)}\parallel s_{\theta }\left(x_t,{\sigma }_t\right)+\frac{x_t-x}{{\sigma }_t}{\parallel }_2^2$$ (8)

in which λ(σ_t) is a weighting function.

During the sampling phase, NCSNs use the annealed Langevin dynamics algorithm, which employs a Markov Chain Monte Carlo procedure (Box 2) to sample from a distribution according to its score function ∇_xlogp(x). The Langevin method recursively computes x_i as follows:

$$x_i=x_{i-1}+\frac{\gamma }{2}{\nabla }_x\text{log}p(x)+\sqrt{\gamma }{\omega }_i$$ (9)

where γ determines the amplitude of the update in the score’s direction; x₀ is sampled from the prior distribution; and the noise is drawn according to ω_i ~N(0,I).

NCSNs and DDPMs both operate on the principle of converting a basic noise distribution into a more intricate data distribution by collecting information during the introduction of noise, which is then reapplied when removing the noise. Both models are trained to tackle a noise regression problem, based on the principle of maximum likelihood estimation. Notably, the objective formulation of score matching with Langevin dynamics in NCSN aligns with that of the re-weighted variant of the evidence lower bound of DDPM^35,126,127. In terms of sample generation, both models employ ancestral sampling, which progressively transforms a noise sample into a data sample, guided by data distribution gradients.

Score stochastic differential equations

With unlimited time steps or noise levels, DDPMs and NCSNs can be further generalized to a situation in which the perturbation and denoising processes can be described as SDEs. This generalized approach³⁵ of gradually transforming data into noise is called score SDE. The forward process of score SDE uses SDEs and requires an estimated score function of the noisy data distribution. It is equivalent to the Itô SDE¹²⁸ solution, which consists of a drift component for mean transformation and a diffusion coefficient for describing noise:

$$\text{d}x=f(x,t)\text{d}t+g(t)\text{d}w,t\in [0,T]$$ (10)

where w represents the standard Wiener process known as Brownian motion, and f (x, t) and g (t) are the drift and diffusion coefficients of SDE, respectively. The forward process in DDPMs and score-based generative models is a special case of the discretizational SDE.

The formulation of the reverse diffusion process of SDE is given by equation (11)¹²⁹, also called reverse-time SDE:

$$\text{d}x=\left[f(x,t)-g^2(t){\nabla }_x\text{log}{p}_t(x)\right]\text{d}t+g(t)\text{d}\overline{w}$$ (11)

where $\overline{w}$ is the standard Brownian motion running backward time, and dt represents the infinitesimal negative time step. The reverse SDE and forward SDE share the same marginal densities but in the opposite time direction³⁵. As in DDPMs and NCSNs, to numerically solve reverse-time SDE, a trainable neural network s_θ(x, t) is employed to estimate the actual score function ∇_xlogp_t(x). The objective function can be defined as:

$${\mathbb{E}}_{x(t)~p(x(t)|x(0)),x(0)~p_{\text{data }}}\left[\frac{\lambda (t)}{2}\parallel s_{\theta }(x(t),t)-{\nabla }_{x(t)}\text{log}{p}_t(x(t)|x(0)){\parallel }_2^2\right]$$ (12)

where t ~ u([0, T]) denotes the uniform distribution over [0, T] and λ is a weighting function. In addition, several sampling techniques, such as the predictor–corrector sampler, can be employed to generate good samples. This procedure uses a score-based method (that is, annealed Langevin dynamics) as a corrector after using a numerical approach to sample data from the reverse-time SDE.

Improving diffusion models

The aforementioned diffusion models can be further improved through extension in training speed^{126,130–133}, increasing data sampling (data generation) speed^97,134–139, integration with other neural networks^{38,120,140–142}, and applications to different data types^{53,73,143–151}. Many of these improvement strategies are available as open-source tools¹⁵² (Table 1), which has opened up their application to a diverse range of bioinformatics problems (Box 3). Importantly, diffusion models can handle different data types, such as one-dimensional (1D) DNA and protein sequences, two-dimensional (2D) biomedical images, three-dimensional (3D) protein structures and vectorized gene-expression data.

Table 1 |.

Open-source tools for implementing and improving diffusion models

Tool Name	Year	Model type or improvement	Diffusion frameworka	Network architecture	GitHub
NCSN34	2019	Foundational model	NCSN	Variant of U-Net	CODE
DDPM33	2020	Foundational model	DDPM	U-Net237	CODE
Score SDE35	2020	Foundational model	Score SDE	Variant of U-Net	CODE
EDP-GNN151	2020	Diffusion on graph data	NCSN	Variant of GNN	CODE
Improved diffusion130	2021	Speed up training	DDPM	U-Net	CODE
Cold diffusion131	2021	Speed up training	Score SDE	U-Net	CODE
DDIM97	2021	Speed up sampling	DDPM	U-Net	CODE
Diffusion-point-cloud143	2021	Point cloud	DDPM	Autoencoder	CODE
LSGM140	2021	Mixed modelling	Score SDE	Autoencoder	CODE
ConfGF144	2021	Diffusion on graph data	Score SDE	GNN	CODE
PVD145	2021	Point cloud	DDPM	Autoencoder	CODE
sdeflow-light252	2021	SDE unification	Score SDE	VAE	CODE
Gotta Go Fast134	2021	Speed up sampling	Score SDE	Variant of U-Net	CODE
Score-flow120	2021	Mixed modelling	Score SDE	U-Net	CODE
DiffFlow141	2021	Mixed modelling	Score SDE	U-Net	CODE
FastDPM132	2021	Speed up training	Score SDE	U-Net	CODE
argmax_flows73	2021	Categorical data	DDPM	VAE, DenseNet	CODE
Soft Truncation253	2021	Likelihood optimization	Score SDE	Variant of U-Net	CODE
ARDM146	2022	Categorical data	DDPM	Variant autoregressive models	CODE
k-Diffusion135	2022	Speed up sampling	Score SDE	Variant of U-Net	CODE
DPM-Solver136	2022	Speed up sampling	Score SDE	Plugin method	CODE
VQ-diffusion254	2022	Vector quantized	DDPM	Vector quantized VAE	CODE
Improved VQ-Diff255	2022	Vector quantized	DDPM	Vector quantized VAE	CODE
Diffusion GAN38	2022	Mixed modelling	DDPM	GANs	CODE
DiffuseVAE142	2022	Mixed modelling	DDPM	VAE	CODE
PNDM137	2022	Speed up sampling	Score SDE	Variant of U-Net	CODE
GeoDiff147	2022	Diffusion on graph data	DDPM	Variant of VAE	CODE
VDM126	2022	Speed up training	DDPM	Variant of VAE	CODE
Analytic-DPM138	2022	Speed up sampling	NCSN	U-Net	CODE
Point Diffusion-Refinement53	2022	Point cloud	DDPM	Variant of U-Net	CODE
GDSS148	2022	Diffusion on graph data	Score SDE	Multiple stage architecture	CODE
Riemannian-score-sde149	2022	Diffusion on manifold data	Score SDE	Multilayer perceptron	CODE
Diffusion Distillation133	2022	Speed up training	DDPM	Variant of U-Net	CODE
bit-diffusion150	2022	Diffusion on discrete data	DDPM	Variant of U-Net	CODE
DreamFusion256	2022	Generate 3D data	DDPM	Multi-stage architecture	CODE
diffusers257	2022	Improve sample quality	DDPM	Multiple network architectures	CODE
DPM-Solver++139	2022	Speed up sampling	DDPM	Multiple network architectures	CODE

^aThe diffusion frameworks are general models that are typically combined with specific deep learning architectures (network architectures) to generate or denoise a specific type of data. Tools intended to increase sampling speed and the quality of diffusion are tested with various network architectures, and their associated network architectures are categorized as ‘multiple network architectures’. NCSN, noise-conditioned score network; U-Net, U-shaped neural network; DDPM, denoising diffusion probabilistic model; score SDE, score stochastic differential equation; GNN, graph neural network; VAE, variational autoencoder; GAN, generative adversarial network.

Box 3. A practical guide for applying diffusion models in bioinformatics.

Diffusion models are particularly useful in the generation, design or analysis of small molecules, proteins and biological images. To decide which diffusion model to apply to a specific problem, the representation of the specific data type (for example, small molecules) needs to be considered to be suitable for processing by a deep learning model in the diffusion process. The conformations of small molecules and drugs can be represented in several ways to facilitate the diffusion process; for example, they can be treated as a string, such as the SELF-referencing embedded string, which can be converted into a two-dimensional (2D) matrix. This matrix can be used as input for graph neural networks (GNNs) under a diffusion model framework to generate three-dimensional (3D) molecular graphs, as exemplified by dynamic graph score matching¹¹⁰. Alternatively, they can be presented as 3D graphs that contain spatial direction and torsion angles between atoms, which can be used by a combination of SE(3)-equivariant GNNs¹⁶² and diffusion models, such as the E(3)-equivariant diffusion model⁵⁴ to capture their essential properties. In addition, small molecules can be represented as 3D atomic point clouds to be processed by equivariant GNNs, as in DiffLinker¹¹³. Proteins can be represented as either one-dimensional (1D) sequential features suitable for a 1D transformer or 2D contact and distance maps suitable for processing by convolutional neural networks. The 3D structure of proteins is usually represented as graphs that consist of nodes denoting residues and edges that represent residue pairs in contact, which can be handled by both standard GNNs and SE(3)-equivariant GNNs in combination with diffusion models. For imaging data, such as cryo-electron microscopy images, various diffusion models initially developed for image generation, such as CascadedDiff⁶⁰, can be applied. Biomolecules or cell shapes may also be represented by 3D images, which can be reconstructed from 2D images by a combination of autoencoder or U-Net²³⁷ with a diffusion model, as in CryoDRGN⁹⁸ and DISPR⁹⁹. These can model the distribution of ground-truth data to generate higher-quality 3D images than other generative artificial intelligence methods. For example, DISPR outperforms a VAE-based deep generative model SHAPR²³⁸ in the context of 3D cell shape reconstruction.

Protein design and generation

The computational generation of new, physically foldable protein structures allows the design of proteins with specific functions or structural properties for protein engineering and drug discovery. However, deep generative models (Box 2), such as VAEs and GANs^153–159, are limited to generating only small proteins or domains of large proteins (for example, of immunoglobulins). Alternatively, diffusion models can be applied to protein design and generation, because large and diverse proteins can be generated by guiding the model at each step of the iterative generation process.

Protein structures in protein generation^153,154 are typically described by a 2D matrix (map) that contains the pairwise distances and angles between all the residues in the protein. For example, ProteinSGM, based on a score-based generative model⁹¹, applies a diffusion model of 2D image generation using such a representation to create protein structures: a score-based generation diffusion model with SDEs is used to generate a series of 2D matrices that include inter-residue pairwise distances d, and the ω, θ and φ angles between two residues. These constraints are then fed into Rosetta¹⁶⁰ to build native-like protein structures. For unconditional protein structure generation, ProteinSGM can generate proteins from random noise. For conditional protein structure generation, such as scaffold inpainting and functional site inpainting, the tool can generate protein structures that satisfy user-defined constraints, similar to solving an image inpainting problem. However, ProteinSGM requires post-processing by Rosetta using Markov Chain Monte Carlo (Box 2), which makes the prediction computationally expensive.

Unlike ProteinSGM, Foldingdiff¹⁰¹ represents the protein backbone structures (only N–Ca–C atoms for each residue) with a series of consecutive angles to capture the relative orientation of the constituent atom acid residues. A simple language transformer model³¹ with DDPM can then be applied to generate protein structures unconditionally, as the angles are invariant to translation and rotation. However, using a transformer to predict sequence-like consecutive angles has the drawback that errors from the early prediction accumulate and considerably affect the final structure, including collisions between atoms. In addition, the approach cannot be generalized to generate complex structures with more than one chain.

Inspired by Foldingdiff, DiffSDS¹⁰² introduces a 1D directional representation derived from invariant atom features, similar to torsion angle representation, which enables an encoder–decoder language model to perform the diffusion process. In the language model, the encoder (with a hidden atom-direction-space layer) transforms the invariant features into equivalent direction vectors, whereas the decoder reverses the transformation. By performing the diffusion process in this direction and by conditioning angle spaces on geometric restraints, DiffSDS can restore protein backbone structures of higher quality than the deep-learning-based protein design method RFDesign¹⁵⁶: DiffSDS is two times better at generating proteins that resemble natural proteins (protein likeness), as measured by Rosetta energies, about 18 times better in terms of connectivity errors and 60% better at generating non-overlapping scores with existing backbones than RFDesign.

The integration of diffusion models with GNNs³⁰ (Box 2) enables the direct generation of 3D protein coordinates, resulting in an end-to-end generative model. SE(3)-equivariant^161,162 (Box 2) DDPMs, which are usually used in small-molecule generation, can also be applied to generate protein structures in a representation-frame-independent manner¹⁶³. For example, independent DDPM models equipped with invariant point attention¹⁶³ structural modules can be trained with the distribution of atom features (for example, coordinates in a canonical frame with respect to backbone atoms, residue type and side-chain angles) to generate a protein’s backbone, sequence and side-chain rotamers⁸⁴. By jointly diffusing the structure and sequence, while incorporating coarse structural constraints, the model can gradually generate the fully atomistic protein structure and sequence, allowing controllable protein backbone generation and protein structure inpainting. The sequence recovery rate of this method is comparable to that of other machine-learning-based and physics-based methods, such as 3DConv¹⁶⁴, RosettaFixBB and RosettaReIBB¹⁶⁵. Similarly, Genie¹⁰³ makes use of the SE(3)-equivariant feature from the invariant point attention module in conjunction with DDPM to generate protein backbones unconditionally, also introducing geometric asymmetry with an invariant encoder to directly inject noise into residue coordinates, as well as an SE(3) equivariant decoder with an invariant point attention module to predict noise.

SMCDiff¹⁰⁴ applies a similar deep learning architecture (that is, an SE(3)-equivariant GNN) (Box 2) to the motif-scaffolding generation problem, dividing the problem into two parts: unconditional protein backbone generation (ProtDiff) and conditional sampling in diffusion models based on a protein motif (SMCDiff), similar to inpainting. Unconditional protein generation is achieved by training a SE(3)-equivariant GNN (Box 2), built from residue coordinates and embedded features from the protein sequence, to generate protein backbones. By contrast, conditional sampling is formulated on an unconditional diffusion model as a sequential Monte Carlo simulation problem, which may be solved by particle filtering. However, the network does not include torsion angles as features and may therefore generate unnatural proteins (for example, left-handed helices). SMCDiff was the first deep generative model that lever-aged the power of diffusion models to address the motif-scaffolding generation problem.

RFdiffusion¹⁰⁵, which integrates a conditional DDPM diffusion model with the pre-trained protein 3D structure prediction model RoseTTAFold¹⁶⁶, can directly generate final 3D coordinates. Inspired by the recycling process in AlphaFold2, a self-conditioning prediction strategy is applied, in which the current prediction is conditioned on the prediction from the previous timesteps, thereby considerably improving the performance of the model. Starting from random noise, RFdiffusion can generate large protein structures unconditionally, which can then be used in the design of protein monomers. Using protein motif coordinates as input, RFdiffusion can also construct scaffolds conditionally for functional motif and enzyme active site scaffolding¹⁰⁵. Given a point group symmetry, RFdiffusion can maintain the symmetry during the prediction owing to the equivariance design of RoseTTAFold. Therefore, this approach can be applied to symmetric protein oligomer and motif scaffolding (for example, for the design of therapeutic¹⁶⁷ and metal-binding proteins^168,169). We note that compared to the other methods discussed in this section, some proteins designed by RFdiffusion have not only been validated in silico, but also by biochemical and biophysical experiments^170,171, making it one of the first generative artificial intelligence methods of protein design that have been experimentally validated. Furthermore, RFdiffusion outperforms other methods, such as RFDesign, in the design of large protein structures and high-order protein oligomers, demonstrating the advantage of diffusion models.

FrameDiff¹⁰⁶ applies diffusion models to explore whether a pre-trained protein structure predictor is necessary for protein backbone generation. Here, using denoising score matching, a principled SE(3) diffusion model can better formulate the protein backbone generation problem, achieving comparable performance with four-fold fewer network weights and without the need to train another protein structure prediction network, compared to RFdiffusion.

Chroma¹⁰⁷ is a GNN³⁰-based conditional diffusion model designed to generate large single-chain proteins and protein complexes with desired properties and functions. This model can generate protein structures that are over 3,000 residues in size, which surpasses the size limit for proteins generated by several other networks (that is, ProteinSGM, Foldingdiff, DiffSDS and SMCDiff) (<2,000 residues). To reduce computational complexity, Chroma uses a random graph generation procedure that preserves both short- and long-range interactions. As a result, Chroma can produce high-quality, diverse new protein structures, and enables the programmable generation of proteins that are conditioned on several different properties, such as residue–residue distances, symmetry and shape.

Small-molecule generation and drug design

Drug discovery involves the identification and optimization of small molecules that can interact with specific biological targets, such as enzymes or receptors, to modulate their activity and ultimately achieve a therapeutic effect. Deep learning, particularly deep generative models, enables the rapid generation and evaluation of a large number of such potential drug candidates^172–175.

The conditional diffusion model, which is a deep learning method based on discrete graph structures (CDGS), allows the generation of molecular graphs of small molecules with similar data distributions to real-number molecular graphs¹⁰⁸. This method employs a hybrid message-passing block architecture, which comprises a standard message-passing layer for collecting local features, such as node-edge dependencies, and an attention-based message-passing layer for extracting and transmitting global information in the architecture. The molecular graphs are embedded with distinct components for node features and edge matrices, with channels for edge existence and edge types. The CDGS model has enabled the application of diffusion models in the molecular graph domain, which is crucial for drug discovery and material science. This approach accurately models the complex dependency between graph structures and features during the generative process, using SDEs to describe the graph diffusion process. The continuous forward process is applied directly to edge existence variables, and the reverse process first decodes discrete graph structures, which serve as the condition for each sampling step. A specialized hybrid graph noise prediction model is used to extract global and local node-edge dependencies from intermediate graph states. This diffusion-based model can obtain high-fidelity samples in 200 steps of network evaluations using the Euler–Maruyama method¹⁷⁶. In addition, a fast ordinary differential equation solver, which applies the semi-linear structure of probability flow ordinary differential equations for graphs, promotes rapid, high-quality graph sampling. CDGS outperforms other methods in molecular graph generation, including flow-based methods (for example, GraphAF¹⁷⁷, GraphDF¹⁷⁸, MoFlow¹⁷⁹ and GraphCNF¹⁸⁰) and other diffusion models (EDP-GNN¹⁵¹, GraphEBM¹⁸¹ and GDSS¹⁴⁸). CDGS also performs better in generic graph generation than ER¹⁸², VGAE¹⁸³, GpraphRNN¹⁸⁴ and GRAN¹⁸⁵, demonstrating its potential to facilitate drug discovery and material design by representing molecular structures and restricting the molecule search space.

The E(3)-equivariant diffusion model (EDM)⁵⁴ (Box 2) performs the diffusion process on atom coordinates and atom types in the Euclidean space to generate small molecule structures with up to 29 atoms, compared to nine heavy atoms that can be achieved with equivariant normalizing flows¹⁸⁶. An EDM represents each small molecule as a point cloud that can be described by a graph with nodes v_i ∈ V representing atoms in the molecule based on an equivalent transformation, thereby combining the equivariant GNN and the diffusion process. The former contains L layers of equivariant graph convolutional layers that take each atom’s 3D coordinates and features as input to model molecule structures with geometric symmetries, whereas the latter gradually adds Gaussian noise to both the coordinates and features of the atom, thereby improving training, performance and scalability, compared to other E(3)-equivariant models, such as G-Schnet¹⁸⁷ and equivariant normalizing flows¹⁸⁶ as well as graph-based molecule-generative models, such as GraphVAE¹⁸⁸, GraphTransformer¹⁸⁹ and Set2GraphVAE¹⁹⁰.

Based on the equivariant GNN architecture and inspired by the physics governing the formation of small molecules, the Lyapunov function applies physical and statistics prior information (diffusion informative prior bridge)¹⁰⁹ to guide the diffusion process in model training and generate high-quality and realistic molecules. In this approach, problem-dependent prior information, in particular, physical and statistics information, is injected into the diffusion process instead of imposing or improving deep learning architectures. Several energy functions, integrated with the physical and statistical prior information, are then used as a prior bridge to guide the model training without any extra modification of the equivariant GNN architecture. Thereby, the Lyapunov function shows better molecule-generation performance in terms of physical energy and molecule stability¹⁰⁹ and better uniformity-promoted 3D point cloud generation compared to EDM⁵⁴ and point cloud diffusion¹⁴³, which apply the traditional Gaussian noise in model training, as well as equivariant normalizing flows¹⁸⁶.

Dynamic graph score matching (DGSM)¹¹⁰ is a deep learning model developed for predicting stable 3D conformations from 2D molecular graphs, primarily used in computational chemistry. The model can also be extended to protein sidechain conformation prediction and complex multi-molecular prediction (for example, predicting the interaction of more than three small molecules without explicit bonds)¹¹⁰. Deep learning methods often consider only the local interactions between bonded atoms, while neglecting the long-range interactions among unbound atoms, which are crucial for constructing accurate 3D molecular structures. To overcome this limitation, DGSM treats each molecule as a graph g = <v, e>, where a node in v represents an atom and its features (for example, coordinates), and an edge in e represents a bond between two atoms. The distance D_ij between each pair of atoms, that is, the edge length in the graph, can then be computed from their coordinates. For each pair of unbound atoms, the distance D_ij can be perturbed by a Gaussian noise level at each training step. A message passing neural network¹⁹¹ is then applied, using edge length and edge type in the graph as inputs to dynamically embed the molecular 2D graph by adding Gaussian noise to the distance between pairs of unbound atoms. Using the score-matching method, the model can then directly estimate the gradient fields of the logarithm density of atomic coordinates. Importantly, the model can be trained in an end-to-end fashion, thereby addressing the limitation of physics-based simulation methods that do not account for long-range interactions between non-bounded atoms. Thus, DGSM outperforms other methods, including RDKit¹⁹², CGCF¹⁹³ and ConfGF¹⁴⁴ in terms of matching score and coverage score, confirming the benefit of modelling long-range interactions.

SDEGen¹¹¹ is a multi-stage diffusion model that can generate molecules by adopting multiple architectures in different stages with different purposes; here, molecular conformations, including distances between two atoms within three-hop edges, edge type and atom type, and their corresponding graphs, are used as inputs for three different multilayer perceptrons to generate their embeddings. The distance embeddings are corrupted by Gaussian noise and the atom-type embeddings are then updated by a GNN (Box 2). The noisy distance embeddings, edge-type embeddings and the updated atom-type embeddings are then combined into final bond embeddings. Finally, the SDE network is parameterized. This multi-stage model is not as streamlined as end-to-end models, but it outperforms several other models, including DGSM¹⁹⁴, CGCF¹⁹³, ConfGF¹⁴⁴, CVGAE¹⁹⁵ and DMCG¹⁹⁶, by multiple metrics, such as coverage score and matching score, in particular, when considering long-range interactions in molecules.

DiffMD¹¹² is a score-based denoise diffusion model that can be applied to improve molecular dynamics simulations. Deep-learning-based molecular dynamics models typically depend on intermediate force fields and can thus only be applied to static molecules, not considering thermodynamics. DiffMD addresses this problem by applying score-based conditional diffusion models, employing the equivariant geometric transformer to take atomic coordinates, velocity and features embedded in molecular dynamics trajectories directly as input. In each layer, the model introduces velocities, directions and other geometric information using the spherical Fourier–Bessel transformation to update the input information. During the diffusion process, the conditional noise, based on the accelerations of atoms in previous frames, is added to the inputs for the equivariant geometric transformer to estimate the score function, that is, the gradient of the log density of the biomolecule conformations. DiffMD outperforms several deep-learning-based molecular dynamics methods, including tensor field networks¹⁶², radial fields¹⁹⁷, SE(3)-transformers¹⁹⁸, graph mechanics networks¹⁹⁹ and SCFNN²⁰⁰ in terms of average root-mean-squared error.

Fragment-based drug design can also be used for the discovery of new small molecules in a 3D space. Here, the aim is to design linkers consisting of atoms that can connect molecular fragments into a complete molecule. DiffLinker¹¹³ uses an E(3)-equivariant 3D conditional diffusion model to generate these molecular linkers and to connect multiple molecular fragments to form a single connected molecule. The prediction is made by applying a GNN to predict the linker size (the atom number of the linker) and atom types. The coordinates of the atoms are sampled from the normal distribution, followed by a reverse diffusion process of the atom features conditioned on the input fragments. Compared to DeLinker²⁰¹ and 3DLinker²⁰², DiffLinker can perform better in terms of average quantitative estimation of drug-likeness, synthetic accessibility, the average number of rings in the linker, and the validity, uniqueness and novelty of the samples, thereby generating more realistic molecules.

Protein–ligand interaction modelling

Predicting the conformation of a ligand bound to a protein is important in the investigation of protein–ligand interactions and protein function as well as for the discovery of new drugs. Various protein–ligand docking, machine learning and auto-regressive models have been developed to address this problem^10,203–206; however, these approaches are limited by their low geometrical accuracy. Alternatively, DiffBP¹¹⁴ can generate ligands that bind to a specific protein pocket without requiring the ligand structure as input; here, a pre-generation network is used to generate the centre of mass and atom number of the ligand, followed by diffusion models in conjunction with equivariant GNNs^161,207 to generate high-quality ligand candidates^33,35. Compared to auto-regressive methods, such as 3DSBDD²⁰⁵, Pocket2Mol²⁰⁶ and GraphBP²⁰³, which generate one atom at a time without considering interactions among all atoms, DiffBP can generate all atoms of a ligand that bind to a target protein, exhibiting high binding affinities (for example, 41.07%¹¹⁴ for DiffBP, compared to 12.22%¹¹⁴ for 3DSBDD, 23.98%¹¹⁴ for Pocket2Mol and 29.54%¹¹⁴ for GraphBP) on the CrossDocked²⁰⁸ dataset curated from protein–ligand complex structures in the Protein Data Bank (PDB).

DiffSBDD¹¹⁵ adopts a DDPM equipped with an E(3)-equivariant neural network to generate new ligands, including atomic features binding to specific protein pockets; here, ligand generation can either be protein-conditioned, based on the binding site to the protein, or the ligand can be impainted after learning the joint distribution of the protein–ligand complexes. Compared to 3DSBDD and Pocket2Mol, DiffSBDD can generate more diverse ligands with higher affinity on the CrossDocked dataset¹¹⁵.

Unlike diffusion models applied for protein pocket docking, DiffDock¹¹⁶ uses the structure of the protein and ligand as input and does not require knowledge of the location of the binding site (that is, blind docking); here, the diffusion process is applied to ligand positions, represented by ligand translation and rotation, sampling multiple positions, which are then ranked based on a confidence score using a trained scoring model and a trained confidence model, which are built on top of SE(3)-equivariant GNNs (Box 2). The scoring model samples different positions of the ligand, and the confidence model selects the ligand positions with the highest confidence score, similar to the structural and scoring modules of AlphaFold2¹⁶³ for protein structure prediction. DiffDock has been tested on the PDBBind dataset, outperforming search-based methods, such as SMINA²⁰⁹, QuickVina-W²¹⁰, GLIDE²¹¹ and GNINA²¹², and the deep learning methods EquiBind²¹³ and TANKBind²¹⁴. Specifically, DiffDock achieved a top-1 success rate of 38.2% (the percentage of top-1 predictions with root-mean-square deviation <2 Å), which is significantly better than the energetics-based method GLIDE (21.8%; P = 2.7 × 10⁻⁷) and the geometric deep-learning-based method TANKBind (20.4%; P = 1.0 × 10⁻¹²)¹¹⁶.

Similar to DiffDock, NeuralPLexer¹¹⁷ is a deep generative network that leverages SDEs to predict complex protein–ligand structures based on the protein structure and molecular graphs of the ligand as input in blind docking. The key component in the model is an equivariant structure diffusion module, which predicts the atomic coordinates on a heterogeneous graph formed by protein atoms, ligand atoms, protein backbone frames and ligand local frames. Using SDEs, the model can handle unbound or predicted protein structure inputs and can automatically accommodate changes in the protein structure in response to ligand binding. Compared with the deep learning method EquiBind²¹³ and the physics-based method CB-Dock²¹⁵ on the PDBBind²¹⁶ dataset, NeuralPLexer can generate a more accurate ligand structure with higher geometrical accuracy, with an approximately 70% success rate for a ligand with root-mean-square deviation <2 Å, which is higher than that of EquiBind (about 40%) and CB-Dock (about 38%) and has a lower steric clash rate of 0.105.

Finally, a deep generative energy-based diffusion model can predict the binding affinity for a protein–ligand pair, if trained with a set of protein–ligand complexes, without requiring labels for binding affinities¹¹⁸. During training, the network first predicts the rotation score for the perturbed ligand with respect to the protein pocket using an equivariant rotation prediction network, called Neural Euler’s Rotation Equation (NERE). By training the model with the SE(3) denoising score matching, the log-likelihood is considered to be the binding affinity between the protein and ligand in a pair. Tested on the protein–ligand dataset PDBbind²¹⁶ and the structural antibody database SAbDab²¹⁷, the model achieves an accuracy of 0.656 in predicting protein–ligand binding affinity, which is better than that of other unsupervised methods: 0.647 for Molecular Mechanics Generalized Born Surface Area²¹⁸ (MM/GBSA), 0.617 for Astex Statistical Potential²¹⁹ (ASP) and 0.602 for DrugScore2018 (ref. 220). This model further performs comparably to other supervised methods in predicting antibody–antigen binding¹¹⁸: Zlab RerANK²²¹, ZRANK2²²², RosettaDock²²³, PyDock²²⁴, Scoring by Intermolecular Pairwise Propensities of Exposed Residues (SIPPER)²²⁵, Atomic Potential Protein Interactions Scored Atomically (AP_PISA)²²⁶, Coarse Grained Protein Interaction Energy (CP_PIE)²²⁷ and FIREDOCK²²⁸.

Cryo-electron microscopy data analysis

Single-particle cryo-electron microscopy (cryo-EM)^229–235 is a key imaging technique for determining and visualizing the 3D conformation (structure) of large biomolecular complexes (for example, protein complexes) at atomic resolution; here, the images of protein complexes obtained by cryo-EM are used to reconstruct their 3D conformation represented by 3D density maps.

The protein structure reconstruction method CryoDRGN²³⁶ introduces a latent variable Z to define a conformational space V for a protein complex on cryo-EM density maps. CryoDRGN is based on a VAE framework that learns a continuous distribution in the latent space for protein structures from cryo-EM data. However, although CryoDRGN can simulate complicated structural dynamics, the Gaussian prior distribution of VAE does not match the posterior aggregate approximation, which limits the generative capability of the model²³⁶. Alternatively, a continuous-time diffusion model (that is, score SDEs) can be implemented in CryoDRGN to learn a high-quality generative model for capturing protein conformations directly from cryo-EM imaging data. This CryoDRGN⁹⁸ model is first trained with the standard VAE model using cryo-EM images in Fourier space. The latent space Z, which is predicted by the encoder of the trained VAE, is then fed into the denoise diffusion model based on a ResNet architecture²⁸ to approximate the distribution of the latent variable Z. Finally, the synthesized latent variable Z, which is sampled from the diffusion model and is similar to the target protein’s distribution, is used as input for the decoder of the VAE to generate protein structures with better quality (higher similarity with the target proteins’ distribution) than a VAE, which directly reconstructs protein structures by learning continuous distribution in latent space.

Single-cell image and gene-expression analysis

Reconstructing the 3D shape of a cell from a single-cell 2D microscopy image using computational methods is useful for studying the morphological features of cells. However, each 2D image may permit multiple 3D reconstructions, and therefore, different 2D slices may lead to different predictions of the 3D shape. To tackle this issue, DISPR⁹⁹ employs the U-net architecture²³⁷ and a diffusion process to generate a single-cell 3D shape from 2D images. During training and evaluation, this approach uses a 2D image of an individual cell as an inductive bias. The 2D image is then concatenated with its 3D Gaussian noisy segmentation mask as input for the diffusion-based model to predict realistic 3D cell shapes. DISPR benefits from this training approach and its stochastic property. Unlike VAE-based architectures used in SHAPR²³⁸ and its variants²³⁹, which produce a single, deterministic reconstruction, DISPR employs a stochastic model trained on Gaussian noise and is thus capable of predicting an infinite number of cell shapes, providing a more comprehensive representation of dynamic cell structures. DISPR represents the first use of a diffusion model in the context of 3D cell shape reconstruction, outperforming VAE-based deep generative models, such as SHAPR²³⁸, in terms of volume, surface area and roughness reconstruction⁹⁹.

Single-cell RNA sequencing can assess the expression of genes in individual cells. However, cells typically contain low quantities of RNA, which may cause noisy measurements (for example, varied measurements and experimental bias) of gene expression; moreover, values may be missed (dropouts). Therefore, it is important to denoise single-cell RNA-sequencing data and impute missing values. DEWAKSS¹⁰⁰ applies a diffusion model with a K-nearest-neighbour (KNN) graph to select denoising hyperparameters using the noise2self self-supervision method, thereby not depending on an explicit noise model but on an invariant function of data features. Unlike heuristic-based methods, such as MAGIC²⁴⁰ and KNN-smoothing²⁴¹, which also use KNN graph architecture but may lead to over-smoothing of data variance, DEWAKSS can preserve variances across multiple gene-expression dimensions.

Open-source diffusion model tools

Some diffusion models that can be applied to bioinformatics have been implemented as open-source tools (Table 2). However, these tools do not use NCSNs³⁴ as the diffusion framework, mainly because NSCNs face problems in terms of sampling and training and can thus not achieve high accuracy in image generation. Therefore, NCSNs are less adopted in bioinformatics and computational biology than DDPMs³³ and score SDEs³⁵, which are equipped with efficient sampling and training methods³² for high-definition image generation. Nevertheless, as the first diffusion model, NSCN has made substantial contributions to the development of the field. Furthermore, many bioinformatics applications also include deep learning components to deal with data generation and denoising challenges specific to their application.

Table 2 |.

Open-source diffusion model tools for bioinformatics

Applications	Tool name	Denoising condition	Diffusion framework	Network architecture	Github
Protein design and generation	ProteinSGM91	Conditioned and unconditioned	Score SDE	Convolutional neural network	CODE
	FoldingDiff101	Unconditioned	DDPM	Transformer	CODE
	Genie103	Unconditioned	DDPM	Variant SE(3)-transformer	CODE
	SMCDiff104	Conditioned	Score SDE	EGNN	CODE
	FrameDiff106	Unconditioned	Score SDE	SE(3)-transformer	CODE
	RFdiffusion105	Conditioned and unconditioned	DDPM	SE(3)-transformer	CODE
	Chroma107	Conditioned	Score SDE	GNN	CODE
Small-molecule generation and drug design	EDM54	Conditioned	DDPM	EGNN	CODE
	SDEGen111	Conditioned	Score SDE	GNN	CODE
	DiffLinker113	Conditioned	DDPM	EGNN	CODE
Protein-ligand interaction modelling	DiffBP114	Conditioned	DDPM	EGNN	CODE
	DiffSBDD115	Conditioned	DDPM	EGNN	CODE
	DiffDock116	Conditioned	Score SDE	Variant EGNN	CODE
Cryo-EM data analysis	CryoDRGN98	Conditioned	Score SDE	VAE	CODE
Single-cell image and gene-expression data analysis	DISPR99	Conditioned	DDPM	U-Net	CODE

Score SDE, score stochastic differential equation; DDPM, denoising diffusion probabilistic model; EGNN, equivariant GNN; GNN, graph neural network; cryo-EM, cryogenic electron microscopy; VAE, variational autoencoder; U-Net, U-shaped neural network.

Outlook

Diffusion models can be applied in several bioinformatics applications and may be further extended to other computational biology areas owing to their ability to denoise data and generate realistic new data (Table 3).

Table 3 |.

Applications of diffusion models in bioinformatics

Applications	Tool name	Key function and strength	Input	Output	Diffusion targeta
Protein design and generation	ProteinSGM91	Inpaints plausible backbones/domains; generates native-like structures; allows precise and modular design	Inter-residue 6D feature maps	Full-atomistic structure	Inter-residue 6D feature maps
	FoldingDiff101	Mirrors native folding process; alleviates the need for equivariant networks; unconditionally generates realistic protein structures	6 consecutive backbone angles	Protein backbone structure	6 consecutive backbone angles
	DiffSDS102	Reduces computational complexity and cost; efficiently imposes geometric constraints; outperforms previous strong baselines	6 consecutive backbone angles	Masked protein backbone structure	6 consecutive backbone angles
	ProSSDG84	Operates at large scales; generates realistic proteins structures with sequences; allows interactive structure generation	Secondary structure; coarse constraints	All-atomistic protein structure	Coarse constraints
	Genie103	Dual representation for protein residues; designability and diversity	Oriented reference frames	Protein backbone structure	Oriented reference frames
	SMCDiff104	Efficiently samples scaffolds; samples conditioned on given motif; theoretically guarantees conditional samples	Molecular graph structure	Scaffold structure given input motif	Molecular graph structure
	RFdiffusion105	Generates diverse outputs; can be guided toward specific design objectives; explicitly models 3D structure	Sequence, predicted structure	Diverse, complex, functional protein	RF frames from a predicted structure
	FrameDiff106	Generates designable monomers and diverse protein backbones; does not require pretrained structure predictor	Molecular graph structure	Designable monomer backbone structure	Molecular graph structure
	Chroma107	Jointly models structures and sequences; sub-quadratic computational scaling; arbitrary conditional sampling	Protein graph structure	Proteins with desired functions	Protein graph structure
Small-molecule generation and drug design	CDGS108	Incorporates discrete graph structures; specialized graph noise prediction model; similarity-constrained molecule optimization pipeline	Graph structures and inherent features	Molecular graphs	Discrete graph structure
	EDM54	Equivariant to Euclidean transformations; operates on continuous and categorical features; admits likelihood computation	Atom coordinates, atom types	3D molecular graphs	Coordinates and categorical features
	DiffBridge109	Incorporates prior information; generates realistic molecules; uses physically informed diffusion bridges	Masked points in 3D Euclidean space	Realistic molecules or point cloud	Molecular graph structure
	DGSM110	Models local and long-range interactions; dynamically constructs graph structures; estimates gradient fields of logarithm density	2D molecular graphs	Stable 3D conformations	Molecular graph structure
	SDEGen111	Captures multimodal conformation distribution; quickly searches low-energy conformations and higher efficiency	Small molecules	Representative conformations	Encoding of graph structure
	DiffMD112	No intermediate variables; directly estimates gradient of log density; incorporates directions and velocities of atomic motions	3D coordinates, velocities, and invariant features	Molecule simulation trajectories	Atomic position coordinates
	DiffLinker113	Links arbitrary number of fragments; generates diverse and synthetically accessible molecules; conditions on protein pockets	Molecule structure	A molecule incorporating all the input fragments	Molecular graph structure
Protein–ligand interaction modelling	DiffBP114	Non-autoregressive generation; generates molecules with high affinity to target proteins and desirable drug properties	Protein–ligand structure	Protein–ligand structure given input protein	Protein graph structure
	DiffSBDD115	Generates diverse drug-like ligands; efficient in silico experiments; uses experimentally determined binding data	Protein–ligand structure	High-affinity ligands given protein pockets	Protein graph structure
	DiffDock116	Maps manifold to product space; provides confidence estimates; maintains precision on computationally folded structures	Protein–ligand structure	Ranked ligand poses, confidence scores	Ligand poses
	NeuralPLexer117	Repacks failed AlphaFold2 sites; enables end-to-end design; generalizes to ligand-unbound or predicted protein structure inputs	Protein backbone template and ligand molecular graphs	Full-atom protein–ligand structure	Contact maps, geometry prior
	NERE118	Benchmarks on protein–ligand and antibody–antigen dataset; outperforms other unsupervised methods	Protein–ligand complex	Binding affinity of the protein–ligand structure	Ligand atom coordinates
Cryo-EM data analysis	CryoDRGN98	Accurate data distribution sampling; fast latent space traversal; unlocks generative modelling tools	Single-particle cryo-EM imaging	High-quality 3D structure	Latent space of image embeds
Single-cell image and gene-expression analysis	DISPR99	Realistic 3D reconstructions; data augmentation tool in single-cell classification task; inverse biomedical problems	2D microscopy images as a prior	3D cell shape predictions	3D microscopy point cloud
	DEWAKSS100	Maintains cellular identity; preserves data variance; maintains cluster homogeneity	Gene-expression data	Single-cell genomics data	Gene-expression matrix

^aThe term ‘diffusion’ refers to the application of the forward and reverse diffusion processes on a specific data representation, that is, the target. Cryo-EM, cryogenic electron microscopy. RF, RosettaFold.

3D genomics data analysis

High-throughput chromosome conformation capture (Hi-C) is a key technology for studying 3D conformations of chromosomes and genomes, applying next-generation sequencing techniques to sequence chromosomal regions that are spatially close to each other (that is, in contact)²⁴². Thus, Hi-C data captures the interactions between chromosomal regions of a genome to build 3D conformations of the genome^243,244 and study long-range gene-enhancer interactions. This approach typically requires the data to be converted into 2D chromosomal contact matrices (maps), which store the frequency at which chromosome region i interacts with chromosome region j, where i and j are the indices of chromosome regions. Therefore, a Hi-C contact matrix can be considered an image. However, Hi-C data, in particular, single-cell Hi-C data, are usually noisy and incomplete, so that chromosomal interactions in chromosomal contact matrices may be false positives or interactions may be missing in the matrices. Deep learning methods (for example, GANs) can be applied to denoise Hi-C data²⁴⁵; in addition, diffusion models (for example, DDPM) may enable denoising of Hi-C chromosomal contact matrices to improve 3D genome conformation modelling and to study spatial interactions between genes and regulatory elements (for example, enhancers). However, the deep learning architecture of DDPM is typically the U-Net, which may not be as powerful as the deep residual network used in the Hi-C data denoising method ScHiCEDRN²⁴⁶. Thus, if applied to Hi-C data denoising, the architecture of DDPM would have to be updated to deep residual networks to improve its denoising ability.

Single-cell reconstruction and inference

The activity of a single cell can be captured by various ‘omics data, such as transcriptomics (RNA-seq), proteomics, chromosome accessibility (ATAC-Seq) and epigenetics (bisulfite sequencing), which may benefit from diffusion models; for example, data could be inferred to one modality (for example, RNA-Seq) from another modality, such as ATAC-Seq data and genome methylation data; missing spots in single-cell spatial transcriptomic data could be calculated; spots (each consisting of multiple cells) in 10× spatial transcriptomic data could be decomposed into single cell data (super-resolution); and single-cell data could be used to build 3D models of the spatial arrangement of cells. Moreover, diffusion models designed to denoise images could also be applied to denoise single-cell ‘omics data, such as transcriptomics, proteomics, metabolomics and epigenetics data.

DNA regulatory element design

The expression of genes is modulated by short DNA sequences on genomes, called regulatory elements, such as enhancers and promoters. Designing regulatory elements is an important approach to designing synthetic cells using synthetic biology. Generative models, such as GANs²⁴⁷, can be applied to design enhancers that regulate the expression of genes and the development of cell types. However, diffusion models have shown better performance in image synthesis than GANs⁵⁰ and may thus be more suitable for the design of enhancers and other gene regulatory elements.

Cryo-electron microscopy image denoising

Diffusion models can reconstruct complex protein structures from 3D cryo-EM density maps, which are typically made of noisy and low-contrast 2D cryo-EM protein particle images, isolated from large 2D cryo-EM protein images (also called cryo-EM micrographs). However, denoising original cryo-EM images to build better 3D cryo-EM density maps remains challenging. Although image preprocessing techniques, such as EMAN2 (ref. 248), can denoise cryo-EM images²⁴⁹, diffusion models trained on many noisy images at various noise levels and their clean counterparts may allow the recovery of clean cryo-EM images more effectively than conventional image processing techniques that have not been trained to learn the noise distribution of cryo-EM images²⁵⁰.

Peptide design

Peptides, which are short, typically unfolded amino acid sequences, can bind to proteins to modulate their function, which has been explored for drug design. Diffusion models can not only be designed to generate new proteins but could also be adapted to create peptides that can modulate protein function. For example, diffusion models pretrained for protein design may be retrained on a peptide dataset to design peptides through transfer learning.

Protein structure refinement and mutation prediction

The tertiary and quaternary structures of many proteins and protein complexes can be fairly accurately predicted by AlphaFold2 (ref. 163) and AlphaFold-multimer²⁵¹, respectively. However, such predicted structures may contain structural errors and may thus need to be refined. Conditioned on a predicted structure input, diffusion models may be able to remove noise from the predicted structure to bring it closer to the native structure. Similarly, it remains challenging to predict how an amino acid mutation alters the structure of a protein, which may affect protein function and phenotype. Diffusion models have generative capability, demonstrated in protein design, and may therefore be able to transform the known structure of a protein without mutation (wild-type protein) to the structure of the same protein with mutations (mutant) to predict structural changes induced by mutations.

Limitations of current diffusion models

Although diffusion models may be applied for various bioinformatics applications, potentially outperforming GANs and VAEs, some limitations remain to be addressed. First, the training process of diffusion models involves the introduction of Gaussian noise to the data, resulting in a long training time. Second, although considerable efforts have been directed towards increasing the sampling speed in diffusion models, the sampling time of most models still exceeds that of other deep generative models (for example, GANs and VAEs). The long sampling time hinders some real-time applications of diffusion models. Developing a streamlined approach of single-step noise addition and removal may reduce the training and sampling time. Third, the computational resource requirements of diffusion models are higher than those of GANs and VAEs. Therefore, the trade-off between performance improvement and computational resource demand needs to be evaluated when deciding which model to use. Furthermore, new applications of diffusion models are often non-trivial and may require validation of suitable data representations (embeddings), types of diffusion model and deep learning architectures.

Key points.

• Diffusion models are a generative artificial intelligence technology that can be applied in natural language processing, image synthesis and bioinformatics.

• Diffusion models have contributed greatly to computational protein design and generation, drug and small-molecule design, protein–ligand interaction modelling, cryo-electron microscopy data enhancement and single-cell data analysis.

• Many diffusion models are also available as open-source tools.

• Although diffusion models may potentially outperform other generative approaches, such as generative adversarial networks and variational auto-encoders, their computational resource requirements remain high.

Citation diversity statement

We acknowledge that papers authored by scholars from historically excluded groups are systematically under-cited. Here, we have made every attempt to reference relevant papers in a manner that is equitable in terms of racial, ethnic, gender and geographical representation.