Table 2. Summary of diffusion models in text generation, grouped by type.

Model	Noise schedule	Sampling	Space	Generation process	Pretrain
Conditional text generation (Text-driven generation)
DiffuSeq	Partial noising	Minimum Bayes Risk	Ca	NARb	/
DiffuSum	Partial noising	/	C	NAR	/
DiffusER	Edit-based reconstruction	Beam search, 2D Beam search, Nucleus sampling	D	NAR	/
SeqDiffuSeq	Adaptive noise schedule	Self-conditioning	C	NAR	/
Zero-Shot Diffusion	Partial noising	Classifier-free conditional denoising	C	NAR	/
GENIE	/	Continuous paragraph denoise	C	NAR	Arge-scale pretrained diffusion language model
RDMs	Mask	Reparameterized sampling, stochastic routing mechanism	D	NAR	Pre-trained autoregressive Transformer
Diffusion-NAT	Mask	Self-prompting	D	NAR	BART
CDCD	Time warping	Inverse transform sampling, time warping	C	NAR	BERT
DiNoiSer	Manipulated noises	MBR	C	NAR	/
AR-DIFFUSION	Square-root	Multi-level diffusion strategy, dynamic movement speeds, MBR	C	AR	/
Conditional text generation (Fine-grained control generation)
Diffusion-LM	Cosine	MBR	C	NAR	/
Masked-Diffuse LM	Strategically soft-masking	MBR	D	NAR	BERT
Difformer	Sqrt noise	2D parallel decoding	C	NAR	/
Text-driven generation and Fine-grained control generation
LDEBM	/	/	C	NAR	/
Unconstrained text generation
D3PM	Uniform transition matrices	/	D	NAR	/
DiffusionBERT	Spindle schedule	$x_0$-Parameterization	D	NAR	BERT
Multi-mode text generation
SED	Span masking	Self-conditioning	C	NAR	Embedding pretraining
SUNDAE	Uniform transition matrices	Unrolled denoising, low-temperature sampling, argmax-unrolled decoding, updating fewer tokens	C	NAR	/
LD4LG	Cosine	Self-conditioning	C	NAR	BART
SSD-LM	Logits-generation	Sampling, multi-hot and greedy	C	NAR	/

Notes:

^a“C” and “D” respectively represent continuous and discrete.

^b“AR” and “NAR” respectively stand for autoregressive and non-autoregressive.