Table 1.

Literature overview of the state-of-the-art methods for medical image synthesis.

Reference (Year)	Task	Dataset Information	Core Methodology	Remarks
Rivenson et al. (2019) (37)	Virtual staining	Whole slides of 211,475 and 59,344 of Liver and Kidney tissues	GAN framework, U-Net generator combined with an FCN discriminator	Pros: GAN-generated virtual staining can provide similar results as conventional staining, providing time and cost-saving Cons: The GAN framework is not validated for other contrast-generating methods multiple excitation and emission wavelengths
Muckley et al. (2020) (34)	MR image reconstruction	7,299 clinical brain scans subsampled k-space data	Comparative analysis of networks for MR image reconstruction for fastMRI challenge 2020	Pros: Deep learning methodologies decrease the minimum requirement for MR image reconstruction set by parallel imaging and compressed sensing methods Cons: Pseudo-regular sampling of the MR data lacks realism and is not equivalent to the perfectly equidistant sampling pattern used on MRI systems
Qu et al. (2020) (35)	Image enhancement (Image super-resolution)	15 pairs of 3T and 7T brain images	WATNet, an encdoer decoder network with wavelet priors and conditional normalization	Pros: Wavelet coefficient can allow learning feature map normalization weights Cons: Other tasks, such as MRI to CT and T2 images from T1 translation have not been explored in the work
He et al. (2020) (39)	RF super resolution	50 human subjects, 50-90 frames per patient	Super-resolution radio-frequency neural network (SRRFNN) inspired by a super-resolution GAN	Pros: Laterally upsampled RF data processed by SRRFNN performs better than conventional bi-cubic interpolation approach Cons: The method does not utilize the actual high-frequency US data using novel beam-forming technology for training
Li et al. (2021) (36)	MR image reconstruction	305 paired brain MRI samples with a thickness of 1.0 mm and 6.5 mm	3D U-Net followed by a convolutional LSTM network for MRI slice refinement	Pros: Practical and clinical value of generated thin MRI is higher than other voxel-based morphometry Cons: The quality of reconstruction is directly dependent on the accuracy of statistical parametric mapping (SPM)
Dalmaz et al. (2022) (41)	MRI to CT translation, MRI missing slices generation	IXI dataset (53 subjects), BRATS dataset (55 subjects) (42), multi-modalpelvic MRI-CT dataset (15 subjects) (43)	ResViT architecture with vision transformers’ block at the bottleneck and convolution operators in the encoder and decoder of the GAN generator.	Pros: Convolutional and transformer branches within a residual bottleneck of the generator preserves both local precision and contextual sensitivity Cons: Architecture needs further validation with unpaired sets of medical images using cycle consistency loss.
Ozbey et al. (2022) (44)	MRI to CT translation	IXI dataset (40 subjects), BRATS dataset (55 subjects) (42), multi-modal pelvic MRI-CT dataset (15 subjects) (43)	Adversarial diffusion modeling using conditional diffusion for capturing and correlating the image distributions.	Pros: Cycle-consistent architecture is used with coupled diffusive and non-diffusive components to bilaterally translate between imaging modalities. Cons: Adversarial loss in diffusion models introduce training instability and suboptimal convergence
Zhang et al. (2022) (38)	Elastogram generation	726 thyroid US elastography images of 397 patients	AUE-Net GAN framework, U-Net generator with spatial and color attention.	Pros: L1 loss can be added to the generator loss for improving the color distributions of generated elastograms Cons: The method does not perform qualitative evaluation of the generated elastograms
Yu et al. (2023) (40)	Elastogram generation	4580 breast cancer cases from 15 medical centers	GAN with a U-Net generator, global and local tumor discriminator, with L1 loss and color coefficient	Pros: AR-EUS improves the diagnosis accuracy of pocket US Cons: The GAN framework has been only validated for the Chinese population