PET image denoising based on denoising diffusion probabilistic model

Abstract

Purpose:

Due to various physical degradation factors and limited counts received, PET image quality needs further improvements. The denoising diffusion probabilistic model (DDPM) was a distribution learning-based model, which tried to transform a normal distribution into a specific data distribution based on iterative refinements. In this work, we proposed and evaluated different DDPM-based methods for PET image denoising.

Methods:

Under the DDPM framework, one way to perform PET image denoising was to provide the PET image and/or the prior image as the input. Another way was to supply the prior image as the network input with the PET image included in the refinement steps, which could fit for scenarios of different noise levels. 150 brain [¹⁸F]FDG datasets and 140 brain [¹⁸F]MK-6240 (imaging neurofibrillary tangles deposition) datasets were utilized to evaluate the proposed DDPM-based methods.

Results:

Quantification showed that the DDPM-based frameworks with PET information included generated better results than the nonlocal mean, Unet and generative adversarial network (GAN)-based denoising methods. Adding additional MR prior in the model helped achieved better performance and further reduced the uncertainty during image denoising. Solely relying on MR prior while ignoring the PET information resulted in large bias. Regional and surface quantification showed that employing MR prior as the network input while embedding PET image as a data-consistency constraint during inference achieved the best performance.

Conclusions:

DDPM-based PET image denoising is a flexible framework, which can efficiently utilize prior information and achieve better performance than the nonlocal mean, Unet and GAN-based denoising methods.

Introduction

Positron Emission Tomography (PET) was widely used in oncology, cardiology and neurology studies because of its high sensitivity and quantitative merits. Due to various physical degradation factors and limited counts received, the signal-to-noise ratio (SNR) and resolution of PET was low, which comprised its clinical values in diagnosis, prognosis, staging and treatment monitoring. Additionally, to improve the hospital’s throughput or reduce the radiation exposures to patients, faster or low-dose PET imaging was desirable, where the counts received during the scan was even less. This further challenged our ability to attain high-quality PET images from limited counts.

For the past decades, various approaches were explored for PET image denoising, e.g., wavelet [1], highly constrained backprojection (HYPR) [2], nonlocal mean [3, 4], guided image filtering [5], block matching [6], and dictionary learning [7]. These methods were mainly based on specifically designed similarity calculation or small-scale feature extraction/learning. With the availability of enormous training data and strong computational power, deep learning-based image processing were extensively studied recently. More specifically, convolutional neural network (CNN)-based PET image denoising achieved superior performance than other state-of-the-art methods [8–16]. However, over-smoothness of the network output was one pitfall of most CNN-based approaches. Conditional generative adversarial networks (cGANs)-based PET image denoising were developed to address this issue [17–22], where the learned discriminative loss could further push the image appearance closer to the ground truth. One issue of the CNN and cGAN-based methods was that if there were data-distribution mismatches between training and testing data, the network performance could be compromised. Transfer learning was investigated as one approach to address the mismatch issue and was demonstrated effective in various PET studies [23–27].

The deep learning approaches described above were deterministic models, where only one denoised image was obtained based on the pre-specified network input. The denoising diffusion probabilistic model (DDPM) and score function-based model [28–30], denoted as diffusion models here, were a category of distribution learning-based models, which tried to transform a normal distribution into a specific data distribution based on iterative refinements. During the forward pass, noise was injected in the image gradually. During the reverse pass, data samples were generated by removing noise from the image after multiple refinement steps. Fig. 1 showed the diagram of the forward and reverse pass. Additional prior image could be added to the model to conduct conditional image generation. By running the reverse pass multiple times, an uncertainty map could also be calculated from the generated data samples, which could be used to further guide diagnosis and treatment monitoring [31–33]. The diffusion models have shown encouraging results in various computer-vision tasks, e.g., image generation [28–30, 34, 35], super resolution [34, 36], image in-painting [37], and MR image reconstruction [38, 39].

Figure 1:

Diagram of the DDPM framework.

As the dynamic range of different organs’ uptake was high and the noise levels of different scans varied, the data distribution of PET images was more complicated than natural images. Additionally, apart from PET image itself, there was patients’ anatomical prior information (e.g., CT or MR images) available from the same or separate scans. Thus, further evaluating the diffusion models and exploring approaches to embed the patient’s prior information for PET image denoising deserved further investigations. In this work, we evaluated DDPM for PET image denoising and further explored different approaches of embedding the PET and the prior MR information in DDPM. Nonlocal mean, Unet and Pix2Pix [40] methods were employed as the reference methods. Unet and Pix2Pix were widely used CNN and cGAN-based methods, respectively. 150 brain [¹⁸F]FDG datasets and 140 [¹⁸F]MK-6240 (imaging neurofibrillary tangles deposition) datasets were utilized in two evaluation studies to evaluate the performance of different methods based on global and regional quantitative metrics.

Materials and methods

DDPM model

Suppose the target data ${\mathit{x}}_0\sim q\left({\mathit{x}}_0\right)$. During the forward process, we could define a Markov diffusion process $q$ that added Gaussian noise to ${\mathit{x}}_0$ in each step, which could be written as [28]

$$q\left({\mathit{x}}_1,\mathrm{..},{\mathit{x}}_T\mid {\mathit{x}}_0\right)=\prod _{t=1}^{T}q\left({\mathit{x}}_t\mid {\mathit{x}}_{t-1}\right),q\left({\mathit{x}}_t\mid {\mathit{x}}_{t-1}\right)=𝒩({\mathit{x}}_t;\sqrt{1-{\beta }_t}{\mathit{x}}_{t-1},{\beta }_t\mathit{I}).$$ (1)

One property of the forward process $q$ was that it allowed us to sample at any arbitrary step directly conditioned on ${\mathit{x}}_0$. Denoting ${\alpha }_t=1-{\beta }_t$ and ${\bar{\alpha }}_t={\prod }_{l=1}^t {\alpha }_l$, we could have

$$q\left({\mathit{x}}_t\mid {\mathit{x}}_0\right)=𝒩\left({\mathit{x}}_t;\sqrt{{\overline{\alpha }}_t}{\mathit{x}}_0,\left(1-{\overline{\alpha }}_t\right)\mathit{I}\right),{\mathit{x}}_t=\sqrt{{\overline{\alpha }}_t}{\mathit{x}}_0+\left(1-{\overline{\alpha }}_t\right)\mathit{ϵ},$$ (2)

where $\mathit{ϵ}\sim 𝒩(\mathbf{0},\mathit{I})$. Based on Bayes theorem, the posterior $q\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)$ also followed Gaussian distribution,

$$q\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)=𝒩({\mathit{x}}_{t-1};{\tilde{\mathit{\mu }}}_t\left({\mathit{x}}_t,{\mathit{x}}_0\right),\frac{1-{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_t}{\beta }_t\mathit{I}),$$ (3)

$$\text{with}{\tilde{\mathit{\mu }}}_t\left({\mathit{x}}_t,{\mathit{x}}_0\right)=\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}{\mathit{x}}_0+\frac{\sqrt{{\alpha }_t}\left(1-{\bar{\alpha }}_{t-1}\right)}{1-{\bar{\alpha }}_t}{\mathit{x}}_t.$$ (4)

If we aimed to sample from the target data distribution $q\left({\mathit{x}}_0\right)$, we could first sample from $q\left({\mathit{x}}_T\right)$, which was an isotropic Gaussian distribution given a large enough $T$. Then based on the posterior distribution $q\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)$, we could obtain samples of ${\mathit{x}}_0$. However, $q\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)$ was not computable as the distribution of ${\mathit{x}}_0$ was unknown. The DDPM framework tried to approximate $q\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)$ by $p_{\mathit{\theta }}\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)$ through a network with parameter $\mathit{\theta }$, where

$$p_{\mathit{\theta }}\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)=𝒩\left({\mathit{x}}_{t-1};{\tilde{\mathit{\mu }}}_{\mathit{\theta }}\left({\mathit{x}}_t,t\right),{\sigma }_t^2\mathit{I}\right).$$ (5)

Instead of directly approximating ${\tilde{\mathit{\mu }}}_{\theta }\left({\mathit{x}}_t,t\right)$ by a neural network, Ho et al [28] proposed to approximate the noise $\mathit{ϵ}$ in equation (2) by a network as ${\mathit{ϵ}}_{\mathit{\theta }}\left({\mathit{x}}_t,t\right)$, which could also be interpreted as the gradient of the data log-likelihood, known as the score function. Based on equations (2) and (4), ${\tilde{\mathit{\mu }}}_{\mathit{\theta }}\left({\mathit{x}}_t,t\right)$ could expressed as

$${\tilde{\mathit{\mu }}}_{\mathit{\theta }}\left({\mathit{x}}_t,t\right)=\frac{1}{\sqrt{{\alpha }_t}}\left[{\mathit{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathit{ϵ}}_{\mathit{\theta }}\left({\mathit{x}}_t,t\right)\right].$$ (6)

Based on the trained score function ${\mathit{ϵ}}_{\stackrel{\mathit{ˆ}}{\mathit{\theta }}}\left({\mathit{x}}_t,t\right)$, and equations (5) and (6), each refinement step during inference was

$${\mathit{x}}_{t-1}=\frac{1}{\sqrt{{\alpha }_t}}\left[{\mathit{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathit{ϵ}}_{\stackrel{\mathit{ˆ}}{\mathit{\theta }}}\left({\mathit{x}}_t,t\right)\right]+{\sigma }_t\mathit{z},\text{where}\mathit{z}\sim 𝒩(\mathbf{0},\mathit{I}).$$ (7)

Conditional PET image denoising based on DDPM

The original DDPM stated above was for unconditional image generation. For PET image denoising, the noisy PET image ${\mathit{x}}_{\text{noisy}}$ existed and the patient’s prior image ${\mathit{x}}_{\text{prior}}$ might also be available. We could directly supply ${\mathit{x}}_{\text{noisy}}$ and ${\mathit{x}}_{\text{prior}}$ (if available) as the additional network input of the score function, as was done in image super-resolution [36], and the score function ${\mathit{ϵ}}_{\mathit{\theta }}\left({\mathit{x}}_t,t\right)$ changed to ${\mathit{ϵ}}_{\mathit{\theta }}\left({\mathit{x}}_t,t,{\mathit{x}}_{\text{noisy}},{\mathit{x}}_{\text{prior}}\right)$. This framework required specific low- and high-quality training pairs during the score function training. Another approach was to only supply ${\mathit{x}}_{\text{prior}}$ as the additional input to the score function, while including ${\mathit{x}}_{\text{noisy}}$ during inference. Compared to the previous approach, this approach did not need low- and high-quality pairs during training and could thus work for noisy PET images of various noise levels during inference. In this approach, we could approximate $q\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)$ by $p_{\mathit{\theta }}\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t,{\mathit{x}}_{\text{noisy}},{\mathit{x}}_{\text{prior}}\right)$ instead of $p_{\mathit{\theta }}\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t\right)$ in the original DDPM framework, which could be rewritten as

$$p_{\mathit{\theta }}\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t,{\mathit{x}}_{\text{noisy}},{\mathit{x}}_{\text{prior}}\right)\propto p_{\mathit{\theta }}\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t,{\mathit{x}}_{\text{prior}}\right)p\left({\mathit{x}}_{\text{noisy}}\mid {\mathit{x}}_{t-1},{\mathit{x}}_t,{\mathit{x}}_{\text{prior}}\right).$$ (8)

Suppose the PET image noise followed Gaussian distribution and was independent of ${\mathit{x}}_{\text{prior}}$, we could have

$$p\left({\mathit{x}}_{\text{noisy}}\mid {\mathit{x}}_{t-1},{\mathit{x}}_t,{\mathit{x}}_{\text{prior}}\right)\approx 𝒩\left({\mathit{x}}_t,{\sigma }_d^2\mathit{I}\right),$$ (9)

where ${\sigma }_d$ indicated the noise level of ${\mathit{x}}_{\text{noisy}}$. As for $p_{\mathit{\theta }}\left({\mathit{x}}_{t-1}\mid {\mathit{x}}_t,{\mathit{x}}_{\text{prior}}\right)$, similar to equations (5) and (6), $\mathit{ϵ}$ was approximated by ${\mathit{ϵ}}_{\mathit{\theta }}\left({\mathit{x}}_t,t,{\mathit{x}}_{\text{prior}}\right)$, where ${\mathit{x}}_{\text{prior}}$ was supplied as an additional input to the score function. Based on equation (10) from Ref. 34 and equation (9), the refinement step during inference changed to

$${\mathit{x}}_{t-1}=\frac{1}{\sqrt{{\alpha }_t}}\left[{\mathit{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathit{ϵ}}_{\stackrel{\mathit{ˆ}}{\mathit{\theta }}}\left({\mathit{x}}_t,t,{\mathit{x}}_{\text{prior}}\right)\right]-\frac{{\sigma }_t^2}{{\sigma }_d^2}\left({\mathit{x}}_{\text{noisy}}-{\mathit{x}}_t\right)+{\sigma }_t\mathit{z}.$$ (10)

Datasets

Two experiments based on [¹⁸F]FDG and [¹⁸F]MK-6240 datasets were conducted to evaluate the performance of DDPM for PET image denoising and also explore different approaches of embedding prior information. Fig. 1 showed a general description of the datasets used for the two experiments.

Firstly, 150 brain [¹⁸F]FDG datasets acquired from the GE DMI PET-CT scanner at MD Anderson Cancer Center were utilized. The scans were performed 1 hour post injection with 2 min per bed position. The injected doses were 359.82 ± 41.49 MBq. No head and neck position immobilization nor PET/CT alignment were performed. Of all the datasets, 85 datasets were used for training, 5 datasets for validation and the remaining 60 datasets for testing. 87 data pairs were generated from each patient dataset and there were in total 7395 training data pairs. Normal-dose datasets were used as the ground truth and low-dose datasets were generated by extracting 1/4 events from the normal-dose listmode datasets. All the datasets were reconstructed into a matrix of 256 × 256 × 89 with a voxel size of 1.17 × 1.17 × 2.79 mm³ based on the ordered subset expectation maximization (OSEM) algorithm (3 iterations and 17 subsets) including point spread function (PSF) and time of flight (TOF) modeling. The images were later scaled to the standardized uptake value (SUV) unit before being further processed using different methods. For the proposed DDPM-based method, low-dose PET images were supplied as the network input and this method was denoted as DDPM-PET.

The other experiment was based on [¹⁸F]MK-6240 datasets with corresponding T1-weighted MR images. In total 140 dynamic [¹⁸F]MK-6240 datasets acquired on the GE DMI PET/CT scanner (injected doses around 185 MBq) were utilized in this experiment. The datasets were acquired from 66 subjects. Some of the subjects had longitudinal scans. The testing data did not include datasets scanned from subjects that were included in the training/validation population. The demographics of the subjects were shown in Table 1. For each dataset, the events from 0 min to 10 min post injection, were extracted and combined to construct the normal-dose dataset. These early-time-frame datasets mainly contained brain perfusion information, which was actively investigated as a potential biomarker for neurodegenerative diseases [41–43]. The image reconstruction algorithm, matrix size and pixel size were the same as the above-mentioned [¹⁸F]FDG datasets. Additional T1-weighted MR images were acquired from the Siemens 3T MR scanner and were registered to the PET images through rigid registration by ANTS [44]. To simulate low-dose scenerios, we further down sampled the listmode events to generate 1/4 low-dose datasets. Of all the 140 data pairs, 116 were employed for training, 4 for validation and the remaining 20 for testing. 87 data pairs were generated from each patient dataset and there were in total 10092 training data pairs. To further test the robustness of different methods to mismatches between training and testing datasets, we also downsampled the normal-dose test datasets to 1/16 low-dose test datasets. In this experiment, we have different DDPM-based methods by varying embedding approaches of the low-dose PET and MR prior images. The method using PET image only as the network input was denoted as DDPM-PET (the same as in the [¹⁸F]FDG experiment). The method using MR image only as the input was denoted as DDPM-MR. The method using both PET and MR images as the input was denoted as DDPM-PETMR. Finally, the method using the MR image as the network input while using the PET image in the data consistency item, as shown in equation (10), was denoted as DDPM-MR-PETCon.

Table 1:

Demographics of the subjects in the [¹⁸F]MK-6240 study. CN: cognitive normal. MCI: mild cognitive impairment. AD: Alzheimer’s disease. MMSE: Mini-Mental State Examination.

	CN (N = 57)	MCI (N = 6)	AD (N = 3)
Age	68.0 ±11.5	69.9 ±10.0	69.1±12.9
Females	47.4%	50.0%	67.7%
MMSE	28.6±1.3	21.7 ±5.0	21.7 ±6.1

Training details and reference methods

The network used to generate the score function ${ϵ}_{\theta }$ was a Unet structure with attention and residual blocks as described in [34]. Two neighboring axial slices were additionally supplied as the network input, to avoid the axial artifacts. During network training, the batch size was set to 20 for all methods. The network input was 256 × 256 ×3, 256 × 256 ×3, 256 × 256 ×3, and 256 × 256 ×6, for DDPM-PET, DDPM-MR, DDPM-MR-PETCon, and DDPM-PETMR, respectively. For the score function training, the loss function used was the same as in [28],

$$L(\mathit{\theta })={\mathbb{E}}_{t\sim [1,T],{\mathit{x}}_0\sim q\left({\mathit{x}}_0\right),\mathit{ϵ}\sim 𝒩(0,\mathit{I})}({∥\mathit{ϵ}-{\mathit{ϵ}}_{\mathit{\theta }}\left({\mathit{x}}_t,t\right)∥}_2^2).$$ (11)

where $\mathbb{E}$ denoted expectation, the number of time steps $T$ was set to 1000, and $t$ was uniformly chosen from 1 to $T.{\beta }_t$ was set to increase linearly from ${\beta }_1=0.0001$ to ${\beta }_T=0.02$ as in [28]. For each DDPM-based method, the training took around 10 days using 4 NVIDIA RTX 8000 GPUs. The inference took around 50 mins based on 1 GPU.

For both [¹⁸F]FDG and [¹⁸F]MK-6240 experiments, the nonlocal mean (NLM), Unet, and Pix2Pix-based GAN were adopted as the reference methods. For the Unet and GAN methods, the training datasets and the number of training parameters are the same as the DDPM-based methods. For the [¹⁸F]FDG experiment, the similarity in the NLM method was calculated based on the low-dose PET image. The three-channel input of the Unet and GAN is also the low-dose PET image. In the [¹⁸F]MK-6240 experiment, the similarity in the NLM method was calculated based on the MR prior image. The six-channel input of the Unet and GAN contains both the low-dose PET and MR prior images.

Data analysis

For [¹⁸F]FDG and [¹⁸F]MK-6240 test datasets, the peak signal-to-noise ratio (PSNR) and and structural similarity index measure (SSIM) were calculated for different methods with the normal-dose PET images as the ground truth. PSNR was calculated as

$$PSNR=10\cdot {\mathrm{l}\mathrm{o}\mathrm{g}}_{10}\left(\frac{MA{X}_I^2}{MSE}\right)$$ (12)

where $MA{X}_I$ was the maximum possible pixel value of the image. $MSE=\frac{1}{n}{\sum }_{i=1}^n {\left({\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{post}}^i-{\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{normal}}^i\right)}^2$, where ${\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{post}}$ and ${\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{normal}}$ were the low-dose image after post-processing and the normal-dose image, respectively. Given two image vectors $\mathbf{x}$ and $\mathbf{y}$, the general definition of SSIM was

$$\mathrm{S}\mathrm{S}\mathrm{I}\mathrm{M}(\mathbf{x},\mathbf{y})=[l(\mathbf{x},\mathbf{y}){]}^{\alpha }\cdot [c(\mathbf{x},\mathbf{y}){]}^{\beta }\cdot [s(\mathbf{x},\mathbf{y}){]}^{\gamma },$$ (13)

where $l(\mathbf{x},\mathbf{y})=\frac{2{\mu }_x{\mu }_y+C_1}{{{\mu }_x}^2+{{\mu }_y}^2+C_1},c(\mathbf{x},\mathbf{y})=\frac{2{\sigma }_x{\sigma }_y+C_2}{{{\sigma }_x}^2+{{\sigma }_y}^2+C_2}$ and $s(\mathbf{x},\mathbf{y})=\frac{{\sigma }_{xy}+C_3}{{\sigma }_x{\sigma }_y+C_3}$ were the luminance, contrast and similarity measures, respectively. Here ${\mu }_x,{\mu }_y,{\sigma }_x^2,{\sigma }_y^2$ were the means and variances of $\mathbf{x}$ and $\mathbf{y}$, respectively, and ${\sigma }_{xy}$ was the covariance between $\mathbf{x}$ and $\mathbf{y}.C_1,C_2$ and $C_3$ were small constants used to stabilize the results. All parameters were based on the default setting in MATLAB 2021a (Mathworks, MA) The Wilcoxon signed-rank tests of PSNR and SSIM were conducted to compare different methods. In the [¹⁸F]MK-6240 experiment, as the patient’s MR image was available, Freesurfer [45] was used for cortical parcellation based on the MR image to obtain the region of interests (ROIs). Further quantification of the cortical ROIs based on PSNR was performed. The Braak stage-related cortices—hippocampus, entorhinal, parahippocampal, amygdala, inferior temporal, fusiform, posterior cingulate, lingual, precuneus, insula, pericalcarine, cuneus and precentral—were chosen as the evaluation ROIs.

We have further performed surface analysis to comprehensively evaluate different methods’ performance on generating high-quality PET images in the [¹⁸F]MK-6240 experiment. Firstly, the relative PET error, ${\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{error}}$, was calculated as

$${\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{error}}=\frac{{\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{post}}-{\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{normal}}}{{\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{normal}}}.$$ (14)

Afterwards, the surface map of ${\mathrm{P}\mathrm{E}\mathrm{T}}_{\text{error}}$ was generated for each [¹⁸F]MK-6240 test dataset and was registered to the FSAverage template in Freesurfer to construct the averaged surface map of all the 20 test datasets.

One characteristic of the diffusion models is the ability to generate the uncertainty map based on multiple forward passes during inference. We forwarded pass the same test dataset through the network 20 times. The variance of the 20 network outputs were used to represent the uncertainty map of the network output. Note that the uncertainty of the network output is not the same as the error of the model prediction. To understand the relationship between the uncertainty of the network output and the model prediction error, we calculated the correlation between the output uncertainty at different brain regions and the mean squared error (MSE) calculated at the same brain regions. The MSE was calculated by utilizing the normal-dose data as the ground truth.

Results

Fig. 3 showed three views of one [¹⁸F]FDG dataset processed with different methods. The error images generated by different methods compared to the normal-dose image were also shown. It could be observed that both the Unet and the DDPM-PET methods could generate results with better cortical details and smaller noise in the while matter region compared to the NLM method. From the error images, we observed that the DDPM-PET method could generate a smaller error compared with other methods. Fig. 4 presented the PSNR and SSIM values of different methods based on 60 test datasets, which showed that the DDPM-PET method had better quantification results than other reference methods.

Figure 3:

Three views of one [¹⁸F]FDG test dataset processed using different methods along with the 1/4 low-dose image (1st column) and the normal-dose image (last column). For each view, the second row shows the difference images of difference methods when compared with the normal-dose image.

Figure 4:

The PSNR (left) and SSIM (right) values using different methods calculated based on 60 [¹⁸F]FDG test datasets. ***, **, ns, located at the top of the bar plot represents p-value< 0.001, p-value< 0.01 and p-value≥ 0.05, respectively.

Fig. 5 showed three views of one [¹⁸F]MK-6240 dataset processed using different methods along with the co-registered MR prior image for the 1/4 and 1/16 low-dose datasets. Compared to DDPM-PET, DDPM-PETMR had higher image resolution, which demonstrated the benefit of including the additional MR prior image as the network input. It was observed that the DDPM-MR method could generate results with the highest resolution and the lowest noise. However, as shown in the error images, large bias existed at some regions, e.g., the caudate and putamen regions. Based on the additional PET data-consistency constraint, results of the DDPM-MR-PETCon method shared more similarity with the normal-dose results. Compared to Unet, DDPM-MR-PETCon had a higher image contrast. Fig. 6 showed the PSNR and SSIM values of different methods based on 20 test datasets. These global quantification values showed that DDPM-PETMR had the best performance and was more robust to the noise levels during testing, followed by DDPM-MR-PETCon. The regional PSNR plot at 14 cortical regions for different methods was presented at Fig. 7. Fig. 8 further showed the surface plots of the left hemisphere regarding the PET relative error for different methods. The regional and the surface results both showed that DDPM-MR-PETCon had the lowest error and was also more robust to the noise levels during testing, followed by DDPM-PETMR.

Figure 5:

Three views of one [¹⁸F]MK-6240 test dataset processed using different methods along with the MR prior image (1st column), the 1/4 low-dose image (2nd column) and the normal-dose image (last column). For each view, the second row shows the difference images of difference methods when compared with the normal-dose image.

Figure 6:

The PSNR (left column) and SSIM (right column) values using different methods calculated based on 20 [¹⁸F]MK-6240 test datasets for 1/4 low-dose (top row) and 1/16 low-dose datasets. ***, **, *, ns, located at the top of the bar plot represents p-value< 0.001, p-value< 0.01, p-value< 0.05, and p-value≥ 0.05, respectively.

Figure 7:

PSNR values of the 14 cortical regions for different methods calculated based on 20 [¹⁸F]MK-6240 test datasets. The top figure is for the case of 1/4 low-dose datasets and the bottom figure is for the case of 1/16 low-dose datasets.

Figure 8:

Surface maps of the left hemisphere showing mean PET relative errors for different methods calculated based on 20 [¹⁸F]MK-6240 test datasets regarding the (a) 1/4 low-dose datasets and (b) 1/16 low-dose datasets. For (a), the color-bar range is [0 0.13]. For (b), the color-bar range is [0 0.18].

Fig. 9 shows the uncertainty maps of the DDPM-MR, DDPM-PET, DDPM-PETMR and DDPM-MR-PETCon methods based on 20 realizations from the same dataset as in Fig. 5. Comparing the results of DDPM-PET and DDPM-PETMR, we observed that adding additional MR prior image could reduce the uncertainty. Comparing the results of DDPM-MR and DDPM-MR-PETCon, we observed that adding PET data-consistency constraint could further reduce the uncertainty. The uncertainty values of DDPM-MR-PETCon were smaller than that of DDPM-PETMR, which showed that including the PET image during the refinement steps could further reduce the uncertainty compared to supplying it as the network input. We also calculated the correlation between the MSE and the generated predictive uncertainty for different brain regions, which was 0.6048, 0.947, 0.9209, and 0.9246 for DDPM-MR, DDPM-PET, DDPM-PETMR, and DDPM-MR-PETCons respectively. This showed that the DDPM generated results with PET info had high correlation with the MSE, and the correlation of DDPM-MR with MSE was low.

Figure 9:

Three views of the uncertainty maps of one [¹⁸F]MK-6240 test dataset calculated from 20 realizations for different DDPM-based methods.

Discussion

In this work we developed DDPM-based approaches for PET image denoising and evaluated their performance on datasets from two different tracers. Quantitative results showed that the DDPM-based frameworks with PET information included had better performance than the NLM, Unet, and GAN-based denoising methods. One benefit of the DDPM-based framework compared to Unet and GAN is the ability to calculate the uncertainty map by generating multiple realizations, which has the potential to be used in progression tracking and longitudinal studies.

Regarding different DDPM-based methods, by comparing DDPM-PET and DDPM-PETMR, our evaluations showed that further adding the MR prior images as the additional network input helped improve the model’s performance both globally (as shown in Fig. 6) and regionally (as shown in Figs. 7 and 8). It also further reduced the uncertainty values as shown in Fig. 9. The results of DDPM-MR presented in Figs. 5 and 6 showed that only replying on the MR prior image was difficult to synthesize a pseudo-PET image with high accuracy as large bias existed in the results due to the high dynamic range of PET uptakes at different regions. Further adding PET information as a data-consistency constraint in the inference step helped improve the results as shown by the performance of DDPM-MR-PETCon.

Fig. 6 showed that DDPM-PETMR had better performance than DDPM-MR-PETCon regarding global quantification. Figs. 7 and 8 revealed that DDPM-MR-PETCon was better than DDPM-PETMR with respect to local quantification. One advantage of DDPM-MR-PETCon compared to DDPM-PETMR and Unet was that it was more robust regarding difference PET image noise levels as demonstrated in Figs. 6, 7 and 8. This was because during the training phase of the score function ${\mathit{ϵ}}_{\mathit{\theta }}$, PET information was not included. Another advantage of DDPM-MR-PETCon was that it could be further extended to PET image reconstruction. This could be achieved by changing the Gaussian distribution assumption in equation (9) to Poisson distribution with PET sinograms included [46]. Considering the performance, the invariability to nosie levels, and the potential extension to PET image reconstruction, the DDPM-MR-PETCon framework was preferable over DDPM-PETMR.

For the two experiments with [¹⁸F]FDG and [¹⁸F]MK-6240, the ground truth utilized in the quantification was based on the normal-dose datasets, whose quality was not good enough and this was one limitation of the quantification. In our work, a 2D network was used in the DDPM-based methods and the neighboring two axial slices were supplied as the network input to reduce axial artifacts. We also tested the 3D network and the training was 4 times slower than the 2D network, which was not realistic based on our current computing resources. Additionally, the inference step took around 50 mins for one 3D dataset. Compared to CNN and GAN-based methods, though the diffusion models could be adapted to PET images of different noise levels and also provide additional predictive uncertainty map, the high computational cost during inference was one issue of the diffusion models. A lot of on-going works are trying to reduce the inference time, e.g., finding a better initialization instead of starting from the random noise [47]. Further extending the framework to enable efficient 3D network training and reducing the inference time while preserving the sampling quality were our future research directions. In our current study, the experiment based on [¹⁸F]MK-6240 datasets was based on 20 test datasets due to the limitation of datasets available for the experiment, which limited its statistical power. Additionally, we believe results of the human reader studies will be the best metric to evaluate deep learning-based image enhancement methods. Further evaluating the performance of the DDPM-based methods on more clinical datasets of different tracers and performing human reader studies are also our future research directions.

Conclusion

In this work, DDPM-based PET imaging denoising frameworks were proposed and evaluated based on [¹⁸F]FDG and [¹⁸F]MK-6240 datasets. Results showed that DDPM-based frameworks including PET information had better performance than the nonlocal mean and the Unet-based denoising methods. Adding additional MR prior in the model achieved better performance and further reduce the uncertainty during image denoising. Future work will focus on extending the framework to 3D network, further reducing the inference time, and performing more evaluations based on different tracers.

Figure 2:

Diagram of the datasets used in the training/validation/test scenerios for the two experiments.