Synthetic CT generation from non-attenuation corrected PET images for whole-body PET imaging

Abstract

Attenuation correction (AC) of PET/MRI faces challenges including inter-scan motion, image artifacts such as truncation and distortion, and erroneous transformation of structural voxel-intensities to PET mu-map values. We propose a deep-learning-based method to derive synthetic CT (sCT) images from non-attenuation corrected PET (NAC PET) images for AC on whole-body PET/MRI imaging.

A 3D cycle-consistent generative adversarial networks (CycleGAN) framework was employed to synthesize CT images from NAC PET. The method learns a transformation that minimizes the difference between sCT, generated from NAC PET, and true CT. It also learns an inverse transformation such that cycle NAC PET image generated from the sCT is close to true NAC PET image. A self-attention strategy was also utilized to identify the most informative component and mitigate the disturbance of noise. We conducted a retrospective study on a total of 119 sets of whole-body PET/CT, with 80 sets for training and 39 sets for testing and evaluation.

The whole-body sCT images generated with proposed method demonstrate great resemblance to true CT images, and show good contrast on soft tissue, lung and bony tissues. The mean absolute error (MAE) of sCT over true CT is less than 110 HU. Using sCT for whole-body PET AC, the mean error of PET quantification is less than 1% and normalized mean square error (NMSE) is less than 1.4%. Average normalized cross correlation on whole body is close to one, and PSNR is larger than 42 dB.

We proposed a deep learning-based approach to generate sCT from whole-body NAC PET for PET AC. sCT generated with proposed method shows great similarity to true CT images both qualitatively and quantitatively, and demonstrates great potential for whole-body PET AC in the absence of structural information.

1. Introduction

The emerging hybrid system, PET/MRI has attracted enormous attention in both research and clinical applications for the last decades (Quick 2014, Vandenberghe and Marsden 2015). MRI produces no ionization radiation, resulting in significantly reduced radiation dose comparing to PET/CT. MRI also provides superior soft tissue contrast as well as functional information such as perfusion and diffusion. However, predicting the attenuation correction (AC) factors from MR images is challenging. MR signal is related to tissue’s proton density and magnetic relaxation properties, which is not correlated to electron density required for AC. There is no one-to-one mapping to convert MR signals to linear attenuation coefficients. A number of studies have been performed to develop accurate AC map from MR images. The most straightforward MR-based AC (MRAC) is to segment MR image into different tissue types, such as air, lung, tissue, fat and bone, and assign the corresponding linear attenuation coefficients at 511 keV (Zaidi et al 2003, Hofmann et al 2009, Fei et al 2012)]. Segmenting lung and cortical bone are particularly difficult because the two only produce weak signals with conventional MR sequences. Though ultrashort echo time (UTE) pulse sequences has been investigated for bone visualization and segmentation, its performance is limited by high level of noise and image artefacts (Catana et al 2010, Keereman et al 2010). Moreover, due to considerable long acquisition time, the application of UTE MR sequence is usually limited to brain imaging or small field-of-view. Atlas–based techniques provide an alternative approach to the MRAC problem. This technique utilizes the co-registration of MR image and CT templates to predict a pseudo-CT or sCT and obtain an attenuation map (Hofmann et al 2008, 2011, Malone et al 2011). The merit of atlas–based AC methods is that it predicts attenuation maps on a continuous scale, which outperforms segmentation-based AC methods in some applications. However, registration is computational costly, especially when multiple registrations are required. Moreover, the accuracy of this technique highly depends on the registration accuracy, while accurate registration is not always guaranteed due to organ morphology and variability across patients.

Besides the issues mentioned above, both of segmentation-based and atlas-based methods face common challenges, such as image artefacts and lengthy acquisition times. Truncation happens when patient body falls out of field of view, and usually results in imaging distortion and missed tissue contours. Truncation artefacts may be less of an issue for brain imaging, but is critical in whole body MRI due to the unavoidable extended image volumes (Hofmann et al 2011). It is reported that arm truncation artefacts could result in standardized uptake value (SUV) estimation errors over the arm in the range of 16%–57% (Schramm et al 2012). Even accepting the imperfections, one may find that MR images are not always available due to the prolonged scanning time of PET/MRI on whole body imaging. Compared to brain imaging, PET/MR is quite limited for whole-body applications, majorly due to the considerably increased acquisition time. For conventional PET/CT, the PET duration is the limiting factor, which takes 2–3 min per bed position while whole-body PET/MRI usually takes 5–10 min or even longer for each bed position primarily due to the lengthy MR sequences (Lindemann et al 2018). Though the acquisition time can be reduced by eliminating some MR sequences while keeping acceptable AC accuracy, sequences, such as T1-weighted Dixon-type sequences (Martinez-Möller et al 2009, Lindemann et al 2018) and UTE sequences (Robson and Bydder 2006, Keereman et al 2010, Berker et al 2012), have to be performed for the sole purpose of AC. Lengthening acquisition time increases patient discomfort and patient motion, and reduces patient throughput. Therefore, it is desirable to find alternative AC methods without the help of MR images.

The development of deep learning has demonstrated tremendous potential in computer vision as well as medical imaging. Deep learning could help to generate sCT from MR images to predict AC map (Spuhler et al 2018, Yang et al 2019). Different from those approaches, we want to make full use of existing NAC PET, and develop a method that make use of NAC PET to synthesize CT images for AC. This paper is organized as follows. Section 2 presents how we structured a cycle-generative-adversarial-network (CycleGAN) to establish the transformation between NAC PET and CT. In section 3, we described the validation studies. We then discussed the limitations and potentials of the proposed methods in section 4, ad concluded in section 5.

2. Methods

2.1. CT synthesis from NAC PET

We propose a new AC method by using a sCT-aided strategy. Figure 1 outlines the schematic flow chart of this CT synthesis process. The method consists of two major steps: the training step and the synthesis step. In the training step, CT images were used as the learning-based target of corresponding NAC PET images. A 3D CycleGAN network (Harms et al 2019) architecture was employed to learn the mapping between CT and NAC PET. The CycleGAN architecture structures the transformation to be a closed loop by introducing both a targeted mapping (NAC PET to CT) and an inverse mapping (CT to NAC PET). The input patch size for the CycleGAN architecture was set to 64 × 64 × 64. Each patch was extracted from NAC PET and CT images by sliding the window with overlap size of 18 × 18 × 18 to its neighbouring patches.

Figure 1.

The schematic flow diagram of the proposed method. The upper part shows the training procedure, and the lower part shows the CT synthesis procedure.

The generator architecture we used is a self-attention U-Net, which was structured with encoding path, decoding path and long skip connections. As shown in the generator architecture of figure 1 (upper part), after two convolution layers with max-pooling operators (which is called the encoding path) to reduce the feature map sizes, the feature map went through nine dense blocks and then two deconvolution layers and a tanh layer (which is called the decoding path) to perform end-to-end mapping, i.e. equal-sized input and output. The tanh layer works as a nonlinear activation function and makes it easy for the model to generalize or adapt to a variety of data and to map structures with similar CT intensities to different structures on NAC PET, such as heart and bladder. The dense block is implemented by six convolution layers. A first layer is applied to the input to create k feature maps, which are concatenated to the input. A second layer is then applied to create another k feature maps, which are again concatenated to the previous feature maps. The operation is repeated five times then the output of these layers goes through the last layer to shorten the feature maps to k. In the long skip connection, three attention gates (AGs) were used to focus on identifying the most informative features from the feature maps extracted from the encoding path. In order to enforce a closed loop, two generators and two discriminator were utilized.

The discriminator architecture of CycleGAN was used as a typical fully convolutional network (FCN), where the input patch was either the output of the generators, the true CT or NAC PET. The input patch went through three convolution layers followed by pooling layer to reduce the output size. The sigmoid operation was used to polarize this output.

During the synthesis stage (lower part of figure 1), the patches were extracted from the NAC PET image and fed into the trained generator, which maps NAC PET patches to sCT patches. Finally the sCT used for PET AC was obtained by patch fusion.

2.2. Self-attention

In order to well represent the image patch in our generators, a long skip connection was used to concatenate the high-frequency information from the encoding path to low-frequency information from the decoding path to integrate multi-level features. However, high-frequency information often include irrelevant and uninformative components since NAC PET images contain a large amount of noise. These components may prevent the model from obtaining an accurate mapping from NAC PET to CT. In order to overcome this challenge, we used a self-attention strategy on the feature maps generated from the encoding path to identify the most informative component and mitigate the disturbance of noise. Previous work demonstrated that by integrating attention gates (AGs) into a standard U-Net model, the most relevant semantic contextual information can be captured without enlarging the receptive field, which is highly beneficial for organ localization (Mishra et al 2018). As for our study, clear identification of organ outlines is important to estimate the intensity values inside each structure.

As is shown in AG architecture of figure 1, the feature maps extracted from the coarse scale were used in gating to disambiguate irrelevant and noisy responses in skip connections. This is performed immediately prior to the concatenation operation to merge only relevant activations. By using AGs, the most salient features from the encoding path are highlighted and are passed through the long skip connection.

2.3. Loss function

As described above, the network relies on continuous improvement of a generator network and a discriminator network. The accuracy of both networks is directly dependent on the design of their corresponding loss functions. The original cycle-GAN study optimized the networks in tandem according to a two-part loss function, consisting of an adversarial loss and a cycle consistency loss. The adversarial loss function, which relies on the output of the discriminators, applies to both the NAC-to-CT generator (G_NAC−CT) and the CT-to-NAC generator (G_NAC−CT) and the CT-to-NAC generator (G_CT−NAC), but here we present only formulation for G_NAC−CT for clarity. The adversarial loss function in this work is defined by

$$L_{\text{adv}}\left(G_{\text{NAC}-\text{CT}},D_{\text{CT}},I_{\text{NAC}},I_{\text{CT}}\right)=\text{SCE}\left(D_{\text{CT}}\left(G_{\text{NAC}-\text{CT}}\left(I_{\text{NAC}}\right)\right),1\right)$$ (1)

where I_NAc is the NAC image and G_NAC−CT(I_NAC) is the output of the NAC-to-CT generator, i.e. the sCT image. D_CT is the CT discriminator which is designed to return a binary value indicating whether a pixel region is real (from a CT) or fake (from a sCT), so this measures the number of incorrectly generated pixels in the sCT image. The function SCE (·, 1) is the sigmoid cross entropy (SCE) between the logits obtained by discriminator and a unit as label.

The cycle consistency loss function in this work consists of a compound loss function. In the original cycle-GAN paper, this loss function constrained the inverse transformation, which in this work would be cycle image to original image, for both generators. In addition to these constraints, we minimize the distance between the synthetic image and the original image, which we call the synthetic consistency, to directly enforce them to have same intensity distribution. The first component of the loss function is the mean absolute loss (MAL):

$$\text{MAL}\left(G_{\text{NAC}-\text{CT}},G_{\text{CT}-\text{NAC}}\right)=\frac{1}{n\left(I_{\text{CT}}\right)}\left[{\lambda }_{\text{loss }}^{\text{cycle }}\left[\text{MAE}\left(G_{\text{CT}-\text{NAC}}\left(G_{\text{NAC}-\text{CT}}\left(I_{\text{NAC}}\right)\right),I_{\text{NAC}}\right)+\text{MAE}\left(G_{\text{sCT}-\text{CT}}\left[G_{\text{CT}-\text{sCT}}\left(I_{\text{CT}}\right)\right],I_{\text{CT}}\right)\right]+{\lambda }_{\text{loss }}^{\text{syn }}\left[MAE\left({\text{G}}_{\text{CT}-\text{sCT}}\left({\text{I}}_{\text{CT}}\right)-{\text{I}}_{\text{sCT}}\right)+MAE\left({\text{G}}_{\text{sCT}-\text{CT}}\left({\text{I}}_{\text{sCT}}\right)-{\text{I}}_{\text{CT}}\right)\right]\right]$$ (2)

where n(·) is the total number of pixels in the image, ${\lambda }_{\text{loss }}^{\text{cycle }}$ and ${\lambda }_{\text{loss }}^{\text{syn }}$ are parameters which control the cycle consistency and synthetic consistency, respectively. The symbol MAE denotes the mean absolute error (MAE) between two images.

The second component of the loss function is the gradient magnitude distance (GMD). Between any two images, the GMD is defined as:

$$\text{GMD}(Z,Y)=\sum _{i\text{,}j\text{,}k}\left\{{\left(\left|Z_{i,j,k}-Z_{i-1,j,k}\right|-\left|Y_{i,j,k}-Y_{i-1,j,k}\right|\right)}^2+{\left(\left|Z_{i,j,k}-Z_{i,j-1,k}\right|-\left|Y_{i,j,k}-Y_{i,j-1,k}\right|\right)}^2+{\left(\left|Z_{i,j,k}-Z_{i,j,k-1}\right|-\left|Y_{i,j,k}-Y_{i,j,k-1}\right|\right)}^2)\right\}$$ (3)

where Z and Y are any two images, and i, j, and k represent pixels in x, y, and z. To provide additional clarity in this work, we define a gradient magnitude loss (GML), which is a function of the generator networks

$$\text{GML}\left(G_{\text{NAC}-\text{CT}},G_{\text{CT}-\text{NAC}}\right)={\lambda }_{\text{dist }}^{\text{cycle }}\left[\text{GMD}\left(G_{\text{CT}-\text{NAC}}\left(G_{\text{NAC}-\text{CT}}\left(I_{\text{NAC}}\right)\right),I_{\text{NAC}}\right)+\text{GMD}\left(G_{\text{NAC}-\text{CT}}\left(G_{\text{CT}-\text{NAC}}\left(I_{\text{CT}}\right)\right),I_{\text{CT}}\right)\right]+{\lambda }_{\text{dist }}^{\text{syn}}\left[\text{GMD}\left(G_{\text{CT}-\text{NAC}}\left(I_{\text{CT}}\right),I_{\text{sCT}}\right)+\text{GMD}\left(G_{\text{NAC}-\text{CT}}\left(I_{\text{NAC}}\right),I_{\text{CT}}\right)\right].$$ (4)

The total cycle- and synthetic-consistency loss function is then optimized according to

$$L_{\text{cyc}}\left(G_{\text{NAC}-\text{CT}},G_{\text{CT}-\text{NAC}}\right)={\lambda }_{\text{MAL}}\text{MAL}\left(G_{\text{NAC}-\text{CT}},G_{\text{CT}-\text{NAC}}\right)+{\lambda }_{\text{GML}}\text{GML}\left(G_{\text{NAC}-\text{CT}},G_{\text{CT}-\text{NAC}}\right)$$ (5)

where λ_MPL and λ_GML are tuning parameters that balance the MPL and GML functions. The global generator loss function can then be written

$$\left(G_{\text{CT}-\text{NAC}},G_{\text{NAC}-\text{CT}}\right)=\underset{G_{\text{CT}}{}_{-\text{NAC}},G_{\text{NAC}-\text{CT}}}{\text{arg}\text{min}}\left\{{\lambda }_{\text{adv}}\left[L_{\text{adv}}\left(G_{\text{NAC}-\text{CT}},D_{\text{CT}},I_{\text{NAC}},I_{\text{CT}}\right)+L_{\text{adv}}\left(G_{\text{CT}-\text{NAC}},D_{\text{NAC}},I_{\text{CT}},I_{\text{NAC}}\right)\right]+L_{\text{cyc}}\left(G_{\text{NAC}-\text{CT}},G_{\text{CT}-\text{NAC}}\right)\right\}$$ (6)

where λ_adv is a regularization parameter that control the weights of the adversarial loss. The discriminators are optimized in tandem with the generators according to

$$\left(D_{\text{CT}},D_{\text{NAC}}\right)=\underset{D_{\text{CT}},D_{\text{NAC}}}{\text{argmin}}\left\{\text{SCE}\left(D_{\text{CT}}\left(G_{\text{NAC}-\text{CT}}\left(I_{\text{NAC}}\right)\right),0\right)+\text{SCE}\left(D_{\text{CT}}\left(I_{\text{CT}}\right),1\right)+\text{SCE}\left(D_{\text{NAC}}\left(G_{\text{CT}-\text{NAC}}\left(I_{\text{CT}}\right)\right),0\right)+\text{SCE}\left(D_{\text{NAC}}\left(I_{\text{NAC}}\right),1\right)\right\}.$$ (7)

The hyperparameter values from equations (6) and (7) are listed as follows: λ_adv = 1, ${\lambda }_{\text{loss }}^{\text{cycle }}=10$, ${\lambda }_{\text{loss }}^{\text{syn }}=1$, λ_MPL = 1, λ_GML = 1. Since it will be more difficult to generate an accurate cycle image from a real image due to the two transformations introduced, the value of ${\lambda }_{\text{loss }}^{\text{cycle }}$ should be larger than ${\lambda }_{\text{loss }}^{\text{syn }}$.

2.4. Validation and evaluations

A retrospective sample of 119 whole-body FDG oncology patient datasets were collected for model training (80 sets) and evaluation (39 sets). All PET data were acquired with Discovery 690 PET/CT scanner (General Electric) using a body mass index (BMI)-based administration protocol of either 370 MBq (BMI < 30) or 444 MBq (BMI ⩾ 30) followed by a 60 min uptake period. Emission data were collected based on BMI for 2.5 min (BMI ⩾ 25), 2min (18.5 < BMI < 25) or 1.5 min (BMI ⩽ 18.5). Images were reconstructed with a 3D ordered-subset expectation maximization (OSEM) algorithm (3 iterations, 24 subsets) with time-of-flight and corrections for attenuation, scatter, randoms, normalization and deadtime (Iatrou et al 2004). A post-image Gaussian filter of 6.4 mm was applied to all images. The reconstruction matrix was 192×192 with a pixel size of 3.65 × 3.65 × 3.27 mm³

The accuracy of sCT is quantified as MAE, which is calculated as,

$$\text{MAE}=\frac{1}{N}\sum _{i\in V}\frac{\left|I_{\text{CT}}(i)-I_{\text{sCT}}(i)\right|}{I_{\text{CT}}(i)}$$ (8)

where I_CT and I_sCT are true CT and sCT intensities, N is the number of voxels, and i represents index of each voxel on CT and sCT images. We also calculated the normalized cross correlations (NCC) to quantify the structure similarity between CT and sCT. NCC is defined as,

$${\text{NCC}}_{\text{CT}}=\frac{{\sum }_{i\in V}\frac{1}{N}\cdot \left(I_{\text{CT}}(i)-\text{mean}\left(I_{\text{CT}}\right)\right)\left(I_{\text{sCT}}(i)-\text{mean}\left(I_{\text{sCT}}\right)\right)}{\text{std}\left(I_{\text{CT}}\right)\cdot \text{std}\left(I_{\text{sCT}}\right)}$$ (9)

where mean (·) and std (·) calculates the mean and standard deviation (STD) intensity inside the ROI volume.

The performance of proposed method on PET quantification is quantified with mean error (ME), normalized mean square error (NMSE) and peak signal to noise ratio (PSNR) calculated inside those contoured volumes,

$$\text{ME}=\frac{1}{N}\sum _{i\in V}\frac{I(i)-I^{\prime }(i)}{I(i)}$$ (10)

$$\text{NMSE}=\frac{{\Vert I-I^{\prime }\Vert }_2^2}{{\Vert I\Vert }_2^2}$$ (11)

$$\text{PSNR}=10{\text{log}}_{10}\left(\frac{N\cdot \text{max}{\left(I^{\prime }(i)\right)}^2}{{\Vert I-I^{\prime }\Vert }_2^2}\right).$$ (12)

ME and NMSE are averaged over all the voxels, i, inside the contoured organs or whole-body volume, V. The organs and whole body were delineated on CT images, where N is the total number of PET voxels inside the volumes. I and I′ are PET intensities after AC with true CT and sCT respectively. max (·) is the max intensity inside the delineated volume. The NCC is calculated to quantify the intensity and structure similarity between AC PET (AC performed with true CT) and sCT AC PET (AC performed with sCT).

$${\text{NCC}}_{\text{PET}}=\frac{{\sum }_{i\in V}\frac{1}{N}\cdot (I(i)-\text{mean}(I))\left(I^{\prime }(i)-\text{mean}\left(I^{\prime }\right)\right)}{\text{std}(I)\cdot \text{std}\left(I^{\prime }\right)}.$$ (13)

We randomly selected 80 sets out of totally 119 sets of whole-body PET images for model training, and then applied the well-trained model on the remaining 39 sets of images to evaluate the reliability of the proposed method. Regions of interest (ROI) were delineated over brain, lung, heart, bilateral kidneys, liver, whole-body and lesions, as indicated in the patient’s clinical report.

3. Results

Figure 2 shows the side-by-side comparison of sCT and true CT on one patient. The sCT images generated with the proposed method demonstrate great resemblance to true CT images. sCT images not only recover lung, soft tissue and bony structures such as humerus and femur, but also demonstrate good contrast on fat and soft tissues around extremities, as indicated by the red arrows in figure 2. The proposed method obtained MAE of 108.9 ± 19.1 HU, NCC of 0.854 ± 0.031, and PSNR larger than 22.3 ± 1.27 dB on the 39 testing datasets.

Figure 2.

Comparison of true CT and sCT images on a female received breast implants. From left to right, the four columns are NAC PET, true CT, sCT, and difference images between CT and sCT (unit: HU). The top two rows show two transverse slices, and the last row shows one coronal slice.

The qualitative sCT AC PET reconstruction results on one representative patient are illustrated in figure 3. sCT images generated with the proposed method show anatomical structures that are close to those on true CT images. Lung and the bone of extremities can be clearly visualized on sCT. After sCT AC, the generated PET images are almost identical to the PET images corrected with true CT (figures 3(c) and (d)). Figure 4 shows the profile comparison and joint histogram of AC PET and sCT AC PET. The PET profile generated with sCT AC matches the true AC PET profile very well. The joint histogram shows a distribution that is close to the line of identity, which indicates great correlation between AC PET and sCT AC PET. Figure 5 shows sCT and sCT PET images on another patient. Similarly, sCT shows great contrasts on lung, soft tissue and bone structures. The PET images after sCT AC demonstrate excellent matching to the groundtruth AC PET images. The quantitative results are shown in figure 6, where the image profile is greatly enhanced.

Figure 3.

Qualitative results on Patient 1. Images are (a) true CT, (b) sCT, (c) AC PET and (d) sCT AC PET. The dashed line on (c) indicates the position of a sagittal cranial-caudal profile displayed in figure 4.

Figure 4.

Quantitative results on Patient 1. Images are PET image profiles (left) and joint histograms (right) of AC PET and sCT AC PET. The red line in the right figure is the line of identity.

Figure 5.

Qualitative results on Patient 2, whom is a female received breast implants. Images are (a) true CT, (b) sCT, (c) AC PET and (d) sCT AC PET. The dashed line on (c) indicates the position of a sagittal cranial-caudal profile displayed in figure 6.

Figure 6.

Quantitative results on Patient 2. Images PET image profiles (left) and joint histograms (right) of AC PET and sCT AC PET.

The quantitative AC results are listed in table 1. The ME on the whole-body volume obtained with the proposed method is 0.12% ± 2.98%, and NMSE is 1.36% ± 1.08%. NCC is close to one, indicating excellent correlation between the PET images obtained with the proposed method and the groundtruth PET images. The ME and NMSE calculated on brain, heart, left kidney, right kidney liver and lesion are from −1.06% to 3.57%, and from 0.43% to 1.80%, respectively. The quantification on lung volume is challenging due to tissue inhomogeneity, and the proposed method was able to obtain average ME and NMSE of 10.72% and 6.50%.

Table 1.

AC performances on 39 sets of whole-body PET images. Data are reported as mean ± STD.

ROI	ME	NMSE	NCC	PSNR (dB)
Brain	−1.05% ± 3.94%	0.43% ± 0.0.78%	0.981 ± 0.014	31.6 ± 4.1
Lung	10.72% ± 7.71%	6.50% ± 4.17%	0.854 ± 0.067	25.5 ± 4.9
Heart	−0.67% ± 8.4%	1.40% ± 1.22%	0.964 ± 0.034	27.6 ± 3.9
Lt kidney	2.45% ± 9.13%	1.41% ± 3.12%	0.975 ± 0.069	32.3 ± 8.2
Rt kidney	3.57% ± 10.36%	1.50% ± 2.88%	0.983 ± 0.059	31.9 ± 7.9
Liver	−1.06% ± 7.047%	1.80% ± 2.29%	0.856 ± 0.136	25.5 ± 5.2
Lesion	1.07% ± 9.01%	0.91% ± 1.31%	0.972 ± 0.024	27.9 ± 6.3
Whole body	0.12% ± 2.98%	1.36% ± 1.08%	0.988 ± 0.009	42.6 ± 3.4

4. Discussion

We proposed a deep learning-based method to synthesize CT images with only NAC PET images for AC of whole body PET images. We trained and evaluated the proposed model with 119 sets of whole-body PET images. The evaluation demonstrated excellent quantitative accuracy of sCT as well as sCT AC PET. The proposed Cycle GAN model uses only NAC PET images for AC, which has great potential to avoid lengthy MR acquisition used only for AC map estimation.

Synthesizing CT images with NAC PET images are inherently difficult due to the considerably low spatial resolution and limited anatomical information provided by NAC PET images. However, the proposed method demonstrates competitive results when compared to MRAC methods in literature. Hofmann et al proposed a MRAC method combined atlas and pattern-recognition, and applied this method on both brain and whole body imaging (Hofmann et al 2008, 2011). This AC method obtained a mean PET quantification error of 3.2% ± 2.5% SUV on brain imaging and 7.7% ± 8.4% SUV on whole body PET images with 14.0% ± 11.4% SUV in thorax region. Paulus et al implemented a model-based bone segmentation method on Dixon sequence, and obtained SUV quantification errors of 2.7% and 5.2% on normal soft tissue and soft tissue lesions (Paulus et al 2015). The proposed method obtains whole body ME of less than 1% and NMSE less than 2% within common organs, and around 1% quantification errors on lesions.

The possibility of using NAC PET to perform AC was investigated in literature. Liu et al trained a deep convolutional encoder-decoder network for brain CT synthesis from NAC PET images (Liu et al 2018). The network generated pseudo-CT with MAE of 111HU, resulted with average errors of less than 1% on the whole brain region and these results are well matched to our findings. The MAE on sCT with the proposed CycleGAN is less than 110 HU, and the ME on PET quantification is less than 1%.

The sCT shows more blurring patterns around lung, as shown in figure 2. This is due to respiratory motion over the duration of the bed acquisitions covering the thorax. Deriving sCT directly from NAC PET, the proposed method tends to propagate the respiratory motion to sCT. This suggests that the proposed CT synthesis scheme could possibly mitigate the mis-registration or mis-alignment between NAC PET and CT, since sCT is directly generated from NAC PET. This improvement was also studied in (Liu et al 2018). In the future, we will validate HU accuracy and corresponding AC accuracy with motion-averaged CT and PET scans.

The OSME iterations and FWHM of post-smoothing kernels majorly change the noise level the NAC PET images. In order to evaluate the effect of different noise level on our proposed method, we conducted study to test the robustness of the proposed method under different noise levels. Specifically, we added different level of noise (Gaussian distribution with different standard deviation with different ratio 5%, 10%, 20% and 30%) into the NAC PETs, generated the sCTs using our original trained model and then performed some comparisons. The MAE on sCT is 108.4 ± 9.7 HU, 115.7 ± 11.1 HU, 115.7 ± 11.1 HU, 116.7 ± 11.2 HU and 117.3 ± 11.3 HU on no noise, 5% noise, 10% noise, 20% noise and 30% noise images. The corresponding ME on whole-body PET is 0.12% ± 3.01%, 0.20% ± 3.35%, 0.21% ± 3.58%, 0.21% ± 4.53% and 0.22% ± 4.68% respectively. Based on these results, the proposed method is robust to different noise.

Although lung and the bone of extremities can be clearly visualized on sCT, the sacrum and coccyx are not predicted precisely in figures 3(b) and 5(b). The reason may be that the PET artifacts caused by extreme activity contrast in the pelvic region would be influential on sCT. The other reason may be the intensity of bladder is much larger than the rest region in NAC PET image. Thus, for the batch normalization, the intensity of other contrast region, such as the bone in the torso, is reduced. This would affect the accuracy of bone in the torso. In our future work, one solution is to use a threshold to clip the intensity value of bladder region in NAC PET image. Another solution is to train a specific model only for pelvic site to solve this problem.

In table 1, the solid organs are almost negative ME values (overestimation) except for both kidneys and lesion. The reason may be caused by the large variance of intensity values for whole body NAC PET images. As we trained the model on patches and used a uniformed normalization $(x-\overline{x})/\left(x_{99}-x_5\right)$, where x is the mean intensity, x₉₉ and x₅ are the 99th and 5th percentile intensities. Since x₉₉ of NAC PET is higher in brain, heart and bladder, the uniform normalization would reduce the contrast of other organs in NAC PET image. The low contrast region would be a challenge for proposed model to predict the accurate CT intensity value. In future, we will work on region-based networks to predict the sCT for different sites.

The proposed method synthesized CT images from NAC PET images, which was then used for PET AC. Though designed for the propose of AC in PET/MRI imaging, the proposed method was implemented and evaluated with PET/CT data in this study. The quantification performances could be affected by MR coil attenuation and differences in protocol and machine setting on PET/MRI data. With appropriate scanning protocol and machine calibration, their impact on the quantification accuracy could be minimized. In the future, we will implement the proposed method with PET/MRI data for further validation.

The proposed method was designed to serve AC for PET/MR system, and could be applied to PET/CT studies. Though CT-based AC methods is the prevailing method, it has it has limitations (Berker and Li 2016, Mehranian et al 2016). Misalignment between CT and PET due to patient motion could result in errors when calculating AC factors (Mehranian et al 2016). Moreover, the CT imaging dose is undesirable in some applications, such as paediatric imaging due to the increased radiosensitivty of these patients. Depending imaging parameters and scanning length, a whole body PET/CT scan could result in 13–32mSv total effective dose (Huang et al 2009). It results in more imaging dose when serial PET/CT scans are required for restaging or assessing treatment response. The proposed method estimates AC maps with only NAC PET images, and has the potential to obtain accurate AC without the need of extra CT doses.

5. Conclusion

In this work, we proposed a deep learning-based method to synthesize CT images with only NAC PET images for whole body PET AC. The method demonstrates excellent sCT estimation accuracy and PET quantification accuracy and has great potential to facilitate AC in hybrid systems without the need of additional scans.