Attenuation Correction for Brain PET Imaging Using Deep Neural Network Based on Dixon and ZTE MR Images

Abstract

Positron Emission Tomography (PET) is a functional imaging modality widely used in neuroscience studies. To obtain meaningful quantitative results from PET images, attenuation correction is necessary during image reconstruction. For PET/MR hybrid systems, PET attenuation is challenging as Magnetic Resonance (MR) images do not reflect attenuation coefficients directly. To address this issue, we present deep neural network methods to derive the continuous attenuation coefficients for brain PET imaging from MR images. With only Dixon MR images as the network input, the existing U-net structure was adopted and analysis using forty patient data sets shows it is superior than other Dixon based methods. When both Dixon and zero echo time (ZTE) images are available, we have proposed a modified U-net structure, named GroupU-net, to efficiently make use of both Dixon and ZTE information through group convolution modules when the network goes deeper. Quantitative analysis based on fourteen real patient data sets demonstrates that both network approaches can perform better than the standard methods, and the proposed network structure can further reduce the PET quantification error compared to the U-net structure.

1. Introduction

Positron Emission Tomography (PET) can produce three dimensional images of biochemical processes in the human body by using specific radioactive tracers. It has wide applications in neuroscience studies, such as measurement of metabolism for brain tumor imaging, dopamine neurotransmitter imaging related to addiction, β-amyloid and tau imaging in Alzheimer’s disease, translocator protein (TSPO) imaging related to microglial activation and so on. Due to various physical degradation factors, correction items, such as randoms, scatters, normalization and attenuation correction, should be included in the reconstruction process to obtain meaningful quantitative results. For attenuation correction, information from computed tomography (CT) has been treated as a reference standard to reflect the attenuation coefficients in 511 Kev after a bilinear scaling (Kinahan et al 2003).

Recently, PET/MR systems begin to be adopted in clinics due to MR’s excellent soft tissue contrast and the ability to perform functional imaging. In addition, simultaneously acquired MR images can provide useful information for PET motion compensation (Catana et al 2011) and partial volume correction (Gong et al 2017c). One concern is that the MR signal is not directly reflective of attenuation coefficients, and hard to be used for attenuation correction without approximation. Many methods have been proposed to generate the attenuation map based on T1-weighted, Dixon, ultra-short echo time (UTE) or zero echo time (ZTE) MR images, which can majorly be summarized into four categories. The first category is segmentation based methods. The MR image is segmented into different tissue classes with the corresponding attenuation coefficients assigned to produce the attenuation map (Martinez-Möller et al 2009, Keereman et al 2010, Berker et al 2012, Ladefoged et al 2015, Sekine et al 2016, Leynes et al 2017, Khalifé et al 2017, Yang et al 2017). Another category relies on the atlas generated from prior patients’ CT and MR pairs. Pseudo CT will be created by non-rigidly registering the atlas to patient MR images (Wollenweber et al 2013, Burgos et al 2014, Izquierdo-Garcia et al 2014, Yang et al 2017a). With the availability of time-of-flight (TOF) information, emission based methods have been developed to estimate the activity image and the attenuation map simultaneously without the use of MR information (Defrise et al 2012, Rezaei et al 2012, Li et al 2017), or aided by MR information (Mehranian and Zaidi 2015, Kim et al 2016, Mehranian et al 2017). Finally, there are efforts adopting machine learning based approaches to pseudo CT generation driven by prior MR and CT pairs, such as the random forest (Huynh et al 2016) and neural network methods(Han 2017, Nie et al 2017, Liu et al 2017, Leynes et al 2017b).

Over the past several years, deep neural networks have been widely and successfully applied to computer vision tasks because of the availability of large data sets, advances in optimization algorithms and emerging of effective network structures. Recently, it has been applied to medical imaging, such as image denoising (Wang et al 2016, Kang et al 2016, Chen et al 2017), image reconstruction (Wu et al 2017a, Gong et al 2017b) and end-to-end lesion detection (Wu et al 2017b). Several pioneering works have shown that neural networks can be employed to generate the pseudo CT images from T1-weighted MR images for the brain region, with evaluations on the pseudo CT image quality only (Han 2017, Nie et al 2017). Take one step further, Liu et al (2017) used convolutional auto-encoder (CAE) to generate the CT tissue labels (air, bone, and soft tissue) from T1-weighted MR images and evaluated its performance for PET images. In that work additional CT segmentation is needed and the attenuation coefficients were assigned based on tissue labels, which are not continuous. Recently Leynes et al (2017b) combined ZTE and Dixon images to generate the pseudo CT for the pelvis region using the U-net structure (Ronneberger et al 2015).

In this work, we focus on using neural network based methods to predict the continuous attenuation map specifically for brain PET imaging under two scenarios:

1. When there are only Dixon MR images available, we adopted the U-net structure (Ronneberger et al 2015) to generate the pseudo CT images. Forty patients’ data sets were used in the experiment and cross-validated to evaluate the performance. The segmentation and atlas methods based on Dixon MR images provided by the vendor were used as comparison methods;

2. When both Dixon and ZTE MR images are available, we proposed a new network structure based on group convolution modules to more efficiently combine ZTE and Dixon information. Fourteen patient data sets with both Dixon and ZTE images were employed in the experiments. The ZTE segmentation method provided by the vendor was adopted as the comparison methods.

The main contributions of this paper include (1) using deep neural networks to generate continuous attenuation maps for brain PET imaging;(2) proposing a new network structure to generate the attenuation maps utilizing multiple MR inputs; (3) a comprehensive quantitative comparison with the standard methods.

2. Method

2.1. PET attenuation model

For PET image reconstruction, the measured sinogram data y ∈ ℝ^M×1 can be modeled as a collection of independent Poisson random variables and its mean $\overline{\mathit{y}}\in {\mathrm{ℝ}}^{M\times 1}$ is related to the unknown image x ∈ ℝ^N×1 through an affine transform

$$\overline{\mathit{y}}=\mathit{P}\mathit{x}+\mathit{s}+\mathit{r},$$ (1)

where P ∈ ℝ^M×N is the detection probability matrix, s ∈ ℝ^M×1 is the expectation of scattered events, and r ∈ ℝ^M×1 denotes the expectation of random coincidences. M is the number of lines of response (LOR) and N is the number of pixels in image space. The reconstructed image quality strongly depends on the accuracy of the detection probability matrix P, which can be decomposed to (Qi et al 1998)

$$\mathit{P}=\mathit{N}\mathit{A}\mathit{B}\mathit{G}\mathit{R},$$ (2)

where G ∈ ℝ^M×N is the geometric projection matrix whose element g_i,j denotes the probability of a photon pair produced in voxel j reaching the front faces of detector pair i, R ∈ ℝ^N×N models the image domain blurring effects, B ∈ ℝ^M×M is the sinogram domain blurring matrix (Gong et al 2017a), diagonal matrix N ∈ ℝ^M×M contains the normalization effects, and diagonal matrix A ∈ ℝ^M×M models the attenuation factors. The ith diagonal element of attenuation matrix A is calculated as

$$a_{ii}=e^{-{\displaystyle {\sum }_j{l}_{ij}{\mu }_j}},$$ (3)

where $\mathit{\mu }\in {\mathrm{ℝ}}^{N_{\mu }\times 1}$ is the attenuation map, l_ij denotes the interaction length of LOR i with voxel j. In PET/CT, CT images are used for the attenuation map generation by the bilinear scaling method (Carney et al 2006)

$${\mu }_j=\{\begin{array}{c}9.6{\mathrm{e}}^{-5}\left({\text{HU}}_j+1000\right)\text{if}{\text{HU}}_j<\text{Threshold},\\ a\left({\text{HU}}_j+b\right)\text{if}{\text{HU}}_j>\text{Threshold}.\end{array}$$ (4)

Here HU_j represents the HU units in CT voxel j. a, b and Threshold are values depending on the energy of the CT and are given in Carney et al (2006).

2.2. Pseudo CT generation using deep neural network

The basic module of a convolutional neural network includes a convolution layer and an activation layer. The input and output relationship of the ith module can be denoted as

$${\mathit{y}}_i=f_i\left({\mathit{y}}_{i-1}\right)=g\left({\mathit{w}}_i⊛{\mathit{y}}_{i-1}+{\mathit{b}}_i\right),$$ (5)

where y_i−1 ∈ ℝ^N×N×C is the module input with spatial size N × N and channel size C, y_i ∈ ℝ^N×N×H denotes the module output with spatial size N × N and H channels, w_i ∈ ℝ^{M × M × C × H} is the convolutional filter with kernel width M, b ∈ ℝ^1×H is the bias term, ⊛ stands for the convolution operation, and g represents the non-linear activation function. The rectified linear unit (ReLU) activation function, defined as g(x) = max(x, 0), is employed as the activation function. To stabilize and accelerate the deep network training, batch normalization (Ioffe and Szegedy 2015) is often added after the convolution operation. After stacking L units together, the network output can be calculated as

$${\mathit{y}}_{\text{out}}\left({\mathit{x}}_{\text{input}}\right)=f_L\left(f_{L-1}\left(\dots f_1\left({\mathit{x}}_{\text{input}}\right)\right)\right).$$ (6)

In this work, MR images are treated as the network input and pseudo CT images are output of the network. The network is trained based on prior acquired MR and CT pairs from different patients, with the objective function

$$L=\left|{\text{CT}}_{\text{true}}-{\mathit{y}}_{\text{out}}\left(\text{MR}\right)\right|,$$ (7)

which is the L1 norm of the difference between the ground truth CT image CT_true and the output from the neural network y_out(MR). We have also tried L2 norm and found that L1 norm can produce less blurred structures.

2.2.1. Single input

In many cases, only one MR sequence is available for attenuation correction, either T1-weighted, Dixon or UTE/ZTE. The network implemented for this scenario is based on the U-net structure (Ronneberger et al 2015). The overall network architecture is summarized in Fig. 1. It consists of repetitive applications of 1) 3×3 convolutional layer, 2) batch normalization layer, 3) ReLU layer, 4) convolutional layer with stride 2 for down-sampling, 5) transposed convolutional layer with stride 2 for up-sampling, and 6) mapping layer that concatenates the left-side features to the right-side. The input has nine channels with a spatial size of 144 × 144 and the bottom layer has an spatial size of 9 × 9. The number of features N after the first convolution module is 16. To make full use of the axial information, nine neighboring axial slices were stacked occupying nine input channels to reduce the axial aliasing artifacts. As only Dixon images are utilized as single input, this method is referred as Dixon-Unet.

Figure 1.

The schematic diagram of the U-net architecture. Numbers on top of the module stand for the number of features in the channel. Numbers on the left size of the module indicate the spatial input size. N is the number of features after the first convolution module. For the proposed GroupU-net structure, the convolution module inside the dashed box will be replaced by the group convolution module as indicated in Fig. 2. The group module will only be used when the block input has features ≥ 4N. The number of groups in the group convolution module is set to be N.

2.2.2. Multiple inputs

For current PET/MR scanners, more than one MR sequence can be acquired for attenuation correction. For example, both Dixon and ZTE MR images are available in GE SIGNA scanner. When multiple MR images are included as network input, the number of features N after the first convolution module should be enlarged to digest the additional spatial information. For the U-net structure, the number of trainable parameters increases quadratically with N, and overfitting can be a serious pitfall when increasing the network complexity while not providing enough training pairs. It is shown in previous studies that designing a “wider” network can make more efficient use of model parameters (Szegedy et al 2016, Chollet 2016, Xie et al 2017). To preserve the network capacity while restricting the network complexity, the group convolution module as illustrated in Fig. 2 was adopted to replace the convolution module when the network goes deeper. The group convolution module is similar to the module presented in ResNeXt network structure (Xie et al 2017). Traditionally the convolution kernel considers cross-channel correlations and spatial correlations together. The group convolutional module presented in Fig. 2 first deals with the cross-channel correlation through 1×1 convolution and then handles the spatial correlation in smaller groups. The hypothesis is that when the network goes deeper, the spatial content and the cross-channel correlations can be decoupled (Chollet 2016). In our implementation, N is set to be 19 for the U-net with both Dixon and ZTE as input. For GroupU-net, the number of groups is set to be N and we only use the group convolution module when the input channel size is ≥ 4N. N is set to 32 to match with the number of trainable parameters in U-net (2.7 million). These two methods are labeled as DixonZTE-Unet and DixonZTE-GroupUnet, respectively.

Figure 2.

The schematic diagram of the group convolution module. The ReLU and Batch normalization layers are added after each convolution operation during implementation.

3. Experimental evaluations

3.1. Data sets

The patient study was approved by the Institutional Review Board and all patients signed an informed consent before the examinations. In total forty patients acquired from 2014 to 2016 were used in this study. All patients had whole-body PET/CT, followed by additional PET/MRI scanning without second tracer administration. For both PET/CT and PET/MR, only data acquired in the bed position that includes the head are used in the study. No pathology in the brain was reported for any of the patients. The average patient weight was 73.2241 ± 17.0 kg (range, 39.5-109.8 kg). For PET/MRI, the average scan duration of the whole brain was 224.6 ± 133.7 s (range, 135-900 s). All forty patient data sets have Dixon MR images and fourteen patient data sets with additional acquired ZTE MR images. Thirty seven of the total forty patient data sets had FDG scans. The average administered dose of FDG was 305.2 ± 73.9 MBq (range, 170.2-468.1 MBq). Twelve of the fourteen patients with additional ZTE scans had FDG PET scans.

PET/CT examinations were performed in the GE Discovery PET/CT scanner or the Siemens Biograph HiRez 16 PET/CT scanner. For CT images acquired from the GE Discovery PET/CT scanner, the reconstruction has a axial field of view (FOV) of 700 mm and the matrix size is 512 × 512 with voxel size 2.73 × 2.73 × 3.75mm³. For CT images acquired from the Siemens Biograph HiRez 16 PET/CT system, the reconstruction has a axial FOV of 500 mm and the matrix size is 512 × 512 with voxel size 1.95 × 1.95 × 5.00mm³. PET/MR examinations were performed in the GE SIGNA PET/MR system (Grant et al 2016). The transaxial and axial FOV of the PET/MR system is 600 mm and 250 mm, respectively. The crystal size is 4.0 × 5.3 × 25 mm³. PET images were reconstructed using the ordered subset expectation maximization (OSEM) algorithm with TOF information. The point spread function (PSF) (Alessio et al 2010) was also included to improve the image quality. Two iterations with sixteen subsets were run. The voxel size is 1×1×2.87 mm³ and the image size is 300×300×89. Dixon MR images were acquired using the head and neck coil array (repetition time, ~4 ms; first echo time/second echo time, 1.3/2.6 ms; flip angle, 5°; acquisition time, 18 s) and the image size is 256 × 256 × 120 with voxel size 1.93 × 1.93 × 2.6 mm³. ZTE images were acquired using the same head and neck coil array (repetition time, ~0.7 ms; echo time, 0 ms; flip angle, 0.6°; transmit/receive switching delay, 28 ms; readout duration, 440 ms; acquisition time, 41 s) and the reconstructed image size is 110 × 110 × 110 with voxel size 2.4 × 2.4 × 2.4 mm³ (Yang et al 2017).

3.2. Implementation details

When preparing the training pairs, we first registered CT images and ZTE images (if applicable) to the Dixon MR images through rigid transformation using the ANTs software (Avants et al 2009). Then random rotation and permutation was performed on the training pairs to avoid over-fitting. Fig. 3 shows some of the example pairs from different patient data sets used in the training phase for the multiple input scenario. When using only Dixon images as the input, in order to make full use of all the data sets in both the training and testing periods, the forty patient data sets were randomly separated into five groups. For each group, the whole eight data sets were used for testing and the remaining thirty two from other groups were employed in training. Among the forty patients, there are fourteen patients with additional ZTE scans. When using both Dixon and ZTE images as inputs, the fourteen patient data sets were randomly separated into seven groups. For each group the network was trained using the data sets from other groups.

Figure 3.

Examples of the training pairs used in the network training. Top row is the CT label image, middle and bottom rows are the corresponding Dixon MR images (middle) and ZTE MR images (bottom).

The network structures were implemented in TensorFlow using Adam algorithm as the optimizer (Kingma and Ba 2014).The learning rate and the decay rates used are the default settings in Tensorflow. For the single input case, the batch size was set to 60 and for the multiple input case, the batch size was set to 30. 1000 epochs were run for both cases as the training cost function becomes steady after 1000 epochs.

3.3. References methods

From the Dixon MR image, water and fat tissues were segmented and corresponding attenuation coefficients were assigned to generate the attenuation map. This method is labelled as Dixon-Seg. Alternatively, the patient MRI image can be registered to the MR template enabled by prior patients’ MR and CT pairs through non-rigid registration. Air, soft tissue, sinus and bone exist in the generated CT image. This method is named as Dixon-Atlas (Wollenweber et al 2013). For the segmentation method using ZTE images, the ZTE images were first N4 bias corrected (Tustison et al 2010) and then normalized by the median tissue value. Thresholding was performed to segment the images into air, soft tissue and bone regions. This method is labeled as ZTE-Seg. All of these three methods are available in the PET reconstruction tool box provided by the vendor.

3.4. Evaluation metrics

The predicted pseudo CT image quality was evaluated using the relative validation loss, defined as

$$\text{Relative validation loss}=\frac{\left|{\text{CT}}_{\text{pseudo}}-{\text{CT}}_{\text{true}}\right|}{\left|{\text{CT}}_{\text{true}}\right|},$$ (8)

where CT_pseudo is the generated CT using different methods, and CT_true denotes the ground-truth CT. Bone regions were also quantified using the Dice index, defined as

$$\text{Dice index}=2\frac{{\text{Bone}}_{\text{pseudo}}\cap {\text{Bone}}_{\text{true}}}{{\text{Bone}}_{\text{pseudo}}+{\text{Bone}}_{\text{true}}}.$$ (9)

Regions with attenuation coefficient higher than 0.1083 cm⁻¹ (200 HU unit) were classified as the bone area. For PET image quantification, the relative PET error was used which is defined as

$$\text{Relative PET error}=\frac{\left|{\text{PET}}_{\text{pseudoCT}}-{\text{PET}}_{\text{CT}}\right|}{\left|{\text{PET}}_{\text{CT}}\right|},$$ (10)

where PET_pseudoCT is the PET image reconstructed using the pseudo CT, and PET_CT is the PET image reconstructed using the ground-truth CT. The reason we use absolute value here is to ensure the total error will not vanish when summing up the voxel errors inside a region. As it is hard to visualize the error for all pixels, we calculated the relative PET error inside specific regions using the corresponding predefined masks.

3.4.1. Global quantification

We performed a global brain quantification using the brain mask from MNI-ICBM 152 nonlinear 2009 version (Fonov et al 2009). The Dixon image of each patient was first registered to the MNI template. Then the MNI template was back-warped to the Dixon image space. Besides, a mask is defined to include the pixels whose intensity is larger than 30 percent of the max PET intensity (Ladefoged et al 2017). The final global brain mask is defined as the intersection of these two masks. Besides, the histograms of the error image inside the global brain mask, defined as PET_pseudoCT – PET_CT, were calculated to compare the global performance regarding the bias and standard deviation.

3.4.2. Regional quantification

Apart from the whole brain quantification, we are also interested in the regional brain quantifications as they each play crucial roles in specific neuroscience studies. The automated anatomical labeling (AAL) template (Holmes et al 1998) was back-warped to the PET image space and defined the regions. Four cortex lobes as well as the inner deep regions were used in the quantification. The mean and standard deviation of the relative PET error across all patients for each of the methods were calculated for all the regions and the whole brain.

4. Results

4.1. Using Dixon MR images as input

We first performed a comparison of the proposed Dixon-Unet method with the Dixon-Seg and Dixon-Atlas methods using all data sets. Fig. 4 shows three orthogonal views of the ground truth CT images and the generated pseudo CT images using different Dixon-based methods for one patient. Compared with the atlas method, the CT image produced by the proposed Dixon-Unet method has better bone and sinus structures. The Dixon-Seg method only shows the water and fat tissues. Table. 1 presents the quantitative comparison of the predicted CT images using relative validation loss and the Dice index. Clearly the Dixon-Unet method has the smallest validation loss and the highest Dice index in the bone region. Fig. 5 presents the PET reconstruction error images using the attenuation map produced from the pseudo CT images shown in Fig. 4. Evidently the Dixon-Seg method has the largest error, especially near the bone and air-cavity regions. The Dixon-Atlas method produces smaller errors compared with the Dixon-Seg method, but still has large errors near the bone and the air cavity. Compared with these two methods, Dixon-Unet method shows smaller errors for the whole brain.

Figure 4.

Three views of the attenuation maps (unit, cm⁻¹) derived from the true CT image (first column) and the generated pseudo CT images using the Dixon-Seg method (second column), Dixon-atlas method (third column) and the proposed Dixon-Unet method (last column).

Table 1.

Comparisons of the generated pseudo-CT images when only Dixon images are available (based on 40 patient data sets). The Dice index of bone regions was computed for the whole brain, regions above and below the eyes.

Methods	Relative validation loss (%)	Dice of bone whole	Dice of bone above eye	Dice of bone below eye
Dixon-Seg	32.70 ± 5.38	–	–	–
Dixon-Atlas	22.86 ± 2.34	0.52 ± 0.05	0.61 ± 0.06	0.30 ± 0.05
Dixon-Unet	13.84 ± 1.43	0.76 ± 0.04	0.82 ± 0.04	0.63 ± 0.06

Figure 5.

Three views of the PET reconstruction error images (PET_pseudoCT – PET_CT, unit: SUV) using the Dixon-Seg method (left column), the Dixon-atlas method (middle column) and the proposed Dixon-Unet method (right column).

To quantitatively characterize the influence of different attenuation correction methods on PET images, the mean relative PET error across all the data sets for the whole brain and different regions were calculated and presented in Fig. 6. Clearly in all regions the Dixon-Unet method is the best among all Dixon methods. The Dixon-Seg method has the largest error due to the missing of bone signals. Comparing the standard deviations, Dixon-Unet method has the smallest standard deviation in all regions, meaning it is robust across different populations by using the information from other patient data sets. For all regions, the error of the Dixon-Unet method is below 3%. Fig. 7 shows the histogram plot of the PET error images for the three methods. The plot indicates that the error image of the Dixon-Unet method has the smallest standard deviation and the histogram shape is more like a Gaussian distribution with zero mean. The histogram shapes of the Dixon-Seg and Dixon-Atlas methods are more screwed. Specially the Dixon-Seg method is negatively biased due to missing bone.

Figure 6.

The bar plot of the mean relative PET error for all the patient data sets. Standard deviation of the relative PET error for all the patients are plotted as the error bar.

Figure 7.

The histogram of PET error images for the three Dixon methods.

4.2. Using both Dixon and ZTE MR images as input

In the following analysis, results using Dixon-Atlas, ZTE-Seg, DixonZTE-Unet and DixonZTE-GroupUnet methods were presented and compared using the twelve patient data sets with FDG scans. Fig. 8 shows three orthogonal views of the ground truth CT images as well as the generated pseudo CT images using different methods for one patient. Compared to the Dixon-Atlas method, the ZTE-Seg method can recover most of the bone regions as the contrast between the bone and neighboring pixels is good in the ZTE MR image. The images generated using the neural network methods are generally similar to the images generated using the ZTE-Seg method, but with more details revealed and closer to the CT ground truth. To compare the pseudo-CT qualities for each data set, Fig. 9 shows the CT validation loss using the U-net and GroupU-net structures. The proposed GroupU-net method has lower validation loss in 13 out of 14 data sets. Table. 2 presents the quantitative comparison of the predicted CT images. The proposed GroupU-net method has the smallest validation loss and the highest Dice index in the bone region. Fig. 10 gives three views of the PET reconstruction error images based on the corresponding pseudo CT images presented in Fig. 8. The Dixon-Atlas method has the largest error and the DixonZTE-GroupUnet method has the smallest error. Fig. 11 shows the plot of the mean relative PET error for all twelve patients across different regions. Clearly the Dixon-Atlas method has the largest mean error in all regions and the ZTE-Seg method generates smaller errors as compared with the Dixon-Atlas method. The proposed neural network methods can be better than both Dixon-Atlas and ZTE-Seg methods. Specially, the DixonZTE-GroupUnet can produce the smallest errors in all regions. This trend can also be observed in the histogram plot of the PET error images shown in Fig.12. The DixonZTE-GroupUnet method has both the smallest standard deviation and the smallest systematic bias.

Figure 8.

Comparison of the true CT image (first column) with generated pseudo CT images using the Dixon-Atlas method (second column), the ZTE-Seg method (third column), the DixonZTE-Unet method (fourth column) and the DixonZTE-GroupUnet method (last column).

Figure 9.

Comparison of the validation loss regarding the predicted CT images using U-net and the proposed GroupU-net when both Dixon and ZTE MR images are used as network input.

Table 2.

The comparison of the generated pseudo-CT when both Dixon and ZTE images are available (based on 14 patient data sets). The Dice index of bone was computed for the whole brain, regions above and below the eyes.

Methods	Relative validation loss (%)	Dice of bone whole	Dice of bone above eye	Dice of bone below eye
Dixon-Atlas	23.33 ± 3.23	0.52 ± 0.05	0.61 ± 0.06	0.29 ± 0.05
ZTE-Seg	16.20 ± 2.28	0.69 ± 0.05	0.75 ± 0.05	0.56 ± 0.07
DixonZTE-Unet	13.58 ± 1.53	0.77 ± 0.04	0.83 ± 0.04	0.66 ± 0.07
DixonZTE-GroupUnet	12.62 ± 1.46	0.80 ± 0.04	0.86 ± 0.03	0.69 ± 0.06

Figure 10.

PET reconstruction error images (PET_pseudoCT – PET_CT, unit: SUV) using the Dixon-Atlas method (first column), the ZTE-Seg method (second column), the DixonZTE-Unet method (third column) and the DixonZTE-GroupUnet method (last column).

Figure 11.

The bar plot of the mean relative PET error for the patient data sets with both Dixon and ZTE images. Standard deviations of the absolute error for all the patients are plotted as the error bar.

Figure 12.

The histogram plot of the PET SUV difference inside the whole brain for Dixon-Atlas, ZTE-Seg, DixonZTE-Unet and DixonZTE-GroupUnet methods.

5. Discussion

Dixon MR acquisition is simple and fast. It is widely deployed in current PET/MR systems as an option for further attenuation map derivation. As the signal intensity is low in the bone region, it is hard to segment the bone out. In this work, we employed the deep neural network method to predict pseudo CT images from Dixon images. From the CT images presented in Fig. 4, we can notice that the shape of the bone region predicted by the neural network method is much better than the atlas method. This indicates the neural network can recognize the bone region from the Dixon image input. Further quantitative analysis based on 40 patient data sets reveals that the mean relative PET error of the whole brain using the neural network method is within 3%, which demonstrates the reproducibility of the proposed method.

With the developments of new MR sequences, multiple MR images are available during the same scan. It is thus crucial to find an optimum way integrating the information from multiple MR images while not increasing the network complexity much, especially when the training data sets are not large enough. In this work we have proposed a modified U-net structure, named GroupU-net, to digest both Dixon and ZTE information through group convolution modules when the network goes deeper. The group convolution module first considers the cross-channel correlation through 1×1 convolution, and then handles the spatial correlation in smaller groups. Quantification analysis shows that the GroupU-net structure has better performance than the U-net structure when the network complexity is the same. This demonstrates that model parameters can be used more efficiently by making the network wider when the network goes deeper. It also shows that improving the network structure can generate better attenuation maps. Designing and testing different network structures will be one of our future work.

For the case of using both Dixon and ZTE images as network input, there are 12 patient data sets in each training group. Quantitative analysis demonstrates that 12 patient data sets can be used to train a network which provides higher CT prediction accuracy than the state-of-art methods. One limitation of this work is that no brain pathology was reported for the brain data sets employed in this study. We are unsure about the prediction accuracy for MR images with abnormal regions. If the test data do not lie in the training space due to population difference, the trained network may not accurately recover unseen structures. The robustness of the trained network to diseased data sets deserves further evaluations.

As for the objective function employed in the network training, L1 norm was found to be better than L2 norm. L2 norm results in blurrier images. We also tried another objective function by including additional L1 difference between the gradient images of the ground-truth CT and pseudo CT in both the horizontal and vertical directions. Though the generated CT image had a sharper bone, quantification for the inner regions, such as the putamen and caudate, showed worse results. The sizes of the air-cavity regions in the ZTE and Dixon MR images are different. As different methods extract information from Dixon only, ZTE only, or Dixon and ZTE combined, there will be difference about the delineations of air-cavity as shown in Fig. 4 and Fig. 8. Additionally, we noticed that for the MR and CT images acquired in two different scanners, the jaw and head-neck regions could not be registered well in some cases due to position difference. This can produce errors as the training presumes that the CT and MR images match perfectly. Generalized adversarial networks which do not depend on the paired MR and CT images might help solve this problem.

6. Conclusion

We have proposed a neural network method to generate the continuous attenuation map for brain PET imaging based on the Dixon MR images only, and based on Dixon and ZTE images combined. Analysis using real data sets shows that the neural network method can produce smaller PET quantification errors as compared to other standard methods. When both Dixon and ZTE images are available, the proposed GroupU-net structure, which extracts features from Dixon and ZTE images through group convolution modules when the network goes deeper, can have better performance than the U-net structure. Future work will focus on designing and testing different network structures to better improve the results as well as testing the robustness of the trained network to diseased data sets.