Joint Correction of Attenuation and Scatter in Image Space Using Deep Convolutional Neural Networks for Dedicated Brain ¹⁸F-FDG PET

Abstract

Dedicated brain positron emission tomography (PET) devices can provide higher-resolution images with much lower dose, compared to conventional whole-body PET systems, which is important to support PET neuroimaging particularly useful for the diagnosis of neurodegenerative diseases. However, when a dedicated brain PET scanner does not come with a combined CT or transmission source, there is no direct solution for accurate attenuation and scatter correction, both of which are critical for quantitative PET. To address this problem, we propose joint attenuation and scatter correction (ASC) in image space for non-corrected PET (PET_NC) using deep convolutional neural networks (DCNN). This approach is a one-step process, distinct from conventional methods that rely on generating attenuation maps first that are then applied to iterative scatter simulation in sinogram space. For training and validation, time-of-flight PET/MR scans and additional Helical CTs were performed for 35 subjects (25/10 split for training and test dataset). A DCNN model was proposed and trained to convert PET_NC to DCNN-based ASC PET (PET_DCNN) directly in image space. For quantitative evaluation, uptake differences between PET_DCNN and reference CT-based ASC PET (PET_CT-ASC) were computed for 116 automated anatomical labels (AAL) across 10 test subjects (1160 regions in total). MR-based ASC PET (PET_MR-ASC), a current clinical protocol in PET/MR imaging, was another reference for comparison. Statistical significance was assessed using a paired t test. The performance of PET_DCNN was comparable to that of PET_MR-ASC, in comparison to reference PET_CT-ASC. The mean SUV differences (mean ± SD) from PET_CT-ASC were 4.0 ± 15.4 % (P < 0.001) and −4.2 ± 4.3 % (P < 0.001) for PET_DCNN and PET_MR-ASC, respectively. The overall larger variation of PET_DCNN (15.4%) was prone to the subject with the highest mean difference (48.5 ± 10.4 %). The mean difference of PET_DCNN except the subject was substantially improved to −0.8 ± 5.2% (P < 0.001) that was lower than that of PET_MR-ASC (−5.07 ± 3.60%, P < 0.001). In conclusion, we demonstrated the feasibility of directly producing PET images corrected for attenuation and scatter using a DCNN (PET_DCNN) from PET_NC in image space without requiring conventional attenuation map generation and time-consuming scatter correction. Additionally, our DCNN-based method provides a possible alternative to MR-ASC for simultaneous PET/MRI.

1. Introduction

Positron Emission Tomography (PET) enables an understanding of biochemical changes in the brain at early stages of disease prior to structural changes or clinical symptoms. PET neuroimaging is particularly useful in the diagnosis of neurodegenerative diseases (e.g. Alzheimer’s disease, frontotemporal dementia, dementia with Lewy bodies, Parkinson’s disease, and Huntington’s disease) such as diseases associated with significant changes in brain metabolism (Silverman and Alavi, 2005). The most commonly-used tracer in clinical brain PET imaging is ¹⁸F-fluorodeoxyglucose (FDG) but the development of other radiotracers such as dopaminergic, amyloid plaque and tau imaging agents is still an active development area (Brown et al., 2014).

To keep pace with the expanding demand of PET neuroimaging, dedicated high-resolution brain PET systems such as High-Resolution Research Tomograph (HRRT) (Wienhard et al., 2002), CerePET™ (Brain Biosciences) and CareMiBrain (Oncovision) have been introduced in the market for human brain imaging, supporting significantly improved high-resolution images typically with lower dose, compared to a conventional whole-body PET system. Additionally, wearable PET scanners such as ambulatory micro-dose PET (AM-PET) (Melroy et al., 2017) and helmet-PET (Tashima and Yamaya, 2016) have been developed and are being evaluated for functional brain research. However, these dedicated or wearable PET systems do not provide direct solutions for attenuation and scatter correction. These systems are not combined with computed tomography (CT) or magnetic resonance imaging (MRI) that can provide attenuation maps used for attenuation and scatter correction. Thus, a challenge for dedicated brain PET systems is to develop a practical and robust method for accurate attenuation and scatter correction without an additional imaging modality, such as CT or MRI.

Attenuation and scatter correction (ASC) is critical for quantitative accuracy as well as image quality in PET (Meikle and Badawi, 2005). Attenuated and scattered events occur due to photoelectric effects and Compton scattering induced by the presence of dense material along lines of response (LORs). Without attenuation correction, regions near the skin appear darker (emitting more photons) and regions surrounding brain tissues appear brighter (emitting less photons). Scatter fraction can reach 50% to 60% of LORs recorded in whole-body 3D PET and, without scatter correction, LORs recorded outside an object boundary due to scatter contribute noise in image reconstruction. Therefore, it is important to compensate for attenuation and scatter for quantitative PET. In a hybrid PET/CT or PET/MR imaging, the current implementation for attenuation correction is to transform CT (Lonn et al., 2003) or MR-derived pseudo CT images into attenuation maps (Wollenweber et al., 2013; Yang et al., 2017b; Berker and Li, 2016); while, the current implementation for scatter correction is to estimate scatter iteratively by a 3D model-based simulation using down-sampled attenuation and emission images (Iatrou et al., 2006; Watson et al., 2004; Zaidi and Montandon, 2007). Both attenuation and scatter correction are separately performed due to the difference of photoelectric effects and Compton scattering (Meikle and Badawi, 2005) in sinogram space where LORs are conventionally recorded as counts according to their locations and orientations and then reconstructed to PET images by an ordered subset expectation maximization (OSEM) algorithm (Defrise et al., 2005).

Recently, deep convolutional neural networks (DCNN) have been widely applied to medical imaging based on the success of deep learning to computer vision tasks (Ronneberger et al., 2015). DCNN demonstrated the direct conversion to pseudoCT from T1-weighted MR (Liu et al., 2018a), Dixon and Zero-TE MR (Gong et al., 2018) or non-attenuation-corrected PET (Liu et al., 2018b) for attenuation correction in PET neuroimaging. Also, DCNN demonstrated to improve the quality of noisy attenuation maps generated by simultaneous maximum-likelihood reconstruction of activity and attenuation by time-of-flight (TOF) information (Hwang et al., 2018).

In this paper, we propose a new approach for joint ASC in image space using only non-corrected PET images without depending on another imaging modality and performing a scatter simulation. Since brain tissues and their boundaries (e.g., white and gray matters, skin, bone, etc) are perceptible in non-corrected PET, DCNN can extract important patterns successfully for joint ASC in PET neuroimaging. The proposed joint ASC is a one-step process, distinct from conventional methods that rely on generating attenuation maps first that are then applied to iterative scatter simulation in sinogram space.

2. Methods

2.1. Patient information

The patient study was approved by the Institutional Review Board, and all patients signed an informed consent form before the examinations. 35 patients (16 male and 19 female) underwent whole-body ¹⁸F-FDG PET/MRI and helical CT scans. The average patient age was 57.7 ± 11.5 y (range, 29–76y), the average weight was 73.7 ± 17.4 kg (range, 39.5–109.8 kg), and the average administered dose of ¹⁸F-FDG was 308.5 ± 74.6 MBq (range, 170.2–468.1 MBq). The average scan duration of the whole brain was 227.2 ± 137.5 s (range, 135–900 s), and the average time difference between injection and scan was 150.8 ± 24.5 min (range, 111.0–190.1 min). A tumor was observed in the head for only one subject who was included in the test set (Figure 1).

Figure 1.

Schematic of conventional attenuation and scatter correction (ASC) performed in sinogram space during PET image reconstruction (left) compared with proposed deep convolutional neural networks (DCNN)-based ASC performed in image space (NC: non-corrected). A lesion is observed at the border of the brain. Note that the RF brain coil and the couch were included in the attenuation sinogram and that the scanner geometry effects (gaps between detector panels) were added to the scatter simulation, so that the scatter sinogram contains the same effects that are found in the emission sinogram.

2.2. PET/MRI and CT data acquisition

TOF PET/MRI examinations were performed on a SIGNA PET/MR scanner (GE Healthcare). PET had a 600-mm transaxial field of view (FOV) and 250-mm axial FOV, with a TOF timing resolution of approximately 400 ps and average measured sensitivity of 22.65 cps/kBq. While PET data were acquired, Dixon MR (FOV, 500 × 500 × 312 mm; resolution, 1.95 × 1.95 mm; slice thickness, 5.2 mm; slice spacing, 2.6 mm; scan time, 18 s) sequences were acquired for MR-based attenuation and scatter correction (MR-ASC), using the head and neck coil array. Helical CT images of the patients were acquired on a Discovery PET/CT (GE Healthcare) or Biograph HiRez 16 (Siemens Healthcare) scanner with variable parameter settings (120 kVp; 105–599 mA; rotation time, 0.5 s; pitch, 0.98 and 0.75; rotation, 39.37 and 34.45 mm; axial FOV, 700 and 500 mm; slice thickness, 3.75 and 5.00 mm; matrix size, 512 × 512; and voxel sizes, 2.73 × 2.73 × 3.75 and 1.95 × 1.95 × 5.00 mm³ for GE and Siemens systems, respectively) for CT-based attenuation and scatter correction (CTASC). The methodology described in our previous work (Yang et al., 2017a; Yang et al., 2017b) was used for CT image preprocessing and coregistration to MR images: MR and CT image pairs were coregistered using the vendor-developed registration toolkit or Advanced Normalization Tools (Avants BB, 2009) based on the Insight Segmentation and Registration Toolkit (Kitware, Clifton Park, NY).

2.3. PET image reconstruction

As depicted in Figure 1, non-corrected PET (PET_NC) and CT-based attenuation/scatter-corrected PET (PET_CT-ASC) images were reconstructed by a TOF OSEM algorithm (4 iterations; 28 subsets; axial FOV, 350 mm; matrix size, 256 × 256 × 89; voxel size, 1.37 × 1.37 × 2.78 mm³; 4.0 mm in-plane Gaussian filter followed by axial 3-slice 1:4:1 filtering) in the offline PET/MR toolbox (REL_1_28, GE Healthcare). Also, MR-based attenuation/scatter-corrected PET (PET_MR-ASC) images were reconstructed as a silver standard for evaluation, providing the bottom line of a clinically-acceptable performance limit. Corrections including normalization, dead time, decay, point-spread function, and randoms were applied during the reconstruction.

2.4. Deep convolutional neural networks (DCNN)

The aim of this work is to develop a DCNN model that can transform PET_NC to PET_CT-ASC directly in image space (PET_DCNN), without generating attenuation maps and performing an iterative scatter simulation.

2.4.1. DCNN Architecture.

The proposed DCNN consists of five encoder-decoder stages with symmetrically concatenated with skip connections (Figure 2) based on the U-Net (Ronneberger et al., 2015). In each stage, convolution (Conv) with 3×3 kernels, batch normalization (BN) (Ioffe and Szegedy, 2015), and rectified linear unit (ReLU) is sequentially performed twice. Between stages, the downsampling and upsampling are done by 2×2 max pooling and bilinear interpolation (Xu et al., 2017), respectively. In order to preserve local information and resolution of the image, skip connections transfer the 2^nd convolution layer of the encoder, performed prior to the BN and ReLU activation, to the decoder after upsampling at the same level of stage (Liu et al., 2018b).

Figure 2.

Deep Convolution Neural Networks (DCNN) architecture (Conv: convolution; BN: batch normalization; ReLU: rectified linear unit).

2.4.2. Preprocessing.

PET_NC and PET_CT-ASC images were utilized as paired input and output for training/testing our proposed DCNN architecture. For each training dataset, PET raw values (Bq/ml) were scaled down to (kBq/ml) to reduce the dynamic range of input/output values and PET slices above the top of the head and below the cerebellum were removed to focus on the brain. Also, activities out of the head were considered as a noise and removed by binary masking.

2.4.3. Model Training.

The DCNN model was trained with three-slice input to provide volumetric information due to attenuation and scatter in the axial direction. Training multi-slice inputs can provide higher efficiency with fewer parameters than training with depth-wise operation of 3D convolution (Xu et al., 2017). Before being fed into the model, the input was randomly rotated (−10 ~ 10 degree), flipped horizontally, and translated horizontally (< 50 pixels in FOV) for data augmentation to simulate a larger dataset and to avoid overfitting. Model training was performed with the mean squared error (or L2 loss) and RMSprop optimizer (Hinton et al.) with a learning rate initialized by 0.001 which halved automatically if the loss did not decrease in 10 epochs. Weights for convolution were initialized with truncated Gaussian distributions with zero mean and standard deviation of 0.02. All biases were initialized with zero. A mini-batch of 32 input/output patches (1375 patches in total) was used for training and the loss was converged in 140 epochs.

2.4.4. Computation.

Training and testing our proposed DCNN were performed on a Ubuntu server with a single Tesla P100 (NVIDIA) graphics processing unit. The proposed model was implemented using Tensorflow (version 1.9.0) and Keras libraries (version 2.2.0). Model training takes approximately 160 minutes to reach stability, which occurred at approximately 5,880 iterations (140 epochs × 42 iterations per epoch). At that point, the training was stopped. After training the model, it took only 0.4 s on average to generate PET_DCNN volumetric images (89 slices) with the single Tesla P100.

2.5. Evaluation metrics

2.5.1. Quantitative analysis

All PET images were spatially registered to a brain template with 116 automated anatomical labels (AAL) for a generalized regional analysis. Dixon MRAC T1 images simultaneously acquired with PET were registered to the T1 brain template provided by the Montreal Neurological Institute (MNI) (Tzourio-Mazoyer et al., 2002) using Advanced Normalization Tools (Avants BB, 2009). The derived registration parameters were applied to deform PET images to the template. Absolute and relative (%) differences (mean ± SD) of standardized uptake values (SUV = image-derived uptake [MBq/mL] ⁄ injection dose [MBq] × patient’s weight [g]) between PET_DCNN and reference PET_CT-ASC were computed for 116 AAL regions across 10 test subjects (1160 regions in total) as follows:

$$\begin{gathered} \text{Difference }(SUV)=\text{mean}({\text{voxels of PET}}_{\text{DCNN}}\text{ in AAL}\#) \\ -{\text{ mean(voxels of PET}}_{\text{CT-ASC}}\text{ in AAL\#)} \end{gathered}$$

$$\begin{gathered} {\text{Difference (\%) = [mean(voxels of PET}}_{\text{DCNN}}\text{ in AAL\#)} \\ -{\text{ mean(voxels of PET}}_{\text{CT-ASC}}\text{ in AAL\#)]} \\ {\text{/ mean(voxels of PET}}_{\text{CT-ASC}}\text{ in AAL\#) }\times 100 \end{gathered}$$

The distributions of the differences were presented in Bland–Altman plots. A difference between PET_MR-ASC and PET_CT-ASC was also calculated as a reference for comparison. Statistical significance was assessed using a paired t test and a P value < 0.05 was deemed statistically significant.

Additionally, to evaluate a performance variation of DCNN between subjects, subject-specific SUV differences across 116 AAL regions were computed separately. Also, to assess a performance variation of DCNN in the brain regions, SUV differences were calculated in 8 merged regions: cerebellum, temporal lobes, occipital lobes, parietal lobes, frontal lobes, thalamus, putamen, and caudate nucleus.

2.5.2. Qualitative analysis

PET_CT-ASC, PET_DCNN and their difference images were illustrated for representative three test subjects: subject 1 with the longest scan duration (900 s) selected for presenting the effect of a scan time variation (mean scan duration: 227.2 ± 137.5 s); subject 2 with a tumor in the head chosen for presenting the effect of a pathological variation; and subject 3 with outliers selected for presenting the visual effect of a large difference.

2.5.3. Voxelwise analysis

A joint histrogram was used to show the distribution of voxel-based PET uptake correlation between PET_DCNN and reference PET_CT-ASC within the SUV range of 0.5–20.0 (g/mL). Also, an error histogram was used to show the distribution of voxel-based PET uptake differences within the same SUV range.

All the processing and analyses above were performed in MATLAB (MathWorks).

3. Results

The average difference of PET_DCNN from PET_CT-ASC was 0.20 ± 0.92 (3.98 ± 15.42 %, P < 0.001) and the average difference of PET_MR-ASC from PET_CT-ASC was −0.31 ± 0.31 (−4.24 ± 4.29 %, P < 0.001) (Table 1). PET_DCNN was slightly overestimated (4.0%; range, −13.4 ~ +63.1%) with a larger variation (15.4%); while, PET_MR-ASC was slightly underestimated (−4.2%; range, −28.8 ~ +9.3%) with a smaller variation (4.3%). In the Bland–Altman plot (Fig. 3), most of the differences (circles) with DCNN were positioned within ±10%; whereas, most of the differences (triangles) with MR-ASC were prone to negative areas within ±10%. Box plots of 116 AAL regions across test subjects are available in Supplementary Figure 1.

Table 1.

Uptake differences (SUV and %) of PET_DCNN and PET_MR-ASC from reference PET_CT-ASC across 116 regions × 10 test subjects (1160 regions in total).

	Difference
	SUV	%	P
PETDCNN	0.20 ± 0.92	3.98 ± 15.42	<0.001
PETMR-ASC	−0.31 ± 0.31	−4.24 ± 4.29	<0.001

Differences are mean ± SD. A paired t test was performed for the pair of PET_CT-ASC and PET_DCNN.

Figure 3.

Bland–Altman plots of (a) PET_DCNN and (b) PET_MR-ASC for uptake differences (%) from reference PET_CT-ASC across 116 regions × 10 test subjects (1160 regions in total). In (a), the isolated cloud of circles is in the range of 40–60% belongs to subject 3.

Because the overall larger variation of DCNN (15.4%) was mostly due to the outliers between 40–60% differences in the Bland–Altman plot (Figure 3), we calculated subject-specific differences across 116 regions to evaluate a performance variation of DCNN across subjects. Figure 4a demonstrated that subject 3 with the highest mean difference contributed the outliers to the Bland-Atlman plot of DCNN. In order to derive a more generalized result without the outliers, the average differences of PET_DCNN and PET_MR-ASC from PET_CT-ASC were recalculated after excluding subject 3 and the updated results were summarized in Table 2. The average difference of DCNN was substantially reduced from 4.0% to −0.8% without the outliers, that was much smaller than the average difference of MR-ASC (−5.1%); while, the SD of DCNN (±5.2%) was still slightly higher than the SD of MR-ASC (± 3.6%).

Figure 4.

Box plots of PET uptake differences (%) of DCNN and MR-ASC from CT-ASC: (a) 10 test subjects across 116 AAL regions, (b) 8 merged regions across 10 test subjects, (c) 8 merged regions across 9 subjects without subject 3. Subject-3 has the highest mean ± SD difference (48.52 ± 10.36%) for DCNN.

Table 2.

PET uptake differences (SUV and %) of DCNN and MR-ASC from reference PET_CT-ASC across 116 regions × 9 subjects (excluding subject 3).

	Difference
	SUV	%	P
PETDCNN	−0.08 ± 0.33	−0.83 ± 5.20	<0.001
PETMR-ASC	−0.36 ± 0.27	−5.07 ± 3.60	<0.001

Differences are mean ± SD. A paired t test was performed for the pair of PET_CT-ASC and PET_DCNN.

Also, the average differences were calculated in 8 merged regions to assess a regional performance variation of DCNN. Figure 4b–c compared the generalized regional differences with and without subject 3 (outliers), which was also consistent with the overall result above. For example, the average difference of DCNN was substantially reduced from 6.2% to 0.4% for cerebellum after excluding subject-3. The reduced difference of DCNN (0.4%) was much smaller than the recalculated difference of MR-ASC (−7.1 %); while, the SD of DCNN (± 11.3%) was still higher than the SD of MR-ASC (± 7.1%).

Next, Figure 5 illustrates qualitative differences of PET_DCNN and PET_MR-ASC from reference PET_CT-ASC with examples of representative three subjects (subject 1, 2, and 3). The examples show the overall similarity between PET images with different ASC methods (reference CT-ASC, DCNN, and MR-ASC) and the voxel-based difference patterns of DCNN and MR-ASC from CT-ASC. In Figure 5a (subject 1), the major difference between DCNN and MR-ASC is that DCNN differences are randomly distributed with the mixture of over- and underestimated patterns; while, MR-ASC differences depict overall slight underestimation across the brain but a strong underestimation near the skull. In Figure 5b (subject 2), the SUV_max of the tumor with DCNN was underestimated by −13.5%; while, the SUV_max of the tumor with MR-ASC was underestimated by −6.7%, compared to the SUV_max with CT-ASC. In Figure 5c (subject 3), both PET_DCNN (structural similarity index (SSIM) (Wang et al., 2004) = 0.9670) and PET_MR-ASC (SSIM = 0.9968) are qualitatively similar to PET_CT-ASC, though PET_DCNN was substantially overestimated across the brain in the result of Figure 4a (48.5 ± 10.4% vs. 4.0 ± 4.1%).

Figure 5.

PET examples of representative subjects for CT-ASC, DCNN, and MR-ASC: (a) subject 1 with the longest scan duration (900 s), (b) subject 2 with a tumor in the head, and (c) subject 3 with the highest mean difference.

Further voxelwise evaluation was performed by the joint histogram (Figure 6) and error histogram (Figure 7). The analysis shows the voxel-wise similarity between PET_DCNN and reference PET_CT-ASC with the slope of 1.01 and R² of 0.98 (Figure 6a). PET_DCNN achieved higher accuracy for lower uptake voxels with smaller variation but lower accuracy for larger uptake voxels with larger variation; whereas, PET_MR-ASC achieved smaller variation with a trend of underestimation with the slope of 0.97 and R² of 0.99 (Fig. 6b), which was consistent with the result of the error histograms of PET_DCNN and PET_MR-ASC (Figure 7).

Figure 6.

Joint histogram of PET voxels within SUV range of 0.5–20.0 (g/mL): (a) PET_DCNN versus PET_CT-ASC and (b) PET_MR-ASC versus PET_CT-ASC. Note that the counts were log-scaled (i.e., log₁₀(counts)) to visualize small counts.

Figure 7.

Error histogram of PET voxels (PET_DCNN – PET_CT-ASC and PET_MR-ASC – PET_CT-ASC) within SUV range of 0.5–20.0 (g/mL).

4. Discussion

Our results demonstrated that DCNN can achieve joint ASC, that converts PET_NC to PET_ASC directly in image space without depending on a conventional approach that performs attenuation and scatter correction separately in sinogram space. To our knowledge, this is the first work to investigate the feasibility of performing ASC simultaneously in image space through DCNN. In this study, DCNN demonstrated comparable quantitative (−0.83 ± 5.20% difference without outliers) and qualitative results, compared to conventional CT-ASC or MR-ASC performed in sinogram space using CT or MR-derived attenuation maps and time-consuming iterative scatter correction.

Encouragingly, the results demonstrated the great potential of DCNN-based joint ASC to promote the clinical feasibility for a dedicated brain PET system such as a small-FOV PET or a wearable PET (Tashima and Yamaya, 2016; Oncovision; Brain Biosciences; Melroy et al., 2017). Since it is technically difficult and not practical to combine such small PET systems with an additional CT or MR system, it is important to devise a practical and robust way for ASC without requiring additional anatomical imaging. Additionally, it is encouraging to provide a possible alternative to MR-based attenuation correction (MRAC) in simultaneous PET/MRI. Although the solutions for MRAC of simultaneous brain PET/MRI have been improved (Ladefoged et al., 2017), these solutions are still not regarded as on the same level as CT-based attenuation correction. The comparison of PET_DCNN and PET_MR-ASC (−0.83% versus −5.07% difference without subject 3) demonstrated that DCNN could be a potential alternative for MRAC, which was the motivation of using PET/MR data instead of PET/CT data in this study.

The DCNN-based ASC approach is fully automated and fully data-driven: PET_NC is directly converted to PET_ASC in image space almost in real time (0.4 s on average for 89 volumetric images) without requiring additional anatomical information or time-consuming scatter correction. In the offline toolbox, scatter simulation took approximately 150 seconds and TOF-PET reconstruction with ASC took 147.8 minutes that was 2.2 times longer than the processing time without ASC (67.8 minutes) in our workstation (Intel i7 CPU with 3.40GHz × 8 cores and 31.4 GB memory). Scatter correction increased overall processing time substantially in the course of image reconstruction, but we can overcome this issue by applying our DCNN to reconstructed images without ASC. In general, DCNN applications for PET are more challenging than those for MR and CT due to the low resolution and noise characteristics of PET. Nevertheless, the success of low-dose FDG PET reconstruction demonstrated the capability of DCNN to deal with noisy FDG PET data (Xu et al., 2017); and the feasibility study of DCNN-based conversion from FDG PET_NC to pseudo CT demonstrated the perceptibility of brain tissues and their boundaries in FDG PET_NC (Liu et al., 2018b). Therefore, it is not surprising to expect the successful result of our proposed DCNN-based joint ASC for FDG PET. In our study, the pattern differences were derived from how attenuation and scatter correction changes uncorrected image patterns simultaneously, which enabled our proposed DCNN to predict attenuation/scatter-correction patterns for corrected images.

For the model training, we did not consider the information about a table couch and external head coils that should be always included in attenuation maps derived from CT or MR for accurate attenuation correction and scatter simulation. Surprisingly, however, the omitted information was not problematic since the attenuation information caused by the external materials could be imbedded in training images themselves. For this reason, it is important not to perform vertical flipping for data augmentation because the back of the head was always positioned near the couch in the field of view and brain tissues near the couch were more attenuated in this setting.

The proposed DCNN achieved the high accuracy (−0.83% difference) on average except for one test subject (subject 3) who seemed to be not represented in the training cohort. To clarify, since the average skull density is 685.6 ± 61.1 HU (min: 569.6 HU, max: 805.1 HU) in our data set (34 subjects except subject 3), such a low skull density (e.g., subject-3: 475.1 HU) is not common. The low bone density of subject 3 was qualitatively and quantitatively presented in additional supplementary figures 2–3. Although the quantitative difference (48.5% difference) of this subject was clinically not acceptable, we expect that this overall overestimation problem might not be problematic for diagnostic purposes because the relative contrast was consistent for both DCNN and standard approaches for subject 3 (Figure 5). Therefore, it may be acceptable to use our DCNN for quantifying pre and follow-up scans if DCNN is consistently used for an outlier such as subject 3, since the values would be consistently overestimated with an acceptable contrast.

Despite the promising results, our study has several limitations. First, the amount of training and test data (25/10 data split) may not be enough to derive a fully-generalized optimal DCNN model. Since the generalization power of DCNN is largely dependent on the amount of data set, we did data augmentation to simulate a larger dataset and to avoid overfitting. Nevertheless, as shown in the result of subject 3, the model substantially overestimated PET_DCNN compared to reference PET_CT-ASC (Figure 4a), though the qualitative similarity between them was still high (Figure 5c). Additionally, the training set did not include pathological subjects, which resulted in the relatively larger difference (−13.5%) of DCNN than the difference of MR-ASC (−6.7%) for the tumor uptake in subject 2. Therefore, in order to deal with any pathological pattern in our DCNN model, it is very important to include the variety of pathological patterns that may include a similar pattern for a target pathology in our extended training set. With the addition of an increased number of subjects with various pathologies in the training dataset, we expect that our model could be robust to a potential source of bias or variation. Second, the model has not been tested for other radiotracers with different mean/maximum positron ranges (¹⁸F, 0.66/2.63 mm; ¹¹C, 1.27/4.46 mm; ¹³N, 1.73/5.57 mm; ¹⁵O, 2.97/9.13 mm; ⁶⁸Ga, 3.56/10.27 mm; ⁸²Rb, 7.49/18.6 mm in water (Champion and Le Loirec, 2007; Le Loirec and Champion, 2007)) or biological distributions that can change image resolution and visually-recognized brain patterns. Radiotracer-specific model training is likely necessary to evaluate the radiotracer-specific performance variation of our DCNN. Third, we have not investigated the performance of our DCNN for dynamic scans. Neuroreceptor studies potentially done with a dedicated brain PET system may be performed as dynamic scans, and the image patterns of such a study change continuously from the perfusion phase to the final image of neuroreceptor binding. Basically, since the theory of attenuation and scatter correction is consistent for static and dynamic PET, the DCNN model could derive dynamic-imaging-specific ASC patterns if enough dynamic brain PET data were used to train the model. As discussed above, the flexibility of our DCNN model is substantially dependent on the amount of pattern information in training data as well as the network architecture, we may need to tailor our model architecture and to retrain the model, considering data diversity such as inter-patient variation (pathological and anatomical difference) and inter-radiotracer variation (positron-range and biological distribution). Finally, there were large variations between PET protocols for the brain (e.g., scan duration, post-injection scan time, etc) since the brain PET scan was taken from a whole-body scan. However, DCNN-based joint ASC performed considerably well in spite of the variation, considering the result of subject 1 with the longest scan duration (900 seconds for subject 1, 227.2 ± 137.5 seconds for the others).

In future work, substantially increasing training data is our most important task to derive a fully-generalized model that can interpret inter-patient and inter-radiotracer variations. Since many clinical and research PET/CT and PET/MR scans are on-going in our department, we will access archived data and prospectively acquired data to increase our training/validation data set. We can increase training/validation data set using PET/CT and PET/MR data simultaneously; however, it may be necessary to train the model according to PET/CT and PET/MR data separately and to compare their results, since PET system resolution and reconstruction details are different between PET/CT and PET/MR systems. For future validation, it will be worth to use cross validation to perform a more generalized evaluation in a large cohort study. Second, although subjects with a very low bone density such as subject 3 are generally not likely, it is still important to control this potential problem to avoid an unexpectedly large quantitative error. To address this problem, it is probably feasible to develop a peripheral DCNN that might be able to derive a mean skull density for preventing unacceptable over/under estimation by training bone-specific voxel pairs of noncorrected PET and corrected PET, which is not a voxel-by-voxel conversion but a regression that might be resolved by DCNN. Third, we may increase the confidential level of our DCNN using conditional Generative adversarial networks (c-GANs). The c-GANs include a generator network and a discriminator network which are trained simultaneously in order to guarantee the predicted output to be close to the ground truth (Wang et al., 2018). Finally, we plan to investigate a trade off between image-based (our DCNN) and sinogram-based (Liu et al., 2018b) approaches using only PET data in terms of robustness and practice in a large cohort study.

5. Conclusion

We demonstrated the feasibility of directly producing PET images corrected for attenuation and scatter using DCNN (PET_DCNN) from noncorrected PET (PET_NC) in image space without requiring additional anatomical imaging and time-consuming scatter correction. This approach is a one-step process, distinct from conventional methods that rely on generating attenuation maps first that are then applied to iterative scatter simulation in sinogram space. In particular, DCNN-based joint ASC has great potential to promote the clinical feasibility for a dedicated brain PET system that needs a practical and robust way for ASC without requiring anatomical imaging. Additionally, our DCNN-based method provides a possible alternative to MR-ASC for simultaneous PET/MRI.