Deep‐TOF‐PET: Deep learning‐guided generation of time‐of‐flight from non‐TOF brain PET images in the image and projection domains

Abstract

We aim to synthesize brain time‐of‐flight (TOF) PET images/sinograms from their corresponding non‐TOF information in the image space (IS) and sinogram space (SS) to increase the signal‐to‐noise ratio (SNR) and contrast of abnormalities, and decrease the bias in tracer uptake quantification. One hundred forty clinical brain ¹⁸F‐FDG PET/CT scans were collected to generate TOF and non‐TOF sinograms. The TOF sinograms were split into seven time bins (0, ±1, ±2, ±3). The predicted TOF sinogram was reconstructed and the performance of both models (IS and SS) compared with reference TOF and non‐TOF. Wide‐ranging quantitative and statistical analysis metrics, including structural similarity index metric (SSIM), root mean square error (RMSE), as well as 28 radiomic features for 83 brain regions were extracted to evaluate the performance of the CycleGAN model. SSIM and RMSE of 0.99 ± 0.03, 0.98 ± 0.02 and 0.12 ± 0.09, 0.16 ± 0.04 were achieved for the generated TOF‐PET images in IS and SS, respectively. They were 0.97 ± 0.03 and 0.22 ± 0.12, respectively, for non‐TOF‐PET images. The Bland & Altman analysis revealed that the lowest tracer uptake value bias (−0.02%) and minimum variance (95% CI: −0.17%, +0.21%) were achieved for TOF‐PET images generated in IS. For malignant lesions, the contrast in the test dataset was enhanced from 3.22 ± 2.51 for non‐TOF to 3.34 ± 0.41 and 3.65 ± 3.10 for TOF PET in SS and IS, respectively. The implemented CycleGAN is capable of generating TOF from non‐TOF PET images to achieve better image quality.

We aim to synthesize brain time‐of‐flight (TOF) PET images/sinograms from their corresponding non‐TOF information in the image space (IS) and sinogram space (SS) to increase the signal‐to‐noise ratio (SNR) and contrast of abnormalities, and decrease the bias in tracer uptake quantification. The implemented CycleGAN is capable of generating TOF from non‐TOF PET images to achieve better image quality.

1. INTRODUCTION

Positron emission tomography (PET) provides a solid foundation for nuclear medicine and molecular imaging‐based examination for in vivo measurement of the metabolic activity or receptor density at the cellular level. Significant advances in hardware and software developments enabled the translation of quantitative PET imaging capabilities in clinical diagnosis, prognosis, and for outcome prediction (Czernin et al., 2007). However, the accuracy of PET image quality and quantitative accuracy is affected by several degrading factors, including the technological limitation of detection modules (spatial/temporal resolution), annihilation photons interaction (attenuation and scattering) within the patients' body, and the imperfections of reconstruction algorithms (Zaidi & Karakatsanis, 2018).

The potential advantages of Time‐of‐Flight (TOF) were realized during the early stages of PET instrumentation development. Yet, the limited performance of available scintillators in terms of decay time of the scintillating light did not enable their implementation (Melcher, 2000; Ter‐Pogossian et al., 1981). Hypothetically, ideal TOF information enables localization of the annihilation position along the Line of Response (LOR) without the need for image reconstruction (Karp et al., 2008; Kwon et al., 2021). In commercially available TOF PET scanners, arrival time differences between detected coincident photons are measured to estimate the localization information, hence enabling the reduction of noise propagation (Conti, 2011a). The main advantage of TOF PET imaging lies in the sensitivity gain, which is defined as the total count ratio of TOF PET versus non‐TOF PET when random noise is similar (Budinger, 1983). It has been demonstrated that TOF improved the quality of PET images depending on scanner's coincidence timing resolution and patient's size (Lois et al., 2010; Vandenberghe et al., 2016). A number of PET scanners using detector modules based on slow decay time scintillators (e.g., Bismuth Germanate ‐ BGO) are still being used in the clinic and there is an interest to use these relatively inexpensive crystals compared with Lutetium (Lu)‐based scintillators, such as LSO and LYSO on new generation PET scanners, provided they can provide similar image quality without TOF capability. As such, the capability of generating TOF from non‐PET images is of paramount importance to realize the full potential of this imaging modality.

The introduction of machine/deep learning algorithms in recent years has revolutionized medical imaging research, particularly in areas linked to human interpretation/intervention (e.g., segmentation, diagnostic, prognostic, radiomics, etc.) as well as other technical areas, including optimization of image acquisition, reconstruction, quantification, motion correction and image denoising (Akhavanallaf et al., 2021; Arabi et al., 2021; Arabi & Zaidi, 2021; Sanaat et al., 2020; Sanaat et al., 2022; Sanaat, Mirsadeghi, et al., 2021; Sanaat, Shiri, et al., 2021; Sanaat & Zaidi, 2020; Shiri et al., 2020; Zaidi & El Naqa, 2021).

Reader et al. reviewed the potential of deep learning algorithms in PET image reconstruction (Reader et al., 2021). A number of studies focused on resolution recovery of PET images using various convolutional neural network architectures (Hu et al., 2019; Song et al., 2020). Arabi and Zaidi (Arabi & Zaidi, 2020) proposed a novel method to predict attenuation correction factors from TOF information using a deep learning method. In another work Sanaat et al. used a similar approach for generating full‐dose TOF information from low‐dose TOF images/sinograms (Sanaat, Shooli, et al., 2021).

In this study, we introduce the concept of PET signal recovery through estimating TOF information from non‐TOF PET signal in both the image space (IS) and sinogram space (SS) domains. Thereby, a deep learning algorithm was developed to predict TOF PET images/sinograms from non‐TOF signals for ¹⁸F‐FDG brain PET/CT images. The same approach could be expanded for applications in cardiovascular or whole‐body PET imaging.

2. MATERIALS AND METHODS

The current study was applied on a dataset consisting of 140 ¹⁸F‐FDG brain PET/CT images acquired between June 2017 and May 2019 at Geneva University Hospital, Switzerland. The dataset contained 67 males (73 ± 9 yrs) and 73 females (72 ± 11 yrs) all with cognitive symptoms and possible neurodegenerative disease. Table 1 summarizes the demographic information of the dataset. The study protocol was approved by the institution's ethics committee and all patients gave written informed content. PET/CT imaging was performed on a Biograph mCT scanner (Siemens Healthcare, Erlangen, Germany) using the routine clinical protocol consisting of 20 min PET acquisition 35 min postinjection of 205 ± 10 MBq of ¹⁸F‐FDG. An ultra‐low dose CT scan (120 kVp, 20 mAs) was applied for attenuation correction. The data were acquired in list‐mode format. Then, the TOF sinogram was histogrammed using the e7‐tools (Siemens Healthcare) toolkit. The Biograph mCT PET scanner has a coincidence time resolution of ~530 ps (Jakoby et al., 2011) and its generated TOF sinogram contains 13 time bins (0, ±1, ±2, ±3, ±4, ±5, ±6) in a large matrix of 400 × 168 × 621 × 13. In most PET images, patients' head was located at the center of the field‐of‐view (FOV). However, some patients with off‐center head positioning were used to evaluate the performance of the model. By considering the average adult's head size and the scanner's time resolution, seven bins (0, ±1, ±2, ±3) were selected for the training and reconstruction (to avoid unnecessary processing time). For non‐TOF PET reconstruction, a sinogram containing the summation of the entire seven bins was generated. The reduced TOF bin sinograms were reconstructed in TOF and non‐TOF mode using the above‐mentioned toolkit with a Poisson ordered subsets‐expectation (OP‐OSEM) and point spread function modeling with five iterations and 21 subsets. Gaussian postreconstruction filtering with 2 mm FWHM was applied as in the clinical protocol. The reconstructed images have a matrix size of 200 × 200 × 109 and 2.03 × 2.03 × 2.2 mm³ voxel size.

TABLE 1.

Demographics of patients included in this study

	Training	Test
Number	120	20
Male/female	57/63	10/10
Age (Mean ± SD)	72 ± 8	69 ± 9
Indication/diagnosis	Cognitive symptoms of possible neurodegenerative a etiology

Following two scenarios, the performance of a cycle generative adversarial network (CycleGAN) for synthesizing the TOF information from non‐TOF data in the image and projection space (Figure 1) was investigated. In the image space implementation, we trained a CycleGAN to estimate TOF directly from non‐TOF PET images whereas implementation in the projection space involved the use of seven CycleGANs independently trained to generate different time bins from non‐TOF sinogram. The estimated TOF sinograms were reconstructed and evaluated versus the reference TOF PET images.

FIGURE 1.

Schematic diagram of TOF PET data generation models. In the first strategy (projection space), seven CycleGAN models were trained separately to generate the different TOF bin sinograms (−3, −2, −1, 0, 1, 2, 3) from non‐TOF sinograms. In the second strategy (image space), a CycleGAN model was trained to directly generate TOF images from non‐TOF PET images. In the lower panel, the generator and discriminator of the deep learning model is presented.

2.1. CycleGAN model

Style transformation is one of the attractive features in the field of deep learning‐assisted medical image analysis. The purpose of style transfer is mapping an image belonging to domain X to another image belonging to domain Y. CycleGAN is one of the well‐established architectures to translate domain X to Y while maintaining image consistency. The translated images should be similar to the original ones with some stylistic variation applied. Our optimized CycleGAN included a generator for learning a map from non‐TOF images/sinograms to TOF images/sinograms and a discriminator to evaluate the generated TOF images/sinograms. Since non‐TOF and TOF in both image and sinogram domains have almost similar structures, it is essential to consider a residual network as the generator of CycleGAN to learn based on residual images. This emphasizes the variance between non‐TOF and TOF images/sinograms, rather than the global image patterns. Overall, the goal of the generator is to synthesize a very precise representation of TOF images/sinograms (decrease the error between synthetic and actual images) to fool the discriminator network (increase the discriminator judgment error) by generating synthetic or cycle image/sinogram that are indistinguishable from the input images. The generator and discriminator networks were trained carefully to reach the plateau of loss error training. In this work, the number of convolution layers in discriminator architecture is nine, where eight layers are followed by batch normalization, the last layer is followed by a sigmoid function. The flowchart of this architecture is presented in Figure 1. The details of our models are as follows: Batch size = 20, Learning rate = 4 × 10⁻⁴, Linear decay from initial value to 10⁻⁷, number of epoch = 288, lambda = 10.0, beta_1 = 0.5 and beta_2 = 0.999 for Adam optimizer, loss function of discriminator and generator = Mean Absolute Error, L1 Loss.

A number of pre‐developed CycleGAN models are implemented in different libraries, such as PyTorch and Keras and Tensorflow backend. Inspired by the work of (Zhu et al., 2017), we consider all of these libraries, and developed/optimized a new code on Keras. Supplemental Table 1 summarizes the network's layers details for both generator and discriminator. The model and hyperparameter tuning was one of the main challenges in this study. Both were optimized based on our previous judgment/experience (Sanaat, Shooli, et al., 2021). The model accuracy was evaluated and listed for each set of hyperparameters and then the sets with the lowest error were selected.

2.2. Image normalization

Since retaining the quantitative aspect of PET images when using deep learning models is one of the most critical parts in model development, our model was trained to estimate the standardized uptake values (SUVs)‐based images. For image normalization, we converted the intensity values to SUV, and then the patient with maximum SUV among all patients was found and all images divided by this number. After model training and testing on unseen datasets, the images were multiplied by the constant normalization factor to recover the original SUVs.

Model training and evaluation was performed on a NVIDIA 2080Ti GPU with 11 GB memory running under windows 10 operating system. We did not use cross‐validation because the training process was time consuming and the sinogram matrix size very large.

2.3. Quantitative analysis

The performance of our developed models was assessed through quantitative analysis of the test dataset. Well‐established quantitative metrics, including the root mean squared error (RMSE), peak signal‐to‐noise ratio (PSNR), and structural similarity index metrics (SSIM) were calculated for non‐TOF, and predicted TOF PET images in IS and SS with respect to reference TOF PET images.

In image to image translation tasks, when the goal is to improve image quality, it is important to have an insight into the differences between low‐ and high‐quality images. Hence, the mentioned parameters were calculated for non‐TOF PET images to set a baseline regarding the extent of the model capability in improving signal properties and spatial resolution.

To assess the performance of our models for lesion detectability, the SNR, contrast and noise were estimated for reference TOF, non‐TOF, and predicted TOF in IS, and SS. The SNR is defined as the SUV_mean difference between volumes of interest (VOIs) drawn on lesions/hot‐spots (VOI_hot‐spot) and background (VOI_background) divided by the noise in the background. In this definition, the noise is regarded as the standard deviation (SD) of VOI_background and VOI_background was defined in a uniform area outside of the lesion/background.

$$\mathrm{SNR}=\frac{{\mathrm{VOI}}_{\mathrm{hot}–\text{spot}}-{\mathrm{VOI}}_{\text{background}}}{\mathrm{SD}\left({\mathrm{VOI}}_{\text{background}}\right)}$$ (1)

Contrast resolution is crucial in brain PET imaging because it can help in distinguishing different brain structures, those correlated with the presence of neurodegenerative disease and healthy membranes from cancer cells. The contrast was calculated using the following formula to assess the impact of TOF PET capability.

$$\text{Contrast}=\frac{{\mathrm{VOI}}_{\mathrm{hot}-\text{spot}}}{{\mathrm{VOI}}_{\text{background}}}$$ (2)

The statistical noise inherently present in PET images degrades image quality and might lead to wrong clinical decisions. TOF PET imaging could enhance image quality by reducing the noise. The noise was estimated to assess the performance of our models in terms of noise reduction.

$$\text{Noise}=\frac{\mathrm{SD}\left({\mathrm{VOI}}_{\text{background}}\right)}{{\mathrm{VOI}}_{\text{background}}}$$ (3)

Finding a reasonably homogeneous region close to lesion/hot‐spot for background VOI is not straightforward and could be considered as one of the limitations of our work. Therefore, we selected the center of the brain (ventricle region) as background where the uptake is mostly uniform.

Twenty‐eight radiomic features were calculated for 83 brain regions to assess the agreement of radiotracer uptake between predicted IS, SS, and TOF PET images. The PMOD medical image analysis software (PMOD Technologies LLC) was used to normalize and register TOF, non‐TOF, IS, and SS to a standard ¹⁸F‐FDG brain template. Subsequently, the LIFEx analysis toolkit was used to quantify 28 radiometric features, including seven conventional indices, five first‐order features, six gray‐level run‐length matrix features, three gray‐level co‐occurrence matrix features, and seven gray‐level zone length matrix features (Table 2). The heat map of the relative error was produced based on Equation 4 for non‐TOF, IS, and SS by considering TOF PET image as reference.

$$\mathrm{\delta }\left(\%\right)=\frac{\mathrm{\alpha }-\mathrm{\beta }}{\mathrm{\beta }}$$ (4)

In Equation (4), $\alpha$ and $\beta$ denote the actual and predicted value of a specific radiomic feature calculated in a brain region, respectively. A pairwise t‐test was calculated for statistical analysis of PSNR, RMSE, and SSIM using the MedCalc software (Schoonjans et al., 1995). The significance level was set at a p value <0.05 for all comparisons.

TABLE 2.

Summary of the 28 radiomic features belonging to the 6 main categories estimated for the 83 brain regions

Radiomic feature category	Radiomic feature names
Conventional indices	SUVmean
	SUVstd
	SUVmax
	SUV Q1
	SUV Q2
	SUV Q3
	TLG (ml)
First order features—histogram	Kurtosis
	Entropy_log10
	Entropy_log2
First order features—shape	SHAPE_Volume (ml)
	SHAPE_Volume (# Voxel)
Grey‐level zone length matrix (GLZLM)	Short‐zone emphasis (SZE)
	Long‐zone emphasis (LZE)
	Short‐zone low gray‐level emphasis (SZLGE)
	Short‐zone high gray‐level emphasis (SZHGLE)
	Long‐zone low gray‐level emphasis (LZLGLE)
	Long‐zone high gray‐level emphasis (LZHGLE)
	Zone percentage (ZP)
Grey‐level run length matrix (GLRLM)	Short‐run emphasis (SRE)
	Long‐run emphasis (LRE)
	Short‐run low gray‐level emphasis (SRLGLE)
	Short‐run high gray‐level emphasis (SRHGLE)
	Run length non‐uniformity (RLNU)
	Run percentage (RP)
Grey‐level co‐occurrence matrix (GLCM)	Homogeneity
	Energy
	Dissimilarity

3. RESULTS

The qualitative visual inspection of the results demonstrated the acceptable performance of the deep learning model for generating artifact‐free, high‐quality TOF from non‐TOF PET images both in image and projection space. Figure 2 shows transverse, coronal, and sagittal views of reference and predicted brain PET images along with their corresponding bias maps. Although the predicted images (IS and SS) exhibited good image quality, overestimation of tracer uptake (positive bias) was observed between IS and SS compared with TOF PET. The qualitative assessment was supported by quantitative evaluation through calculating RMSE, PSNR, and SSIM (Table 3). The signal enhancement, noise reduction, and quality improvement are higher for implementation in image space compared with the one in projection space.

FIGURE 2.

Representative ¹⁸F‐FDG brain PET images of a 49‐year‐old male patient. (a) The reference TOF image and the corresponding (b) Non‐TOF image and predicted TOF images in (c) the image space (IS) and (d) sinogram space (SS) are presented. The SUV bias maps for (e) non‐TOF, (f) IS and (g) SS PET images with respect to the reference TOF PET image are also shown.

TABLE 3.

Comparison of the results obtained from image quality assessment in non‐TOF and predicted TOF images in image (IS) and sinogram (SS) space for the validation dataset

Dataset	SSIM	PSNR	RMSE
non‐TOF	0.97 ± 0.03	72.10 ± 2.82	0.22 ± 0.12
Predicted TOF PET in image space (IS)	0.99 ± 0.03	84.25 ± 3.75	0.12 ± 0.09
Predicted TOF PET in sinogram space (SS)	0.98 ± 0.02	81.30 ± 3.92	0.16 ± 0.04
p value (IS vs. SS)	<0.05	<0.02	<0.05
p value (IS vs. non‐TOF)	<0.02	<0.01	<0.01
p value (SS vs. non‐TOF)	<0.02	<0.02	<0.02

Abbreviations: PSNR, peak signal to noise ratio; RMSE, root mean squared error; SSIM, structural similarity index metrics.

Figure 3 illustrates an example of the beneficial impact of TOF PET imaging, where the hotspots/lesions can be overlooked completely or partially in non‐TOF PET images. Both predicted TOF images (IS and SS) were successful in extracting the information missed or hidden among the noise in non‐TOF PET images.

FIGURE 3.

A representative clinical study showing a 59‐year‐old male patient displaying the patterns of detailed anatomical structures revealed by: (a) TOF and (b) non‐TOF, and the predicted TOF PET images for (c) IS and (d) SS models. The images exhibited promising performance to capture/reveal detailed brain structures.

Since TOF information has a direct influence on lesion‐to‐background ratio, both predicted TOF PET images (IS and SS) showed a larger putamen‐to‐background ratio compared with non‐TOF image (Figure 4), with IS providing slightly higher contrast. Figure 5 depicts the correlation between ¹⁸F‐FDG tracer uptake in non‐TOF, IS, and SS vs. TOF PET images based on pixelwise linear regression analysis. The data points scatter distribution decreased from non‐TOF to SS and then IS and the correlation and error level enhanced for SS (R ² = 0.98, MSE = 0.025) compared with non‐TOF (R ² = 0.96, MSE = 0.048). The predicted image in image space achieved the highest correlational and lowest deviation from the identity line (R ² = 0.99, MSE = 0.014). The bias and variance of non‐TOF and predicted TOF PET images in the 83 brain regions were evaluated by Bland & Altman analysis (Figure 6). Each single black point denotes a brain region for each patient. The range of variance gradually decreases from non‐TOF (95% CI: −0.58%, +0.42) to TOF PET SS (95% CI: −0.37%, +0.28%) and IS (95% CI: −0.21%, +0.17%), perfectly supporting the joint histogram analysis results. The average brain region's tracer uptake bias follows the same trend and decreased from −0.058% for non‐TOF to −0.043% for SS, and −0.026% for IS.

FIGURE 4.

Plots of average Putamen‐to‐background ratio calculated across the entire test dataset.

FIGURE 5.

The joint histogram analysis for non‐TOF (left), IS (midfle), and SS (right) brain PET images versus TOF PET images.

FIGURE 6.

Bland & Altman plots of SUV differences in the 83 brain regions calculated for non‐TOF (left), IS (middle) and SS (right) PET images with respect to the reference TOF PET images in the test dataset. The dashed blue and solid red lines denote the mean and 95% confidence interval (CI) of the SUV differences, respectively.

The SNR, contrast, and noise show substantial improvement in reference TOF and predicted IS and SS TOF PET compared with non‐TOF PET images (Table 4). In particular, the reference TOF, IS, and SS improved the SNR by about 10%, 7%, and 3%, while they reduced the noise by 22%, 16%, and 7%, respectively. The contrast was enhanced by 15%, 13%, and 4% for TOF, IS, and SS, respectively.

TABLE 4.

The SNR, contrast, and noise calculated for malignant lesions across the entire test group

Parameter	Non‐TOF	TOF	IS	SS
SNR	32.34 ± 29.21	35.67 ± 37.18	34.76 ± 31.74	33.55 ± 25.11
Contrast	3.22 ± 2.51	3.71 ± 3.26	3.65 ± 3.10	3.34 ± 0.41
Noise	0.31 ± 0.12	0.24 ± 0.27	0.26 ± 0.1	0.29 ± 0.33

Figure 7 and Supplemental Figure 1 and 2 depict the relative error (%) heat map of 28 radiomic features calculated for 44 brain regions (the symmetric left and right sides of the 83 regions were merged for better illustration) belong to the test dataset for predicted images in image and sinogram space, and non‐TOF. The mean relative errors of all the radiomic features calculated across all brain regions were <16.6%, 13.7%, and 11.8% for non‐TOF, SS, and IS, respectively. The large radiomics bias was mostly observed in regions with relatively large distances from the center of the brain where the effect of TOF information is more visible. This pattern was observed in other predicted TOF images. For instance, SUV_mean bias for medial and lateral orbital gyrus was around 1% and 7.7% for non‐TOF and 0.5% and 5.5% for IS, respectively. Other regions close to the center of the brain with low bias (under 5% for non‐TOF) were straight gyrus, subcellular area, parietal gyrus, caudate, etc. the high bias regions (>8%) are temporal lobe, cerebellum, frontal horn, etc. The highest of the homogeneity radiomic feature belonging to the gray‐level co‐occurrence matrix category was 10.6%, 13.7%, and 13.9% for IS, SS, and non‐TOF, respectively. The mean of all radiomic features is 3.6%, 5%, and 6.2% for IS, SS, and non‐TOF, respectively. The heat map also confirmed the overestimation of tracer uptake by the proposed model.

FIGURE 7.

Heat map of the relative error of the 28 radiomic features calculated across 83 brain regions for predicted IS TOF PET images with respect to reference TOF PET images.

The average SUV bias calculated across 83 brain regions confirmed the higher systematic overestimation bias for non‐TOF, SS, and IS, respectively. The results indicated that our developed models are capable of improving the average SUV_mean bias from 9.3 ± 2.21% for non‐TOF to 6.9 ± 0.08% and 6.2 ± 0.91% for predicted images in SS and IS, respectively.

4. DISCUSSION

The TOF information boost the informative signals relative to statistical noise in PET images. A number of studies comparing TOF and non‐TOF PET reported that TOF information improves the spatial resolution and SNR, and enhances contrast recovery for malignant lesions. Hence, improvement of SNR opens the implementation of additional options, such as reduction of the injected dose and scanning time (Lois et al., 2010).

Although the impact of TOF is more visible in regions residing far from the center of the FOV and could be more relevant in whole‐body PET imaging, its influence is still noticeable in brain imaging, especially in the presence of artifacts or misalignment between anatomical and functional images. Yoshida et al. reported a significant improvement in image quality for a brain‐sized phantom (Hoffman phantom) in TOF relative to non‐TOF imaging (Yoshida et al., 2020). It is noteworthy to point out that the test dataset used in this in this study was carefully selected to encompass images with relatively large misalignments from the center of the FOV to highlight the relevant impact of TOF information. The heat map revealed that brain regions with a large distance from the axis of the scanner had a larger bias compared with regions located close to the axis. These results are in good agreement with Budinger's rule which expresses the relation between uptake position and improvement of SNR (Budinger, 1983). According to this rule, the SNR gain is related to $\sqrt{\frac{2D}{∆T\times 0.03}}$ where D is the diameter of the positron‐emitting distribution in cm (in our case, the diameter of the head) and $∆T$ is the scanner's coincidence timing resolution, about 530 ps for the Siemens mCT Biograph scanner. By assuming an adult's head diameter about 20 cm, we can expect an SNR improvement of about a factor of 1.58 for the Biograph mCT having a TOF resolution of 530 ps and a gain factor of 2.51 for the Biograph Vision with a TOF resolution of 210 ps (van Sluis et al., 2019).

In this study, we suggested a model for improving non‐TOF PET image quality by incorporating TOF information artificially. Our model was trained in both image and projection space, which renders the model's outcome flexible as it leaves open the choice of image reconstruction protocol. To the best of our knowledge, there is no similar work enabling to generate TOF sinograms from non‐TOF sinograms that we can compare our results with. Recently a commercially available deep learning model, referred to as SubtlePET, was employed on non‐TOF ¹⁸F‐FDG PET images with 33% less injected activity on an analog PET/CT scanner without TOF capability to generate equivalent standard dose non‐TOF PET images (Katsari et al., 2021). Another study reported promising results for fast TOF and non‐TOF PET scanning with 75% scanning time reduction (Chaudhari et al., 2021). More recently, Mehranian et al. independently reported a non‐TOF to TOF PET conversion of whole‐body images implemented in image space, demonstrating promising results in to improve noise degradation, image sharpness, and diagnostic value (Mehranian et al., 2022). This work did not consider the specificity of brain PET scanning and the developed model was limited to image space implementation, which prevents the possibility to use the desired image reconstruction protocol.

Detailed visual assessment of our test dataset revealed that in some cases (8 from 20) the hotspots/lesions can be missed/overlooked partially or completely on non‐TOF PET images while they were depicted on TOF PET images. Both IS and SS models successfully depicted the missed lesions/hotspots. Furthermore, the anatomical structures, such as the pattern of the gyrus, were better portrayed in the predicted TOF relative compared with corresponding non‐TOF PET images.

The Bland–Altman analysis supported the scatter plot results where the model trained in the image space led to better correlation, lowest bias, and variance relative to the model trained in the sinogram space. The lower performance of the model in the projection space can be attributed to the number of trained models where the summation of all model errors and more importantly the inherent difference between the input (single non‐TOF sinogram) and the output (several TOF bin sinograms) and the uncertainties associated with synthesizing the corner's bin which is more noisy in the non‐TOF sinogram (synthesis of seven separate TOF bins from non‐TOF). In other words, in sinogram space, our model was trained to project non‐TOF sinogram to a number of TOF bin sinograms and since the off‐center TOF bins are different from the non‐TOF sinogram, it can lead to large errors during training. It is worth highlighting that in previous work, we trained seven DNN models for synthesizing full‐dose TOF bin sinograms from their corresponding low‐dose TOF bin sinograms, and compared its performance with a similar DNN model generating full‐dose PET images from low‐dose PET images (Sanaat, Shooli, et al., 2021). In this work, each model was trained with a specific sinogram (low‐dose TOF bins 0, 1, 2, 3 to full dose TOF bins 0, 1, 2, 3). The results achieved by model implementation in the projection space were superior to the implementation in the image domain.

In dedicated brain PET scanners, the parallax error plays a significant role in spatial resolution degradation owing to because of the small diameter of the gantry. Hence, there has always been a desire to have both depth of interaction (DOI) and TOF capabilities, although the tradeoff between DOI capability and TOF power makes this task more complicated. Detector modules using light sharing can lead to a degradation in TOF performance.

Artifact reduction is one of the sidelong advantages of TOF capability (Conti, 2011b), which proved useful in brain and whole‐body imaging. Voert et al. reported that PET image artifacts can significantly reduce with consideration of TOF information. The role of TOF was observed predominantly in patients with dental filling (Ter Voert et al., 2017).

The main difference between TOF information in the image and projection domains is the way they represent the additional information. In the TOF sinogram, we have access to the spatial bins. For instance, in our study the TOF sinogram of the Biograph mCT (~530 ps TOF resolution) contained seven time bins (0, ±1, ±2, ±3), meaning that we have access to 7 separate spatial bins along the line of response with a size of about 5.38 cm. When the TOF sinogram is reconstructed, the information of all seven TOF bins will be merged to generate the final image and the initial TOF information lost. The TOF information in the projection domain is reflected through assigning the coincidence events to the different TOF bins, which represent the approximate spatial location/origin of the events. Conversely, the TOF information in the image domain would be reflected in higher signal‐to‐noise ratio, reduced noise levels, better convergence, and robustness to inconsistent data. In this regard, we hypothesized that we would achieve better performance by training seven separate models to generate the TOF bins from a non‐TOF sinogram compared with training a single model to generate TOF images from non‐TOF images. Yet, the model trained in image space outperformed the one trained in projection space. A plausible explanation for this observation is that the problem (mapping non‐TOF data to TOF information) is much easier to solve in the image domain than in the projection space since there is a one‐to‐one (image‐to‐image) correspondence in the image domain. We believe that when we train seven models, each producing a certain magnitude of error, the final error resulting from summing up the errors of all models is higher than a single model in the image space.

Furthermore, generating the different time bins from a non‐TOF sinogram might wrongly propagate the information of a certain region to another one. The results are better for implementation in image space, but the sinogram space provides the freedom of reconstructing the generated TOF sinograms with any desired reconstruction algorithm, which compensates for the slight difference in results. Therefore, both strategies can be useful depending on the targeted application.

The extraction of TOF data from non‐TOF data would not be achievable for high TOF resolution data since a single non‐TOF sinogram should be translated into many TOF bin sinograms. For instance, for a TOF resolution of 240 ps, 23 sinogram bins should be generated from a single non‐TOF data, which is memory and computationally demanding. However, in the image domain, owing to the one‐to‐one correspondence, non‐TOF PET images could be translated into a corresponding image reconstructed with extremely high TOF resolution. There is no limit to the implementation of extremely high TOF resolution models in the image domain. Another avenue to explore would be deep learning‐based PET reconstruction using for instance the model developed by Whiteley et al. (2020) wherein the non‐TOF PET data (in the sinogram domain) could be directly translated into the image space. In this regard, ideally reconstructed PET images (for instance using extremely high TOF resolution) could be considered as target for model training. Owing to the fixed size of the input data in the form of non‐TOF data and the small size of the output image (ideally reconstructed PET image), there is no memory or computational limitations for this solution.

Among the limitations of this study is that the model was evaluated on brain PET images while TOF capability is more relevant in whole‐body PET imaging. This limitation is challenging to address because access to whole‐body TOF bin sinograms is demanding owing to their large size in whole‐body imaging. Future work will address this limitation in the image space.

Our method can be applied on any PET scanner without TOF technology, such as analog PET scanners and BGO‐based scanners and even PET scanners with low TOF time resolution. The availability of this technique might revive the interest in commercial deployment of low‐cost standalone BGO‐based PET scanners, particularly in developing countries. Furthermore, this study opened the way toward TOF or high time resolution TOF enhancement through deep learning in both image and projection space. For instance, PET images with low TOF time resolution can be generated through simulations and used as input to train a deep learning model to generate high TOF time resolution PET images from the low resolution TOF PET images corresponding to currently available TOF resolution PET scanners. Among the limitations of this study is that the model was evaluated on brain PET images while TOF capability is more relevant in whole‐body PET imaging. This limitation is challenging to address because access and handling of whole‐body TOF bin sinograms is computationally and memory demanding owing to their large size. Future work might address this limitation in image space.

It should be noted that for the implementation in the image domain, the improvement of the SNR, contrast and noise of non‐TOF images was achieved by training the network on high‐SNR TOF images. Yet, there is a lack of specific metrics enabling to demonstrate that the model fully “recovered” valid TOF information. For example, the SNR performance could come from denoising features of machine learning methods and even conventional techniques. The metrics evaluated in this study (in the image domain) are not solely dependent on TOF information. Other factor, such as image reconstruction algorithm, hyperparameters, and postreconstruction filters might have an impact.

5. CONCLUSION

We developed a CycleGAN model capable of learning a non‐linear transformation to generate TOF from non‐TOF PET images, hence improving image quality and spatial resolution in addition to enhancing lesion detectability of PET images acquired on conventional inexpensive non‐TOF PET scanners. In this regard, a technological gap in conventional PET scanners can be addressed through enabling the prediction TOF PET images from non‐TOF scanners to achieve a higher image quality without hardware upgrades. Our developed model is able to transform non‐TOF PET sinograms to seven TOF bins allowing the reconstruction of the synthesized sinograms using any desired reconstruction as well as pre‐ and postreconstruction filtering. Quantitative evaluation revealed the slightly better performance of the model in the image space compared with the projection space. Nevertheless, the TOF model in the projection space would be useful for applications where different reconstruction algorithms, image correction modeling (such as point spread function modeling), and attenuation/scatter correction are required to be implemented.

AUTHOR CONTRIBUTIONS

Amirhossein Sanaat and Habib Zaidi contributed to the study conception and design. Amirhossein Sanaat designed, implemented, and evaluated the proposed technique and deep learning framework. Amirhossein Sanaat, Azadeh Akhavanalaf, Isaac Shiri, Yazdan Salimi, Hossein Arabi and Habib Zaidi collated the data set and performed data analysis. Amirhossein Sanaat and Habib Zaidi contributed to the initial draft of the manuscript. All authors contributed to the data preparation and revision of the manuscript for important content.

Supporting information

APPENDIX S1 Supporting Information

Sanaat, A. , Akhavanalaf, A. , Shiri, I. , Salimi, Y. , Arabi, H. , & Zaidi, H. (2022). Deep‐TOF‐PET: Deep learning‐guided generation of time‐of‐flight from non‐TOF brain PET images in the image and projection domains. Human Brain Mapping, 43(16), 5032–5043. 10.1002/hbm.26068

DATA AVAILABILITY STATEMENT

The clinical studies used in this work are not available.