Neural MLAA for PET-enabled Dual-Energy CT Imaging

Abstract

The PET-enabled dual-energy CT method allows dual-energy CT imaging on PET/CT scanners without the need for a second x-ray CT scan. A 511 keV γ-ray attenuation image can be reconstructed from time-of-flight PET emission data using the maximum-likelihood attenuation and activity (MLAA) algorithm. However, the attenuation image reconstructed by standard MLAA is commonly noisy. To suppress noise, we propose a neural-network approach for MLAA reconstruction. The PET attenuation image is described as a function of kernelized neural networks with the flexibility of incorporating the available x-ray CT image as anatomical prior. By applying optimization transfer, the complex optimization problem of the proposed neural MLAA reconstruction is solved by a modularized iterative algorithm with each iteration decomposed into three steps: PET activity image update, attenuation image update, and neural-network learning using a weighed mean squared-error loss. The optimization algorithm is guaranteed to monotonically increase the data likelihood. The results from computer simulations demonstrated the neural MLAA algorithm can achieve a significant improvement on the γ-ray CT image quality as compared to other algorithms.

1. INTRODUCTION

Conventionally, dual-energy computed tomography (DECT) uses two different x-ray energies to obtain energy-dependent tissue attenuation information to allow quantitative material decomposition.¹ DECT can be implemented with simultaneous data acquisition using a dedicated DECT scanner or sequential data acquisition on conventional single-energy CT scanners. Combined use of DECT and positron emission tomography (PET) functional imaging provides a multi-parametric characterization of disease states in cancer and other diseases. It will be valuable to enable simultaneous PET/DECT imaging for clinical applications. However, the integration of DECT with PET would not be trivial, either requiring costly CT hardware upgrade or significantly increasing CT radiation dose.

We have proposed a new dual-energy CT imaging method that is enabled using standard time-of-flight PET/CT scans without change of scanner hardware or adding additional radiation dose or scan time.^2,3 Instead of using two different x-ray energies, our PET-enabled dual-energy CT method combines a high-energy “γ-ray CT (GCT)” at 511 keV with the low-energy x-ray CT (usually ≤ 140 keV) to produce a pair of dual-energy CT images for multi-material decomposition. Note that the GCT is not acquired using an external radiation source but the internal γ-rays generated by annihilation radiation of PET radiotracer decays in a subject. The reconstruction of GCT can be achieved by the maximum likelihood attenuation and activity (MLAA) estimation from the PET emission scan.⁴

However, the GCT image reconstructed by standard MLAA is commonly noisy. The noise may affect the quantitative accuracy of GCT for multi-material decomposition. To suppress noise, the kernel MLAA approach has been developed by use of x-ray CT as image prior and has demonstrated substantial improvements,³ but can be further improved. It is worth noting that all the existing kernel methods for image reconstruction (e.g.,^5–7) follow an unregularized formulation, while the kernel methods in machine learning for classification or regression commonly use a regularized framework.⁸ Adding a penalty function in the kernel method for image reconstruction may be helpful to stabilize the solution but will make the reconstruction formulation more complicated and result in a more challenging optimization problem. A regularization-based method also often requires a convergent solution to achieve the optimal solution, which can be computationally challenging in the MLAA framework.

Instead of using any explicit penalty function, in this paper we propose to use a neural network representation to impose an implicit regularization to the kernel method for MLAA reconstruction. The ML estimation of kernel coefficients then becomes the ML estimation of neural network weights from raw projection data. This usage of neural networks shares the same spirit as the work of Gong et al.⁹ for PET activity image reconstruction based on the deep image prior method.¹⁰

One challenge with this kind of neural network reconstruction problem is the complex optimization because the forward model is now nonlinear. The neural network learning and tomographic reconstruction are also coupled with each other in the projection domain. Gong et al.⁹ used the alternating direction of multiplier method (ADMM) to decouple the network training and image reconstruction steps. However, the hyper parameters associated with an ADMM algorithm is challenging to tune in practice. In this work, we apply the optimization transfer principle to obtain a modularized optimization algorithm that can be easily implemented and does not involve hyper-parameter tuning, in the same spirit as the work of Wang and Qi on parametric PET image reconstruction.¹¹

As the kernel representation in this work can also be deemed as a neural network layer with predetermined weights (see Fig. 1 for illustration), we call the proposed algorithm the neural MLAA to feature its nature of neural network representation and differentiate it from the original kernel MLAA.³

Figure 1:

Graphical illustration of a neural network representation of the attenuation image μ for MLAA reconstruction.

2. PET-ENABLED γ-RAY CT RECONSTRUCTION BY MLAA

2.1. Statistical Model

In time-of-flight (TOF) PET, the expectation of the PET projection data is related to the radiotracer activity image λ and object attenuation image μ at 511 keV via

$${\overline{y}}_m(\lambda ,\mu )=\text{diag}\left\{n_m(\mu )\right\}{G}_m\lambda +r_m,$$ (1)

where m denotes the mth TOF bin and G_m is the PET detection probability matrix. r_m accounts for the expectation of random and scattered events. n_m(μ) is the normalization factor with the ith element being

$$n_{i,m}(\mu )=c_{i,m}\cdot \text{exp}{\left(-[A\mu ]\right)}_i,$$ (2)

where c_i,m represents the multiplicative factor excluding the attenuation correction factor and A is the system matrix for transmission imaging.

The TOF PET measurement y can be well modeled as independent Poisson random variables using the log-likelihood function,

$$L(y\mid \lambda ,\mu )=\sum _{i=1}^{N_d}\sum _{m=1}^{N_t}{y}_{i,m}\text{ log }{\overline{y}}_{i,m}(\lambda ,\mu )-{\overline{y}}_{i,m}(\lambda ,\mu ),$$ (3)

where i denotes the index of PET detector pair and N_d is the total number of detector pairs. N_t is the number of TOF bins.

2.2. MLAA Reconstruction

The maximum-likelihood attenuation and activity (MLAA⁴) reconstruction jointly estimates the attenuation image μ and the activity image λ from the projection data y by maximizing the Poisson loglikelihood,

$$\widehat{\lambda },\widehat{\mu }={\text{arg max}}_{\lambda \ge 0,\mu \ge 0}L(y\mid \lambda ,\mu ).$$ (4)

An iterative interleaved updating strategy is commonly used to seek the solution. In each iteration of the algorithm, λ is first updated with fixed attenuation image $\widehat{\mu }$,

$$\widehat{\lambda }=\text{arg }\underset{\lambda \ge 0}{\text{max}}L(y\mid \lambda ,\widehat{\mu }).$$ (5)

which can be solved by the maximum-likelihood expectation maximization (MLEM) algorithm.¹² Next, μ is updated with fixed λ using the following kernel maximum-likelihood transmission reconstruction (MLTR) method,

$$\widehat{\mu }=\text{arg }\underset{\alpha \ge 0}{\text{max}}L(y\mid \widehat{\lambda },\mu ),$$ (6)

which can be solved by the separable paraboloidal surrogate (SPS) algorithm.¹³

In the PET-enabled dual-energy CT method,^2,3 the estimated GCT image μ is combined with the x-ray CT image to form a pair of dual-energy CT images for multi-material decomposition.

3. PROPOSED NEURAL MLAA

3.1. Image Representation Using Kernelized Neural Networks

Following the kernel method for image reconstruction,⁵ the attenuation image μ can be described as a linear function of kernels,

$$\mu =K\alpha$$ (7)

where α is the kernel coefficient image of the same size as μ. The kernel matrix K is sparse and generated from image prior, e.g., the x-ray CT image x.³ Substituting the kernel model into Eq. 4 leads to the kernel MLAA algorithm.³ It is also possible to add an explicit penalty function to stabilize the estimation of α as indicated in the original kernel method,⁵ which however may result in a more challenging optimization problem and add one more hyperparameter to tune.

Inspired by the work of Gong et. al.⁹ using the deep image prior (DIP),¹⁰ we further describe the kernel coefficient image α as a function of neural networks,

$$\alpha =f(\theta \mid z)$$ (8)

where f denotes the neural network mapping from an input image z to the coefficient image α with θ the parameters of the neural network.

Combining the neural network model and kernel model together, we have a kernelized neural network representation to describe the GCT image,

$$\mu =Kf(\theta \mid z).$$ (9)

Fig. 1(a) shows a graphical illustration of the overall network model, of which the last layer has fixed weights as determined by the sparse matrix K. If the sub network f(θ|z) is an identity mapping, i.e., α = z, then the neural MLAA proposed in this paper is the same as the kernel MLAA.³ Note that f(θ|z) can be a more complex neural network model such as the residual U-net⁹ as shown in Fig. 1(b), which in turn introduces an implicit regularization to stabilize the estimation of the kernel coefficient image α.

It is worth noting that f(θ|z) can be a pretrained model by which θ is learned and fixed and z is to be estimated in image reconstruction. It can also be used without pretraining in the same way as the DIP model.^9,10 In this work, we focus on the latter. z is fixed and θ is to be estimated.

3.2. Incorporation into MLAA

Substituting Eq. (9) into the MLAA formulation gives the following neural MLAA formulation,

$$\widehat{\lambda },\widehat{\theta }={\text{arg max}}_{\lambda \ge 0,\theta \ge 0}L(y\mid \lambda ,Kf(\theta \mid z)),$$ (10)

with z either an image of random noise or the x-ray CT image x. Similar to the standard MLAA⁴ and kernel MLAA,³ an interleaved updating strategy is used to update λ and θ.

In each iteration, the λ-step remains the same as other MLAAs given an known $\widehat{\mu }$. The θ-step is a neural MLTR step that finds the ML solution of the neural network weights by

$$\widehat{\theta }=\text{arg\hspace{0.17em}}\underset{\theta }{\text{max}}\underset{\widehat{J}(\theta )}{\underbrace{L(y\mid \widehat{\lambda },Kf(\theta \mid z))}}.$$ (11)

where we use a simpler notation $\widehat{J}(\theta )$ to refer to the objective function.

Once $\widehat{\theta }$ is estimated, the GCT image is obtained by

$$\widehat{\mu }=Kf(\widehat{\theta }\mid z).$$ (12)

3.3. Neural Network Reconstruction Using Optimization Transfer

The optimization problem defined by Eq. (11) for neural network reconstruction is challenging to solve because the unknown θ is non-linearly involved in the projection domain. Here we develop an optimization transfer algorithm using a similar concept as used for parametric PET image reconstruction.¹¹

Our method transfers the complex optimization into a learning-friendly weighted mean squared-error (wMSE) formulation:

$${\theta }^{n+1}=\text{arg }\underset{\theta }{\text{min}}\text{\hspace{0.17em}wMSE}\left(\theta ;{\theta }^n\right),$$ (13)

where

$$\text{wMSE}\left(\theta ;{\theta }^n\right)\triangleq {\Vert {\widehat{\alpha }}^{n+1}-f(\theta \mid z)\Vert }_{{\omega }^n}^2,$$ (14)

with

$${\widehat{\alpha }}^{n+1}={\alpha }^n+\frac{{\mathit{g}}^n}{{\omega }^n}$$ (15)

and αⁿ ≜ f(θⁿ|z). The gradient image gⁿ and weight image ωⁿ are calculated based on αⁿ and y.

Note that the surrogate function wMSE(θ;θⁿ) is not arbitrarily defined but minorizes the original likelihood function $\widehat{J}(\theta )$:

$$\text{wMSE}\left(\theta ;{\theta }^n\right)-\text{wMSE}\left({\theta }^n;{\theta }^n\right)\le \widehat{J}(\theta )-\widehat{J}\left({\theta }^n\right),$$ (16)

$${\nabla }_{\theta }\text{wMSE}\left({\theta }^n;{\theta }^n\right)={\nabla }_{\theta }\widehat{J}\left({\theta }^n\right).$$ (17)

where ∇_θ denotes the gradient with respect to θ.

Thus, each iteration of the neural MLTR optimization can be decomposed into two separable steps:

• Step 1: Obtain an intermediate image update ${\widehat{\alpha }}^{n+1}$ from the projection data y using Eq. (15);

• Step 2: Perform a DIP-like neural network learning of ${\widehat{\alpha }}^{n+1}$ using the wMSE loss.

Step 1 can be implemented in a conventional image reconstruction platform, while Step 2 is fairly easy to implement in a deep-learning platform such as the PyTorch.

The algorithm is guaranteed to monotonically increase the original MLAA likelihood function, i.e.,

$$L\left(y\mid \widehat{\lambda },Kf\left({\theta }^{n+1}\mid z\right)\right)\ge L\left(y\mid \widehat{\lambda },Kf\left({\theta }^n\mid z\right)\right).$$ (18)

Compared with a possible ADMM optimization algorithm as used in,⁹ the proposed optimization transfer algorithm has the advantage that does not involve hyper parameter tuning.

4. SIMULATION RESULTS

4.1. Simulation Setting

We simulated a GE Discovery 690 PET/CT scanner in 2D. The TOF timing resolution of this PET scanner was 550 ps. The true PET activity image and 511 keV attenuation image are shown in Fig. 2(a) and (b), respectively. The images were first forward projected to generate noise-free sinogram of 11 TOF bins. A 40% uniform background was included to simulate random and scattered events. Poisson noise was then generated using 5 million expected events. The x-ray CT image at a low-energy 80 keV is shown in Fig. 2(c).

Figure 2:

The digital phantom used in the PET/CT computer simulation. (a) PET activity image in Bq/cc; (b) PET attenuation image at 511 keV in cm⁻¹; (c) x-ray CT image at 80 keV.

4.2. Reconstruction Methods for Comparison

Four types of MLAA reconstruction algorithms were compared, which includes (1) standard MLAA, (2) kernel MLAA with x-ray CT as image prior, and (3–4) proposed neural MLAA with and without kernels. Note that the neural MLAA without kernel (i.e., when K is an identity matrix) applies a DIP model to the GCT image μ directly. The kernel matrix was built using using image patches of size 3 × 3 extracted from the x-ray CT image and 50 nearest neighbors. The reconstruction steps was implemented in MATLAB and the neural network learning step was implemented in PyTorch. All the reconstructions were run for 150 iterations. Different MLAA methods were compared for the image quality of GCT using the mean squared error (MSE) defined by

$$\text{MSE}(\widehat{\mu })=10{\text{ log}}_{10}\left(\frac{{\Vert \widehat{\mu }-{\mu }^{\text{true}}\Vert }^2}{{\Vert {\mu }^{\text{true}}\Vert }^2}\right)(\text{dB}),$$ (19)

where $\widehat{\mu }$ represents the reconstructed GCT image by each MLAA method and μ^true denotes the ground truth.

4.3. Comparison Results for GCT Image Quality

Fig. 3 shows the comparison of GCT images reconstructed by different algorithms. The images were chosen according to the best image MSE in each method. Fig. 3(b) shows the standard MLAA with a uniform image initial for μ and Fig. 3(c) shows the MLAA with the μ initial set to the 511 keV linear attenuation coefficient map converted from the the x-ray CT using bilinear scaling. Clearly, a good initial from x-ray CT provided a better image quality. Hereafter we mainly present the results that use the CT initial. The results by the kernel MLAA and the neural MLAA without kernel are shown in Fig. 3(d) and 3(e), respectively. Both reconstructions had a better MSE than the standard MLAA but also demonstrated some artifacts. In comparison, the neural MLAA with kernel, as shown in Fig. 3(f), demonstrated a better visual quality with a higher image MSE.

Figure 3:

True and reconstructed GCT images from PET emission data by different algorithms with the best image MSE. (a) Ground truth; (b-c) standard MLAA with (b) uniform initial or (c) CT initial for μ; (d) Kernel MLAA; (e) Neural MLAA without kernel; (f) Neural MLAA with kernel.

Fig. 4 further shows image MSE as a function of iteration number for different algorithms. As compared to the kernel MLAA, the two neural MLAA approaches had a significant acceleration to reach their own best MSE. Among different algorithms, the neural MLAA with kernel achieved the lowest image MSE.

Figure 4:

Plots of image MSE for different algorithms.

5. CONCLUSION AND FUTURE WORK

In this paper, we have developed a neural MLAA approach that combines the kernel model and neural network-based deep image prior model for reconstruction of γ-ray CT image from PET emission data. The neural networks are used to regularize the estimation of kernel coefficients without adding an explicit regularization term. An optimization transfer algorithm is also developed for efficient neural network reconstruction. Computer simulation results have demonstrated the improvement of the neural MLAA over conventional MLAA algorithms. Our future work will evaluate the impact of the improved image quality for PET-enabled dual-energy CT multi-material decomposition. The proposed method will also be combined with the design of neural kernels¹⁴ to further improve the MLAA reconstruction for PET-enabled dual-energy CT imaging.