SPULTRA: Low-Dose CT Image Reconstruction with Joint Statistical and Learned Image Models

Abstract

Low-dose CT image reconstruction has been a popular research topic in recent years. A typical reconstruction method based on post-log measurements is called penalized weighted-least squares (PWLS). Due to the underlying limitations of the post-log statistical model, the PWLS reconstruction quality is often degraded in low-dose scans. This paper investigates a shifted-Poisson (SP) model based likelihood function that uses the pre-log raw measurements that better represents the measurement statistics, together with a data-driven regularizer exploiting a Union of Learned TRAnsforms (SPULTRA). Both the SP induced data-fidelity term and the regularizer in the proposed framework are nonconvex. The proposed SPULTRA algorithm uses quadratic surrogate functions for the SP induced data-fidelity term. Each iteration involves a quadratic subproblem for updating the image, and a sparse coding and clustering subproblem that has a closed-form solution. The SPULTRA algorithm has a similar computational cost per iteration as its recent counterpart PWLS-ULTRA that uses post-log measurements, and it provides better image reconstruction quality than PWLS-ULTRA, especially in low-dose scans.

I. Introduction

Recent years have witnessed the growing deployment of X-ray computed tomography (CT) in medical applications. Simultaneously there has been great concern to reduce the potential risks caused by exposure to X-ray radiation. Strategies for reducing the X-ray radiation in CT include reducing the photon intensity at the X-ray source, i.e., low-dose CT (LDCT), or lowering the number of projection views obtained by the CT machine, i.e., sparse-view CT. In the case where the X-ray radiation is extremely low, the CT image may not be suitable for medical diagnosis, but it is still quite helpful for non-diagnostic applications such as attenuation correction for PET/CT imaging [1]–[3] and virtual CT colonoscopy screening [4]. Reconstructing CT images with reduced radiation is challenging, and many reconstruction methods have been proposed for this setting. Model-based iterative reconstruction (MBIR) is widely used [5] among these approaches. Based on maximum a posteriori (MAP) estimation, MBIR approaches form a cost function that incorporates the statistical model for the acquired measurements and the prior knowledge (model) of the images. This section first reviews some of the statistical models for CT measurements along with recent works on extracting prior knowledge about images for LDCT image reconstruction, and then presents our contributions.

A. Background

Accurate statistical modeling of the measurements in CT scanners is challenging, especially in low-dose imaging, when the electronic noise in the data acquisition system (DAS) becomes significant [6]–[12]. Approximations of the measurement statistics can be categorized into post-log and pre-log models [13], which are detailed next.

The post-log models work on data obtained from the logarithmic transformation of the raw measurements, which is often assumed Gaussian distributed. Since the logarithmic transformation approximately linearizes the raw measurements, methods based on post-log data can readily exploit various optimization approaches and regularization designs with efficiency and convergence guarantees for this reconstruction problem [14]–[16]. The post-log methods however have a major drawback: the raw measurements may contain non-positive values on which the logarithmic transformation cannot be taken (or near-zero positive measurements whose logarithm can be very negative), particularly when the electronic noise becomes significant as compared to the photon statistical noise in low-dose cases.

There are many pre-correction approaches to deal with non-positive raw measurements for post-log methods. Examples of such approaches include using a statistical weight of zero for such measurements [17], replacing the non-positive measurements with a small positive value [18] and filtering neighboring measurements [6]. Thibault et al. [19] proposed a recursive filter which preserves the local mean to pre-process noisy measurements, but still used a non-linear function to map all noisy measurements to strictly positive values. Chang et al. [20] applied the local linear minimum mean-square error (LLMMSE) filter to pre-process the raw measurements, but the LLMMSE filter does not guarantee positivity in its output sinograms and introduces correlations among neighbouring channels. This correlation violates the assumption of independence of sinogram data on which MAP reconstruction formulations rely. Chang et al. [20] also proposed a pointwise Bayesian restoration (PBR) approach, which better preserves the independence of sinogram data while reducing bias for photon-starved CT data. When pre-processing a large percentage of non-positive values for LDCT measurements, these pre-correction methods may still introduce bias in the reconstructed image and can degrade image quality [13], [19]. The logarithmic transformation itself causes a positive bias in the line integrals from which the image is reconstructed [13], [21]. A typical method for reconstructing images from the post-log data is penalized weighted least squares (PWLS) [18] that optimizes an objective consisting of a weighted least squares data fidelity term and a regularization penalty. However, the pre-correction process and non-linear logarithmic operation create challenges in estimating the statistical weights for the PWLS methods [19], [22].

Contrary to the post-log methods, the pre-log methods work directly with the raw measurements. A robust statistical model for the pre-log raw CT measurements is the shifted-Poisson (SP) model. This model shifts the measurements by the variance of the electric readout noise. The shifted measurement has its variance equal to its mean, so that it could be approximated to be Poisson distributed. Since the shifted-Poisson model is a better approximation for CT measurement statistics compared to the Gaussian model [13], [23]–[26], and no pre-correction of the data is needed for most LDCT dose levels [13], this paper uses this SP model for LDCT image reconstruction.

There has been growing interest in improving CT image reconstruction by extracting prior knowledge from previous patient scans. Many methods have been proposed in this regard, such as prior image constrained compressed sensing methods (PICCS) [27]–[29], or the previous normal-dose scan induced nonlocal means method [30], [31]. More recently, inspired by the success of learning-based methods in image processing and computer vision, researchers have incorporated data-driven approaches along with statistical models for LDCT image reconstruction. One such approach proposed by Xu et al. [32] combined dictionary learning techniques with the PWLS method for LDCT image reconstruction. The dictionary they used was either pre-learned from a training image set (consisting of 2D images) and fixed during reconstruction, or adaptively learned while reconstructing the image. The 2D dictionary model for image patches was later extended to a 2.5D dictionary (where different dictionaries were trained from 2D image patches extracted from axial, sagittal, and coronal planes of 3D data) [33], and then to a 3D dictionary trained from 3D image patches [34]. These dictionary learning and reconstruction methods are typically computationally expensive, because they involve repeatedly optimizing NP-hard problems [35] for estimating the sparse coefficients of patches. The learning of sparsifying transforms (ST) was proposed in recent works [36], [37] as a generalized analysis dictionary learning method, where the sparse coefficients are estimated directly by simple and efficient thresholding. Pre-learned square sparsifying transforms have been recently incorporated into 2D LDCT image reconstruction with both post-log Gaussian statistics [38] and pre-log SP measurement models [39]. Especially, Zheng et al. [38] showed promise for PWLS with a union of pre-learned sparsifying transforms [40] regularization that generalizes the square sparsifying transform approach.

In addition to the dictionary learning-based approaches, some works have incorporated neural networks in CT image reconstruction. Adler and Öktem proposed a learned primal-dual reconstruction method [41], that uses convolutional neural networks (CNNs) to learn parameterized proximal operators. This method was applied to relatively simple 2D phantoms. Wu et. al [42] proposed a k-sparse autoencoder (KSAE) based regularizer for LDCT image reconstruction, where they trained three independent KSAEs from axial, sagittal and coronal slices for 3D reconstruction via artificial neural networks. Chen et al. [43] proposed to unfold the classical iterative reconstruction procedure into a CNN-based recurrent residual network so that the original fixed regularizers and the balancing parameters within the iterative scheme can vary for each layer. The reconstruction with this network was only performed slice by slice. He et al. proposed a parameterized plug-and-play alternating direction method (3pADMM) for PWLS model based low-dose CT image reconstruction [44]. By regarding the ADMM optimization steps as network modules, this method can optimize the 3p prior and the related parameters simultaneously. These methods are fully supervised learning methods requiring large datasets consisting of both undersampled images or measurements and the corresponding high-quality images. Some post-processing approaches involving neural networks such as a U-Net or a residual net also improve CT image quality [45], [46], but such post-processing methods usually construct an image-to-image mapping without fully incorporating the physics of the imaging process. Additionally, the generalization of supervised learning methods may be limited in the sense that the trained model may only work well on the data that is similar to the training set.

B. Contributions

Considering the robustness and accuracy offered by the SP statistics, and inspired by the data-driven image modeling methods not requiring paired training data or previous registered normal-dose images, here we propose a new LDCT image reconstruction method named SPULTRA that combines robust SP measurement modeling with a union of learned sparsifying transforms (ULTRA) based regularizer. Since the SP model leads to a nonconvex data-fidelity term, we design a series of quadratic surrogate functions for this term in our optimization. For each surrogate function combined with the ULTRA regularizer (a majorizer of the SPULTRA objective), we optimize it by alternating between an image update step and a sparse coding and clustering step. The proposed SPULTRA scheme is proved to converge to the critical points of the overall nonconvex problem. In the experiments, we compare SPULTRA with the recent PWLS-ULTRA scheme [38] under different incident photon intensity levels for 3D XCAT phantom simulations. The results demonstrate that the proposed method avoids bias in image regions caused by the PWLS-ULTRA method, especially for low X-ray doses. At the same time, SPULTRA achieves better image reconstruction quality than PWLS-ULTRA given the same number of iterations, or alternatively, SPULTRA achieves a desired image reconstruction quality much faster than the competing PWLS-ULTRA scheme, especially for low X-ray doses. We verify the bias avoidance property of SPULTRA on a synthesized 3D clinical chest scan, and an ultra low-dose 2D shoulder phantom scan simulated from standard-dose raw measurements that also involve beam-hardening effects. We compared SPULTRA with a recent deep-learning based denoising framework [46] on the 2D data demonstrating the better reconstruction and generalization ability of SPULTRA.

This paper significantly extends our previous conference work [39] by incorporating the ULTRA regularizer and proposing a faster optimization procedure with convergence guarantees. We performed extensive numerical evaluations compared to the 2D LDCT XCAT phantom results in [39].

C. Organization

The rest of this paper is organized as follows. Section II presents the proposed problem formulation for low-dose CT image reconstruction. Section III briefly reviews the ULTRA learning method and describes the proposed SPULTRA image reconstruction algorithm. Section IV discusses the convergence properties of the SPULTRA methodology. Section V presents detailed experimental results and comparisons. Section VI presents conclusions.

II. Problem Formulation for SPULTRA

The goal in LDCT image reconstruction is to estimate the linear attenuation coefficients $x\in {ℝ}^{N_p}$ from CT measurements $y\in {ℝ}^{N_d}$. We propose to obtain the reconstructed image by solving a SP model-based penalized-likelihood problem:

$$\widehat{x}={\text{arg\hspace{0.17em}min}}_{x\in \mathcal{X}}G(x),\hspace{1em}G(x)=\text{L}(x)+\text{R}(x),$$ (P0)

where $\mathcal{X}=\left\{\text{x}|0\le {\text{x}}_j\le x_{\text{max}}\right\}$ is a large constant. The objective function G(x) is composed of a negative log-likelihood function L(x) based on the SP model for the measurements, and a penalty term R(x) that is based on the ULTRA model [38], [47]. The SP model can be described as $Y_i~\text{Poisson}\left\{I_0{e}^{-f_i\left({[Ax]}_i\right)}+{\sigma }^2\right\}$, where Y_i is the shifted quantity of the ith measurement for i = 1,…,N_d, σ² is the variance of the electronic noise, I₀ is the incident photon count per ray from the source, f_i(·) models the beam-hardening effect, and $A\in {ℝ}^{N_d\times N_p}$ is the CT system matrix. Denoting l_i(x) ≜ [Ax]_i (or l_i in short), the data-fidelity term L(x) can be written as

$$\text{L}(\text{x})=\sum _{i=1}^{N_d}{h}_i\left(l_i(x)\right),$$ (1)

where

$$h_i\left(l_i\right)\triangleq \left(I_0{e}^{-f_i\left(l_i\right)}+{\sigma }^2\right)-Y_i\text{log}\left(I_0{e}^{-f_i\left(l_i\right)}+{\sigma }^2\right).$$ (2)

The beam-hardening model f_i(·) is usually approximated as a polynomial [13]. For simplicity, we use a second order polynomial, i.e., $f_i\left(l_i\right)=s_{1_i}{l}_i+s_{2_i}{l}_i^2$, where $s_{1_i}$ and $s_{2_i}$ are coefficients of the polynomial for the ith measurement.

The ULTRA regularizer R(x) has the following form [38]:

$$\begin{gathered} \text{R}(x)\triangleq \underset{\left\{z_j,C_k\right\}}{\text{min}}\beta \sum _{k=1}^K\left\{\sum _{j\in C_k}{\tau }_j\left\{{\Vert {\Omega }_k{P}_{j}x-z_j\Vert }_2^2+{\gamma }_c^2{\Vert z_j\Vert }_0\right\}\right\} \\ s.t.\left\{C_k\right\}\in \mathcal{G}, \end{gathered}$$ (3)

where $\mathcal{G}$ denotes the set consisting of all possible partitions of {1,2,…,N_p} into K disjoint subsets, K is the number of classes and C_k denotes the set of indices of patches belonging to the kth class. The operator $P_j\in {ℝ}^{v\times N_p}$ is the patch extraction operator that extracts the jth patch of υ voxels for j = 1,…,Ñ, from x, where Ñ is the number of extracted patches. The learned transform corresponding to the kth class ${\Omega }_k\in {ℝ}^{v\times v}$ maps the patches to the transform domain. Vector $Z_j\in {ℝ}^v$ denotes the sparse approximation of the transformed jth patch, with the parameter ${\gamma }_c^2\left({\gamma }_c>0\right)$ controlling its sparsity level. We use the ℓ₀ “norm” (that counts the number of nonzero elements in z_j) to enforce sparsity. The patch-based weight τ_j is defined as ‖P_jκ‖₁/υ [38], [48], where $\kappa \in {ℝ}^{N_p}$ is defined to help encourage resolution uniformity as ${\kappa }_j\triangleq \sqrt{{\sum }_{i=1}^{N_d}{a}_{ij}{\tilde{w}}_i{\sum }_{i=1}^{N_d}{a}_{ij}}$ [49, eq(39)], with a_ij denoting the entries of A, and ${\tilde{w}}_i$ is approximated as ${\tilde{w}}_i={\dot{f}}_i{\left({\tilde{l}}_i\right)}^2\frac{y_i^2}{y_i+{\sigma }^2}$ [13, eq(10)], where ${\tilde{l}}_i$ is the beam-hardening corrected, post-log sinogram data. To balance the data-fidelity term and the regularizer in the formulation, R(x) is scaled by a positive parameter β.

III. Algorithm

The proposed SPULTRA algorithm is based on a pre-learned union of sparsifying transforms. The process of learning such a union of transforms from a dataset of image patches has been detailed in [38]. The learning problem in [38] simultaneously groups the training patches into K classes and learns a transform in each class along with the sparse coefficients (in the transform domain) of the patches. This learning is accomplished by an alternating algorithm (see [38]). This section focuses on describing the algorithm in the reconstruction stage for SPULTRA, i.e., for (P0).

The data-fidelity term L(x) in (P0) is nonconvex when the electronic noise variance σ² is nonzero. It is challenging to directly optimize such a logarithmic nonconvex function. We propose to iteratively design quadratic surrogate functions for this data-fidelity term L(x). In each iteration, we optimize the surrogate function that is a quadratic data-fidelity term together with the ULTRA regularizer using alternating minimization that alternates between an image update step and a sparse coding and clustering step that has closed-form solution [40]. We use the relaxed OS-LALM algorithm for the image update step [50]. We perform only one alternation between the two steps for each designed surrogate function, which saves runtime and works well in practice.

A. Surrogate function design

We design a series of quadratic surrogate functions for L(x) as follows:

$$\varphi \left(x;x^n\right)=\text{L}\left(x^n\right)+d_h\left(l^n\right)A\left(x-x^n\right)+\frac{1}{2}{\left(x-x^n\right)}^T{A}^T{W}^{n}A\left(x-x^n\right),$$ (4)

where (·)ⁿ denotes values at the nth iteration and $d_h\left(l^n\right)\in {ℝ}^{N_d}$ is a row vector capturing the gradient information and is defined as $d_h\left(l^n\right)\triangleq {\left[{\dot{h}}_i\left(l_i^n\right)\right]}_{i=1}^{N_d}$. The curvatures of the nth updated parabola (surrogate) are described by $W^n\triangleq \text{diag}\left\{c_i\left(l_i^n\right)\right\}$. In this paper, we use the optimum curvatures [51] that are defined as follows:

$$c_i\left(l_i^n\right)=\{\begin{array}{ll}{\left[2\frac{h_i(0)-h_i\left(l_i^n\right)+\left(l_i^n\right){\dot{h}}_i\left(l_i^n\right)}{{\left(l_i^n\right)}^2}\right]}_{+},\hfill & l_i^n>0\hfill \\ {\left[{\ddot{h}}_i(0)\right]}_{+},\hfill & l_i^n=0,\hfill \end{array}$$ (5)

where ${\ddot{h}}_i$ is the second-order derivative, and operator [·]₊ sets the non-positive values to zero. In practice, we replace negative values with a small positive number so that the diagonal matrix Wⁿ is invertible. Due to numerical precision, (5) might become extremely large when $l_i^n$ is nonzero but small. To avoid this problem, we use an upper bound of the maximum second derivative ${\left[{\ddot{h}}_i(0)\right]}_{+}$ for the curvature $c_i\left(l_i^n\right)$ when $l_i^n>0$.

By ignoring the terms irrelevant to x in (4), we get the following equivalent form of φ(x;xⁿ):

$$\varphi \left(x;x^n\right)\equiv \frac{1}{2}{\Vert {\tilde{y}}^n-Ax\Vert }_{W^n}^2,$$ (6)

where “≡” means equal to within irrelevant constants of x, and ${\tilde{y}}^n\triangleq A{x}^n-{\left(W^n\right)}^{-1}{\left[d_h\left(l^n\right)\right]}^T$. The overall surrogate function at the nth iteration for the penalized-likelihood objective function G(x) in (P0) is then

$$\Phi \left(x;x^n\right)=\frac{1}{2}{\Vert {\tilde{y}}^n-Ax\Vert }_{W^n}^2+\text{R}(x),\text{\hspace{0.17em}s}.\text{t}.x\in \mathcal{X}.$$ (7)

We descend the surrogate function Φ(x;xⁿ) in (7) by alternating once between an image update step, and a sparse coding and clustering step.

B. Image Update Step

In the image update step, we update the image x with fixed sparse codes {z_j} and class assignments {C_k}. The relevant part of the majorizer for this step is

$${\Phi }_1\left(x;x^n\right)=\varphi \left(x;x^n\right)+\beta \sum _{k=1}^K\sum _{j\in C_k}{\tau }_j{\Vert {\Omega }_k{P}_{j}x-z_j\Vert }_2^2$$ (8)

Although we have a box constraint on x, i.e., $x\in \mathcal{X}$, in practice, the upper bound x_max can be set high such that it will not be active. We applied the relaxed OS-LALM algorithm [50] to minimize (8) with the constraint. This algorithm is shown in Algorithm 1 (steps 7–10). The OS-LALM method uses majorizing matrices. In particular, the matrix A^TWⁿA is majorized by D_A ≜ diag{A^TWⁿA1}, where 1 denotes a vector of ones. Denoting the regularization term in (8) as R₂(x), its gradient is

$$\nabla {\text{R}}_2(\text{x})=2\beta \sum _{k=1}^K\sum _{j\in C_k}{\tau }_j{P}_j^T{\Omega }_k^T\left({\Omega }_k{P}_{j}x-z_j\right).$$ (9)

The Hessian of R₂(x) is majorized by the following diagonal matrix:

$$D_{\text{R}}\triangleq 2\beta \left\{\underset{k}{\text{max}}{\Vert {\Omega }_k^T{\Omega }_k\Vert }_2\right\}\sum _{k=1}^K\sum _{j\in C_k}{\tau }_j{P}_j^T{P}_j.$$ (10)

The (over-)relaxation parameter α ∈ [1,2) and the parameter ρ_t > 0 decreases with iterations t in OS-LALM according to the following equation [50]:

$${\rho }_t(\alpha )=\{\begin{array}{ll}1,\hfill & t=0\hfill \\ \frac{\pi }{\alpha (t+1)}\sqrt{1-{\left(\frac{\pi }{2\alpha (t+1)}\right)}^2},\hfill & \text{otherwise}.\hfill \end{array}$$ (11)

C. Sparse Coding and Clustering Step

Here, with x fixed, we jointly update the sparse codes and the class memberships of patches. The relevant part of the cost function for the sparse coding and clustering step is

$$\underset{\left\{z_j,C_k\right\}}{\text{min}}\sum _{k=1}^K\left\{\sum _{j\in C_k}{\tau }_j\left\{{\Vert {\Omega }_k{P}_{j}x-z_j\Vert }_2^2+{\gamma }_c^2{\Vert z_j\Vert }_0\right\}\right\}.$$ (12)

Problem (12) is separable in terms of the patches, so each patch is clustered and sparse coded independently in parallel. The optimal sparse code z_j in (12) is obtained by hard-thresholding, i.e., $z_j=H_{{\gamma }_c}\left({\Omega }_{k_j}{P}_{j}x\right)$, where $H_{{\gamma }_c}(\cdot )$ represents a vector hard-thresholding operator that zeros out elements whose magnitudes are smaller than γ_c, and leaves other entries unchanged. Then the optimized class ${\widehat{k}}_j$ for the jth patch is computed as follows [38]:

$${\widehat{k}}_j=\underset{1\le k\le K}{\text{argmin}}{\Vert {\Omega }_k{P}_{j}x-H_{{\gamma }_c}\left({\Omega }_k{P}_{j}x\right)\Vert }_2^2+{\gamma }_c^2{\Vert H_{{\gamma }_c}\left({\Omega }_k{P}_{j}x\right)\Vert }_0.$$ (13)

We compute the cost values on the right hand side of (13) for each k = 1,···, K, and determine the ${\widehat{k}}_j\in \{1,\cdots ,K\}$ that gives the minimal cost value, i.e., patch P_jx is grouped with the transform that provides the smallest value of the cost in (13). Then, the corresponding optimal sparse code is ${\widehat{z}}_j=H_{{\gamma }_c}\left({\Omega }_{{\widehat{k}}_j}{P}_{j}x\right)$.

Algorithm 1 illustrates the proposed optimization algorithm for Problem (P0).

D. Computational Cost

The SPULTRA algorithm has a similar structure in each iteration as the recent PWLS-ULTRA [38], except for several initializations in the image update step. Since forward and backward projections are used to compute D_A and ${\tilde{y}}^n$ during initialization, the image update step of SPULTRA is slightly slower than PWLS-ULTRA. In our experiments, we observed that the initializations took around 20% of the runtime in each outer iteration. However, in practice, especially for low doses, SPULTRA reconstructs images better than PWLS-ULTRA for a given number of outer iterations. Or alternatively, SPULTRA takes much fewer outer iterations (and runtime) to achieve the same image reconstruction quality as PWLS-ULTRA. These results are detailed in Sec. V.

Algorithm 1.

SPULTRA Algorithm

IV. Convergence Analysis

The objective function (P0) of SPULTRA is highly nonconvex due to the non-convexity of the data-fidelity term and the regularizer. The proposed algorithm efficiently optimizes it by using surrogate functions and alternating minimization. This section provides a convergence analysis for the general optimization approach. While a recent work [47] analyzed the convergence of a related optimization method, it did not involve the use of surrogate functions and involved adaptive learning of transforms.

In the proposed method, the sparse coding and clustering step is solved exactly. For the image update step, where the cost function is quadratic as in (8), many approaches may be used to optimize it, e.g., [16], [50], [52]. Our convergence proof in the supplement assumes for simplicity that the image update step is solved exactly.

The convergence result uses the following notation. We use Z for the sparse code matrix concatenated by column vectors z_j, and use a vector $\Gamma \in {ℝ}^{\tilde{N}}$, whose elements represent the classes indices for the patches, i.e., Γ_j ∈ {1,···, K}. For an initial (x⁰,Z⁰,Γ⁰), we let {xⁿ,Zⁿ,Γⁿ} denote the sequence of iterates generated by alternating algorithm. The objective function in (P0) is denoted as G(x,Z,Γ) and includes the constraint on x as an added barrier penalty (which takes the value +∞ when the constraint is violated and is zero otherwise). The convergence result is as follows.

Theorem 1. Assume the image update step is solved exactly. For an initial (x⁰,Z⁰,Γ⁰), iterative sequence (xⁿ,Zⁿ,Γⁿ) generated by the SPULTRA algorithm is bounded, and the corresponding objective sequence {G(xⁿ,Zⁿ,Γⁿ)} decreases monotonically and converges to G* ≜ G*(x⁰,Z⁰,Γ⁰). Moreover, all the accumulation points of the iterate sequence are equivalent and achieve the same value G* of the objective. Each accumulation point (x*,Z*,Γ*) also satisfies the following partial optimality conditions:

$$\begin{array}{l}{0\in {\partial }_{x}G\left(x,Z^{*},{\Gamma }^{*}\right)|}_{x=x*},\\ \left(Z*,\Gamma *\right)\in \underset{Z.\Gamma }{\text{argmin}}G\left(x*,Z,\Gamma \right),\end{array}$$ (14)

where ∂_x denotes the sub-differential operator for the function G with respect to x [53]–[55]. Finally, ‖xⁿ⁺¹ − xⁿ‖₂ → 0 as n → ∞.

The above theorem implies that for each initial (x⁰,Z⁰,Γ⁰), the objective sequence converges (although the limit may depend on initialization) and the iterate sequence in the optimization framework converges to an equivalence class of accumulation points (i.e., all accumulation points have the same objective value G*) that are also partial optimizers satisfying (14). Moreover, the image sequence satisfies ‖xn+1 − xn‖₂ → 0.

When K = 1, (14) readily implies that the iterate sequence in the algorithm converges to an equivalence class of critical points [53] (that are generalized stationary points) of the nonconvex cost G(x,Z,Γ).

A detailed proof is included in the supplement¹.

V. Experimental Results

Here we present numerical experiments demonstrating the behavior of SPULTRA. We evaluated the proposed SPULTRA method on the 3D XCAT phantom [56] and synthesized clinical data at multiply X-ray doses, as well as an ultra low-dose 2D shoulder phantom scan simulated from real raw data, and compared its performance with that of the state-of-the-art PWLS-ULTRA [38]. We computed the root mean square error (RMSE) and structural similarity index (SSIM) [32], [57] of XCAT images reconstructed by various methods in a region of interest (ROI). The RMSE is defined as $\sqrt{{\sum }_{i\in \text{ROI}}{\left({\widehat{x}}_i-x_i^{*}\right)}^2/N_{p,ROI}}$, where N_p,ROI is the number of pixels in the ROI, $\widehat{x}$ is the reconstructed image, and x* is the ground-truth image. We also compared to PWLS reconstruction with an edge-preserving regularizer (PWLS-EP) $\text{R}(\text{x})={\sum }_{j=1}^{N_p}{\sum }_{k\in N_j}{\kappa }_j{\kappa }_k\phi \left(x_j-x_k\right)$, where N_j represents the neighborhood of the jth pixel, κ_j and κ_k are elements of κ that encourages resolution uniformity [49]. The potential function for 3D reconstruction was ϕ(t) = δ²(|t/δ| − log(1 + |t/δ|)) with² δ = 10 HU, and that for 2D shoulder phantom simulations was $\phi (t)={\delta }^2\left(\sqrt{1+|t/\delta {|}^2}-1\right)$ with δ = 100 HU. The results obtained by PWLS-EP were taken as initial images for methods we compared with in this section.

The SPULTRA method shifts uncorrected pre-log data by the variance of electronic noise. Such un-preprocessed pre-log data and the variance of the electronic noise on a CT scanner are proprietary to CT vendors, especially for LDCT. In our experiments of XCAT phantom simulations and the synthesized clinical data, we generated pre-log data ŷ from the XCAT phantom as well as from a clinical image $\tilde{x}$ reconstructed by the PWLS-ULTRA method as follows:

$${\widehat{y}}_i=\text{Poisson}\left\{I_0{e}^{-{[A\tilde{x}]}_i}\right\}+\mathcal{N}\left\{0,{\sigma }^2\right\},$$ (15)

where $\mathcal{N}\left\{\mu ,{\sigma }^2\right\}$ denotes a Gaussian distribution with mean μ and variance σ². We refer to the image $\tilde{x}$ used for generating the synthesized clinical data as the “true” clinical image. We also simulated an ultra low-dose scan from raw (pre-log) measurements of a standard-dose scan of a 2D shoulder phantom as:

$${\widehat{y}}_i=\text{Poisson}\left\{\frac{1}{\alpha }{y}_{i_s}\right\}+\mathcal{N}\left\{0,{\sigma }^2\right\},$$ (16)

where α is a scale factor we used to lower the dose from standard-dose measurements, and $y_{i_s}$ denotes the raw standard-dose measurements. We set σ = 5 for all the simulations, as suggested in prior works [13], [38]. We implemented the system model A via the separable footprint projector methods [58]. MATLAB code to reproduce the results in this work is released at http://web.eecs.umich.edu/~fessler/. Some additional results are included in the supplement.

A. XCAT phantom results

1). Framework:

We pre-learned a union of 15 square transforms from 8 × 8 × 8 overlapping patches extracted from a 420×420×54 XCAT phantom with a patch stride 2×2×2. These transforms were initialized during training [38] with 3D DCT, and the clusters were initialized randomly. We simulated 3D axial cone-beam scans using a 840 × 840 × 96 XCAT phantom with Δ_x = Δ_y = 0.4883 mm and Δ_z = 0.625 mm. We generated sinograms of size 888 × 64 × 984 using GE LightSpeed cone-beam geometry corresponding to a mono-energetic source with I₀ = 1×10⁴, 5 × 10³, 3 × 10³, and 2 × 10³ incident photons per ray and no scatter, respectively. Tab. I shows percentages of non-positive measurements under different dose levels. We set these non-positive measurements to 1×10⁻⁵ for generating the post-log sinogram that PWLS-based methods rely on [13]. We reconstructed the 3D volume with a size of 420 × 420 × 96 at a coarser resolution of Δ_x = Δ_y = 0.9766 mm and Δ_z = 0.625 mm. The patch size during reconstruction was 8 × 8 × 8 and the stride was 3 × 3 × 3. For evaluating reconstruction performance, we chose an ROI that was composed of the central 64 out of 96 axial slices, and refer to it as the reconstruction targeted ROI. Fig. 1 shows the central slices of the true XCAT phantom inside this ROI along three directions. In the reconstruction stage of PWLS-ULTRA and SPULTRA, we used 4 iterations for the image update step, i.e., P = 4, for a good tradeoff between algorithms’ convergence and computational costs. We used 12 ordered subsets, i.e., M = 12, to speed up the algorithm. The initial image for the ULTRA methods was reconstructed by PWLS-EP, whose regularization parameter was set empirically to ensure good reconstruction quality as β_ep = 2¹³ for all the experimented dose cases. We used an analytical filtered back-projection (FBP) method FDK [59] to initialize PWLS-EP. The FDK images of XCAT phantom for all the dose levels are shown in the supplement. Due to the fact that SPULTRA has a similar cost function as PWLS-ULTRA in each outer iteration, we used the same parameter settings for both methods: β = 4×10⁴ and γ_c = 4×10⁻⁴, which we observed worked well for all the dose levels we tested.

TABLE I:

Percentages of non-positive measurements under different dose levels for XCAT phantom simulations.

I0	1 × 104	5 × 103	3 × 103	2 × 103
Non-positive Percentage (%)	0.06	0.20	0.48	0.96

Fig. 1:

Reconstruction targeted ROI of the true XCAT phantom displayed with central slices along the axial, sagittal and coronal directions. The display window is [800, 1200] HU.

2). Behavior of the learned ULTRA Models:

The learned union of transforms contributes to the clustering and sparsification of image patches. To illustrate the behavior of the learned transforms, we selected 3 of the 15 transforms that capture important structures/features of the reconstructed image (with I₀ = 1 × 10⁴) in their classes. Fig. 2 (first column) shows three voxel-level classes (voxels are clustered by majority vote among patches overlapping them) for the reconstructed central axial slice. The top image only contains soft tissues, whereas the middle image shows some edges and bones in the vertical direction, and the bottom image captures some high-contrast structures. Fig. 2 (second column) shows the transforms for the corresponding classes. Each learned transform has 512 8 × 8 × 8 filters, and we show the first 8 × 8 slice of 256 of these filters that show gradient-like and directional features. Fig. 2 also shows the central axial slice of the sparse coefficient maps (volumes) for different filters of the transforms in the third, fourth and fifth columns. Each voxel value in a sparse coefficient map is obtained by applying the specific 3D filter to a 3D patch (whose front top left corner is at that voxel) and hard-thresholding the result. Coefficients for patches not belonging to the specific class are set to zero (masked out). The sparse code maps capture different types of image features (e.g., edges at different orientations or contrasts) depending on the filters and classes.

Fig. 2:

The three rows correspond to the 1st, 13th, and 14th classes respectively. The first column displays three voxel-level clustered images of the central axial slice. Each of them is formed by image patches lie in the corresponding class. The second column displays part of the transforms for the corresponding classes. The third, fourth and fifth columns show the central axial slice of the sparse coefficient maps obtained by applying specific filters (shown in the top left corner) to patches belonging to the corresponding classes. The patch stride for plotting these figures was 1 × 1 × 1.

3). Numerical Results:

We compare the RMSE and the SSIM for SPULTRA with those for PWLS-EP and PWL-SULTRA. Tab. II lists the two metrics for the reconstruction targeted ROI after sufficient iterations (800 iterations) for convergence of PWLS-EP, PWLS-ULTRA, and SPULTRA, for various dose levels. The results show that SPULTRA achieves significant improvements in RMSE and SSIM in low-dose situations. Notably, compared to PWLS-ULTRA, SPULTRA further decreases the RMSE by up to 1.3 HU when I₀ = 3 × 10³, and by around 3.3 HU when I₀ = 2 × 10³.

TABLE II:

RMSE (HU) and SSIM of the reconstruction targeted ROI at various dose levels (I₀) using the PWLS-EP, PWLS-ULTRA, and SPULTRA methods for the XCAT phantom simulations.

(a) RMSE (HU)					(b) SSIM
I0	PWLS-EP	PWLS-ULTRA	SPULTRA		I0	PWLS-EP	PWLS-ULTRA	SPULTRA
1 × 104	45.3	29.1	28.9		1 × 104	0.941	0.974	0.974
5 × 103	47.1	33.3	32.8		5 × 103	0.937	0.969	0.970
3 × 103	49.7	37.7	36.4		3 × 103	0.927	0.961	0.963
2 × 103	53.5	43.2	39.9		2 × 103	0.911	0.948	0.956

The RMSE improvement of SPULTRA over PWLS-ULTRA can be more clearly observed from Fig. 3 that shows the RMSE evolution with the number of outer iterations under different dose levels. At low-doses, SPULTRA decreases the RMSE more quickly (from the same initial value) and to much lower levels than PWLS-ULTRA. Fig. 3 shows that to achieve the same RMSE as PWLS-ULTRA at 600 outer iterations, SPULTRA takes 487, 365, 251 and 133 outer iterations under I₀ = 1×10⁴, 5 × 10³, 3 × 10³, and 2 × 10³, respectively.

Fig. 3:

RMSE comparison of SPULTRA and PWLS-ULTRA. The cursors indicate the RMSEs (Y) at specific number of outer iterations (X).

4). Computational Costs:

As discussed in Sec. III-D, SPULTRA has a similar computational cost per iteration as PWL-SULTRA, except for computing some initializations for image update. Fig. 3 shows that the SPULTRA method requires much fewer number of outer iterations than PWLS-ULTRA to achieve the same RMSE for the reconstruction, especially at low doses. When the dose is very low, e.g., when I₀ = 2×10³, SPULTRA takes only a quarter the number of outer iterations as PWLS-ULTRA to achieve the same RMSE. Thus, the total runtime to achieve a specific reconstruction quality at low doses is typically much lower for SPULTRA than for PWL-SULTRA. When the dose is not very low, for example when I₀ = 1×10⁴, the SPULTRA and the PWLS-ULTRA methods have similar computational costs and runtimes. To achieve RMSE of 29.26 HU (see Fig. 3a), PWLS-ULTRA requires 600 outer iterations, while SPULTRA requires 487×120% ≈ 584 effective outer iterations where the additional 20% runtime is associated with initializations in each SPULTRA outer iteration.

5). Visual Results and Image Profiles:

Fig. 4 shows the reconstructed images and the corresponding error images for PWLS-EP, PWLS-ULTRA, and SPULTRA, at I₀ = 3 × 10³ and I₀ = 2 × 10³. Compared to the PWLS-EP result, both PWLS-ULTRA and SPULTRA achieved significant improvements in image quality in terms of sharper reconstructions of anatomical structures such as bones and soft tissues, and suppressing the noise. However, the PWLS-ULTRA method introduces bias in the reconstructions, which leads to larger reconstruction errors compared to the proposed SPULTRA method. In Fig. 4, we marked three 3D ROIs in the axial plane, i.e., ROI 1, ROI 2, and ROI 3. Fig. 5 shows the zoom-in images of a 3D plot of ROI 1, and those of ROI 2 and ROI 3 are shown in the supplement. We also plot the evolution of RMSE through the axial slices of the three 3D ROIs in Fig. 6. The figures demonstrate that SPULTRA clearly outperforms the competing PWLS-EP and PWLS-ULTRA schemes.

Fig. 4:

Comparison of reconstructions and reconstruction errors at (a) I₀ = 3×10³ and (b) I₀ = 2×10³ dose levels. The 3D images are displayed with the central slices along the axial, sagittal, and coronal directions. The unit of the display windows is HU.

Fig. 5:

3D displays of reconstructions of ROI 1 defined in Fig. 4. The display windows are [900, 1200] HU.

Fig. 6:

RMSE (HU) for each axial slice of the 3D ROIs (ROI 1, ROI 2, and ROI 3). The X-axis shows slice indices of the central 64 out of 96 axial slices. Left plot: I₀ = 3 × 10³. Right plot: I₀ = 2 × 10³.

The above advantages of SPULTRA can be seen more clearly when observing the image profiles. Fig. 7 plots the image profiles for the three methods together with that of the ground-truth image. Fig. 4 shows the horizontal green solid line and the vertical red dashed line, whose intensities are plotted in Fig. 7. It is obvious that the profiles for SPULTRA are closest to the ground-truth among the three compared methods. The gap between the profiles of the PWLS-based methods and the ground-truth shows the bias caused by the compared PWLS methods.

Fig. 7:

Image profiles along the horizontal and vertical lines indicated in Fig. 4. Left plot: I₀ = 3 × 10³. Right plot: I₀ = 2 × 10³.

B. Synthesized Clinical Data

1). Framework:

We used the pre-learned union of 15 square transforms from the XCAT phantom simulations to reconstruct the synthesized helical chest scan volume of size 420 × 420 × 222 with Δ_x = Δ_y = 1.1667 mm and Δ_z = 0.625 mm. The sinograms were of size 888×64×3611. Since the clinical data is synthesized via the PWLS-ULTRA reconstruction, the noise model for this synthesized data is obscure, making it difficult to determine appropriate low-dose levels for such data. We tested the radiation dose of I₀ = 1 × 10⁴ with an electronic noise variance the same as the XCAT phantom simulation, i.e., σ² = 25. The percentage of non-positive pre-log measurements for the synthesized clinical data in this case was around 0.14%. Such non-positive values were replaced by 1 × 10⁻⁵ for PWLS-based methods. Fig. 8a shows the “true” clinical image that was reconstructed from real clinical regular-dose sinogram using the PWL-SULTRA method. Similar to the XCAT phantom simulation, the initial image for both SPULTRA and PWLS-ULTRA was a reconstruction obtained using PWLS-EP. We set the regularizer parameter β_ep for PWLS-EP to 2¹⁵ to generate a smoother (with less noise) initial image, which led to good visual image equality for the SPULTRA and PWLS-ULTRA reconstructions. Since the optimization problem for PWLS-EP is strictly convex, we simply initialized PWLS-EP with a zero image. Fig. 8b shows the PWLS-EP reconstructed image for I₀ = 1 × 10⁴. We set the regularizer parameters for both PWLS-ULTRA and SPULTRA as γ_c = 5 × 10⁻⁴, and β = 1.5 × 10⁴.

Fig. 8:

(a) “true” clinical image (HU), (b) the reconstruction (HU) of the synthesized data with PWLS-EP for I₀ = 1×10⁴ with β_ep = 2¹⁵. The central axial, sagittal, and coronal slices of the volume are shown.

2). Reconstruction results for the synthesized clinical data:

Fig. 9 shows three axial slices from the 3D reconstructions with SPULTRA and PWLS-ULTRA at I₀ = 1 × 10⁴: the middle slice (No. 67) and two slices located farther away from the center (No. 90 and No. 120). The image profiles along a horizontal line (shown in green) in the displayed slices are also shown in Fig. 9. The reconstructed slices using PWLS-ULTRA appear darker around the center compared to the “true” clinical image and the reconstructions with SPULTRA. This means PWLS-ULTRA produces a strong bias in the reconstruction. The bias can be observed more clearly in the profile plots: the pixel intensities for the SPULTRA reconstruction better follow those of the “true” clinical image, while those for the PWL-SULTRA reconstruction are much worse than the “true” values. Moreover, SPULTRA achieves sharper rising and failing edges compared to PWLS-ULTRA. In other words, SPULTRA also achieves better resolution than PWLS-ULTRA. Fig. 9 also shows a zoomed-in ROI for each of the chosen slices, and highlights some small details with arrows. It is clear that in addition to reducing the bias, SPULTRA reconstructs image details better than PWLS-ULTRA.

Fig. 9:

Reconstructed images (columns 1 to 3) and the image profiles (the 4th column) along the green line in the “true” clinical image for the synthesized clinical data with I₀ = 1×10⁴ and σ² = 25. (a) Results for axial slice No. 67, (b) results for slice No. 90, and (c) results for slice No. 120. We selected one ROI for each of these three slices and the arrows point out some small structures in the image. The display windows for reconstructed images are [800, 1200] HU, and those for the zoomed-in ROIs are [950, 1200] HU.

C. Ultra Low-dose Experiments with Raw Data

1). Framework:

We obtained from GE a 2D fan-beam raw (pre-log) scan of a shoulder phantom, which included the beam-hardening effect. The provided 200 mA with 1 second scan can be viewed as a standard-dose scan and all the raw measurements are positive. Based on this standard-dose scan, we simulated an ultra low-dose scan as shown in (15) with α = 200, and added Poisson and Gaussian noise (σ = 5) to the measurements. The simulated measurements have about 0.4% non-positive values. The sinograms were of size 888 × 984, and reconstructed images were of size 512 × 512 with Δ_x = Δ_y = 0.9766 mm.

For PWLS-ULTRA and SPULTRA, we pre-learned a union of five square transforms using 8 × 8 overlapping image patches with stride 1 × 1 from five 512 × 512 XCAT phantom slices [38]. Here, we also compared SPULTRA with a recent deep-learning based low-dose CT denoising framework “WavResNet” combined with an RNN architecture [46]. The iterative RNN version of WavResNet was pre-trained based on the 2016 Low-Dose CT Grand Challenge data set [46]. During reconstruction, WavResNet, PWLS-ULTRA, and SPULTRA were initialized with the image reconstructed by PWLS-EP with β_ep = 0.1. The parameters (β,γ_c) for both PWLS-ULTRA and SPULTRA were set as (0.05,80). These values worked well in our experiment. In the supplement, we discuss in detail the parameter selection procedure of (β, γ_c) for both PWLS-ULTRA and SPULTRA. Parameters for testing WavResNet were set according to [46], and the pixel values of the input to WavResNet were converted to match the network required scalings. Since the WavResNet was trained with images reconstructed with the filtered backprojection (FBP) method [46], we also tested on this shoulder phantom that initialized WavResNet with an FBP reconstructed image. Although initializing WavResNet with an FBP reconstructed image better matches the trained model than the PWLS-EP reconstructed image does, the latter still provided better results. We included in the supplement the denoised image initialized with the FBP reconstruction.

2). Results:

Fig. 10 shows the reconstructions for the 200 mA scan (reference image) along with the reconstructions for the simulated ultra low-dose scan obtained with PWLS-EP, WavResNet, PWLS-ULTRA, and SPULTRA. Visually, WavResNet fails to reconstruct the image but improves over the initial PWLS-EP reconstruction, while PWLS-ULTRA and SPULTRA provide better image quality. This indicates that the ULTRA-based methods may have a better generalization property than WavResNet, since they learn more fundamental features of CT images (also see [38]). We selected three smooth ROIs, where the pixel values are approximately constant. Tab. III shows the mean and the standard deviation of pixel values for these ROIs for various methods and the standard-dose reference. Since the iterative RNN version of WavResNet only has small improvements over PWLS-EP, the pixel values do not change much compared with PWLS-EP. PWLS-ULTRA however reduces the bias in the central region of the image (ROI 2), but fails to correct the bias in the regions near the bones (ROI 1 and ROI 3). SPULTRA reduces the bias in the central region of the image, and also significantly corrects the bias near the bone regions. The standard deviations of the ROIs reconstructed by SPULTRA are comparable to those reconstructed by PWLS-ULTRA, and are close to those of the reference ROIs. Additionally, SPULTRA reconstructs the bone (indicated by the magenta arrow in the last two subfigures of Fig. 10) better than PWLS-ULTRA.

Fig. 10:

Reconstructions for ultra low-dose 2D scan simulated from raw measurements. The leftmost image is the PWLS-EP reconstructed image for the 200 mA scan. The second image is the PWLS-EP reconstruction for the simulated ultra low-dose scan, and it is the initial image for WavResNet [46], PWLS-ULTRA [38], and SPULTRA. The display windows are [800, 1400] HU.

TABLE III:

Mean (HU) and standard deviation (STD) (HU) of the ROIs for ultra low-dose shoulder phantom simulations

(a) Mean (HU)					(b) STD (HU)
Methods	ROI1	ROI 2	ROI 3		Methods	ROI 1	ROI 2	ROI 3
Reference	1052.1	1060.1	1053.4		Reference	8.12	8.81	6.98
PWLS-EP	1032.7	977.5	1026.3		PWLS-EP	19.45	19.45	30.46
WavResNet [46]	1037.6	981.1	1031.2		WavResNet [46]	18.91	18.91	30.16
PWLS-ULTRA [38]	1031.1	1043.0	1024.2		PWLS-ULTRA [38]	14.82	10.92	19.29
SPULTRA	1054.7	1044.0	1049.6		SPULTRA	16.34	11.42	11.60

VI. Conclusions

This paper proposes a new LDCT reconstruction method dubbed SPULTRA that combines the shifted-Poisson statistical model with the union of learned transforms or ULTRA regularizer. To deal with the nonconvex data-fidelity term arising from the shifted-Poisson model, we iteratively designed quadratic surrogate functions for this term in the proposed algorithm. In each surrogate function update iteration, the overall cost function (i.e., majorizer) has a similar structure as in the very recent PWLS-ULTRA method, and is optimized by performing an image update step with a quadratic cost and a sparse coding and clustering step with an efficient closed-form update. We evaluated the proposed SPULTRA scheme with numerical experiments on the XCAT phantom, synthesized clinical data, and beam-hardened ultra low-dose raw measurement simulations. SPULTRA outperformed prior methods in terms of eliminating bias and noise in the reconstructed image while maintaining the resolution of the reconstruction under very low X-ray doses. SPULTRA was also much faster than PWLS-ULTRA in achieving a desired reconstruction quality for low-doses, and it had better generalization property than the WavResNet based denoising scheme. Moreover, we investigated the convergence guarantees of the proposed surrogate function based alternating minimization scheme. We will investigate variations or generalizations of the SPULTRA model such as exploiting unions of overcomplete or tall transforms, or rotationally invariant transforms in future work.