Alternating Direction Method of Multiplier for Tomography With Nonlocal Regularizers

Abstract

The ordered subset expectation maximization (OSEM) algorithm approximates the gradient of a likelihood function using a subset of projections instead of using all projections so that fast image reconstruction is possible for emission and transmission tomography such as SPECT, PET, and CT. However, OSEM does not significantly accelerate reconstruction with computationally expensive regularizers such as patch-based nonlocal (NL) regularizers, because the regularizer gradient is evaluated for every subset. We propose to use variable splitting to separate the likelihood term and the regularizer term for penalized emission tomographic image reconstruction problem and to optimize it using the alternating direction method of multiplier (ADMM). We also propose a fast algorithm to optimize the ADMM parameter based on convergence rate analysis. This new scheme enables more sub-iterations related to the likelihood term. We evaluated our ADMM for 3-D SPECT image reconstruction with a patch-based NL regularizer that uses the Fair potential function. Our proposed ADMM improved the speed of convergence substantially compared to other existing methods such as gradient descent, EM, and OSEM using De Pierro’s approach, and the limited-memory Broyden–Fletcher–Goldfarb–Shanno algorithm.

I. Introduction

Incorporating noise models in tomographic image reconstruction can improve image quality. However, unlike analytical image reconstruction methods such as filtered back-projection (FBP), statistical image reconstruction methods such as the expectation-maximization (EM) algorithm [1], often require gradient-based iterative algorithms. Since the gradient of a likelihood function should be evaluated at each iteration, these algorithms (including EM) are undesirably slow.

Statistical image reconstruction for emission tomography started to be used widely in clinics and in commercial scanners after the fast algorithm called ordered-subset expectation-maximization (OSEM) was developed [2]. The main idea was to speed up gradient computation by approximating it using a subset of projections instead of using all projections (ordered-subset or OS approximation). This approximation has been used for unregularized emission tomographic image reconstruction [2] and regularized emission and transmission tomographic image reconstruction with simple quadratic or edge-preserving regularizers [3]. Since the computation cost for these regularizers is fairly low compared to that for the likelihood term, OS algorithms that approximate the gradient of the likelihood term often speed up penalized likelihood (PL) image reconstruction, too.

Recently, nonlocal (NL) regularizers have been proposed that improve image quality substantially compared to conventional regularizers such as quadratic or edge-preserving functions in image deconvolution [4], emission image reconstruction with convex functions [5], [6], and magnetic resonance imaging (MRI) image reconstruction with nonconvex functions [7]. NL regularizers have been extended to use high-resolution CT or MRI side information for emission and super-resolution image reconstruction for further improvement of image quality [8]–[11]. For emission tomography problems such as [6], [9]–[11], various optimization algorithms were used for image reconstruction such as gradient descent (GD) [10], EM (or OSEM) algorithm based on optimization transfer using De Pierro’s lemma [6], the EM algorithm using one-step late approach [11], and the quasi-Newton algorithm called the limited-memory Broyden–Fletcher–Goldfarb–Shanno with a Box constraint (L-BFGS-B) [9]. They showed promising improvement of image quality, but their algorithms were undesirably slow since the computation cost of the NL regularizers can be comparable to or even higher than that of the likelihood. Therefore, the OS approximation does not significantly accelerate the convergence rate of existing PL image reconstruction algorithms with these NL regularizers.

In this paper, we propose to use variable splitting to separate the likelihood term and the regularizer term for PL image reconstruction problem and to optimize it using the alternating direction method of multipliers (ADMM) [12]. We also propose a fast algorithm to optimize the ADMM parameter based on convergence rate analysis that was extended from the work of Ghadimi et al. with quadratic data fitting and regularizer terms [13]. There are existing methods that use variable splitting for the data fidelity term and the regularizer term [4], [14]–[16]. These previous methods address nonsmooth regularizers such as total variation (TV). A sub-problem with nonsmooth regularizers can be solved quickly by using efficient proximal operators such as shrinkage. Our proposed variable splitting has different motivation. We divide the original optimization into a few sub-problems and we solve the sub-problem related to the likelihood term using OS approximation and we update the sub-problem related to the NL regularizer less often.

We evaluated our new ADMM for 3-D SPECT image reconstruction with a patch-based convex NL regularizer that uses the Fair potential [6]. Our simulation using the XCAT phantom [17] shows that our proposed ADMM for image reconstruction with NL regularizers accelerated convergence substantially compared to existing methods such as GD, EM, and OSEM using De Pierro’s approach [3], and the L-BFGS-B algorithm [18]. This paper is organized as follows. Section II reviews statistical image reconstruction in emission tomography, OS approximation, and various NL regularizers. Section III proposes an efficient method for NL regularized image reconstruction by using variable splitting and ADMM. Section IV proposes an analytical method to select automatically a suitable ADMM parameter for fast convergence rate. Lastly, Section V presents 3-D SPECT simulation results with the XCAT phantom [17] for an application in I-131 radioimmunotherapy (RIT) [19].

II. Image Reconstruction With NL Regularizers

A. Statistical Image Reconstruction for Tomography

The usual form of statistical image reconstruction is to perform the following optimization with respect to an image f:

$$\widehat{\mathit{f}}\triangleq \underset{\mathit{f}\ge \mathbf{0}}{argmin}\mathrm{L}(\mathit{y}\mid \mathit{f})+\beta \mathrm{R}(\mathit{f})$$ (1)

where y is a measured sinogram data, L denotes a negative log-likelihood function, β is a regularization parameter, and R is a regularizer.

For emission tomography, the negative Poisson log-likelihood is

$$\mathrm{L}(\mathit{y}\mid \mathit{f})=\sum _{l\in \mathrm{\Omega }}{\overline{y}}_l(\mathit{f})-y_{l}log{\overline{y}}_l(\mathit{f})$$ (2)

where y_l is the lth element of the measurement y, Ω is the set of indexes of all measurements, and ȳ_l (f) ≜ Σ_jaljf_j + s_l where a_lj is the element of the system matrix A at the lth row and the jth column, f_j is the jth element of the image vector f, and s_l is a random and scatter component for the lth measurement. We focus on SPECT imaging where we incorporate an attenuation map and a depth-dependent point spread function model including penetration tails [20] in the system matrix A. We assume known s_l; in practice, this scatter component can be estimated by using a triple energy window (TEW) method or by Monte Carlo methods [21].

B. Ordered-Subset Approximation

Iterative image reconstruction algorithms for (1) usually require calculating the gradient of L(y|f) and R(f) at every iteration. The gradient is evaluated at f⁽ⁿ⁾ where f⁽ⁿ⁾ is an estimate of f at the nth iteration. These algorithms include GD, EM [1], and L-BFGS-B [18]. Calculating the gradient of typical R(f) such as quadratic or edge-preserving regularizers is very fast. Evaluating the gradient of L(y|f) is much slower since this requires one forward projection and one back projection of A for each iteration.

OS methods [2] approximate the gradient of L(y|f) at f⁽ⁿ⁾ with the gradient of L_k(y|f) at f^(n+k/K) where

$$L_k(\mathit{y}\mid \mathit{f})\triangleq K\sum _{l\in {\mathrm{\Omega }}_k}{\overline{y}}_l(\mathit{f})-y_{l}log{\overline{y}}_l(\mathit{f})$$ (3)

Ω_k are mutually exclusive, Ω = ∪_kΩ_k, and k = 1, ···, K. Evaluating the OS approximated gradient in (3) is about K times faster than calculating the original gradient in (2). In this way, OSEM achieves faster image reconstruction. Note that OS methods with a fixed K > 1 do not guarantee convergence, but yield approximate PL images.

C. Nonlocal Regularizers

Recently, many researchers have formed high-quality images in many image reconstruction problems by replacing R(f) in (1) with a NL regularizer [4]–[7]. A typical NL regularizer looks like

$$\mathrm{R}(\mathit{f})\triangleq \sum _{i,j\in S_i}{p}_{ij}({\Vert {\mathit{N}}_i\mathit{f}-N_j\mathit{f}\Vert }^2)$$ (4)

where p_ij(t) is a function of a scalar variable t, ||·|| is the ℓ₂ norm, S_i is the search neighborhood around the ith voxel (usually the set of all voxels within a fixed ℓ_∞ distance from the ith voxel), and N_i is an operator on the image f such that N_if is a vector of image intensities of all voxels within a fixed ℓ_∞ distance from the ith voxel (cube-shaped patch).

A typical choice for the function p_ij is [4], [5]

$$p_{ij}(t)=\frac{w_{i,j}(\stackrel{\sim }{\mathit{f}})}{2N_{\mathrm{f}}}t$$ (5)

where N_f is the number of voxels in the patch N_if (assuming that the patch size is the same for all i), a weighting function is

$$w_{i,j}(\mathit{f})=exp\left(-\frac{{\Vert {\mathit{N}}_i\mathit{f}-{\mathit{N}}_j\mathit{f}\Vert }^2}{2N_{\mathrm{f}}{\sigma }_{\mathrm{f}}^2}\right)$$ (6)

and σ_f is a design parameter. For the image f̃, Lou et al. used an initial image from any analytical image reconstruction (e.g., FBP) [5] and Zhang et al. used an estimated image from the previous iteration f⁽ⁿ⁻¹⁾ so that p_ij(t) changes over iterations [4].

Yang et al. used a few nonconvex potentials including the Welsh potential [22]

$$p_{ij}(t)={\sigma }_{\mathrm{f}}^2\left(1-exp\left(-\frac{t}{2N_f{\sigma }_{\mathrm{f}}^2}\right)\right)$$ (7)

and showed that using (7) is equivalent to using (5) with an estimated value f⁽ⁿ⁻¹⁾ for f̃ at the nth iteration [7]. Wang et al. used the Fair potential [23], [24]

$$p_{ij}(t)={\sigma }_{\mathrm{f}}^2\left(\sqrt{\frac{t}{{\sigma }_{\mathrm{f}}^2{N}_{\mathrm{f}}}}+log\left(1+\sqrt{\frac{t}{{\sigma }_{\mathrm{f}}^2{N}_{\mathrm{f}}}}\right)\right).$$ (8)

Both (7) and (8) do not depend on an initial image and (8) is convex while (7) is nonconvex. It has been reported that non-convex functions yielded better image quality than a convex function [7].

One can also design NL regularizers that incorporate high-resolution side information such as CT or MR images [9]–[11] to further improve image quality. One way to incorporate high-resolution side information into the NL regularizer is to use the following type of NL regularizer [10]:

$$p_{ij}(t)=\frac{w_{i,j}\left({\mathit{f}}^{(n-1)}\right)v_{i,j}(\mathit{g})}{2N_{\mathrm{f}}}t$$ (9)

where

$$v_{i,j}(\mathit{g})=exp\left(-\frac{{\Vert {\mathit{M}}_i\mathit{g}-{\mathit{M}}_j\mathit{g}\Vert }^2}{2N_{\mathrm{a}}{\sigma }_{\mathrm{a}}^2}\right)$$ (10)

g is a high-resolution image such as MR or CT, M_i is an operator on the image g such that M_ig is a vector of image intensities in a patch around the ith voxel, N_a is the number of voxels in the patch M_ig, and σ_a is another design parameter. Another NL regularizer incorporating high resolution side information is [9]

$$p_{ij}(t)=\frac{w_{i,j}\left({\mathit{f}}^{(n-1)}\right)+v_{i,j}(\mathit{g})}{4N_{\mathrm{f}}}t.$$ (11)

All of these NL regularizers are computationally expensive due to the calculation of (6) at each iteration. The gradients of both L(y|f) and R(f) should be evaluated at each iteration for optimization. Using OS approximations of the gradient of L(y|f) does not improve the speed of convergence much since one can not use OS approximations for R(f) so the gradient of R(f) must be evaluated at each sub-iteration.

III. Alternating Direction Method of Multipliers

A. ADMM for NL Regularization

To benefit from an OS approximation for L(y|f), while avoiding heavy computation of the gradient of R(f) at each sub-iteration, we split the variable for the likelihood term and the regularizer term by replacing (1) with the following equivalent constrained optimization problem:

$$\widehat{\mathit{f}}\triangleq \underset{\mathit{f}\ge \mathbf{0},\mathit{u}}{argmin}\mathrm{L}(\mathit{y}\mid \mathit{f})+\beta \mathrm{R}(\mathit{u}),\text{subject}\text{to}\mathit{u}=\mathit{f}.$$ (12)

The augmented Lagrangian for (12) is

$$\mathrm{L}(\mathit{y}\mid \mathit{f})+\beta \mathrm{R}(\mathit{u})+\frac{\mu }{2}{\Vert \mathit{f}-\mathit{u}-\mathit{d}\Vert }^2,\text{subject}\text{to}\mathit{f}\ge \mathbf{0}$$ (13)

where μ is a scalar value (design parameter) and d is a Lagrangian multiplier vector. We need to find a saddle point of the augmented Lagrangian (13).

We solve (13) using the ADMM algorithm [25], [26] as follows:

For n = 1,2, …

$${\mathit{u}}^{(n)}\in \underset{\mathit{u}}{argmin}\frac{\mu }{2}{\Vert \mathit{u}-{\mathit{f}}^{(n-1)}+{\mathit{d}}^{(n-1)}\Vert }^2+\beta \mathrm{R}(\mathit{u})$$ (14)

$${\mathit{f}}^{(n)}=\underset{\mathit{f}\ge \mathbf{0}}{argmin}\mathrm{L}(\mathit{y}\mid \mathit{f})+\frac{\mu }{2}{\Vert \mathit{f}-{\mathit{u}}^{(n)}-{\mathit{d}}^{(n-1)}\Vert }^2$$ (15)

$${\mathit{d}}^{(n)}={\mathit{d}}^{(n-1)}-\left({\mathit{f}}^{(n)}-{\mathit{u}}^{(n)}\right)$$ (16)

End.

For convex R, this ADMM algorithm is guaranteed to converge for any μ > 0 [26]. We can solve the sub-problems of (14) and (15) using any existing method. One need not solve the subproblems exactly to guarantee the convergence of the ADMM algorithm [26, Th. 8].

B. Optimization for the Sub-Problem (14)

We used the GD algorithm to solve (14) as follows:

$${\mathit{u}}^{(n-1)}={\mathit{u}}^{(n)}-{\alpha }^{(n)}\nabla {\mathrm{\Phi }}^{(n)}\left({\mathit{u}}^{(n)}\right)$$ (17)

where α is a step size and

$${\mathrm{\Phi }}^{(n)}(\mathit{u})\triangleq \frac{\mu }{2}{\Vert \mathit{u}-{\mathit{f}}^{(n)}+{\mathit{d}}^{(n)}\Vert }^2+\beta \mathrm{R}(\mathit{u}).$$ (18)

We plug (17) into (14) to determine the step size as follows:

$${\alpha }^{(n)}=\underset{\alpha }{argmin}{\varphi }^{(n)}(\alpha )$$ (19)

where

$${\varphi }^{(n)}(\alpha )\triangleq {\mathrm{\Phi }}^{(n)}\left({\mathit{u}}^{(n)}-\alpha \nabla {\mathrm{\Phi }}^{(n)}\left({\mathit{u}}^{(n)}\right)\right).$$ (20)

The gradient of Φ⁽ⁿ⁾(u) is

$$\nabla {\mathrm{\Phi }}^{(n)}(\mathit{u})=\mu \left(\mathit{u}-{\mathit{f}}^{(n)}+{\mathit{d}}^{(n)}\right)+\beta \nabla \mathrm{R}(\mathit{u})$$ (21)

where

$$\nabla \mathrm{R}(\mathit{u})=\sum _{i,j\in S_i}2{\stackrel{.}{p}}_{ij}({\Vert {\mathit{N}}_i\mathit{u}-{\mathit{N}}_j\mathit{u}\Vert }^2)·{({\mathit{N}}_i-{\mathit{N}}_j)}^{\prime }({\mathit{N}}_i-{\mathit{N}}_j)\mathit{u}$$ (22)

and ṗ_ij(t) is the derivative of p_ij(t). Since solving (19) is an intermediate step of solving (14), we do not need to find an exact α value to minimize (19). We chose to use one step of Newton’s method for (19) as follows [27]:

$${\alpha }^{(n)}=-\frac{{\stackrel{.}{\varphi }}^{(n)}(0)}{{\ddot{\varphi }}^{(n)}(0)}$$ (23)

where ϕ̇⁽ⁿ⁾(α) and ϕ̈⁽ⁿ⁾(α) are the first and second derivatives of ϕ⁽ⁿ⁾(α) with respect to α

$${\stackrel{.}{\varphi }}^{(n)}(0)=-{\Vert \nabla {\mathrm{\Phi }}_{\mathit{u}}^{(n)}\Vert }^2$$ (24)

where $\nabla {\mathrm{\Phi }}_{\mathit{u}}^{(n)}\triangleq \nabla {\mathrm{\Phi }}^{(n)}({\mathit{u}}^{(n)})$ and

$${\ddot{\varphi }}^{(n)}(0)\approx \nabla {\mathrm{\Phi }}_{\mathit{u}}^{{(n)}^{\prime }}\left(\mu \nabla {\mathrm{\Phi }}_{\mathit{u}}^{(n)}+\nabla \mathrm{R}\left(\nabla {\mathrm{\Phi }}_{\mathit{u}}^{(n)}\right)\right).$$ (25)

We approximate ϕ̈⁽ⁿ⁾(α) by excluding the second derivative of p(t) as suggested in [28, p. 683].

C. Optimization for the Sub-Problem (15)

One can solve (15) using any statistical image reconstruction algorithms with a slight modification for a shifted quadratic regularizer. OS approximation can be usually used to speed up convergence rate. Using this splitting, one may focus more computational resources on solving the sub-problem (15) instead of solving the sub-problem (14) so that one may achieve faster overall convergence rate.

We modified De Pierro’s EM algorithm [3] for (15) by considering a simple shifted quadratic regularizer. The surrogate function for the likelihood term in (15) is

$$\mathrm{L}(\mathit{y}\mid \mathit{f})\le \sum _j{\stackrel{\sim }{Q}}_j\left(f_j\mid {\mathit{f}}^{(n-1)}\right)$$ (26)

where Q̃_j(f_j|f⁽ⁿ⁻¹⁾) is a surrogate function that can be found in [3] and [6]. We use a surrogate function Q_j(f_j|f⁽ⁿ⁻¹⁾) that is equivalent to Q̃_j(f_j|f⁽ⁿ⁻¹⁾) by omitting the terms that are independent of f_j as follows:

$$Q_j\left(f_j\mid {\mathit{f}}^{(n-1)}\right)=a_j{f}_j-e_j\left({\mathit{f}}^{(n-1)}\right)f_j^{(n-1)}log(f_j)$$ (27)

where

$$a_j=\sum _l{a}_{lj}$$

and

$$e_j(\mathit{f})=\sum _l{a}_{lj}\frac{y_l}{{\overline{y}}_l(\mathit{f})}.$$ (28)

Note that Q̃_j(f_j|f⁽ⁿ⁻¹⁾) and Q_j(f_j|f⁽ⁿ⁻¹⁾) are minimized at the same f_j.

The regularizer in (15) is separable in the image domain

$$\frac{\mu }{2}{\Vert \mathit{f}-{\mathit{u}}^{(n)}-{\mathit{d}}^{(n-1)}\Vert }^2=\sum _j\frac{\mu }{2}{\left(f_j-u_j^{(n)}-d_j^{(n-1)}\right)}^2.$$ (29)

Therefore, one must minimize the following surrogate function to solve the sub-problem in (15):

$$Q_j\left(f_j\mid {\mathit{f}}^{(n-1)}\right)+\frac{\mu }{2}{\left(f_j-u_j^{(n)}-d_j^{(n-1)}\right)}^2$$ (30)

for all j. Differentiating (30) with respect to f_j and setting it to be zero leads to the following second-order polynomial with respect to f_j

$$\mu f_j^2+\left(a_j-\mu \left(u_j^{(n)}+d_j^{(n-1)}\right)\right)f_j-e_j\left({\mathit{f}}^{(n-1)}\right)f_j^{(n-1)}=0.$$ (31)

The nonnegative root of (31) is the minimizer of (30). This root always exists because e_j(f⁽ⁿ⁻¹⁾) and $f_j^{(n-1)}$ are nonnegative [29].

An OS approximation for this modified De Pierro’s algorithm can be easily done by substituting the most time-consuming part (28) in (15) at each iteration with the following approximate term

$$e_j(\mathit{f})\approx K\sum _{l\in {\mathrm{\Omega }}_k}{a}_{lj}\frac{y_l}{{\overline{y}}_l(\mathit{f})}.$$ (32)

This new term (32) requires about K times less computation than the term (28) does. Since calculating (28) dominates the overall calculation of (31) for each iteration, OS approximation for (15) substantially reduces the computation time per update.

In this Section, we proposed to use variable splitting and ADMM for efficient computation of NL regularized image reconstruction in (14), (15), and (16). We also described detailed algorithms for each sub-problem: gradient descent with one step of Newton’s method for (14) and modified De Pierro’s OSEM algorithm for (15). Even though our proposed ADMM algorithm for convex R guarantees to converge for any μ, the parameter μ in (13) affects convergence rate. In the next Section, we propose a method to optimize μ automatically.

IV. Parameter Selection for ADMM

A. Ideal ADMM Update and Approximation

Ghadimi et al. optimized μ in ADMM for ℓ₂-regularized minimization with a quadratic data fitting term [13]. We generalize and extend their analysis to optimize μ for our case of having NL regularizers with the negative Poisson likelihood term.

Let us derive the “ideal” ADMM update for (14), (15), and (16). Using the gradient of (14) with respect to u, the first update for (14) at the nth iteration is

$${\mathit{u}}^{(n)}={\left[\mathit{I}+\frac{\beta }{\mu }\mathit{R}\left({\mathit{u}}^{(n-1)}\right)\right]}^{-1}\left({\mathit{f}}^{(n-1)}-{\mathit{d}}^{(n-1)}\right)$$ (33)

where the approximate Hessian of (4) is

$$\mathit{R}(\mathit{u})\triangleq \sum _{i,j\in S_i}2{\stackrel{.}{p}}_{ij}\left({\Vert ({\mathit{N}}_i-{\mathit{N}}_j)\mathit{u}\Vert }^2\right){({\mathit{N}}_i-{\mathit{N}}_j)}^{\prime }({\mathit{N}}_i-{\mathit{N}}_j).$$

We used the GD algorithm in (17) since (33) is impractical. Similarly, using the gradient of (15) with respect to f and ignoring the non-negative constraint for f, “ideally” the update for (15) at the nth iteration is

$${\mathit{f}}^{(n)}={\left[{\mathit{A}}^{\prime }{\mathit{W}}^{(n-1)}\mathit{A}+\mu \mathit{I}\right]}^{-1}\times \left({\mathit{A}}^{\prime }{\mathit{W}}^{(n-1)}(\mathit{y}-\mathit{s})+\mu {\mathit{u}}^{(n)}+\mu {\mathit{d}}^{(n-1)}\right)$$ (34)

where

$${\mathit{W}}^{(n-1)}\triangleq \mathit{D}\left(\frac{1}{{\overline{y}}_l\left({\mathit{f}}^{(n-1)}\right)}\right).$$

Because (34) is impractical, we used De Pierro’s EM algorithm in (31). We can use (16) itself as the third “ideal” ADMM update at the nth iteration.

To facilitate the analysis using these ideal ADMM updates (33), (34), and (16), we need to fix R(u⁽ⁿ⁻¹⁾) and W⁽ⁿ⁻¹⁾ for all iterations or for all n. We conjecture that we can set u⁽ⁿ⁻¹⁾ = f⁽ⁿ⁻¹⁾ = f̂^init for R(u⁽ⁿ⁻¹⁾) and W⁽ⁿ⁻¹⁾ so that we can fix

$$\mathit{R}\triangleq \mathit{R}({\widehat{\mathit{f}}}^{\text{init}})$$ (35)

and

$$\mathit{W}\triangleq \mathit{D}\left(\frac{1}{{\overline{y}}_l({\widehat{\mathit{f}}}^{\text{init}})}\right).$$ (36)

We used these approximations only for selecting μ. The initial image f̂^init can be anything like the FBP image or the reconstructed image using the OSEM with a few iterations. For the results in Section V, we used the reconstructed image using un-regularized OSEM with five iterations (six subsets). Please see Appendix A for technical details for the approximation of (35) and (36).

B. Nearly Optimal ADMM Parameter

To study the convergence rate, we need to derive the update equation of f⁽ⁿ⁾ in terms of f⁽ⁿ⁻¹⁾. Here, we generalize the approach of [13] for our “ideal” ADMM update equations for (14)–(16).

First, rearrange (34) with (36) and use it for (16). Then, (16) becomes

$${\mathit{d}}^{(n)}=\frac{1}{\mu }{\mathit{A}}^{\prime }\mathit{W}\left[\mathit{A}{\mathit{f}}^{(n)}+\mathit{s}-\mathit{y}\right]$$ (37)

and this also works for n − 1. Secondly, we use (37) with n − 1 for (14) with (35) to obtain

$${\mathit{u}}^{(n)}={\left[\mathit{I}+\frac{\beta }{\mu }\mathit{R}\right]}^{-1}\left(\mathit{I}-\frac{1}{\mu }{\mathit{A}}^{\prime }\mathit{WA}\right){\mathit{f}}^{(n-1)}+\mathit{c}$$ (38)

where c is a constant vector. Lastly, combining (37) (with n − 1), (38), and (34) [with (36)] yields

$${\mathit{f}}^{(n)}={\mathit{S}}_{\mu }{\mathit{f}}^{(n-1)}+\stackrel{\sim }{\mathit{c}}$$ (39)

where c̃ is a constant vector and

$${\mathit{S}}_{\mu }\triangleq {[\mathit{H}+\mu \mathit{I}]}^{-1}{[\beta \mathit{R}+\mu \mathit{I}]}^{-1}(\beta \mathit{RH}+{\mu }^2\mathit{I})$$ (40)

where H ≜ A′WA.

With the goal of approximately optimizing the convergence rate, we choose the ADMM parameter μ as follows:

$${\mu }^{\ast }=arg\underset{\mu }{min}\rho ({\mathit{S}}_{\mu })$$ (41)

where ρ(S_μ) is the spectral radius of the matrix S_μ, which is the maximum eigen value of the matrix S_μ. However, it is very challenging to find eigen values for large matrices such as S_μ and it is also infeasible to find the matrix S_μ itself due to the inversion of large matrices such as H + μI and βR + μI.

To make the problem (41) tractable, we approximately optimize the ADMM parameter μ for the center of the image and use that parameter everywhere else. We approximate H and R as being locally circulant around that local area; a similar assumption often is used when analyzing spatial resolution properties [30], [31]. Then, the eigen value of the matrix S_μ for the pth eigen vector (or the pth discrete Fourier basis) becomes

$${\lambda }_p({\mathit{S}}_{\mu })=\frac{\beta r_p{h}_p+{\mu }^2}{(h_p+\mu )(\beta r_p+\mu )}$$ (42)

where r_p and h_p are the eigen values of R and H for the corresponding pth eigen vector, respectively. Then, our choice for the ADMM parameter μ is

$${\mu }^o=arg\underset{\mu \in \mathrm{\Lambda }}{min}\underset{p}{max}{\lambda }_p({\mathit{S}}_{\mu }).$$ (43)

Unlike the case in [13], it is not easy to find the analytical solution for (43). Instead, we can quickly solve the problem (43). We find r_p and h_p for all p using the fast Fourier transform (FFT) of R1_c and H1_c, respectively, where 1_c is a unit vector where the element of 1_c is one at the center voxel of the image and zero otherwise. Then, we calculated max_p λ_p(S_μ) for a finite set of μ values on $\mathrm{\Lambda }=[0,{max}_p\sqrt{\beta r_p{h}_p}]$, and then obtain μ^∘ from max_p λ_p(S_μ). Since λ_p(S_μ) is a nondecreasing function of μ for $\mu >\sqrt{\beta r_p{h}_p}$, μ^o must be less than or equal to ${max}_p\sqrt{\beta r_p{h}_p}$. The computational cost for evaluating max_p λ_p(S_μ) is linear in the number of voxels. This procedure is fast since it is parallelizable. One can speed up this procedure using coarse-to-fine approach. One could also use the Golden section search [28, p. 397] to find μ^∘ to minimize max_p λ_p(S_μ).

V. Result

A. Simulation Setup

We simulated the Siemens Symbia Truepoint 3-D SPECT system with high energy collimators (parallel hexagonal collimators with a septal thickness of 2 mm, a hole diameter of 4 mm, and a hole length of 59.7 mm) with a nonuniform attenuation map, depth-dependent collimator-detector response [20], and scatter component (128 × 21, 4.8² mm² pixel size). The system resolution at 10 cm was 13.4 mm and 60 views were collected around 360°. We used the XCAT phantom [17] to generate the true SPECT image with activity distributions realistic for I-131 RIT. The dimension of the SPECT image was 128 × 128 × 21, 4.8³ mm³ voxel size. Three spherical lesions were placed within the XCAT phantom with volumes 176 cc, 32 cc, and 9 cc as shown in the bottom-right figure of Fig. 4(a) or (b). Poisson noise was added after scaling the projections to the count-level corresponding to day 2 post-therapy in I-131 RIT (about 600 K total counts per slice with about 300 K scatter counts per slice).

Fig. 4.

Estimated images of different methods at 500 and 1000 s and the true image for the case of the Fair potential. ADMM yielded the best contrast recovery among all other methods. (a) 500 s. (b) 1000 s.

We set the common regularization parameters for all optimization methods as follows: β = 2⁻¹³, σ_f = 2^1.5, the patch size 3 × 3× 3, and the search neighborhood size 7 × 7× 7. Six subsets were used for OSEM and ADMM. ADMM separates the likelihood update and the regularizer update by split-ting and we chose to run more sub-iterations for the likelihood update (2 outer-iterations ×6 subsets) than for the regularizer update (one outer-iteration). We used six threads for computation (Intel Core i7 2.8 GHz) for all methods and they used the same compiled ANSI C99 code to evaluate the cost function and the gradient of the cost function. We measured the normalized root mean square error (RMSE) over the whole image at each (outer) iteration

$$\text{RMSE}=\frac{\Vert \widehat{\mathit{f}}-{\mathit{f}}_{\text{TRUE}}\Vert }{\Vert {\mathit{f}}_{\text{TRUE}}\Vert }.$$ (44)

B. Parameter Selection

There are parameters to tune for each optimization method. We selected the step size for GD and the number of past estimated images for hessian approximation of L-BFGS-B. We chose those values to yield the fastest convergence rate empirically: the step size for GD was 0.04 and the number of past estimated images for L-BFGS-B was 5. In our experiment, not shown here, the step size for GD was critical for convergence. GD diverged with too large step size and GD converged slowly with too small step size. However, the number of past images used for L-BFGS-B did not affect convergence rate much.

We selected the ADMM parameter μ using (43). We first obtained max_p λ_p(S_μ) for μ = 0, 0.0001, 0.0002, ···, 0.2125 as shown in Fig. 1 where ${max}_p\sqrt{\beta r_p{h}_p}=0.2125$. Based on this plot, we chose the ADMM parameter μ to be 0.0106, which is the red star mark (*) in Fig. 1.

Fig. 1.

Plot of max_p λ_p(S_μ) for μ = 0, 0.0001, 0.0002, ···, 0.1. Red star (*) denotes our choice of μ^∘.

We evaluated the ADMM convergence rate as a function of μ empirically. Fig. 2 shows that our choice of ADMM parameter μ = 0.0106 achieved reasonably fast convergence compared to other choices of μ. Too large μ such as μ = 0.1250 (green square mark) yielded slower convergence rate and too small μ value such as μ = 0.0005 (magenta triangle mark) resulted in fluctuating tails. Due to the approximations such as local shift invariance and fixing some matrices like R and W, we can not claim that our choice of μ is optimal, but Fig. 2 suggests that our choice is adequate for fast convergence of ADMM.

Fig. 2.

RMSEs of estimated images over time using ADMM with different μ values. Atomatically selected value (μ = 0.0106, red star *) yielded relatively fast convergence rate of ADMM compared to other choices of μ.

C. Simulation Results for NL Regularization

We reconstructed images using different optimization algorithms for the cost function (1) with the (convex) Fair potential (8).

Fig. 3 shows the plots of RMSE values versus computation time for different methods: GD, EM, and OSEM using De Pierro’s lemma, L-BFGS-B, and proposed ADMM. EM yielded faster convergence rate than GD with fixed step size. OSEM does improve convergence speed as compared to EM, but it provides little acceleration due to computationally expensive NL regularizer calculation for all sub-iterations. L-BFGS-B yielded similar convergence rate to OSEM with six subsets. Our proposed ADMM substantially improved convergence speed over all other methods. Other methods did not reach the minimum RMSE before 2000 s, but ADMM achieved the minimum RMSE before 1000 s. These simulation results illustrate that repeated likelihood updates are more important for fast convergence than regularizer updates.

Fig. 3.

RMSEs of estimated images using different algorithms versus time for NL regularization with the Fair potential. Proposed ADMM showed faster convergence rate than other methods.

Fig. 4(a) and (b) shows reconstructed images by different methods at 500 and 1000 s and the true image for NL regularization with the Fair potential. At this early time of 500 s, ADMM yielded the best contrast recovery among all other methods. As time goes by (at 1000 s), other methods also started to yield similar images to that of ADMM since all optimization methods minimize the almost same cost function. However, they may not be exactly the same due to the OS approximation.

Fig. 5(a)–(c) shows recovery coefficients (RCs) of different methods over time when using the Fair potential. The larger the lesion is, the faster it approached to the achievable RC value. Note that we did not optimize NL regularizer parameters such as σ_f and one may achieve better RC for smaller lesions like 9 cc than that in Fig. 5(c).

Fig. 5.

Recovery coefficients for different size lesions over time when using the Fair potential. ADMM yielded the best recovery coefficients among all other methods. (a) 176 cc. (b) 32 cc. (c) 9 cc.

VI. Discussion

We developed a new algorithm for tomography with computationally expensive NL regularizers using ADMM. We also proposed a method to determine automatically a suitable μ value for fast convergence rate. By combining with the OS approach, our proposed ADMM approached convergence much faster than existing methods such as GD, EM, and OSEM using the De Pierro lemma, and L-BFGS-B. Since it seems more important to update the likelihood part frequently, our ADMM yielded faster convergence. Since the cost function has both the likelihood term and the regularization term, increasing the number of iterations for the likelihood term did not always further accelerate convergence. We chose a good combination of iterations for both terms empirically. Similarly, one could use an approximate gradient for the NL regularizer to reduce computation, but in our simulation not shown here, our ADMM outperformed this approximation.

In this paper, we minimized the same NL regularized cost function (the Fair potential) as Wang et al. did [6]. Whereas Wang et al. used De Pierro’s algorithm with the surrogate function of their NL regularizer, we use De Pierro’s algorithm with a shifted quadratic regularizer, which requires far less computation. Zhang et al. also applied a splitting approach to the optimization problem with NL regularizers [4]. However, their motivation for splitting was to apply shrinkage operator to the nonsmooth potential function such as TV. In addition, our way of splitting in (12) was different from theirs. Xu et al. used the same type of splitting as ours for the case of using non-smooth regularizer [16]. They used a similar formula for the sub-problem of (15) except a Lagrangian multiplier vector, but their motivation was to deal with nonsmooth regularizer rather than to deal with computation-intensive NL regularizer.

Our approach to optimizing μ was based on ‘ideal’ updates. In other words, we assumed fully converged images for sub-iterations of (14) and (15). Nevertheless, our optimized μ worked well even for sub-iterations that did not converge fully. This may be because a few sub-iterations yielded good approximations of fully converged images for the sub-problems (14) and (15).

The proposed method worked well for SPECT image reconstruction with the patch-based convex Fair potential function [6]. Our proposed method can be easily extended to other computationally expensive NL regularizers [4], [5], [7] and NL regularizers that use high-resolution side information [9]–[11] for both emission and transmission tomography. Even though ADMM works only with convex cases theoretically, our proposed ADMM can be a practical method for many nonconvex NL regularizers. Improving image quality with proper regularizers and appropriate regularization parameter selection using our fast ADMM algorithm will be an important and interesting future work.

Appendix A. Approximation for Ideal ADMM Update

For R, even though f̂^init and u⁽ⁿ⁻¹⁾ differ substantially, the patch selection operator N_i − N_j and the ℓ₂ norm can make this difference much smaller. Therefore, it seems reasonable to use the approximation (35). For W, this type of approximation in (36) has been used in different gradient-based analysis such as mean-variance analysis [32], spatial resolution analysis [30], [31], and noise analysis [33]. Even though there may be non-negligible difference between f̂^init and f⁽ⁿ⁻¹⁾, the approximation A′WA ≈ A′W⁽ⁿ⁻¹⁾A still holds fairly well. Thus, it also seems reasonable to use the approximation (36).