Optimization Methods for Magnetic Resonance Image Reconstruction: Key Models and Optimization Algorithms

Abstract

The development of compressed sensing methods for magnetic resonance (MR) image reconstruction led to an explosion of research on models and optimization algorithms for MR imaging (MRI). Roughly 10 years after such methods first appeared in the MRI literature, the U.S. Food and Drug Administration (FDA) approved certain compressed sensing methods for commercial use, making compressed sensing a clinical success story for MRI. This review paper summarizes several key models and optimization algorithms for MR image reconstruction, including both the type of methods that have FDA approval for clinical use, as well as more recent methods being considered in the research community that use data-adaptive regularizers. Many algorithms have been devised that exploit the structure of the system model and regularizers used in MRI; this paper strives to collect such algorithms in a single survey.

I. Introduction

A. Scope

Although the paper title begins with “optimization methods,” in practice one first defines a model and cost function, and then applies an optimization algorithm. There are several ways to partition the space of models, cost functions and optimization methods for MRI reconstruction, such as: smooth vs non-smooth cost functions, static vs dynamic problems, single-coil vs multiple-coil data. This paper focuses on the static reconstruction problem because the dynamic case is rich enough to merit its own survey paper. This paper emphasizes algorithms for multiple-coil data (parallel MRI) because modern systems all have multiple channels and advanced reconstruction methods with under-sampling are most likely to be used for parallel MRI scans. Main families of parallel MRI methods include “SENSE” methods that model the coil sensitivities in the image domain [1], “GRAPPA” methods that model the effect of coil sensitivity in k-space [2], and “calibrationless” methods that use low-rank or joint sparsity properties [3]. This paper considers all three approaches, emphasizing SENSE methods for simplicity.¹

B. Measurement model

The signals recorded by the sensors (receive coils) in MR scanners are linear functions of the object’s transverse magnetization. That magnetization is a complicated and highly nonlinear function of the RF pulses, gradient waveforms, and tissue properties, governed by the physics of the Bloch equation [4]. Quantifying tissue properties using nonlinear models is a rich topic of its own [5], but we focus here on the problem of reconstructing images of the transverse magnetization from MR measurements.

Ignoring noise, a vector $s\in {ℂ}^M$ of signal samples recorded by a MR receive coil is related (typically) to a discretized version $x\in {ℂ}^N$ of the transverse magnetization via a linear Fourier relationship:

$$s=Fx,F_{ij}=\text{exp}\left(-ı2\pi {\overrightarrow{\nu }}_i\cdot {\overrightarrow{\text{x}}}_j\right),\begin{array}{l}i=1,\dots ,M\hfill \\ j=1,\dots ,N,\hfill \end{array}$$ (1)

where ${\overrightarrow{v}}_i$ denotes the k-space sample location of the ith sample (units cycles/cm) and ${\overrightarrow{x}}_j$ denotes the spatial coordinates of the center of the jth pixel (units cm). In the usual case where the pixel coordinates $\left\{{\overrightarrow{x}}_j\right\}$ and k-space sample locations $\left\{{\overrightarrow{\nu }}_i\right\}$ are both on appropriate Cartesian grids, matrix F is square corresponds to the (2D or 3D) discrete Fourier transform (DFT). In this case $F^{-1}=\frac{1}{N}{F}^{\prime }$ so reconstructing x from s is simply an inverse fast Fourier transform (FFT), and that approach is used in many clinical MR scans.

The reconstruction problem becomes more interesting when the k-space sample locations are on a non-Cartesian grid [6], when the scan is “accelerated” by recording M < N samples, when non-Fourier effects like magnetic field inhomogeneity are considered [7] and/or when there are multiple receive coils. In parallel MRI, let s_l denote the samples recorded by the lth of of L receive coils. Then one replaces the model (1) with

$$s_l=F{C}_{l}x,$$ (2)

where C_l is a N × N diagonal matrix containing the coil sensitivity pattern of the l coil on its diagonal. Note that F does not depend on l; all coils see the same k-space sampling pattern. Stacking up the measurements from all coils and accounting for noise yields the following basic forward model in MRI:

$$\left[\begin{array}{c}{y}_1\\ ⋮\\ y_L\end{array}\right]=y=\underset{A}{\underbrace{\left(I_L\otimes F\right)C}}x+\epsilon ,C=\left[\begin{array}{c}{C}_1\\ ⋮\\ C_L\end{array}\right],$$ (3)

where $A\in {ℂ}^{ML\times N}$ denotes the system matrix, $y\in {ℂ}^{ML}$ denotes the measured k-space data, and $x\in {ℂ}^N$ denotes the latent image. The noise in k-space is well modeled as complex white Gaussian noise. For extensions that consider other physics effects like relaxation and field inhomogeneity, see [8].

The goal in MR image reconstruction is to recover x from y using the model (3). All MR image reconstruction problems are under-determined because the magnetization of the underlying object being scanned is a space-limited continuous-space function on ${ℝ}^3$, yet only a finite number of samples are recorded. Nevertheless, the convention in MRI is to treat the object as a finite-dimensional vector $x\in {ℂ}^N$ for which M ≥ N appropriate Cartesian k-space samples is considered “fully sampled” and any M < N is considered “accelerated.” Sampling pattern design is a topic of ongoing interest, with renewed interest in data-driven methods [9].

The matrix F in (3) is known prior to the scan, because the k-space sample locations $\left\{{\overrightarrow{\nu }}_i\right\}$ are controlled by the pulse sequence designer. In contrast, the coil sensitivity maps {C_l} depend on the exact configuration of the receive coils for each patient. To use the model (3), one must determine the sensitivity maps from some patient-specific calibration data, e.g., by joint estimation [10], regularization [11], or subspace methods [12].

II. Cost functions and algorithms

A. Quadratic problems

When ML ≥ N, i.e., when the total number of k-space samples acquired across all coils exceeds the number of unknown image pixel values, the linear model (3) is over-determined. If additionally A is well conditioned, which depends on the sampling pattern and coil sensitivity maps, then it is reasonable to consider an ordinary least-squares estimator

$$\begin{gathered} \widehat{x}=\underset{x\in {ℂ}^N}{\text{arg}\text{min}}\frac{1}{2}\Vert Ax-y{\Vert }_2^2={\left(A^{\prime }A\right)}^{-1}{A}^{\prime }y \\ ={\left({\sum }_{l=1}^L{C}_l^{\prime }{F}^{\prime }F{C}_l\right)}^{-1}\left({\sum }_{l=1}^L{C}_l^{\prime }{F}^{\prime }y\right). \end{gathered}$$ (4)

In particular, for fully sampled Cartesian k-space data where $F^{-1}=\frac{1}{N}{F}^{\prime }$, this least-squares solution simplifies to $\widehat{x}={\left({\sum }_{l=1}^L{C}_l^{\prime }{C}_l\right)}^{-1}\left({\sum }_{l=1}^L{C}_l^{\prime }{F}^{-1}y\right)$, which is trivial to implement because each C_l is diagonal. This is known as the optimal coil combination approach [13]. For regularly under-sampled Cartesian data, where only every nth row of k-space is collected, the matrix F′F has a simple block structure with n × n blocks that facilitates non-iterative block-wise computation known as SENSE reconstruction [1]. This form of least-squares estimation is used widely in clinical MR systems.

B. Regularized least-squares

For under-sampled problems (ML < N) the LS solution (4) is not unique. Furthermore, even when ML ≥ N often A is poorly conditioned, particularly for non-Cartesian sampling. Some form of regularization is needed in such cases. Some early MRI reconstruction work used quadratically regularized cost functions leading to optimization problems of the form:

$$\widehat{x}=\underset{x\in {ℂ}^N}{\text{arg}\text{min}}\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\beta \Vert Tx{\Vert }_2^2,$$ (5)

where β > 0 denotes a regularization parameter and T denotes a K × N matrix transform such as finite differences. The conjugate gradient (CG) algorithm is well-suited to such quadratic cost functions [6], [7]. The Hessian matrix A′A + βT′T often is approximately Toeplitz [14], so CG with circulant preconditioning is particularly effective. Although the quadratically regularized least-squares cost function (5) is passé in the compressed sensing era, CG is often an inner step for optimizing more complicated cost functions [15].

C. Edge-preserving regularization

The drawback of the quadratically regularized cost function (5) with T as finite differences is that it blurs image edges. To avoid this blur, one can replace the quadratic regularizer $\Vert Tx{\Vert }_2^2$ with a nonquadratic function ψ(Tx) where typically ψ is convex and smooth, such as the Huber function, a hyperbola, or the Fair potential function ψ(z) = δ² (|z/δ| − log(1 + |z/δ|)), among others as follows:

$$\widehat{x}=\underset{x\in {ℂ}^N}{\text{arg min}}\Psi (x)\triangleq \frac{1}{2}\Vert Ax-y{\Vert }_2^2+\beta \psi (Tx).$$ (6)

Such methods have their roots in Bayesian methods based on Markov random fields. The nonlinear CG algorithm is an effective optimization method for cost functions with such smooth edge-preserving regularizers. Another appropriate optimization algorithm is the optimized gradient method (OGM), a first-order method having optimal worst-case performance among all first-order algorithms for convex cost functions with Lipschitz continuous gradients [16]. OGM has a convergence rate bound that is twice better than that of Nesterov’s fast gradient method [17].

Fig. 1 compares two of these methods for the case where T is finite differences and ψ is the Fair potential with δ = 0.1, which approximates TV fairly closely while being smooth.

Fig. 1.

Comparison of CG and OGM convergence for single-coil MRI reconstruction with edge-preserving regularization (akin to anisotropic TV with corner rounding). From left to right: Top row: k-space sampling pattern where only 34% of the phase-encodes are collected, true image, initial image from zero-filled k-space data, minimizer $\widehat{x}$ of (6). (Both CG and OGM converge to the same limit $\widehat{x}$.) Bottom row: cost function Ψ(x_k) in (6) and normalized root mean squared error (NRMSE) ‖x_k − x‖₂ / ‖x‖₂ versus iteration k.

D. Sparsity models: synthesis form

Scan time in MRI is proportional to the number of k-space samples recorded. Reducing scan time in MRI can reduce cost, improve patient comfort, and reduce motion artifacts. Reducing the number of k-space samples ML to well below N, necessitates stronger modeling assumptions about x, and sparsity models are prevalent [18] [19]. Two main categories of sparsity models are the synthesis approach and the analysis approach. In a synthesis model, one assumes x = Bz for some N × K matrix B where coefficient vector $z\in {ℂ}^K$ should be sparse. In an analysis model, one assumes Tx is sparse, for some K × N transformation matrix T.

A typical cost function for a synthesis model is

$$\widehat{x}=B\widehat{\mathit{z}},\widehat{\mathit{z}}=\underset{\mathit{z}\in {ℂ}^K}{\text{arg}\text{min}}\frac{1}{2}\Vert ABz-y{\Vert }_2^2+\beta \Vert z{\Vert }_1,$$ (7)

where the 1-norm is a convex relaxation of the ℓ₀ counting measure that encourages z to be sparse. Typically B is a wide matrix (often called an over-complete dictionary) so that one can represent x well using only a fraction of the columns of B. The classical approach for (7) is the iterative soft thresholding algorithm (ISTA) [20], also known as the proximal gradient method (PGM) [21] and proximal forward-backward splitting [22], having the simple form

$$z_{k+1}=\text{soft}\left(z_k-D^{-1}{B}^{\prime }{A}^{\prime }\left(AB{z}_k-y\right),\beta /d\right),$$ (8)

where the soft thresholding function is defined by soft(z, c) = sign(z) max(|z| − c, 0) and D = diag{d} is any positive definite diagonal matrix such that D – B′A′AB is positive semidefinite [23].

The ISTA update (8) applies to the 1-norm in (7). If we replace that 1-norm with some other function ψ(z) then one replaces (8) with the more general PGM update of the form

$$z_{k+1}={\text{prox}}_{\beta \psi }\left(z_k-D^{-1}{B}^{\prime }{A}^{\prime }\left(AB{z}_k-y\right),\beta /d\right),$$

where the proximal operator of a function f is defined by

$${\text{prox}}_f(v)\triangleq \underset{x}{\text{arg}\text{min}}\frac{1}{2}\Vert x-v{\Vert }_2^2+f(x).$$ (9)

Traditionally D = ⦀B′A′AB⦀₂I, but computing that spectral norm (via the power iteration) requires considerable computation for parallel MRI problems in general. However, for Cartesian sampling, F′F ≼ NI so it suffices to have NB′C′CB ≼ D. Often the sensitivity maps are normalized such that C′C = I in which case NB′B ≼ D suffices. If in addition B′ is a Parseval tight frame, then B′B ≼ I so using D = NI is appropriate. For non-Cartesian sampling, or non-normalized sensitivity maps, or general choices of B, finding D is more complicated [23].

Although ISTA is simple, it has an undesirably slow O(1/k) convergence bound, where k denotes the number of iterations. This limitation was first overcome by the fast iterative soft thresholding algorithm (FISTA) [24], also known as the fast proximal gradient method (FPGM) that has an O(1/k²) convergence bound. A recent extension is the proximal optimized gradient method (POGM) that has worst-case convergence bound about twice better than that of FISTA/FPGM [25]. Both FISTA and POGM are essentially as simple to implement as (8). Recent MRI studies have shown POGM converging faster than FISTA, as one would expect based on the convergence bounds [26], particularly when combined with adaptive restart [25]. So POGM (with restart) is a recommended method for optimization problems having the form (7). Figure 2 provides POGM pseudo-code for solving composite optimization problems like the MRI synthesis reconstruction model (7).

Fig. 2.

Comparison of ISTA/PGM, FISTA/FPGM and POGM for single-coil MRI reconstruction with orthogonal discrete wavelet transform sparsity regularizer using the 1-norm. Minimizer $\widehat{x}$ of (7); NRMSE versus iteration k. FISTA requires about 40% more iterations to converge than POGM, consistent with the 2× better worst-case bound of POGM.

Fig. 2 shows that POGM converges faster than FISTA and ISTA for minimizing (7).

E. Sparsity models: analysis form

A potential drawback of the synthesis formulation (7) is that x ≈ Bz may be a more realistic assumption than the strict equality x = Bz when z is sparse. The analysis approach avoids constraining $\widehat{x}$ to lie in any such subspace (or union of subspaces when B is wide). For an analysis form sparsity model, a typical optimization problem involves a composite cost function consisting of the sum of a smooth term and a non-smooth term:

$$\widehat{x}=\underset{x}{\text{arg}\text{min}}\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\beta \Vert Tx{\Vert }_1,$$ (10)

where T is a sparsifying operator such as a wavelet transform, or finite differences, or both [18]. The expression (10) is general enough to handle combinations of multiple regularizers, such as wavelets and finite differences [19], by stacking the operators in T and possibly allowing a weight 1-norm. When T is finite differences, the regularizer is called total variation (TV) [11], and combinations of TV and wavelet transforms are useful [19]. Although the details are proprietary, the FDA-approved method for compressed sensing MRI for at least one manufacturer is related to (10) [27].

When T is invertible, such as an orthogonal wavelet transform, one rewrites the optimization problem (10) as

$$\widehat{x}=T^{-1}\widehat{z},\widehat{z}=\underset{z}{\text{arg}\text{min}}\frac{1}{2}{\Vert A{T}^{-1}z-y\Vert }_2^2+\beta \Vert z{\Vert }_1,$$

which is simply a special case of (7) with B = T⁻¹. Typically B is wide and T is tall so this simplification is not possible in general.

In the general case (10) where T is not invertible, the optimization problem is much harder than (7) due to the non-differentiability of the 1-norm with the matrix T. The non-invertible case (with redundant Haar wavelets) is used clinically [27]. The PGM for (10) is

$$x_{k+1}=\underset{x}{\text{arg}\text{min}}\frac{\mathcal{L}}{2}{\Vert x-{\tilde{x}}_k\Vert }_2^2+\beta \Vert Tx{\Vert }_1,$$ (11)

where ${\tilde{x}}_k\triangleq x_k-\frac{1}{\mathcal{L}}{A}^{\prime }\left(A{x}_k-y\right)$ denotes the usual gradient update and the Lipschitz constant is $\mathcal{L}={|\Vert A\Vert |}_2^2$. Unfortunately there is no simple solution for computing the proximal operator (9) in (11) in general, so inner iterative methods are required, typically involving dual formulations [24]. This challenge makes PGM and FPGM and POGM less attractive for (10) and has led to a vast literature on algorithms for problems like (10), with no consensus on what is best. The difficulty of (11) is the main drawback of analysis regularization, whereas a possible drawback of the synthesis regularization in (7) is that often K ≫ N for overcomplete B.

1). Approximate methods:

One popular “work around” option is to “round the corner” of the 1-norm, making smooth approximations like $|z|\approx \sqrt{|z{|}^2+ϵ}$. This approximation is simply the hyperbola function that has a long history in the edge-preserving regularization literature. All of the gradient-based algorithms mentioned for edge-preserving regularization above are suitable candidates when a smooth function replaces the 1-norm. Smooth functions can shrink values towards zero, but their proximal operators never have a thresholding effect that induces sparsity by setting many values exactly to zero. Whether a thresholding effect is truly essential is an open question. Hereafter we focus on methods that tackle the 1-norm directly without any such approximations.

2). Variable splitting methods:

Variable splitting methods replace (10) with an exactly equivalent constrained minimization problem involving an auxiliary variable such as z = Tx, e.g.,

$$\widehat{x}=\underset{x}{\text{arg}\text{min}}\underset{z:z=Tx}{\text{min}}\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\beta \Vert z{\Vert }_1.$$ (12)

This approach underlies the split Bregman algorithm [28], various augmented Lagrangian methods [29], and the alternating direction multiplier method (ADMM) [30]. The augmented Lagrangian for (12) is

$$\begin{gathered} L(x,z;\gamma ,\mu )=\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\beta \Vert z{\Vert }_1 \\ +\text{real}\left\{\langle \gamma ,Tx-z\rangle \right\}+\frac{\mu }{2}\Vert Tx-z{\Vert }_2^2, \end{gathered}$$

where $\gamma \in {ℂ}^K$ denotes the vector of Lagrange multipliers and μ > 0 is an AL penalty parameter that affects the convergence rate but not the final image. Defining the scaled dual variable η ≜ 1/μγ and completing the square leads to the following scaled augmented Lagrangian:

$$\begin{gathered} L(x,z;\eta ,\mu )=\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\beta \Vert z{\Vert }_1 \\ +\frac{\mu }{2}\left(\Vert Tx-z+\eta {\Vert }_2^2-\Vert \eta {\Vert }_2^2\right). \end{gathered}$$

An augmented Lagrangian approach alternates between descent updates of the primal variables x, z and an ascent update of the scaled dual variable η. The z update is simply soft thresholding:

$$z_{k+1}=\text{soft}\left(T{x}_k+{\eta }_k,\beta /\mu \right).$$

The x update minimizes a quadratic function:

$$x_{k+1}={\left(A^{\prime }A+\mu T^{\prime }T\right)}^{-1}\left(A^{\prime }y+\mu T^{\prime }\left(z_{k+1}+{\eta }_k\right)\right).$$

A few CG iterations is a natural choice for approximating the x update. Finally the η update is

$${\eta }_{k+1}={\eta }_k+\left(T{x}_{k+1}-z_{k+1}\right).$$

The unit step size here ensures dual feasibility [31]. A drawback of variable splitting methods is the need to select the parameter μ. Adaptive methods have been proposed to help with this tuning [31].

One could apply ADMM to the synthesis regularized problem (7), though again it would require parameter tuning that is unnecessary with POGM.

The conventional variable split in (12) ignores the specific structure of the MRI system matrix A in (3). Important properties of A include the fact that F′F is circulant (for Cartesian sampling) or Toeplitz (for non-Cartesian sampling) and that each coil sensitivity matrix C_l is diagonal. In contrast, the Gram matrix A′A for parallel MRI is harder to precondition, though possible [32]. An alternative splitting that simplifies the updates is [29]:

$$\begin{gathered} \underset{x\in {ℂ}^N}{\text{arg}\text{min}}\underset{u\in {ℂ}^{NL},z\in {ℂ}^K,v\in {ℂ}^N}{\text{min}}\frac{1}{2}{\Vert F_{L}u-y\Vert }_2^2+\beta \Vert z{\Vert }_1 \\ \text{sub. to }u=Cx,z=Tv,v=x, \end{gathered}$$ (13)

where F_L ≜ I_L ⊗ F. With this splitting, the z update again is simply soft thresholding, and the x update involves the diagonal matrix C′C which is trivial. The v update involves the matrix T′T that is circulant for periodic boundary conditions or is very well suited to a circulant preconditioner otherwise, using simple FFT operations. The u update involves the matrix $F_L^{\prime }{F}_L$ that is circulant or Toeplitz. This approach exploits the structure of A to simplify the updates; the primary drawback is that it requires selecting even more AL penalty parameters; condition number criteria can be helpful [29]. Another splitting with fewer auxiliary variables leads to an inner update step that requires solving denoising problems similar to (11).

3). Primal-dual methods:

A key idea behind duality-based methods is the fact:

$$\Vert Tx{\Vert }_1=\underset{z\in {ℂ}^K:\Vert z{\Vert }_{\infty }\le 1}{\text{max}}\text{real}\left\{\langle z,Tx\rangle \right\}.$$

Thus the (nonsmooth) analysis regularized problem (10) is equivalent to this constrained problem:

$$\underset{x}{\text{arg}\text{min}}\underset{z\in \mathcal{Z}}{\text{min}}\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\beta \text{real}\left\{\langle z,Tx\rangle \right\},$$ (14)

where $\mathcal{Z}\triangleq \left\{z\in {ℂ}^K:\Vert z{\Vert }_{\infty }\le 1\right\}$. The primal-dual methods typically alternate between updating the primal variable x and the dual variable z, using more convenient alternatives to (14) that involve separate multiplication by A and by A′ without requiring inner CG iterations. These methods provide convergence guarantees and acceleration techniques that lead to O(1/k²) rates [33]. A drawback of such methods is they typically require power iterations to find a Lipschitz constant, and, like AL methods, have tuning parameters that affect the practical convergence rates. Finding a simple, convergent, and tuning-free method for the analysis regularized problem (10) remains an important open problem.

F. Patch-based sparsity models

Using (10) with a finite-difference regularizer is essentially equivalent to using patches of size 2×1. It is plausible that one can regularize better by considering larger patches that provide more context for distinguishing signal from noise. There are two primary modes of patch-based regularization: synthesis models and analysis methods.

A typical synthesis approach attempts to represent each patch using a sparse linear combination of atoms from some signal patch dictionary. Let P_p denote the d × N matrix that extracts the pth of P patches (having d pixels) when multiplied by an image vector x. Then the synthesis model is that P_px ≈ Dz_p where D is a d × J dictionary, such as the discrete cosine transform (DCT) [34], and $z_p\in {ℂ}^J$ is a sparse coefficient vector for the pth patch. Under this model, a natural regularizer is

$$R(x)=\underset{\left\{z_p\right\}}{\text{min}}\sum _{p=1}^P\frac{1}{2}{\Vert P_{p}x-D{z}_p\Vert }_2^2+\alpha {\Vert z_p\Vert }_1.$$ (15)

See [34] for an extension to the case of multiple images. The regularizer has an inner minimization over the sparse coefficients {z_p}, so the overall problem involves both optimizing the image x and those coefficients. This structure lends itself to alternating minimization algorithms. The work in [34] used ISTA for updating z_p; the results in Fig. 2 suggest that POGM may be beneficial.

A typical analysis approach for patches assumes there is a sparsifying transform Ω such that ΩP_px tend to be sparse. For example, [35] uses a directional wavelet transform for each patch. Under this model, a natural regularizer is

$$R(x)=\underset{\left\{z_p\right\}}{\text{min}}\sum _{p=1}^P\frac{1}{2}{\Vert {\mathbf{\Omega }\mathit{P}}_{p}x-z_p\Vert }_2^2+\alpha {\Vert z_p\Vert }_1.$$ (16)

Again a double minimization over the image x and the transform coefficients {z_p} is needed, so alternating minimization algorithms are natural. For alternating minimization (block coordinate descent), the update of each z_p is simply soft thresholding, and the update of x is a quadratic problem involving $A^{\prime }A+\beta {\sum }_p{P}_p^{\prime }{\Omega }^{\prime }\Omega P_p$. When the transform Ω is unitary and the patches are selected with periodic boundary conditions and a stride of one pixel, then this simplifies to A′A + βI. A few inner iterations of the (preconditioned) CG algorithm is useful for the x update. Under these assumptions, and using just a single gradient descent update for x, an alternating minimization algorithm for least-squares with regularizer (16) simply alternates between a denoising step and a gradient step:

$$\begin{gathered} {\tilde{\mathit{x}}}_k={\sum }_{p=1}^P{\mathit{P}}_p^{\prime }{\mathbf{\Omega }}^{\prime }\text{soft}\left(\mathbf{\Omega }{\mathit{P}}_p{\mathit{x}}_k,\alpha \right) \\ {\mathit{x}}_{k+1}={\mathit{x}}_k-{\left(D+\beta I\right)}^{-1}\left({\mathit{A}}^{\prime }(\mathit{A}\mathit{x}-\mathit{y})+\beta {\tilde{\mathit{x}}}_k\right). \end{gathered}$$ (17)

For this algorithm the cost function is monotonically nonincreasing.

G. Adaptive regularization

The patch dictionary D in (15) or the sparsifying transform Ω in (16) can be chosen based on mathematical models like the DCT, or they can be learned from a population of preexisting training data and then used in (15) or (16) for subsequent patients. A third possibility is to adapt D or Ω to each specific patient [36]. The “dictionary learning MRI” (DLMRI) approach [36] uses a non-convex regularizer of the following form:

$$R(x)=\underset{D\in \mathcal{D}}{\text{min}}\underset{\left\{z_p\right\}}{\text{min}}\sum _{p=1}^P{\Vert P_{p}x-D{z}_p\Vert }_2^2+\alpha {\Vert z_p\Vert }_1,$$ (18)

where $\mathcal{D}$ is the feasible set of dictionaries (typically constrained so that each atom has unit norm). Now there are three set of variables to optimize: x, {z_p}, D, so alternating minimization methods are well suited. The update of the image x is a quadratic optimization subproblem, the z_p update is soft thresholding, and the D update is simple when considering one atom at a time.

The “transform learning MRI” (TLMRI) approach uses a regularizer of this form:

$$R(x)=\underset{\Omega }{\text{min}}\underset{\left\{z_p\right\}}{\text{min}}\sum _{p=1}^P{\Vert \Omega P_{p}x-z_p\Vert }_2^2+\alpha {\Vert z_p\Vert }_1+\gamma r(\Omega ),$$

where r(Ω) enforces or encourages properties of the sparsifying transform such as orthogonality. Again, alternating minimization methods are well suited; the Ω update involves (small) SVD operations.

H. Convolutional regularizers

An alternative to patch-based regularization is to use convolutional sparsity models [37]. A convolutional synthesis regularizer replaces (15) with

$$R(x)=\underset{\left\{z_k\right\}}{\text{min}}\frac{1}{2}{\Vert x-{\sum }_{k=1}^K{h}_k*z_k\Vert }_2^2+\alpha {\Vert z_k\Vert }_1,$$

where {h_k} is a set of filters learned from training images [37] and ∗ denotes convolution. Again, alternating minimization algorithms are a natural choice because the x update is quadratic and the z_k update is a sparse coding problem for which proximal methods like POGM are well-suited.

A convolution analysis regularizer replaces (16) with

$$R(x)=\underset{\left\{z_k\right\}}{\text{min}}\sum _{k=1}^K\frac{1}{2}{\Vert h_k*x-z_k\Vert }_2^2+\alpha {\Vert z_k\Vert }_1.$$

Again, alternating minimization algorithms are effective, where the z_k update is soft thresholding. One can either learn the filters {h_k} from good quality (e.g., fully sampled) training data, or adapt the filters for each patient by jointly optimizing x, {h_k} and {z_k} using alternating minimization.

I. Other methods

The summation in (17) is a particular type of patch-based denoising of the current image estimate x_k. There are many other denoising methods, some of which have variational formulations well-suited to inverse problems, but many of which do not, such as nonlocal means (NLM) [38] and block-matching 3D (BM3D) [39]. One way to adapt most such denoising methods for image reconstruction is to use a plug-and-play ADMM approach [40] that replaces a denoising step like (17) with a general denoising procedure.

J. Non-SENSE methods

The measurement model (2) and (3) has a single latent image x, viewed by each receive coil. An alternate formulation is to define a latent image for each coil x_l ≜ C_lx and write the measurement model as y_l = Fx_l + ε_l. For such formulations, the problem becomes to reconstruct the L images X = [x₁ … x_L] from the measurements, while considering relationships between those images. Because multiplication by the smooth sensitivity map C_l in the image domain corresponds to convolution with a small kernel in the frequency domain, any point in k-space can be approximated by a linear combination of its neighbors in all coil data [2]. This “GRAPPA modeling” leads to an approximate consistency condition vec(X) ≈ G vec(X) where G is a matrix involving small k-space kernels that are learned from calibration data [2]. This relationship leads to “SPIRiT” optimization problems like:

$$\begin{gathered} \widehat{X}=\underset{X\in {ℂ}^{N\times L}}{\text{arg}\text{min}}\frac{1}{2}{\left|\Vert FX-Y\Vert \right|}_{\text{Frob}}^2 \\ +{\beta }_1\frac{1}{2}\Vert (G-I)\text{vec}(X){\Vert }_2^2+{\beta }_{2}R(X), \end{gathered}$$

where $Y=\left[y_1\dots y_L\right]\in {ℂ}^{M\times L}$ and R(X) is a regularizer that encourages joint sparsity because all of the images {x_l} have edges in the same locations. No sensitivity maps C are needed for this approach. When β₂ = 0 the problem is quadratic and CG is well suited. Otherwise, ADMM is convenient for splitting this optimization problem into parts with easier updates. The ESPIRiT approach uses the redundancy in k-space data from multiple coils to estimate sensitivity maps from the eigenvectors of a certain block-Hankel matrix [12]; this approach helps bridge the SENSE and GRAPPA approaches while building on related signal processing tools like subspace estimation and multichannel blind deconvolution.

III. Summary

Although the title of this paper is “optimization methods for…” before selecting an optimization algorithm it is far more important (for under-sampled problems) to first select an appropriate cost function that captures useful prior information about the latent object x. The literature is replete with numerous candidate models, each of which often lead to different optimization methods. Nevertheless, common ingredients arise in most formulations, such as alternating minimization (block coordinate descent) at the outer level, preconditioned CG for inner iterations related to quadratic terms, and soft thresholding or other proximal operators for nonsmooth terms that promote sparsity.

This survey has focused on 1-norm regularizers for simplicity, but (nonconvex) p “norms” with 0 ≤ p < 1 have also been investigated and appear to be beneficial particularly for very undersampled measurements. This survey considers a single image x but many MRI scan protocols involve several images with different contrast and it may be useful to reconstruct them jointly, e.g., by considering common sparsity or subspace models.

There are many open problems in optimization that are relevant to MRI. The analysis form regularized problem (10) remains challenging, and further investigation of analysis vs synthesis approaches is needed. There has been considerable recent progress on finding optimal worst-case methods [16], but these optimality results are for very broad classes of cost functions, whereas the cost functions in MRI reconstruction have particular structure. Finding algorithms with optimal complexity (fastest possible convergence) for MRI-type cost functions would be valuable both for clinical practice and for facilitating research.

Finally, the current trend is to use convolutional neural network (CNN) methods to process under-sampled images, or for direct reconstruction, or as denoising operators. The stochastic gradient descent method currently is the universal optimization tool for training CNN models. Many “deep learning” methods for MRI are based on network architectures that are “unrolled” versions of iterative optimization methods like PGM. Thus, familiarity with “classical” optimization methods for MR image reconstruction is important even in the machine learning era.

Fig. 3.

POGM method for minimizing f(x) + g(x) where f is convex with L-Lipschitz smooth gradient and g is convex. See [25] for adaptive restart version.