A Generalized Structured Low-Rank Matrix Completion Algorithm for MR Image Recovery

Abstract

Recent theory of mapping an image into a structured low-rank Toeplitz or Hankel matrix has become an effective method to restore images. In this paper, we introduce a generalized structured low-rank algorithm to recover images from their undersampled Fourier coefficients using infimal convolution regularizations. The image is modeled as the superposition of a piecewise constant component and a piecewise linear component. The Fourier coefficients of each component satisfy an annihilation relation, which results in a structured Toeplitz matrix, respectively. We exploit the low-rank property of the matrices to formulate a combined regularized optimization problem. In order to solve the problem efficiently and to avoid the high memory demand resulting from the large-scale Toeplitz matrices, we introduce a fast and memory efficient algorithm based on the half-circulant approximation of the Toeplitz matrix. We demonstrate our algorithm in the context of single and multi-channel MR images recovery. Numerical experiments indicate that the proposed algorithm provides improved recovery performance over the state-of-the-art approaches.

I. Introduction

The recovery of images from their limited and noisy measurements is an important problem in a wide range of biomedical imaging applications, including microscopy [1], magnetic resonance imaging (MRI) [2], and computed tomography [3]. The common method is to formulate the image reconstruction as an optimization problem, where the criterion is a linear combination of data consistency error and a regularization penalty. The regularization penalties are usually chosen to exploit the smoothness or the sparsity priors in the discrete image domain. For example, compressed sensing methods are capable of recovering the original MR images from their partial k space measurements [2] using the L₁ norm in the total variation (TV) or wavelet domain. The reconstruction performance is determined by the effectiveness of the regularization. In order to improve the quality of the reconstructed images, several extensions and generalizations of TV are also proposed, such as total generalized variation (TGV) [4], [5], Hessian-based norm regularization [6], and higher degree total variation (HDTV) [7], [8]. All of these regularization penalties are formulated in the discrete domain, and hence suffer from discretization errors as well as lack of rotation invariance.

Recently, a new family of reconstruction methods, which are based on the low-rank property of structured Hankel or Toeplitz matrices built from the Fourier coefficients of the image, have been introduced as powerful continuous domain alternatives to the above discrete domain penalties [9], [10], [11], [12], [13]. Since these methods minimize discretization errors and rotation dependence, they provide improved reconstruction. These algorithms can be viewed as the multidimensional extensions of the finite-rate-of-innovation (FRI) frame-work [14], [15]. All of these methods exploit the “annihilation property”, which implies that image derivatives can be annihilated by multiplication with a bandlimited polynomial function in the spatial domain; this image domain relation translates to a convolutional annihilation relationship in the Fourier domain. Since the locations of the discontinuities are not isolated in the multidimensional setting, the theoretical tools used to show perfect recovery are very different from the 1-D FRI setting [11], [16]. The convolution relations are compactly represented as a multiplication between a block Hankel structured matrix and the Fourier coefficients of the filter. It has been shown that the above structured matrix is low-rank, which allows the recovery of the unknown matrix entries using structured low-rank matrix completion. Empirical results show improved performance over classical total variation methods [17], [16], [9], [18]. Haldar proposed a Hankel structured low-rank matrix algorithm (LORAKS) for the reconstructions of single coil MR images [10] with the assumption that the image has limited spatial support and smooth phase. The effectiveness of the algorithm was also investigated in parallel MRI [19], [20], [21].

In this paper, we extend the structured low-rank framework to recover the sum of two piecewise smooth functions from their sparse measurements. This work is inspired by the infimal convolution framework [22], where the sum of a piecewise constant and a piecewise linear function was recovered; the infimal convolution of functions with first and second order derivatives were considered as penalties in [22]. The algorithm was then applied in a general discrete setting for image denoising [23] to obtain improved performance over standard TV. The extension of TV using infimal convolution (ICTV) was applied in video and image reconstruction in [24]. The infimal convolution of TGV (ICTGV) was proposed in the context of dynamic MRI reconstruction by balancing the weights for the spatial and temporal regularizations [25]. In [26] and [27], the infimal convolution of two total variation Bregman distances are applied to exploit the structural information in the reconstruction of dynamic MR datasets and the joint reconstruction of PET-MRI, respectively. In [28], the authors adopted the robust PCA method [29] into dynamic MRI, where the dataset is decomposed into low-rank and sparse component (L+S). In [30], instead of imposing low-rank assumptions, the k-t PCA method was improved using model consistency constraints (MOCCO) to obtain temporal basis functions from low resolution dynamic MR data. In this paper, we propose to model the image as the combination of a piecewise constant component and a piecewise linear component. For the piecewise constant component, the Fourier coefficients of the gradient of the component satisfy the annihilation relation. We thus build a structured Toeplitz matrix, which can be proved to be low-rank. Similarly, we can obtain a structured low-rank Toeplitz matrix from the Fourier coefficients of the second order partial derivatives of the piecewise linear component. By introducing the generalized structured low-rank method, the image can be automatically separated into components where either the strong edges and feature details or the smooth regions of the image can be accurately recovered. Thus, the optimal balance can be obtained between the first order and higher order penalties.

Since the proposed method involves the recovery of large-scale first and second order derivatives lifted Toeplitz matrices, the implementation of the method is associated with high computational complexity and memory demand. In order to solve the corresponding optimization problem efficiently, we introduce an algorithm based on the half-circulant approximation of Toeplitz matrices, which is a generalization of the Generic Iteratively Reweighted Annihilating Filter (GIRAF) algorithm proposed in [17], [31]. This algorithm alternates between the estimation of the annihilation filter of the image, and the computation of the image annihilated by the filter in a least squares formulation. By replacing the linear convolution by circular convolution, the algorithm can be implemented efficiently using Fast Fourier Transforms, which significantly reduces the computational complexity and the memory demand. We investigate the performance of the algorithm in the context of compressed sensing MR images reconstruction. Experiments show that the proposed method is capable of providing more accurate recovery results than the state-of-the-art algorithms. The preliminary version of the work was accepted as a conference paper. Compared to the work [32], the theoretical and algorithmic frameworks are further developed here. We have significantly more validation in the current version, in addition to the generalization of the method to parallel MRI setting.

II. Generalized Structure Low-Rank Matrix Recovery

A. Image recovery model

We consider the recovery of a discrete 2-D image ρ ∈ ℂ^N from its noisy and degraded measurements b ∈ ℂ^M. We model the measurements as $\mathbf{b}=\mathcal{A}(\mathit{\rho })+\mathbf{n}$, where $\mathcal{A}\in {ℂ}^{M\times N}$ is a linear degradation operator which maps ρ to b, and n ∈ ℂ^M is assumed to be the Gaussian distributed white noise. Since the recovery of ρ from the measurements b is ill-posed in many practical cases, the general approach is to pose the recovery as a regularized optimization problem, i.e.,

$$\mathit{\rho }=\text{arg}\underset{\mathit{\rho }}{\text{min}}{\Vert \mathcal{A}\left(\mathit{\rho }\right)-\mathbf{b}\Vert }^2+\mathrm{\lambda }\mathcal{J}\left(\mathit{\rho }\right)$$ (1)

where ${‖\mathcal{A}(\mathbf{\rho })-\mathbf{b}‖}^2$ is the data consistency term, λ is the balancing parameter, and $\mathcal{J}(\mathit{\rho })$ is the regularization term, which determines the quality of the recovered image. Common choices for the regularization term include total variation, wavelet, and their combinations. Researchers have also proposed the extensions of total variations [4], [6], [8] to improve the performance.

B. Structured low-rank matrix completion

Consider the general model for a 2-D piecewise smooth image ρ(r) at the spatial location r = (x, y) ∈ ℤ²:

$$\rho \left(\mathbf{r}\right)=\sum _{i=1}^N{g}_i\left(\mathbf{r}\right){\mathrm{\chi \Omega }}_i\left(\mathbf{r}\right)$$ (2)

where χΩ_i is a characteristic function of the set Ω_i and the functions g_i(r) are smooth polynomial functions which vanish with a collection of differential operators D ={D₁, …, D_N} within the region Ω_i. It is proved that under certain assumptions on the edge set $\partial \mathrm{\Omega }={\cup }_{i=1}^N\partial {\mathrm{\Omega }}_i$, the Fourier transform of derivatives of ρ(r) satisfies an annihilation property [11]. We assume that a bandlimited trigonometric polynomial function µ(r) vanishes on the edge set of the image:

$$\mu \left(\mathbf{r}\right)=\sum _{\mathbf{k}\in {\mathrm{\Delta }}_1}c\left[\mathbf{k}\right]e^{j2\pi \langle \mathbf{k},\mathbf{r}\rangle }$$ (3)

where c[k] denotes the Fourier coefficients of µ and ∆₁ is any finite set of ℤ². According to [13], the family of functions in (2) is a general form including many common image models by choosing different set of differential operators D.

• 1)
  Piecewise constant images: Assume ρ₁(r) is a piecewise constant image function, thus the first order partial derivative of the image D₁ρ₁ = ∇ρ₁ = (∂_xρ₁, ∂_yρ₁) is annihilated by multiplication with µ₁ in the spatial domain:

  $${\mu }_1\mathbf{\nabla }{\rho }_1=0$$ (4)

  The multiplication in spatial domain translates to the convolution in Fourier domain, which is expressed as:

  $$\sum _{\mathbf{k}\in {\mathrm{\Delta }}_1}\widehat{\mathbf{\nabla }{\rho }_1}\left[\mathbf{\ell }-\mathbf{k}\right]c_1\left[\mathbf{k}\right]=\mathbf{0},\forall \mathbf{\ell }\in {\mathbb{Z}}^2$$ (5)

  where $\widehat{\mathbf{\nabla }{\rho }_1}\left[\mathbf{k}\right]=j2\pi \left(k_x{\widehat{\rho }}_1\left[\mathbf{k}\right],k_y{\widehat{\rho }}_1\left[\mathbf{k}\right]\right)$ for k = (k_x, k_y). Thus the annihilation property can be formulated as a matrix multiplication:

  $${\mathcal{T}}_1\left({\widehat{\rho }}_1\right)\mathbf{\text{c}}=\left[\begin{array}{c}{\mathcal{T}}_x\left(\widehat{\rho }\right)\\ {\mathcal{T}}_y\left(\widehat{\rho }\right)\end{array}\right]{\mathbf{\text{c}}}_{\mathbf{1}}=0$$ (6)

  where ${\mathcal{T}}_1\left({\widehat{\rho }}_1\right)$ is a Toeplitz matrix built from the entries of ${\widehat{\rho }}_1$, the Fourier coefficients of ρ₁. Specifically, ${\mathcal{T}}_x\left({\widehat{\rho }}_1\right)$, ${\mathcal{T}}_y\left({\widehat{\rho }}_1\right)$ are matrices corresponding to the discrete 2-D convolution of $k_x{\widehat{\rho }}_1\left[\mathbf{k}\right]$ and $k_y{\widehat{\rho }}_1\left[\mathbf{k}\right]$ for (k_x, k_y) ∈ Γ, omitting the irrelevant factor j2π. Here c₁ is the vectorized version of the filter c₁[k]. Note that c is supported in ∆₁. Thus, we can obtain that:

  $${\widehat{\rho }}_1\left[\mathbf{k}\right]*d\left[\mathbf{k}\right]=0,\mathbf{k}\in \mathrm{\Gamma }$$ (7)

  Here d[k] = c₁[k] * h[k], where h[k] is any FIR filter. Note that ∆₁ is smaller than Γ, the support of d. Thus, if we take a larger filter size than the minimal filter c₁[k], the annihilation matrix will have a larger null space. Therefore, ${\mathcal{T}}_1\left({\widehat{\rho }}_1\right)$ is a low-rank matrix. The method corresponding to this case is referred to as the first order structured low-rank algorithm (first order SLA) for simplicity.
• 2)
  Piecewise linear images: Assume ρ₂(r) is a piecewise linear image function, it can be proved that the second order partial derivatives of the image ${\mathbf{D}}_{2\rho 2}=\left({\partial }_{xx}^2{\rho }_2,{\partial }_{xy}^2{\rho }_2,{\partial }_{yy}^2{\rho }_2\right)$ satisfy the annihilation property [13]

  $${\mu }_2^2{\mathbf{D}}_{2\rho 2}=0$$ (8)

  Thus, the Fourier transform of D₂ρ₂ is annihilated by convolution with the Fourier coefficients c₂[k]; k ∈ ∆₂ of ${\mu }_2^2$:

  $$\sum _{\mathbf{k}\in {\mathrm{\Delta }}_2}\widehat{{\mathbf{D}}_2{\rho }_2}\left[\mathbf{\ell }-\mathbf{k}\right]c_2\left[\mathbf{k}\right]=\mathbf{0},\forall \mathbf{\ell }\in {\mathbb{Z}}^2$$ (9)

  where $\widehat{{\mathbf{D}}_2{\rho }_2}\left[\mathbf{k}\right]={\left(j2\pi \right)}^2\left(k_x^2{\widehat{\rho }}_2\left[\mathbf{k}\right],k_x{k}_y{\widehat{\rho }}_2\left[\mathbf{k}\right],k_y^2{\widehat{\rho }}_2\left[\mathbf{k}\right]\right)$ for k = (k_x, k_y). Similarly, the annihilation relation can be expressed as:

  $${\mathcal{T}}_2\left({\widehat{\rho }}_2\right)\mathbf{d}=\left[\begin{array}{c}{\mathcal{T}}_{xx}\left({\widehat{\rho }}_2\right)\\ {\mathcal{T}}_{xy}\left({\widehat{\rho }}_2\right)\\ {\mathcal{T}}_{yy}\left({\widehat{\rho }}_2\right)\end{array}\right]{\mathbf{\text{c}}}_2=0$$ (10)

  where ${\mathcal{T}}_2\left({\widehat{\rho }}_2\right)$ is a Toeplitz matrix.${\mathcal{T}}_{xx}\left({\widehat{\rho }}_2\right)$, ${\mathcal{T}}_{xy}\left({\widehat{\rho }}_2\right)$ and ${\mathcal{T}}_{yy}\left({\widehat{\rho }}_2\right)$ are matrices corresponding to the discrete convolution of $k_x^2{\widehat{\rho }}_2\left[\mathbf{k}\right]$, $k_x{k}_y{\widehat{\rho }}_2\left[\mathbf{k}\right]$, and $k_y^2{\widehat{\rho }}_2\left[\mathbf{k}\right]$, omitting the insignificant factor, and c₂ is the vectorized version of d[k]. Similar to the piecewise constant case, the Toeplitz matrix ${\mathcal{T}}_2\left({\widehat{\rho }}_2\right)$ is also a low-rank matrix. The method exploiting the low-rank property of ${\mathcal{T}}_2\left({\widehat{\rho }}_2\right)$ is referred to as the second order structured low-rank algorithm (second order SLA).

C. Generalized structured low-rank image recovery (GSLR)

We assume that a 2-D image ρ is a piecewise smooth function, which can be decomposed into two components ρ = ρ₁ + ρ₂, such that ρ₁ represents the piecewise constant component of ρ, while ρ₂ represents the piecewise linear component of ρ. We assume that the gradient of ρ₁ and the second derivative of ρ₂ vanish on the zero sets of µ₁ and µ₂, respectively. This relation transforms to a convolution relation between the weighted Fourier coefficients of ρ₁ and ρ₂ with c₁ and c₂, respectively, based on the analysis in Section. II-B. Inspired by the concept of infimal convolution, we consider the framework of a combined regularization procedure, where we formulate the reconstruction of the Fourier data $\widehat{\mathbf{\rho }}$ from the undersampled measurements b as follows:

$$\begin{gathered} \underset{{\widehat{\mathbf{\rho }}}_1+{\widehat{\rho }}_2=\widehat{\mathbf{\rho }}}{\text{min}}\text{rank}\left[{\mathcal{T}}_1\left({\widehat{\rho }}_1\right)\right]+\text{rank}\left[{\mathcal{T}}_2\left({\widehat{\rho }}_2\right)\right] \\ \text{such that\hspace{0.17em}}\mathbf{\text{b}}=\mathcal{A}\left(\widehat{\mathbf{\rho }}\right)+\mathbf{\text{n}} \end{gathered}$$ (11)

Since the above problem is NP hard, we choose the Schatten p (0 ≤ p < 1) norm as the relaxation function, which makes (11) as the following optimization problem:

$$\begin{gathered} \{{{\widehat{\mathbf{\rho }}}_1}^{\star },{{\widehat{\mathbf{\rho }}}_2}^{\star }\}=\text{arg\hspace{0.17em}}\underset{{\widehat{\mathbf{\rho }}}_1,{\widehat{\mathbf{\rho }}}_2}{\text{min}}{\mathrm{\lambda }}_1{\Vert {\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right)\Vert }_p+{\mathrm{\lambda }}_2{\Vert {\mathcal{T}}_2\left({\widehat{\mathbf{\rho }}}_2\right)\Vert }_p \\ +{\Vert \mathcal{A}\left({\widehat{\mathbf{\rho }}}_1+{\widehat{\mathbf{\rho }}}_2\right)-\mathbf{b}\Vert }^2 \end{gathered}$$ (12)

where ${\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right)$ and ${\mathcal{T}}_2\left({\widehat{\mathbf{\rho }}}_2\right)$ are the structured Toeplitz matrices in the first and second order partial derivatives weighted lifted domain, respectively. λ₁ and λ₂ are regularization parameters which balance the data consistency and the degree to which ${\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right)$ and ${\mathcal{T}}_2\left({\widehat{\mathbf{\rho }}}_2\right)$ are low-rank. ǁ·ǁ_p is the Schatten p norm (0 ≤ p < 1), defined for an arbitrary matrix X as:

$${\Vert \mathbf{X}\Vert }_p=\frac{1}{p}\text{Tr}[{\left({\mathbf{X}}^{*}\mathbf{X}\right)}^{\frac{p}{2}}]=\frac{1}{p}\text{Tr}[{\left(\mathbf{X}{\mathbf{X}}^{*}\right)}^{\frac{p}{2}}]=\frac{1}{p}\sum _i{\sigma }_i^p$$ (13)

where σ_i are the singular values of X. Note that when p → 1, ǁXǁ_p is the nuclear norm; when p → 0, ${\Vert \mathbf{\text{X}}\Vert }_p=\sum _i\text{log\hspace{1em}}{\sigma }_i$. The penalty ǁXǁ_p is convex for p = 1, and non-convex for 0 ≤ p < 1.

III. Optimization Algorithm

We apply the iterative reweighted least squares (IRLS) algorithm to solve the optimization problem (12). Based on the equation ${\Vert \mathbf{X}\Vert }_p={\Vert \mathbf{X}{\mathbf{H}}^{\frac{1}{2}}\Vert }_F^2$ where $\mathbf{H}={\left(\mathbf{X}*\mathbf{X}\right)}^{\frac{p}{2}-1}$, let $\mathbf{X}={\mathcal{T}}_i\left({\widehat{\mathbf{\rho }}}_i\right)\left(i=1,2\right)$, (12) becomes:

$$\begin{gathered} \left\{{\widehat{\mathbf{\rho }}}_1^{\star },{\widehat{\mathbf{\rho }}}_2^{\star }\right\}=\text{arg\hspace{0.17em}}\underset{{\widehat{\mathbf{\rho }}}_1,{\widehat{\mathbf{\rho }}}_2}{\text{min\hspace{0.17em}}}{\mathrm{\lambda }}_1\Vert {\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right){\mathbf{H}}_1^{\frac{1}{2}}{\Vert }_F^2+{\mathrm{\lambda }}_2\Vert {\mathcal{T}}_2\left({\widehat{\mathbf{\rho }}}_2\right){\mathbf{H}}_2^{\frac{1}{2}}{\Vert }_F^2 \\ +{\Vert \mathcal{A}\left({\widehat{\mathbf{\rho }}}_1+{\widehat{\mathbf{\rho }}}_2\right)-\mathbf{b}\Vert }^2 \end{gathered}$$ (14)

In order to solve (14), we can use an alternating minimization scheme, which alternates between the following subproblems: updating the weight matrices H₁ and H₂, and solving a weighted least squares problem. Specifically, at nth iteration, we compute:

$${\mathbf{H}}_1^{\left(n\right)}=\underset{{\mathbf{\text{G}}}_1}{\underbrace{[{\mathcal{T}}_1({\widehat{\mathbf{\rho }}}_1^{\left(n\right)})*{\mathcal{T}}_1({\widehat{\mathbf{\rho }}}_1^{\left(n\right)})}}+{ϵ}_n{\mathbf{I}]}^{\frac{p}{2}-1}$$ (15)

$${\mathbf{H}}_2^{\left(n\right)}=\underset{{\mathbf{\text{G}}}_2}{\underbrace{[{\mathcal{T}}_2({\widehat{\mathbf{\rho }}}_2^{\left(n\right)})*{\mathcal{T}}_2({\widehat{\mathbf{\rho }}}_2^{\left(n\right)})}}+{ϵ}_n{\mathbf{I}]}^{\frac{p}{2}-1}$$ (16)

$$\begin{gathered} \{{\widehat{\mathbf{\rho }}}_1^{\left(n\right)},{\widehat{\mathbf{\rho }}}_2^{(n)}\}=\text{arg\hspace{0.17em}}\underset{{\widehat{\mathbf{\rho }}}_1,{\widehat{\mathbf{\rho }}}_2}{\text{min}}{\Vert \mathcal{A}\left({\widehat{\mathbf{\rho }}}_1+{\widehat{\mathbf{\rho }}}_2\right)-\mathbf{b}\Vert }^2 \\ +{\mathrm{\lambda }}_1\Vert {\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right){({\mathbf{H}}_1^{(n)})}^{\frac{1}{2}}{\Vert }_F^2+{\mathrm{\lambda }}_2\Vert {\mathcal{T}}_2\left({\widehat{\mathbf{\rho }}}_2\right){({\mathbf{H}}_2^{(n)})}^{\frac{1}{2}}{\Vert }_F^2 \end{gathered}$$ (17)

where ϵ_n → 0 is a small factor used to stabilize the inverse. We now show how to efficiently solve the subproblems.

A. Update of least squares

First, let ${\mathbf{H}}_1^{\frac{1}{2}}=[{\mathbf{h}}_1^{\left(1\right)},\cdot \cdot \cdot ,{\mathbf{h}}_1^{\left(L\right)}],$${\mathbf{H}}_2^{\frac{1}{2}}=[{\mathbf{h}}_2^{\left(1\right)},\cdot \cdot \cdot ,{\mathbf{h}}_2^{\left(M\right)}],$ we rewrite the least squares problem (17) as follows:

$$\begin{gathered} \{{\widehat{\mathbf{\rho }}}_1^{\left(n\right)},{\widehat{\mathbf{\rho }}}_2^{\left(n\right)}\}=\text{arg\hspace{0.17em}}\underset{{\widehat{\mathbf{\rho }}}_1,{\widehat{\mathbf{\rho }}}_2}{\text{min}}{\Vert \mathcal{A}\left({\widehat{\mathbf{\rho }}}_1+{\widehat{\mathbf{\rho }}}_2\right)-\mathbf{b}\Vert }^2 \\ +{\mathrm{\lambda }}_1\sum _{l=1}^L\Vert {\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right){\mathbf{h}}_1^{\left(l\right)}{\Vert }_F^2+{\mathrm{\lambda }}_2\sum _{m=1}^M\Vert {\mathcal{T}}_2\left({\widehat{\mathbf{\rho }}}_2\right){\mathbf{h}}_2^{\left(m\right)}{\Vert }_F^2 \end{gathered}$$ (18)

We now focus on the update of ${\widehat{\mathbf{\rho }}}_1$. The update of ${\widehat{\mathbf{\rho }}}_2$ can be derived likewise. From the structure property of ${\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right)$ and the convolution relationship, we can obtain:

$$\begin{gathered} {\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right){\mathbf{\text{h}}}_1^{(l)}={\mathcal{P}}_{{\mathrm{\Gamma }}_1}({\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1*{\mathbf{\text{h}}}_1^{(l)})={\mathcal{P}}_{{\mathrm{\Gamma }}_1}({\mathbf{\text{h}}}_1^{\left(l\right)}*{\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1) \\ ={\mathbf{\text{P}}}_1{\mathbf{\text{C}}}_1^{\left(l\right)}{\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1,l=1,\dots ,L \end{gathered}$$ (19)

where ${\mathbf{\text{C}}}_1^{\left(l\right)}$ denotes the linear convolution by ${\mathbf{\text{h}}}_1^{\left(l\right)},{\mathcal{P}}_{{\mathrm{\Gamma }}_1}$ is the projection of the convolution to a finite set Γ₁ of the valid k space index, which is expressed by the matrix P₁. M₁ is the linear transformation in k space, which denotes the multiplication with the first order Fourier derivatives j2πk_x and j2πk_y, referred to as the gradient weighted lifting case. We can approximate ${\mathbf{\text{C}}}_1^{\left(l\right)}$ by a circular convolution by ${\mathbf{\text{h}}}_1^{\left(l\right)}$ on a sufficiently large convolution grid. Then, we can obtain ${\mathbf{\text{C}}}_1^{\left(l\right)}=\mathbf{F}{\mathbf{S}}_1^{\left(l\right)}{\mathbf{F}}^{*}$, where F is the 2-D DFT and ${\mathbf{\text{S}}}_1^{\left(l\right)}$ is a diagonal matrix representing multiplication with the inverse DFT of ${\mathbf{\text{h}}}_1^{\left(l\right)}$. Assuming ${\mathbf{P}}_1^{*}{\mathbf{P}}_1\approx \mathbf{I}$, we can thus rewrite the second term in (18) as:

$$\begin{gathered} {\mathrm{\lambda }}_1\sum _{l=1}^L\Vert {\mathbf{\text{P}}}_1{\mathbf{\text{C}}}_1^{\left(l\right)}{\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1{\Vert }^2={\mathrm{\lambda }}_1{\widehat{\mathbf{\rho }}}_1*{\mathbf{\text{M}}}_1^{*}\mathbf{\text{F}}\sum _{l=1}^L{\mathbf{\text{S}}}_1^{\left(l\right)*}{\mathbf{\text{S}}}_1^{\left(l\right)}{\mathbf{\text{F}}}^{*}{\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1 \\ ={\mathrm{\lambda }}_1\Vert {\mathbf{\text{S}}}_1^{\frac{1}{2}}{\mathbf{\text{F}}}^{*}{\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1{\Vert }^2 \end{gathered}$$ (20)

where S₁ is a diagonal matrix with entries ${\sum }_{l=1}^N{\left|{\mu }_l\left(\mathbf{\text{r}}\right)\right|}^2$, and μ_l(r) is the trigonometric polynomial of inverse Fourier transform of ${\mathbf{\text{h}}}_1^{\left(l\right)}$. S₁ is specified as

$${\mathbf{\text{S}}}_1=\sum _{l=1}^L{\mathbf{\text{S}}}_1^{\left(l\right)}*{\mathbf{\text{S}}}_1^{\left(l\right)}=\text{diag}({\mathbf{\text{F}}*\mathbf{\text{P}}}_1^{*}{\mathbf{\text{h}}}_1)$$ (21)

Similarly, the third term in (18) can be rewritten as ${\mathrm{\lambda }}_2\Vert {\mathbf{\text{S}}}_2^{\frac{1}{2}}{\mathbf{\text{F}}*\mathbf{\text{M}}}_2{\widehat{\mathbf{\rho }}}_2{\Vert }^2$, where S₂ is given by

$${\mathbf{\text{S}}}_2=\sum _{m=1}^M{\mathbf{\text{S}}}_2^{\left(m\right)}*{\mathbf{\text{S}}}_2^{\left(m\right)}=\text{diag}\left({\mathbf{\text{F}}*\mathbf{\text{P}}}_2^{*}{\mathbf{\text{h}}}_2\right)$$ (22)

Therefore, we can reformulate the optimization problem (18) as:

$$\begin{gathered} \underset{{\widehat{\mathbf{\rho }}}_1,{\widehat{\mathbf{\rho }}}_2}{\text{min}}{\Vert \mathcal{A}\left({\widehat{\mathbf{\rho }}}_1+{\widehat{\mathbf{\rho }}}_2\right)-\mathbf{\text{b}}\Vert }^2+{\mathrm{\lambda }}_1\Vert {\mathbf{\text{S}}}_1^{\frac{1}{2}}{y}_1{\Vert }_F^2+{\mathrm{\lambda }}_2\Vert {\mathbf{\text{S}}}_2^{\frac{1}{2}}{y}_2{\Vert }_F^2 \\ \text{s.t.\hspace{0.17em}}{\mathbf{\text{Fy}}}_1={\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1,{\mathbf{\text{Fy}}}_2={\mathbf{\text{M}}}_2{\widehat{\mathbf{\rho }}}_2 \end{gathered}$$ (23)

The above constrained problem can be efficiently solved using the alternating directions method of multipliers (ADMM) algorithm [33], which yields to solving the following subproblems:

$${\mathbf{\text{y}}}_1^{\left(n\right)}=\underset{{\mathbf{\text{y}}}_1}{\text{min}}\Vert {\mathbf{\text{S}}}_1^{\frac{1}{2}}{\mathbf{\text{y}}}_1{\Vert }_2^2+{\gamma }_1\Vert {\mathbf{\text{q}}}_1^{\left(n-1\right)}+{\mathbf{\text{F}}}^{*}{\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1^{\left(n-1\right)}-{\mathbf{\text{y}}}_1{\Vert }_2^2$$ (24)

$${\mathbf{\text{y}}}_2^{\left(n\right)}=\underset{\mathbf{\text{y}}2}{\text{min}}\Vert {\mathbf{\text{S}}}_1^{\frac{1}{2}}{\mathbf{\text{y}}}_1{\Vert }_2^2+{\gamma }_1\Vert {\mathbf{\text{q}}}_2^{\left(n-1\right)}+{\mathbf{\text{F}}}^{*}{\mathbf{\text{M}}}_2{\widehat{\mathbf{\rho }}}_2^{\left(n-1\right)}-{\mathbf{\text{y}}}_2{\Vert }_2^2$$ (25)

$${\widehat{\mathbf{\rho }}}_1^{\left(n\right)}=\underset{{\widehat{\mathbf{\rho }}}_1}{\text{min}}{\Vert \mathcal{A}\left({\widehat{\mathbf{\rho }}}_1+{\widehat{\mathbf{\rho }}}_2\right)-\mathbf{\text{b}}\Vert }_2^2+{\gamma }_1{\mathrm{\lambda }}_1\Vert {\mathbf{\text{q}}}_1^{\left(n-1\right)}+{\mathbf{\text{F}}*\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1-{\mathbf{\text{y}}}_1^{\left(n\right)}{\Vert }_2^2$$ (26)

$${\widehat{\mathbf{\rho }}}_2^{\left(n\right)}=\underset{{\widehat{\rho }}_2}{\text{min}}\Vert \mathcal{A}\left({\widehat{\mathbf{\rho }}}_1+{\widehat{\mathbf{\rho }}}_2\right)-\mathbf{\text{b}}{\Vert }_2^2+{\gamma }_2{\mathrm{\lambda }}_2\Vert {\mathbf{\text{q}}}_2^{\left(n-1\right)}+\mathbf{\text{F}}*{\mathbf{\text{M}}}_2{\widehat{\mathbf{\rho }}}_2-{\mathbf{\text{y}}}_2^{\left(n\right)}{\Vert }_2^2$$ (27)

$${\mathbf{\text{q}}}_i^{\left(n\right)}={\mathbf{\text{q}}}_i^{\left(n-1\right)}+{\mathbf{\text{F}}*\mathbf{\text{M}}}_i{\widehat{\mathbf{\rho }}}_i^{\left(n\right)}-{\mathbf{\text{y}}}_i^{\left(n\right)}i=1,2$$ (28)

where q_i (i = 1, 2) represent the vectors of Lagrange multipliers, and γ_i (i = 1, 2) are fixed parameters tuned to improve the conditioning of the subproblems. Subproblems (24) to (27) are quadratic and thus can be solved easily as follows:

$${\mathbf{\text{y}}}_1^{\left(n\right)}={\left({\mathbf{\text{S}}}_1+{\gamma }_1\mathbf{\text{I}}\right)}^{-1}[{\gamma }_1({\mathbf{\text{q}}}_1^{\left(n-1\right)}+{\mathbf{\text{F}}*\mathbf{\text{M}}}_1{\widehat{\mathbf{\rho }}}_1^{\left(n-1\right)})]$$ (29)

$${\mathbf{\text{y}}}_2^{\left(n\right)}={\left({\mathbf{\text{S}}}_2+{\gamma }_2\mathbf{\text{I}}\right)}^{-1}[{\gamma }_2({\mathbf{\text{q}}}_2^{\left(n-1\right)}+{\mathbf{\text{F}}*\mathbf{\text{M}}}_2{\widehat{\mathbf{\rho }}}_2^{\left(n-1\right)})]$$ (30)

$$\begin{gathered} {\widehat{\mathbf{\rho }}}_1^{(n)}={({\mathcal{A}}^{*}\mathcal{A}+{\gamma }_1{\mathrm{\lambda }}_1{\mathbf{\text{M}}}_1^{*}{\mathbf{\text{M}}}_1)}^{-1}[{\gamma }_1{\mathrm{\lambda }}_1({\mathbf{\text{M}}}_1^{*}\mathbf{\text{F}})({\mathbf{\text{y}}}_1^{(n)}{\mathbf{\text{q}}}_1^{(n-1)}) \\ +{\mathcal{A}}^{*}\mathbf{\text{b}}-{\mathcal{A}}^{*}\mathcal{A}{\widehat{\mathbf{\rho }}}_2^{(n-1)}] \end{gathered}$$ (31)

$$\begin{gathered} {\widehat{\mathbf{\rho }}}_2^{(n)}={({\mathcal{A}}^{*}\mathcal{A}+{\gamma }_2{\mathrm{\lambda }}_2{\mathbf{\text{M}}}_2^{*}{\mathbf{\text{M}}}_2)}^{-1}[{\gamma }_2{\mathrm{\lambda }}_2({\mathbf{\text{M}}}_2^{*}\mathbf{\text{F}})({\mathbf{\text{y}}}_2^{(n)}{\mathbf{\text{q}}}_2^{(n-1)}) \\ +{\mathcal{A}}^{*}\text{b}-{\mathcal{A}}^{*}\mathcal{A}{\widehat{\mathbf{\rho }}}_1^{(n-1)}] \end{gathered}$$ (32)

B. Update of weight matrices

We now show how to update the weight matrices H₁ and H₂ in (15) and (16) efficiently based on the GIRAF method [11]. In order to obtain the weight matrices H₁ and H₂, we first compute the Gram matrix G₁ and G₂ as:

$${\mathbf{\text{G}}}_1={\mathcal{T}}_1{\left({\widehat{\mathbf{\rho }}}_1\right)}^{*}{\mathcal{T}}_1\left({\widehat{\mathbf{\rho }}}_1\right)$$ (33)

$${\mathbf{\text{G}}}_2={\mathcal{T}}_2{\left({\widehat{\mathbf{\rho }}}_2\right)}^{*}{\mathcal{T}}_2\left({\widehat{\mathbf{\rho }}}_2\right)$$ (34)

Let (V₁, Λ₁) denote the eigen-decomposition of G₁, where V₁ is the orthogonal basis of eigenvectors ${\mathbf{\text{v}}}_1^{\left(l\right)}$ and Λ₁ is the diagonal matrix of eigenvalues ${\mathrm{\lambda }}_{p1}^{\left(l\right)}$, which satisfy G₁ = V₁ Λ₁ ${\mathbf{\text{V}}}_1^{*}$. Then we can rewrite the weight matrix H₁ as:

$${\mathbf{\text{H}}}_1={\left[{\mathbf{\text{V}}}_1\left({\mathbf{\Lambda }}_1+ϵ\mathbf{\text{I}}\right){\mathbf{\text{V}}}_1^{*}\right]}^{\frac{p}{2}-1}={\mathbf{\text{V}}}_1{\left({\mathbf{\Lambda }}_1+ϵ\mathbf{\text{I}}\right)}^{\frac{p}{2}-1}{\mathbf{\text{V}}}_1^{*}$$ (35)

Thus, one choice of the matrix square root ${\mathbf{\text{H}}}_1^{\frac{1}{2}}$ is

$$\begin{gathered} {\mathbf{\text{H}}}_1^{\frac{1}{2}}={({\mathbf{\Lambda }}_1+ϵ\mathbf{\text{I}})}^{\frac{p}{4}-\frac{1}{2}}{\mathbf{\text{V}}}_1^{*} \\ =[{({\text{λ}}_{p_1}^{(1)}+ϵ)}^{\frac{p}{4}-\frac{1}{2}}({\mathbf{\text{v}}}_1^{(1)})*,\dots ,{({\text{λ}}_{p_1}^{(L)}+ϵ)}^{\frac{p}{4}-\frac{1}{2}}({\mathbf{\text{v}}}_1^{(L)})*] \\ =[{\mathbf{\text{h}}}_1^{(1)},\dots ,{\mathbf{\text{h}}}_1^{(L)}] \end{gathered}$$ (36)

Similarly, we can obtain ${\mathbf{\text{H}}}_2^{\frac{1}{2}}$ as:

$$\begin{gathered} {\mathbf{\text{H}}}_2^{\frac{1}{2}}=[{({\text{λ}}_{p_2}^{(1)}+ϵ)}^{\frac{p}{4}-\frac{1}{2}}{({\mathbf{\text{v}}}_2^{(1)})}^{*},\dots ,{({\text{λ}}_{p_2}^{(M)}+ϵ)}^{\frac{p}{4}-\frac{1}{2}}({\mathbf{\text{v}}}_2^{(M)})*] \\ =[{\mathbf{\text{h}}}_2^{(1)},\dots ,{\mathbf{\text{h}}}_2^{(M)}] \end{gathered}$$ (37)

C. Implementation details

The details for solving the optimization problem (14) are shown in the following pseudocode Algorithm 1. In order to investigate how the SNR values of the recovered image behave as the function of the balancing parameters λ₁ and λ₂, we plot the parameters optimization results for two images in Fig. 1, where (a) correspond to the parameters for the compressed sensing reconstruction of an ankle MR image with the acceleration factor of 6, and (b) correspond to the parameters choices for the recovery of a phantom image with 4-fold undersampling. We find that the tuning of the two parameters is not very time consuming, since we observe that the optimal parameters are localized in a narrow range between 10⁵ and 10⁶ for λ₁ and between 10⁷ and 10⁸ for λ₂, for different images under different scenarios.

Fig. 1:

Parameters optimization results for the recovery of two images. (a) shows the SNR as the function of the two parameters λ₁ and λ₂ for the 6-fold undersampling recovery of an ankle MR image (as shown in Fig. 5). (b) shows the SNR as the function of λ₁ and λ₂ for the 4-fold undersampling recovery of a piecewise smooth phantom image (as shown in Fig. 3). The black rectangles correspond to the optimal parameters with the highest SNR values.

IV. Experiments and Results

A. 1-D signal recovery

We first experiment on a 1-D signal to investigate the performance of the algorithm on recovering signals from their undersampled measurements. Fig. 2 (a) shows the original signal, which is 2-fold undersampled in k space using variable density undersampling pattern, indicated in (b). The direct IFFT recovery is shown in (c). (d) shows the recovered signal (in blue solid line) using the proposed GSLR method in 1-D, and the decomposition results of the piecewise constant component ρ₁ (in black dotted line) and the piecewise linear component ρ₂ (in red dotted line). We then experiment on the signal recovery using a 4-fold random undersampling pattern, shown in (e). (f) is the direct IFFT of the undersampled measurements. (g) represents the recovered signal and the decomposition results. The results clearly show that by the GSLR method, both the jump discontinuities and the linear parts of the signal are nicely restored.

Fig. 2:

1-D signal recovery using the GSLR method. (a) is the original signal, which is undersampled by a 2-fold variable density random undersampling pattern, indicated in (b). (c) is the direct IFFT recovery of the undersampled measurements. (d) shows the recovered signal (in blue solid line) and the decomposition results of the two components, namely the piecewise constant component (showed in black dotted line) and the piecewise linear component (showed in red dotted line). (e)–(g) represent the results of a 4-fold random undersampling compressed sensing signal recovery experiment. Note that the proposed scheme recovers the Fourier coefficients of the signal; the ringing at the edges is associated by the inverse Fourier transform of the recovered Fourier coefficients.

B. MR images recovery

The performance of the proposed method is investigated in the context of compressed sensing MR images reconstruction. We compare the proposed GSLR method with the first order and the second order structured low-rank algorithms. We also study the improvement of the image quality offered by the GSLR algorithm over standard TV, TGV algorithm [4], and the LORAKS method [10]. For all of the experiments, we have manually tuned the parameters to ensure the optimal performance in each scenario. Specifically, we determine the parameters to obtain the optimized signal-to-noise ratio (SNR) to ensure fair comparisons between different methods. The SNR of the recovered image is computed as:

$$\text{SNR}=-10{\text{\hspace{0.17em}log\hspace{0.17em}}}_{10}\left(\frac{\Vert f_{\text{orig}}-\widehat{f}{\Vert }_F^2}{\Vert f_{\text{orig}}{\Vert }_F^2}\right)$$ (38)

where $\widehat{f}$ is the recovered image, f_orig is the original image, and ${\Vert \cdot \Vert }_F$ is the Frobenius norm.

In the experiments, we consider two types of undersampling trajectory: a radial trajectory with uniform angular spacing, and a 3-D variable density random retrospective undersampling trajectory. For the 3-D sampling pattern, since the readout direction is orthogonal to the image plane, such undersampling patterns can be implemented on the scanner.

We first study the performance of the proposed method for the recovery of a piecewise smooth phantom image from its noiseless k space data. We assume the data was sampled with 26 k radial spokes, with the approximate acceleration factor of 10.7. Fig. 3 (a) is the actual image, (b) to (g) are the recovered images using GSLR with filter size of 51 × 51, GSLR with filter size of 31 × 31, the 1st SLA and the 2nd SLA with filter size of 31 × 31, TGV, and the standard TV, respectively. The second row show the zoomed regions of the corresponding images. (o) is the undersampling pattern, (q) to (v) are the error images.

Fig. 3:

Recovery of a piecewise smooth phantom image from around 10-fold undersampled measurements. (a): the actual image. (b)–(g): Reconstructions using the proposed GSLR method with filter size of 51 × 51, GSLR with filter size of 31 × 31, the 1st and 2nd SLA with filter size of 31 × 31, TGV, and the standard TV, respectively. (h)–(n): the zoomed regions of the area indicated in the red rectangle for the corresponding images. (o): the undersampling pattern. (q)–(v): the error images. Note that recovered images using TGV and TV methods present undersampling artifacts, indicated in green arrows, while the GSLR methods outperform the other methods in recovering both the edges and the smooth regions, indicated in red arrows.

$$\begin{array}{l}\underset{\_}{\overline{\text{Algorithm\hspace{0.17em}}1:\text{\hspace{0.17em}GSLR image recovary algorithm}}}\\ \underset{\_}{\begin{array}{l}\hspace{1em}\text{Initialization:\hspace{0.17em}}{\widehat{\mathbf{\rho }}}^{\left(0\right)}\leftarrow \mathcal{A}*\mathbf{\text{b}},n\leftarrow 1,{ϵ}^{\left(0\right)}>0;\\ \hspace{1em}\mathbf{\text{while}}\hspace{0.17em}n<N_{\text{max}}\hspace{0.17em}\mathbf{\text{do}}\\ \hspace{1em}\hspace{1em}\hspace{1em}\mathbf{\text{Step}}\mathbf{1}:\hspace{0.17em}\mathbf{\text{Update of weight matrices}};\\ \hspace{0.17em}\hspace{0.17em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\text{Compute the Gram matrices}\hspace{0.17em}{\mathbf{\text{G}}}_i(i=1,2)\hspace{0.17em}\text{using}\hspace{0.17em}\left(33\right)\text{\hspace{0.17em}and\hspace{0.17em}}(34);\\ \hspace{0.17em}\hspace{0.17em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\text{Compute eigendecompositions\hspace{0.17em}}{({\lambda }_{pi}^{(k)},{\text{v}}_i^{\left(k\right)})}_{k=1}^K\text{of}\hspace{0.17em}{\mathbf{\text{G}}}_i(i=1,2);\\ \hspace{0.17em}\hspace{0.17em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\text{Compute\hspace{0.17em}}{\mathbf{\text{h}}}_1^l\text{\hspace{0.17em}and\hspace{0.17em}}{\mathbf{\text{h}}}_2^{\left(m\right)}\text{\hspace{0.17em}using\hspace{0.17em}}(36)\text{\hspace{0.17em}and\hspace{0.17em}}(37);\\ \hspace{1em}\hspace{1em}\hspace{1em}\mathbf{\text{Step}}\mathbf{2}:\hspace{0.17em}\mathbf{\text{Update of least squares}};\\ \hspace{0.17em}\hspace{0.17em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\text{Compute the weight matrices}\hspace{0.17em}{\mathbf{\text{S}}}_1\text{\hspace{0.17em}and\hspace{0.17em}}{\mathbf{\text{S}}}_2\text{\hspace{0.17em}using\hspace{0.17em}}\left(21\right)\text{\hspace{0.17em}and\hspace{0.17em}}(22);\\ \hspace{0.17em}\hspace{0.17em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\text{Solve the following least squares problem :}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\underset{{\widehat{\mathbf{\rho }}}_1,{\widehat{\mathbf{\rho }}}_2}{\text{min}}{\mathrm{\lambda }}_1\Vert {\mathbf{\text{S}}}_1^{\frac{1}{2}}{\mathbf{\text{y}}}_1{\Vert }_F^2+\lambda 2\Vert {\mathbf{\text{S}}}_2^{\frac{1}{2}}{\mathbf{\text{y}}}_2{\Vert }_F^2+\Vert \mathcal{A}\left({\widehat{\mathbf{\rho }}}_1+{\widehat{\mathbf{\rho }}}_2\right)-\mathbf{\text{b}}{\Vert }^2\text{\hspace{0.17em}using ADMM iterations\hspace{0.17em}}(24)\text{\hspace{0.17em}to\hspace{0.17em}}(28);\\ \hspace{1em}\hspace{1em}\hspace{1em}\text{Choose\hspace{0.17em}}{ϵ}^{\left(n\right)}\text{\hspace{0.17em}such that\hspace{0.17em}}0<{ϵ}^{\left(n\right)}\le {ϵ}^{\left(n-1\right)};\\ \hspace{1em}\mathbf{\text{end}}\end{array}}\\ \hspace{1em}\hspace{1em}\hspace{1em}\end{array}$$

We observe that the structured low-rank algorithms outperform TGV and the standard TV algorithms under this scenario, in that the recovered images by TGV and TV methods suffer from obvious undersampling artifacts, indicated in green arrows. For the structured low-rank algorithms with filter size of 31 × 31, it is shown that GSLR performs better than the 1st SLA and the 2nd SLA in recovering the edges, indicated in red arrows. It is shown that with larger filter sizes (51 × 51), the GSLR method provides the best reconstruction result with the SNR improvement of around 3dB over standard TV.

In the following experiments, we investigate the proposed GSLR method on the reconstruction of single-coil real MR images. The reconstructions of a brain MR image at the acceleration of 4 is shown in Fig. 4, where we compare the proposed GSLR method using different filter sizes with the first and second order SLA, S-LORAKS method, TGV, and the standard TV method. Fig. 4 (a) is the original image. (b) to (h) are the reconstructions using the GSLR with filter size of 51 × 51, GSLR with filter size 31 × 31, the 1st and 2nd SLA with filter size 31 × 31, S-LORAKS, TGV, and the standard TV. (i) to (p) are the zoomed versions of the images, indicated by the red rectangle. (q) is the variable density random undersampling pattern. (r) to (x) are the error images using the corresponding methods. It is seen that among all of the methods, GSLR performs the best in preserving the details and providing the most accurately recovered image. Note that by increasing the filter size from 31 × 31 to 51 × 51, the image quality is significantly improved, with only a modest increase in runtime (85 s versus 36 s).

Fig. 4:

Recovery of a brain MR image from 4-fold undersampled measurements. (a): the original image. (b)–(h): Reconstructions using GSLR with filter size of 51 × 51, GSLR with filter size 31 × 31, the 1st and 2nd SLA with filter size 31 × 31, S-LORAKS, TGV, and the standard TV, respectively. (i)–(p): The zoomed versions of the area shown in the red rectangle in (a). (q): The undersampling pattern. (r)–(x): the error images.

We demonstrate the performance of the proposed method on the reconstruction of an ankle MR image at the approximate acceleration rate of 6.7 using the radial undersampling pattern in Fig. 5. In this experiment, we compare the proposed GSLR method with 1st and 2nd SLR, S-LORAKS, G-LORAKS, TGV, and the standard TV. All of the structured low rank methods are with filter size of 51 × 51. (a) is the original image. (b) to (h) are the reconstructed images using GSLR, 1st SLA, 2nd SLA, S-LORAKS, G-LORAKS, TGV, and TV, respectively. (j) to (p) are the zoomed versions of the images indicated by the red rectangle shown in (a). (q) is the radial undersampling pattern. (r) to (x) are the error images. We observe that TV method gives blurry reconstruction. The images recovered by S-LORAKS, G-LORAKS, and TGV methods have undersampling artifacts, indicated in the green arrow. The structured low rank methods provide improved results, among which GSLR performs better in preserving image details, shown in red arrows.

Fig. 5:

Recovery of an ankle MR image from 6.7-fold undersampled measurements with radial undersampling pattern. (a): The actual image. (b)–(f): The recovery images using GSLR, 1st SLA, 2nd SLA with filter size of 51 × 51, S-LORAKS, G-LORAKS, TGV, and TV, respectively. (i)–(p): The zoomed versions of the images. (q): The undersampling pattern. (r)–(s): Error images using the corresponding methods. Note that the proposed GSLR method performs the best in preserving the edges and eliminating the artifacts caused by the undersampling, compared with the other methods.

In Fig. 6, we experiment on a brain MR image using radial undersampling pattern with the acceleration factor around 4.8. (a) is the actual image. (b) to (h) are the reconstruction results using GSLR with filter size of 51 × 51, 1st and 2nd SLA with filter size of 51 × 51, S-LORAKS, G-LORAKS, TGV, and the standard TV, respectively. The second row are the zoomed versions of the red rectangular area shown in (a) for different methods. (q) shows the undersampling pattern. (r) to (x) are the error images of the corresponding methods, respectively. We observe that TV and TGV methods provide blurry reconstructions, and the LORAKS methods preserve fine details better while suffers from undersampling artifacts. Among the methods, GLSR provides the best reconstruction and improves the SNR by around 2dB compared over standard TV.

Fig. 6:

Recovery of a brain MR image from 4.85-fold undersampled measurements. (a): The actual image. (b)–(h): The recovery images using GSLR, 1st and 2nd SLA with 51 × 51 filter size, S-LORAKS, G-LORAKS, TGV, and the standard TV, respectively. (i)–(p): Zoomed versions of the red rectangular area for different methods. (q): the undersampling pattern. (r)–(x): Error images of the corresponding methods.

In Fig. 7, we compare different methods on the recovery of a multicoil MR dataset acquired using four coils from 8-fold undersampled measurements. The data was retrospectively undersampled using the variable density random undersampling pattern. (a) is the actual image. (b) to (g) show the reconstruction results using GSLR, 1st and 2nd SLA with filter size of 31 × 31, S-LORAKS, G-LORAKS, and the standard TV, respectively. (h) to (n) are the zoomed regions indicated in the red rectangle. (o) is the undersampling pattern. (p) to (u) indicate error images by different methods. According to the results, we observe that compared with the other methods, GLSR performs the best in preserving the image features and providing the recovered image with highest SNR value. We have shown the phase images of all the datasets in Fig. 8. We note that all of the images are associated with reasonable phase variations, expected from a typical MR acquisition. We note that GSLR relies on the compact representation of the image, enabled by its decomposition into piecewise constant and linear components. Since S-LORAKS and G-LORAKS do not exploit these property, we obtain improved reconstructions with filter sizes larger than 31 × 31.

Fig. 7:

Recovery of a multicoil brain MR dataset from 8-fold undersampled measurements. (a): The actual image. (b)–(g): The reconstructions using GSLR, the first and second order SLA with 31 × 31 filter size, S-LORAKS and G-LORAKS method, and the standard TV. (h)–(n): The zoomed regions. (o): The undersampling pattern. (p)–(u): The error images. Note that GLSR provides the most accurate reconstruction result compared with the other methods, indicated by red arrows.

Fig. 8:

Phase images of the real datasets. (a) Brain image in Fig.4. (b) Ankle image in Fig.5. (c) Brain image in Fig.6. (d) Brain image in Fig.7.

The SNRs of the recovered images using variable density random undersampling patterns are shown in Table I, and the reconstruction results using radial undersampling patterns are shown in Table II. We compare the 1st and 2nd SLA, S-LORAKS, G-LORAKS, TGV, the standard TV with the proposed GSLR method. For the structured low-rank algorithms, we compare the performance by different filter sizes. Specifically, we use three different filter sizes, 15 × 15, 31 × 31 and 51 × 51 for the variable density undersampling experiments, and two filter sizes for the radial undersampling experiments. Note that when the filter size is 15 × 15, the results provided by GSLR are not comparable to the other methods for some cases. However, using larger filter sizes leads to significantly improved image quality. For filter sizes 31 × 31 and 51 × 51, GSLR consistently obtains the best results, with the SNR improvement by around 2–3 dB over standard TV. The reason is that the size of the filter specifies the type of curves or edges that its zero set can capture. Specifically, smaller filters can only represent simpler and smoother curves, while larger filters can represent complex shapes (see [11] for an illustration). When complex structures are presented in the image, the use of a smaller filter fails to capture the intricate details. We note that for most images, we need to use larger filters to ensure that the details are well captured. The use of larger filters is made possible by the proposed IRLS algorithm, which does not require us to explicitly compute the Toeplitz matrices.

Table I:

Comparison of MR image recovery algorithms using variable density random undersampling pattern

filter size	[15,15]	[31,31]	[51,51]	[15,15]	[31,31]	[51,51]	[15,15]	[31,31]	[51,51]
Phantom	acc=2			acc=4			acc=5
First SLA	47.63	53.78	55.34	38.06	44.20	46.20	34.87	39.93	42.06
Second SLA	48.77	56.26	57.66	33.73	43.30	45.53	33.62	38.07	41.10
TGV	42.17	42.17	42.17	37.86	37.86	37.86	35.40	35.40	35.40
TV	41.31	41.31	41.31	35.88	35.88	35.88	35.06	35.06	35.06
GSLR	51.13	56.94	58.21	39.22	45.31	47.18	35.93	40.79	43.11
Brain Fig.4	acc=2			acc=4			acc=5
First SLA	34.93	36.14	36.72	24.85	26.40	26.80	23.00	24.66	25.24
Second SLA	34.10	35.67	36.31	24.28	25.90	26.43	22.63	24.30	24.73
TGV	33.09	33.09	33.09	25.36	25.36	25.36	23.34	23.34	23.34
TV	32.40	32.40	32.40	24.71	24.71	24.71	22.90	22.90	22.90
S-LORAKS	35.43	35.43	35.43	25.86	25.86	25.86	24.22	24.22	24.22
G-LORAKS GSLR	33.36	33.36	33.36	24.93	24.93	24.93	23.01	23.01	23.01
	35.34	36.38	37.19	25.34	26.72	27.36	23.58	25.03	26.07
Brain Fig.6	acc=2			acc=4			acc=5
First SLA	30.31	32.30	32.61	20.64	23.63	24.17	18.77	21.36	22.54
Second SLA	30.16	31.99	32.33	19.24	22.81	23.62	17.56	20.70	22.19
TGV	30.26	30.26	30.26	23.05	23.05	23.05	21.13	21.13	21.13
TV	30.10	30.10	30.10	22.65	22.65	22.65	20.83	20.83	20.83
S-LORAKS	29.83	29.83	29.83	22.71	22.71	22.71	21.02	21.02	21.02
G-LORAKS	28.69	28.69	28.69	22.06	22.06	22.06	20.69	20.69	20.69
GSLR	30.70	32.51	33.17	21.41	24.20	24.93	19.51	22.15	23.08
Ankle	acc=2			acc=4			acc=5
First SLA	37.80	38.13	38.42	30.01	30.89	31.17	27.37	28.26	28.50
Second SLA	37.96	38.38	38.61	29.65	30.69	30.90	26.65	27.31	28.08
TGV	36.89	36.89	36.89	30.43	30.43	30.43	28.05	28.05	28.05
TV	33.43	33.43	33.43	28.38	28.38	28.38	26.22	26.22	26.22
S-LORAKS	37.91	37.91	37.91	30.18	30.18	30.18	27.44	27.44	27.44
G-LORAKS	37.02	37.02	37.02	29.27	29.27	29.27	26.85	26.85	?26.85
GSLR	38.05	38.47	39.05	30.46	31.00	31.66	27.96	28.43	28.96
Multi-coil	acc=4			acc=6			acc=8
First SLA	29.64	30.97	31.32	23.48	25.45	25.81	21.43	23.68	24.04
Second SLA	29.70	31.22	31.68	23.74	25.67	25.99	21.02	23.39	23.62
TGV	27.48	27.48	27.48	21.53	21.53	21.53	21.70	21.70	21.70
TV	26.68	26.68	26.68	22.15	22.15	22.15	21.26	21.26	21.26
S-LORAK	27.83	27.83	27.83	23.92	23.92	23.92	21.53	21.53	21.53
G-LORAKS	27.22	27.22	27.22	22.81	22.81	22.81	20.83	20.83	20.83
GSLR	30.08	31.48	32.24	23.88	25.82	26.29	22.00	24.16	24.58

Table II:

Comparison of MR image recovery algorithms using radial undersampling pattern

filter size	[31,31]	[51,51]	[31,31]	[51,51]
Brain Fig.4	acc≈4.8		acc≈6.7
First SLA	26.64	26.85	23.19	23.80
Second SLA	26.05	26.55	22.82	23.32
TGV	25.90	25.90	23.11	23.11
TV	25.26	25.26	22.59	22.59
S-LORAKS	26.31	26.31	22.85	22.85
G-LORAKS	25.97	25.97	21.30	21.30
GSLR	26.77	27.25	23.45	24.18
Brain Fig.6	acc≈4.8		acc≈6.7
First SLA	24.99	26.25	21.17	22.62
Second SLA	23.82	24.95	20.86	22.29
TGV	24.26	24.26	21.63	21.63
TV	23.65	23.65	20.88	20.88
S-LORAKS	24.23	24.23	21.20	21.20
G-LORAKS	23.75	23.75	21.53	21.53
GSLR	25.23	26.62	22.47	23.01
Ankle	acc≈4.8		acc≈6.7
First SLA	31.02	31.37	26.15	26.76
Second SLA	31.14	31.42	26.07	26.49
TGV	30.53	30.53	25.59	25.59
TV	28.53	28.53	24.15	24.15
S-LORAKS	31.03	31.03	25.15	25.15
G-LORAKS	29.89	29.89	25.24	25.24
GSLR	31.39	31.69	26.60	27.08
Multi-coil	acc=5.2		acc=10
First SLA	27.78	29.01	22.33	22.87
Second SLA	27.24	28.15	21.76	22.60
TGV	26.08	26.08	20.77	20.77
TV	24.92	24.92	18.53	18.53
S-LORAK	27.32	27.32	21.01	21.01
G-LORAKS	26.58	26.58	20.22	20.22
GSLR	27.90	29.43	22.51	23.15

V. Conclusion

We proposed a novel generalized structured low-rank algorithm to recover images from their undersampled k space measurements. We assume that an image can be modeled as the superposition of two piecewise smooth functions, namely a piecewise constant component, and a piecewise linear component. Each component can be annihilated by multiplication with a bandlimited polynomial function, which yields to a structured Toeplitz matrix. We formulate a combined regularized optimization algorithm by exploiting the low-rank property of the Toeplitz matrix. In order to solve the corresponding problem efficiently, we adapt the iteratively reweighted least squares method which alternates between the computation of the annihilation filter and the least squares problem. We investigate the proposed algorithm on the compressed sensing reconstruction of single-coil and multi-coil MR images. Experiments show that the proposed algorithm provides more accurate recovery results compared with the state-of-the-art approaches.