A Convergent Generalized Krylov Subspace Method for Compressed Sensing MRI Reconstruction with Gradient-Driven Denoisers

Abstract

Model-based reconstruction plays a key role in compressed sensing (CS) MRI, as it incorporates effective image regularizers to improve the quality of reconstruction. The Plug-and-Play and Regularization-by-Denoising frameworks leverage advanced denoisers (e.g., convolutional neural network (CNN)-based denoisers) and have demonstrated strong empirical performance. However, their theoretical guarantees remain limited, as practical CNNs often violate key assumptions. In contrast, gradient-driven denoisers achieve competitive performance, and the required assumptions for theoretical analysis are easily satisfied. However, solving the associated optimization problem remains computationally demanding. To address this challenge, we propose a generalized Krylov subspace method (GKSM) to solve the optimization problem efficiently. Moreover, we also establish rigorous convergence guarantees for GKSM in nonconvex settings. Numerical experiments on CS MRI reconstruction with spiral and radial acquisitions validate both the computational efficiency of GKSM and the accuracy of the theoretical predictions. The proposed optimization method is applicable to any linear inverse problem.

I. Introduction

MAGNETIC resonance imaging (MRI) scanners acquire k-space data that represents the Fourier coefficients of the image of interest. However, the acquisition process is inherently slow due to physical, hardware, and sampling constraints [1]. This slow acquisition presents several practical challenges, including patient discomfort, motion artifacts, and reduced throughput. Since the seminal work in [2], compressed sensing (CS) MRI has attracted significant attention in the MRI community [3, 4] for accelerating the acquisition process through structured sampling patterns. Modern CS MRI methods incorporate multiple receiver coils (a.k.a. parallel imaging [5, 6]) to further improve acquisition speed. Image reconstruction in CS MRI requires solving the following composite minimization problem:

$$x^{\ast }=\text{arg}\underset{x\in {ℂ}^N}{\text{min}}F(x)\triangleq \underset{h(x)}{\underbrace{\frac{1}{2}\Vert Ax-y{\Vert }_2^2}}+\lambda f(x),$$ (1)

where $\mathbf{A}\in {\mathbb{C}}^{MC\times N}$ denotes the forward operator that maps the image $\mathbf{x}\in {\mathbb{C}}^N$ to the measured k-space data $\mathbf{y}\in {\mathbb{C}}^{MC}$. Here, we consider $C$ receiver coils. The encoding operator $\mathbf{A}$ is a stack of $C$ submatrices ${\mathbf{A}}_c\in {\mathbb{C}}^{M\times N}$, each defined as ${\mathbf{A}}_c={\mathbf{P}\mathbf{F}\mathbf{S}}_c$, where $\mathbf{P}$ is the sampling mask, $\mathbf{F}$ is the (nonuniform) Fourier transform, and ${\mathbf{S}}_c$ is the coil sensitivity map corresponding to the cth coil, which is patient-dependent. The trade-off parameter $\lambda >0$ balances $h\left(\mathbf{x}\right)$ and $f\left(\mathbf{x}\right)$.

The data-fidelity term $h\left(\mathbf{x}\right)$ in (1) promotes consistency with the acquired k-space data. In practice, often $M\ll N$ due to under-sampling, making (1) ill-posed. Therefore, incorporating prior knowledge through the regularizer $f\left(\mathbf{x}\right)$ is essential for stabilizing the reconstruction. The choice of regularization plays a crucial role in reconstruction quality. Traditional hand-crafted regularizers include wavelets [7], total variation (TV) [8, 9], combinations of wavelets and TV [2, 10], dictionary learning [11, 12], and low-rank models [13], to name a few. For reviews of various choices for $f\left(\mathbf{x}\right)$, see [4, 14, 15].

In the past decade, deep learning (DL) has gained prominence in MRI reconstruction due to its capacity to learn complex image priors directly from large training datasets [16]. Roughly speaking, DL-based approaches can be broadly categorized into end-to-end networks [17] and physics-driven unrolled algorithms [18–20]. Recently, generative models have emerged as a powerful class for modeling priors in MRI, achieving impressive results across various settings [21].

An alternative to classical DL pipelines is the Plug-and-Play (PnP) [22] and REgularization-by-Denoising (RED) [23] frameworks. PnP and RED integrate powerful denoisers into iterative reconstruction algorithms and have demonstrated competitive performance across various imaging tasks [24–29]. Early PnP/RED frameworks employed classical denoisers such as the median filter [30], non-local means [31], and BM3D [32]. As convolutional neural network (CNN)–based denoisers have shown superior performance to classical ones, modern PnP/RED methods typically integrate CNN-based denoisers. Unlike end-to-end or unrolled DL methods that require retraining for each imaging task, PnP and RED leverage learned image priors to flexibly adapt to changes in the forward model without retraining. This adaptability is particularly beneficial in CS MRI reconstruction, where scanspecific variations (e.g., different sampling trajectories and patient-specific sensitivity maps) are common. See [33] for a review of PnP methods that incorporate both classical and CNN-based denoisers in MRI reconstruction.

Despite the empirical success of PnP and RED, their theoretical convergence guarantees remain an active area of research; see [26, 34–38]. These works typically require that the denoisers either approximate maximum a posteriori or minimum mean squared error estimators, or satisfy the nonexpansiveness condition. However, many successful denoisers—especially those based on CNNs—do not satisfy these assumptions. As a result, PnP and RED with such denoisers cannot be rigorously interpreted as optimization algorithms. Although optimization-free perspectives have been proposed, understanding the behavior of these frameworks remains challenging [26]. One alternative is to train denoisers with additional regularization that enforces a bounded Lipschitz constant [36, 37]. However, guaranteeing strict and tight boundedness in practice remains an open challenge.

Recent efforts have aimed to close the gap between the theoretical foundations and practical effectiveness of PnP and RED by introducing gradient-driven denoisers [39–41]. In this approach, the unknown image $\mathbf{x}$ is recovered by solving

$$x^{\ast }=\text{arg}\underset{x\in 𝒞}{\text{min}}F(x):=\underset{h(x)}{\underbrace{\frac{1}{2}\Vert Ax-y{\Vert }_2^2}}+\lambda f_{\theta }(x),$$ (2)

where $f_{𝛉}(\mathbf{x})$ is a scalar-valued energy function parameterized by CNNs that serves as a learned image prior and $𝒞$ is a closed convex set in ${\mathbb{C}}^N$. The parameters $𝛉$ are learned by enforcing $\mathbf{x}-{\nabla }_{\mathbf{x}}{f}_{𝛉}(\mathbf{x})$ to act as a denoiser. Thus, the only required assumption is the differentiability of $f_{𝛉}$ with respect to $\mathbf{x}$, which allows one to integrate deep learning into inverse problems while maintaining a degree of interpretability—an essential requirement in medical imaging, where reconstructions directly influence diagnostic decisions. For notational simplicity, we omit the subscript $𝛉$ and use $\nabla f(\mathbf{x})$ instead of ${\nabla }_{\mathbf{x}}f(\mathbf{x})$. Moreover, we absorb $\lambda$ into $f(\mathbf{x})$ in the following discussion since $\lambda$ is fixed throughout the minimization once it is selected.

Although both $h(\mathbf{x})$ and $f(\mathbf{x})$ in (2) are differentiable, $f(\mathbf{x})$ is generally nonconvex, which poses challenges for designing convergent and efficient algorithms. Cohen et al. [39] applied a projected gradient descent method with a line search to solve (2). Alternatively, Hurault et al. [40] employed the proximal gradient descent method with line search. Both approaches provide convergence guarantees under the assumption that $\nabla f$ is Lipschitz continuous. However, these methods typically require hundreds of iterations to converge, which limits their practical applicability. Recently, Hong et al. [42] proposed a convergent complex quasi-Newton proximal method (CQNPM) that significantly reduces the computational time required to solve (2). Their convergence is established under the assumptions that $\nabla f$ is Lipschitz continuous and that the proximal Polyak-Łojasiewicz condition holds. Although CQNPM converges faster than existing methods for solving (2), it requires solving a weighted proximal mapping (as defined in [42, equation (3)]) at each iteration. This step requires computing $\mathbf{A}\mathbf{x}$ and ${\mathbf{A}}^{\mathrm{H}}\mathbf{x}$ multiple times,¹ which which can increase the overall computational complexity. Computing Ax is expensive in MRI reconstruction with many coils, high-resolution images, or many interleaves or spokes in non-Cartesian acquisitions. Drawing inspiration from Krylov subspace methods (KSMs) [43], we propose a generalized Krylov subspace method (GKSM) for efficiently solving (2), which requires computing $\mathbf{A}\mathbf{x},{\mathbf{A}}^{\mathrm{H}}\mathbf{x}$, and $\nabla f(\mathbf{x})$ only once per iteration. Our main contributions are summarized as follows:

• We propose a generalized Krylov subspace method (GKSM) for efficiently solving (2).

• We present a rigorous convergence analysis of GKSM in nonconvex settings, along with the convergence rate of the cost function values.

• We extensively evaluate the performance of GKSM on brain (respectively, knee) images from the dataset described in the MoDL paper [18] (respectively, the NYU fastMRI dataset [44]). The k-space data are simulated from the reconstructed complex-valued images using spiral and radial sampling trajectories. We also empirically validate the accuracy of the convergence analysis.

The rest of this paper is organized as follows. Section II reviews the preliminaries on KSMs and discusses related work that generalizes KSMs for solving inverse problems. Section III describes GKSM in detail. Section IV provides a rigorous convergence analysis of GKSM. Section V reports experimental results that evaluate the performance of GKSM and empirically validate the theoretical analysis.

II. Preliminaries on Krylov Subspace Methods

This section first introduces KSMs, which were primarily developed for solving linear equations. We then review related generalized Krylov methods for linear inverse problems, along with existing theoretical results. Our main goal in this section is to provide a sketch of the key developments in KSMs, from their origins in solving linear equations to their generalization for inverse problems.

KSMs are a class of iterative algorithms for solving problems of the form

$$\overline{A}x=b,$$ (3)

where $\overline{\mathbf{A}}\in {\mathbb{R}}^{N\times N}$ is typically sparse, ill-conditioned, and large-scale. At kth iteration, KSMs construct an approximate solution to ${\mathrm{x}}^{\ast }$ within the Krylov subspace:

$${𝒦}_k\left(\overline{\mathbf{A}},{\overline{\mathbf{r}}}_1\right)=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}\left\{{\overline{\mathbf{r}}}_1,\overline{\mathbf{A}}{\overline{\mathbf{r}}}_1,{\overline{\mathbf{A}}}^2{\overline{\mathbf{r}}}_1,\dots ,{\overline{\mathbf{A}}}^{k-1}{\overline{\mathbf{r}}}_1\right\},$$ (4)

where ${\overline{\mathbf{r}}}_1=\mathbf{b}-\overline{\mathbf{A}}{\mathbf{x}}_1$ is the initial residual. The approximate solution ${\mathbf{x}}_{k+1}$ is obtained by seeking ${\mathbf{x}}_{k+1}\in {\mathbf{x}}_1+{𝒦}_k\left(\overline{\mathbf{A}},{\overline{\mathbf{r}}}_1\right)$ that minimizes a chosen norm of the residual. The most widely used KSMs include, but are not limited to, the conjugate gradient method [45], LSQR [46], BiCGSTAB [47], and the generalized minimal residual method [48]. These methods are designed for different types of matrices $\overline{\mathbf{A}}$, such as symmetric positive definite, non-symmetric, or indefinite, to name a few [43]. Moreover, KSMs can incorporate preconditioners to further accelerate convergence [49].

Many inverse problems with variational regularizers [50, 51] can be modeled as the following ${\ell }_p-{\ell }_q$ optimization problem:

$$\underset{x\in {ℝ}^N}{\text{min}}\frac{1}{p}{\Vert Ax-y\Vert }_p^p+\frac{\lambda }{q}{\Vert Wx\Vert }_q^q,$$ (5)

where $0<p,q\le 2$, and $\mathbf{W}$ represents a transform such as a wavelet transform. KSMs have been generalized to solve problems such as (5). For $p=q=2$, Lampe et al. [52] presented a generalized KSM to address (5) in which the parameter $\lambda$ is adaptively adjusted to keep $\Vert \mathbf{A}\mathbf{x}-\mathbf{y}{\Vert }_2$ sufficiently close to a prescribed tolerance. Lanza et al. [53] proposed to solve (5) using a KSM along with an iteratively reweighted approach. Moreover, Huang et al. combined Krylov subspace-based method with majorization minimization for solving (5) [54]. To avoid the inner-outer iterations when using the iteratively approach in [53], several flexible Krylov subspaces were proposed to improve efficiency [55–57]. See [50, 58] for a review of using KSMs for inverse problems.

Besides their extension to inverse problems, rigorous convergence analyses of KSMs remain an open research area. Lanza et al. [53] showed that the iterates converged to a minimizer of (5) for $1\le p,q\le 2$ if $\mathrm{k}\mathrm{e}\mathrm{r}\left({\mathbf{A}}^{\mathrm{T}}\mathbf{A}\right)\cap \mathrm{k}\mathrm{e}\mathrm{r}\left({\mathbf{W}}^{\mathrm{T}}\mathbf{W}\right)=\left\{0\right\}$, where $\mathrm{k}\mathrm{e}\mathrm{r}(\cdot )$ denotes the null space of the matrix, and the constructed Krylov subspace fully represents the entire image domain. Similar convergence results can also be found in [54, 59]. Other works [55–57] proved that the cost function values are monotonically decreasing and that the iterates converge to a stationary point. For brevity, we present only the convergence results of KSMs for inverse problems. See [60, 61] and the references therein for discussions on KSMs and their convergence in other contexts.

III. Proposed Method

This section provides the details of our GKSM for solving (2). We first discuss the case where $𝒞={\mathbb{C}}^N$ and then describe how to incorporate a convex constraint. Lastly, we provide further discussion of GKSM to offer additional insights.

Given a subspace basis ${\mathbf{V}}_k\in {\mathbb{C}}^{N\times k}$ satisfying ${\mathbf{V}}_k^{\mathrm{H}}{\mathbf{V}}_k={\mathbf{I}}_k$ where ${\mathbf{I}}_k$ is the identity matrix with dimension $k$, a Hermitian positive definite matrix ${\mathbf{B}}_k\in {\mathbb{C}}^{N\times N},{\mathbf{B}}_k\succ 0$, and ${\alpha }_k\in \mathbb{R},{\alpha }_k>0$, GKSM solves the following problem at the kth iteration to obtain the coefficient ${𝛃}_k\in {\mathbb{C}}^k$:

$${𝛃}_k=\mathrm{a}\mathrm{r}\mathrm{g}\underset{𝛃\in {\mathbb{C}}^k}{\mathrm{m}\mathrm{i}\mathrm{n}}\underset{\bar{F}\left(\mathbf{x},{\mathbf{x}}_k\right)}{\underbrace{\frac{1}{2}{\Vert \mathbf{A}\mathbf{x}-\mathbf{y}\Vert }_2^2+\bar{f}\left(\mathbf{x},{\mathbf{x}}_k,{\mathbf{B}}_k,{\alpha }_k\right)}},$$ (6)

where $\mathbf{x}={\mathbf{V}}_k𝛃,\bar{f}\left(\mathbf{x},{\mathbf{x}}_k,{\mathbf{B}}_k,{\alpha }_k\right)\equiv ⟨\nabla f\left({\mathbf{x}}_k\right),\mathbf{x}⟩+\frac{1}{2{\alpha }_k}{\Vert \mathbf{x}-{\mathbf{x}}_k\Vert }_{{\mathbf{B}}_k}^2$ is a quadratic proximal term, and $\Vert \mathbf{x}{\Vert }_{{\mathbf{B}}_k}^2={\mathbf{x}}^{\mathrm{H}}{\mathbf{B}}_k\mathbf{x}$. Then the next image iterate is ${\mathbf{x}}_{k+1}={\mathbf{V}}_k{𝛃}_k$. Rewriting (6) in terms of $𝛃$ and reorganizing yields

$${𝛃}_k=\mathrm{a}\mathrm{r}\mathrm{g}\underset{𝛃\in {\mathbb{C}}^k}{\mathrm{m}\mathrm{i}\mathrm{n}}{‖\left[\begin{array}{c}\mathbf{A}{\mathbf{V}}_k𝛃\\ {\overline{\mathbf{B}}}_k^{\frac{1}{2}}{\mathbf{V}}_k𝛃\end{array}\right]-\left[\begin{array}{c}\mathbf{y}\\ {\overline{\mathbf{B}}}_k^{\frac{1}{2}}{\mathbf{w}}_k\end{array}\right]‖}_2^2,$$ (7)

where ${\mathbf{w}}_k={\mathbf{x}}_k-{\alpha }_k{\mathbf{B}}_k^{-1}\nabla f\left({\mathbf{x}}_k\right)$ and ${\overline{\mathbf{B}}}_k^{1/2}$ denotes the principal matrix square root of ${\overline{\mathbf{B}}}_k={\mathbf{B}}_k/{\alpha }_k$ and it is unique [62]. Note that ${\mathbf{A}\mathbf{V}}_k$ is built incrementally during the algorithm. Compared with the image size, the dimension of ${𝛃}_k$ is relatively low because the number of iterations will be significantly smaller than the image dimension. Therefore, we solve (7) directly, i.e.,

$${𝛃}_k={\left({\mathbf{V}}_k^{\mathrm{H}}{\mathbf{A}}^{\mathrm{H}}\mathbf{A}{\mathbf{V}}_k+{\mathbf{V}}_k^{\mathrm{H}}{\overline{\mathbf{B}}}_k{\mathbf{V}}_k\right)}^{-1}{\mathbf{V}}_k^{\mathrm{H}}\left({\mathbf{A}}^{\mathrm{H}}\mathbf{y}+{\overline{\mathbf{B}}}_k{\mathbf{w}}_k\right).$$ (8)

Algorithm 1.

Generalized Krylov Subspace Method (GKSM)

Initialization: ${\mathbf{\text{x}}}_1$, stepsize ${\alpha }_k>0,V_1=\frac{A^{\text{H}}y}{\Vert A^{\text{H}}y\Vert },A{V}_1,$ maximal number of subspace iterations K, and maximal number of total iterations Max_Iter
Iteration:
1:	for k = 1, 2,…, Max_Iter do
2:	Compute $\nabla f\left(x_k\right)$
3:	Set $B_k$ using Algorithm 2
4:	Compute ${𝛃}_k$ using (8) (or solve (11) for ${𝛃}_k$ if a convex constraint is enforced)
5:	Compute $x_{k+1}\leftarrow V_k{𝛃}_k$
6:	if $k\le K$ then
7:	Compute $r_k\leftarrow {\nabla }_x\overline{F}\left(x_{k+1},x_k\right)$
8:	${\tilde{r}}_k\leftarrow \left(I-V_k{V}_k^{\text{H}}\right)r_k$
9:	if $\Vert {\tilde{r}}_k\Vert \ne 0$ then
10:	$v_{k+1}\leftarrow {\tilde{r}}_k/\Vert {\tilde{r}}_k\Vert$
11:	$V_{k+1}\leftarrow \left[\begin{array}{ll}{V}_k\hfill & v_{k+1}\hfill \end{array}\right]$
12:	$A{V}_{k+1}\leftarrow \left[\begin{array}{ll}A{V}_k\hfill & A{v}_{k+1}\hfill \end{array}\right]$
13:	else
14:	$V_{k+1}\leftarrow V_k$
15:	$A{V}_{k+1}\leftarrow A{V}_k$
16:	end if
17:	else
18:	$V_{k+1}\leftarrow I_N$
19:	end if
20:	end for

Here the matrix being inverted is only $k\times k$ with $k\ll N$.

To enrich the subspace after the kth iteration, we first compute the gradient of the objective function in (6) with respect to $\mathbf{x}$ at $\mathbf{x}={\mathbf{x}}_{k+1}$, i.e.,

$${\mathbf{r}}_k={\nabla }_{\mathbf{x}}\bar{F}\left({\mathbf{x}}_{k+1},{\mathbf{x}}_k\right).$$ (9)

Then we set ${\mathbf{v}}_{k+1}={\tilde{\mathbf{r}}}_k/‖{\tilde{\mathbf{r}}}_k‖$ with ${\tilde{\mathbf{r}}}_k=\left({\mathbf{I}}_N-{\mathbf{V}}_k{\mathbf{V}}_k^{\mathrm{H}}\right){\mathbf{r}}_k$. The new subspace basis ${\mathbf{V}}_{k+1}$ is formulated as $\left[\begin{array}{ll}{\mathbf{V}}_k& {\mathbf{v}}_{k+1}\end{array}\right]$. If $‖{\tilde{\mathbf{r}}}_k‖=0$, we simply skip the update of ${\mathbf{V}}_k$. Note that the dimension of ${𝛃}_k$ is equal to the number of columns of ${\mathbf{V}}_k$. This dimension may be smaller than $k$ if the event $‖{\tilde{\mathbf{r}}}_k‖=0$ occurs, in which case the number of columns of ${\mathbf{V}}_k$ is also smaller than $k$. Algorithm 1 summarizes the detailed steps of GKSM. To establish the convergence rate of the cost function values, we introduce an additional step 18 in Algorithm 1, as the generated ${\mathbf{V}}_k$ does not necessarily span the entire image domain. Note that GKSM reduces to CQNPM [42] when ${\mathbf{V}}_k={\mathbf{I}}_N$. For this case, we simply apply the accelerated gradient descent method to solve (6). If the regularizer $f$ in (2) is a quadratic function, then the subspace spanned by ${\mathbf{V}}_k$ simplifies to the classical Krylov subspace in (4). To obtain ${\mathbf{B}}_k$, we adopt the algorithm presented in [42, Algorithm 2] such that ${\mathbf{B}}_k$ is an estimate of the Hessian matrix of $f(\mathbf{x})$ that is guaranteed to be Hermitian positive definite. For completeness, Algorithm 2 provides the detailed steps for computing ${\mathbf{B}}_k$. The operator $\Re (\cdot )$ in Algorithm 2 extracts the real part.

Algorithm 2.

Modified Memory Efficient Self-Scaling Hermitian Rank-1 Method

Initialization: ${\text{x}}_{k-1},{\text{x}}_k,\nabla f\left({\text{x}}_{k-1}\right),\nabla f\left({\text{x}}_k\right),\delta >0,{\nu }_1\in (0,1)$ and ${\nu }_2\in (1,\infty )$
1:	Set $s_k\leftarrow x_k-x_{k-1}$ and $m_k\leftarrow \left(\nabla f\left(x_k\right)-\nabla f\left(x_{k-1}\right)\right)$
2:	Compute a such that $$\begin{gathered} {\min }_a\left\{a\in [0,1]\mid {\overline{m}}_k=a{s}_k+(1-a)m_k\right\} \\ \text{satisfies}{\nu }_1\le \frac{\Re \left(\langle s_k,{\overline{m}}_k\rangle \right)}{\langle s_k,s_k\rangle }\text{and}\frac{\langle {\overline{m}}_k,{\overline{m}}_k\rangle }{\Re \left(\langle s_k,{\overline{m}}_k\rangle \right)}\le {\nu }_2 \end{gathered}$$ (10)
3:	Compute ${\tau }_k\leftarrow \frac{\langle s_k,s_k\rangle }{\Re \left(\langle s_k,{\overline{m}}_k\rangle \right)}-\sqrt{{\left(\frac{\langle s_k,s_k\rangle }{\Re \left(\langle s_k,{\overline{m}}_k\rangle \right)}\right)}^2-\frac{\langle s_k,s_k\rangle }{\langle {\overline{m}}_k,{\overline{m}}_k\rangle }}$
4:	${\rho }_k\leftarrow \Re \left(\langle s_k-{\tau }_k{\overline{m}}_k,{\overline{m}}_k\rangle \right)$
5:	if ${\rho }_k\le \delta \Vert s_k-{\tau }_k{\overline{m}}_k\Vert \Vert {\overline{m}}_k\Vert$ then
6:	$u_k\leftarrow 0$
7:	else
8:	$u_k\leftarrow s_k-{\tau }_k{\overline{m}}_k$
9:	end if
10:	${\rho }_k^B\leftarrow {\tau }_k^2{\rho }_k+{\tau }_k{u}_k^{\text{H}}{u}_k$
11:	Return: $B_k\leftarrow {\tau }_k^{-1}{I}_N-\frac{u_k{u}_k^{\text{H}}}{{\rho }_k^{\text{B}}}$

A. Incorporating A Convex Constraint

This part extends GKSM to handle a convex constraint on $\mathbf{x}$ with a slight increase in computational cost. To ensure $\mathbf{x}\in 𝒞$, we solve the following problem for ${𝛃}_k$ instead of (7):

$${𝛃}_k=\mathrm{a}\mathrm{r}\mathrm{g}\underset{\left({\mathbf{V}}_k𝛃\right)\in 𝒞}{\mathrm{m}\mathrm{i}\mathrm{n}}{‖\left[\begin{array}{c}\mathbf{A}{\mathbf{V}}_k𝛃\\ {\overline{\mathbf{B}}}_k^{\frac{1}{2}}{\mathbf{V}}_k𝛃\end{array}\right]-\left[\begin{array}{c}\mathbf{y}\\ {\overline{\mathbf{B}}}_k^{\frac{1}{2}}{\mathbf{w}}_k\end{array}\right]‖}_2^2.$$ (11)

Letting $\mathbf{z}={\mathbf{V}}_k𝛃$ and using the fact that ${\mathbf{V}}_k^{\mathrm{H}}{\mathbf{V}}_k={\mathbf{I}}_k$, we have $𝛃={\mathbf{V}}_k^{\mathrm{H}}\mathbf{z}$. Thus we rewrite (11) as

$${\mathbf{x}}_{k+1}=\mathrm{a}\mathrm{r}\mathrm{g}\underset{\mathbf{z}\in 𝒞}{\mathrm{m}\mathrm{i}\mathrm{n}}{‖\left[\begin{array}{c}\mathbf{A}{\mathbf{V}}_k{\mathbf{V}}_k^{\mathrm{H}}\mathbf{z}\\ {\overline{\mathbf{B}}}_k^{\frac{1}{2}}{\mathbf{V}}_k{\mathbf{V}}_k^{\mathrm{H}}\mathbf{z}\end{array}\right]-\left[\begin{array}{c}\mathbf{y}\\ {\overline{\mathbf{B}}}_k^{\frac{1}{2}}{\mathbf{w}}_k\end{array}\right]‖}_2^2.$$ (12)

Since the objective function of (12) is differentiable, we simply apply the accelerated projection gradient method [63] to solve (12) efficiently. Although $\mathbf{z}$ has the same dimension as $\mathbf{x}$, solving (12) only requires simple matrix-vector multiplications since $\mathbf{A}{\mathbf{V}}_k$ and ${\overline{\mathbf{B}}}_k^{\frac{1}{2}}{\mathbf{V}}_k$ are precomputed and saved. In practice, to extend Algorithm 1 to handle the convex constraint, we only need to replace the computation of ${𝛃}_k$ at step 4 with the solution of (11) by solving (12).

B. Discussion

The dominant computations at each iteration in Algorithm 1 involve computing $\nabla f(\mathbf{x}),\mathbf{A}\mathbf{x}$ and ${\mathbf{A}}^{\mathrm{H}}\mathbf{x}$ once, and the overall computational cost per iteration is lower than that of the methods proposed in [40, 42] that require dozens of evaluations of $\mathbf{A}\mathbf{x}$ and ${\mathbf{A}}^{\mathrm{H}}\mathbf{x}$ per iteration. Apart from computational efficiency, GKSM requires additional memory, because it must store ${\mathbf{V}}_k$ and $\mathbf{A}{\mathbf{V}}_k$. Thus, GKSM may become memoryprohibitive for very large-scale problems. A practical heuristic to address this challenge is to use a restart strategy, in which we cyclically set ${\mathbf{V}}_k={\mathbf{x}}_{k+1}/‖{\mathbf{x}}_{k+1}‖$. This not only reduces the memory usage but also lowers the computational cost. Algorithmically, when a restart is triggered, this is equivalent to rerunning Algorithm 1 with ${\mathbf{x}}_{k+1}$ as the new initial value and ${\mathbf{V}}_1={\mathbf{x}}_{k+1}/‖{\mathbf{x}}_{k+1}‖$. Moreover, in practice, there is no guarantee that a column-orthogonal matrix ${\mathbf{V}}_{k+1}$ can always be constructed from ${\mathbf{r}}_k$, since $‖{\tilde{\mathbf{r}}}_k‖$ may be zero. The restart strategy typically helps to escape such a situation. However, in our experimental settings, we never found that $‖{\tilde{\mathbf{r}}}_k‖=0$. We leave the study of restart strategies to future work.

The following convergence analysis shows that GKSM is guaranteed to monotonically decrease the cost function value every iteration. Thus, it is safe to set $K=\mathtt{Max\_Iter}$ in practice. However, the convergence rate of the cost function values to a minimum remains unclear since we cannot guarantee that ${\mathbf{V}}_k$ will span the entire image space after a finite number of iterations. To better characterize the convergence rate of the cost values, we introduce step 18 in Algorithm 1. This addition allows us to explicitly quantify the cost convergence after $K$ iterations.

IV. Convergence Analysis

This section provides a rigorous convergence analysis of using Algorithm 1 to solve (2). Because the unconstrained problem is a special case of the constrained one, we focus our analysis on using GKSM for problems with constraints. We use the notation $F_{𝒞}(\mathbf{x})=F(\mathbf{x})+{\iota }_{𝒞}(\mathbf{x})$ in the following analysis, where ${\iota }_{𝒞}(\mathbf{x})$ denotes the characteristic function, defined as ${\iota }_{𝒞}(\mathbf{x})=0$ if $\mathbf{x}\in 𝒞$, and ${\iota }_{𝒞}(\mathbf{x})=+\mathrm{\infty }$ otherwise. Here, we assume that $𝒞$ is convex and that its indicator function ${\iota }_{𝒞}$ is lower semicontinuous. Before presenting our main convergence results, we first review the definition of Kurdyka–Łojasiewicz (KL) inequality and make one assumption, followed by four supporting lemmas.

Definition 1 (Kurdyka–Łojasiewicz inequality [64, 65]). Let $\chi \left(\mathrm{x}\right):{\mathbb{C}}^N\to (-\mathrm{\infty },+\mathrm{\infty }]$ be a proper, lower semicontinuous function. We say that $\chi$ satisfies the Kurdyka–Łojasiewicz (KL) inequality at a point $\overline{\mathbf{x}}\in \mathrm{d}\mathrm{o}\mathrm{m}(\partial \chi )$ if there exist a $\eta >0$, a neighborhood $𝒰$ of $\overline{\mathbf{x}}$, and a continuous concave function $\phi :[0,\eta )\to {\mathbb{R}}_{+}$ that is continuously differentiable on $(0,\eta )$ and satisfies $\phi \left(0\right)=0$ and ${\phi }^{\prime }(s)>0$ for all $s\in (0,\eta )$, such that

$${\phi }^{\prime }\left(\right|\chi (\mathbf{x})-\chi \left(\overline{\mathbf{x}}\right)\left|\right)\cdot \mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}(\mathbf{0},\partial \chi (\mathbf{x}))\ge 1.$$

holds for all $\mathbf{x}\in 𝒰\cap \left\{\mathbf{x}\in {\mathbb{C}}^N:|\chi (\mathbf{x})-\chi (\overline{\mathbf{x}}\left)\right|<\eta \right\}$. Here, $\partial \chi (\mathbf{x})$ denotes the subgradient of $\chi (\mathbf{x})$, and $\text{dist}(\cdot ,\cdot )$ denotes Euclidean distance.

In Definition 1, $\phi (s)$ is called the desingularization function. If $\phi (s)$ holds the form $\phi (s)=c{s}^{1-t}$ for $t\in [0,1)$ and $c>$ 0, then we say that $\chi (\mathbf{x})$ has the KL property at $\overline{\mathbf{x}}$ with an exponent of $t$.

Assumption 1 ($L$-Smooth $f$). Assume that $f:{\mathbb{C}}^n\to (-\mathrm{\infty },+\mathrm{\infty }]$ is a proper, lower semicontinuous, and lower bounded function. Further assume that the gradient of $f$ is L-Lipschitz continuous. That is, $\forall {\mathbf{x}}_1,{\mathbf{x}}_2\in {\mathbb{C}}^N$, there exists a $L>0$ such that the following inequality holds:

$$\Vert \nabla f\left(x_1\right)-\nabla f\left(x_2\right)\Vert \le L\Vert x_1-x_2\Vert .$$ (13)

Lemma 1 (Majorizer of $f$ [42, Lemma 1]). Let $f:{\mathbb{C}}^N\to (-\mathrm{\infty },\mathrm{\infty }]$ be an $L$-smooth function. Then for any ${\mathbf{x}}_1,{\mathbf{x}}_2\in {\mathbb{C}}^N$, we have

$$f\left(x_2\right)\le f\left(x_1\right)+\Re \left\{\nabla f\left(x_1\right),x_2-x_1\right\}+\frac{L}{2}{\Vert x_1-x_2\Vert }_2^2.$$ (14)

Lemma 2 (Bounded Hessian [42, Lemma 4]). The approximate Hessian matrices ${\mathbf{B}}_k$ generated by Algorithm 2 satisfy the following inequality

$$\underset{\_}{\eta }\mathbf{I}⪯{\mathbf{B}}_k⪯\bar{\eta }\mathbf{I},$$

where $0<\underset{\_}{\eta }<\bar{\eta }<\mathrm{\infty }$.

Lemma 3. By running Algorithm 1 for solving (2), we have the following inequality at $k$th iteration,

$$\begin{gathered} \Re \{⟨\nabla f\left(x_k\right),x_{k+1}-x_k⟩\}\le h\left(x_k\right)-h\left(x_{k+1}\right) \\ -\frac{1}{2}{\Vert x_k-x_{k+1}\Vert }_{\left(\frac{2}{{\alpha }_k}{B}_k-\underset{\_}{\eta }{I}_N\right)}^2. \end{gathered}$$ (15)

Lemma 4. Suppose the elements in ${\left\{{\varphi }_k>0\right\}}_{k\ge 1}$ satisfy

$${\varphi }_{k+1}^{2t}\le \gamma \left({\varphi }_k-{\varphi }_{k+1}\right)\mathit{and}{\varphi }_{k+1}\le {\varphi }_k,$$

where $t\in (0,1)$ and $\gamma >0$. Then we have the following upper bounds for ${\varphi }_{k+1}$,

$${\varphi }_{k+1}\le \{\begin{array}{ll}{\left(1-\frac{{\varphi }_1^{2t-1}}{\gamma +{\varphi }_1^{2t-1}}\right)}^k{\varphi }_1,& t\in \left(0,\frac{1}{2}\right]\\ {\left({\varphi }_1^{1-2t}+\frac{(2t-1){(1-\sigma )}^{2t}}{2\gamma }k\right)}_{+}^{\frac{1}{1-2t}},& t\in \left(\frac{1}{2},1\right),\end{array}$$ (16)

where $\sigma \in (0,1)$ and $(\cdot {)}_{+}=\mathrm{m}\mathrm{a}\mathrm{x}(\cdot ,0)$.

Lemmas 1 and 2 were already demonstrated in [42], so we omit their proofs here. The proofs of Lemmas 3 and 4 are provided in Appendices A and B. Lemma 4 is used to establish the convergence rates of the cost function sequence for different values of $t$ in the KL inequality when running Algorithm 1. Theorems 1 and 2 summarize our main convergence results.

Theorem 1 (Descent properties of Algorithm 1, $K\le +\mathrm{\infty }$). Let ${\alpha }_k\in \left(0,\frac{2\underset{\_}{\eta }}{\underset{\_}{\eta }+L}\right)$ and ${\mathrm{\Delta }}_k\triangleq \underset{k^{\prime }\le k}{\mathrm{m}\mathrm{i}\mathrm{n}}{‖{\mathrm{x}}_{k^{\prime }+1}-{\mathrm{x}}_{k^{\prime }}‖}_2^2$. Under Assumption 1, by running $k<K$ iterations of Algorithm 1 to solve (2), we have

• ${\mathrm{\Delta }}_k\le \frac{F\left({\mathbf{x}}_1\right)-F^{\ast }}{vk}$ and $F\left({\mathbf{x}}_{k+1}\right)\le F\left({\mathbf{x}}_k\right)$, where $v=\underset{k}{\mathrm{m}\mathrm{i}\mathrm{n}}\left\{\underset{\_}{\eta }/{\alpha }_k-(\underset{\_}{\eta }+L)/2\right\},F^{\ast }$ denotes the minimum of (2), and ${\mathbf{x}}_1$ is the initial iterate.

• $‖{\mathbf{x}}_{k+1}-{\mathbf{x}}_k‖\to 0$ as $k\to \mathrm{\infty }$.

Theorem 2 (Convergence rates, $K<+\mathrm{\infty }$). Let ${\alpha }_k\in \left(0,\frac{2\underset{\_}{\eta }}{\underset{\_}{\eta }+L}\right)$ and $ℬ\left({\mathrm{x}}^{\ast },\mathrm{\Lambda }\right)=\left\{\mathrm{x}\in {\mathbb{C}}^N\mid ‖\mathrm{x}-{\mathrm{x}}^{\ast }‖\le \mathrm{\Lambda }\right\}$. Under Assumption 1, by running Algorithm 1 $k>K$ iterations to solve (2), we have

• $‖{\mathbf{x}}_{k+1}-{\mathbf{x}}_k‖\to 0$ as $k\to \mathrm{\infty }$ and all cluster points of the sequence ${\left\{{\mathbf{x}}_k\right\}}_{k>K}$ are critical points of (2).

• Assume ${\mathbf{x}}_k$ converges to ${\mathbf{x}}^{\ast }$ and $F_{𝒞}$ satisfies the $KL$ inequality. There exists $\mathrm{\Lambda }>0$ such that $F_{𝒞}$ at $\overline{\mathrm{x}}={\mathrm{x}}^{\ast }$ in a neighborhood of $𝒰$ containing $ℬ\left({\mathrm{x}}^{\ast },\mathrm{\Lambda }\right)$. Then there also exists $K^{\prime }>K$, such that, ${\mathbf{x}}_k\in ℬ\left({\mathbf{x}}^{\ast },\mathrm{\Lambda }\right)$ and $\left|F_{𝒞}\left({\mathbf{x}}_k\right)-F^{\ast }\right|<\eta$ for all $k\ge K^{\prime }$. For $k\ge K^{\prime }$, we have the following convergence rates of the cost function values for $t\in [0,1)$:

  1. $F\left({\mathbf{x}}_{k+1}\right)-F^{\ast }\le {\left(F_{K^{\prime }}-\frac{1}{\gamma }\left(k-K^{\prime }+1\right)\right)}_{+}$, $t=0$,
  2. $F\left({\mathbf{x}}_{k+1}\right)-F^{\ast }\le {\left(1-\frac{F_{K^{\prime }}^{2t-1}}{F_{K^{\prime }}^{2t-1}+\gamma }\right)}^{k-K^{\prime }+1}{F}_{K^{\prime }}$, $t\in \left(0,\frac{1}{2}\right]$,
  3. $F\left({\mathbf{x}}_{k+1}\right)-F^{\ast }\le {\left(F_{K^{\prime }}^{1-2t}+q\left(k-K^{\prime }+1\right)\right)}_{+}^{\frac{1}{1-2t}}$, $t\in \left(\frac{1}{2},1\right)$.

  where $F_{K^{\prime }}=F\left({\mathbf{x}}_{K^{\prime }}\right)-F^{\ast },\sigma \in (0,1),\gamma ={\mathrm{m}\mathrm{a}\mathrm{x}}_k({[c(L{\alpha }_k+\bar{\eta })]}^2)/(v{(1-t)}^2{\alpha }_k^2)$, and $q=(t-1/2){(1-\sigma )}^{2t}/\gamma$.

Theorem 1 states that GKSM leads to $‖{\mathbf{x}}_{k+1}-{\mathbf{x}}_k‖\to 0$ as $k\to \mathrm{\infty }$ for any $K$, either finite or infinite. Our first result in Theorem 2 establishes all cluster points of the sequence ${\left\{{\mathbf{x}}_k\right\}}_{k>K}$ are critical points of (2) for finite $K$. The second result in Theorem 2 provides convergence rates of the cost values for $t\in [0,1)$ after $K^{\prime }>K$ iterations. During the algorithm’s execution, all iterates remain in $𝒞$, so that $F_{𝒞}\left({\mathrm{x}}_k\right)=F\left({\mathrm{x}}_k\right)$ for all $k$. For ease of notation, we use $F(\mathbf{x})$ instead of $F_{𝒞}(\mathbf{x})$ to express the convergence rates in Theorem 2. Note that if $F\left({\mathrm{x}}_{k+1}\right)=F^{\ast }$, the left hand side is identically zero and therefore the convergence rate bounds in Theorem 2 hold trivially.

Note that Algorithm 1 reduces to CQNPM [42] when $K$ is finite and $k>K$. Theorem 2 then extends the theoretical analysis in [42] to a more general class of functions $F_C$. Specifically, our analysis relies on the KL inequality, which is weaker than the Polyak-Łojasiewicz (PL) inequality² used in [42]. Section V-C empirically studies the convergence behavior of Algorithm 1 to validate our theoretical analysis.

V. Numerical Experiments

This section studies the performance of GKSM for CS MRI reconstruction with spiral and radial sampling trajectories. Note that the experimental and algorithmic settings used here are similar to those in our previous work [42]. For convenience, we briefly re-describe them in this paper. Then, we present the reconstruction results and study the convergence behavior of GKSM empirically.

Experimental Settings:

The performance of GKSM are evaluated on both brain and knee MRI datasets. For the brain images, we used the dataset from [18], which contains 360 training and 164 test images. For the knee images, we adopted the multi-coil knee dataset from the NYU fastMRI [44]. The ESPIRiT algorithm [66] was used to obtain the complex-valued images from the raw k-space data. All images were then resized to a uniform image size of $256\times 256$ and normalized such that the maximum magnitude was one. The network architecture proposed in [39] with the addition of bias terms is used to construct $f(\mathbf{x})$, see Fig. 1. The number of layers is set to six. Note that other neural network architectures can also be used to train the gradient-driven denoiser. The only requirement is that the nonlinear activation functions be differentiable. For example, Hurault et al. [40] used DRUNet by replacing the ReLU activations with ELU to construct f(x). The noisy images were generated by adding i.i.d. Gaussian noise with variance 1/255 to the clean images. The network was trained using the mean squared error loss. We employed the ADAM optimizer [67] with an initial learning rate of 10−3, which was halved every 4, 000 iterations. Training was performed for a total of 18, 000 iterations with a batch size of 64. Although we trained separate denoisers for the brain and knee datasets, different sampling trajectories used the same denoiser.

Fig. 1.

The neural network architecture used to construct the energy function $f_{𝛉}(\mathbf{x})$ is based on [39]. The convolutional kernels have size $3\times 3$ with a stride of one.

To assess reconstruction quality, we selected six brain and knee test images as ground truth. Fig. 2 displays the magnitudes of these images. For spiral acquisition, we used six interleaves with 1688 readout points and 32 coils. Radial acquisition employed 55 spokes with golden-angle rotation, 1024 readout points, and 32 coils. Fig. 3 describes these sampling trajectories. To simulate k-space data, we applied the forward model to the ground-truth images and then added the complex i.i.d. Gaussian noise (zero mean, variance 10⁻⁴), resulting in an input SNR of approximately 21 dB. In the reconstruction, we employed coil compression [68] to reduce the number of coils from 32 to 20 virtual coils, thereby lowering the computational cost. All experiments used simulated k-space data and were implemented in PyTorch and ran on an NVIDIA A100 GPU.

Fig. 2.

The magnitude of the six brain and knee complex-valued ground truth images.

Fig. 3.

The spiral (a) and radial (b) sampling trajectories.

Algorithmic Settings:

The previous work [10] already showed that the accelerated proximal gradient method (dubbed APG) [69] is faster than the projected gradient descent method [39] and the proximal gradient method [40] for addressing (2). Thus, we mainly compared GKSM with CQNPM [10] and APG in this paper. To test the applicability of GKSM for a constrained problem, we used the constraint set $𝒞=\left\{\mathbf{x}\mid \Vert \mathbf{x}{\Vert }_{\mathrm{\infty }}\le 1\right\}$ in all competing methods. However, in practical CS MRI reconstruction, we generally do not impose such a constraint. For plots involving $F^{\ast }$, we ran APG for 500 iterations and defined $F^{\ast }=F\left({\mathbf{x}}_{500}\right)-\epsilon$ for a small constant $\epsilon >0$. Unless otherwise specified, we set $K=\mathtt{Max\_Iter}$ in the following experiments. Algorithm 2 used $\delta ={10}^{-8}$, ${\nu }_1=2\times {10}^{-6}$, and ${\nu }_2=200$.

A. Spiral Acquisition Reconstruction

Fig. 4 summarizes the cost and PSNR values versus the number of iterations and wall time for each method on the brain 1 image. Fig. 4.s (a) and (c show CQNPM was the fastest algorithm in terms of the number of iterations. GKSM achieved similar results to CQNPM after enough iterations. Fig. 4s (b) and (d) report the cost and PSNR values versus wall time, where GKSM was the fastest algorithm in terms of wall time. In multi-coil CS MRI reconstruction with non-Cartesian sampling, computing $\mathbf{A}\mathbf{x}$ and ${\mathbf{A}}^{\mathrm{H}}\mathbf{x}$ typically dominate the computational cost. CQNPM (respectively, APG) requires solving a weighted proximal mapping (respectively, a proximal mapping) at each iteration, which involves multiple evaluations of $\mathbf{A}\mathbf{x}$ and ${\mathbf{A}}^{\mathrm{H}}\mathbf{x}$. In contrast, GKSM requires only a single evaluation of $\mathbf{A}\mathbf{x}$ and ${\mathbf{A}}^{\mathrm{H}}\mathbf{x}$ per iteration, which significantly reduces computational cost while maintaining relatively fast convergence in terms of the number of iterations.

Fig. 4.

Comparison of different methods with spiral acquisition on the brain 1 image for $\epsilon =5\times {10}^{-3}$. (a), (b): cost values versus iteration and wall time; (c), (d): PSNR values versus iteration and wall time.

Fig. 5 presents the reconstructed images and the corresponding error maps at the 50th and 100th iterations of each method. GKSM eventually achieved a reconstruction quality similar to that of CQNPM with same number of iterations, but with significantly less wall time. Table I summarizes the PSNR performance on the rest of five brain images within 150 iterations. Clearly, we observed that GKSM was approximately $7\times$ faster than CQNPM in terms of wall time required to exceed the performance of APG within 150 iterations. Moreover, GKSM achieves nearly the same performance as CQNPM at the 150th iteration, while requiring approximately $9\times$ less time, illustrating the superior performance of our method. The supplementary material includes additional results on the knee images with spiral acquisition, as well as the structural similarity index measure (SSIM) metrics, which exhibit similar behavior.

Fig. 5.

First row: the reconstructed brain 1 images of each method at 50th and 100th iterations with spiral acquisition. The PSNR (respectively, SSIM) values are labeled at the left (respectively, right) bottom corner of each image. Second row: the associated error maps (8×) of the reconstructed images.

TABLE I.

PSNR performance of each method for reconstructing five additional brain test images with spiral acquisition. For APG, we report the maximum PSNR (within 150 iterations), the corresponding number of iterations, and wall time. For other methods, we report the earliest iteration count that exceeds the APG PSNR (along with its PSNR and wall time) in the first row. The PSNR and wall time at the 150th iteration were summarized in the second row. Bold indicates the shortest wall time at which the PSNR of APG was exceeded. The PSNR values and wall time at the 150th iteration of CQNPM and GKSM are marked with an underline. The blue digits denote the shortest wall time at the 150th iteration.

Methods╲Index	2			3			4			5			6
	PSNR↑	iter.↓	sec.↓	PSNR↑	iter.↓	sec.↓	PSNR↑	iter.↓	sec.↓	PSNR↑	iter.↓	sec.↓	PSNR↑	iter.↓	sec.↓
APG	42.1	150	86.3	42.8	150	86.4	43.3	150	87.1	42.0	150	86.4	40.5	150	88.7
CQNPM	42.1	29	16.9	42.8	30	19.7	43.3	31	19.1	42.1	30	17.6	40.5	29	17.6
	44.2	150	82.0	44.7	150	70.1	45.0	150	65.5	44.0	150	68.9	42.7	150	77.0
GKSM	42.2	49	2.4	42.9	50	2.4	43.3	50	2.4	42.1	51	2.5	40.5	49	2.3
	44.2	150	8.4	44.7	150	8.2	45.1	150	8.2	43.9	150	8.3	42.5	150	8.2

B. Radial Acquisition Reconstruction

Fig. 6 presents the cost and PSNR values versus the number of iterations and wall time of each method on the knee 1 image with radial acquisition. Fig. 6s (a) and (c) show CQNPM converged faster than APG in terms of the number of iterations. In this experimental setting, we found that CQNPM made faster progress than GKSM in the early iterations. Then GKSM exceeded CQNPM in the later iterations. This observation is slightly different from Fig. 4. Although all methods have convergence guarantees under the same assumptions, we cannot guarantee that their iterates follow the same trajectory. One possible explanation is that using a subspace in GKSM may sometimes act as an additional constraint, guiding the iterates along a more favorable path toward a minimizer. The detailed study of this direction is beyond the scope of this paper and is left for future work. Fig. 6s (b) and (d) display the cost and PSNR values of each method versus wall time. Evidently, GKSM converged faster than others in terms of wall time, which is consistent with the previous observation.

Fig. 7 reports the reconstructed images of each method at 50th and 100th iterations. From this experiment, we observed that GKSM demonstrated the best visual quality among all methods. Table II reports the PSNR and wall time performance of each method on the remaining five knee test images. We ran APG for 100 iterations and then compared how many iterations were required by the other methods to exceed the PSNR value achieved by APG. Table II also reports the PSNR values and wall time of CQNPM and GKSM at 100 iterations. Consistently, we observed similar beharior as in Table I. The supplementary material includes additional results on the brain images with radial acquisition, as well as the SSIM metrics that show similar trends.

C. Effect of K and Convergence Validation

This part empirically studies the effect of $K$ and the convergence behavior of GKSM using the brain 1 image and spiral acquisition settings as in Fig. 4. Fig. 8 reports the PSNR values versus the number of iterations and wall time for GKSM with varying $K$ and CQNPM. We observed that different values of $K$ converged to similar PSNR values as CQNPM. Fig. 8(b) presents the PSNR values versus wall time, where we observed that larger values of $K$ led to faster convergence compared with smaller ones. This observation is consistent with our earlier results, as GKSM avoids solving a weighted proximal mapping for iterations $k\le K$.

We have now empirically validated our theoretical analysis. Fig. 9 presents the cost values and the values of ${\mathrm{\Delta }}_k/{\mathrm{\Delta }}_1$ for GKSM with spiral acquisition on six brain test images. As expected, the cost values converged to a constant across all test images, and ${\mathrm{\Delta }}_k/{\mathrm{\Delta }}_1\to 0$, consistent with our theoretical analysis.

VI. Conclusion

A well-established theoretical foundation is especially important for ensuring reliability in medical imaging applications. Compared with the PnP and RED frameworks, gradient-driven denoisers offer a significantly stronger theoretical foundation. In particular, the only required assumptions are the differentiability of f and the Lipschitz continuity of ∇f, which are easier to satisfy in practice. To efficiently solve the associate nonconvex minimization problem, we developed a generalized Krylov subspace method with convergence guarantees in nonconvex settings. Numerical experiments on multi-coil CS-MRI reconstruction with non-Cartesian sampling trajectories demonstrate that the proposed method can recover images within seconds on a GPU platform. This significantly improves the efficiency of solving the associated optimization problem and enhances the practical applicability of gradient-driven denoisers.

Appendix A

Proof of Lemma 3

Since ${\mathbf{B}}_k\succ 0$ (cf. Lemma 2), we know the objective function in (6) is $\underset{\_}{\eta }$-strongly convex. By combining with the fact that ${\mathbf{V}}_k^{\mathrm{H}}{\mathbf{V}}_k={\mathbf{I}}_k$ and the $\underset{\_}{\eta }$-strongly convex inequality, we have the following inequality at kth iteration for $\forall \mathbf{x}={\mathbf{V}}_k𝛃,\mathbf{x}\in 𝒞$:

$$\begin{gathered} \Re \{V_k^{\text{H}}[\nabla h\left(x_{k+1}\right)+\nabla \overline{f}\left(x_{k+1},x_k,B_k,{\alpha }_k\right) \\ +\frac{\eta }{2}\left(V_k\beta -x_{k+1}\right)],\beta -{\beta }_k\}\ge 0. \end{gathered}$$ (17)

TABLE II.

PSNR performance of each method for reconstructing five additional knee test images with radial acquisition. For APG, we report the maximum PSNR (within 100 iterations), the corresponding number of iterations, and wall time. For other methods, we report the earliest iteration that exceeds the APG PSNR (along with its PSNR and wall time) in the first row. The PSNR and wall time at the 100th iteration were summarized in the second row. Bold indicates the shortest wall time at which the PSNR of APG was exceeded. The PSNR values and wall time at the 150th iteration of CQNPM and GKSM are marked with an underline. The blue digits denote the shortest wall time at the 100th iteration.

Methods╲Index	2			3			4			5			6
	PSNR↑	iter.↓	sec.↓	PSNR↑	iter.↓	sec.↓	PSNR↑	iter.↓	sec.↓	PSNR↑	iter.↓	sec.↓	PSNR↑	iter.↓	sec.↓
APG	42.6	100	61.0	41.4	100	61.3	44.6	100	60.7	42.1	100	63.6	44.2	100	64.7
CQNPM	42.7	26	15.8	41.4	30	19.1	44.7	25	16.4	42.2	28	17.0	44.2	25	16.3
	44.1	100	59.9	41.9	100	64.3	45.5	100	65.4	44.1	100	61.9	44.9	100	66.3
GKSM	42.7	41	2.0	41.4	33	1.6	44.7	38	1.8	42.3	36	1.8	44.2	39	1.9
	44.1	100	5.6	43.3	100	5.5	45.5	100	5.5	44.4	100	5.7	44.9	100	5.5

Fig. 6.

Comparison of different methods with radial acquisition on the knee 1 image for $\epsilon =6\times {10}^{-3}$. (a), (b): cost values versus iteration and wall time; (c), (d): PSNR values versus iteration and wall time.

Letting $𝛃=\left[\begin{array}{c}{𝛃}_{k-1}\\ 0\end{array}\right]$, we rewrite (17) as

$$\begin{gathered} \Re \{[\nabla h\left(x_{k+1}\right)+\frac{1}{{\alpha }_k}{B}_k\left(x_{k+1}-x_k\right)+\nabla f\left(x_k\right) \\ +\frac{\eta }{2}\left(x_k-x_{k+1}\right)],x_k-x_{k+1}\}\ge 0. \end{gathered}$$ (18)

Note that if $‖{\tilde{\mathbf{r}}}_k‖=0$, we choose $𝛃={𝛃}_{k-1}$ and (18) is still held. By reorganizing (18) and using the convexity of $h(\mathbf{x})\left(h\left({\mathbf{x}}_k\right)\ge h\left({\mathbf{x}}_{k+1}\right)+\Re \left\{⟨\nabla h\left({\mathbf{x}}_{k+1}\right),{\mathbf{x}}_k-{\mathbf{x}}_{k+1}⟩\right\}\right)$, we get the desired result

$$\begin{gathered} \Re \left\{\nabla f\left(x_k\right),x_{k+1}-x_k\right\}\le h\left(x_k\right)-h\left(x_{k+1}\right) \\ -\frac{1}{2}{\Vert x_k-x_{k+1}\Vert }_{\left(\frac{2}{{\alpha }_k}{B}_k-\underset{\_}{\eta }{I}_N\right)}^2. \end{gathered}$$ (19)

Appendix B

Proof of Lemma 4

Our goal is to derive an upper bound for ${\varphi }_{k+1}$ by using the facts that ${\varphi }_k-{\varphi }_{k+1}\ge {\varphi }_{k+1}^{2t}/\gamma$ and $0<{\varphi }_{k+1}\le {\varphi }_k$. Rewrite ${\varphi }_k-{\varphi }_{k+1}\ge {\varphi }_{k+1}^{2t}/\gamma$ as ${\varphi }_k\ge {\varphi }_{k+1}\left(1+\frac{1}{\gamma }{\varphi }_{k+1}^{2t-1}\right)$. Considering $t\in (0,1/2)$, we know ${\varphi }_{k+1}^{2t-1}$ is monotonically decreasing since $2t-1<0$. So we have ${\varphi }_{k+1}^{2t-1}\ge {\varphi }_1^{2t-1}$, which implies ${\varphi }_k\ge {\varphi }_{k+1}\left(1+\frac{1}{\gamma }{\varphi }_1^{2t-1}\right)$, yielding ${\varphi }_{k+1}\le {\left(1-\frac{{\varphi }_1^{2t-1}}{\gamma +{\varphi }_1^{2t-1}}\right)}^k{\varphi }_1$. If $t=\frac{1}{2}$, we have ${\varphi }_{k+1}\le \gamma \left({\varphi }_k-{\varphi }_{k+1}\right)$, which yields ${\varphi }_{k+1}\le \frac{\gamma }{1+\gamma }{\varphi }_k$. Therefore, we can establish the desired result immediately: ${\varphi }_{k+1}\le {\left(1-\frac{1}{1+\gamma }\right)}^k{\varphi }_1$.

Denote by $\psi \left(x\right)=x^{1-2t}$, where $x>0$. Let $\bar{t}=2t-1$. Using the mean value theorem, we have

$$\psi \left({\varphi }_{k+1}\right)-\psi \left({\varphi }_k\right)=-\bar{t}{\bar{\varphi }}_k^{-\bar{t}-1}\left({\varphi }_{k+1}-{\varphi }_k\right),$$ (20)

with ${\varphi }_{k+1}\le {\bar{\varphi }}_k\le {\varphi }_k$. Since ${\bar{\varphi }}_k^{-\bar{t}-1}$ is monotonically decreasing and ${\varphi }_k-{\varphi }_{k+1}\ge {\varphi }_{k+1}^{2t}/\gamma$, we can get the following inequalities from (20) for $t\in (1/2,1)$

$$\begin{gathered} \psi \left({\varphi }_{k+1}\right)-\psi \left({\varphi }_k\right)\ge \overline{t}{\varphi }_k^{-2t}\left({\varphi }_k-{\varphi }_{k+1}\right) \\ \ge {\varphi }_k^{-2t}{\varphi }_{k+1}^{2t}\frac{\overline{t}}{\gamma }. \end{gathered}$$ (21)

Since $0<{\varphi }_{k+1}\le {\varphi }_k$, we have ${\varphi }_{k+1}/{\varphi }_k\le 1$. Suppose we run $k$ iterations. For any $\sigma \in (0,1)$, we can split the whole iterate indices into two subsets ${ℐ}_1$ and ${ℐ}_2$ such that ${ℐ}_1=\left\{k^{\prime }\mid \right.\left.{\varphi }_{k^{\prime }+1}/{\varphi }_{k^{\prime }}\le 1-\sigma \right\}$ and ${ℐ}_2=\left\{k^{\prime }\mid {\varphi }_{k^{\prime }+1}/{\varphi }_{k^{\prime }}>1-\sigma \right\}$. So, we know that either $\left|{ℐ}_1\right|\ge k/2$ or $\left|{ℐ}_2\right|\ge k/2$.

If $\left|{ℐ}_1\right|\ge k/2$, we get

$${\varphi }_{k+1}\le (1-\sigma {)}^{k/2}{\varphi }_1.$$ (22)

Next, we consider $\left|{ℐ}_2\right|\ge k/2$. By summing up (21) from 1 to $k$, we reach

$$\psi \left({\varphi }_{k+1}\right)\ge \psi \left({\varphi }_1\right)+\frac{(2t-1)(1-\sigma {)}^{2t}}{2\gamma }k.$$

Using the definition of $\psi (\cdot )$ and the fact that $1-2t<0$, we derive

$${\varphi }_{k+1}\le {\left({\varphi }_1^{1-2t}+\frac{(2t-1)(1-\sigma {)}^{2t}}{2\gamma }k\right)}_{+}^{\frac{1}{1-2t}}.$$ (23)

If $k$ is large enough, the bound in (22) is smaller than that in (23), yielding the desired result.

Fig. 7.

First row: the reconstructed knee 1 images of each method at 50th and 100th iterations with radial acquisition. The PSNR (respectively, SSIM) values are labeled at the left (respectively, right) bottom corner of each image. Second row: the associated error maps (8×) of the reconstructed images.

Fig. 8.

Comparison of varying K with spiral acquisition on the brain 1 image. (a), (b): PSNR values versus iteration and wall time.

Fig. 9.

(a) Averaged cost values (a) and ${\mathrm{\Delta }}_k/{\mathrm{\Delta }}_1$ (b) versus iteration for GKSM. The shaded region of each curve represents the range of the cost values and ${\mathrm{\Delta }}_k$ across six brain test images with spiral acquisition.

Appendix C

By using Lemma 1, we have the following inequalities

$$\begin{gathered} f\left(x_{k+1}\right)\le f\left(x_k\right)+\frac{L}{2}{\Vert x_{k+1}-x_k\Vert }_2^2 \\ +\Re \left\{\nabla f\left(x_k\right),x_{k+1}-x_k\right\} \\ \le f\left(x_k\right)+h\left(x_k\right)-h\left(x_{k+1}\right) \\ -\frac{1}{2}{\Vert x_k-x_{k+1}\Vert }_{\left(2B_k/{\alpha }_k-(\underset{\_}{\eta }+L)I_N\right)}^2. \end{gathered}$$ (24)

The second inequality comes from Lemma 3. Reorganizing (24), we get

$$\frac{1}{2}{‖{\mathbf{x}}_k-{\mathbf{x}}_{k+1}‖}_{(2{\mathbf{B}}_k/{\alpha }_k-(\underset{\_}{\eta }+L){\mathbf{I}}_N)}^2\le F\left({\mathbf{x}}_k\right)-F\left({\mathbf{x}}_{k+1}\right).$$

Letting ${\alpha }_k<\frac{2\underset{\_}{\eta }}{\underset{\_}{\eta }+L},v={\mathrm{m}\mathrm{i}\mathrm{n}}_k\{\underset{\_}{\eta }/{\alpha }_{k^{\prime }}-(\underset{\_}{\eta }+L)/2\}$, and using Lemma 2, we reach

$$v{‖{\mathbf{x}}_k-{\mathbf{x}}_{k+1}‖}_2^2\le F\left({\mathbf{x}}_k\right)-F\left({\mathbf{x}}_{k+1}\right).$$ (25)

Since $v>0$, we have $F\left({\mathbf{x}}_{k+1}\right)\le F\left({\mathbf{x}}_k\right)$. Summing up (25) from $k^{\prime }=1$ to $k$, we get

$$\begin{gathered} \sum _{k^{\prime }=1}^{k}v{\Vert x_{k^{\prime }}-x_{k^{\prime }+1}\Vert }_2^2\le F\left(x_1\right)-F\left(x_{K+1}\right) \\ \le F\left(x_1\right)-F^{\ast }, \end{gathered}$$ (26)

where $F^{\ast }$ denotes the minimal value of $F(\mathbf{x})$. Letting ${\mathrm{\Delta }}_k={\mathrm{m}\mathrm{i}\mathrm{n}}_{k^{\prime }\le k}\left\{{‖{\mathbf{x}}_{k^{\prime }}-{\mathbf{x}}_{k^{\prime }+1}‖}_2^2\right\}$, we get the desired result

$${\mathrm{\Delta }}_k\le \frac{F\left({\mathbf{x}}_1\right)-F^{\ast }}{vk}.$$ (27)

Let $k\to \mathrm{\infty }$, we get ${\mathrm{\Delta }}_k\to 0$. Together with the summation in (26), we obtain $‖{\mathrm{x}}_{k+1}-{\mathrm{x}}_k‖\to 0$ as $k\to \mathrm{\infty }$.

Appendix D

Proof of Theorem 2

For $k>K$, we have ${\mathbf{V}}_k={\mathbf{I}}_N$, so ${\mathbf{V}}_k^H{\mathbf{V}}_k={\mathbf{I}}_N$ still holds. Therefore, (26) and (27) remain valid for $k>K$. Consequently, we still have $‖{\mathrm{x}}_{k+1}-{\mathrm{x}}_k‖\to 0$ as $k\to \mathrm{\infty }$. Next, we prove that all cluster points of the sequence ${\left\{{\mathbf{x}}_k\right\}}_{k>K}$ are critical points of (2).

Let $G(𝛃)={‖{\overline{\mathbf{A}}}_k𝛃-{\overline{\mathbf{y}}}_k‖}_2^2$ denote the cost function of (11) with ${\overline{\mathbf{A}}}_k=\left[\begin{array}{c}\mathbf{A}\\ {\overline{\mathbf{B}}}_k^{\frac{1}{2}}\end{array}\right]$ and ${\overline{\mathbf{y}}}_k=\left[\begin{array}{c}\mathbf{y}\\ {\overline{\mathbf{B}}}_k^{\frac{1}{2}}{\mathbf{w}}_k\end{array}\right]$. Then we rewrite (11) as an unconstrained problem, i.e.,

$${𝛃}_k=\mathrm{a}\mathrm{r}\mathrm{g}\underset{𝛃}{\mathrm{m}\mathrm{i}\mathrm{n}}G(𝛃)+{\iota }_{𝒞}(𝛃).$$ (28)

From the first-order optimality condition of (28) and using the fact that ${\mathbf{x}}_{k+1}={𝛃}_k$, we have

$$\mathbf{0}\in \nabla h\left({\mathbf{x}}_{k+1}\right)+\partial {\iota }_{𝒞}\left({\mathbf{x}}_{k+1}\right)+\nabla f\left({\mathbf{x}}_k\right)+\frac{1}{{\alpha }_k}{\mathbf{B}}_k\left({\mathbf{x}}_{k+1}-{\mathbf{x}}_k\right)$$

which implies

$$\begin{gathered} \nabla f\left(x_{k+1}\right)-\nabla f\left(x_k\right) \\ +\frac{1}{{\alpha }_k}{B}_k\left(x_k-x_{k+1}\right)\in \nabla h\left(x_{k+1}\right)+\nabla f\left(x_{k+1}\right) \\ +\partial {\iota }_{𝒞}\left(x_{k+1}\right). \end{gathered}$$ (29)

Here, we use the definitions of $\bar{f}\left(\mathbf{x},{\mathbf{x}}_k,{\mathbf{B}}_k,{\alpha }_k\right)$ and $h(\mathbf{x})$.

Note that $F_{𝒞}(\mathbf{x})=F(\mathbf{x})+{\iota }_{𝒞}(\mathbf{x})$ with $F(\mathbf{x})=h(\mathbf{x})+f(\mathbf{x})$. By using (29), we have

$$\begin{gathered} \text{dist}\left(\mathbf{0},\partial F_{𝒞}\left(x_{k+1}\right)\right)\le \Vert \nabla f\left(x_{k+1}\right)-\nabla f\left(x_k\right) \\ +\frac{1}{{\alpha }_k}{B}_k\left(x_k-x_{k+1}\right)\Vert , \\ \le \Vert \nabla f\left(x_{k+1}\right)-\nabla f\left(x_k\right)\Vert \\ +\frac{\overline{\eta }}{{\alpha }_k}\Vert x_k-x_{k+1}\Vert \\ \le \frac{L{\alpha }_k+\overline{\eta }}{{\alpha }_k}\Vert x_k-x_{k+1}\Vert . \end{gathered}$$ (30)

Notice that $‖{\mathbf{x}}_k-{\mathbf{x}}_{k+1}‖\to 0$ for $k\to \mathrm{\infty }$ and that $\frac{L{\alpha }_k+\bar{\eta }}{{\alpha }_k}$ remains finite. So we have $\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\left(0,\partial F_{𝒞}\left({\mathbf{x}}_{k+1}\right)\right)\to 0$ for $k\to \mathrm{\infty }$, which implies that all cluster points of ${\left\{{\mathbf{x}}_k\right\}}_{k>K}$ are critical points of (2). This completes the proof of the first term.

Since $F_{𝒞}$ satisfies the KL inequality and ${\mathbf{x}}_k$ is converging to ${\mathbf{x}}^{\ast }$, there exist $K^{\prime }>K$ and $\mathrm{\Lambda }>0$ such that, for all $k\ge K^{\prime }$, we have ${\mathrm{x}}_k\in ℬ\left({\mathrm{x}}^{\ast },\mathrm{\Lambda }\right)$, where $ℬ\left({\mathrm{x}}^{\ast },\mathrm{\Lambda }\right)=\left\{\mathrm{x}\in {\mathbb{C}}^N\mid \Vert \mathrm{x}-\right.\left.{\mathbf{x}}^{\ast }\Vert \le \mathrm{\Lambda }\right\}$, and $\left|F_C\left({\mathbf{x}}_k\right)-F^{\ast }\right|<\eta$. By letting $\overline{\mathbf{x}}={\mathbf{x}}^{\ast }$, $\phi (s)=c{s}^{1-t}$, and the assumption $ℬ\left({\mathbf{x}}^{\ast },\mathrm{\Lambda }\right)\subseteq 𝒰$, we get

$$\begin{gathered} {\left(F_{𝒞}\left(x_{k+1}\right)-F^{\ast }\right)}^{2t}\le c^2{\left(1-t\right)}^2\text{dist}{\left(\mathbf{0},\partial F_{𝒞}\left(x_{k+1}\right)\right)}^2 \\ \stackrel{\left(30\right)}{\le }\frac{{\left[c\left(L{\alpha }_k+\overline{\eta }\right)\right]}^2}{{(1-t)}^2{\alpha }_k^2}{\Vert x_k-x_{k+1}\Vert }_2^2 \\ \stackrel{\left(25\right)}{\le }\frac{{\left[c\left(L{\alpha }_k+\overline{\eta }\right)\right]}^2}{v{(1-t)}^2{\alpha }_k^2}\left(F\left(x_k\right)-F\left(x_{k+1}\right)\right). \end{gathered}$$ (31)

Note that during the algorithm’s progress, all iterates remain in $𝒞$, so that $F_{𝒞}(\mathbf{x})=F(\mathbf{x})$. For simplicity, we write $F(\mathbf{x})$ instead of $F_{𝒞}(\mathbf{x})$ in what follows. Denote by $\gamma =\left.{\mathrm{m}\mathrm{a}\mathrm{x}}_k\left({\left[c\left(L{\alpha }_k+\bar{\eta }\right)\right]}^2\right)/(v{(1-t)}^2{\alpha }_k^2)\right)$. For $t=0$, we have

$$F\left({\mathbf{x}}_{k+1}\right)-F^{\ast }\le F\left({\mathbf{x}}_k\right)-F^{\ast }-\frac{1}{\gamma },$$

resulting in

$$F\left({\mathbf{x}}_{k+1}\right)-F^{\ast }\le {\left(F\left({\mathbf{x}}_{K^{\prime }}\right)-F^{\ast }-\frac{1}{\gamma }\left(k-K^{\prime }+1\right)\right)}_{+}.$$

By using Lemma 4, we get the desired results for $t\in (0,1)$.