Reducing the Complexity of Model-Based MRI Reconstructions via Sparsification

Abstract

Model-based reconstruction methods have emerged as a powerful alternative to classical Fourier-based MRI techniques, largely because of their ability to explicitly model (and therefore, potentially overcome) moderate field inhomogeneities, streamline reconstruction from non-Cartesian sampling, and even allow for the use of custom designed non-Fourier encoding methods. Their application in such scenarios, however, often comes with a substantial increase in computational cost, owing to the fact that the corresponding forward model in such settings no longer possesses a direct Fourier Transform based implementation. This paper introduces an algorithmic framework designed to reduce the computational burden associated with model-based MRI reconstruction tasks. The key innovation is the strategic sparsification of the corresponding forward operators for these models, giving rise to approximations of the forward models (and their adjoints) that admit low computational complexity application. This enables overall a reduced computational complexity application of popular iterative first-order reconstruction methods for these reconstruction tasks. Computational results obtained on both synthetic and experimental data illustrate the viability and efficiency of the approach.

I. Introduction

Standard magnetic resonance imaging (MRI) formulations require highly uniform magnetic fields and linear field gradients in order to allow a Fourier interpretation of the forward model and received signal. This has obvious benefits by allowing the reconstruction process to be as simple as applying a discrete Fourier Transform (DFT). However, recent efforts in the community have deviated from these requirements in order to enable a wide array of new techniques, including parallel imaging [1], [2], novel motion correction [3], and non-standard hardware [4], [5]. The trade-off for these techniques is that the reconstruction process requires using “model-based” methods—and usually involves solving a large-scale (possibly constrained or regularized) inverse problem [4]–[10]. Particularly when a high resolution or 3D reconstruction is desired, this may be too computationally demanding without finding some additional method for making the problem tractable (with a particular emphasis on the memory issues of storing a full forward model).

The need for model-based reconstruction can be particularly profound when the MRI acquisition method uses a frequency-swept pulse that imprints a spatially- and temporally-dependent quadratic phase on the MRI signals. Such phase variation can be advantageous due to the imparted resilience to magnetic field imperfections, and can help avoid some image artifacts (e.g., [11]–[14]). Conversely, under some conditions, the quadratic phase can lead to signal-to-noise (SNR) loss and blurring when simple DFT image reconstruction is used. Related work on the rationale for model-based MRI can be found in [15].

The approach proposed in this paper is relatively general; we expect it to enable model-based reconstructions in situations with experimental imperfections such as inhomogeneous fields and (modelable) hardware limitations. Despite this generality, the focus of this paper will be on spatiotemporal encoding (SPEN) with frequency-swept pulses. SPEN has many advantages over conventional Fourier encoding, notably its robustness to field inhomogeneities, which would otherwise severely degrade a Fourier encoded image. However, in its simplest form, SPEN has a significantly lower signal-to-noise ratio (SNR) when compared to Fourier encoding [6], [16], [17], as Fourier encoding has a signal multiplexing advantage. As shown in [6], reconstructing images using a model-based approach, as advocated for here, remedies the inferior SNR typically associated with SPEN, in addition to restoring the loss in resolution originating from using SPEN as opposed to Fourier techniques.

This paper proposes a novel method for enabling a computationally tractable solution of regularized linear inverse problems of the type found in MRI applications. In particular, the method is well-adapted to the application where the forward model is described by the Bloch equations and is constructed by numerically solving the corresponding differential equations. In this regime we propose approximating the forward model by a “sparsified” operator, which has both a compact representation in memory and a fast method of applying it (both for it and its adjoint)—enabling the use of so-called first-order optimization methods which only utilize the values and gradients of the function to be optimized.

Compressed sensing based works (e.g., [18] and numerous others) utilize sparsity as a kind of prior on the signal to be reconstructed. These methods succeed when the signal of interest is sparse (either in the spatial domain, in its gradient, or in a transformed domain), and the appropriate constraints or regularizers are utilized in the reconstruction process to steer the solution toward a correspondingly sparse candidate. In contrast, while we also utilize structured signal priors here, our main contribution comes in the form of sparse approximation of an imaging system’s forward operator, motivated by a desire for computational efficiency in non-Fourier imaging scenarios. In this sense, our effort here is related to other recent works that utilize sparsity in the design of task-specific excitations for MRI. In [19], Zelinski et al. consider a shimming application where they aim to overcome B₁ inhomogeneity by sparse selection of slice selective pulses in excitation k-space. For certain convex linear inverse problems, the systematic reduction of the dimension of the forward operator (and thus, reduction of the overall problem complexity) via premultiplication by a random structured matrix has also been explored in the broader literature on sparse approximation; see, e.g., [20].

This work is also in line with previous work considering computationally tractable ways to reconstruct images in MRI in the absence of a direct Fourier model. In this context, for example, gridding can be a viable way to reconstruct when measuring Fourier coefficients off-grid. More closely related, Fessler et al. [21] took advantage of Toeplitz-structured matrices with a conjugate gradient method in order to correct for inhomogeneous magnetic fields with a significant reduction in computation time. Regarding more advanced reconstruction approaches, a very interesting recent work [22] considers computationally efficient methods for quantitative imaging in MRI. That work requires the solution to very large-scale non-linear optimization problems in a distributed setting and is designed for high performance computing environments.

The current work also extends our own previous conference paper [23], which provides a summary of the methods outlined here, with some preliminary experimental validations.

A. Notation

We write $ℝ$ and $ℂ$ for the real and complex fields respectively. Lower-case boldface notation is used for vectors (e.g., x) and upper-case boldface notation for matrices and vector-valued functions (e.g., $\mathbf{A}\in {ℂ}^{m\times n}$ and $\mathbf{M}(x,y,z,t)\in {ℂ}^3$). Unbolded symbols are reserved for scalars or scalar-valued functions. An asterisk (^*) refers to the conjugate transpose operation on vectors and matrices. The symbol ∥·∥_F denotes the matrix Frobenius norm, ∥·∥_p the vector p norm (or its induced matrix norm if the argument is a matrix), and, via an abuse of notation ∥·∥₀ will refer to the number of non-zero entries of a matrix or vector. Entries of a matrix or vector will be denoted using subscripts (e.g., x_i, A_ij) while iterations in an algorithm will be rendered as superscripts in parentheses, e.g., x^(k).

B. Limitation of Fourier Encoding in Spatiotemporal Methods

In this section, to serve as a general motivation for model-based reconstruction methods, we demonstrate via simulation that standard Fourier encoding is not an appropriate model when using a frequency-swept pulse under certain (arguably, natural) conditions.

In the presence of a field gradient, the frequency sweep excites spins sequentially in time along the direction of the simultaneously imposed gradient. When the gradient polarity is reversed for acquisition, the spins excited first are locally refocused last, and vice versa. Early parts of the time-domain signal predominantly originate from spins excited last, and late parts of the signal originate from spins excited earlier. To demonstrate the difference between encoding following frequency-swept excitation and conventional amplitude-modulated excitation, an encoding matrix for a one dimensional imaging experiment was generated by modeling the spin dynamics during a hyperbolic secant (HS) pulse [24] in the presence of a gradient and recording the spin evolution during a frequency-encoded acquisition. The real component of this matrix and a corresponding trace through its respective Fourier reconstruction are shown in Fig.1. When using an amplitude-modulated pulse for excitation, the encoding matrix reduces to the DFT matrix; whereas for a frequency-swept pulse, the encoding matrix is dependent on the excitation and acquisition parameters. As the acquisition resolution drops below the time-bandwidth product of the pulse, the quadratic phase variation across the spins becomes sufficiently large that signal dropout occurs. This can be seen in Fig. 1 by the skewed encoding matrix and the distorted Fourier reconstruction.

Fig. 1.

Encoding matrix produced by a frequency-swept pulse, displayed at various resolutions. Real part of the DFT operator (a), along with the real part of the encoding matrix at various resolutions (b-d). For b) - d), a HS1 selective excitation was used, which excited less than the full Field of View (FOV), corresponding to the horizontal direction in the image. The time-bandwidth of the pulse was 64, while the grid sizes simulated on were b) 128, c) 64, and d) 32. The pulse duration was 1 ms and the FOV was 20 cm. A trace corresponding to the standard Fourier reconstruction is plotted above each corresponding resolution, showing the effect of the quadratic phase on the standard Fourier reconstruction.

Modeling these differences can allow a model-based reconstruction to more faithfully represent the underlying physics and recover information that would normally be lost by standard Fourier techniques. In Fig. 2, a frequency-swept pulse was used to excite a disc in a uniform phantom and then reconstructed using both standard Fourier reconstruction and a model-based reconstruction. As the model is simulated using the Bloch equations, it contains information regarding spatially varying flip angle and quadratic phase resulting from the applied 2D radiofrequency (RF) pulse, whereas the Fourier reconstruction does not. In the excited region, the model-based reconstruction is significantly flatter, showing that it is not constrained, as the Fourier reconstruction is, by the spatially varying flip angle and phase.

Fig. 2.

A comparison of reconstructions of a uniform phantom resulting from Fourier reconstruction and from a model-based method following 2D frequency-swept excitation of a disc. The pulse used was a 2D HS1 [25] with time-bandwidth product of 7, 17.4 ms duration, and sampled on an 18 turn spiral. In each case the first row represents the magnitude of the image while the second row represents the phase. The first column (a,d) shows one time point of the encoding matrix corresponding to end of the pulse, the second column (b,e) shows the standard Fourier reconstruction using convolutional gridding [26] onto a Cartesian grid, while the third column (c,f) shows the magnitude and phase of the unregularized model-based reconstruction. The vertical lines and graphs to the right of the magnitudes show the profile of a slice through the middle of the image. The model incorporates spatially-varying flip angle and phase into the forward operator, so the model reconstructs an apparently wider disc accordingly without spatially varying phase.

II. Methods

Let $\mathbf{M}(x,y,z,t)\triangleq M_i(x,y,z,t)\mathbf{i}+M_j(x,y,z,t)\mathbf{j}+M_k(x,y,z,t)\mathbf{k}$ denote the time-varying net nuclear magnetization vector in a region of interest. Assume that we have as an initial condition that M is aligned in the k direction: M(x, y, z, 0) = M₀(x, y, z)k. In the presence of an external, time-varying magnetic field $\mathbf{B}\triangleq B_i(x,y,z,t)\mathbf{i}+B_j(x,y,z,t)\mathbf{j}+B_k(x,y,z,t)\mathbf{k}$, the time evolution of M(x, y, z, t) is given by the Bloch equations

$$\frac{d\mathbf{M}}{dt}=\gamma \mathbf{M}\times \mathbf{B}-\frac{M_i\mathbf{i}+M_j\mathbf{j}}{T_2}-\frac{\left(M_k-M_0\right)\mathbf{k}}{T_1},$$ (1)

where γ is the gyromagnetic ratio, $T_1\triangleq T_1(x,y,z)$ is the longitudinal relaxation time constant and $T_2\triangleq T_2(x,y,z)$ is the transverse relaxation time constant.

By Faraday’s law, the signal s measured at time t with a constant system-dependent gain Λ is given by

$$s(t)=-\Lambda {\int }_V\frac{d{M}_{xy}}{dt}dV.$$ (2)

Here, the transverse magnetization is defined by M_xy = M_x+ iM_y while integrating over the imaging volume V. Generally, the time-dependence of M_xy is dominated by Larmor precession, i.e., $M_{xy}(x,y,z;t)\approx M_{xy}(x,y,z;t)\exp \left\{-i{\omega }_{0}t\right\}$, so the following substitution is made:

$$\frac{d{M}_{xy}}{dt}\to -i{\omega }_0{M}_{xy}.$$ (3)

The time-derivatives of other time-dependent terms are negligibly small compared to Larmor precession and are ignored in subsequent analysis. Setting the constant gain term iω₀Λ to unity, the signal is then written as

$$\begin{gathered} s(t)={\int }_V{M}_{xy}(x,y,z;t)dV, \\ ={\int }_V\frac{M_{xy}(x,y,z;t)}{M_0(x,y,z)}{M}_0(x,y,z)dV. \end{gathered}$$ (4)

In (4), the magnetization may be represented as the product of a normalized transverse magnetization vector and the underlying image M₀(x, y, z). If unknown, relaxation terms may be safely absorbed into the image, presuming the data acquisition window is short relative to T₂ and the longitudinal magnetization, M_z, is in a steady-state. These conditions are satisfied in many commonly used MRI techniques, with one exception being echo-planar imaging (EPI) [27] (in EPI, the transverse magnetization decays significantly during data acquisition). Other examples include inversion recovery sequences with rapid acquisition (such as MPRAGE [28]), where the longitudinal magnetization is not in a steady-state during the rapid gradient echo acquisition. Additionally, during fast spin echo sequences, the magnetization is not in a steady-state between acquisitions due to the application of one or more refocusing pulses. However, in all these cases, direct Fourier reconstruction with relaxation neglected does not cause substantial image artifacts. Hence, even in these cases, ignoring relaxation is not anticipated to cause significant issues in the present application. Known relaxation effects could be incorporated into the framework herein, although this is beyond the scope of this work. Thus, a forward operator is constructed by simulation of a uniform object via numerical integration of the Bloch equations, absent relaxation. The magnetization M is then represented at each point in space as the product of the simulated, normalized magnetization and the target image. Discretizing the signal equation (4) results in a linear model: the signal measured at time t is proportional to the inner product of a vector representing the normalized net magnetization at time t of a uniform object multiplied by the underlying image.

To fix notation going forward, consider the problem of reconstructing an image x with n voxels from m measurements b = [b₁, · · · , b_m]^T given the nominal data model Ax = b. Each row of A corresponds to the collection of net magnetizations at each spatial location, at a particular instant in time (subsequent rows of A correspond to subsequent time instants).

A. First-order methods for regularized linear inverse problems

In this section methods for solving regularized linear inverse problems in either Lagrangian

$$\underset{\mathbf{x}}{\mathrm{minimize}}\Vert \mathbf{A}x-\mathbf{b}{\Vert }_2^2+\lambda \mathrm{\Omega }(\mathbf{x}),$$ (5)

or constrained

$$\begin{array}{ll}\underset{\mathbf{x}}{\mathrm{minimize}}\hfill & \Vert \mathbf{A}x-\mathbf{b}{\Vert }_2^2\hfill \\ \text{subject}\text{to}\hfill & \Omega (\mathbf{x})\le \tau ,\hfill \end{array}$$ (6)

forms are discussed. Here, Ω(·) is a regularization function, which we assume to be convex but not necessarily differentiable. In practice, we are interested in the large-scale regime m, n ≫ 0. In fact, we are most interested in the regime where even storing A in memory is expensive, making matrix-vector operations involving A prohibitive. For example, note that even for a 2D, moderately high-resolution image of size 256 × 256, a fully determined (square) encoding matrix A will contain more than four billion entries and require more than 34GB of memory just to store in double precision. Closed-form solutions of the optimization problems above do not exist for most regularizers (with Tikhonov regularization, $\mathrm{\Omega }(\mathbf{x})\triangleq \Vert x{\Vert }_2^2$, a notable exception) which restricts our attention to iterative schemes. Higher-order iterative methods (e.g., Newton’s method, L-BFGS [29], et al.) are too expensive from both a computational and a memory perspective to be feasible for problems of this size on consumer hardware.

In light of this, only first-order algorithms—those that only require knowledge of the gradient of the objective function and not any higher-order derivatives—will be considered. For example, one standard approach to this problem for general non-smooth convex functions f is to use a proximal algorithm [30]. Given a closed proper convex function f, the scaled proximal operator with scale α > 0 is

$${\mathrm{prox}}_{\alpha f}(\mathbf{u})\triangleq \underset{\mathbf{x}}{\mathrm{argmin}}\left[f(\mathbf{x})+\frac{1}{2\alpha }\Vert \mathbf{x}-\mathbf{u}{\Vert }_2^2\right].$$ (7)

The proximal operator provides a balance, determined by the scale α, between minimizing the underlying convex function f and not moving too far away from the “starting point” u. It generalizes the projection operator onto a convex set when f is the indicator function of a convex set C. Setting f = Ω as in (5), proximal algorithms have update steps of the form

$${\mathbf{x}}^{(k+1)}={\mathrm{prox}}_{\lambda \mathrm{\Omega }/\eta }\left({\mathbf{x}}^{(k)}-\frac{1}{\eta }\mathbf{A}*\left(\mathbf{A}{\mathbf{x}}^{(k)}-\mathbf{b}\right)\right).$$ (8)

Notice that the argument of prox is simply gradient descent for step-size $\frac{1}{\eta }$. Each iteration requires a matrix-vector multiplication involving A and A^* and an application of the proximal operator. For an interesting class of functions Ω(x) that we will call prox-capable [31] (i.e., functions for which ${\mathrm{prox}}_{\lambda \mathrm{\Omega }/\eta }(\cdot )$ can be computed quickly), the cost of such algorithms is dominated by the cost of applying A and A^*.

In the remainder of this section we briefly discuss alternative approaches for solving (5) or (6) in high-dimensional settings, for general matrices A, providing additional context on why these methods may not be ideally suited to our specific problem.

Stochastic Gradient Descent:

Given the problem of minimizing a cost function f₀(x) which can be written as a sum of (sub)differentiable functions $f_0(x)={\sum }_{i=1}^n{f}_i(x)$, stochastic gradient descent is an iterative method with each iteration given by (a) choosing a random i ∈ {1, . . . , n} and then (b) updating our current iterate x^(k) using gradient descent on the ith function f_i(x):

$${\mathbf{x}}^{(k+1)}={\mathbf{x}}^{(k)}-\gamma \nabla f_i\left({\mathbf{x}}^{(k)}\right).$$ (9)

A natural setting would be to use a projected or proximal variant of stochastic gradient descent by decomposing the data fidelity term as

$$\Vert \mathbf{A}x-\mathbf{b}{\Vert }_2^2=\sum _{i=1}^n\underset{f_i(\mathbf{x})}{\underbrace{{\left(\sum _{j=1}^n{\mathbf{A}}_{ij}{\mathbf{x}}_j-{\mathbf{b}}_j\right)}^2}}.$$ (10)

Using this formalism, each row of A could be computed one at a time via Bloch simulation, operated with, and then discarded or stored. However this is still unnatural when solving (6) when A is generated by numerical integration of a differential equation. When A is the encoding matrix for the Bloch equations, then the rows of A are necessarily generated in sequence, so selecting indices randomly is an unnatural fit.

This problem could be mitigated by iterating over A cyclically (instead of stochastically). However, the convergence theory for (6) in this domain is much trickier (see, e.g., [32]) and, in general, significantly weaker. Furthermore, even in the stochastic case the slower convergence of SGD-type methods matters more when the regularizer requires solving a (comparatively expensive) proximal sub-problem.

Coordinate descent:

If Ω(x) is separable in the coordinates of $\mathbf{x}:\mathrm{\Omega }(\mathbf{x})={\sum }_{i=1}^n{\mathrm{\Omega }}_i\left({\mathbf{x}}_i\right)$ (e.g., in the case $\mathrm{\Omega }(\mathbf{x})=\Vert \mathbf{x}{\Vert }_2^2$ then we could write $f_0(\mathbf{x})={\sum }_{i=1}^n{\left({\mathbf{A}}_i-{\mathbf{x}}_i\right)}^2+\lambda {\mathrm{\Omega }}_i\left({\mathbf{x}}_i\right)$ and optimize for each coordinate in parallel (see, e.g., [33]). This has the advantage of only requiring one column of A to be held in memory at a given time. However, requiring separability of Ω is an extremely strong restriction. Furthermore, even in the case that Ω is (at least partially) separable, the total memory requirements would only be partially mitigated (by spreading the operator A across multiple machines instead of just one). Alternatively, the solution to the Bloch equations could be re-computed many times. We defer investigations along these lines to a subsequent effort.

B. Fast approximate first-order methods via operator approximation

In lieu of directly solving the inverse problem above, we propose to instead approximate the forward operator A using another operator $\tilde{\mathbf{A}}$, which can be used to solve the inverse problem efficiently.

Let C denote the computational complexity of applying a linear transformation T to a vector. Some transformations T that have fast implementations include the FFT (computational complexity C ~ n log n) or the discrete wavelet transforms (computational cost of C ~ n in certain cases). We propose to approximate A with the operator $\tilde{\mathbf{A}}\triangleq \mathbf{S}\circ \mathbf{T}$ where T is has low implementation complexity, S is a sparse operator, and ◦ denotes composition. In such cases, the cost of applying $\tilde{\mathbf{A}}$ to a vector x is ∥S∥₀C, which can potentially be significantly smaller than mn if the matrix S is sparse.

In fact, this is applicable in the spatiotemporal MRI context with T being the Fourier transform $\mathcal{F}$. In Fig. 3 for example, a 2D HS1 pulse [14], [25] was used for excitation and spatiotemporal encoding; a single snapshot of the net magnetization obtained by Bloch simulation is shown. It is evident that the encoding matrix, which was used in real experiments, has many nonzero entries but the Fourier transform of that matrix appears quite sparse. A full description of the experimental parameters is given in Section III-D.

Fig. 3.

Demonstration of the sparsity of the encoding matrix during readout following excitation with the 2D HS1 pulse described by Jang et al. [25]. Shown is a single snapshot in time, approximately 1/8 of the way into the readout period. The first row is in physical space, while the bottom row is in the Fourier domain. Note the simplicity and sparsity of the encoding matrix represented in the Fourier domain in the bottom row. Although only one point in time is shown, the same level of sparsity persists throughout the readout period.

Further note that with this approach, the sparse matrix S can be created by hard thresholding to remove all values below a certain level. In practice, the threshold is determined by choosing how much approximation error one is willing to tolerate in A ≈ S ◦ T. This error is controlled by setting the threshold to retain the largest entries that maintain a certain percentage of the energy, and can be computed by sorting the net magnetizations corresponding to each time step of the encoding matrix. Note this process can be performed one row at a time and can be interleaved with the original solution to the Bloch equations. Thus, this process incurs an additional computational cost of one application of T^* (cost C) and one sort (cost ∼ n log n) for each of the m rows of the forward operator A. At most one row of the full matrix A ever has to be stored in its “uncompressed” form.

This method can be extended to non-orthogonal matrices T, but at the cost of a more complicated process of finding the sparse matrix S. Doing this for an arbitrary linear operator T necessitates solving a sparse regression problem for each row of the forward operator (see the discussion in Section IV for details). To the extent that a single pulse with the same set of parameters can be re-used many times, this may be an acceptable up-front cost to be amortized over many uses.

C. Theoretical Accuracy

A natural question arises as to the accuracy of the approximation method we propose; e.g., whether solving the constrained problem (6) using the approximation $\tilde{\mathbf{A}}\approx \mathbf{A}$ will result in a solution that is in some sense close to the true solution. This is quantified via the analytical result below.

Define the (non-squared) objective function regarded as a function of the input matrix A by

$$F_{\mathbf{A}}(\mathbf{x})\triangleq \Vert \mathbf{A}x-\mathbf{b}{\Vert }_2.$$ (11)

Assume that the approximation is ϵ close to the original matrix in the following relative sense:

$$\Vert \mathbf{A}-\tilde{\mathbf{A}}{\Vert }_F<ϵ\Vert \mathbf{A}{\Vert }_F.$$ (12)

Then the following result provides a guarantee that the optimal value of the minimization problem in Equation (6) with A is close to the corresponding problem with $\tilde{\mathbf{A}}$.

Theorem 2.1:

Let Ω be a proper convex function and let $\tilde{\mathbf{A}}$ be ϵ close to A as in (12). Define

$$F_{\mathbf{A}}(\mathbf{x})\triangleq \Vert \mathbf{A}x-\mathbf{b}{\Vert }_2,$$ (13)

$$F_{\tilde{\mathbf{A}}}(\text{x})\triangleq \Vert \tilde{\mathbf{A}}\mathbf{x}-\mathbf{b}{\Vert }_2,$$ (14)

$${\mathbf{x}}_{\mathbf{A}}\triangleq \underset{\{\mathbf{x}:\mathrm{\Omega }(\mathbf{x})\le \tau \}}{\mathrm{argmin}}{F}_{\mathbf{A}}(\mathbf{x}),$$ (15)

and

$${\mathbf{x}}_{\tilde{\mathbf{A}}}\triangleq \underset{\{\mathbf{x}:\mathrm{\Omega }(\mathbf{x})\le \tau \}}{\mathrm{argmin}}{F}_{\tilde{\mathbf{A}}}(\mathbf{x}).$$ (16)

Let $M\triangleq \max \left\{{\Vert x_{\tilde{\mathbf{A}}}\Vert }_2,{\Vert x_A\Vert }_2\right\}$. Then

$$\left|F_{\tilde{A}}\left(x_{\mathbf{A}}\right)-F_A\left(x_{\tilde{A}}\right)\right|\le \frac{ϵ}{\sqrt{2}}\Vert \mathbf{A}{\Vert }_{F}M.$$ (17)

In other words, the difference in the optimal objective value when minimizing (13) as opposed to (14) can be bounded by a term that grows only linearly in the approximation fidelity ϵ, so that close approximations of A incur correspondingly close reconstructions as quantified by the objective values. Note that a bound of this form (showing the closeness in the objective values) is likely the strongest that can be expected without further conditions on the underlying problem, as the functions F_A and $F_{\tilde{A}}$ are not strictly convex when A or $\tilde{\mathbf{A}}$, respectively do not have full column rank. The proof of Theorem 2.1 can be found in Appendix A.

In the next section this theorem is discussed in the context of numerical simulations and in vivo experiments.

III. Results

In this section the utility of the proposed method is demonstrated using both simulated and in vivo data, and compared with standard image reconstruction methods in MRI. In experiments we will consider the Lagrangian form of the optimization problem (5), where λ > 0 is the regularization parameter, for a variety of different (prox-capable) regularizers Ω. Regularization has been used for different applications in MRI [34]–[36]. In this case, we require some form of regularization because the data are often undersampled, and so there is no unique solution to the unregularized objective. The regularization is also useful for promoting prior knowledge of the object to be imaged—for example, Total Variation regularization is often applied to natural images [37].

A. Simulation

A Bloch simulator written in-house was used to construct forward operators via simulation, which represent different types of data acquisition patterns, including both after and during excitation (for simultaneous transmit and receive, STAR [38]). The input parameters to the simulation need to match those used in experiments for the simulated forward operator to accurately represent the true forward operator. The parameters used in this study are described in Section III-D. These forward operators were of size n × n × t, where n was the resolution of the image and t was the number of time steps in the acquisition. At the largest, these forward operators reached 256×256×14500 in dimension, requiring more than 15GB of memory to store in double precision. We consider several scenarios including simulated experiments (where the data were generated by applying the full forward operator to phantom images) as well as real data. Fast (sparsified) approximations were created using the Fast Fourier Transform $(\mathbf{T}=\mathcal{F})$ and time step-wise processing of the forward operator to retain a specified fraction of the energy of the corresponding row of the unsparsified operator, resulting in $\mathbf{A}\approx \mathbf{S}\circ \mathcal{F}$. In our reconstructions we utilize the Fast Iterative Shrinkage and Thresholding (FISTA) Algorithm [39] to solve the regularized least squares problem, and consider at times three different regularizers: Tikhonov, Total Variation [37], and Wavelet Sparsity (see e.g., [40]).

We first validate in simulations (on synthetic data) that the given approximation (in conjunction with the regularized inversion scheme) works well in practice. The simulation used a 2D HS1 pulse [14], [25] sampled on a 64-turn spiral which excited a 25.6 cm disc at full-width half maximum. The time-bandwidth product of the 2D pulse was 85.3, the duration was 58 ms, and the k-space trajectory sampled to 3 mm resolution. The time-reversed excitation gradients were used for an echo to achieve spatiotemporal encoding. The simulation grid size was 256 × 256 with a 25.6 × 25.6 cm field-of-view (FOV). Additionally, 10 × 10 intra-voxel averaging was used to more reliably model intra-voxel dephasing effects. (Later we use the same pulse for simulations of simultaneous transmit and receive [38] encoding.) The results in Fig. 4 demonstrate the promise of this strategy. In this figure, we compare the difference between using the true encoding matrix and the sparsified version of the encoding matrix (retaining 99.99% of the energy of the original operator) in reconstructions and show that they do not lead to visibly different reconstructions.

Fig. 4.

Reconstruction results for two imaging paradigms; conventional spiral echo readout (top) and simultaneous transmit and receive, STAR (bottom). Shown at the far left are the ground truth image (top, 1 mm resolution) and the effective region of excitation region (bottom) for both paradigms. The remaining columns from left-to-right show: TV-regularized inverse results using full encoding matrices, TV-regularized inverse results with sparsified encoding matrices, and relative difference images (absolute image difference, normalized by the Frobenius norm of the full inverse solution), all sampled at 3 mm resolution. In both settings the sparsified reconstructions are visually very similar to the “brute force” reconstructions, though the location(s) of the errors vary.

B. Simultaneous Transmit and Receive

In order to show a more complete example in which Fourier-based techniques would not be possible, we also include (in Fig. 4) another experiment on the same image with measurements simulated under a STAR acquisition [38]. In this acquisition, a 2D HS1 pulse [14], [25] was used with the same parameters as in the preceding section. Although this experiment yielded slightly reduced data fidelity as compared to that using an echo readout, it was still possible to perform relatively accurate reconstructions using an approximate encoding matrix that took up only approximately 13% of the space of the original.

C. Speedup

We next quantify the effectiveness of the speedup using three different regularizers for (real, acquired) in vivo data. For this experiment a 2D HS1 pulse [14], [25] designed for zoom imaging was utilized for excitation, with a single-shot spiral readout. A 2D pulse was used with restricted readout FOV to increase spatial resolution given the single-shot readout. This experiment gave rise to a 192 × 192 × 32768 sized encoding matrix, reshaped to be of the form $\mathbf{A}\in {ℂ}^{32768\times 36864}$. In this case, we observed that the Fourier transform of A was relatively sparse and more than 99.9% of the energy was contained in only 1.3% of the entries. By keeping only those entries, we formed an approximation as described in Section II and solved the corresponding inverse problem. Fig. 5 compares the reconstructions obtained by using this approximation with reconstructions obtained using the original encoding matrix. For each of the three regularizers, there was more than an order of magnitude speedup compared to reconstructing with the full dense matrix, and the relative difference of the reconstructions was fairly negligible. The apparent blurring of the images is likely due to the B₀ inhomogeneity sensitivity of long duration spiral acquisitions, which was not addressed in this work.

Fig. 5.

Images reconstructed from experimentally acquired data using the full forward model (top row) along with the images reconstructed using the sparsified approximation (middle row) for four different regularization strategies. The sparsification resulted in a forward operator that had 1.2% nonzeros in the transform domain. The speedup factors were computed as ratios of the time taken for the full reconstruction relative to the sparsified one for each regularization approach are shown. The bottom row shows the absolute value of the difference image, normalized by the Frobenius norm of the corresponding “unsparsified” reconstruction, with a corresponding numerical scale. The data were acquired with a selective 2D RF pulse and single-shot spiral acquisition, with a restricted FOV to allow higher spatial resolution readout. The TV and wavelet regularized reconstructions appear to show some more image detail at the periphery of the excited FOV relative to the other recovery methods. Within the FOV of interest, the difference images all show a similar spiral pattern, which is seemingly due to sparsification of the forward operator.

In Fig. 6 we compare the reconstruction time and reconstruction error that arises when performing Tikhonov regularized reconstruction with a fixed choice of regularization parameter using the (transform domain) sparsified forward operators of varying sparsity levels, applied to a synthetic image cropped from the abdomen phantom available at http://www.imp.uni-erlangen.de/phantoms/.

Fig. 6.

Relative error, relative reconstruction time, and an image feature quality metric, all as a function of the forward operator approximation quality, for the same operator as used in Fig. 5. The solid red line shows the relative time taken to solve the optimization problem (5) with the approximate operator Ae where the time taken to solve (5) with A is normalized to be one. The dashed blue line shows the relative reconstruction error ${\Vert x_A-{\text{x}}_{\tilde{\mathbf{A}}}\Vert }_F/{\Vert x_A\Vert }_F$. Both correspond to the logarithmic scale on the left. The dash-dot green line, corresponding to the linear scale at the right, shows the computed full-width at half maximum (FWHM) of a narrow image feature for each reconstruction, relative to the width of that same feature as reconstructed using the “non-sparsified” forward operator (see Fig. 7 for a visual depiction of the specific image feature). As this figure shows, there is a region in which the solution can be solved in approximately 2% of the time with ∼1% relative error; see Fig. 7(c).

We can see that the error decreases and the reconstruction time increases as the effective (transform domain) sparsity of the forward operator increases; in some settings a reconstruction with relative error on the order of 10⁻³ can be achieved while still only taking about 2% of the time it takes to reconstruct the image using the full forward operator (with no sparsification). Further, for this example, our sparsification does not result in any degradation in the quality of reconstruction of a narrow feature in the image. Indeed, some additional detail on this experiment is provided in Fig. 7. There, the ground truth image is depicted with a highlighted (in red) vertical trace showing the axis along which the feature thickness giving rise to the green dash-dot curve in Fig. 6 is assessed. The subsequent three panels show reconstructions at varying levels of (transform domain) sparsity; see figure caption for details.

Fig. 7.

Ground truth and reconstructed images in a synthetic example. Panel (a) shows the cropped region of the abdomen phantom image available at http://www.imp.uni-erlangen.de/phantoms/ used here as “ground truth,” along with a vertical line along which the reconstructed feature thickness is to be assessed. We are using selective excitation here, so the recovered image will itself be the center portion of this image. Panel (b-c): reconstructions using operators whose transform domain representations are only 0.003% and 1.3% nonzeros, respectively. Panel (d): reconstruction from an “unsparsified” operator whose transform domain representation is 53.4% nonzeros. In each reconstruction, the narrow feature is faithfully recovered with some magnitude variations in the highly sparsified case; hence, the flatness of the green dash-dot curve in Fig. 6. The results show that extreme sparsification can introduce substantial reconstruction artifacts, though there is a usable regime where sparsified reconstructions are visually indistinguishable from the unsparsified one, at reconstruction times that are orders of magnitude less (than the unsparsified case).

D. Experiments using Spatiotemporal Encoding

All experiments were performed with a Varian DirectDrive console (Agilent Technologies, Santa Clara, CA) interfaced with a 4T, 90-cm magnet (Oxford Magnet Technology, Oxfordshire, UK) and a clinical gradient system (model SC72, Siemens, Erlangen, Germany) with a single channel transmit, single channel receive RF coil. A protocol approved by our Institutional Review Board (IRB) was followed for human brain imaging of healthy volunteers after obtaining written, informed consent.

Spatiotemporal encoding was performed using the STEREO sequence [8]. In STEREO, image encoding is performed using frequency-swept pulses in the presence of a time-varying field gradient. Specifically, an HS8 pulse [41] was applied in the presence of sin/cos modulated gradient waveforms to excite spins along a spiral trajectory in space. To encode the entire 2D image, 128 spirals were interleaved. Slice selection was performed in the third dimension using two HS refocusing pulses operating with sufficient RF power to be in the adiabatic regime. Regarding sequence parameters, the maximum excitation radius was 10 cm, the pulse duration was 6 ms, and the time-bandwidth product of the pulse was 256. The first spiral was repeated 3 times prior to data collection to ensure the magnetization was in a steady-state. The acquisition bandwidth was 89.285 kHz and 600 complex points were collected per spiral at readout, where the receiver was gated on at the beginning of the ramp-up of the readout gradients. The repetition time, TR, was 2 sec and the echo time, TE, was 46 ms. For simulations, 10 × 10 intra-voxel averaging was performed at each time point during the acquisition, and the field-of-view (FOV) was 20 cm.The simulation FOV was set to twice the maximum excitation radius to ensure that all excited spins were simulated within the FOV. The peak amplitude of the excitation pulse in the simulation was set to 3.5 μT, which was determined by comparison of the transmit power to a power calibration scan.

To reconstruct images from the experimentally acquired signals, the STEREO sequence with the parameters described above was simulated on a 2D grid of spins at three different grid sizes: 64 × 64, 96 × 96, and 128 × 128, as shown in Fig. 8. Three different voxel sizes were used in the Bloch simulation to generate the encoding matrix, since there is not yet an analytic method to understand the encoded spatial resolution of STEREO. Reconstructions were performed using both the full and sparse models with 95% of the model energy preserved, with sparsity levels noted in Fig. 8. The reconstructions at these various resolutions exhibit varying image artifacts, which also do not have a theoretical underpinning. However, the spiral-like artifacts in Fig. 8 clearly increase in severity as the voxel size decreases. The artifacts are not due to sparsification of the encoding matrix, as the artifacts are visually identical using the full encoding matrix. To the best of the authors’ knowledge, these are the highest quality STEREO images in the literature.

Fig. 8.

Reconstructions of experimentally acquired spatiotemporally-encoded data from the STEREO sequence reconstructed onto three different grid sizes: 3mm, 2mm, and 1.5mm pixel widths. The first row is the reconstruction with the full model, while the second row is the reconstruction with the sparse model containing 95% energy. The red box in the third column indicates the zoomed inset in the fourth column. The ratio of nonzero entries to total model size for the 3mm, 2mm, and 1.5mm models are 0.132, 0.089, and 0.056, respectively. The presence of imaging artifacts becomes increasingly evident as the resolution increases, and are independent of the sparse approximation. Detailed anatomic structures and artifacts are challenging to see for the larger pixel sizes in zoomed insets, and hence are not included here.

An additional experiment was performed using a 2D HS1 pulse [14], [25], where excitation and acquisition k-space were both sampled on a 28.5 turn spiral trajectory. The pulse duration was 26 ms, with a maximum k-space radius of 1/(3mm). The repetition time was TR = 2 s, TE = 38 ms for the center of k-space, and the flip angle was 45°. The flip angle was set to less than 90° as the 2D pulse performance degrades at high flip angles. The number of data points collected along the spiral was 6500. For comparison, a Cartesian spin-echo acquisition was performed with the same TR, TE, and flip angle as the spatiotemporal acquisition. The acquisition bandwidth was 250 kHz. This sequence was simulated with an in-house Bloch simulator on a 64 × 64 grid with FOV = 20 cm.

In Fig. 9 we compare our reconstruction results using the 2D pulse described above with the standard convolutional gridding algorithm [26]. The difference between the two reconstruction methods is fairly significant. While our method takes into account the full encoding matrix using the Bloch equations and treats each data point gathered as a linear combination of the points given by the Bloch simulation, gridding instead treats each point as a single sample corresponding to an impulse and interpolates to a uniform grid before taking the Fourier transform to reconstruct the image. The relative success of gridding on this dataset at full-sampling densities is a function of the fact that the encoding matrix is sparse and localized in the Fourier domain. On the other hand, the success of the model-based reconstruction in the undersampled regime shows that the forward model is accurate, as no regularization was used to better visualize the effects of using the sparse approximation.

Fig. 9.

Comparison between gridding and model-based reconstruction for a single-shot experiment as the number of measurements decreases. Down-sampling was achieved by averaging sequential data points along the spiral acquisition. Averaging of data was performed to improve the SNR of the data, as would be the case if using a lower acquisition bandwidth. All images, except the first column, were reconstructed onto a 64 × 64 matrix. The first column is a reference image with standard Cartesian acquisition, where the red box indicates the zoomed FOV of the spiral acquisition following excitation by a 2D RF pulse. The first row is a zoomed inset of the red box in the reference image, reconstructed by gridding. The second row shows the result of the full model-based reconstruction with no regularization using different amounts of data downsampling. The third row shows the reconstruction with no regularization using the sparse model. The original number of data points acquired along the spiral trajectory was 6500. To facilitate a fair comparison all reconstruction parameters are kept constant between different reconstructions.

IV. Discussion

This work proposes a new method of doing model-based reconstructions which will allow for significantly more computationally efficient reconstructions while admitting negligible error. As model-based reconstructions become more popular in practice, the need to perform the reconstructions in a computationally efficient way will also grow. This work is best viewed as a technology for enabling high-resolution - reconstructions under computational constraints, and we expect it to be especially useful in the cases of spatially non-linear magnetic fields [4], [5], [9], [10] and frequency-swept pulses when used with spatiotemporal encoding [6] or with zero echo time [7], [8], [42]. Non-linear fields arise, for example, in the practice of using compact magnets for MRI, where the imaging volume is comparable to the magnet size, as opposed to significantly smaller than the magnet.

The approach taken here can be extended in several interesting and useful directions. Although only Fourier-based sparsity is presented in experiments here (i.e., the operator $\mathbf{T}=\mathcal{F}$, the Fourier Transform) it is immediately straightforward to allow certain other fast operators, including the discrete wavelet transform. The same idea can be further extended to multiple bases. That is, given k fast transforms T₁, . . . , T_k, we can attempt to approximate the forward operator A as

$$\mathbf{A}\approx \tilde{\mathbf{A}}=\sum _{i=1}^k{\mathbf{S}}_i{\mathbf{T}}_i$$ (18)

Given this representation, solving for the coefficients S_i, in general, is a sparse coding problem, with a rich literature (see [43]–[45] and related literature for examples).

As implemented here, the forward operator does not incorporate relaxation effects, T₁(x, y, z) and T₂(x, y, z), or magnetic field inhomogeneities, B₁(x, y, z) and B₀(x, y, z). The primary challenge associated with including relaxation into the forward model is obtaining accurate, robust quantitative relaxation information, which takes significant effort to collect. Measuring field maps is straightforward, after which magnetic field inhomogeneity may be included into the forward operator. However, in the work presented here, the presence of field inhomogeneities was not noticeably detrimental to the results due to the short acquisition periods used with high acquisition bandwidths. To minimize the complexity of the analyses of sparsified model-based image reconstruction, this initial study did not explore the use of parallel receiver coils. However, further gains are anticipated and will be investigated in future work.

In special cases other modifications could be made to the forward operator to achieve even greater speedups. For example, many encoding matrices at readout can be well-approximated by a sparse convolutional operator; potentially vastly improving memory savings. Similarly, techniques which apply the same excitation profile with each RF pulse allow one to factor the excitation profile out of the forward operator for additional speedup. For example, the 2D selective pulses used herein fall into this class, while the STEREO experiment does not. Note that extracting the excitation profile from the forward operator would again require the solution to a sparse coding problem when solving for the coefficients S, and so is particularly amenable to being used in conjunction with multiple fast operators T₁, . . . , T_k.

Further exploration is also warranted in the analysis of this method. Namely, while Theorem 2.1 provides an analysis of the objective value of the optimal solutions of the original and perturbed problems, under certain conditions on the encoding matrices or the signals themselves, it may be possible to show that the solution sets themselves are in some way close. We defer this to a future work.

Appendix

A. Proof of Theorem 2.1

Let $\mathbf{A}\in {ℂ}^{m\times n}$ and let b = Ax^*+η for some noise vector η. We seek to understand the difference between the following two optimization problems:

$$\underset{\mathbf{x}}{\mathrm{minimize}}\underset{F_{\mathbf{A}}(\mathbf{x})}{\underbrace{\frac{1}{2}\Vert \mathbf{A}\mathbf{x}-\mathbf{b}{\Vert }_2^2+\lambda \mathrm{\Omega }(\mathbf{x})}}$$ (P1)

$$\underset{\mathbf{x}}{\mathrm{minimize}}\underset{F_{\tilde{\mathbf{A}}}(\mathbf{x})}{\underbrace{\frac{1}{2}\Vert \tilde{\mathbf{A}}\mathbf{x}-\mathbf{b}{\Vert }_2^2+\lambda \mathrm{\Omega }(\mathbf{x})}},$$ (P2)

where

$$\Vert \mathbf{A}-\tilde{\mathbf{A}}{\Vert }_F<ϵ\Vert \mathbf{A}{\Vert }_F.$$ (19)

Define ${\mathbf{x}}_{\mathbf{A}}={\mathrm{argmin}}_{\mathbf{x}}{F}_{\mathbf{A}}(\mathbf{x})$, ${\mathbf{x}}_{\tilde{\mathbf{A}}}={\mathrm{argmin}}_{\mathbf{x}}{F}_{\tilde{\mathbf{A}}}(\mathbf{x})$, and $\mathbf{E}:=\tilde{\mathbf{A}}-\mathbf{A}$ and recall that the characteristic function of a convex set C is given by

$${\chi }_C(\mathbf{x})=\{\begin{array}{ll}0\hfill & \text{if}\mathbf{x}\in C\hfill \\ \infty \hfill & \text{if}\mathbf{x}\notin C\hfill \end{array}.$$ (20)

Now, for $C\ne \varnothing$ convex, define $\mathrm{\Omega }(\mathbf{x})={\chi }_C(\mathbf{x})$. We establish two inequalities; first,

$$\begin{gathered} \sqrt{F_{\tilde{\mathbf{A}}}(\mathbf{x})}=\frac{1}{\sqrt{2}}\Vert \mathbf{b}-\tilde{\mathbf{A}}\mathbf{x}{\Vert }_2 \\ =\frac{1}{\sqrt{2}}\Vert \mathbf{b}-(\mathbf{A}+\mathbf{E})\mathbf{x}{\Vert }_2 \\ \le \frac{1}{\sqrt{2}}\Vert \mathbf{b}-\mathbf{A}\mathbf{x}{\Vert }_2+\frac{1}{\sqrt{2}}\Vert \mathbf{E}\mathbf{x}{\Vert }_2 \\ =\sqrt{F_{\mathbf{A}}(\mathbf{x})}+\frac{1}{\sqrt{2}}\Vert \mathbf{E}\mathbf{x}{\Vert }_2. \end{gathered}$$ (21)

and second,

$$\begin{gathered} \sqrt{F_{\mathbf{A}}(\mathbf{x})}=\frac{1}{\sqrt{2}}\Vert \mathbf{b}-\mathbf{A}\mathbf{x}{\Vert }_2 \\ =\frac{1}{\sqrt{2}}\Vert \mathbf{b}-\mathbf{A}\mathbf{x}-\mathbf{E}\mathbf{x}+\mathbf{E}\mathbf{x}{\Vert }_2 \\ \le \frac{1}{\sqrt{2}}\Vert \mathbf{b}-(\mathbf{A}+\mathbf{E})\mathbf{x}{\Vert }_2+\frac{1}{\sqrt{2}}\Vert \mathbf{E}\mathbf{x}{\Vert }_2 \\ =\sqrt{F_{\tilde{\mathbf{A}}}(\mathbf{x})}+\frac{1}{\sqrt{2}}\Vert \mathbf{E}\mathbf{x}{\Vert }_2 \end{gathered}$$ (22)

We can now develop our bounds on the different objective values. Since ${\mathbf{x}}_{\tilde{\mathbf{A}}}$ minimizes $F_{\tilde{\mathbf{A}}}(\mathbf{x})$,

$$\begin{gathered} \sqrt{F_{\tilde{\mathbf{A}}}\left({\mathbf{x}}_{\tilde{\mathbf{A}}}\right)}\le \sqrt{F_{\tilde{\mathbf{A}}}\left({\mathbf{x}}_{\mathbf{A}}\right)} \\ \le \sqrt{F_{\mathbf{A}}\left({\mathbf{x}}_{\mathbf{A}}\right)}+\frac{1}{\sqrt{2}}{\Vert \mathbf{E}{\mathbf{x}}_{\mathbf{A}}\Vert }_2 \\ =\sqrt{F_{\mathbf{A}}\left({\mathbf{x}}_{\mathbf{A}}\right)}+\frac{1}{\sqrt{2}}ϵ{\Vert {\mathbf{x}}_{\mathbf{A}}\Vert }_2\Vert \mathbf{A}{\Vert }_F. \end{gathered}$$ (23)

Thus,

$$\sqrt{F_{\tilde{\mathbf{A}}}\left({\mathbf{x}}_{\tilde{\mathbf{A}}}\right)}-\sqrt{F_{\mathbf{A}}\left({\mathbf{x}}_{\mathbf{A}}\right)}\le \frac{1}{\sqrt{2}}ϵ{\Vert {\mathbf{x}}_{\mathbf{A}}\Vert }_2\Vert \mathbf{A}{\Vert }_F.$$ (24)

Similarly, since x_A minimizes F_A(x), we have

$$\begin{gathered} \sqrt{F_A\left({\mathbf{x}}_A\right)}\le \sqrt{F_A\left({\mathbf{x}}_{\tilde{\mathbf{A}}}\right)} \\ \le \sqrt{F_{\tilde{\mathbf{A}}}\left({\mathbf{x}}_{\tilde{\mathbf{A}}}\right)}+\frac{1}{\sqrt{2}}{\Vert \mathbf{E}{\mathbf{x}}_{\tilde{\mathbf{A}}}\Vert }_2 \\ =\sqrt{F_{\tilde{\mathbf{A}}}\left({\mathbf{x}}_{\tilde{\mathbf{A}}}\right)}+\frac{1}{\sqrt{2}}ϵ{\Vert {\mathbf{x}}_{\tilde{\mathbf{A}}}\Vert }_2\Vert \mathbf{A}{\Vert }_F, \end{gathered}$$ (25)

implying

$$\sqrt{F_{\tilde{\mathbf{A}}}\left({\mathbf{x}}_{\tilde{\mathbf{A}}}\right)}-\sqrt{F_A\left({\mathbf{x}}_A\right)}\ge \frac{1}{\sqrt{2}}ϵ{\Vert {\mathbf{x}}_1\Vert }_2\Vert \mathbf{A}{\Vert }_F.$$ (26)

Combining these facts we get:

$$\begin{gathered} \frac{1}{\sqrt{2}}ϵ{\Vert {\mathbf{x}}_{\tilde{\mathbf{A}}}\Vert }_2\Vert \mathbf{A}{\Vert }_F\le \sqrt{F_{\tilde{\mathbf{A}}}\left({\mathbf{x}}_{\tilde{\mathbf{A}}}\right)}-\sqrt{F_{\mathbf{A}}\left({\mathbf{x}}_{\mathbf{A}}\right)} \\ \le \frac{1}{\sqrt{2}}ϵ{\Vert {\mathbf{x}}_{\mathbf{A}}\Vert }_2\Vert \mathbf{A}{\Vert }_F, \end{gathered}$$ (27)

which can be expressed as

$$\begin{gathered} \left|\sqrt{F_{\tilde{\mathbf{A}}}\left({\mathbf{x}}_{\tilde{\mathbf{A}}}\right)}-\sqrt{F_{\mathbf{A}}\left({\mathbf{x}}_{\mathbf{A}}\right)}\right| \\ \le \frac{1}{\sqrt{2}}ϵ\Vert \mathbf{A}{\Vert }_F\max \left\{{\Vert {\mathbf{x}}_{\mathbf{A}}\Vert }_2,{\Vert {\mathbf{x}}_{\tilde{\mathbf{A}}}\Vert }_2\right\} \end{gathered}$$ (28)

completing the proof.