M-MRI: A Manifold-based Framework to Highly Accelerated Dynamic Magnetic Resonance Imaging

Abstract

High-dimensional signals, including dynamic magnetic resonance (dMR) images, often lie on low dimensional manifold. While many current dynamic magnetic resonance imaging (dMRI) reconstruction methods rely on priors which promote low-rank and sparsity, this paper proposes a novel manifold-based framework, we term M-MRI, for dMRI reconstruction from highly undersampled k-space data. Images in dMRI are modeled as points on or close to a smooth manifold, and the underlying manifold geometry is learned through training data, called “navigator” signals. Moreover, low-dimensional embeddings which preserve the learned manifold geometry and effect concise data representations are computed. Capitalizing on the learned manifold geometry, two regularization loss functions are proposed to reconstruct dMR images from highly undersampled k-space data. The advocated framework is validated using extensive numerical tests on phantom and in-vivo data sets.

I. Introduction

Dynamic magnetic resonance imaging (dMRI) plays a principal role in numerous clinical applications for studying qualitative and quantitative dynamics of various physiological phenomena and body organs. Nevertheless, dMRI’s spatio-temporal resolution is often compromised because of inherent long data acquisition times [1], [2]. State-of-the-art techniques in compressed sensing (CS) [3]–[5] as well as low-rank (LR) modeling [6]–[9] have established solid results in reconstructing dMR images from sub-Nyquist sampled k-space data, exploiting sparse and/or low-rank priors along spatio-temporal directions. While most of sparsity-promoting techniques in MRI focus on Fourier and wavelet bases, and low-rank approximation is mainly based on principal component analysis (PCA) to capture the variability of the data, new developments in dimensionality-reduction methods, called manifold learning [10]–[12], have established new ways to look in data correlations. Although the majority of manifold learning techniques are used for data classification and visualization, manifold-based priors have very recently drawn interest in MRI reconstruction [13]–[17].

Capitalizing on manifold learning arguments, this paper presents a manifold based framework, we term as M-MRI and within this framework, develop two novel methods $\text{M-MRI}{\mathcal{R}}_1$ and $\text{M-MRI}{\mathcal{R}}_2$ for image reconstruction in dMRI from highly undersampled k-space data. Image x_i, taken from a dMRI series and embedded in a high dimension space ${ℂ}^N$, is modeled as a point on or close to a manifold $\mathcal{M}$ such that $\mathcal{M}\subset {ℂ}^N$. Relations amongst images in dMRI are learned from appropriately defined neighborhoods in $\mathcal{M}.$ Unlike [13], [17], where neighborhoods are determined from a predefined Gaussian kernel, that has been shown to be successful in modeling diffusion and heat-map processes, we propose here instead a data-driven learning approach where neighborhoods are defined by the data themselves based on data-specific sparse affine relations [12]. In addition, to promote low-dimensional descriptions of data, a low-dimensional embedding that preserves the manifold geometry described by the learned neighborhood is also computed [12].

The rest of the paper is organized as follows. Section II describes the basic principles behind dMRI with undersampled data. Section III presents the proposed methods, and section IV validates the advocated approach using extensive numerical tests on data. Finally, section V concludes the paper.

II. Background

The data acquisition process in N_c - channel dMRI with N_fr frames can be formulated as:

$${\mathbf{y}}_{ij}={\mathbf{\text{S}}}_i\mathbf{\text{F}}{\mathbf{\text{C}}}_j{\mathbf{x}}_i+{\mathbf{\eta }}_{ij,}$$ (1)

where S_i is the undersampling pattern in i^th frame, i = 1, 2, … N_fr, F is the Fourier matrix, C_j is the sensitivity of the j^th coil, j = 1, 2, … N_c and η is noise. Defining the dynamic image series as an N × N_fr Casorati matrix X = [x₁, x₂, …x_Nfr], where N = Number of phase encoding lines (N_p) × Number of frequency encoding lines (N_f) and each column x_i is the vectorized image, (1) can be rewritten as,

$$\mathbf{\text{Y}}=\varphi (\mathbf{\text{X}})+\mathbf{\eta },$$ (2)

where, ϕ is measurement operator that incorporates Fourier undersampling and coil sensitivity in (1). To reconstruct dynamic image series X from acquired undersampled k-space data Y an optimization problem, regularized via $\mathcal{R}\left(\cdot \right)$ is solved:

$$\text{arg}\mathbf{\hspace{0.17em}}{\text{min}}_{\mathbf{\text{X}}}{‖\mathbf{\text{Y}}-\varphi (\mathbf{\text{X}})‖}_{\text{F}}^2+\lambda \mathcal{R}(\mathbf{\text{X}}).$$ (3)

Typically, $\mathcal{R}\left(\cdot \right)$ is the temporal Fourier sparsity-aware loss ${‖F_t(\mathbf{\text{X}})‖}_1$, where $F_t(\cdot )$ is the Fourier transform along temporal direction or, the low-rank prior rank(X). However, this paper proposes two novel regularizers based on a manifold-based approach. As a first step, neighborhood relation amongst images are learned via our underlying manifold-based modeling. Secondly, a nonlinear mapping which embeds $\mathcal{M}$ into ${ℂ}^M$, $M\ll N$, is computed, and two regularizers are defined based on the learned manifold geometry. Finally, we enforce those regularizers on undersampled k-space data to reconstruct the desired dMR image.

III. Proposed Framework

A. Manifold learning

Let x_i, i = 1,…, N_fr, be the N × 1 vectored form of size N_p × N_f images. We postulate that x_i lies on or close to an M-dimensional $\left(M\ll N\right)$ manifold $\mathcal{M}\subset {ℂ}^N$, as shown in Fig. 1. The first step is to find the neighborhood relation between ${\left\{{\mathbf{\text{x}}}_i\right\}}_{i=1}^{N_{\text{fr}}}$. Unlike the case of many learning applications, where neighborhoods are defined by the Euclidean distance of the ambient space ${ℂ}^N$, we capitalize here on the smoothness of the unknown manifold $\mathcal{M}$, and postulate that each image vector x_i can be approximated by the affine combination of its neighboring image vectors:${\mathbf{\text{X}}}_i\approx {\sum }_{n=1}^{N_{\text{fr}}}{\omega }_i^n{\mathbf{\text{x}}}_n$, mimicking well-known properties of tangent spaces of smooth manifolds [12]. In this way, each image is related to its neighbors by the weight vector ω_i. The weight vector ω_i is constrained to be sparse, i.e., each image has a limited number of neighbors. Such a neighborhood relation can be captured by solving the following ${\ell }_1$-constrained least-squares problem:

$${\mathbf{\omega }}^i\in \underset{{\mathbf{\omega }}_i^{H}1N_{\text{fr}}=1,{\omega }_i^i=0}{\text{arg min}}{‖{\mathbf{\text{X}}}_i-{\sum }_{n=1}^{N_{\text{fr}}}{\omega }_i^n{\mathbf{\text{x}}}_n‖}^2+\beta {\Vert {\mathbf{\omega }}^i\Vert }_1,$$ (4)

where ${\mathbf{1}}_{N_{\text{fr}}}$ is the all-one vector. The constraint ${\mathbf{\omega }}_i^H{\mathbf{1}}_{N_{\text{fr}}}=1,{(·)}^H$ denotes Hermitian transposition, ascertains that weights sum up to 1, justifying the affine neighboring relations, ${\omega }_i^i=0$ excludes x_i from being a neighbor of itself, β ≥ 0 can be tuned to constrain the numbers of neighbors K_n. Eq. (4) is solved using arguments similar to the minimization framework based on the celebrated least square shrinkage and selection operator (LASSO).

Fig. 1.

Illustration of a smooth manifold. Three-dimensional data points lying close to the 2-dimensional smooth manifold surface.

It is important to note here that, in undersampled MRI acquisition, full images x_i are not available, rather, they are the desired output. Often times, undersampling is carried out in a way such that certain low frequencies are sampled in all frames, whereas random undersampling is mimicked along phase encoding and time plane as shown in Fig. 2, in the case of Cartesian undersampling. Such low frequency signals, called “navigators,” are used to calculate weights that define neighborhoods. Later, in section IV-A, we show that such navigator lines are sufficient enough to describe neighborhoods and approximate a manifold embedding. Hence, navigators $\widehat{x}$ are used to estimate weights ωⁱ using (4).

Fig. 2.

1-D Cartesian undersampling pattern with navigator lines.

B. Manifold embedding and regularization

In accelerated dMRI the spatial data is highly undersampled, however the temporal direction is often fully sampled. This allows to learn the temporal basis of the desired dynamic image series. Typically, the temporal basis are learned through the PCA or singular value decomposition (SVD) of navigator signal [6], [7], [9]. In this paper, we exploit the neighborhood relation to learn such temporal basis. Let W be a N_fr × N_fr weight matrix with entries ${\omega }_i^n$. Having learned W, the nonlinear embedding Ψ, that maps ${\mathbf{\text{X}}}_i\in {ℂ}^N$ into ${ℂ}^M$ is given by solving

$$\underset{\begin{array}{l}\mathbf{\Psi }\in {ℂ}^{M\times N_{\text{fr}}},\mathbf{\Psi }{\mathbf{\Psi }}^H={\mathbf{\text{I}}}_M,\\ \hspace{1em}\hspace{1em}\mathbf{\Psi }{\mathbf{1}}_M=0_M\end{array}}{\text{arg min}}‖{\psi }_i-{\sum }_{n=1}^{N_{\text{fr}}}{\omega }_i^n{\psi }_n‖,$$ (5)

where $\text{Ψ=}\left[{\psi }_1{\psi }_2\dots {\psi }_{N_{\text{fr}}}\right]$. The constraint $\mathbf{\Psi }{\mathbf{\Psi }}^H={\mathbf{\text{I}}}_M$, where I_M is M × M identity matrix, excludes trivial all zero solution, whereas $\mathbf{\Psi }{\mathbf{1}}_M=0$ centers the columns of $\mathbf{\Psi }$ around 0. The solution of (5) is given by the eigen-decomposition of appropriate N_fr × N_fr matrices [10], [18]. Define matrices

$$\mathcal{K}:=\left(\mathbf{\text{I}}-\mathbf{\text{W}}\right){\left(\mathbf{\text{I}}-\mathbf{\text{W}}\right)}^H,\hspace{1em}\hspace{1em}\mathcal{L}:=\mathbf{\text{D}}-\mathbf{\text{W}},$$

where I is the identity matrix, D is a diagonal matrix with non-zero entries $d_{ii}:={\sum }_{n=1}^{N_{\text{fr}}}{\omega }_i^n$. Based on these two matrices we develop two regularizers ${\mathcal{R}}_1\left(\cdot \right)$ and ${\mathcal{R}}_2\left(\cdot \right)$ specific to $\text{M-MRI}{\mathcal{R}}_1$ and $\text{M-MRI}{\mathcal{R}}_2$, respectively to reconstruct dynamic MRI from undersampled k-space data.

1. $M-MRI{\mathcal{R}}_1$ (affine combination): The first regularizer is based on our modeling that each image (point on a manifold) is closely approximated by the affine combination of its neighbors. Define approximation error matrix operation $X_{\text{err}}=\mathbf{\text{X}}\left(\mathbf{\text{I}}-\mathbf{\text{W}}\right)$. Elementary algebra reveals that the i^th column ${\mathbf{\text{x}}}_{\text{err}}^i={\mathbf{\text{x}}}_i-{\sum }_{n=1}^{N_{\text{fr}}}{\omega }_i^n{\mathbf{\text{x}}}_n$ provides the error of approximation for x_i. As such, to enforce the affine-combination modeling assumption, the following loss function is introduced

   $$\begin{array}{l}{\mathcal{R}}_1\left(\mathbf{\text{X}}\right)={‖\mathbf{\text{X}}\left(\mathbf{\text{I}}-\mathbf{\text{W}}\right)‖}_{\text{F}}^2=\text{tr}\left(\mathbf{\text{X}}\left(\mathbf{\text{I}}-\mathbf{\text{W}}\right){\left(\mathbf{\text{X}}\left(\mathbf{\text{I}}-\mathbf{\text{W}}\right)\right)}^H\right)\\ \mathbf{\hspace{1em}}\mathbf{\hspace{1em}}\mathbf{\hspace{1em}}=\text{tr}\left(\mathbf{\text{X}}\mathcal{K}{\mathbf{\text{X}}}^H\right),\end{array}$$ (6)

   where tr denotes the trace of a matrix.

2. $M-MRI{\mathcal{R}}_2$ (smoothness): The Laplacian matrix $\mathcal{L}$ can be decomposed as $\mathcal{L}=\mathbf{\text{G}}{\mathbf{\text{G}}}^H$, where G^H stands for the incidence matrix [18]. It has been shown that the matrix G acts as a finite difference operator and ${‖\mathbf{\text{XG}}‖}_F^2$ resembles Tikhonov regularization [13], [18]. Hence, defining the second regularizer as,

$${\mathcal{R}}_2\left(\mathbf{\text{X}}\right)={‖\mathbf{\text{XG}}‖}_{\text{F}}^2=\text{tr}(\mathbf{\text{X}}\mathcal{L}{\mathbf{\text{X}}}^H).$$ (7)

The regularizer ${\mathcal{R}}_2\left(\cdot \right)$ is similar to the “${\mathcal{l}}_2$ STORM” one, reported in [13], as both are inspired by the graph Laplacian and enforce manifold smoothness to the design. However, unlike [13], where neighborhoods are user-defined and thus data-agnostic, the neighborhoods of the present framework are defined by the data themselves via affine relations and by calculating weights as in (4). Using a generic kernel to learn data relations maybe simple and beneficial to learning systems that use diffusion maps and heat flow, but such an “one-fits-all” approach often appears to be prone to modeling errors.

C. Manifold regularization and reconstruction

The singular vectors of the N_fr × N_fr matrices $\mathcal{K}$ and $\mathcal{L}$ provide the solutions of (5) [10], [18]. These singular vectors approximate the temporal basis of dynamic images as illustrated in the Fig. 3. Let the singular value decompositions of $\mathcal{K}$ and $\mathcal{L}$:

$$\mathcal{K}={\mathbf{\Psi }}^k{\mathbf{\Sigma }}^k{\mathbf{\Psi }}^{k^H},$$ (8)

$$\mathcal{L}={\mathbf{\Psi }}^l{\mathbf{\Sigma }}^l{\mathbf{\Psi }}^{l^H},$$ (9)

where superscripts ${\left(\cdot \right)}^k$ and ${\left(\cdot \right)}^l$ indicate the decomposition $\mathcal{K}$ of and $\mathcal{L}$, respectively. Eigen decompositions have been widely used to represent temporal variations in dMRI [6]–[8]. However, it is worth noticing here that, while PCA decompositions preserve data covariances, the “decomposition” of (4) preserves essentially the manifold geometry [10], [18].

Fig. 3.

Reference and approximated eigenvectors (ψ_i) of $\kappa$. Left to Right: 2nd, 3rd, 4th and 5th.

Substituting, (8) in (6),

$$\begin{gathered} {\mathcal{R}}_1\left(\mathbf{\text{X}}\right)=\text{tr}\left(({\mathbf{\text{X}}\mathbf{\Psi }}^k){\mathbf{\Sigma }}^k{({\mathbf{\text{X}}\mathbf{\Psi }}^k)}^H\right)=\text{tr}\left({\mathbf{\text{U}}}^k{\mathbf{\Sigma }}^k{({\mathbf{\text{U}}}^k)}^H\right) \\ \hspace{1em}\hspace{1em}\hspace{1em}={\sum }_{m=1}^M{\sigma }_m^k{‖{\mathbf{\text{u}}}_m^k‖}^2, \end{gathered}$$ (10)

where ${\mathbf{\text{U}}}^k={\mathbf{\text{X}}\mathbf{\Psi }}^k$. Unlike in PCA, M minor singular vectors are used [10], [18], instead of principal, to approximate the solution of (5). Hence, ${\mathbf{\text{u}}}_m^k=\mathbf{\text{X}}{\psi }_m^k$ is the projection of X on the m^th minor singular vector of $\mathcal{K}$. Similarly, substituting (9) in (7) we obtain an equivalent expression for ${\mathcal{R}}_2$:

$${\mathcal{R}}_2\left(\mathbf{\text{X}}\right)={\sum }_{m=1}^M{\sigma }_m^l{‖{\mathbf{\text{u}}}_m^l‖}^2.$$ (11)

Hence, the original problem in (3) can be expressed via U^k and U^l as

$$\text{arg}{\text{min}}_{{\mathbf{\text{U}}}^k}{‖\mathbf{\text{Y}}-\varphi \left({\mathbf{\text{U}}}^k{({\mathbf{\Psi }}^k)}^H\right)‖}_{\text{F}}^2+\lambda {\sum }_{m=1}^M{\sigma }_m^k{‖{\mathbf{\text{u}}}_m^k‖}^2,$$ (12)

$$\text{arg}{\text{min}}_{{\mathbf{\text{U}}}^l}{‖\mathbf{\text{Y}}-\varphi \left({\mathbf{\text{U}}}^l{({\mathbf{\Psi }}^l)}^H\right)‖}_{\text{F}}^2+\lambda {\sum }_{m=1}^M{\sigma }_m^l{‖{\mathbf{\text{u}}}_m^l‖}^2.$$ (13)

Finally, the conjugate gradient algorithm is used to solve (12), (13) and reconstruct desired images from U^k and U^l.

IV. Results and Discussion

To verify our proposed methods we implement them in two data sets, cardiac cine phantom and prospectively undersampled real time cardiac cine data. All experiments are performed in quad core workstation Intel(R) 3.6 GHz with windows 7, 32 GB RAM running MATLAB 2014.

A. Cardiac Cine Phantom

The MRXCAT [19] using extended cardiac torso (XCAT) is used to generate breath hold cardiac cine data. Single channel simulation is mimicked to generate the cardiac cine data of matrix size (N_p × N_f × N_fr) 408 × 408 × 360. The numeric cine phantom has about 15 cardiac cycles and 24 cardiac phases. The data is then retrospectively undersampled using Cartesian sampling as shown in Fig. 2. Each frame has about 12 acquired phase encoding lines out of which 4 are navigator lines from the central k-space data with net reduction factor of about 32.

The navigator lines are used to calculate the weights and effectively the eigenvectors in (5). Fig 3 compares the eigenvectors of $\mathcal{K}$ computed using the fully sampled data (reference) and using navigator lines (approximated). For better visualization, only section of curves ([1:50 311:360]) are shown. It can be seen that few navigator lines are sufficient to approximate first few eigenvectors.

The performance of the proposed methods are compared with the state-of-the-art PS-sparse method [7], as shown in Figs. 4 and 5. The parameters in all methods were tuned heuristically and the results with best RNMSE values are presented. Fig. 4 shows two frames representing two different cardiac stages, systole and diastole, whereas Fig. 5 shows the x-t cross section along the dotted line in the reference image. From Fig. 4, we can see that the proposed methods perform better than the competitive method. PS-sparse tends to have noise-like artifacts, while in x-t cross section we can see that the PS-sparse has ripple-like aliasing artifacts. Computational times for PS-sparse and proposed methods $\text{M-MRI}{\mathcal{R}}_1$ and $\text{M-MRI}{\mathcal{R}}_2$ are 48, 39 and 32 mins, respectively.

Fig. 4.

Spatial results for numeric cine phantom. Systole and diastole phase. Left to Right: Reference, PS-sparse (0.0682), proposed $\left(\text{M-MRI}{\mathcal{R}}_1\right)$ (0.0532), proposed ($\left(\text{M-MRI}{\mathcal{R}}_2\right)$) (0.0547).

Fig. 5.

Temporal x-t cross section results for numeric cine phantom. Top to bottom: Reference, PS-sparse, proposed $\left(\text{M-MRI}{\mathcal{R}}_1\right)$, proposed $\left(\text{M-MRI}{\mathcal{R}}_2\right)$.

To numerically quantify the reconstruction error, we compared the root normalized mean-square error (RNMSE) defined as the normalized difference square between the reference (X_REF) and the reconstructed image (X_REC) as,

$$\text{RNMSE}=\frac{{‖{\mathbf{\text{X}}}_{\text{REF}}-{\mathbf{\text{X}}}_{\text{REC}}‖}_{\text{F}}}{{‖{\mathbf{\text{X}}}_{\text{REF}}‖}_{\text{F}}}.$$ (14)

Numbers in parentheses in Fig. 4 indicate RNMSE values.

B. In-vivo cardiac cine data

For in-vivo experiment, we acquired prospective undersampled free breathing data using 1-D Cartesian sampling. Data is acquired in 12 channels from a volunteer breathing normally. 14,000 phase encoding lines were acquired continuously resulting in 1, 000 frames with about 14 phase encoding lines out of which 4 are navigators, per frame. The acquisition parameters used were TR/TE = 5.8/4 ms, FOV 284 × 350 mm, spatial resolution = 1.8 mm. The data matrix was of size 156 × 192 × 1000 for each channel.

During reconstruction, each channel data was processed separately, and for fair comparisons, final images were obtained using sum of square of images from each channel. The computational times for PS-sparse and the proposed methods $\text{M-MRI}{\mathcal{R}}_1$ and $\text{M-MRI}{\mathcal{R}}_2$ are 132, 125 and 117 mins, respectively.

Fig. 6 shows two representative frames reconstructed by each method, while Fig. 7 shows the x-t cross section along dotted line. The PS-sparse method tend to have sharper spatial images, however they show granular like reconstruction artifacts. The temporal details in PS-sparse method look smoother. This is likely because of over-smoothing the temporal basis approximated by singular value decomposition and Fourier periodic approximation. This is illustrated in the zoom in region in Fig. 7. The arrows point to the areas where it looks substantially smoother than results from proposed methods. The RNMSE values cannot be calculated because the data is prospectively undersampled and do not have the Gold standard for the comparison.

Fig. 6.

Spatial results for real time free breathing cine. Systole and diastole phase. Left to right: PS-sparse, proposed $\left(\text{M-MRI}{\mathcal{R}}_1\right)$, proposed $\left(\text{M-MRI}{\mathcal{R}}_2\right)$.

Fig. 7.

Temporal x-t cross section results for real time free breathing cine. Top to bottom: PS-sparse, proposed $\left(\text{M-MRI}{\mathcal{R}}_1\right)$, proposed $\left(\text{M-MRI}{\mathcal{R}}_2\right)$. Only first 150 frames are shown. Second column shows the zoom in view of region over dotted rectangular region.

v. Conclusion

In this paper, we presented a novel manifold-based framework for dynamic image reconstruction from highly undersampled k-sapce data. We proposed a novel data driven technique to learn the manifold from training data, called “navigators,” and developed two novel methods within the manifold framework for reconstructing dynamic MRI from highly undersampled k-sapce data. The proposed methods are validated using numerical phantom and real time in-vivo data. Extensive validation of the proposed methods on more data sets and applications is desired. It should be noted that the proposed methods do not depend on the periodicity assumption, unlike the state-of-the-art PS-sparse method. This motivates the study and implementation of the proposed framework in many other dMRI applications, such as speech, lungs, and liver imaging. The proposed methods also establish the modality of exploiting a-priori information from manifold geometry and develop new constraints for dynamic MR image reconstruction from undersampled data.