Patch Based Reconstruction Of Undersampled Data (PROUD) for High SNR and High Frame Rate Contrast Enhanced Liver Imaging

Abstract

PURPOSE

High spatial-temporal four-dimensional imaging with large volume coverage is necessary to accurately capture and characterize liver lesions. Traditionally, parallel imaging and adapted sampling are used towards this goal, but they typically result in a loss of signal to noise. Furthermore, residual under-sampling artifacts can be temporally varying and complicate the quantitative analysis of contrast enhancement curves needed for pharmacokinetic modeling. We propose to overcome these problems using a novel patch-based regularization approach called Patch-based Reconstruction Of Under-sampled Data (PROUD).

METHODS

PROUD produces high frame rate image reconstructions by exploiting the strong similarities in spatial patches between successive time frames to overcome the severe k-space under-sampling. To validate PROUD, a numerical liver perfusion phantom was developed to characterize CNR performance compared to a previously proposed method, TRACER. A second numerical phantom was constructed to evaluate the temporal footprint and lag of PROUD and TRACER reconstructions. Finally, PROUD and TRACER were evaluated in a cohort of five liver donors.

RESULTS

In the CNR phantom, PROUD, compared to TRACER, improved peak CNR by 3.66 times while maintaining or improving temporal fidelity. In vivo, PROUD demonstrated an average increase in CNR of 60% compared to TRACER.

CONCLUSION

The results presented in this work demonstrate the feasibility of using a combination of patch based image constraints with temporal regularization to provide high SNR, high temporal frame rate and spatial resolution four dimensional imaging.

INTRODUCTION

The ability to reliably capture the arterial phase in contrast enhanced imaging is crucial for the detection and characterization of liver lesions. Multiple phase imaging within a breath-hold has shown promise towards achieving this goal (1–3). To retain both high spatial and temporal resolution, parallel imaging techniques with adapted sampling schemes have been used, both with Cartesian (4–8) and radial (9–12) trajectories. The success of these techniques lies in the particular balance chosen between undersampling artifacts, signal to noise ratio (SNR), volume coverage, spatial resolution and temporal fidelity. TRACER (13), a nonlinear parallel imaging reconstruction (14) of golden ratio ordered variable density spiral acquisition, allows the reconstruction of volume covering the whole liver with a sub-second frame rate. The high temporal frame rate (equal to the time to acquire a single spiral leaf for all slice encodings) was achieved by assuming small changes of image content between frames, an assumption which is increasingly satisfied as the frame rate increases. Both in simulations and phantom experiments, a high agreement between the reconstructed and the true contrast enhancement curves was observed. However, the reconstructed images are susceptible to noise in the rapid spiral acquisition; the measured temporal footprint of each frame was found to be larger than the apparent frame rate; and residual undersampling artifacts resulted in flickering artifacts visible when viewed in cine mode.

Recently, patch-based image reconstruction methods using dictionary learning have shown promise in reconstructing both static and dynamic under-sampled MRI data (15–24) by exploiting local structure and similarity to a known dictionary. In these methods, an overcomplete basis is learned from prior measurements or the data itself during the reconstruction, using for instance the K-SVD algorithm (25–27). It is assumed that every patch in the unknown image can be written as a sparse linear combination – i.e. with very few non-zero coefficients – of elements in this basis, typically referred to as “atoms”. In this work, the computationally expensive dictionary learning step is replaced by an explicit construction of the dictionary, based on the assumption that image patches change little in structure and location between successive time frames. For every patch, a number of sets with a fixed number of patches are constructed by taking a patch around the same location (allowing for some translation) in a number of reference images, one of which is the time frame immediately preceding the frame to be reconstructed. Combined with an L2 temporal regularization, the resulting method is called Patch-based Reconstruction Of Under-sampled Data (PROUD) and is applied to dynamic imaging data with frames that are ~250 ms apart. Through phantom and in vivo studies, it is shown that PROUD is able to maintain the high spatial and temporal frame rate obtained with TRACER while allowing for increased SNR, reduction of temporal footprint, and reduced undersampling artifacts.

THEORY

The PROUD algorithm reconstructs dynamic highly undersampled data: in this work a single spiral leaf is used to reconstruct each frame, assuming a fully sampled image is available for the first time frame (see below). To allow a reconstruction, the regularization used in PROUD is based on the observation that the local structure of the image is largely preserved between successive frames in high frame rate dynamic imaging. An additional temporal regularization is used to suppress temporally varying residual undersampling artifacts. In previous approaches (13,14), differences between successive frames were constrained on a global, image based level. To constrain differences on a local level, we use image patches. Patch based image regularization has proven very successful in image processing algorithms such as denoising (26).

In this work, each n × n patch R_i,jv around pixel (i,j) of the unknown image v is assumed to be a linear combination of a small number of n × n patches taken from a dictionary of patches P_i,jD specific to the pixel (i,j). This dictionary is constructed by taking n × n patches from an m × m neighborhood around the pixel (i,j) selected by P_i,j from a set D constructed from N_R reference images that are assumed to represent the local structure of the unknown image v well. For each pixel (i,j), the dictionary D_t contains a set of patches d_i,j. Each set d_i,j has N_R patches of size n × n selected around pixel (i,j) in each of the N_R reference images. Therefore, the dictionary D_t has size (N_x × N_y) × N_R × (n × n), where N_x and N_y are the number of columns and rows of v_t, respectively. Note that for pixels near the edge of the field of view, zero padding was used in the construction of patches when needed. Therefore, the following minimization problem was solved in order to obtain a solution $v_t^{\ast }$ for each time t = 1, …, T:

$$\begin{array}{c}{v}_t^{\ast }=\underset{v_t,{\alpha }_{i,j,t}}{\text{argmin}}{\Vert U_t{\mathit{FSv}}_t-y_t\Vert }_2^2+\lambda {\sum }_{i,j}{\Vert R_{i,j}{v}_t-{\alpha }_{i,j,t}{P}_{i,j}{D}_t\Vert }_2^2\\ \text{such}\text{that}{\Vert {\alpha }_{i,j,t}\Vert }_{C0}=1\end{array}$$ [1]

where y_t is the sampled k-space data for all coils, T is the total number of time frames, S is the operator that multiplies the image v_t with each of the coil sensitivities (s_i)_{i =1, …, Nc}, N_c is the number of coils, F is the spatial Fourier transform, U_t is the projection of k-space unto the spiral leaf acquired at time t. Each vector α_i,j,t has linear coefficients α_i,j,t(p, r) with p = 1, …, N_p and r = 1, …, N_r, with N_p = (m − n + 1) × (m − n + 1) the number of overlapping patches of size n × n that fit within a m × m neigborhood. Note that, here, we index the vector α_i,j,t with two indices for clarity. On the vectors α_i,j,t, a composite L₀ norm was defined as ${\Vert {\alpha }_{i,j,t}\Vert }_{C0}=\#\{p=1,\dots ,N_P\mid 0<{\sum }_{r=1}^{N_R}\mid {\alpha }_{i,j,t}(p,r)\mid \}$. Therefore, the constraint in Eq. 1 ensures that only one set of patches from the dictionary P_i,jD_t is selected. This reflects the constraint that an image patch is expected to move only slightly between time frames and can change in scale (DC component) as well. The latter will be the case for contrast enhanced imaging. Note that, compared to conventional dictionary learning approaches, the number of non-zero linear coefficients is always set to N_R and that only certain combinations of patches are allowed, i.e., those that were derived from the same pixel location in the N_R reference images. In this work, D_t was constructed from the following reference images: the previously reconstructed image $v_{t-1}^{\ast }$ and a composite image reconstructed from all acquired data, which in dynamic imaging is fully sampled or even oversampled. In vivo, to further improve robustness to through plane motion, D_t included reference images derived from N_slices neighboring slices reconstructed from all acquired data as well. The minimization of the patch constraint term in Eq. 1 is simplified by orthonormalizing each set of patches from the dictionary P_i,jD_t, such that the optimal linear combinations α_i,j,t(p, r) can be computed using a simple projection (see last paragraph of Appendix B). A general depiction of Eq. 1 for one time frame is shown in Fig. 1. Note that in Fig. 1, the displayed set of patches is not orthonormalized for clarity of presentation only.

Figure 1.

A diagram of the PROUD reconstruction algorithm for one temporal frame. Patch size is 5×5 with a neighborhood size of 7×7.

As in TRACER (13), an initial guess v₀ for the first frame was reconstructed from the first fully sampled set of data, which is typically available in dynamic imaging. For the contrast enhanced imaging application, this assumes no contrast agent has entered the field of view until after the first fully sampled set of data is acquired. This requirement can be considerably relaxed by taking the same fully sampled reconstruction image and perform a PROUD (or TRACER, see below) reconstruction backwards in time from the end of the fully sampled data set towards the beginning, after which the reconstruction is again performed now going forward in time. This allows a better guess for the first time frame. The coil sensitivities (s_i)_{i =1, …, Nc} are derived by dividing low resolution coil images by their root sum of squares. These low resolution coil images were obtained from all acquired data in the dynamic acquisition, which typically is long enough to provide a highly oversampled, high SNR reconstruction. The determination of the regularization parameter λ is described below.

For high temporal resolution imaging, a new frame is reconstructed for each newly acquired set of k-space data (13), which is assumed to be acquired in a small time interval in order to satisfy the assumption of smooth temporal changes. With PROUD (and TRACER, see below), a single spiral leaf (collected for all slice encodings in the case of 3D imaging) is used to update each temporal frame, typically at a rate of ~250 ms. The angle between two successive spiral leaves is based on the golden ratio (13,28,29).

In a second step, temporal smoothness was enforced to overcome the temporally varying residual undersampling artifacts in the reconstructed images. This regularization provides smooth time courses for contrast enhancement curves simplifying post-processing. This was done by solving:

$$\begin{array}{c}{v}_{t=1,\dots ,T}^{\ast }=\underset{\begin{array}{l}{v}_t,{\alpha }_{i,j,t}\\ t=1,\dots ,T\end{array}}{\text{argmin}}\{{\sum }_{t=1}^T{\Vert U_t{\mathit{FSv}}_t-y_t\Vert }_2^2+\lambda {\sum }_{t=1}^T{\sum }_{i,j}{\Vert R_{i,j}{v}_t-{\alpha }_{i,j,t}{P}_{i,j}{D}_t\Vert }_2^2\}\\ +\gamma {\sum }_{t=1}^T{\Vert v_t-(v_{t-1}+v_{t+1})/2\Vert }_2^2\}\\ \text{such}\text{that}{\Vert {\alpha }_{i,j,t}\Vert }_{C0}=1\text{for}\text{all}t=1,\dots ,T,\end{array}$$ [2]

Eqs. 1&2 are solved by extending the method used in (24) to the multi-channel non-Cartesian case. A detailed derivation is shown in Appendix B. The solver algorithm iterates between solving for α_i,j,t and solving for v_t. Since spiral data was acquired in this study, nonuniform fast Fourier transforms were performed using NUFFT (30,31). The dictionary D_t for each frame v_t was constructed in the same way as for Eq. 1.

The patch and temporal regularization parameters, λ and γ, were determined automatically by the solver. To find λ, we estimated a solution for $v_1^{\ast }$ by excluding the regularization term from Eq. 1:

$$v_1^{\ast }=\underset{v_1}{\text{argmin}}{\Vert U_1{\mathit{FSv}}_1-y_1\Vert }_2^2$$ [3]

Because of the severe undersampling by U₁, which projects onto a single leaf, this was iteratively solved by using v₀ as the initial guess and halting the iterations when the image update between iterations no longer decreased, similar to the stopping criterium used in TRACER (13). Next, a vector ${\alpha }_{i,j,1}^{\ast }$ was found by solving (see Appendix B):

$${\alpha }_{i,j,1}^{\ast }=\underset{{\alpha }_{i,j,1}}{\text{argmin}}{\sum }_{i,j}{\Vert R_{i,j}{v}_1^{\ast }-{\alpha }_{i,j,1}{P}_{i,j}{D}_t\Vert }_2^2\text{such}\text{that}{\Vert {\alpha }_{i,j,1}\Vert }_{C0}=1$$ [4]

Using the discrepancy principle (32), the regularization parameter λ was then chosen such that the two terms in Eq. 1 were equal for the values of $v_1^{\ast }$ and ${\alpha }_{i,j,1}^{\ast }$ obtained in Eqs 3 and 4, respectively. The value of λ was then kept constant for all remaining time frames. After Eq. 1 was solved for all time frames, the regularization parameter γ was obtained by demanding that the first term (data term) equaled the third term (temporal term) in Eq. 2 when $v_t^{\ast }$ and ${\alpha }_{i,j,t}^{\ast }$ were set to the solutions of Eq. 1. Appendix A outlines the PROUD reconstruction algorithm in detail.

In the experiments below, the performance of PROUD will be compared TRACER, a previous method to perform high frame rate reconstructions. For clarity, a brief description of this method is included here. Using the notation introduced above, TRACER aims to solve the following equation:

$$v_t^{\ast }=\underset{v_t}{\text{argmin}}{\Vert U_{t}F{S}_t{v}_t-y_t\Vert }_2^2$$ [4a]

which is similar to the data term in Eq. 1 for a single time frame, except for the fact that the coil sensitivities are now considered to be unknown as well and are allowed to vary over time. This makes Eq. 4a a non-linear least squares problem in the variable x_t,k = {v_t,k, s_1,t,k, … s_Nct,k}. It is solved using the Levenberg–Marquardt (LM) algorithm, which iteratively determines an update dx_t,k = {dv_t,k, ds_1,t,k, … ds_Nc,t,k} to an initial guess x_t,0:x_t,k = x_{t,k − 1} + dx_t,k}. To deal with the high undersampling, each frame x_t was reconstructed using the previous frame as the initial guess x_t,0 = x_{t − 1} and the iteration was stopped when the norm of the update ||dx_t,k|| stopped decreasing (13). The underlying assumption is the change in image content dx_t is small between successive time frames. Note that in contrast to PROUD the difference between successive frames is measured using a single norm of the entire image update. All reconstruction parameters were automatically determined based on the data as described in (13).

METHODS

Optimization of reconstruction parameters

To characterize the PROUD algorithm, a numerical phantom simulating dynamic liver perfusion was created (Fig. 2a) similar to the phantom in (13). The noiseless phantom $v_t^{\mathit{REF}}(t=1,\dots ,T)$ was multiplied by simulated coil sensitivity maps (eight channels), and sampled with a spiral trajectory. For each time frame, a cutoff of r_cutoff = 10⁻⁵ with a maximum of 100 and 5 iterations was used for the first (Eq. 1) and the remaining temporal iterations (Eq. 2), respectively. Five temporal iterations were used in solving Eq. 2. The patch size was determined by solving Eq. 1 for a range of sizes: square patches of size n = 3,6,7,9,11 and 13 pixels were tested. The root mean square error RMSE, defined as

Figure 2.

(A) A numerical phantom simulating liver perfusion indicating the various simulated organs, including the inferior vena cava (IVC) (B). Temporal footprint phantom with inner diameter = 0.5*FOV.

$$\mathit{RMSE}=\sqrt{{\sum }_t{\Vert v_t-v_t^{\mathit{REF}}\Vert }_2^2}$$ [5]

was calculated to determine the optimal patch size. The background of the phantom was excluded from the norm in Eq. 5. The neighborhood size was then set m = n+2 to reduce the increase in computation time while still allowing small local translational motions of image patches between successive time frames.

Numerical phantom SNR

Thirty sets of randomly generated Gaussian noise was added to the numerical phantom described above. A fully sampled first frame was assumed. The phantoms were then reconstructed using PROUD with the reconstruction parameters obtained above and using the TRACER algorithm (1) for comparison. A contrast-to-noise ratio CNR was computed as

$$\mathit{CNR}={\mathit{SNR}}_{\mathit{aorta}}-{\mathit{SNR}}_{\mathit{portal}\mathit{vein}}$$ [6]

where SNR_aorta and SNR_{portal vein} are the signal-to-noise ratios within a region of interest (ROI) in the respective anatomies. Pixel-wise SNR was computed at each time-point by taking the mean of each pixel in an ROI over the 30 reconstructions divided by the standard deviation of each pixel. The mean of the pixel-wise SNR values was then taken over the ROI.

Numerical phantom temporal footprint

Temporal phantoms were developed (Fig. 2b) to characterize the temporal response of the PROUD algorithm compared to TRACER. Each phantom consists of a central disk with diameter (DIA) equal to 0.125, 0.1875, 0.25, 0.3125, 0.375, 0.4375, 0.5, 0.5625, 0.625, 0.6875, 0.75, 0.8125, 0.875, 0.9375, 1 times the FOV superimposed onto a larger disk with diameter equal to the FOV. The contrast in the central disk was changing over time, such that the enhancement curve was Gaussian. The temporal footprint (TF), used to characterize the length of the enhancement curves, was defined as the full width-half maximum (FWHM) of the Gaussian curves: $HM=2\sqrt{2ln2}\sigma$. The TF of the curves was varied between 0.5, 1, 2, 5, 7, 10, 15 and 20 seconds. This resulted in 120 unique numerical phantoms. These noiseless phantoms were multiplied by simulated coil sensitivity maps and sampled along a spiral trajectory. A fully sampled first frame was assumed and the phantom was reconstructed with PROUD and TRACER for comparison. For PROUD, the phantoms were reconstructed both with Eq. 1 and 2 to allow for analysis of the effects of temporal regularization. The phantoms allowed for the estimation of the temporal footprint and lag. The reconstructed curve was temporally shifted and compressed to maximize its cross-correlation with respect to the true simulated curve. The measured temporal footprint (TF_MEAS) of the reconstructed curve was taken as the true temporal footprint divided by the obtained compression factor. The measured temporal lag was defined as the obtained temporal shift (in sec). Differences in the amplitude of the measured curve compared to the true curve were not taken into account.

In vivo liver MRI experiments

In vivo data were acquired using a golden angle 3D dynamic multi-phase spiral LAVA sequence using a stack of variable density spiral trajectory (1). Five candidate liver donors imaged with this acquisition were selected retrospectively in accordance with an institutionally approved IRB protocol. The acquisition acquired data continuously for 60 sec, while the patients were given repeated breath-hold instructions. Other imaging parameters were: 1.5T (Ge Healthcare, Waukesha, WI), 48 variable density spiral leaves (density of 2 in the center of k-space, 0.7 at the periphery), acquisition matrix 256×256× (34–60), slice thickness 5 mm, FOV 32–48cm, bandwidth ±62kHz, fat suppression, 8 channel cardiac coil, and Magnevist/Eovist (Bayer Healthcare) injected at a dose based on patient weight. Since breathing motion was present, additional high SNR images created from all sampled data at 3 slices above and below the reconstructed slice were added to the dictionary D_t.

CNR was computed in vivo as

$$\mathit{CNR}=(S_{\mathit{aorta}}-S_{\mathit{portal}\mathit{vein}})/N_{\mathit{liver}}$$ [7]

where S_aorta and SNR_{portal vein} S_PV is the mean signal in a ROI in the aorta and portal vein, respectively, and N_liver N_L the signal standard deviation in an ROI of homogenous liver tissue near both the aorta and portal vein ROIs. This is computed for each time frame and is a surrogate measure for how well the arterial phase is visualized (13).

All PROUD and TRACER reconstructions were done either on a Dell Studio XPS 8100 (Intel i7 2.8 GHz processor, 16 GB RAM, Windows 7, Matlab R2011b) or a Dell PowerEdge R910 Server (64 cores, 64 GB RAM, Red Hat Linux, Matlab R2009a). Statistical significance (p <0.05) was determined by using a paired two-tailed student’s t-test in Microsoft Excel 2013.

RESULTS

Parameter selection

Optimal patch size was determined to be 7×7 pixels as it resulted in the smallest RMSE (Figure 3) and neighborhood size was then set to be 9×9 pixels.

Figure 3.

Root Mean Squared Error (RMSE) vs. patch size. 7×7 pixels was chosen to be the optimal patch size.

Phantom SNR test

Both algorithms provided similar contrast enhancement curves (Fig 4a). The PROUD aorta curve slightly preceded the TRACER aorta curve (Fig. 4a&b), but both lag the reference aorta curve. PROUD provided almost 3.66× higher peak CNR compared to TRACER (Fig. 4c). PROUD without the temporal regularization (denoted by PROUD γ = 0) obtained essentially the same peak CNR. The dip in the PROUD CNR was due to increased undersampling artifacts associated with rapid signal changes, such as the enhancing aorta and portal vein.

Figure 4.

Comparison of PROUD, PROUD γ = 0 (i.e., without temporal regularization) and TRACER in a numerical liver phantom (A) Average signal intensity of various regions of interest in the numerical phantom with noise standard deviation = 0.003. (B) Magnified region of interest showing the aorta enhancement curves around their peak enhancement. (C) CNR averaged over 30 noisy phantoms. The dip in CNR of the PROUD curve is due to increased reconstruction errors at the edges of the aorta and portal vein, which increase the standard deviation in the ROIs near the edges.

Phantom temporal footprint

Fig. 5 shows the results from the testing of the temporal response of PROUD compared to TRACER. For larger objects (>50% FOV) and longer temporal footprint (> 5 sec) – indicated by a black dot in Fig. 5 – the curves reconstructed with TRACER and PROUD had measured temporal footprints that were very close to those of the reference curves and temporal lags below 0.5 seconds.

Figure 5.

(A) Temporal (Temp.) lag and (B) measured temporal footprint (TF_MEAS) values determined using cross-correlations of the reconstructions from TRACER and PROUD compared to the true phantom curves with varying diameter (DIA) and temporal footprint (TF). PROUD γ = 0 is the solution to Eq. 1 and PROUD to Eq. 2. A black dot indicates the position corresponding to DIA>50% FOV and TF>5sec in each of the plots.

For all methods, temporal lag decreased for larger objects and increased for larger temporal footprint. Overall TRACER had a higher temporal lag than PROUD. In addition, there was a larger discrepancy between TRACER and PROUD for smaller objects (FOV 12.5%–43.8%). For larger objects (FOV = 81.3% and greater) with very short temporal events (0.5 – 1 sec temporal footprint), PROUD had a longer temporal lag than TRACER.

The measured temporal footprint for TRACER and PROUD were similar for objects of size 37.5% of the FOV and greater. The PROUD reconstruction using temporal regularization (Eq. 2) had a slightly larger measured temporal footprint compared to the PROUD reconstruction without (Eq. 1). In addition, for fast temporal events (~ 5 sec temporal footprint and less), there was some discrepancy in the methods. In general, for smaller objects TRACER had an increased measured temporal footprint. However, from objects sized equal or greater than 50% of the FOV, the temporal regularization of Eq. 2 made the measured temporal footprint larger than both TRACER and PROUD without the temporal regularization, with the largest difference around 1 sec. Finally for the three smallest objects (FOV = 12.5 – 25%), the measured TRACER temporal footprint is smaller than the true temporal footprint, when the latter exceeds 2 to 3 sec.

Fig. 6 shows examples for three of the curves analyzed in Fig. 5. In Fig. 6A for a short TF with small DIA, TRACER (TF_MEAS =7.5s, Temp. Lag =1.9s), PROUD No Temp. Constr. (TF_MEAS = 1.6s, Temp. Lag =1.6s) and PROUD w/ Temp. Constr. (TF_MEAS =2.4s, Temp. Lag =1.4s) had large errors in reconstructing the true curve. In Fig. 6B as the TF and DIA grow, TRACER (TF_MEAS =5.1s, Temp. Lag =1.8s), PROUD No Temp. Constr. (TF_MEAS = 5.1s, Temp. Lag =1.4s) and PROUD w/ Temp. Constr. (TF_MEAS = 5.3s, Temp. Lag =1.3s), had improved performance compared to the smaller DIA/TF. Finally in Fig. 6C with a large TF and DIA, all methods provided curves similar to the truth: TRACER (TF_MEAS = 20.0, Temp. Lag =0.1s), PROUD No Temp. Constr. (TF_MEAS = 20.0s, Temp. Lag =0.3s) and PROUD w/ Temp. Constr. (TF_MEAS =20.1s, Temp. Lag =0.3s).

Figure 6.

Examples of curves analyzed in Fig. 5. (A) A very short temporal event (TF = 0.5 sec) with a small diameter object (12.5% FOV). (B) A medium size temporal event (TF = 5 sec) with a mid-sized diameter object (50% FOV). (C) A long temporal event (TF = 20 sec) with a large diameter object (100% FOV).

In vivo testing

Over the 5 in vivo datasets, the images reconstructed using PROUD had a peak CNR of 42 ± 18 compared to 26 ± 12 using TRACER resulting in a 60% improvement in peak CNR (p = 0.008). Fig. 7 shows a single temporal frame reconstructed with the PROUD and TRACER methods in two patients. Improvements in SNR were seen using PROUD, while TRACER clearly suffered from increased noise in the reconstruction. The magnified regions of interest for each of the liver images indicates that PROUD allowed preservation of image detail. A video comparing PROUD and TRACER reconstructions in the same subject is available as online supplementary material.

Figure 7.

In vivo image reconstruction using PROUD and TRACER in two patients. Note the reduction in noise (increase in SNR) utilizing the PROUD method. The second row for each patient shows a magnified region of interest. Note that image details are preserved on PROUD compared to TRACER.

Fig. 8 shows the signal and CNR curves for patient in Fig. 7. In-vivo for Eq. 1, the preliminary pure Matlab (Natick, MA) implementation of PROUD took an average of 4.3 ± 0.1 minutes to reconstruct one temporal frame resulting in a total reconstruction time of 10.3 hours for 144 frames (as measured in one volunteer).

Figure 8.

(A) In vivo perfusion curves for aorta, portal vein and liver and (B) aorta—to—portal vein CNR curve for the same volunteer as in Fig. 7. A comparison is made between PROUD, PROUD γ = 0 (i.e., without temporal regularization) and TRACER.

DISCUSSION

The results presented in this work demonstrate the feasibility of using a combination of a patch based image constraint with a temporal regularization to provide improved CNR in phantoms while maintaining or improving temporal fidelity and to provide an increase in CNR of up to 60% in vivo compared to TRACER. This reduction in noise (as seen by the increase in CNR) can be attributed to the patch-based dictionary methods employed in the PROUD method. Patch level dictionary learning has previously been shown to be useful in de-noising problems (26,27) and has been shown to improve PSNR compared to a ground truth in MRI data (24).

Temporal characterization of PROUD against TRACER gave similar performance for larger objects and slower temporal events. For short temporal events or small objects there were some discrepancies between the methods. TRACER had increased temporal lag and somewhat larger measured temporal footprint with smaller object size at shorter temporal events. This may be due to differences in the reconstruction methods of TRACER and PROUD. Importantly, TRACER is subject to blurring of edges during fast signal changes. This is shown in Fig. 4d of (13), where the blurring of the aorta edge around peak aorta enhancement increased its apparent diameter. This blurring will effect smaller objects more and is most likely the cause of the increase in the measured temporal footprint and some of the temporal lag. This artifact was reduced in PROUD, however, since the dictionary-based method allows for a better delineation of edges during contrast changes. This is because this edge information is available in the image reconstructed from all available data which was used in the dictionary for each frame. In addition, with PROUD regularization is done on a local level while TRACER implements global image regularization. In our analysis, the method of determining the temporal lag and compression factor via compression and cross-correlation may be affected by the fact that response function in TRACER only is non-zero for positive time, see Fig. 3a in (13), while, due the temporal smoothness constraint, PROUD has a bi-directional temporal spread. These reasons may also explain the reason why TRACER exhibits an apparent negative lag for the three smallest objects (FOV = 12.5 – 25%), which is likely an artifact of the specific way in which lag was measured in this work. Overall it is important to note, that for both methods, temporal response was a factor of both the temporal event length and the size of the feature being evaluated.

Finally, in comparison to TRACER, PROUD is able to reduce residual temporal flickering artifacts due to the addition of Eq. 2. This temporal fidelity constraint forces neighboring frames to be similar over time therefore penalizing rapid temporal changes (e.g. flickering). There are differences to be noted however in the temporal performance of PROUD with Eq. 1 only and PROUD with Eq. 2. In general, Eq. 1 (and sometimes TRACER with larger objects) provided slightly better measured temporal footprint than Eq. 2. This is mainly due to the fact that the temporal term in Eq. 2. may introduce some degree of temporal blurring. The effect of temporal blurring was observed to be small, except in fast changing temporal signals. Improvement may be possible in future implementations replacing the L₂-norm on the temporal term with an L₁ norm, as was done in (23). It should be noted, however, that a small degree of temporal blurring is still present for large acceleration factors using this norm (23).

There were some limitations in this work. The size of the neighborhood in constructing the atoms for the dictionary was relatively small: 2 pixels larger than the patch size itself. This was done two reasons: 1) the assumption was made that the motion between frames was small, and 2) increasing the neighborhood increased the computational cost of the algorithm, since each patch in the unknown image would have to be compared with a larger number of atoms in the dictionary. In vivo validation was only done in subjects that were assumed to be healthy. Further studies may be warranted to evaluate the performance of PROUD in patients presenting with liver dysfunction (i.e. fibrosis, hepatocellular carcinoma, etc.). In addition, reconstruction time for the current pure Matlab implementation of PROUD was long (4.3 minutes to reconstruct one frame of one slice with Eq. 1). However, reconstruction time can be reduced through implementation in C code and GPU parallelization. Future implementations of PROUD may warrant automatic patch and neighborhood size determination, which could be done by analyzing the residual in Eq. 10a for varying patch/neighborhood size.

CONCLUSION

The proposed PROUD algorithm combines an image patch based regularization combined with a temporal regularization to enable high SNR, high temporal frame rate and spatial resolution 4D imaging that can improve in vivo peak CNR by up to 60% compared to previous methods while maintaining adequate temporal fidelity.

TABLE 1.

Symbols Used in the Text

t	Time index
i,j	Image space pixel locations
ki,kj	K-space pixel locations
vt	Image to be reconstructed at time t
Nx	Number of columns of vt
Ny	Number of rows of vt
n	Patch size (in pixels)
Ri,jvt	Selects an n × n patch centered around a pixel i,j
m	Neighborhood size (in pixels). m ≥ n
Dt	Image-based dictionary at time t
NR	Number of reference images used for the reconstruction of the dictionaries Dt
Pi,jvt−1	Selects all sets of NR patches of size n × n with an m × m neighbourhood centered at pixel i,j
yt	Multi-channel spiral k-space data acquired at time t
Nc	Number of RF coils used
si	Coil sensitivity map of the ith channel, i = 1, …, Nc
S	Operator that maps an image v to the coil images (siv)i = 1, …, Nc
F	Fourier transform
Ut	Under-sampling operator projecting Cartesian k-space on the spiral k-space acquired at time t
αi,j,t	Vector of dictionary coefficients for a patch at time t around pixel i,j
Ns	Number of spiral leaves needed for a Nyquist reconstruction
rcutoff	Relative change between successive solutions that will halt the iterative solver.
$N_{\mathit{iter}}^t$	Number of temporal iterations for solving Eq. 2.
TF	True temporal footprint (FWHM) of the numerical temporal phantoms
TFMEAS	Measured temporal footprint of the numerical temporal phantoms
DIA	Diameter (as a % of the FOV) of the central disk in the numerical temporal phantoms

APPENDIX A: PROUD Reconstruction Algorithm

v_t, α_i,j,t ← PROUD (y_t)

1. Reconstruct a reference v₀ using the first N_s spiral leaves

2. Reconstruct a composite image v_ALL using all spiral leaves

3. Compute the coil sensitivity maps (s_i)_{i = 1, … ,Nc} using v_ALL

4. Solve Eqs 3&4, obtaining initial estimates of $v_1^{\ast }$ and ${\alpha }_{i,j,1}^{\ast }$

5. Set λ such that the two terms in Eq. 1 are equal for the solutions in step 4.

6. Solve Eq. 1 for all time frames:

   for t =1, …, T

   $v_t^0\leftarrow v_{t-1}+S^H{F}^H{U}_t^H(y_t-U_t{\mathit{FSv}}_{t-1})$

   $v_{\mathit{prev}}\leftarrow v_t^0$ and k ← 0

   $r\leftarrow \Vert v_t^0-v_{\mathit{prev}}\Vert /\Vert v_t^0\Vert$

   Dt ← DICTIONARY (vt−1, vALL)

   while r ≥ rcutoff

   find ${\alpha }_{i,j,t}^k$ by fitting $v_t^k$ against the patch dictionary Dt using Eq. 10a,b

   create the patch averaged image $v_p^k\leftarrow {\sum }_{i,j}{R}_{i,j}^H{\alpha }_{i,j,t}^k{P}_{i,j}{D}_t$

   compute new image estimate $v_t^k$ using Eq. 9a

   $r\leftarrow \Vert v_t^k-v_{\mathit{prev}}\Vert /\Vert v_t^k\Vert$

   $v_{\mathit{prev}}\leftarrow v_t^k$ and k ← k + 1

   end

   $v_t\leftarrow v_t^k$ and ${\alpha }_{i,j,t}\leftarrow {\alpha }_{i,j,t}^k$

   end

7. Set γ such that the terms 1 and 3 in Eq 2 are equal for the solutions obtained in step 5.

8. Solve Eq. 2 using the solutions obtained in step 5 as initial guess:

   vt,1 ← vt and αi,j,t,1 ← αi,j,t

   for m = 2, …, $N_{\mathit{iter}}^t$

   for t =1, …, T

   $v_{t,m}\leftarrow v_{t-1,m}+S^H{F}^H{U}_t^H(y_t-U_t{\mathit{FSv}}_{t-1,m})$

   va ← 1/2(vt−1,m + vt+1,m−1)

   Dt,m ← DICTIONARY (vt−1,m, vALL)

   find αi,j,t,m by fitting vt,m against the patch dictionary Dt,m using Eq. 10a&b

   create the patch averaged image $v_{p,m}\leftarrow {\sum }_{i,j}{R}_{i,j}^H{\alpha }_{i,j,t,m}^k{P}_{i,j}{D}_{t,m}$

   compute new image estimate vt,m using Eq. 9b

   end

   end

   vt ← vt,m and αi,j,t ← αi,j,t,m

D ← DICTIONARY (v₁, …, v_NR)

1. From each v₁, …, v_NR, take a patch of size n × n around pixel (i,j) to form a set d_i,j.

2. Orthonormalize each set d_i,j

3. For every pixel (i,j) collect all sets d_a,b in an m × m neighborhood centered on pixel i,j

APPENDIX B

In the following, we detail the solver used for Eqs. 1&2. The method goes back and forward between solving for $v_t^{\ast }$ and solving for the patch weights α_i,j,t.

Eqs. 1&2 are solved by extending the method in (24) to the multi-channel non-Cartesian case. We start by assuming α_i,j,t to be fixed to some appropriate initial values. Then Eq. 1

$$v_t^{\ast }=\underset{v_t}{\text{argmin}}{\Vert U_t{\mathit{FSv}}_t-y_t\Vert }_2^2+\lambda \sum _{i,j}{\Vert R_{i,j}{v}_t-{\alpha }_{i,j,t}{P}_{i,j}{D}_t\Vert }_2^2$$

A solution for $v_t^{\ast }$ is obtained by setting the derivative of the cost function to zero, resulting in

$$[S^H{F}^H{U}_t^H{U}_{t}FS+\lambda \sum _{i,j}{R}_{i,j}^H{R}_{i,j}]v_t^{\ast }=S^H{F}^H{U}_t^H{y}_t+\lambda \sum _{i,j}{R}_{i,j}^H{\alpha }_{i,j,t}{P}_{i,j}{D}_t$$

Where $R_{i,j}^H{R}_{i,j}$ is equal to βI where β is a scaling factor equal to n², after appropriately taking into account patches at the edge of the FOV. Therefore:

$$[S^H{F}^H{U}_t^H{U}_{t}FS+\lambda \beta I]v_t^{\ast }=S^H{F}^H{U}_t^H{y}_t+\lambda \sum _{i,j}{R}_{i,j}^H{\alpha }_{i,j,t}{P}_{i,j}{D}_t$$

Note that S^H F^H FS = S^H S = I, by the fact that the Fourier transform is unitary and the fact that S^H S is a diagonal matrix with entries equal to ${\sum }_i\mid s_i^2\mid =1$ (by construction). Therefore

$$S^H{F}^H(U_t^H{U}_t+\lambda \beta I){\mathit{FSv}}_t^{\ast }=S^H{F}^H{U}_t^H{y}_t+\lambda \sum _{i,j}{R}_{i,j}^H{\alpha }_{i,j,t}{P}_{i,j}{D}_t$$

By applying FS on both sides, we obtain

$$(U_t^H{U}_t+\lambda \beta I){\mathit{FSv}}_t^{\ast }=U_t^H{y}_t+FS\lambda \beta v_p$$ [8]

where $v_p=\frac{1}{\beta }{\sum }_{i,j}{R}_{i,j}^H{\alpha }_{i,j,t}{P}_{i,j}{D}_t$ is the “patch averaged image”. Here $U_t^H{U}_t$ “zeroes out” any k-space that is not on the spiral trajectory corresponding to time frame t. By writing $FS\lambda \beta v_p=U_t^H{U}_{t}FS\lambda \beta v_p+(1-U_t^H{U}_t)FS\lambda \beta v_p$, we can apply the inverse of the diagonal matrix on both sides of Eq. 8:

$${\mathit{FSv}}_t^{\ast }=\frac{1}{1+\lambda \beta }(U_t^H{y}_t+U_t^H{U}_{t}FS\lambda \beta v_p)+\frac{1}{\lambda \beta }(1-U_t^H{U}_t)FS\lambda \beta v_p$$

$U_t^H{y}_t$ is simply the sampled k-space locations zero-padded to the matrix size. β can be incorporated into λ such that we can substitute λ ← β λ as in (24). This can be simplified to

$$v_t^{\ast }=v_p+\frac{1}{1+\lambda }{S}^H{F}^H{U}_t^H(y_t-U_t{\mathit{FSv}}_p)$$ [9a]

The added temporal regularization term in Eq. 2 similarly leads to:

$$v_t^{\ast }=\frac{\lambda v_p+\gamma v_a}{\lambda +\gamma }+\frac{1}{1+\lambda +\gamma }{S}^H{F}^H{U}_t^H\left(y_t-U_{t}FS\frac{\lambda v_p+\gamma v_a}{\lambda +\gamma }\right)$$ [9b]

Where $v_a=\frac{1}{2}(v_{t-1}^{\ast }+v_{t+1}^{\ast })$. To speed up convergence, new estimates of $v_t^{\ast }$ are used in the initialization and in the regularization term for subsequent time frames as soon as they become available.

Once an updated guess for an image v is available, the patch weights α_i,j,t are updated as follows. For every pixel (i,j), a set d^* of N_R patches is found that maximizes the projection of patch R_i,jv onto the set d^*:

$$d^{\ast }=\underset{d\in P_{i,j}D}{\text{argmax}}\mid A(d,R_{i,j}v)\mid$$ [10a]

where $A(d,R_{i,j}v)\stackrel{\text{def}}{=}{\sum }_{w\in d}{w}^H{R}_{i,j}v$. Note that, by construction, each set d ∈ P_i,jD is an orthonormal basis with N_R elements. Once the optimal patch set d^* is found, the patch weights are set equal to the linear coefficients in this basis:

$${\alpha }_{i,j}={\{w^H{R}_{i,j}v\}}_{w\in d^{\ast }}$$ [10b]