Deep Learning-Based Parameter Mapping with Uncertainty Estimation for Fat Quantification using Accelerated Free-Breathing Radial MRI

Abstract

Deep learning has been applied to remove artifacts from undersampled MRI and to replace time-consuming signal fitting in quantitative MRI, but these have usually been treated as separate tasks, which does not fully exploit the shared information. This work proposes a new two-stage framework that completes these two tasks in a concerted approach and also estimates the pixel-wise uncertainty levels. Results from accelerated free-breathing radial MRI for liver fat quantification demonstrate that the proposed framework can achieve high image quality from undersampled radial data, high accuracy for liver fat quantification, and detect uncertainty caused by noisy input data. The proposed framework achieved 3-fold acceleration to <1 min scan time and reduced the computational time for signal fitting to <100 ms/slice in free-breathing liver fat quantification.

1. Introduction

Fat quantification by chemical shift-encoded MRI is used to diagnose diseases such as non-alcoholic fatty liver disease. Chemical shift-encoded MRI requires image acquisition at multiple echo times (TEs), fitting the acquired data to a multiparametric nonlinear fat-water signal model, and calculation of proton-density fat fraction (PDFF, 0–100%) maps [1]. Conventional Cartesian MRI is sensitive to motion and requires breath-holding (10–25 sec), which can be challenging for some patients. Non-Cartesian radial MRI with improved motion robustness enables free-breathing liver fat quantification [1] but requires 2–5 min scans. In addition to challenges in acquisition, fat-water signal fitting is time-consuming, even with state-of-the-art graph-cut (GC) algorithms [2, 3], which have computational time on the order of 1–10 sec/slice.

MRI can be accelerated by acquiring an undersampled set of data, followed by iterative constrained reconstruction (e.g., compressed sensing [4]). Compared with these iterative algorithms, deep learning (DL)-based methods provide rapid inference time, which is preferred in clinical practice. The majority of research on DL-based MRI reconstruction or enhancement have focused on Cartesian undersampling [5, 6]. There are recent DL-based developments for undersampled radial MRI [7, 8], but not for multi-echo quantitative imaging.

The computational time for MRI fat mapping can also be substantially reduced using DL-based methods [9–11]. However, the confidence levels for DL-based parameter quantification accuracy were not explicitly characterized. Unlike errors in qualitative MRI contrast, quantification errors can be hard to detect by visual inspection and can have a direct impact on clinical decisions that rely on the pixel-wise numerical values. Recent developments for estimating uncertainty using DL [12] have the potential to address this crucial gap.

Thus far, DL-based reconstruction/enhancement of undersampled MRI and DL-based MRI parameter mapping have usually been treated as separate tasks, which does not fully exploit the shared information. In this work, we propose a new two-stage DL framework that performs image enhancement and parameter mapping with pixel-wise uncertainty estimation to accelerate free-breathing radial MRI scans to <1 minute and also reduce the computational time for fat mapping to <100 ms/slice.

2. Methods

2.1. Two-stage image-to-image-to-map (IIM) framework

We propose an IIM framework that includes an image enhancement stage to suppress the radial undersampling artifacts followed by a parameter mapping stage to reconstruct quantitative maps (Fig. 1). For the image enhancement stage, we used a 2D U-Net with a residual path [5], which improves recognition and removal of radial undersampling artifacts. A 5×5 convolutional filter size was used in the convolutional layers to increase the receptive field. Instance normalization [13] was used to handle contrast variations across subjects. Images acquired at multiple TEs were stacked along the channel dimension to exploit shared information (Fig. 1). This stage was pre-trained using mean squared error (MSE) loss.

Fig. 1.

The two-stage image-to-image-to-map (IIM) framework for quantitative liver fat mapping using accelerated free-breathing radial MRI. Black solid lines indicate how the framework generates results. Orange dashed lines indicate how the training samples were generated. Ndim: dimensions of the images/maps.

The parameter mapping stage learns a mapping from image enhancement results to quantitative maps (e.g., PDFF), with probabilistic outputs using a Gaussian model. We used a 2D U-Net structure with modified layers at the output (Fig. 1) to estimate pixel-wise means y and variances σ² of the target quantitative parameter. A sigmoid layer was added before the output of σ² to generate positive values. Following [12], we leveraged maximum a posteriori (MAP) inference and trained this stage to predict y and σ² by using the loss function ${\parallel y-\widehat{y}\parallel }^2/(N_c\bullet 2{\sigma }^2)+(1/2)\bullet log{\sigma }^2$, where y and $\widehat{y}$ denote the network output and the reference parameter maps, and N_c is the number of channels in the network output. During training, when the network output y had large differences versus the reference, the network increased σ², thus characterizing the uncertainty. During inference, we used the predicted means as the reconstructed parameter maps and the predicted variances as uncertainty maps.

For end-to-end training of the two-stage IIM framework, we used the loss function $({\parallel y-\widehat{y}\parallel }^2+\lambda {\parallel p-\widehat{p}\parallel }^2)/\left(\right(N_t+\lambda N_c)\bullet 2{\sigma }^2)+(1/2)\bullet log{\sigma }^2$, where p and $\widehat{p}$ denote the image enhancement output and the reference images, λ is the regularization parameter balancing errors associated with images and parameter maps, and N_t is the number of channels in the reference images.

2.2. Dataset and data processing

We acquired axial free-breathing multi-echo stack-of-radial (”FB radial”) data [1] from 68 adults (male/female=41/27) and 22 children (male/female=12/10) on 3T MRI scanners (MAGNETOM Skyra and Prisma, Siemens Healthcare, Erlangen, Germany), and retrospectively undersampled by a factor of R=3, in which a contiguous subset of radial readouts corresponding to 1/3 of the entire scan was extracted (Table 1). To augment training data, we included 3 sets of R=3 undersampled data from each scan. Fully sampled images (R=1) were defined based on Nyquist criteria with the number of radial spokes = matrix size in X·π/2. The subjects were separated into training, validation, and testing datasets with a 3/1/1 ratio, and each set had similar adult/children and male/female ratios (Table 2).

Table 1.

Representative radial MRI parameters

TE (ms)	1.23, 2.46, 3.69, 4.92, 6.15, 7.38
TR (ms)	8.85
Flip angle (degree)	5
Field of view (X × Y × Z)	360 × 360 × 200 mm3
Matrix size (X × Y × Z)	224 × 224 × 40
Radial spokes	353	118
Radial undersampling	N/A	R = 3-fold
Scan time (min:s)	2:05	0:42 *

(* indicates the equivalent scan time for undersampled data).

Table 2.

Datasets

Dataset	# of Adults	# of Children	Total 2D slices
Training	15F, 25M	6F, 8M	6756*
Validation	6F, 8M	2F, 2M	792
Testing	6F, 8M	2F, 2M	754

(*training set includes R=3 sets of undersampled data from each subject). F: female, M: male.

R=1 and R=3 images were reconstructed using the non-uniform fast Fourier transform (NUFFT), coil-combined [14], and normalized based on the mean signal intensities in the first-echo image in each slice. We calculated reference PDFF maps from R=1 FB radial data using GC algorithms [2, 3] to fit the signals from the 6 TEs to a 7-peak fat model [15] with single effective $R_2^{*}$ per voxel. We also used body masks to mask out the background in the reference PDFF maps.

2.3. Two-stage IIM framework training

The two stages were first pre-trained separately by using (1) pairs of 2D 6-echo R=3 and R=1 images for the image enhancement stage, and (2) pairs of R=1 images and corresponding fat (PDFF) and water fraction (1-PDFF) maps from R=1 + GC for the parameter mapping stage. Next, we performed end-to-end training of the IIM framework using sets of R=3 images, R=1 images, and corresponding fat/water fraction maps from R=1 + GC. Hyperparameters were chosen as batch size=16, learning rate=0.01 (halved every 25 epochs), and number of epochs=100. The regularization parameter was set to λ = 5 to balance image quality and quantification accuracy based on validation results. All models were trained using the Adam optimizer.

2.4. Two-stage IIM framework evaluation

We evaluated our IIM framework in three aspects. (1) Image quality: We compared the structural similarity index (SSIM) and normalized root mean squared error (NRMSE) of the IIM image enhancement stage results versus NUFFT for R=3 data, with respect to R=1 reference images. (2) PDFF quantification: We compared the PDFF quantification accuracy of our IIM framework to a single-stage 2D U-Net that directly reconstructs parameter maps from undersampled images (“single U-Net”). We chose the single U-Net for comparison because it was previously used for Cartesian MRI fat mapping [9, 10]. The single U-Net was trained using the same pairs of R=3 images and reference fat/water fraction maps (R=1 + GC). To measure liver PDFF, we manually placed 5cm² circular regions of interest (ROIs) on an axial mid-liver slice in each subject while avoiding large vessels and bile ducts. We used Bland-Altman analysis to calculate the mean difference and 95% limits of agreement (LoA) of the IIM and single U-Net results using R=3 images, compared to R=1 + GC. (3) Uncertainty estimation: We added artificial noise patterns in the R=3 input images in the testing set and analyzed the relative intensity increase on the IIM uncertainty maps. Three different levels of zero-mean white additive Gaussian noise were added to the circular liver ROIs in the input images, with noise variances/mean signal intensity = 6.77%, 10%, 20%.

3. Results

Representative image enhancement results from the IIM framework are depicted in Fig. 2. Compared with R=3 +NUFFT, IIM suppressed radial undersampling streaking artifacts outside the body and inside the liver (red arrows in Fig. 2). SSIM and NRMSE of the out-of-phase (echo 1, 3 and 5) and in-phase (echo 2, 4 and 6) images are reported (Table 3). The IIM framework achieved average SSIM>0.86 and NRMSE<0.12 for R=3, which were significantly better than R=3 + NUFFT (p<0.01, Wilcoxon signed-rank test).

Fig. 2.

Magnitude/phase images from echo 1 (out-of-phase) and echo 2 (in-phase) for fully sampled data (R=1), R=3 undersampled data with NUFFT, and R=3 data after image enhancement by the IIM framework from a 37-year-old male. Red arrows point to radial undersampling artifacts that are suppressed by the IIM framework.

Table 3.

Image quality in the testing set

Metric	Image type	R=3+NUFFT	R=3+IIM
SSIM	Out-of-phase	0.766 ± 0.107	0.869 ± 0.048*
SSIM	In-phase	0.847 ± 0.063	0.886 ± 0.049*
NRMSE	Out-of-phase	0.140 ± 0.102	0.112 ± 0.040*
NRMSE	In-phase	0.125 ± 0.081	0.103 ± 0.044*

(* p<0.01 for NUFFT vs. IIM using Wilcoxon signed-rank test).

Representative PDFF maps reconstructed using different methods are shown in Fig. 3. IIM had sharper features, less blurring, and more accurate liver PDFF compared with the single U-Net. Bland-Altman analysis of the liver PDFF measurements (Fig. 4) show that R=3 + IIM achieved close agreement (mean difference=0.24%) with R=1 + GC, and had a tighter 95% LoA compared with R=3 + single U-Net.

Fig. 3.

PDFF maps using (a) GC algorithms with R=1 data, (b) single U-Net with R=3 data, and (c,d) proposed IIM with R=3 data from the same subject in Fig. 2. The IIM results have less blurring and more accurate liver PDFF (red circle), compared to single U-Net. The red arrows point to low signal-to-noise ratio regions (air in the lungs) with unreliable PDFF, which is captured by the IIM uncertainty map. (For uncertainty maps, we used a color map from [16] that is designed for visualization of scientific data.)

Fig. 4.

Bland-Altman plots comparing liver PDFF measurements from different methods (MD: mean difference; LoA: 95% limits of agreement).

Uncertainty estimation analysis results are shown in Fig. 5. The IIM image enhancement results and PDFF maps have noise corruption at the location where noise was added on the input images, and the uncertainty map had high intensities at the same location, which identified the unreliability. Results using different levels of added noise were compared (Fig. 6). As more noise was added (higher noise variance), the intensities in the uncertainty map increased accordingly and characterized the diminishing confidence for the PDFF results at the corresponding location.

Fig. 5.

(a, b) Reference R=1 image and PDFF map using GC, (c) R=3 image with artificially added noise patch (noise variance/mean signal intensity = 20%, red arrow), and (d-f) corresponding IIM results.

Fig. 6.

Relative increase in the IIM uncertainty map results (%) when different levels of noise variance were added to the R=3 input images in the testing set. IQR: interquartile range.

The proposed two-stage IIM framework required 20 hours to train (including pre-training) on a NVIDIA v100 GPU. The average inference time was 65.79 ms/slice on this GPU for image enhancement and calculation of fat/water fraction and uncertainty maps. On the other hand, GC algorithms took ~28 sec/slice on an Intel Xeon E5–2660 CPU.

4. Discussion and Conclusions

We developed a new DL two-stage IIM framework that suppresses radial MRI undersampling artifacts and reconstructs accurate PDFF maps for 3-fold scan acceleration, which corresponds to <1 min free-breathing scan time. This positions FB radial MRI as a compelling alternative to standard breath-holding MRI (10–25 sec). The balance between scan time and IIM reconstruction fidelity for higher undersampling factors warrants additional research.

Compared with a single U-Net, our two-stage IIM used shared information between images and maps to achieve sharper spatial features and less blurring in the PDFF maps, based on undersampled input data. Depending on implementation and hardware, computational time for GC fat mapping is on the order of 1–10 sec/slice. IIM reduced the computational time for fat mapping by two orders of magnitude versus GC, and would enable rapid processing for clinical applications.

Compared with previous work on DL-based fat mapping [9–11], our work demonstrates accurate PDFF mapping from undersampled MRI, and our framework generates additional uncertainty maps to provide more context for using DL in quantitative MRI. In regions with low signal-to-noise ratio (e.g., air), DL-based methods that use MSE-based loss tend to output the mean of the PDFF range (~50%), which is misleading. With our IIM framework, the uncertainty maps highlight the unreliability (red arrows in Fig. 3). In our controlled experiments, the IIM uncertainty maps had higher values at locations with more noise in the input data, and thus detected locations with low confidence in the PDFF results.

The underlying network architecture in each stage of the proposed two-stage IIM framework could be adjusted. More complex network architectures (e.g., unrolled networks [17]) could be investigated in the future. In addition to PDFF mapping, the proposed framework could be adapted to different parameter mapping applications (e.g., T₁, T₂) with modifications in the corresponding input and output data formats.

There are limitations in this work. First, there was a limited amount of training and testing data. Second, the datasets were acquired during free-breathing radial MRI and motion can cause residual streaking artifacts. Self-gating [18] can be applied to compensate motion prior to network training and testing, or incorporated into the IIM framework. Third, uncertainty estimation using MAP inference only captures data uncertainty (i.e., aleatoric uncertainty). Other types of uncertainty, such as model uncertainty [12], could be investigated in the future. Fourth, the current IIM framework only generates relative uncertainty maps. Direct prediction of parameter quantification errors will be the next step.

In conclusion, the IIM framework achieved 3-fold scan acceleration with <100 ms/slice computational time for free-breathing radial MRI fat quantification.