Uncertainty-Aware Physics-Driven Deep Learning Network for Free-Breathing Liver Fat and R₂* Quantification using Self-Gated Stack-of-Radial MRI

Abstract

Purpose:

To develop a deep learning-based method for rapid liver proton-density fat fraction (PDFF) and R₂* quantification with built-in uncertainty estimation using self-gated free-breathing stack-of-radial MRI.

Methods:

This work developed an uncertainty-aware physics-driven deep learning network (UP-Net) to (1) suppress radial streaking artifacts due to undersampling after self-gating, (2) calculate accurate quantitative maps, and (3) provide pixel-wise uncertainty maps. UP-Net incorporated a phase augmentation strategy, generative adversarial network architecture, and an MRI physics loss term based on a fat-water and R₂* signal model. UP-Net was trained and tested using free-breathing multi-echo stack-of-radial MRI data from 105 subjects. UP-Net uncertainty scores were calibrated in a validation dataset and used to predict quantification errors for liver PDFF and R₂* in a testing dataset.

Results:

Compared with images reconstructed using compressed sensing (CS), UP-Net achieved structural similarity index >0.87 and normalized root mean squared error <0.18. Compared with reference quantitative maps generated using CS and graph-cut (GC) algorithms, UP-Net achieved low mean differences (MD) for liver PDFF (−0.36%) and R₂* (−0.37 s⁻¹). Compared with breath-holding Cartesian MRI results, UP-Net achieved low MD for liver PDFF (0.53%) and R₂* (6.75 s⁻¹). UP-Net uncertainty scores predicted absolute liver PDFF and R₂* errors with low MD of 0.27% and 0.12 s⁻¹ compared to CS+GC results. The computational time for UP-Net was 79 ms/slice, while CS+GC required 3.2 min/slice.

Conclusion:

UP-Net rapidly calculates accurate liver PDFF and R₂* maps from self-gated free-breathing stack-of-radial MRI. The pixel-wise uncertainty maps from UP-Net predict quantification errors in the liver.

1. INTRODUCTION

Chronic liver disease is a global health burden^1–3. Liver disease is characterized by histological changes that include hepatic steatosis, inflammation, fibrosis, and iron deposition^4–10. Progressive liver disease is associated with cirrhosis and hepatocellular carcinoma, and can culminate in liver failure¹¹. Biopsy is considered the standard technique for diagnosing liver diseases. However, biopsy suffers from sampling bias, is invasive, and is associated with complications¹².

MRI evaluates hepatic steatosis and iron overload by quantifying proton-density fat fraction (PDFF) and R₂*, using chemical-shift-encoded multi-echo Dixon techniques that acquire and fit data to a signal model that accounts for the multi-peak fat spectrum and R₂* component¹³. Conventional Dixon techniques using a multi-echo 3D Cartesian sequence¹⁴ are sensitive to motion and require breath-holding (10–20 sec). The breath-holding requirement limits the volumetric coverage and resolution, and can be challenging for patients¹⁵. Recently, a multi-echo 3D stack-of-radial technique^16,17 has been developed for free-breathing liver PDFF and R₂* quantification and demonstrated accurate results in subjects with non-alcoholic fatty liver disease (NAFLD)¹⁸. To compensate for respiratory motion in free-breathing radial data acquisition, self-gating is used to reconstruct images from a subset of data with consistent motion behavior. However, motion self-gating introduces radial undersampling artifacts in the images and quantitative maps. These artifacts can be mitigated by acquiring more radial spokes¹⁷ or using constrained reconstruction¹⁹, but these strategies require a longer acquisition and/or computational time.

Accurate and rapid signal fitting is another challenge in PDFF and R₂* quantification. Due to the non-convex structure of the signal model and ambiguities in resonant frequencies of water/fat protons with respect to B₀ field variations, signal fitting can converge to a local minimum solution and lead to fat-water swaps. State-of-the-art graph-cut (GC)-based methods^20,21 impose smoothness constraints on the field map and use optimization algorithms to reduce the occurrence of fat-water swaps. However, the GC-based algorithms are computationally expensive with computation time on the order of 10 sec/slice²¹.

Compared with iterative constrained reconstruction methods for MRI, such as compressed sensing (CS)^22,23, deep learning (DL)-based methods can rapidly enhance or reconstruct images from undersampled data by leveraging datasets from prior scans. Previous studies have developed novel DL networks for MRI enhancement or reconstruction from undersampled Cartesian data^24,25. Although there were DL networks developed for undersampled radial MRI^26–29, there is a lack of investigation regarding multi-echo radial MRI for PDFF and R₂* mapping. On the other hand, DL has also been used to replace the computationally expensive fat-water signal fitting process. Different network architectures^30–33 and loss functions³⁴ have been proposed to separate fat/water signals or generate PDFF/R₂* maps. However, these methods only investigated fully-sampled Cartesian data and did not consider radial acquisition or data undersampling.

Developing, evaluating, and translating DL-based methods for quantitative MRI parameter mapping can be challenging because quantification errors can be difficult to detect by visual inspection. Confidence levels of quantification accuracy from the DL network outputs were not typically characterized in previous studies^30–34. Recently, there have been initial developments in incorporating uncertainty estimation in the DL networks for MR image reconstruction^35–37. These works showed promise by investigating the relationships between estimated uncertainty scores and reconstruction errors. A recent work showed promising results of using uncertainty estimation for quantitative MRI PDFF maps obtained from DL³⁸. The study indicated that the uncertainty scores were related to the noise levels in the input data. However, the relationship between the uncertainty scores and quantification accuracy was not established.

In this work, we developed an uncertainty-aware physics-driven deep learning network (UP-Net) that can rapidly calculate accurate liver PDFF and R₂* maps using multi-echo images from undersampled self-gated free-breathing stack-of-radial MRI data. UP-Net simultaneously (1) suppressed radial streaking artifacts due to undersampling after self-gating, (2) calculated accurate quantitative liver PDFF and R₂* maps, and (3) provided pixel-wise uncertainty maps for each quantitative parameter within a rapid inference time <100 ms/slice. We calibrated the UP-Net uncertainty scores and demonstrated the ability to predict liver PDFF and R₂* quantification errors using the uncertainty scores.

2. METHODS

2.1. Uncertainty-Aware Physics-Driven Deep Learning Network (UP-Net)

We proposed UP-Net (Figure 1) to generate accurate quantitative maps from undersampled 2D multi-echo images and provide pixel-wise uncertainty maps which can be used to predict quantification errors. UP-Net contained two concatenated network modules for artifact suppression and parameter mapping. The first module took 2D multi-echo undersampled images x as the input and generated enhanced 2D multi-echo images $\widehat{m}$ with suppressed undersampling artifacts. For x and $\widehat{m}$, multi-echo images, including both the real and imaginary components, were stacked along the channel dimension. The second module transformed $\widehat{m}$ to quantitative parameter maps $\widehat{p}$ and their corresponding uncertainty maps $\widehat{u}$. In our case of multi-parameter fitting, $\widehat{p}$ and $\widehat{u}$ are 3D tensors where different 2D quantitative maps are stacked along the channel dimension. UP-Net requires reference multi-echo images and reference quantitative maps for training. Details regarding reference data generation are described in section 2.5.

Figure 1.

The proposed uncertainty-aware physics-driven deep learning network (UP-Net) for rapid free-breathing proton-density fat fraction (PDFF) and R₂* quantification from self-gated multi-echo stack-of-radial MR images. The artifact suppression module used a generative adversarial network (GAN) architecture to reduce the radial undersampling artifacts due to self-gating. The parameter mapping module used a bifurcated UNet structure, which had a shared encoder and two decoders, to calculate parameter maps (pixel-wise means) and uncertainty maps (pixel-wise variances). See Supplementary Figure S1 for more details. NUFFT: non-uniform fast Fourier transform.

Convolutional neural networks (ConvNet) have been proposed to effectively suppress artifacts from undersampling^24–29. Recently, there are works showing that generative adversarial networks (GAN) can improve the quality of the reconstructed images for radial MRI compared to conventional ConvNet³⁹. GAN uses a discriminator to help generate images that closely resemble the reference images and match the intended data distribution, which can improve the image reconstruction or artifact suppression results. We investigated a GAN architecture for the artifact suppression module and decided to use it for UP-Net based on results in an ablation study (see section 2.8). The generator was implemented using a 2D UNet architecture⁴⁰, and the discriminator was implemented using the architecture proposed in⁴¹. To deal with image contrast variation across subjects, instance normalization⁴² was used in both the generator and the discriminator. A detailed diagram for the UP-Net implementation is presented in Supplementary Figure S1.

We considered the quantitative parameter output as distributions which can be characterized using pixel-wise means $\widehat{p}$ and pixel-wise variances $\widehat{u}$ from a Bayesian perspective^43,44. We interpreted $\widehat{p}$ and $\widehat{u}$ as the quantitative maps and the corresponding uncertainty maps. For each pixel index j, a larger value of $\widehat{u_j}$ indicates a wider spread of the distribution and therefore the associated $\widehat{p_j}$ has higher uncertainty. By assuming a prior data distribution, the network can be trained to predict $\widehat{p}$ and $\widehat{u}$ simultaneously using the loss function introduced in section 2.2. In light of the deeply correlated nature of $\widehat{p}$ and $\widehat{u}$, we used a “bifurcated UNet” architecture (Figures 1 and S1) for the parameter mapping module. This architecture has one shared encoder that extracts features from multi-contrast images $\widehat{m}$, and two separate decoders that generate parameter maps $\widehat{p}$ and uncertainty maps $\widehat{u}$. Because the uncertainty score, or the variance of a distribution, should always be nonnegative, a softplus layer (Softplus(x) = log(1 + e^x)) was added prior to the uncertainty map output.

2.2. Loss Function for UP-Net Training

We constructed a loss function with 5 components for supervised training of UP-Net:

$$L_{UP-Net}=w_1{L}_{imgMSE}+{w_{2}L}_{imgGAN}+w_3{L}_{mapMSE}+w_4{L}_{physics}+w_5{L}_{uncert}$$ (1)

An image mean squared error (MSE) loss was used to measure the errors between enhanced ($\widehat{m}$) and reference (m) multi-echo images:

$$L_{imgMSE}=\frac{1}{N_j}\sum _j{\left({\widehat{m}}_j-m_j\right)}^2$$ (2)

, where j represents the pixel index and N_j is the total number of pixels in the multi-echo images. We trained the GAN architecture using a Wasserstein GAN loss⁴⁵, which can be formulated as:

$$\underset{G}{\min }\underset{D}{\max }{\mathbb{E}}_{m~p_{train}\left(m\right)}\left[D\left(m\right)\right]-{\mathbb{E}}_{\widehat{m}~p_G(\widehat{m})}[D(G(\widehat{m}))]$$ (3)

, where G represents the generator, D represents the discriminator. The loss for updating the generator G was:

$$L_{imgGAN}={\mathbb{E}}_{\widehat{m}~p_G(\widehat{m})}[D(G(\widehat{m}))]$$ (4)

We also used an MSE loss that measures the errors between quantitative maps from UP-Net ($\widehat{p}$) and reference data (p):

$$L_{imgMSE}=\frac{1}{N_j}\sum _j{({\widehat{p}}_j-p_j)}^2$$ (5)

To promote learning of the signal fitting process we used an MRI physics loss:

$$L_{physics}=\frac{1}{N_j}{(\widehat{m}-Q\left(\widehat{p}\right))}^2$$ (6)

where Q represents an operator that transforms the quantitative maps to multi-echo images based on the MRI signal equation. In this work where we investigated PDFF and R₂* quantification, the operator Q we used was:

$$Q\left(\widehat{p}\right)=Q(W,F,R_2^{\mathrm{*}},\phi ,TE)=\left(W+F·\left(\sum _{m=1}^M{a}_m·e^{i2\pi f_{m}TE}\right)\right)·e^{-R_2^{\mathrm{*}}TE}·e^{i2\pi \phi TE}$$ (7)

where W, F, $R_2^{\mathrm{*}},$ φ represent the quantitative water maps, fat maps, R₂* maps, and B₀ field maps. A 7-peak fat model with amplitudes a_m and frequencies f_m were also included⁴⁶.

To predict quantitative parameter outputs with corresponding uncertainty scores, we used an uncertainty loss:

$$L_{uncert}=\frac{{‖\widehat{p}-p‖}_1}{\widehat{u}}+\mathit{log}\left(\widehat{u}\right)$$ (8)

This uncertainty loss function is equivalent to performing maximum a posteriori (MAP) inference where a Laplace distribution⁴³ is assumed for each quantitative parameter in each pixel. We can also understand this loss function from a more intuitive perspective. First, in regions where the ${‖\widehat{p}-p‖}_1$ error minimization is difficult (e.g., regions with lower signal-to-noise ratio), increased values of $\widehat{u}$ can reduce the loss, therefore capturing uncertainty. Second, the $log\left(\widehat{u}\right)$ term can serve as a regularization term to avoid unconstrained increase in the uncertainty score.

The relative weights for each loss component in Equation 1 can impact the results. We chose the weight combination that achieved the lowest PDFF and R₂* quantification errors in the validation set: w₁ = 0.2, w₂ = 0.2, w₃ = 0.2, w₄ = 0.3, and w₅ = 0.1.

2.3. Training Strategy for UP-Net

To shorten the convergence time for training UP-Net, we used a step-by-step training strategy.

Step 1:

Pre-train the artifact suppression module using pairs of input undersampled images x and reference images m as the training data and using only L_imgMSE and L_imgGAN for the loss function.

Step 2:

Pre-train the parameter mapping module without the uncertainty estimation path using pairs of reference multi-contrast images m and reference quantitative maps p as the training data and using only L_mapMSE and L_physics for the loss function.

Step 3:

Load the weights trained from Steps 1 and 2, and then train the entire UP-Net end-to-end without the uncertainty path using L_imgMSE, L_imgGAN, L_mapMSE and L_physics for the loss function (i.e., not including the uncertainty loss term yet).

Step 4:

Train the entire UP-Net using training sets of undersampled images x, reference images m and reference quantitative maps p with the full loss function L_UP–Net.

For all training steps, data augmentation for images/maps was performed by mirroring (no flip or horizontal flip) and rotating by n*90 degrees (n=0,1,2,3). This augmented the training data size to 8 times that of the original training data. In addition, we used a “phase augmentation” strategy to further augment training data by adding a phase offset to the multi-echo input images, multi-echo reference images, and reference fat and water complex signals at the same time (Supplementary Figure S2). The signal magnitudes were not changed, and the relationship between images and quantitative maps were not modified. In each epoch during training, we generated 3 more instances for each 2D slice in this manner (i.e., in addition to the original data, 3 different phase offsets were applied to generate 3 more instances). The phase offsets were randomly selected between 0~2π. This strategy aimed to improve robustness to phase variations, which is important in separating fat/water signals.

2.4. Data Acquisition

In a HIPAA-compliant and IRB-approved study, we acquired MR images from 105 subjects, including healthy subjects and subjects with suspected or confirmed NAFLD, at 3T (MAGNETOM Skyra or Prisma, Siemens Healthineers, Erlangen, Germany). Fifty-seven of the subjects were adults (34 females, 23 males; age 48.16±19.01 years; body mass index [BMI]: 26.98±5.94kg/m²) and 48 of the subjects were children (19 females, 29 males; age 13.06±2.99 years; BMI: 22.85±8.41kg/m²). Written informed consent, parental permission, and assent, if applicable, were obtained for all subjects prior to research procedures. We scanned using a prototype free-breathing multi-echo gradient-echo 3D stack-of-radial sequence with bipolar readout gradients (FB Radial, parameters in Table 1)^16,17. To compare with standard breath-holding techniques, we acquired an additional breath-hold bipolar multi-echo gradient-echo 3D Cartesian sequence (BH Cartesian, parameters in Table 1)¹⁴. We separated the data into training (N=63), validation (N=21), and testing (N=21) datasets using a 3:1:1 ratio (Table 2). Subject information and data were entered into a secure database for management and analysis⁴⁷.

Table 1.

Representative sequence parameters for free-breathing 3D stack-of-radial (FB Radial) and breath-holding (BH) 3D Cartesian axial MRI scans at 3T. N/A: not applicable.

Sequence parameters	FB Radial	BH Cartesian
TE (ms)	1.23, 2.46, 3.69, 4.92, 6.15, 7.38
TR (ms)	8.85
Flip angle (°)	5	5
Field of view	360–440 × 360–440 mm2	360–440 × 360–440 mm2
Slice thickness (mm)	5	5
Matrix size (x, y, z)	224–288, 224–288, 40–72	224–288, 224–288, 30–40
Acceleration factor	N/A	R=4 (parallel imaging)
Radial spokes	354–454	N/A
Scan time (min:sec)	2:28 – 4:49*	0:19#
Retrospective undersampling	R=2.5 (40% self-gating acceptance rate)	N/A

^*Radial gradient calibration time was not included.

^#Prescan calibration time was not included.

Table 2.

Dataset characteristics. NAFLD: non-alcoholic fatty liver disease. Std: standard deviation.

Datasets	Total number of subjects	Adult subjects	Pediatric subjects	Total number of 2D slices	Range of liver PDFF values
Training set	63	24 NAFLD, 11 healthy	18 NAFLD#, 10 healthy	2528*	Min: 0.4% Max: 33.4% Mean: 10.5% Std: 9.7%
Validation set	21	7 NAFLD, 4 healthy	6 NAFLD#, 4 healthy	812	Min: 0.9% Max: 28.4% Mean: 9.2% Std: 8.8%
Testing set	21	7 NAFLD, 4 healthy	5 NAFLD#, 5 healthy	860	Min: 0.6% Max: 25.2% Mean: 9.8% Std: 8.4%

^*Before performing data augmentation. See text in section 2.3 for details about data augmentation.

^#Suspected or confirmed NAFLD.

We trained and tested UP-Net using only FB Radial data, while the BH Cartesian data served as an external reference for evaluation of PDFF and R₂* quantification accuracy. For BH Cartesian data, images and the quantitative PDFF and R₂* maps were reconstructed using vendor-provided software on the scanner.

2.5. Reference Data Preparation for UP-Net Training

UP-Net was trained in a supervised approach, which demands high-quality multi-echo images and quantitative maps with minimal artifacts to serve as references. However, it is challenging to acquire fully-sampled data for reconstruction of motion-resolved 3D volumetric images in the abdomen. To satisfy the Nyquist sampling criteria after self-gating, longer acquisition time is needed, which may increase sensitivity to motion effects. Previous works have used CS to generate images and quantitative maps for DL network training in applications where a fully-sampled reference dataset is difficult to acquire^48,49. Following a similar strategy, we acquired nominally fully-sampled stack-of-radial data before applying motion self-gating, and used CS to reconstruct motion self-gated images with suppressed undersampling artifacts.

The workflow for generating the training data for UP-Net is shown in Figure 2. Gradient delays were calibrated to correct the radial trajectory for FB Radial data reconstruction¹⁶. We extracted a projection-based self-navigator from the k_x = k_y = 0 line in k-space²³ to track respiratory motion along the z dimension. A sliding window approach was applied along the motion dimension to bin the k-space data into 6 respiratory motion states where each bin contained 40% of the entire k-space data (effective data undersampling factor = 2.5 in each state). We estimated coil sensitivity maps using a phased array beamforming technique^50,57, which has been shown to suppress radial artifacts resulting from hardware imperfections (e.g., gradient non-linearity and field inhomogeneity). We formulated the 2D CS reconstruction problem as²³:

$$x^{\mathrm{*}}={argmin}_x{‖FSx-y‖}_2^2+{\lambda }_1{TV}^{motion}\left(x\right)+{\lambda }_2{\sum }_{echo,state}{‖Wavelet\left(x_{echo,state}\right)‖}_1$$

, where F represents the non-uniform fast Fourier Transform (NUFFT) operator, S denotes coil sensitivity maps, x is the reconstructed multi-echo images, y is the acquired k-space data, and λ₁ and λ₂ are regularization parameters. The regularization parameters were chosen manually to balance between undersampling artifact reduction and image sharpness. After CS reconstruction, we calculated quantitative maps (including complex fat/water components, R₂* map, and B₀ field map) by fitting the CS-reconstructed multi-echo images to a 7-peak fat model with a single R₂* component⁴⁶ (same as Eq. 7) using GC-based algorithms^20,21. Local fat-water swaps still occurred in certain slices and were difficult to correct using GC-based algorithms; we excluded these slices from the training dataset. We generated body masks from the first-echo CS-reconstructed magnitude images, and applied the body masks to the CS-reconstructed images and the corresponding quantitative maps for background artifact and noise suppression. We will refer to the reference CS-reconstructed self-gated free-breathing stack-of-radial images as FB+CS and the corresponding quantitative maps reconstructed by GC-based algorithms as FB+CS+GC.

Figure 2.

The workflow for generating reference data (multi-echo images and quantitative maps) for training UP-Net. Nominally fully-sampled stack-of-radial k-space data were binned into 6 respiratory motion states using projection-based self-navigators. A 2D compressed sensing (CS) framework with beamforming-based coil sensitivity maps was used to reconstruct multi-echo images with reduced undersampling streaking artifacts. Quantitative maps were generated by fitting the multi-echo images to a fat-water signal model with a single R₂* component. Body masks were generated from the CS-reconstructed first-echo images for background suppression.

The input images x to UP-Net were coil-combined 6-echo images using 40% of FB Radial data near the end-expiration state (Figure 1). The real and imaginary components from each echo were stacked along the channel dimension (6 echoes × real/imaginary components = 12 channels). The output from the artifact suppression module had the same data dimensions as the input images (12 channels), and were fed into the parameter mapping module. The output from the parameter mapping module contained (1) complex-valued fat and water components, R₂* map, and field map stacked along the channel dimension and (2) three uncertainty maps for PDFF, R₂*, and field map stacked along the channel dimension. PDFF maps were generated from complex fat/water components for calculation of L_mapMSE and L_uncert, while the complex-valued fat/water components were directly used in L_physics.

Based on PDFF and R₂* quantification accuracy in the validation dataset, the hyperparameters for the end-to-end UP-Net training were chosen as: batch size=32, initial learning rates=0.0001, and epochs=150, using the Adam optimizer.

2.6. Evaluation of UP-Net Image Quality and Quantification Accuracy

We evaluated the performance of UP-Net in terms of image quality and quantification accuracy of the output images and maps in the testing dataset. For image quality, we compared the enhanced image results from UP-Net with the reference FB+CS images using normalized root mean squared error (NRMSE) and structure similarity index (SSIM). For quantification accuracy, we calculated differences in PDFF and R₂* quantification results using liver regions of interest (ROIs) for (1) FB+UP-Net versus FB+CS+GC and (2) FB+UP-Net versus BH Cartesian. ROIs with area of 5-cm² were placed in the right lobe of the liver by a trained researcher while avoiding large vessels and bile ducts¹⁸. A total of 3 ROIs were placed in the upper, middle and lower liver (one ROI at each level) for each subject. Bland-Altman analysis was performed to evaluate PDFF and R₂* accuracy by calculating the mean difference (MD) and 95% limits of agreement (LoA) between different methods.

2.7. Evaluation of UP-Net Uncertainty Estimation

We evaluated the performance of UP-Net uncertainty estimation in terms of its ability to predict quantification errors according to the following steps.

Step 1:

Complete UP-Net training.

Step 2:

In the validation dataset, measure (1) quantification errors (${\sum }_{i\in ROI}\left|{\widehat{p}}_i-p_i\right|$) between UP-Net and reference FB+CS+GC results and (2) UP-Net uncertainty scores in the liver ROIs. Use a linear correlation model to generate “calibration curves” between quantification errors and UP-Net uncertainty scores for each quantitative parameter separately. Calculate Spearman correlation coefficients and test for statistical significance.

Step 3:

Output UP-Net uncertainty scores for liver ROIs in the testing dataset. Transform UP-Net uncertainty scores to predicted quantification errors using the calibration curves.

Step 4:

Perform Bland-Altman analysis on predicted quantification errors versus actual quantification errors in the testing dataset, with respect to FB+CS+GC results, for each quantitative parameter separately.

2.8. UP-Net Ablation Study

We performed an ablation study to assess the contributions of the key components used in UP-Net, including phase augmentation, GAN loss, MRI physics loss, uncertainty estimation, and joint end-to-end training strategy. After training each ablated model with the same training dataset, we compared the results in the testing dataset using NRMSE and SSIM for image quality, and absolute errors in liver PDFF and R₂* for quantification accuracy. We used the Wilcoxon signed-rank test to evaluate if the performance of the ablated network models had significant difference versus the performance of UP-Net. p<0.01 was considered significant.

3. RESULTS

3.1. UP-Net Image Quality and Quantification Accuracy

Figure 3 shows representative first-echo magnitude and phase images from UP-Net input, UP-Net output results, and FB+CS results in two subjects with NAFLD (a 47-year-old male and a 17-year-old male). UP-Net suppressed the radial undersampling streaking artifacts in the liver and in the background (arrows in Figure 3), and achieved high SSIM compared with FB+CS. Figures 4 and 5 show representative quantitative PDFF, R₂* and field map results in axial and coronal orientations and the corresponding uncertainty maps in two NAFLD subjects (same subjects as in Figure 3). UP-Net generated accurate PDFF/R₂*/field maps compared with FB+CS+GC references in the liver ROIs. Most regions show low quantification errors (Figures 4 and 5) in all 3 quantitative parameters. Regions corresponding to air usually had large quantification errors. The uncertainty maps show high intensities (red arrows in Figures 4 and 5) and characterize the lower confidence in these regions in air.

Figure 3.

Representative free-breathing (FB) stack-of-radial first-echo images from self-gated input images, UP-Net output results, and reference images reconstructed by compressed sensing (CS). (a) Results from a 47-year-old male (BMI=28.0kg/m²) in the testing set. (b) Results from a 17-year-old male (BMI=30.4kg/m²) in the testing set. Structural similarity index (SSIM) values comparing UP-Net output with reference images are shown. Arrows point to streaking artifacts in the self-gated images that are suppressed in the UP-Net output and reference images. BMI: body mass index.

Figure 4.

Representative free-breathing (FB) (a) proton-density fat fraction (PDFF), (b) R₂* and (c) field maps and corresponding uncertainty maps from the same subject in Figure 3a. PDFF, R₂* and field map errors were generally low when comparing UP-Net results with reference maps reconstructed using compressed sensing (CS) and graph-cut (GC) algorithms. In regions with larger quantification errors, higher UP-Net uncertainty scores were observed in all 3 quantitative maps (red arrows). Errors and higher uncertainty scores around the body were in regions corresponding to air.

Figure 5.

Representative free-breathing (FB) (a) proton-density fat fraction (PDFF), (b) R₂* and (c) field maps and corresponding uncertainty maps from the same subject in Figure 3b. PDFF, R₂* and field map errors were generally low when comparing UP-Net results with reference maps reconstructed using compressed sensing (CS) and graph-cut (GC) algorithms. In regions with larger quantification errors, higher UP-Net uncertainty scores were observed in all 3 quantitative maps (red arrows). Errors and higher uncertainty scores around the body were in regions corresponding to air.

Bland-Altman plots for liver PDFF and R₂* quantification accuracy are shown in Figure 6. For PDFF quantification, FB+UP-Net achieved MD = −0.36% compared with FB+CS+GC, and MD = 0.53% compared with BH Cartesian. For R₂* quantification, FB+UP-Net achieved MD = −0.37 s⁻¹ compared with FB+CS+GC, and MD = 6.75 s⁻¹compared with BH Cartesian. LoA between FB+UP-Net versus FB+CS+GC was narrower than FB+UP-Net versus BH Cartesian for both PDFF and R₂* quantification.

Figure 6.

(a-b) Bland-Altman plots comparing liver proton-density fat fraction (PDFF) values from UP-Net output maps versus free-breathing (FB) reference maps and breath-holding (BH) Cartesian maps. (c-d) Bland-Altman plots comparing liver R₂* values from UP-Net output maps versus FB reference maps and BH Cartesian maps. The dashed lines represent zero difference. The solid lines represent mean differences (MD) and 95% limits of agreements (LoA).

3.2. UP-Net Uncertainty Estimation

Linear correlation results comparing absolute quantification errors versus uncertainty scores of three quantitative parameters in liver ROIs in the validation dataset are shown in Figure 7a. The Spearman correlation coefficients for PDFF, R₂* and field map were 0.358 (p<0.05), 0.466 (p<0.01), and 0.503 (p<0.01), respectively. These calibrated linear regression curves were used to convert uncertainty scores measured in the testing dataset to predicted quantification errors. The Bland-Altman plots for UP-Net predicted errors versus actual quantification errors in liver ROIs are shown in Figure 7b. MDs between UP-Net predicted errors versus actual absolute quantifications errors were 0.27%, 0.12 s⁻¹, and 0.19 Hz for PDFF, R₂*, and field map, respectively. Note that the quantification errors were all generally low to begin with.

Figure 7.

(a) Correlation plots between absolute quantification errors (UP-Net outputs versus reference compressed sensing and graph-cut fitting results) and UP-Net uncertainty scores in liver regions of interest (ROIs) in the validation dataset. Linear regression was performed to calibrate PDFF, R₂* and field map uncertainty scores with respect to the absolute errors. (b) Bland-Altman plots comparing the errors predicted from UP-Net uncertainty scores versus the actual absolute quantification errors in PDFF, R₂* and field map in liver ROIs in the testing dataset. The black dashed lines represent zero difference. The black solid lines represent mean differences (MD) and 95% limits of agreements (LoA).

3.3. UP-Net Ablation Study Results

Table 3 shows the results of our ablation study. UP-Net achieved higher mean SSIM of 0.872 and lower mean NRMSE of 0.173 compared with the ablated UP-Net models without phase augmentation or GAN loss (all p<0.01). Compared with the ablated UP-Net model without the MRI physics loss, UP-Net achieved lower mean PDFF error of −0.36% and lower mean R₂* error of −0.37 s⁻¹ (both p<0.01). UP-Net without uncertainty estimation did not have significant difference in image quality and quantification accuracy when compared with UP-Net (i.e., the addition of the uncertainty path did not degrade quantification accuracy). UP-Net without joint training achieved higher mean PDFF error of −0.46% (p<0.01) compared to UP-Net with joint training.

Table 3.

Ablation study in the testing dataset for different components used in UP-Net. Structural similarity index (SSIM) and normalized root mean squared error (NRMSE) were evaluated on magnitude images, with respect to compressed sensing results. Proton-density fat fraction (PDFF) and R₂^* quantification errors were evaluated in liver regions of interest, with respect to compressed sensing and graph-cut fitting results. Results are reported as mean ± standard deviation.

	Component					Metric
Network	Phase Augmentation	GAN Loss	Physics Loss	Uncertainty Estimation	Joint Training	SSIM	NRMSE	PDFF Errors	R2* Errors
1		✓	✓	✓	✓	0.851±0.055*	0.182±0.048*	−0.92%±0.95%*	−0.54s−1±3.02s−1*
2	✓		✓	✓	✓	0.858±0.067*	0.194±0.053*	−0.31%±1.05%	−0.68s−1±3.46s−1*
3	✓	✓		✓	✓	0.870±0.049	0.178±0.050	−1.69%±1.49%*	−2.50s−1±5.03s−1*
4	✓	✓	✓		✓	0.877±0.048	0.176±0.042	−0.29%±0.88%	−0.34s−1±3.81s−1
5	✓	✓	✓	✓		0.884±0.050	0.168±0.061	−0.46%±1.47%*	−0.41s−1±3.02s−1
UP-Net	✓	✓	✓	✓	✓	0.872±0.053	0.173±0.059	−0.36%±0.98%	−0.37s−1±3.56s−1

^*represents statistically significant difference (p<0.01, Wilcoxon signed-rank test) compared with UP-Net.

3.4. Processing/Reconstruction Time

The entire CS reconstruction was implemented in MATLAB 2021b (MathWorks, Natick, MA). Data preparation steps of gradient calibration, self-gating, NUFFT, and beamforming-based coil combination (Figures 1 and 2) required a total time of 30 sec/slice. Repeated forward and inverse NUFFT are the bottleneck for the CS reconstruction method. To improve computational performance, we implemented the CS reconstruction method using GPU-based NUFFT packages⁵⁸. CS reconstruction took 3 min/slice on an Intel Xeon E5–2660 CPU with 128GB RAM and an NVIDIA v100 GPU with 32GB memory. We used the ISMRM fat-water toolbox⁵⁹ and code from previous works²¹ for GC fitting algorithms, which required 15 seconds/slice on the same CPU. We implemented UP-Net using Python 3.8.10 and Pytorch 1.12.1. UP-Net required 28 hours to train on an NVIDIA v100 GPU with 32GB memory. With the prepared data as input, UP-Net took 79 msec/slice for network inference (using same hardware as network training). A more detailed analysis of the total operation counts for each method was provided in Supplementary Table S1.

4. DISCUSSION

We developed an uncertainty-aware physics-driven deep learning network that accurately quantifies liver PDFF and R₂* using undersampled self-gated free-breathing multi-echo stack-of-radial MRI. Compared with previous works on DL-based fat/water separation and/or R₂* mapping^30–34, our study has two main contributions. First, we investigated a DL approach for PDFF and R₂* mapping from undersampled radial MRI data. Unlike previously proposed networks that learned mapping from fully-sampled Cartesian images to fat/water signals or quantitative maps^30–34, UP-Net generates accurate quantitative maps from images impacted by radial undersampling artifacts. We incorporated artifact suppression and parameter mapping into one end-to-end network. This substantially reduced the computational time for image artifact suppression compared to time-intensive CS methods and fat-water signal fitting compared to GC algorithms. Second, our proposed network has built-in uncertainty estimation that generates pixel-wise uncertainty maps for different quantitative parameters. Uncertainty estimation to assess the confidence levels in DL-based MRI reconstruction and quantitative parameter mapping results is a nascent direction^35–38. We specifically investigated the application of uncertainty estimation in DL-based PDFF and R₂* quantification and demonstrated that a calibration method for the UP-Net uncertainty scores can be used to predict absolute liver PDFF and R₂* quantification errors in UP-Net parameter maps to within 1% and 3 s⁻¹, respectively, compared to actual errors with respect to reference methods.

To suppress radial undersampling streaking artifacts, we used UNet as the backbone architecture, which has been used in previous work for radial streaking reduction in 2D slices^26,27, dynamic 2D cardiac images²⁹, and 2D images from different respiratory phases⁵¹. In this work, we adapted the input/output dimensions of UNet to accommodate the 2D multi-echo images. We stacked the real and imaginary components from all of the multi-echo images along the channel dimension to preserve the consistency of the magnitude and phase input information for PDFF and R₂* quantification. We also adopted a GAN architecture and a phase augmentation strategy for image quality improvement. Due to limited memory on the GPU for network training, correlations between neighboring slices were not considered in this study. Networks that can efficiently process multi-echo 2D+slice or multi-echo 3D volumetric data could be further investigated. Although it is possible to omit the artifact suppression module and use one single network to generate quantitative maps directly from self-gated undersampled radial images, UP-Net with a modular architecture can provide more accurate quantitative maps with less radial streaking artifacts (example in Supplementary Figure S3).

In addition to rapid computational time, another potential advantage of using UP-Net or other DL-based methods for fat-water separation is reducing the occurrence of fat-water swaps. In our datasets, there were slices with local fat-water swaps (usually around the liver dome) using the GC methods. These slices required manually adjusting GC parameters, such as constraints on B₀ field map smoothness or range, to address the swaps. When training UP-Net, we excluded data with fat-water swaps and performed phase augmentation, which helped the network to learn reliable fat-water separation in the presence of B₀ field map variations. The use of UP-Net can potentially reduce the occurrence of fat-water swaps (Supplementary Figures S4) and avoid the extra time/effort needed to check and fix fat-water swaps.

One concern of DL-based fat-water separation is whether the network could perform accurate mapping for datasets with liver PDFF values outside the range in the training dataset. UP-Net has two advantages that could allow it to generalize to these cases. First, we used an MRI physics loss that will constrain the output to follow the fat-water signal model. Second, even though the training dataset we used in this work has a maximum liver PDFF around 30%, UP-Net still learned from the signal characteristics in fat-dominant tissues (e.g., subcutaneous adipose tissue) with PDFF up to 90%. To investigate this, we created synthetic testing datasets with higher liver PDFF, and used UP-Net to perform PDFF mapping. A representative example in Supplementary Figure S5 shows that UP-Net can indeed quantify higher liver PDFF values (e.g., >40%) that were not included in the training dataset. Another concern of DL-based fat-water separation is whether the network can be adapted to several different body parts. Although different body parts may have different B₀ field map ranges and variations, their signal characteristics are described by the same fat-water signal model. After training UP-Net on a certain dataset (e.g., liver and upper abdomen), the fat-water signal model is implicitly learned. Through transfer learning and fine tuning, UP-Net can potentially be applied to other body parts (e.g., lower abdomen).

Fully-sampled free-breathing motion-resolved volumetric abdominal stack-of-radial MRI data is often impractical to acquire. We used CS to generate images and quantitative maps with suppressed radial streaking artifacts. CS methods have already been validated for PDFF quantification using undersampled Cartesian MRI⁵² and for PDFF and R₂* quantification using undersampled radial MRI data¹⁹. For complete evaluation of our UP-Net quantification accuracy, we also compared UP-Net results with standard 3D BH Cartesian MRI. Many previous DL-based fat-water signal fitting methods were only evaluated on individual fat and water maps^30–33. We evaluated our results on quantitative PDFF maps. Similar to a previous DL-based method for joint PDFF and R₂* mapping using Cartesian MRI³⁴, we also achieved low biases in PDFF, R₂*, and field map values versus reference methods. Notably, we trained and tested our method on a larger dataset (105 subjects). In contrast, a previous report considered 31 subjects³⁴. Previous DL methods did not investigate their results in NAFLD subjects, while our UP-Net was trained and evaluated in a population including healthy subjects and subjects with suspected or confirmed NAFLD. In our Bland-Altman analysis of PDFF and R₂* quantification accuracy, the MD between FB+UP-Net and FB+CS+GC was smaller than the MD between FB+UP-Net and BH Cartesian. This was expected because UP-Net was trained using reference data from the FB+CS+GC method. The MD and LoA of PDFF and R₂* quantification comparing FB+UP-Net versus BH Cartesian are similar to results in previous studies comparing self-gated FB stack-of-radial MRI with BH Cartesian¹⁸.

We carefully examined the contributions from key components in UP-Net, including phase augmentation, GAN loss, and MRI physics loss. Among these components, MRI physics loss was especially important for accurate parameter quantification. From our ablation study, the network without MRI physics loss generated larger biases in both PDFF and R₂* quantification. MRI physics loss considered the relationship between multi-echo signals and quantitative parameters and did not require reference quantitative maps. However, the MRI physics loss alone may not provide sufficient information to resolve fat-water swaps. By adding an MSE loss for quantitative maps and training with reference non-swapped maps, we directed UP-Net to learn the spatial distribution of fat- or water- dominant pixels and reduce fat-water swaps. In applications that focus on the signal magnitude, phase information in DL results is often discarded or overlooked. Our use of a phase augmentation strategy strengthened UP-Net’s ability to learn complex-valued signal relationships by including images with the same magnitude but different phase. This strategy can also be applied in applications that need accurate phase, including temperature mapping and quantitative susceptibility mapping. GAN architectures for imaging tasks is an active research topic. In this work, we used Wasserstein loss in our GAN architecture, which has previously been used in Cartesian MRI reconstruction⁵³. More complicated GAN architectures and loss functions designed for medical images⁵⁴ could be investigated further.

The “black box” nature of DL-based methods for MRI is an important concern and potential barrier to clinical translation. Uncertainty estimation in DL networks^35–38 presents a promising approach to provide context and assess confidence in DL outputs for clinical applications that demand a high level of numerical accuracy, including the use of quantitative maps for diagnostic decisions. In this study, we showed that with calibration, UP-Net uncertainty scores predicted quantification errors in a separate testing dataset. These promising results have some potential applications. For example, confidence masks can be generated by thresholding the uncertainty scores and then overlaid on the UP-Net quantitative parameter maps. Radiologists can avoid making measurements and decisions in areas with higher uncertainty scores and have more confidence in using DL-generated images and quantitative maps. A recent study has demonstrated that by passing uncertainty information in concatenated tasks, the performance of the downstream task (e.g., segmentation or detection) can be improved⁵⁵. The uncertainty maps generated by UP-Net can potentially provide information and improve subsequent automatic liver MRI analysis, such as DL-based liver segmentation and disease classification.

This study has limitations. First, we did not investigate the influence of different data undersampling factors on UP-Net performance. We used a 40% data acceptance window (2.5-fold undersampling) on nominally fully-sampled data, as suggested in previous studies^17,18. Self-gating data acceptance rates can be further reduced to improve motion fidelity. The number of acquired radial spokes can also be reduced to investigate additional scan acceleration. However, higher undersampling factors pose more difficulties in both generating high-quality reference data and training UP-Net. Adjustments such as adding k-space consistency layers⁵⁶ might be required for UP-Net to address higher undersampling factors. Second, we trained and tested UP-Net using data with specific number of echoes, TE, TR, and flip angle. These sequence parameters were closely related to PDFF/R₂* accuracy in the data we used. Our current analysis on the quantification accuracy and uncertainty prediction may not be directly applicable in other datasets with different sequence parameters. Third, the calculation of the UP-Net uncertainty loss term required reference quantitative maps. Therefore, the UP-Net uncertainty values reflect differences between UP-Net results and results from reference methods. Fourth, we calibrated the PDFF and R₂* uncertainty estimation in the validation dataset only using ROIs in the liver. This approach required CS and GC reference reconstruction results for calibration. In addition, different calibration curves may be needed to quantify DL uncertainty in other tissues, such as subcutaneous and visceral adipose tissues. Fifth, we used linear regression to investigate the relationship between UP-Net uncertainty scores and quantification errors. However, this approach may not be sufficient to characterize all the factors at play. PDFF measurements are results of relative amounts of two chemical shift species and can have different inherent uncertainty at different PDFF levels. R₂* uncertainty depends on combinations of the number of echoes, the chosen echo times, and the underlying true R₂* values. Because PDFF and R₂* are incorporated together in the fat-water signal model, multi-variate models can also be considered to improve uncertainty characterization and calibration in the future.

5. CONCLUSION

In this study, we developed an uncertainty-aware physics-driven deep learning network that rapidly calculates accurate liver PDFF and R₂* maps from undersampled free-breathing self-gated multi-echo stack-of-radial images and provides pixel-wise uncertainty maps. GAN architecture, phase augmentation, and MRI physics loss improved the UP-Net image quality and quantification accuracy for liver PDFF and R₂*. We demonstrated that UP-Net uncertainty scores can be used to predict absolute quantification errors in liver PDFF and R₂*.

Supplementary Material

supinfo

Supplementary Table S1. Comparison of the number of floating-point operations (FLOPs) for reconstructing one slice using different methods. One addition or one multiplication of two real numbers counts as one FLOP. UP-Net was implemented using Python and PyTorch. The number of FLOPs for UP-Net was calculated using the ptflops package (https://pypi.org/project/ptflops/). The compressed sensing (CS) algorithm used in this work was implemented using conjugate gradient descent in MATLAB. The graph-cut based fitting algorithm was implemented in MATLAB using code from the ISMRM fat-water toolbox. The CS algorithm is an iterative method and the FLOPs count is based on the average iteration number used in this work. The number of FLOPs for CS and graph-cut algorithms were calculated with the help of the Lightspeed MATLAB Toolbox (https://github.com/tminka/lightspeed) which provides accurate estimated number of FLOPs for different operations in MATLAB.

Supplementary Figure S1. Network architectures for the (a) artifact suppression module and (b) parameter mapping module in UP-Net.

Supplementary Figure S2. Overview of the data augmentation strategy used in this work. For phase augmentation, a phase offset (∅) was added to (1) self-gated images, (2) reference compressed sensing (CS)-reconstructed images, (3) reference complex-valued fat signal and (4) reference complex-valued water signal to generate phase-augmented training data. Signal magnitudes were not modified. R2* and field map values were not changed. In each epoch during training, 3 more instances (in addition to the original data) were generated by using 3 random phase offsets ∅₁, ∅₂ and ∅₃ in the range of [0, 2π].

Supplementary Figure S3. Comparison of results from (a) UP-Net without an artifact suppression module, (b) proposed UP-Net, and (c) reference quantitative maps using compressed sensing and graph-cut algorithms. In (a), we trained UP-Net without an artifact suppression module using pairs of self-gated multi-echo images and reference quantitative maps. This approach used a single network to suppress the radial undersampling artifacts and perform parameter mapping at the same time, which can be challenging. We found that the radial streaking artifacts were not adequately suppressed (yellow arrow) and there were PDFF quantification errors (green arrow).

Supplementary Figure S4. Example cases where a graph cut-based method generated PDFF maps with local fat-water swaps and UP-Net generated PDFF maps without fat-water swaps. The local fat-water swaps usually occur near the liver dome in our free-breathing MRI dataset. In these 2D slices, the liver usually occupies a small portion of the field of view and is more likely to result in local fat-water swaps (arrows) using the conventional method (compressed sensing + graph-cut based fitting). In contrast, these slices do not exhibit fat-water swaps in our UP-Net results.

Supplementary Figure S5. (a) Example of a synthetic testing case with higher PDFF. First, we performed parameter mapping from the dataset using a reference method with graph-cut fitting. Next, we multiplied the fat signal component within the liver by 1.5 and divided the water signal component within the liver by 1.5 to synthesize a case with higher liver PFFF (>40%). (b) We used the same approach as in (a) to generate corresponding synthetic self-gated images with higher liver PDFF, which were used as inputs to UP-Net. (c) Reference images and quantitative maps for the synthetic testing case in (a). (d) The UP-Net results for this higher-PDFF synthetic testing case. The quantitative values from the UP-Net results were consistent with the synthetic reference data.

DATA AVAILABILITY STATEMENT

To support the findings in this work, our code for UP-Net implementation is available at https://github.com/HoldenWuLab/UP-Net.