Confidence Maps for Reliable Estimation of Proton Density Fat-Fraction and R2* in the Liver

Abstract

Purpose:

To develop a fully automated algorithm that generates confidence maps to identify regions valid for analysis of quantitative proton density fat-fraction (PDFF) and R2* maps of the liver, generated with chemical shift encoded MRI (CSE-MRI). Confidence maps are urgently needed for automated quality assurance, particularly with the emergence of automated segmentation and analysis algorithms.

Methods:

Confidence maps for both PDFF and R2* maps are generated based on goodness of fit, measured by normalized root-mean-square error (NRMSE) between measured complex signals and the CSE-MRI signal model. Based on Cramér-Rao Lower Bound and Monte-Carlo simulations, NRMSE threshold criteria were developed to identify unreliable regions in quantitative maps. Simulation, phantom, and in vivo clinical studies were included. To analyze the clinical data, a board-certified radiologist delineated regions of interest (ROI) in each of the 9 liver segments for PDFF and R2* analysis in consecutive clinical CSE-MRI data sets. The percent area of ROIs in areas deemed unreliable by confidence maps was calculated to assess the impact of confidence maps on real-world clinical PDFF and R2* measurements.

Results:

Simulations and phantom studies demonstrated that the proposed algorithm successfully excluded regions with unreliable PDFF and R2* measurements. ROI analysis by the radiologist revealed that 2.6% and 15% of the ROIs were placed in unreliable areas of PDFF and R2* maps, as identified by confidence maps.

Conclusion:

A proposed confidence map algorithm that identifies reliable areas of PDFF and R2* measurements from CSE-MRI acquisitions, was successfully developed and demonstrated technical and clinical feasibility.Introduction

Proton density fat-fraction (PDFF) and transverse relaxivity rate (R2*) estimated using quantitative chemical shift-encoded MRI (CSE-MRI)^1,2 are well-established quantitative biomarkers of liver fat content, ie: steatosis^3,4 and liver iron deposition^5,6. Over the past two decades, numerous studies have described the technical development and validation of CSE-MRI, including approaches that reduce variability and bias in quantitative measurements^7,8. CSE-MRI methods are FDA-approved to evaluate diseases such as non-alcoholic fatty liver disease⁹ and liver iron overload^6,10. More recently, R2* mapping with CSE-MRI been shown to be reproducible across vendors and calibrated to liver iron concentration (LIC, mgFe/g dry)¹¹.

Despite these efforts, reliable estimation of PDFF and R2* remains a challenge because of confounding factors, such as low signal-to-noise ratio (SNR), high iron content, inhomogeneous main magnetic field (B0), and motion. For example, Colgan et al. investigated the reliability of PDFF estimation in the presence of high R2*, showing that PDFF measurements may be unreliable above R2* of ~500s⁻¹ and ~800s⁻¹ at 1.5T and 3.0T, respectively¹².

With low SNR, PDFF estimates demonstrate not only high variability but also bias, due to an asymmetric noise probability density at low SNR¹³. Phase errors caused by hardware imperfections and concomitant field gradients, also introduce bias and poor reliability of PDFF and R2* estimation^14–16.

Further, water-fat swapping can occur in areas of high B0 inhomogeneity, such as the liver dome. Despite sophisticated methods aimed at preventing water-fat swaps¹⁷, such swaps remain a challenge, even with commercial methods. Incorrect estimation of the B0 field results from a natural ambiguity in the water and fat signals in water (fat)-dominant pixels, leading to incorrect estimation of both PDFF and R2*. Ghosting of adipose tissue signal into the liver can also lead to inaccurate estimation of PDFF and R2*.

Unbiased estimation of PDFF and R2* is essential for accurate diagnosis, staging and treatment monitoring. For example, differentiation of normal liver from biopsy-based grade 1 hepatic steatosis occurs at a PDFF threshold of ~5.4-6.4%^18,19. Unfortunately, most commercial and investigational CSE-MRI methods simply provide reconstructed PDFF and R2* maps without any guidance on regions with valid fitting. One commercial PDFF and R2* mapping product (LiverLab, Siemens Healthineers, Erlangen, Germany) provides a goodness-of-fit map for quality control, although it does not provide details or explicit criteria for identifying reliable regions of PDFF and R2* estimation^20,21. Regions with inadequate or corrupted estimates of PDFF or R2* are often not apparent, even to experienced human analysts.

Algorithms that segment the liver²² are emerging and hold promise for automated PDFF and R2* analysis, making automated identification of valid regions necessary. Without automated methods to exclude invalid regions of PDFF and R2* maps, the performance of fully automated analysis of PDFF and R2* mapping may be limited.

Therefore, the purpose of this work is to develop a fully automated algorithm to generate confidence maps that identify regions in PDFF and R2* maps valid for subsequent analysis by human analysts or automated segmentation algorithms.

Theory

Scenarios for Failure in Estimation of PDFF and R2*

We have identified six scenarios that lead to biased estimation or high variability of PDFF and/or R2* values. These include:

1. Low SNR of underlying source images.

2. Moderate iron overload leading to invalid PDFF estimation but valid R2* estimation.

3. Severe iron overload, leading to both invalid PDFF and R2* estimation.

4. Extraneous phase errors in complex CSE-MRI methods.

5. Water-fat swapping.

6. Tissues adjacent to areas of high susceptibility eg. metallic implants.

PDFF and/or R2* in pixels suffering from bias or high variability should be excluded from subsequent analysis and reporting.

PDFF and R2* exhibit increased variability and bias when SNR is low. Complex echo images have independent additive Gaussian noise²³, resulting in variability in both signal magnitude and phase. Poor SNR in underlying echo images can lead to high variability and bias in PDFF and R2* maps¹³. Noise does not propagate linearly in PDFF and R2* maps, making it difficult for experts to determine when SNR is sufficient.

Signal decay caused by moderate iron overload worsens SNR in later echoes at higher echo time (TE) values. Even though SNR may be adequate to estimate R2* reliably, water-fat separation and therefore PDFF estimation may be difficult. In the spectral domain, increasing R2* leads to line-width broadening, and when sufficiently severe, merges water and fat into an inseparably broad peak (Figure S1). More severe iron overload results in insufficient SNR to estimate either R2* and PDFF.

Source of phase errors include eddy currents²⁴, concomitant gradients²⁵, or spurious phase errors. Such phase errors can lead to biased estimates of both PDFF and R2*. Successful strategies have been proposed to address eddy currents^26,27 and concomitant gradients¹⁴. However, it can be challenging to identify incompletely corrected or other spurious phase-related PDFF and R2* estimation errors. Phase errors lead to anatomically accurate and visibly plausible PDFF and R2* maps, but with large quantitative errors not easily appreciated visually.

Water-fat swapping occurs in regions with relatively high B0 inhomogeneity, such as the liver dome²⁸. Swaps lead to bias in not only PDFF estimate, but also R2* because the spectral model of water (single-peak) and fat (multi-peak) are not the same. A pixel with a water-fat swap will inappropriately fit signal to the wrong spectral model, and the multi-peak interference of fat with itself will lead to an apparent under- (over-) estimation of R2*. Water-fat swaps are generally easy for expert human analysts to identify, although this is unlikely the case for automated segmentation algorithms.

Finally, signal dephasing in tissue adjacent to sources of severe focal susceptibility such as metallic implants and or abrupt tissue-air interfaces, can lead to over-estimation of R2*, and bias in PDFF.

Chemical Shift Encoded Magnetic Resonance Imaging (CSE-MRI)

Multi-echo spoiled gradient echo imaging can be used to provide simultaneous confounder-corrected estimation of PDFF and R2*^5,29. In this study, we adopt a commonly used 6-peak model, which assumes multiple species of fat, as first proposed by Yu et al.^2,30:

$$s\left(t\right)=\left({\rho }_w+{\rho }_f\sum _{p=1}^6{\alpha }_p\cdot e^{i2\pi f_{p}t}\right)\cdot e^{i2\pi \psi t},$$ (1)

where ${\rho }_w$ and ${\rho }_f$ are the complex signal amplitude of water and fat components, ${\alpha }_p$ and $f_p$ are relative amplitudes and frequencies for p^th peak of fat such that $\sum _{p=1}^6{\alpha }_p=1,\psi$ is the “complex field map”³⁰ that includes off-resonance frequency ${\psi }_0$ (Hz) and R2* (s⁻¹), defined as:

$$\psi ={\psi }_0+\frac{{iR}_2^{*}}{2\pi }.$$ (2)

PDFF is defined as:

$$PDFF=\frac{\left|{\rho }_f\right|}{\left|{\rho }_w\right|+\left|{\rho }_f\right|}.$$ (3)

PDFF and R2* can be estimated using iterative² or non-linear estimation algorithms³¹. The first and second terms of equation 1 describe signals from the water and fat components in the multi-peak model. Since this problem is non-convex, challenges such as inhomogeneous B0 fields, low SNR, high R2*¹², and water-fat swaps, can result in inaccurate estimation of PDFF and/or R2*. In this work, six-echo imaging, which is commonly used for clinical imaging in the liver, is assumed.

Confidence Maps

Confidence maps enable the identification of inappropriate estimation of PDFF and R2* under the scenarios described above. To develop confidence maps, we consider three classes:

1. Poor SNR and/or phase errors of the acquired signals (scenarios i-iv).

2. Failure of B0 estimation (scenario v).

3. Overestimation of R2* due to severe local susceptibility (scenario vi)

Based on the above, three binary confidence maps are generated for both PDFF ($C_{n,PDFF}$, n=1,2,3) and R2* ($C_{n,R{2}^{*}}$, n=1,2,3), for a total of 6 binary confidence masks. An overview of the proposed confidence map algorithm is summarized in Figure 1.

Figure 1:

Overview of the proposed confidence map algorithm. Multiple source echo images are acquired. PDFF and R2* maps are reconstructed using a CSE-MRI algorithm with multi-peak signal modeling. Masks ($C_{1,PDFF},C_{1,R2*}$) identifies areas with poor quality of signals for PDFF and R2* measurements can be calculated based on the normalized root-mean-square error (NRMSE). Cramer-Rao Lower Bound, and Monte-Carlo simulation provide confidence threshold of NRMSE. The NRMSE for signal fitting with and without off-resonance shifts (±3.4 ppm) are compared to detect water-fat swapping ($C_2$). To exclude areas with strong R2* decay due to susceptibility effect, susceptibility effect masks ($C_{3,PDFF}$ and $C_{3,R2*}$) are calculated based on the local B0 gradient. Finally, overall confidence maps for PDFF and R2* are derived by using the binary AND combination of C₁, C₂, and C₃

Overall confidence maps for PDFF and R2* are determined through the binary AND operation:

$$\begin{gathered} C_{PDFF}\left(\mathit{r}\right)=C_{1,PDFF}\left(\mathit{r}\right)\wedge C_2\left(\mathit{r}\right)\wedge C_{3,PDFF}\left(\mathit{r}\right) \\ {C}_{R{2}^{*}}\left(\mathit{r}\right)=C_{1,R{2}^{*}}\left(\mathit{r}\right)\wedge C_2\left(\mathit{r}\right)\wedge C_{3,R2*}\left(\mathit{r}\right) \end{gathered}$$ (4)

where $\wedge$ represents the AND operator and noting that $C_2\left(\mathit{r}\right)=C_{2,PDFF}\left(\mathit{r}\right)=C_{2,R{2}^{*}}\left(\mathit{r}\right)$ for water-fat swaps. The resulting confidence maps for PDFF and R2* maps indicate areas with high and low reliability for ROI-based analysis.

The first mask identifies areas with poor quality signals for estimation of $C_{1,PDFF}$ and $C_{1,R{2}^{*}}$. Confidence maps are based on the normalized root-mean-square error (NRMSE) between the measured signal and CSE-MRI signal model, in the least-squares sense, ie: L2 norm. Any NRMSE value above pre-determined threshold criteria - determined below using Cramér-Rao Lower Bound (CRLB) and Monte-Carlo analyses - are considered reliable.

The NRMSE can also be used to assess water-fat swaps generated using forced water-fat swaps. By comparing each fit with signal models that have off-resonance frequencies to enforce a water-fat swap, and estimating the probability of a water-fat swap¹⁷. Note that PDFF and R2* share the same mask for water-fat swaps, hence $C_2\left(\mathit{r}\right)=C_{2,PDFF}\left(\mathit{r}\right)=C_{2,R{2}^{*}}\left(\mathit{r}\right)$.

Severe local susceptibility caused by metallic implants or air-tissue interfaces, are evaluated using the local spatial gradient of the B0 inhomogeneity (${\psi }_0$) to generate $C_{3,PDFF}$ and $C_{3,R{2}^{*}}$.

In the following subsections, we describe in detail how these three classes of confidence maps are generated.

Evaluation for quality of acquired signal

The reliability of PDFF and R2* estimates can be evaluated using the NRMSE between the measured and fitted signals and non-linear least squares estimation³². This method evaluates the agreement between the measured and modeled complex signal, defined as:

$$NRMSE=\frac{1}{\left|\widehat{y}\right|}\sqrt{\sum _{n=1}^N\frac{{\left[{\mathfrak{R}(y}_n-{\widehat{y}}_n)\right]}^2+{\left[\mathfrak{Z}(y_n-{\widehat{y}}_n)\right]}^2}{N}},$$ (5)

where $y_i$ and ${\widehat{y}}_n$ are the measured and estimated complex signals at n^th echo and $\widehat{y}$ is the mean value of the measured signals, $\mathfrak{R}$ and $\mathfrak{Z}$ denote real and imaginary parts. There are two terms in the square root that denote the residuals between real and imaginary signals of measurement and modeling. The measured signals, which are reconstructed from k-space data, are multi-echo complex-valued signals that are channel combined. NRMSE is a convenient metric to provide a confidence threshold, defined as:

$$C_1\left(\mathit{r}\right)=\left\{\begin{array}{c}\begin{array}{ccc}1& if& NRMSE\left(\mathit{r}\right)\le \tau \end{array}\\ \begin{array}{ccc}0& if& NRMSE\left(\mathit{r}\right)\end{array}>\tau \end{array}\right.$$ (6)

where $\mathit{r}$ is the spatial coordinate of the map and $\tau$ is a predetermined threshold value, above which the signals from voxel at position $\mathit{r}$ do not adequately fit the signal model to be considered reliable.

The threshold $\tau$ can vary with PDFF and R2*, as well as pulse sequence parameters, most notably first echo time ($T{E}_1$), echo spacing ($\mathrm{\Delta }TE$) and echo train length. In this study, we determine values of $\tau$ as a function of PDFF, R2*, $T{E}_1$, and $\mathrm{\Delta }TE$ using the CRLB and Monte-Carlo simulations as described below, for a fixed echo train length of six.

Detection of water-fat swapping

Water-fat swapping is a well-known challenge with CSE-MRI caused by incorrect estimation of the B0 field map resulting from fitting to a local minimum, particularly in water- or fat-dominant pixels³³. Swaps are commonly seen in the liver dome but can occur anywhere. Since swapping leads to highly biased estimates of both PDFF and R2*, it is critical to exclude these regions.

Water-fat swapping occurs two ways with CSE-MRI. For PDFF values below 50%, when a swap occurs, the estimated field map (${\psi }_0$) is estimated at +3.4ppm higher than the true B0 field, corresponding to the methylene peak of fat. This results in overestimation of PDFF, and is common with typical PDFF values observed in the liver. When PDFF exceeds 50% and a swap occurs, the estimated field map is underestimated by −3.4ppm, which corresponds to the water peak, and PDFF is underestimated. We summarize these two scenarios in Figure S2.

When swaps occur, the agreement between measured and fitted signals decreases, due to differences in the spectral model of water (single peak) and fat (multipeak), leading to an increase in the NRMSE¹⁷. A swap can be enforced by fitting the signal with a shift of +3.4, −3.4ppm relative to the estimated frequency, by substituting the frequency term as

$${\psi }_0\leftarrow {\psi }_0+\mathrm{\Delta }\psi ,$$ (7)

where $\mathrm{\Delta }\psi$ is the off-resonance shift corresponding to ±3.4ppm, eg. ±217Hz at 1.5T. Calculating NRMSE values for each fitted signals can be used to predict the presence of a swap. Since the fitting with lower NRMSE should be correct, the confidence map which is reflecting swapping can be defined as:

$$C_2\left(\mathit{r}\right)=\left\{\begin{array}{c}\begin{array}{ccc}1& if& NRMSE\left(\mathit{r}\right)\le NRMS{E}_{\mathrm{\Delta }\psi =\pm 3.4}\left(\mathit{r}\right)\end{array}\\ \begin{array}{ccc}0& if& NRMSE\left(\mathit{r}\right)>\end{array}NRMS{E}_{\mathrm{\Delta }\psi =\pm 3.4}\left(r\right)\end{array}\right.,$$ (8)

where $NRMS{E}_{\mathrm{\Delta }\psi =\pm 3.4}$ is NRMSE for the fitted signal with the off-resonance shift. This principle has been proposed by Yu et al¹⁷. and others³⁴ to exploit the spectral complexity of fat to mitigate fat-water swaps.

Detection of focal susceptibility gradients

Strong local off-resonance due to sources of susceptibility such as metallic implants or tissue-air interfaces, accelerates signal decay and lead to local over-estimation of the apparent R2*³⁵. The B0 field gradient in the largest voxel dimension (usually slice) can be used to exclude regions of high susceptibility by using gradient threshold $\kappa$ as

$$C_{3,R{2}^{*}}\left(\mathit{r}\right)=\left\{\begin{array}{c}\begin{array}{ccc}1& if& \mathrm{\Delta }\bullet G>\kappa \end{array}\\ \begin{array}{ccc}0& if& \mathrm{\Delta }\bullet G\le \end{array}\kappa \end{array}\right.$$ (9)

with

$$G\equiv \frac{\partial }{\partial z}\left({\psi }_0\right)$$ (10)

where ${\psi }_0$ is the estimated B0 map from equation 2, $\mathrm{\Delta }$ denotes the largest voxel dimension (eg. slice direction), and z is the coordinate system in that direction. The B0 map can also be approximated from the phase difference of any two echo images if B0 estimation from CSE-MRI is unreliable due to rapid signal decay (eg. severe R2*). It is important that echo images be selected such that the phase of water and fat signals are similar, to prevent estimation errors caused by differences in off-resonance between water and fat. Gradient thresholds $\kappa$ can be chosen based on numerical simulations of R2* decay with B0 field gradients.

Specifically, Bloch equation simulations of signal decay due to B0 field gradients using a point spread function approach³⁵ can be used to estimate $\kappa$ for equation 9. Assuming a rectangular slice profile, the additional R2* decay due to the macroscopic B0 field gradients in a specific direction can be expressed by

$$s\left(t,G\right)=\left({\rho }_w+{\rho }_f\sum _{p=1}^P{\alpha }_p\cdot e^{j2\pi f_{p}t}\right)\cdot e^{j2\pi \psi \left(z\right)t}\cdot \mathrm{\Delta }\cdot sinc(\gamma \cdot \mathrm{\Delta }\cdot G\cdot t)$$ (11)

where G is the field gradient defined in equation 10. We assume a field gradient along the largest voxel dimension, which is typically the slice direction with CSE-MRI in the liver. Although these assumptions are not entirely accurate for 3D acquisitions, the proposed method provides practical criteria (Figure S3). The apparent R2* which includes the effects of the B0 gradient, can be calculated from the full width half maximum (FWHM) of frequency domain of the signal as

$$R{2}^{*†}(\mathrm{\Delta }\cdot G)=\frac{1}{\pi \cdot FWHM(\mathrm{\Delta }\cdot G)}.$$ (12)

such that the bias of R2* estimation ($\mathrm{\Delta }R{2}^{*}$) is defined as

$$\mathrm{\Delta }R{2}^{*}(\mathrm{\Delta }\cdot G)=R{2}^{*†}(\mathrm{\Delta }\cdot G)-R{2}^{*}$$ (13)

Threshold criteria ($\kappa$) can be chosen such that the bias on R2* estimates are within an acceptable range such that

$$\kappa =\mathrm{a}\mathrm{r}\mathrm{g}\underset{G}{\min }\left|\mathrm{\Delta }R{2}^{*}(\mathrm{\Delta }\cdot G)-R{2}_T^{*}\right|,$$ (14)

where $R{2}_T^{*}$ is the maximum acceptable R2* bias (Materials and Methods). Equations 12 and 13 indicate that even a moderate field gradient will inevitably lead to some bias at low R2* values, while high R2* values are relatively robust to field gradients.

PDFF estimates are also affected by B0 gradients, although less so than R2*. Mild field gradients still allow for accurate estimation because they broaden the water and fat spectra, which manifests as an R2* overestimation, leaving PDFF relatively unaffected. When the field gradient reaches a sufficient amplitude, the spectra become highly broadened, leading to complicated signal decay, and failure of PDFF estimation.

Therefore, the susceptibility effect map for PDFF can be defined as

$$C_{3,PDFF}\left(\mathit{r}\right)=\left\{\begin{array}{c}\begin{array}{ccc}1& if& \mathrm{\Delta }\cdot G>\kappa \prime \end{array}\\ \begin{array}{ccc}0& if& \mathrm{\Delta }\cdot G\le \end{array}\kappa \prime \end{array},\right.$$ (15)

where $G_{\tau \prime }$ is cutoff gradient value that allows acceptable bias. In this study, values for $\kappa \prime$ of 51Hz and 93Hz for 1.5T and 3.0T were used, respectively. These values were determined using Monte-Carlo simulations (Figure S4).

Cramér–Rao Lower Bound (CRLB) and Monte-Carlo (MC) Simulation

CRLB and MC simulations can be used to determine NRMSE thresholds in two steps. First, we determine the minimum SNR necessary to meet the target variability (as characterized by the standard deviation of PDFF and R2*) by using CRLB. MC and nonlinear regression analyses are then used to obtain regression models that establish the relationship between SNR and NRMSE. These analyses provide the relationship between NRMSE and standard deviation, indirectly, and assumes that the estimators of PDFF and R2* are efficient, which is generally the case for most CSE-MRI methods³⁴.

The CRLB^12,36, derived from the equation for CSE-MRI², provides a theoretical lower bound on the variance of any unbiased estimator for PDFF ($CRL{B}_{PDFF}$) and R2* ($CR{LB}_{R{2}^{*}}$) estimated for a specific combination of PDFF, R2*, TE1, ΔTE, and SNR, assuming a fixed echo train length.

Using ROI analysis, analysts typically estimate the mean of the estimated values from a region of tissue, with Z independent samples. The minimum SNR is chosen such that standard error (SE) of the mean of the estimated PDFF or R2* value is below an acceptable variance threshold. In this work, SE is defined as

$$SE=\frac{\sigma }{\sqrt{Z}}$$ (16)

where $\sigma$ is a standard deviation of PDFF or R2*, and Z is the number of independent samples in an ROI. Threshold criteria for variability in PDFF and R2* estimates, and Z will vary depending on specific applications. The minimum SNR values needed to achieve reliable estimates are obtained by solving the following minimization problems,

$$\begin{gathered} SN{R}_{PDFF}\left(\mathit{x}\right)=\mathrm{a}\mathrm{r}\mathrm{g}\underset{\mathit{SNR}}{\min }\left|\frac{CRL{B}_{PDFF}\left(SNR,\mathit{x}\right)}{\sqrt{Z}}-{SE}_{PDFF}\right| \\ SN{R}_{R{2}^{*}}\left(\mathit{x}\right)=\mathrm{a}\mathrm{r}\mathrm{g}\underset{\mathit{SNR}}{\min }\left|\frac{CR{LB}_{R{2}^{*}}\left(SNR,\mathit{x}\right)}{\sqrt{Z}}-{SE}_{R{2}^{*}}\right| \end{gathered}$$ (17)

with

$$\mathit{x}=(PDFF,R{2}^{*},\mathrm{T}\mathrm{E}1,\mathrm{T}\mathrm{E}),$$ (18)

where $CR{LB}_{PDFF}$ and $CRL{B}_{R{2}^{*}}$ are the minimum standard deviation, calculated from CRLB analysis for PDFF and R2*, respectively; ${SE}_{PDFF}$ and ${SE}_{R{2}^{*}}$ are acceptable SE (Materials and Methods) for PDFF and R2*, respectively. $SN{R}_{PDFF}$ and $SN{R}_{R{2}^{*}}$ are calculated over a range of plausible TE1, $\mathrm{\Delta }TE$, PDFF, and R2* values. Configurations for CRLB analysis including criteria and parameters used in this work are described in Materials and Methods section. The first terms in $argmin$ operator denote SE of PDFF and R2* for a specific set of parameters of SNR, PDFF, R2*, TE1, $\mathrm{\Delta }\mathrm{T}\mathrm{E}$.

Because accurate SNR measurements are a challenge in general, we propose an approach to estimate SNR from NRMSE. To determine the relationship between NRMSE and SNR, MC simulations can be performed with variable PDFF, R2*, TE1, $\mathrm{\Delta }TE$, and SNR values, and repeated Z times for each set of parameters to replicate measurements within a region of interest (ROI) containing Z pixels, simulating an ROI measurement from a PDFF or R2* map. Mean NRMSE and SNR values are measured from the repeated calculations and plotted as shown in Figure 2. An empirical function $F\left(SNR\right)$ representing the NRMSE for a specific SNR value can be obtained by fitting the curves in Figure 2 with a neural network regression model (below).

Figure 2:

A straightforward empirical relationship between NRMSE and SNR can be developed, as shown for PDFF of 0, 10, and 20% at (a) 1.5T and (b) 3.0T, derived from Monte-Carlo simulations. The plots demonstrate that NRMSE decreases monotonically as SNR increases. The relationship has a weak dependence on PDFF unlike R2*, where this is a stronger dependence. The plotted curves are then fit with a neural network regression model to estimate NRMSE threshold values for the SNR value that is required for reliable PDFF/R2* measurements as calculated by CRLB.

Using this relationship, thresholds of NRMSE for PDFF and R2* can be estimated by combining the obtained two regressions as

$$\begin{array}{ccc}{\tau }_{PDFF}\left(\mathit{x}\right)& =& F\left(SN{R}_{PDFF}\left(\mathit{x}\right)\right)\\ {\tau }_{R{2}^{*}}\left(\mathit{x}\right)& =& F\left(SN{R}_{R2*}\left(\mathit{x}\right)\right)\end{array},$$ (19)

where ${\tau }_{PDFF}$ and ${\tau }_{R{2}^{*}}$ are the thresholds used in equation 6 to determine the reliability of PDFF and R2* measurements.

Methods

An algorithm was developed based on the theory described above and is summarized in Figure 1. To evaluate the performance of the confidence map algorithm, MC simulations, and phantom and in vivo experiments were performed. The proposed algorithm was developed in MATLAB (R2021b, Mathworks, Natick, MA). The generation of the quantitative PDFF and R2* maps used for the confidence maps was performed offline using the ISMRM Fat-Water toolbox^37,38. Water and fat images, and PDFF and R2* maps were reconstructed from complex images using a nonlinear least-squares estimation method³⁷. Confidence maps for PDFF and R2* were independently generated pixel-by-pixel using the equation 4, and final PDFF and R2* maps with confidence map overlays were exported as DICOM images for phantom and in vivo experiments.

Confidence Map

In this work, we assumed broad plausible ranges of R2* and PDFF, and broad but plausible ranges for TE1, ΔTE, SNR, and off-resonance frequency. The range for PDFF and was chosen from 0-100%, and the range for R2* was 25-500s⁻¹ at 1.5T and 25-1000s⁻¹ at 3.0T, respectively, based on previous studies which shows upper limits of reliable PDFF and R2*measured using CSE-MRI ^8,12. We used the ranges of TE₁ (0.9-1.8 and 0.6-1.0ms for 1.5T and 3.0T, respectively) and $\mathrm{\Delta }TE$ (1.1-2.2 and 0.55-1.1ms for 1.5T and 3.0T, respectively) based on that typically allowed by vendors, and past optimizations¹⁶. Off-resonance frequencies of −14.5-81.3Hz and −31.7-164.0Hz for 1.5T and 3.0T, respectively were used based on the known variability of B0 in the liver²⁸. SNR of the first echo was varied between 2-50. An echo train length of 6 was chosen, as this value is widely used for clinical CSE-MRI acquisitions.

Criteria for reliable measurements of PDFF and R2* were determined based on previous clinical studies. Specifically, the coefficients of repeatability for PDFF and R2* in the liver were investigated based on prior in-vivo studies. Yokoo et al. has reported that PDFF measurement using CSE-MRI showed repeatability coefficients of 2.99%⁴, respectively. For R2*, Hernando et al. reported the test-retest R2* repeatability using Bland-Altman analysis at both 1.5T(95% limit of agreement (LOA): −14.2-16.9%, Bias: 1.4%) and 3.0T (95% LOA: −16.6-15.5%, Bias: −0.6%) as part of a multi-center, multi-vendor study⁸.

Based on these results, we chose thresholds that would lead to standard error (SE) values much less that these coefficients of repeatability to avoid impacting the test-retest variability of CSE-MRI. Specifically, we aimed to achieve a SE of 1% (absolute) or less for PDFF, and less than 3% (relative) or 5s⁻¹ (whichever is larger) for R2*. Also, we calculated two additional confidence maps with different thresholds (PDFF: 2%, R2*: 10% or 10s⁻¹, and PDFF: 3%, R2*: 20% or 20s⁻¹) to provide a level of confidence reliability. The same thresholds were used for detection of severe local susceptibility. We assumed a circular ROI with 2.8cm diameter, corresponding to 100 pixels for in-plane spatial resolution of 2.0×3.0mm², typical for CSE-MRI. Although voxel size doesn’t directly affect the algorithm, the number of voxels in the ROI considered in this work will affect the proposed threshold. Importantly, the number of voxels in the ROI may vary depending on the application. The map was color-coded based on three different thresholds represented by three different colors: yellow (PDFF: 1%, R2*: 3% or 5s-1), orange (PDFF: 2%, R2*: 10% or 10s-1), and red (PDFF: 3%, R2*: 20% or 20s-1). In addition, regions with water-fat swapping or severe local susceptibility are labeled with cyan hatching.

Simulation Experiments

Monte-Carlo (MC) simulations using signal generated with the model in equation 1 were performed (MATLAB 2021b) to evaluate the performance of the proposed confidence map algorithm. The same ranges of parameters for R2*, PDFF, TE₁, ΔTE, SNR, and off-resonance frequency as those assumed for the confidence map were applied. Zero-mean complex Gaussian noise was added to the simulated signal to vary SNR. The variance of the estimated values of PDFF and R2* for a particular NRMSE was calculated using a binned scatterplot, was performed by grouping NRMSE values with a bin width of 0.025.

Additional MC simulations were used to evaluate the performance of the proposed water-fat swap detection method using the same parameter range explained above. We implemented the algorithm in equation 6 and plotted probabilities that successfully detected swapping against NMRSE for 1.5T and 3.0T. The same ranges for R2* at 1.5 and 3T were used as the confidence map algorithm.

To obtain an empirical relationship between $SN{R}_{PDFF}$ and $SN{R}_{R2*}$ with NMRSE, the results from equation 17 were fit to a shallow neural network regression model consisting of three fully connected hidden layers with 10 neurons using the fitrnet function provided in MATLAB. The same model was used for fitting $F$, which shows the relationship between NRMSE and SNR.

R2* bias due to macroscopic B0 inhomogeneity was calculated with variable R2* (25-1000s⁻¹) and B0 using equations 11–13. An empirical relationship between the B0 gradient and R2* was developed to determine the threshold $\kappa$ in equation 9, obtained by fitting the calculated results to polynomial regression model up to 3^rd order, followed by the application of equation 14.

Phantom Experiments

Phantom experiments were performed to validate the performance of the confidence map algorithm. A prototype phantom³⁹ (Calimetrix, Madison, WI) consisting of 16 vials with simultaneously varying PDFF and R2* enclosed within a spherical housing unit was used. Specific PDFF and R2* values are listed (Figure S5 and Table S1). PDFF and R2* maps were acquired using a 3.0T clinical MR system (Signa Premier, GE Healthcare, Waukesha, WI). Acquisition parameters are summarized in Table 1. To obtain low SNR images, a flip angle of 1° was used. Number of signal averages (NSA) of 1 and 9 were used to vary SNR.

Table 1:

Imaging parameters used for phantom and in vivo studies.

	Phantom	In vivo
	3.0T	1.5T	3.0T
TR	6.2 ms	8.3-13.6 ms	4.9-7.5 ms
TE1	1.0 ms	0.9-1.7 ms	0.9-1.0 ms
ΔTE	0.7 ms	1.8-2.1 ms	0.59-1.0 ms
# Echoes	3	3	3
# Interleaved Echo Trains	2	2	2
Receiver Bandwidth	781 Hz/px	651 Hz/px	781 Hz/px
FOV	40 cm	40-46 cm	36-40 cm
Slice Thickness	2 mm	7.8-10 mm	8-10 mm
Number of slices	20	56-72	28-36
Matrix Size	160×160	184×112-192×160	116×104-192×160
Flip Angle	1	5°	3-4°
NSA	1, 9	0.5	0.5

To perform quantitative analysis, PDFF and R2* values were measured. ROIs were placed in each vial to perform a box plot analysis. The proposed confidence map algorithm was applied to the acquired maps.

In Vivo Experiments

A prospective clinical study was performed by performing a retrospective analysis of CSE-MRI data collected from consecutive patients undergoing PDFF and R2* measurements in the liver as part of clinical MR exams. Approval from our local Institutional Review Board (IRB), including a waiver of informed consent, was obtained. Imaging for the subjects was implemented using various 1.5T and 3.0T clinical MR systems (GE Healthcare, Waukesha, WI). A commercial CSE-MRI method (IDEAL-IQ, GE Healthcare, Waukesha, WI) with imaging parameters listed in Table 1 was used. The specifications, including systems and receiver coils, are summarized in Table S2.

A board-certified radiologist with 6 years of experience delineated ROIs for each of the 9 liver segments according to the method of Campo et al⁴⁰ blinded to the confidence maps. We then calculated the percent area of the ROIs in areas that were identified by the confidence map as invalid. In this way, we aimed to determine the impact of confidence maps on clinical analysis, since those areas would have been avoided if the confidence maps had been made available to the radiologist or an automated segmentation algorithm.

Results

Simulation Experiments

Figure 3 plots SE as a function of NRMSE and R2* for R2* and PDFF calculated using MC simulations with (a) 1.5T and (b) 3.0T. Threshold values calculated using the proposed method (white solid lines) agree with estimates from MC simulations (dotted line). The plots also demonstrate that threshold values vary depending on PDFF and R2*. Calculations were not performed in the lower right regions of the plots, indicated as black regions, due to the absence of any simulated samples with low-moderate R2* and very high NRMSE.

Figure 3:

PDFF and R2* thresholds for NMRSE at different PDFFs (0, 10, 20%) estimated by the proposed CRLB-based method and validated by MC simulations. The standard error (SE) of PDFF (upper row) and R2* (bottom row) measurements with varying R2* is plotted against NRMSE calculated using CRLB and MC simulations at (a) 1.5T and (b) 3.0T. White solid and dotted lines show acceptable NRMSE thresholds estimated using MC simulations and the CRLB, respectively. Although there are slight differences between simulations and estimated results, the threshold values calculated using the proposed CRLB-based method agree closely with MC simulations.

MC simulations reveal water-fat swapping can be successfully detected using the proposed method for wide range of PDFF and R2* values. Figure 4 shows the probability of water-fat swapping as a function of PDFF and R2* for (a) 1.5T and (b) 3.0T. The algorithm can detect swapping with 80% probability for most PDFF and R2* values. The probability of a water-fat swap detection drops dramatically at high R2*, as expected.

Figure 4:

According to Monte-Carlo simulations, swapping can be detected with high probability. The plots show the probability of successful detection of W/F swapping using the proposed algorithm as a function of R2* (50-500 s⁻¹ for 1.5T, 500-1000 s⁻¹ for 3T) and PDFF (0-50%). In most PDFF/R2* regions, swapping can be detected with a probability greater than ~80%. Although swapping is more challenging to identify in regions with low PDFF and high R2*, it is still detectable with more than 65% probability.

Figure 5 plots the estimation bias of R2* in % due to B0 inhomogeneities. There is a strong relationship between estimation bias and B0 gradient and R2*. A relatively small B0 gradient will lead to some bias at low R2* values, whereas high R2* values are less biased. Based on these results, threshold values for B0 gradients are shown as the white dotted lines in Figure 5a−c, for use with equation 14.

Figure 5:

Plots show bias in % of R2* measurement as a function of field gradient and R2*, suggesting estimation bias strongly depends on these variables and could be unreliable in specific conditions. A simulation using the Bloch equation was performed using different R2* values ranging from 50 to 1000 s^-1. When R2* is relatively small, such as less than 100 s⁻¹, a relatively small field gradient could provide non-negligible bias, but as R2* increases, the bias could be neglected regardless of the field gradient. The threshold values (white dotted lines) for the susceptibility map were determined by fitting the calculated results to the polynomial regression model up to the 3rd order to achieve measurements within acceptable R2* bias.

Phantom Experiments

Data from the phantom experiments are shown in Figure 6, demonstrating that the algorithm identified areas not suitable for PDFF and R2* measurements under various SNR conditions. The PDFF and R2* maps obtained with high SNR (NSA=9) had appropriate SE values for PDFF and R2* measurements. Similarly, the algorithm identified regions with poor SE in PDFF and R2* for the low SNR acquisition (NSA=1). ROIs in vials 4, 8, 12, and 16 in the PDFF map acquired with NSA=1 were excluded as unreliable PDFF (SE = 1.1%, 1.7%, 0.79%, 0.71%). As shown in the R2* map (Figure 6b), the algorithm also identified most of vials 8 and 16 as invalid for analysis (SE=3.7%, 2.4%). Finally, box plots also show how these ROIs have relatively high PDFF and/or R2* variability (Figure 6c, d).

Figure 6:

The confidence map algorithm successfully identifies regions of a PDFF/R2* phantom where measurements are unreliable due to high variability in quantitative PDFF and R2* values. For example, ROI 8, whose SE of PDFF and R2* estimates do not meet the threshold SE criteria defined in this work, is excluded by the algorithm in both (a) PDFF (SE=1.7%) and (b) R2* (SE=3.7%) maps. In addition, the algorithm identifies whole regions of ROI 4 for which PDFF (SE=1.1%) is reliable, but not R2* (SE=1.6%). The box plots for PDFF (c) and R2* (d) show the median, first/third quartile, and minimum/maximum values for each vial. Results indicate that vials with high R2* tend to have high variability in both PDFF and R2*, and the algorithm masks out vials with SE that exceeds threshold criteria. Yellow and red hatched regions indicate that measurement variability surpasses two distinct thresholds for PDFF (2%, and 3%) and R2* (10% or 10s⁻¹, and 20% or 20s⁻¹). The cyan hatching indicates areas suspected of water-fat swapping and/or severe local susceptibility.

In Vivo Experiments

Raw data from 100 consecutive patients (1.5T: 51, 3.0T: 49 patients, 51:49 men:women, 56 range:18-82 years) who underwent CSE-MRI of the liver as part of their clinical MR exam was successfully collected and reconstructed as described above. Examples of the proposed algorithm for patients with measurement failures such as high R2* (Figure 7), metallic implant (Figure 8), and water-fat swapping (Figure 9) are also shown. Figure 7 shows R2* and PDFF maps with high R2* (>500s⁻¹) as well as histograms with PDFF and R2* values from ROIs located in the right lobe, demonstrating that the algorithm successfully identified unreliable regions. A measurement of this example indicates unacceptable variability (SE=1.7%) in PDFF maps while R2* (SE=2.3%) is within the defined criteria. Figure 8 contains an example of a patient with a metal implant in the abdominal wall, which causes severe susceptibility artifacts in both PDFF and R2* maps. Most fatty regions in R2* are masked out because of B0 field gradient above the defined threshold($\kappa$) caused by interface between fat and water tissues. These regions were successfully masked using the confidence maps. Finally, Figure 9 shows an example of a water-fat swap at the liver dome successfully identified by the proposed algorithm.

Figure 7:

Example PDFF/R2* maps acquired with 1.5T MRI and corresponding overlaying confidence maps in a patient with elevated liver iron. Histograms plots the distributions of PDFF and R2* values in ROIs placed in the right lobe, indicated as black dotted circles. The hashed-out regions in the confidence maps identify unreliable PDFF/R2* estimates. (a) PDFF measurement fails due to moderate iron overload while R2* is appropriately estimated. Histograms from ROI indicates PDFF has high standard deviation and highly variable values above criteria, while the variability of R2* values in the liver are acceptable.

Figure 8:

Example PDFF/R2* maps acquired with 1.5T MRI with susceptibility artifacts due to a metallic implant in the anterior abdominal well. Regions close to the metallic implant are successfully masked out based on the confidence map algorithm.

Figure 9:

Example PDFF/R2* maps acquired with 1.5T MRI with water-fat swapping successfully identified at the liver dome (red arrow), where both PDFF and R2* are highly biased. Fully automated confidence maps generated using the proposed method excluded this unreliable area.

In total, 832 ROIs were placed by the radiologist, with 68 ROIs not placed due to poor image quality, such as motion artifacts. ROI analysis by the radiologist revealed that 2.6% and 15% of the area of the ROIs in PDFF and R2* maps were placed inside unreliable areas identified by confidence maps. Importantly, 5.4% and 49% of ROIs for PDFF and R2* maps, respectively, had more than 10% of invalid pixels. Further, there were significant differences in PDFF (4.3%±4.4%) and R2* (18.7s⁻¹±34 s⁻¹) measurements with and without confidence maps in 1% and 12% of the cases, respectively. Twenty-six cases had water-fat swapping, and two had severe focal metal artifacts. Furthermore, three cases had unreliable PDFF due to moderate levels of R2* while R2* estimates were acceptable. No cases with extreme R2* values were seen.

Discussion

In this work we have successfully proposed, developed and validated an algorithm to generate confidence maps for PDFF and R2* maps measured in the liver using CSE-MRI. Confidence maps were generated based on NRMSE values between the signal model and measured signals. Further, areas of water-fat swapping and regions of severe susceptibility artifact from metal were excluded. Monte-Carlo simulations, a phantom study and clinical study demonstrated technical and clinical feasibility of the proposed algorithm.

The proposed confidence map approach provides a fully automated method to identify reliable and unreliable regions of PDFF and R2* maps. Such maps are needed to provide accurate and precise estimates of PDFF and R2* needed to diagnose, stage and monitor treatment of patients with liver disease. The proposed algorithm can be used to avoid inappropriate ROI placement by analysts and should improve the quality and consistency of PDFF and R2* measurements.

The proposed method can be used for other CSE-MRI applications, although it was optimized for liver imaging. The threshold values may vary depending on specific applications. However, the proposed method can be applied to any CSE-MRI measurement by changing simply adjusting threshold used to generate the confidence maps.

Automated analysis of quantitative maps is emerging to streamline and automate workflow. Martí-Aguado et al. have proposed a CNN-based whole liver segmentation algorithm for automated measurement of PDFF and R2*²². Despite similar diagnostic accuracy to manual assessment, images with artifacts were manually excluded. Although automated segmentation is beyond the scope and purpose of the current work, automated confidence mapping, such as that proposed here will be necessary for fully automated segmentation methods to provide valid analyses for PDFF and R2* measurements.

As shown in this work, invalid regions of R2* and/or PDFF estimation are relatively common. Furthermore, focal severe susceptibility artifacts occurred with non-negligible frequency. It is important to note that although the expert radiologist can exclude inappropriate areas in many cases, confidence maps are important aids for non-expert analysts, and necessary for fully automated tools.

There are some limitations to the proposed algorithm. One limitation of this work is that only one vendor is evaluated in this study. Future work should include multi-vendor validation of the proposed strategy. There are several estimation algorithms for PDFF and R2*, and the threshold used for the confidence map could vary between algorithms. In addition, pulse sequences parameters, such as TE1 and ΔTE, also vary depending on vendors. However, we expect that the proposed confidence map method can be readily extended or applied across different parameters and estimation algorithms. Moreover, since this study focused on liver-specific parameters, individual optimization is needed for other organs. In addition, we did not consider motion artifacts, which can cause quantitative errors. Further, the algorithm used to predict the presence of water-fat swaps is an approach that if incorporated into the PDFF and R2* map reconstruction, would obviate the need to mask regions of water-fat swaps. With the exception of one commercial PDFF quantification method³⁴, most commercial methods do not use the spectral complexity¹⁷ of fat to avoid water-fat swaps. Our proposed algorithm may be of interest in the presence of denoising algorithms. However, our algorithm assumes zero-mean Gaussian noise, and therefore it may have limited applicability when nonlinear filters are used, as these filters may distort noise characteristics. Further research is needed to investigate this important scenario. Partial volume effects could lead to signal model mismatches that impact the performance of the proposed method.” The proposed method is computationally expensive, currently requiring ~5 minutes per case in the Matlab environment. Finally, further validation studies are needed as future work to evaluate the precision (repeatability and reproducibility) and clinical utility of the algorithm.

In conclusion, we have successfully developed a confidence map algorithm for PDFF and R2* maps generated using CSE-MRI in the liver. Confidence maps are based on NRMSE values, in combination with masking for the presence of water-fat swaps and areas of high local magnetic susceptibility. The proposed algorithm was effective in simulations, phantom, and in vivo experiments.

Supplementary Material

Supinfo

Supporting Information Figure S1 Local field gradients broaden the spectrum (a), resulting in additional R2* decay (b) when the field gradient is mild. A severe field gradient, however, can lead to the spectrum splitting into two lines, which causes signal oscillation in the time domain. In CSE-MRI, the modulated spectra of water and fat may not fit the predefined multi-peak spectra model, causing PDFF to be biased.

Supporting Information Figure S2 There are two possible scenarios that cause water-fat swapping with multi-peak signal modeling. For PDFF<50% (a), the estimated off-resonance frequency is overestimated (b) when with a water-fat swap caused by an error in B0 estimation. For PDFF>50% (c), off-resonance and PDFF is underestimated (d) with a water-fat swap. The over- or under-estimation of off-resonance leads to an increased NRMSE since a residual component of fitting remains, since the spectral models of water (single peak) and fat (multipeak) are different.

Supporting Information Figure S3 The proposed algorithm proposed practical criteria for measurement with a metal implant. Measurement bias of R2* (ΔR2*) and field gradient (ΔG) were measured in a ROI located in the left lobe (a). (b) Plotting ΔR2* against ΔG indicated that the algorithm could provide a reasonable threshold (8.3 Hz) for measurement, although the number is slightly conservative.

Supporting Information Figure S4 Bias in PDFF estimation caused by the field gradient is significant at (a) 1.5T while estimation at (b) 3.0T is relatively prone to the field gradient due to shorter TEs. The results indicate PDFF estimation is relatively robust to the field gradient compared to R2*.

Supporting Information Figure S5 A spherical phantom used in the study, which consists of 16 vials with a unique combination of fat and iron concentration.