Combining deep learning and 3D contrast source inversion in MR‐based electrical properties tomography

Abstract

Magnetic resonance electrical properties tomography (MR‐EPT) is a technique used to estimate the conductivity and permittivity of tissues from MR measurements of the transmit magnetic field. Different reconstruction methods are available; however, all these methods present several limitations, which hamper the clinical applicability. Standard Helmholtz‐based MR‐EPT methods are severely affected by noise. Iterative reconstruction methods such as contrast source inversion electrical properties tomography (CSI‐EPT) are typically time‐consuming and are dependent on their initialization. Deep learning (DL) based methods require a large amount of training data before sufficient generalization can be achieved. Here, we investigate the benefits achievable using a hybrid approach, that is, using MR‐EPT or DL‐EPT as initialization guesses for standard 3D CSI‐EPT. Using realistic electromagnetic simulations at 3 and 7 T, the accuracy and precision of hybrid CSI reconstructions are compared with those of standard 3D CSI‐EPT reconstructions. Our results indicate that a hybrid method consisting of an initial DL‐EPT reconstruction followed by a 3D CSI‐EPT reconstruction would be beneficial. DL‐EPT combined with standard 3D CSI‐EPT exploits the power of data‐driven DL‐based EPT reconstructions, while the subsequent CSI‐EPT facilitates a better generalization by providing data consistency.

A hybrid method for electrical properties tomography (EPT), which combines deep learning EPT (DL‐EPT) and contrast source inversion EPT (CSI‐EPT), is presented. Its accuracy and precision are investigated by means of realistic electromagnetic simulations at 3 and 7 T. The presented hybrid method shows potential improvements compared with standard EPT methods, since it exploits the power of data‐driven DL‐based EPT reconstructions, while the subsequent CSI‐EPT algorithm facilitates a better generalization by providing data consistency.

Abbreviations used

CSF

cerebrospinal fluid

CSI

contrast source inversion

DL

deep learning

EM

electromagnetic

EPs

electrical properties

EPT

electrical properties tomography

GM

gray matter

MR‐EPT

magnetic resonance electrical properties tomography

RRE

relative residual error

SNR

signal‐to‐noise ratio

WM

white matter.

1. INTRODUCTION

Knowledge of in vivo tissue electrical properties (EPs: conductivity σ and relative permittivity ε _r) is of high interest for different applications such as improving the local specific absorption rate quantification used in, for example, hyperthermia treatment planning or safety assessment in MRI. ¹ , ² Furthermore, due to the relation between conductivity and ionic content, in vivo measurements of tissue EPs can in principle provide clinical information about pathological tissues, making them a potentially useful biomarker for diagnostic purposes and treatment monitoring. ³ , ⁴

There have been several approaches to measure in vivo tissue electrical properties noninvasively. ⁵ , ⁶ In 1991, the possibility of retrieving tissue electrical properties in the radio‐frequency range from MR measurements of the circularly polarized transmit magnetic field $\left({\widehat{B}}_1^{+}\right)$ has been shown. ⁷ This technique was referred to as electrical properties tomography (EPT). ⁸ , ⁹

EPT approaches can be divided into two major categories: direct and inverse methods. Direct methods based on the Helmholtz equation aim to reconstruct tissue EPs from MR measurements by computing spatial derivatives of the measured ${\widehat{B}}_1^{+}$ field. However, this operation leads to severe boundary errors and noise amplification in the reconstructed EP maps.

Inverse methods like contrast source inversion electrical properties tomography (CSI‐EPT) ¹⁰ , ¹¹ , ¹² aim to reconstruct EPs by solving a minimization problem iteratively, where the EP model is fitted to the measured ${\widehat{B}}_1^{+}$ field. This avoids the need to compute spatial derivatives of measured data, making these methods in principle more noise‐robust. However, these methods are limited by their computational complexity and the need for EM quantities that are not directly accessible with MRI measurements (such as background fields or transmit phase). ⁸ , ⁹ Furthermore, CSI‐EPT reconstructions suffer from artifacts arising from the low electric field strength at the center of a volume transmit coil and local minima, making CSI‐EPT reconstructions dependent upon their initialization.

Recently, a new approach, called deep learning‐electrical properties tomography (DL‐EPT), ¹³ has been proposed, where the inverse transformation is learned by means of a convolutional neural network. This method relies purely on measurable MR quantities, making it applicable to MR measurements. Preliminary results demonstrated the feasibility of this approach, leading to good‐quality EP maps. However, the major risk of DL‐based EPT reconstructions is that cases not present in the training set will not be accurately reconstructed. Therefore, exhaustive datasets are needed in training, increasing the computational load for DL‐based EPT methods.

In this work, a two‐step approach is proposed, where Helmholtz‐based reconstructions (magnetic resonance electrical properties tomography: MR‐EPT) and deep learning reconstructions (DL‐EPT) are used as data‐driven initializations for 3D CSI‐EPT. We show that an accurate initialization guess provided by DL‐EPT improves CSI‐EPT reconstructions, while CSI‐EPT has the potential to improve tissue structure of DL‐EPT reconstructions.

2. METHODS

2.1. EM simulation setup

Electromagnetic (EM) field simulations were performed using the commercial finite‐difference time‐domain EM simulation software XFdtd (Remcom State College, PA, USA). At 3 T a high‐pass quadrature birdcage body coil (length 58 cm, diameter 70.4 cm) resonant at 128 MHz was simulated, surrounded by a shield (length 70 cm, diameter 74.3 cm), while for simulations at 7 T a high‐pass quadrature birdcage head coil (length 19.5 cm, diameter 30 cm, similar to the dimensions of the Nova Medical birdcage Tx/Rx head coil) resonant at 300 MHz was used, surrounded by a shield (length 22 cm, diameter 36 cm). The head of the male human body model (Duke, Virtual Family ¹⁴ ) used for these simulations was placed at the center of each coil and discretized on a 2×2×2 mm³ grid. For these ${\widehat{B}}_1^{+}$ field simulations, the object is bounded to the reconstruction domain (128×128×56 voxels), to prevent influences from tissues outside the reconstruction domain on the ${\widehat{B}}_1^{+}$ field.

2.2. Reconstruction approaches

Standard Helmholtz‐based MR‐EPT, DL‐EPT, and standard 3D CSI‐EPT using a homogeneous mask as initialization guess were performed as described below. Furthermore, hybrid reconstructions were performed by providing MR‐EPT and DL‐EPT reconstructions as initialization guesses to 3D CSI‐EPT.

2.2.1. MR‐EPT

The conventional implementation of MR‐EPT is based on the Helmholtz equation, given by

$$\frac{{\nabla }^2{\widehat{B}}_1^{+}}{{\widehat{B}}_1^{+}}=-{\omega }^2\mu {\epsilon }_0{\epsilon }_{\mathrm{r}}+\mathrm{i}\omega \mu \sigma$$ (1)

with ω the Larmor frequency (128 or 300 MHz for 3 and 7 T, respectively), μ the permeability of the tissue, which is assumed to be equal to that of free space, and ε ₀ the permittivity of free space. If the left‐hand side of Equation (1) is known, the unknown tissue parameters ε _r and σ can easily be extracted from this equation. To compute the Laplacian of the simulated ${\widehat{B}}_1^{+}$ fields in XFdtd, a large 3D finite‐difference kernel is used (see Mandija et al. ¹⁵ for its description), since small finite‐difference kernels are highly sensitive to spatial fluctuations. ¹⁵ This approach is hereafter referred to as MR‐EPT.

2.2.2. DL‐EPT

Deep learning EPT is a data‐driven approach, where a surrogate model based on accessible MR quantities is learnt. Following the procedure indicated in Mandija et al., ¹³ 1064 unique ${\widehat{B}}_1^{+}$ fields (56 slices for 19 head models) obtained from realistic EM simulations at 3 T performed in Sim4Life (ZMT, Zurich, Switzerland) with noise superimposed were used to train the network. The head models were obtained by changing conductivity and permittivity values of the male (Duke) and female (Ella) human head models from the Virtual Family, and by including geometrical transformations such as compression/dilatation, rotation, and translation. ¹³ The original Duke model used for testing (56 slices) was excluded from training. For these simulations, the same body coil setup as simulated in XFdtd (see Section 2.1) was used. For details on the network, please refer to Isola et al., ¹⁶ while for details on training, validating, or testing please refer to Mandija et al. ¹⁶ Moreover, this network was trained using the transceive phase, that is, the combination of the transmit and receive phase, and not using the transmit phase only, since this latter field cannot be measured in an MR experiment. ¹⁷ Note that only a deep learning network trained for head models at 3 T is currently available. This network provides 2D EP reconstructions, and therefore abrupt changes can appear through slices. These reconstructions are referred to as DL‐EPT.

2.2.3. H‐CSI

Three‐dimensional CSI‐EPT is an iterative method that minimizes a cost functional based on models of the contrast function $\widehat{\chi }$ describing the EPs and the contrast source $\widehat{\mathbf{w}}=\widehat{\chi }\widehat{\mathbf{E}}$, where $\widehat{\mathbf{E}}$ is the electric field strength. The functional that is minimized is given by

$$F(\widehat{\mathbf{w}},\widehat{\chi })=\frac{||\widehat{\rho }|{|}^2}{||{\widehat{B}}_1^{+;\text{sc}}|{|}^2}+\frac{||\widehat{\mathbf{r}}|{|}^2}{||\widehat{\chi }{\widehat{\mathbf{E}}}^{\text{inc}}|{|}^2}$$ (2)

where $\widehat{\rho }$ is the mismatch between measured and modeled data, $\widehat{\mathbf{r}}$ describes the discrepancy in satisfying Maxwell's equations, and the superscripts ‘sc’ and ‘inc’ denote the scattered and incident part of the EM fields (see Leijsen et al. ¹² for more details). A conjugate gradient update step is used for the contrast function to suppress sensitivity to low electric field regions. ¹⁸ Reconstructions are stopped after 500 iterations or when the functional has reached a tolerance level of 10⁻⁵. These stopping criteria are determined empirically to prevent noise overfitting. They act like a simple regularizer, but more sophisticated regularization methods, such as multiplicative total variation, can also be implemented in 3D configurations. ¹⁰ The EPs are extracted from the real and imaginary part of the reconstructed contrast function. The ${\widehat{B}}_1^{+}$ fields simulated in XFdtd were used for standard 3D CSI‐EPT reconstructions. For these reconstructions, a homogeneous mask (H) containing the average expected EP values (σ=0.53 and ε _r=51 for 3 T reconstructions, σ=0.59 and ε _r=43 for 7 T reconstructions) was used as initialization. We refer to these standard CSI reconstructions as H‐CSI.

2.2.4. MR‐CSI and DL‐CSI

As hybrid approaches, we used the performed MR‐EPT and DL‐EPT reconstructions as initialization masks for 3D CSI‐EPT reconstructions. Hereafter we call these two hybrid approaches MR‐CSI and DL‐CSI.

Note that DL‐EPT reconstructions are available only at 3 T. Therefore, for DL‐CSI reconstructions at 7 T, we also used as initialization step the DL reconstruction at 3 T.

2.3. Statistic evaluation and constraints

For all these five methods, EP reconstructions are first performed on data with signal‐to‐noise ratio (SNR) ≈ 1000 by adding Gaussian noise to the real and imaginary parts of the simulated complex ${\widehat{B}}_1^{+}$ fields. Then reconstructions are performed for a realistic SNR value as indicated by Gavazzi et al. ¹⁹ (SNR ≈ 100). Note that CSI‐EPT is based on the scattered field, which produces different effective SNR values at different field strengths: the SNR of the scattered part of ${\widehat{B}}_1^{+}$ is 22 at 3 T, while it is 92 at 7 T.

Since EP reconstruction methods may lead to voxels with unrealistic EP values, minimum and maximum constraints were applied to the final reconstructions, that is, bounding the conductivity in the range [0–2.6] S/m and the permittivity in the range [1–100], where the maximum values are approximately 20% higher than the maximum EP values present in the ground‐truth dielectric models.

For all these reconstructions, mean and standard deviation values were computed in the white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) regions as a proxy of accuracy and precision of these reconstruction methods. Moreover, to evaluate the overall reconstruction accuracy among the investigated EP reconstruction methods, the global relative residual error (RRE) was computed as

$$\text{RRE}=\frac{||\widehat{x}-\tilde{x}||}{||\widehat{x}||},$$ (3)

where $\widehat{x}$ depicts the true conductivity or relative permittivity and $\tilde{x}$ the reconstructed one, and the norm is the Euclidean norm defined over the complete domain of interest.

3. RESULTS

Figure 1 shows EP reconstructions at 3 and 7 T for noiseless simulated ${\widehat{B}}_1^{+}$ data using the standard 3D CSI‐EPT (H‐CSI) and the two hybrid CSI‐EPT approaches (MR‐CSI and DL‐CSI). All the results are obtained after 500 CSI iterations (taking around 60 minutes on an Intel i7‐6700 CPU for this reconstruction domain size), at which point the mismatch functional has decreased to a value of about 5×10⁻⁵.

FIGURE 1.

Reconstructed EP maps from different EPT reconstruction approaches for the male head model at 3 and 7 T based on ${\widehat{B}}_1^{+}$ data with an SNR of 1000. A–H, Conductivity. I–P, Permittivity

At 3 T, H‐CSI produces very poor EP maps. The conductivity map shows a smooth reconstruction, with underestimations of the high conductivity values. The permittivity map shows less of the underlying tissue structure and the white matter region contains clear overestimations. Furthermore, distorted reconstructions are observed in the center of the object in both EP maps, corresponding to the region with low electric field strength. For MR‐CSI conductivity reconstructions, an improvement is observed, especially at the periphery of the head, that is, away from the low $|\widehat{\mathbf{E}}|$‐field region. However, in the low $|\widehat{\mathbf{E}}|$‐field region (ventricles), conductivity reconstructions are still erroneous. MR‐CSI permittivity reconstructions shows severe distortions throughout the brain, reflecting the severe boundary errors of standard MR‐EPT reconstructions (see Figure S1). The DL‐CSI approach shows better tissue reconstructions of the ventricles compared with H‐CSI and MR‐CSI. They are clearly visible in both the conductivity and permittivity maps. However, even though the reconstructed DL‐CSI EP values are close to the ground‐truth values, small errors arising from the DL‐EPT reconstructions used as initialization step (see Figure S1) are visible at the periphery of the head. Note that the DL‐CSI reconstructions presented in this figure assume noiseless data, while DL‐EPT reconstructions used as initialization for DL‐CSI are available only for noisy data, since the available neural network was trained only on the noisy ${\widehat{B}}_1^{+}$ data, to better resemble realistic scenarios from MR‐measurements.

At 7 T, similar results are observed for H‐CSI as at 3 T: a smoothed version for the reconstructed conductivity, and overestimations in the homogeneous WM region for the reconstructed permittivity. Also, the low $|\widehat{\mathbf{E}}|$‐field region, which at 7 T is located further down compared with the 3‐T case (compare the coronal slices of Figure 1b,j with 1f,n), is clearly visible. MR‐CSI at 7 T shows boundary artifacts, which are the result of the intrinsic errors of MR‐EPT at tissue boundaries. However, compared with its initialization (Figure S1), improvements are observed in the conductivity maps. DL‐CSI reconstructions at 7 T (using as initialization DL‐EPT reconstructions at 3 T) show higher structure fidelity compared with DL‐EPT reconstructions, especially around the ventricles.

Figure 2 shows reconstruction results when Gaussian noise is present in the ${\widehat{B}}_1^{+}$ data (SNR=100). The introduction of noise results in nonsignificant differences for H‐CSI reconstructions at both 3 and 7 T. This is in contrast to MR‐CSI, for which initialization maps, that is, EP maps obtained from standard Helmholtz MR‐EPT reconstructions, are extremely sensitive to noise (see Figure S1), thus leading to noise‐corrupted MR‐CSI reconstructions at both 3 and 7 T. DL‐CSI reconstructions are minimally affected by noise, which leads to slightly higher standard deviations in permittivity reconstructions for DL‐CSI compared with DL‐EPT (see Table S1).

FIGURE 2.

Reconstructed EP maps from different EPT reconstruction approaches for the male head model at 3 and 7 T based on ${\widehat{B}}_1^{+}$ data with an SNR of 100. A–H, Conductivity. I–P, Permittivity

In the supplementary material Table S1, the mean and standard deviation values are reported for EP reconstructions in WM, GM, and CSF from noiseless and noisy data for all the aforementioned methods (MR‐EPT, DL‐EPT, H‐CSI, MR‐CSI, and DL‐CSI) as a proxy of accuracy and precision.

Figure 3 gives a qualitative impression of the reconstruction errors of the five different reconstruction methods, by showing the absolute error maps for EP reconstructions from noisy ${\widehat{B}}_1^{+}$ data. For a direct quantitative comparison, the computed RRE in the whole domain is also reported in the figure for each reconstruction method.

FIGURE 3.

Absolute error maps (ground truth − reconstruction) of the reconstructions from the different EPT approaches, for the Duke head model at 3 and 7 T. The values in the subcaptions denote the RRE of the whole volume. A–J, Conductivity. K–T, Permittivity. Note that a DL‐EPT network at 7 T is not available and these reconstructions are therefore not included

MR‐EPT reconstructions show severe errors due to noise amplification in the reconstructed EP maps at 3 T. These errors are reduced for 7‐T MR‐EPT reconstructions. However, the quality of the reconstructed EP maps still remains poor. DL‐EPT reconstructions, available only at 3 T, show good accuracy in homogeneous regions. However, reconstruction errors are present at tissue boundaries, for example, around the ventricles. H‐CSI shows substantial errors arising from the low $|\widehat{\mathbf{E}}|$‐field region. These errors appear as artificial bands/shadow artifacts, which are intrinsically caused by the homogeneous initialization. MR‐CSI is strongly affected by the reconstruction errors present in MR‐EPT, which is used as an initialization guess. Although the RRE is lower compared with MR‐EPT reconstructions, the quality of MR‐CSI is still poor. DL‐CSI reconstructions show good‐quality EP maps. The combined conductivity and permittivity RRE values of DL‐CSI are lower compared with H‐CSI at both 3 and 7 T, showing an advantage in using DL as the initialization guess. Artifacts such as the artificial bands present in H‐CSI reconstructions are highly reduced in DL‐CSI. Improvements are also observed with respect to DL‐EPT (see Figure S1), especially for conductivity reconstructions around the ventricles.

4. DISCUSSION AND CONCLUSION

In this article, we investigated the possible benefits for EP reconstructions achievable by combining standard MR‐EPT, DL‐EPT, and 3D CSI‐EPT (H‐CSI) into a hybrid reconstruction approach, that is, by providing MR‐EPT or DL‐EPT reconstructions as an initialization guess for CSI‐EPT. By doing so, CSI provides data consistency for MR‐EPT or DL‐EPT reconstructions, that is, the data need to satisfy Maxwell's equations, while MR‐EPT and DL‐EPT reconstructions provide, in principle, a better initialization guess for CSI‐EPT compared with the standard approach, which uses a simple homogeneous mask.

Reconstructions obtained with CSI‐EPT depend on the map provided as initialization guess. If a homogeneous mask is provided (H‐CSI), sharp tissue boundaries are in principle reconstructed for a noiseless situation only after a larger number of iterations (for example, about 10 000). ¹² However, for realistic cases including noise, a large number of iterations leads to noise overfitting. To limit this, fewer iterations are performed, resulting in smoother EP maps. Moreover, H‐CSI can converge to suboptimal results, and regions of very low values in the reconstructed EP maps can occur, even in noiseless cases. It is therefore critical to provide CSI‐EPT with a good initialization guess.

If available methods like MR‐EPT are used as the initial guess, improvements can be observed in noiseless cases. However, for realistic scenarios (SNR=100), MR‐EPT reconstructions are severely affected by noise. As shown in this work, this also corrupts MR‐CSI reconstructions.

DL‐EPT is more noise‐robust than MR‐EPT, as the adopted network was trained on noisy data. The training of each network takes approximately 5 hours on a GPU NVIDIA Tesla P100 16‐GB RAM, while DL‐EPT reconstructions can be performed within a minute. An important issue for DL‐EPT is the need for a large training dataset to allow for better generalization, that is, to use DL‐EPT for in vivo models that have different features from the ones the algorithm was trained on. Indeed, initial results for in vivo cases show that geometric variability in training datasets is a major cause of reconstruction errors for DL‐EPT. ²⁰ The inclusion of a tissue mask can partly overcome these issues; however, it increases the chance of overfitting. ¹³ By using CSI‐EPT after DL‐EPT, data consistency, that is, satisfying Maxwell's equations, is incorporated implicitly in the reconstruction process by minimizing the error between simulated and measured ${\widehat{B}}_1^{+}$, resulting in more accurate conductivity reconstructions at 3 and 7 T compared with the other methods presented in this work. DL‐CSI is able to provide data consistency, which, in principle, can allow higher fidelity in the reconstructed tissue structures. For permittivity reconstructions, we did not observe a substantial improvement in the reconstruction accuracy for DL‐CSI compared with DL‐EPT. Still, DL‐CSI reconstructions at 7 T show an improvement compared with DL‐CSI reconstructions at 3 T, indicating the benefit of high field strength MRI for permittivity reconstructions.

A limitation in this work is that DL‐EPT reconstructions were not available at 7 T, since the network was only trained on simulated noisy data at 3 T. We believe that by using 7 T DL‐EPT reconstructions in the future, DL‐CSI reconstructions can improve further due to the higher imprinting of EPs in the measured fields, leading to higher spatial variations in the ${\widehat{B}}_1^{+}$ field magnitude and phase with increasing field strength. A weakness of CSI‐EPT is that knowledge of the transmit phase is required, which is not directly available from MRI measurements. For this purpose, the transceive phase assumption is often applied, but this becomes less accurate at higher field strengths. ¹⁷ An alternative solution, consisting of an iterative phase update, has recently been suggested, showing promising results. ²¹ The implementation of this solution for 3D CSI‐EPT at high fields will be the focus of future work. Finally, we note that all the methods discussed could benefit from the inclusion of a segmentation step in the reconstruction pipeline, since it could help to resolve initialization and boundary issues.

In conclusion, the combination of noise‐robust DL‐EPT reconstructions and 3D CSI‐EPT reconstructions allows, in principle, better generalization, since CSI‐EPT introduces data consistency for the subject at hand. This might reduce the need for an exhaustive training dataset for DL‐EPT. Meanwhile, using DL‐EPT as initialization for CSI‐EPT improves the quality and accuracy of standard 3D CSI‐EPT reconstructions.

Supporting information

Figure S1: MR‐EPT and DL‐EPT reconstructions from 3 T and 7 T B₁ ⁺ data of the Duke head model. These reconstruction are used as initialization for MR‐CSI and DL‐CSI. Note that a DL‐EPT network at 7 T is not available and these reconstructions are therefore not included. Conductivity (a‐h) and permittivity (i‐p).

TABLE S1 The mean and standard deviation of the reconstructed EPs for the segmented regions of white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) for the Duke head model for two noise levels at 3 and 7 T. The difference in averages between the different SNR cases in MR‐EPT (and thus MR‐CSI) is due to the applied minimum and maximum constraint

Leijsen R, van den Berg C, Webb A, Remis R, Mandija S. Combining deep learning and 3D contrast source inversion in MR‐based electrical properties tomography. NMR in Biomedicine. 2022;35(4):e4211. 10.1002/nbm.4211