Opening a new window on MR-based Electrical Properties Tomography with deep learning

Abstract

In the radiofrequency (RF) range, the electrical properties of tissues (EPs: conductivity and permittivity) are modulated by the ionic and water content, which change for pathological conditions. Information on tissues EPs can be used e.g. in oncology as a biomarker. The inability of MR-Electrical Properties Tomography techniques (MR-EPT) to accurately reconstruct tissue EPs by relating MR measurements of the transmit RF field to the EPs limits their clinical applicability. Instead of employing electromagnetic models posing strict requirements on the measured MRI quantities, we propose a data driven approach where the electrical properties reconstruction problem can be casted as a supervised deep learning task (DL-EPT). DL-EPT reconstructions for simulations and MR measurements at 3 Tesla on phantoms and human brains using a conditional generative adversarial network demonstrate high quality EPs reconstructions and greatly improved precision compared to conventional MR-EPT. The supervised learning approach leverages the strength of electromagnetic simulations, allowing circumvention of inaccessible MR electromagnetic quantities. Since DL-EPT is more noise-robust than MR-EPT, the requirements for MR acquisitions can be relaxed. This could be a major step forward to turn electrical properties tomography into a reliable biomarker where pathological conditions can be revealed and characterized by abnormalities in tissue electrical properties.

Introduction

Non-invasive measurements of human tissue electrical properties (EPs), namely conductivity σ and relative permittivity ε_r, are a challenge that attracted several research groups in the past decades^1,2. These properties determine how electromagnetic (EM) fields, such as the MR radiofrequency fields (RF: 64–300 MHz), interact with human tissues. Tissue EPs depend on the tissue structure and composition (water content and ionic concentration). In particular, at RF frequencies where MRI works, tissue conductivity is modulated by the total ionic concentration, which varies in presence of pathologies. Several studies already showed a change in tissue conductivity in presence of tumors^3–8. Therefore, non-invasive measurements of tissue EPs could in principle be used as a new biomarker in oncology for diagnostic purposes and treatment monitoring⁷.

The possibility to non-invasively measure tissue EPs at RF frequencies with clinical MRI systems was first suggested in the early 1990s⁹. However, systematic research only started in the last decade, creating a new branch of research called MR-Electrical Properties Tomography (MR-EPT)¹⁰. Using standard MR hardware, MR-EPT is able to reconstruct tissue EPs from measurements of the RF transmit magnetic field, i.e. the circularly polarized transverse magnetic field referred to as the ${\tilde{\mathrm{B}}}_1^{+}$ field. This field consists of incident and scattered field terms, where the latter component includes contributions from conduction and displacement currents, and thus contains the desired EPs information.

By applying the homogenous Helmholtz equation to the measured ${\tilde{\mathrm{B}}}_1^{+}$ field, EPs map can be reconstructed^9–11. According to this analytical reconstruction model, tissue EPs maps can be obtained by computing second order spatial derivatives of the measured ${\tilde{\mathrm{B}}}_1^{+}$ field^11,12. Spatial derivatives are computed by applying a filter (in this case a 2^nd order finite difference filter) to the ${\tilde{\mathrm{B}}}_1^{+}$ field data, resulting directly in EPs maps. However, this operation is highly sensitive to the intrinsic noise in the MR measurements, and consequently the reconstructed EPs maps lack precision^13,14. To mitigate the impact of noise in the reconstructed EPs maps, large derivative filters in combination with image filters and large voxel sizes are commonly used². Unfortunately, this comes at the cost of severe errors at tissue boundaries, thus making MR-EPT reconstructions challenging, especially for highly spatially convoluted tissue structures such as the human brain¹⁴. Furthermore, for clinical MRI systems (1.5 and 3 Tesla) permittivity reconstructions are not feasible, since the electromagnetic imprint of related displacement currents is too low at these frequencies.

Recently, alternative analytical reconstruction techniques have been presented to improve the quality of MR-EPT reconstructions^15–19. However, these techniques require complex RF setups (multi-transmit array), and high field MR scanners (7 Tesla) are needed to achieve sufficient signal-to-noise-ratio (SNR). From a fundamental point of view, these analytical reconstruction techniques are attractive due to their direct forward mathematical formulation allowing fast reconstructions. However, these methods are sensitive to noise in the input data and therefore require relatively high SNR levels that are not always feasible at clinical MR field strengths.

To overcome this requirement, algebraic algorithms employing a more general inverse approach based on iterative minimization have been suggested^20–23. These methods behave better under noisy conditions. However, this comes at the expense of a higher computational load, challenges related to local minima and more complex electromagnetic modeling. Moreover, these algebraic algorithms need a-priori information (e.g. incident MR electric field), which is not always available. Although some promising results from simulated data have been presented, accurate in-vivo reconstructions have not been shown yet.

Inspired by MR fingerprinting, a different reconstruction method called dictionary-based EPT has been recently proposed²⁴. This method formulated the EPT reconstruction problem as a classification problem and it reconstructs tissue electrical conductivity on a 3D patch level by assigning the conductivity value that corresponds to the simulated ${\tilde{\mathrm{B}}}_1^{+}$ profile that best matched the measured ${\tilde{\mathrm{B}}}_1^{+}$ profile. First results showed the potential of such a matching approach for conductivity reconstructions. No permittivity reconstructions were presented yet. The presented methodology, which exploits a priori data, is not based on data driven learning strategy as in deep learning, where large amounts of realistic data is used to train neural networks.

Instead of relying on analytical or algebraic reconstruction techniques derived from electromagnetic theory, and given the potential of data driven approaches, in this work we investigate the feasibility of using a data driven, supervised deep learning (DL) approach for EPs reconstructions. Deep learning has recently been successfully applied to inverse problems including MRI image reconstruction^25–30. To the best of our knowledge, this is the first time that deep learning is used for EPs reconstructions. Hereafter, we refer to this approach as Deep Learning Electrical Properties Tomography (DL-EPT).

Given the promising performance of Convolutional Neural Networks (CNNs), and in particular of Conditional Generative Adversarial Networks (cGANs)³¹, in this work we train a cGAN to perform EPs reconstructions. Contrary to state-of-the-art MR-EPT techniques which require electromagnetic quantities that are not directly accessible from MRI measurements (e.g. the phase of the MR transmit field, ${\tilde{\varphi }}^{+}$), in DL-EPT a surrogate analytical reconstruction model can be learnt using only MR accessible quantities (e.g. the magnitude of the MR transmit field ${\tilde{\mathrm{B}}}_1^{+}$, and the transceive phase ${\tilde{\varphi }}^{+}$). Electromagnetic simulations including realistic RF coil models, phantoms and body models are used to generate the training dataset. Nowadays, these datasets can be easily generated by exploiting the availability of sophisticated electromagnetic solvers, which allow realistic electromagnetic simulations (e.g. Sim4Life; CST; COMSOL; Remcom). In this way, a high degree of a-priori knowledge, such as the MRI coil setup, can be introduced.

In this work, DL-EPT reconstructions from simulations on phantoms and human head models as well as from phantom and in-vivo MR measurements at 3 Tesla using a clinically available MR setup are presented. The accuracy and precision of the reconstructed EPs maps are assessed, and the impact of different SNR levels has also been investigated. For comparison purposes, Helmholtz-based MR-EPT reconstructions (H-EPT) are presented as a reference for the phantoms and the head models simulations. Although the aim of this study is a proof of principle of DL-EPT, and not an investigation into optimal network and choice of learning parameters, several options are considered. In particular, two cGANs are employed: cGAN_mask, and cGAN_tissue. The former has in input the MR transit ${\tilde{\mathrm{B}}}_1^{+}$field magnitude, the phase ${\tilde{\varphi }}^{+}$ (proportional to the transceive phase ${\tilde{\varphi }}^{\pm }$ measurable in an MR experiment), and a binary mask (1: tissue, 0: air). In the latter, the binary mask is replaced by pseudo Spin Echo MRI images providing tissue contrast information. To the best of our knowledge, with this work we show for the first time that deep learning can provide improved reconstructions of electrical conductivity and permittivity using clinically available MRI scanners, coil setups, and realistic SNR levels.

Results

In Fig. 1, H-EPT and DL-EPT reconstructions are presented for the phantom model 42 including realistic noise (see Supplementary Materials and Methods – Phantom and Head Models). This phantom was used for in-silica testing of the cGAN_mask, and was not included in the training set. Additionally, reconstructions from MRI measurements at 3 T are presented for a cylindrical, homogeneous phantom with the same EPs values. The mean and standard deviation (SD) values of the reconstructed EPs maps are also reported in Fig. 1. To avoid boundary regions that cannot be reconstructed accurately in H-EPT, a smaller region of interest was considered for this evaluation (see Supplementary Fig. S7).

Figure 1.

Conductivity and permittivity maps reconstructed using Helmholtz-based MR-EPT (H-EPT) (b,f) and cGAN_mask (c,g) for the phantom model 42. Ground truth EPs maps (a,e). cGAN_mask EPs reconstructions from MRI measurements at 3 Tesla (d,h). The reported numbers are the mean ± SD of the reconstructed EPs values inside a region of interest (see Supplementary Fig. S7).

Phantom H-EPT reconstructions from simulated data show accurate mean EPs values after exclusion of boundary regions. However, the reported high SD values indicate lack of precision in the reconstructed EPs values due to severe noise amplification (see profiles in Supplementary Fig. S7). The need of high SNR levels is one of the main limitation of current analytical MR-EPT reconstruction methods.

On the contrary, DL-EPT reconstructions from simulated phantom data are less affected by noise. As reported in Fig. 1, DL-EPT reconstructions show a much better precision (low SD) at the cost of a small inaccuracy in the reconstructed mean EPs values (relative error <5%).

DL-EPT reconstructions from MR measurements confirm the results observed in simulations, thus demonstrating the feasibility of reconstructing EPs from MR measurements using DL-EPT. Additionally, permittivity reconstructions are now feasible at 3T, contrary to standard MR-EPT methods.

In Fig. 2, H-EPT, cGAN_mask, and cGAN_tissue EPs reconstructions are shown for the head model Duke M0, which was used for in-silica testing including noise. This head model was not included in the training set. Mean and standard deviation values for the white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) are reported in Table 1.

Figure 2.

Head Model Duke M0 conductivity and permittivity reconstructions at 3 Tesla: (a,e) Ground truth, (b,f) H-EPT, (c,g) cGAN_mask, (d,h) cGAN_tissue.

Table 1.

Reconstructed EPs values for the Human Brain WM, GM and CSF.

	Conductivity σ [S/m]						Permittivity εr [−]
	WM		GM		CSF		WM		GM		CSF
	mean	(SD)	mean	(SD)	mean	(SD)	mean	(SD)	mean	(SD)	mean	(SD)
H-EPT Duke M0	0.33	(0.85)	0.64	(1.27)	3.22	(4.97)	52.9	(130)	67.8	(124)	−43	(350)
cGANmask Duke M0	0.34	(0.15)	0.56	(0.18)	1.83	(0.42)	52.5	(3.9)	72.9	(6.5)	84.1	(3.1)
cGANtissue Duke M0	0.34	(0.03)	0.60	(0.05)	2.03	(0.14)	53.1	(1.3)	74.3	(2.1)	84.4	(1.2)
cGANmask in-vivo subject 1	0.39	(0.08)	0.49	(0.16)	0.85	(0.48)	57.3	(7.2)	61.3	(7.9)	70.4	(10.0)
cGANtissue in-vivo subject 1	0.37	(0.04)	0.53	(0.18)	1.67	(0.47)	54.4	(3.2)	66.0	(6.9)	80.1	(4.9)
Reference	0.34	(−)	0.59	(−)	2.14	(−)	52.6	(−)	73.4	(−)	84	(−)

Mean and SD (inside brackets) of the reconstructed EPs values in the WM, GM, and CSF for the head model Duke M0 using H-EPT, cGAN_mask, and cGAN_tissue, and from in-vivo MR measurements on the first subject using cGAN_mask, and cGAN_tissue. A 3 voxels erosion was performed for each tissue type to avoid boundary regions, since these regions cannot be reconstructed accurately with H-EPT.

H-EPT conductivity reconstructions are severely affected by noise and boundary errors, as previously observed for the phantom reconstructions. Although average H-EPT conductivity and permittivity values for WM and GM have a relative error <10% with respect to input (ground truth) values after excluding boundary regions, the high standard deviations indicate that H-EPT is not suitable to reconstruct EPs on a voxel basis for highly spatially convoluted tissues.

If DL-EPT is used employing the cGAN_mask, the precision of EPs reconstructions is greatly improved (much lower SD). If tissue contrast information (i.e. pseudo Spin Echo MRI images) is provided as additional input for the neural network (cGAN_tissue), the precision of the reconstructed EPs maps is further improved, and the computed mean EPs values (Table 1) agree with the input (ground truth) values. As shown in Fig. 2 and Supplementary Fig. S10, the use of tissue contrast as a-priori information leads to less boundary errors, which are instead a major source of error in H-EPT reconstructions.

In Fig. 3, DL-EPT reconstructions from in-vivo MR measurements at 3T on a healthy subject are shown. Mean and standard deviation values are also reported in Table 1. DL-EPT reconstructions from other two healthy subjects are presented in the Supplementary Results – EPs Reconstructions (Supplementary Fig. S11 and Table S7). The presented results show good quality EPs maps ad exception for the head periphery and the ventricles where cGAN_mask demonstrates less performance. If tissue contrast information is provided, errors at tissue boundaries are considerably reduced. This confirms what was previously observed for the reconstructions from simulated data and shows the feasibility of using DL-EPT to reconstruct in-vivo EPs from MR measurements.

Figure 3.

DL-EPT conductivity and permittivity reconstructions from MR measurements on the first subject using cGAN_mask and cGAN_tissue (a–d). The correspondent MRI magnitude image is also shown as a reference (e).

Finally, in Fig. 4, a comparison between H-EPT and cGAN_mask EPs reconstructions for the head model Duke M0 with a tumor inclusion (sphere, radius 1.5 cm) is presented. Reconstructed mean EPs values and standard deviations of the tumor inclusion are also reported in the figure.

Figure 4.

Ground truth EPs maps for Duke M0 with a tumor inclusion and H-EPT and cGAN_mask EPs reconstructions. The tumor contour is highlighted with a white circle in the reconstructed EPs maps. The numbers reported in the figure are the mean ± SD of the tumor EPs values.

Correct identification of the tumor region is difficult for H-EPT reconstructions, which are highly corrupted by noise. Instead, cGAN_mask EPs reconstructions clearly show a tissue-tumor contrast, especially in the permittivity map. The presented DL-EPT reconstructions show an underestimation for the tumor conductivity value (relative error ≈ 15%), while the reconstructed tumor permittivity value is more accurate (relative error <5%). As a reference, DL-EPT reconstructions using the same network parameters and the same Duke model without tumor inclusion are reported in the Supplementary Materials (Supplementary Fig. S13).

Discussion

In this work, a novel approach for EPs reconstructions is presented, namely deep learning electrical properties tomography (DL-EPT). This technique is based on a data driven learning task, where the training data are obtained from a large number of realistic electromagnetic simulations. We show for the first time that DL-EPT allows high quality conductivity and permittivity reconstructions of human brain tissues at clinically available MR field strengths using standard MR hardware. This has been investigated using in-silica data from realistic phantom and head models, as well as phantom and in-vivo MR measurements at 3 Tesla. The presented results show good accuracy and most notably precision in the reconstructed EPs maps on a voxel basis, demonstrating a large improvement with respect to MR-EPT techniques. Furthermore, DL-EPT is noise-robust and preserves boundary information, while these two aspects are the major issues for conventional MR-EPT techniques.

DL-EPT differs significantly from conventional MR-EPT techniques employing analytical or algebraic reconstruction models. The popular Helmholtz MR-EPT technique, an example of an analytical reconstruction technique, requires the computation of spatial derivatives on measured data¹⁴. This computation is performed by convolving the measured, complex ${\tilde{\mathrm{B}}}_1^{+}$ field with large finite difference kernels such as the 3D kernel adopted in this work¹¹, or the Savitzky-Golay kernel³². These kernels, combined with image filters to further suppress the impact of noise^18,22, lead to a much coarser effective resolution (order of 1 cm) and result in severe errors at tissue boundaries. On the other hand, algebraic MR-EPT reconstruction techniques employing iterative minimization, such as CSI-EPT²⁰, should be more noise-robust. However, these methods require a large degree of regularization to stabilize noise augmentation in specific regions. Furthermore, these reconstruction techniques employ forward models formulated in electromagnetic quantities that are not always accessible with MRI, such as the phase of the transmit ${\tilde{\mathrm{B}}}_1^{+}$ MR field and the incident electric field. Currently, high quality experimental reconstructions using MR-EPT reconstructions are not yet available at clinical MRI field strengths (1.5 and 3 Tesla).

Given these limitations for MR-EPT reconstructions, we investigated the feasibility of using supervised deep learning to reconstruct EPs from accessible MR quantities. Crucial for the success of DL-EPT is the training part where a large degree of a-priori knowledge can be introduced by simulating a realistic coil setup and including realistic head models. The training requires a high number of unique complex ${\tilde{\mathrm{B}}}_1^{+}$ fields, which can be obtained by means of sophisticated, realistic electromagnetic simulations. Realistic electromagnetic simulations are nowadays possible with commonly available electromagnetic simulation software, and therefore represent an elegant solution to overcome the need of a high amount of MR data for training. Another fundamental advantage is that the DL-EPT reconstructions have additional flexibilities in the choice of the input parameters, i.e. the training can be performed based on quantities that are accessible with MRI measurements (e.g. transceive phase). This is contrary to conventional MR-EPT reconstructions models based on electromagnetic theory, which prescribes rigidly the required electromagnetic quantities that need to be measured.

Our results indicate that not only conductivity reconstructions at clinical MRI field strengths (1.5 and 3 Tesla) are feasible, but also permittivity maps can be obtained using DL-EPT. The latter were not yet feasible with conventional MR-EPT approaches due to insufficient SNR levels at clinical field strengths^13,33. Preliminary investigations indicate that DL-EPT is more noise-robust than conventional MR-EPT reconstructions for SNR levels achievable in clinical MRI experiments (SNR ≈ 100). This is highly appealing, as it would permit to relax the requirements in terms of MRI data acquisition, allowing EPs measurements in clinical settings. Erroneous EPs reconstructions appear at SNR levels around 20 (see Supplementary Results – Impact of SNR). Low SNR values combined with CSF pulsation³⁴ could be the cause of the observed inaccuracies at the periphery of the head and around the ventricles when cGAN_mask is used. More accurate EPs reconstructions at tissue boundaries can be obtained by including MR image information (tissue contrast) as a-priori knowledge. Future works should investigate whether other strategies are possible, e.g. providing the network with only boundary information instead of full tissue contrast information.

Of course, the use of a-priori knowledge during training could also create biased reconstructions for cases not included in the training phase. This would generally be the case for patients with pathologies. To test this risk, we provided the cGAN_mask with a pathological case that was not present in the training set, i.e. a head model including a brain tumor with altered EPs. In case of overfitting, which is a known issue for deep learning, reconstructions would not work anymore. Preliminary results at 3 Tesla seem to indicate that DL-EPT can provide a better tumor-normal tissue contrast than MR-EPT.

The presented results indicate the potential of DL-EPT for EPs reconstructions at clinical field strengths and standard MR coil setup. However, more studies are warranted to further validate and generalize DL-EPT.

It has to be further investigated what the impact of different learning parameters is on DL-EPT reconstructions and whether a single network can be trained to generalize to other field strengths and coil setups. We believe that a larger amount of diverse training data is needed for these purposes. Although optimum tuning of network parameters is beyond the scope of the current work, first results indicate that the quality of EPs reconstructions using cGAN_mask improve for different choices of learning parameters (Supplementary Fig. S13 and Table S9).

Additionally, it should be investigated whether the inclusion of in-vivo data during training would be beneficial to allow more accurate in-vivo EPs reconstructions. In-vivo measurements are affected by artifacts such as pulsation and motion, which are not present in simulations. MRI measurements of the ${\tilde{\mathrm{B}}}_1^{+}$ field are also corrupted by noise propagation and systematic errors which depend on the adopted ${\tilde{\mathrm{B}}}_1^{+}$ measurement technique³⁵. These artifacts and variations in ${\tilde{\mathrm{B}}}_1^{+}$ fields may play a crucial role, resulting in less quality DL-EPT reconstructions from in-vivo MRI measurements compared to DL-EPT reconstructions from simulated data. For accurate in-vivo reconstructions, these artifacts may have to be included in simulations. Furthermore, it could be considered to include in-vivo EPT reconstructions in training data, even though the lack of ground truth EPs values for in-vivo cases might increase the level of complexity³⁶.

Moreover, it will be fundamental to understand whether it will be necessary to include an exhaustive database of realistic pathological models (e.g. brain tumors) in the training set for accurate DL-EPT reconstructions of patients. We hypothesize that including more different training data might allow reducing the observed inaccuracies for cases not present in the training set. Future works should address these questions.

In this work, 2D DL-EPT reconstructions were performed due to the available network and computational power. However, given the 3D nature of the EPT reconstruction problem, the use of 3D neural networks for 3D DL-EPT reconstructions should be further studied, and the benefits of 3D patch-based approaches compared to image-based approaches, such as the one adopted in this work, should be addressed. We believe that 3D patch based approaches might allow better generalization of local features and less discontinuities in EPs reconstructions between slices.

In conclusion, to the best of our knowledge, this is the first demonstration of the feasibility of reconstructing in-vivo EPs from MR measurements using supervised deep learning. Although this work is a first proof of principle without aiming at identifying the best network architecture, which is beyond the current scope, the presented results indicate major improvements in the quality of the reconstructed EPs maps compared to MR-EPT approaches. Even permittivity reconstructions are now feasible at 3T with a standard and widely available coil setup. We showed that DL-EPT is noise-robust, thus the requirements in terms of SNR can be relaxed. This will allow faster imaging protocols and higher spatial resolutions. Moreover, DL-EPT can be trained with the transceive phase, thus circumventing the issue of measuring the ${\tilde{\mathrm{B}}}_1^{+}$phase, which is not directly accessible with MRI. The major finding of this work is that the application of supervised training for EPT reconstructions greatly improves the quality of the EP maps. This could have great impact in MR diagnostics as it would turn electrical properties mapping into a new reliable biomarker to locate and characterize pathological conditions based on differences in tissue ionic concentrations resulting in different tissue electrical properties.

Materials and Methods

Database construction

A database consisting of 42 homogeneous phantom models (diameter: 12 cm, length: 12 cm) and 20 head models with piecewise constant EPs values was created in Sim4Life (ZMT AG, Zurich, CH). Different EPs values were assigned to each phantom model and to the WM, GM, and CSF of the adopted head models (Duke and Ella, the Virtual Family³⁷) (see Supplementary Materials and Methods – Phantom and Head Models). These models were placed inside a realistic birdcage body coil model resonant at 128 MHz, thus mimicking the experimental MR setup. With this setup, FDTD simulations were performed in Sim4Life to obtain realistic 3D ${\tilde{\mathrm{B}}}_1^{+}$ field magnitude and transceive phase (${\tilde{\varphi }}^{+}$) maps. Thermal noise was included by independently adding Gaussian noise to the real and imaginary parts of the simulated fields (noiseless ${\tilde{\mathrm{B}}}_1^{+}$field magnitude and transceive phase).

The final SNR was 90 for the obtained ${\tilde{\mathrm{B}}}_1^{+}$ magnitude and the precision of the obtained phase ${\tilde{\varphi }}^{+}$, proportional to the transceive phase, was 9 × 10⁻³ rad (see Supplementary Materials and Methods – Database Construction). This mimics realistic SNR levels in MR experiments. By means of these simulations, 2170 unique 2D complex ${\tilde{\mathrm{B}}}_1^{+}$ field distributions were generated (25 slices for each phantom model and 56 slices for each head model).

Neural network

The neural network used for EPs reconstructions was a Conditional Generative Adversarial Network (cGAN)³¹. In this type of networks, two sub-networks (generator G, and discriminator D) compete with each other in a min-max optimization game during the training phase, in order to learn a conditional generative model. The generator network tries to generate EPs maps from the input images, while the discriminator network tries to discriminate the generated EPs maps from the EPs maps in the training set (ground truth). Like in Isola et al.³¹, the generator was a U-Net and the discriminator was a convolutional PatchGAN classifier. In Pathak et al.³⁸, it was shown that using a cGAN combined with a L2 norm resulted in sharper images compared to a U-Net³⁹. Afterwards, in Isola et al.³¹ it was demonstrated that the use of the L1 norm preserved the boundaries better in the reconstructed images. For EPs reconstructions, it is important to achieve good accuracy at tissue boundaries. Based on these observations, we combined a cGAN with both L1 and L2 norms, yielding to the following cost function (F):

$$\mathrm{F}=\arg {min}_{\mathrm{G}}{max}_{\mathrm{D}}{\lambda }_{\mathrm{c}\mathrm{G}\mathrm{A}\mathrm{N}}{\mathcal{L}}_{\mathrm{c}\mathrm{G}\mathrm{A}\mathrm{N}}(\mathrm{G},\mathrm{D})+{\lambda }_{\mathrm{L}1}{\mathcal{L}}_{\mathrm{L}1}(\mathrm{G})+{\lambda }_{\mathrm{L}2}{\mathcal{L}}_{\mathrm{L}2}(\mathrm{G}).$$ 1

${ℒ}_{cGAN}\left(G,D\right)$ is the GAN objective, ${ℒ}_{L1}$ and ${ℒ}_{L2}$ are respectively the L1 and L2 distance between the ground truth and the output, and λ_GAN, λ_L1, and λ_L2 are the corresponding weights (see Supplementary Materials and Methods – Choice of cGAN).

This network was implemented in TensorFlow⁴⁰ and trained in about four hours on a GPU (NVIDIA Tesla P100 16GB RAM). After training, 2D EPs reconstructions could be performed in less than 1 minute for a volume of 256 × 256 voxels in plane and 56 slices.

We first investigated the effect of providing the network only with EM quantities (cGAN_mask). Then, we investigated the impact of providing the network with additional information, i.e. MRI tissue contrast (cGAN_tissue). Although network optimization is beyond the scope of this work, we also investigated the impact of few different learning parameters on DL-EPT reconstructions for cGAN_mask (Supplementary Fig. S13).

DL-EPT: cGAN_mask

For the training, 2014 2D complex ${\tilde{\mathrm{B}}}_1^{+}$ field distributions were generated using all the simulated models, except for the phantom models 12, 24, 38, and 42, and the head model Duke M0. The inputs for the neural network were: the ${\tilde{\mathrm{B}}}_1^{+}$ magnitude, the phase ${\tilde{\varphi }}^{+}$ (proportional to the transceive phase), and a binary mask (1: object, 0: air). Since only a binary mask was provided as third input and not information about tissue structure, we define this network as cGAN_mask. To reduce the complexity of the problem, two networks with the same input data were trained separately for conductivity and permittivity reconstructions using the same combinations of λ-weights.

For the validation, the complex ${\tilde{\mathrm{B}}}_1^{+}$ field distributions of the phantom models 12 and 24 were used. Although the aim of the paper was not to find the best combination of λ_GAN, λ_L1, and λ_L2 weights, we investigated the impact of various combinations of these parameters on the reconstructed EPs maps. The parameters combination with the lowest average normalized-root-mean-square error (NRMSE) computed over the conductivity and permittivity reconstructions from the validation set was selected for testing: λ_GAN = 2, λ_L1 = 100, and λ_L2 = 200 (see Supplementary Materials and Methods – Choice of cGAN). Among the combinations tested, we investigated whether setting λ_GAN = 0, i.e. employing a less sophisticated network (U-Net)³⁹, would be sufficient for EPs reconstructions (see Supplementary Results – Comparison U-Net and cGAN_mask).

For testing of the selected cGAN_mask, the complex ${\tilde{\mathrm{B}}}_1^{+}$ field distributions of the phantom models 38 and 42, and Duke model M0 were used. The performed realistic electromagnetic simulations provide a controlled environment in which knowledge of the ground truth, i.e. conductivity and permittivity, is possible. This ensured correct assessment of the accuracy (absolute errors: ∆σ, and ∆ε_r) and precision (standard deviation SD) of the performed EPs reconstructions. Additionally, this network was tested using phantom an in-vivo MR measurements. The adopted phantom was a homogeneous, agar-based phantom: diameter: 13 cm, length: 15 cm, σ: 0.88 S/m; ε_r: 80, obtained from probe measurements at 21 °C (85070E, Agilent Technologies, Santa Clara, CA, USA). In-vivo MR measurements were performed on three healthy subjects (male, mean age 26, SD 2.6), after obtaining written informed consent. This was approved by the local institutional review board of the University Medical Center Utrecht, and carried out in accordance with the relevant guidelines and regulations.

Furthermore, to test the generalizability, we investigated the feasibility of detecting a tumor without having trained the neural network with brain tumor models and without providing any information on tissue structure. For this purpose, a head tumor model was created by placing one sphere inside Duke M0 (radius 1.5 cm, σ: 1.4 S/m; ε_r: 73). For this test, the parameter combination with the lowest average NRMSE value computed over conductivity and permittivity reconstructions in the WM, GM and CSF of Duke M0 was chosen: λ_GAN = 2, λ_L1 = 1000, and λ_L2 = 2000 (see Supplementary Table S9). DL-EPT reconstructions for Duke M0 using these network parameters are shown as a reference in the supplementary materials (Supplementary Fig. S13).

DL-EPT: cGAN_tissue

Since MRI images show good contrast between different tissues, we investigated whether providing tissue contrast information as third input instead of adopting a simple mask would improve the EPs reconstructions for the human brain. We therefore trained a cGAN using only the 1064 2D complex ${\tilde{\mathrm{B}}}_1^{+}$ field distributions of the brain models (except for Duke M0, which was used for testing) and the combination of λ-weights previously chosen for the brain reconstructions from simulations and MR measurements, thus allowing direct comparison with the results obtained using the cGAN_mask. Hence, the inputs were: the ${\tilde{\mathrm{B}}}_1^{+}$ magnitude, the phase ${\tilde{\varphi }}^{+}$ and pseudo Spin echo magnitude images obtained after assigning to each tissue type the corresponding magnitude value that would be measured in those tissues with a Spin Echo sequence (see Supplementary Materials and Methods –MR Sequences). We define this network as cGAN_tissue, since the third input provides tissue contrast information. This network was tested on Duke M0 and in-vivo MRI data.

MRI measurements

MRI measurements were performed with a 3 Tesla MR scanner (Ingenia, Philips HealthCare, Best, The Netherlands) with the body coil in transmit and a 15-channel head coil in receive mode. The ${\tilde{\mathrm{B}}}_1^{+}$ magnitude was measured using a dual-TR (AFI) sequence⁴¹. To map the transceive phase, two single echo Spin Echo (SE) sequences with opposite readout gradient polarities were combined¹¹: (ϕ_SE1 − ϕ_SE2)/2, thus minimizing the impact of eddy-currents related artifacts. To convert the receive phase measured with the head coil to the body coil, as if the body coil would have been used both for transmitting and receiving, the vendor specific algorithm CLEAR (Constant Level of Appearance) was used. The sequence parameters for the phantom and the in-vivo MRI measurements are reported in Supplementary Materials and Methods – MR Sequences.

MR-EPT reconstructions: H-EPT

For comparison purposes, standard Helmholtz-based MR-EPT reconstructions (H-EPT) were also performed for the simulated phantom models 38 (see Supplementary Results – EPs Reconstructions) and 42, and the head model Duke M0 with and without tumor inclusion according to¹¹:

$${\mathrm{\epsilon }}_{\mathrm{r}}\left(\mathbf{r}\right)=\frac{-1}{{\mathrm{\mu }}_0{\mathrm{\epsilon }}_0{\mathrm{\omega }}^2}\mathrm{Re}\left(\frac{{\nabla }^2{\tilde{\mathrm{B}}}_1^{+}(\mathbf{r})}{{\tilde{\mathrm{B}}}_1^{+}(\mathbf{r})}\right)$$ 2

$$\mathrm{\sigma }\left(\mathbf{r}\right)=\frac{1}{{\mathrm{\mu }}_0\mathrm{\omega }}\mathrm{Im}\left(\frac{{\nabla }^2{\tilde{\mathrm{B}}}_1^{+}(\mathbf{r})}{{\tilde{\mathrm{B}}}_1^{+}(\mathbf{r})}\right)$$ 3

with ω: Larmor angular frequency, ε₀/μ₀: free space permittivity/permeability, and r: x/y/z-coordinates. To compute the second order spatial derivatives, a 3D noise-robust kernel was used (7 × 7 × 5 voxels)¹¹.

Data Availability

Data are available from the corresponding author upon reasonable request.

Author Contributions

S.M., E.F.M. and N.R.F.H. performed the research; S.M., E.F.M., N.R.F.H. and C.A.T.v.d.B. wrote the paper; P.R.L. and C.A.T.v.d.B. provided technical assistance.

Competing Interests

The authors declare no competing interests.

Supplementary information

Supplementary information accompanies this paper at 10.1038/s41598-019-45382-x.