Machine learning model with physical constraints for diffuse optical tomography

Abstract

A machine learning model with physical constraints (ML-PC) is introduced to perform diffuse optical tomography (DOT) reconstruction. DOT reconstruction is an ill-posed and under-determined problem, and its quality suffers by model mismatches, complex boundary conditions, tissue-probe contact, noise etc. Here, for the first time, we combine ultrasound-guided DOT with ML to facilitate DOT reconstruction. Our method has two key components: (i) a neural network based on auto-encoder is adopted for DOT reconstruction, and (ii) physical constraints are implemented to achieve accurate reconstruction. Both qualitative and quantitative results demonstrate that the accuracy of the proposed method surpasses that of existing models. In a phantom study, compared with the Born conjugate gradient descent (Born-CGD) reconstruction method, the ML-PC method decreases the mean percentage error of the reconstructed maximum absorption coefficient from 16.41% to 13.4% for high contrast phantoms and from 23.42% to 9.06% for low contrast phantoms, with improved depth distribution of the target absorption maps. In a clinical study, better contrast was obtained between malignant and benign breast lesions, with the ratio of the medians of the maximum absorption coefficient improved from 1.63 to 2.22.

1. Introduction

According to the American Cancer Society, breast cancer is estimated to be the most common type of cancer among women, with more than 284,200 new cases and approximately 44,130 deaths projected in the United States for 2021 [1]. X-ray mammography, ultrasound (US), and magnetic resonance imaging (MRI) are widely used to detect and diagnose breast cancers.

Clinically, mammography is one of the most used imaging modalities for breast cancer screening. It provides high resolution images, but it has significant limitations, such as low contrast, ionizing radiation, relatively low sensitivity in early cancer diagnosis, reduced sensitivity in women with dense breasts, and low specificity that results in a large number of unnecessary biopsies [2]. Ultrasound is often used as an adjunct modality to mammography to diagnose solid vs. cystic lesions; however, its diagnostic utility for solid masses is limited [3]. Although MRI has excellent contrast and high resolution, uses nonionizing radiation, and provides high sensitivity in breast cancer detection, the requisite injection of a contrast agent and its high costs make MRI less accessible as a general screening tool [2].

Diffuse optical tomography (DOT) is a noninvasive functional imaging modality that utilizes near-infrared (NIR) light to penetrate deeply into tissue and map its optical properties. Diffuse light in the NIR wavelength range can penetrate several centimeters in soft tissue, such as the breast and brain. DOT image reconstruction is performed using measurements obtained from reflected or transmitted light at the tissue surface [4–7].

DOT is a promising modality for detecting and monitoring functional changes related to tumor angiogenesis. Using multi-wavelength DOT, it is possible to quantify tissue characteristics, such as the oxygenated, deoxygenated, and total hemoglobin concentrations (HbO2, HbR, and HbT), as well as blood oxygen saturation (sO2), and lipid and water concentrations [8,9]. Clinical studies have revealed higher HbT in malignant lesions and lower HbT in both healthy and benign tissues [8,10,11,12,13].

However, the low spatial resolution and lesion location uncertainty caused by intense light scattering in tissue are challenges to DOT’s wide clinical use. In addition, DOT reconstruction is an ill-posed and under-determined problem, which requires a regularization in inversion to improve the convergence [14]. The algebraic reconstruction technique (ART) [15], nonlinear iterative gradient-based optimization methods, and many other reconstruction methods, have been put forward for accurate reconstruction [16]. Multi-modality approaches have also been widely explored [17–19]. A complementary modality can provide prior information about the target, including its depth and size, thus the reconstruction result can be improved. For example, our group has introduced an ultrasound (US)-guided DOT approach that uses co-registered US to identify the breast lesion location and size and uses DOT to reconstruct the lesion optical absorption and then compute hemoglobin concentration [17].

However, model errors due to such factors as imperfect boundary conditions, the chest wall underneath the breast tissue (as shown in Fig. 1), and inaccurate background tissue estimation can result in inaccurate DOT reconstruction [20]. To overcome this challenge, a machine learning (ML) approach, capable of learning a more accurate model for image reconstruction and widely applied for cancer diagnosis [21,22], is a promising solution [23,24]. ML learns directly from the data and formulates the relationship between optical properties and measurements, and thus a more accurate model can be learned for DOT reconstruction.

Fig. 1.

Illustration of DOT breast imaging.

Feng et al. used a back-propagation neural network for DOT reconstruction, which reconstructed more details than the Gradient-based reconstruction with Tikhonov regularization [25]. Yedder et al. used a deep learning method to model the inverse problem and obtained a higher quality of reconstructed images than with a conventional analytic approach [26]. Yoo et al. constructed a CNN model based on Lippman-Schwinger integral equation to achieve a high-quality DOT reconstruction [27]. Mozumder et al. utilized a Bayesian approach and prior information to perform a model-based iterative learning approach for DOT reconstruction [28]. Fang and colleagues solved the ill-posed DOT inverse problem by a CNN model with a prior knowledge [29].

An autoencoder was first introduced as an artificial neural network for learning target data distributions via unsupervised learning [30], and several groups have recently discovered it to be well suited for inverse problems [31–33]. However, one challenge in our study is the significant difference between simulation data and clinical data. First, experimental measurement errors are difficult to reproduce in simulation. Second, there are many parameters to simulate, such as the target’s optical properties and depth, the chest wall’s orientation and its optical properties, and the optical properties of the normal reference breast tissue. Motivated by the idea of physics-informed learning [34], here we propose a ML model with physical constraints (ML-PC) to perform DOT reconstruction which learns an end-to-end mapping between the measurements and target absorption distribution, with Born weights as a constraint. Compared with CNN models developed by Yoo and Fang and the iterative model constructed by Mozumder, we focused on parameters of the imaged medium, such as chest wall depth, chest wall angle, target-reference mismatches, etc. We also added physical constraints to individualize the network in the testing stage. To the best of our knowledge, this is the first ML model to be applied to patient data for DOT reconstruction, and it has provided improved contrast between malignant and benign breast lesions as well as improved lesion depth distributions.

2. Data

Three datasets were utilized to train, validate (fine-tune), and test the ML-PC model: simulation data, phantom data, and clinical data. Simulation data were generated using the finite element method, and they formed a large database for establishing the model in the training stage. Phantom data using tissue-mimicking materials were acquired using a DOT system identical to that used for obtaining the clinical data. The phantom data of homogeneous targets made of same absorption contrast were used for training, and phantom data of inhomogeneous targets made of different absorption contrast were used for validating the ML-PC model. Clinical data from patients with breast lesions were used for testing ML-PC model. The reference measurements were generated without any target in simulation data and collected from a homogenous phantom in phantom studies. In patient data, the reference measurements were obtained from the contralateral normal breast of each individual patient.

2.1. Simulation data

COMSOL software (COMSOL Multiphysics Company Ver. 5.5) was used to simulate the forward process using different background tissue and target configurations. To simulate the breast curvature, a hemisphere with a 20-centimeter diameter was used, and the top plane was kept the same size and configuration as the DOT probe with 9 sources and 14 detectors located at the same positions. As shown in Fig. 2, another layer with different optical properties was added to simulate the chest wall. The diffusion equation was reformulated into the Helmholtz wave equation in COMSOL, and an optical absorption boundary condition was used for all outer boundaries [35]. The optical sources were modeled as isotropic point sources placed at one transport mean free path underneath the probe, and the detectors were modeled as point detectors at the probe-tissue interface. A tetrahedral mesh with a “finer” element size (COMSOL’s terminology) was automatically created and adapted for the model's physical settings.

Fig. 2.

Cross-section of the FEM simulation geometry.

We generated forward measurements with different target radii, target center depths, target absorption coefficients $({\mu }_a)$, background absorption coefficients $({\mu }_{a0})$, reduced scattering coefficients $({\mu }_{s0}^{\mathrm{\prime }})$, chest wall depths, chest wall tilt angles, and chest wall optical properties. During the forward model training of ML-PC, we also added 5% Gaussian noise to the simulation data, which consisted of 104,247 cases (Table 1). Also, simulations of non-homogeneous targets were included: we used top-half and bottom-half spherical targets with different ${\mu }_a$ distributions and left-half and right-half target spherical targets with different ${\mu }_a$ distributions.

Table 1. Simulation parameters.

PARAMETER, UNITS	RANGE (START:STEP:STOP)	NO
${\mathit{\mu }}_{\mathit{a}},\mathit{c}{\mathit{m}}^{-1}$	0.06 : 0.02 : 0.3	13
RADIUS, $\mathit{c}\mathit{m}$	0.5 : 0.25 : 3	11
DEPTH*, $\mathit{c}\mathit{m}$	0.5 : 0.5 : 1.5	3
BACKGROUND ${\mathit{\mu }}_{\mathit{a}},\mathit{c}{\mathit{m}}^{-1}$	0.02 : 0.02 : 0.06	3
CHEST DEPTH, $\mathit{c}\mathit{m}$	0.5 : 0.5 : 1.5	3
CHEST ANGLE, °	-15 : 15 : 15	3
CHEST ${\mathit{\mu }}_{\mathit{a}},\mathit{c}{\mathit{m}}^{-1}$	0.1: 0.05 : 0.2	3
${\mathit{\mu }}_{\mathit{s}}^{\mathit{\prime }}\mathit{c}{\mathit{m}}^{-1}$	4 : 2 : 8	3

2.2. DOT system, phantom and clinical data

Phantom and patient data was acquired from a compact ultrasound guided DOT frequency-domain system [36]. Our combined ultrasound and DOT hand-held probe is shown in Fig. 3. Four laser diodes, with wavelengths 730, 785, 808, and 830 nm, were sequentially switched by two optical switches to 9 source positions on the probe. The system used heterodyne detection: the laser diodes were modulated at 140.02 MHz, and the detected signals were mixed with the 140 MHz reference signal to generate 20 kHz signal. The reflected light from each source was received simultaneously by 14 light guides coupled to 14 photomultiplier detectors. The entire data acquisition (DAQ) was about 3-4 seconds. The distances between the sources and detectors ranged from 3.2 to 8.5 cm.

Fig. 3.

Ultrasound-guided DOT probe with 9 source fibers and 14 light guides.

The breast phantoms mimicked tumors in a real breast. Spherical phantom tumor targets were placed in an intralipid solution with an absorption coefficient of 0.02 cm⁻¹ and a reduced scattering coefficient of 7 cm⁻¹, values that mimicked background tissue optical properties and were also within the range of our simulation. Two sets of homogeneous tumor phantom were used, with absorption coefficients of 0.11 cm⁻¹ and 0.23 cm⁻¹. The diameters of these phantoms were 1 cm, 2 cm, and 3 cm. The phantoms were placed at 1 cm, 1.5 cm, 2 cm, 2.5 cm, and 3 cm depths, measured from the solution surface to the center of the target. There were 30 different phantom configurations in all. Each phantom was measured 3 times, so there were 90 phantom datasets in total, which were used for training.

Non-homogeneous targets consisted of two hemispheres with different ${\mu }_a$, in two configurations (the high ${\mu }_a$ hemisphere on either the top or bottom) and four phantom diameters (2.0, 2.4, 3, and 3.6 cm). Each phantom was measured 3 times, so there were 24 non-homogeneous phantoms, which were all used to test the model.

Patient data were from an on-going clinical study. The study was approved by the local Institutional Review Board (IRB) and was HIPPA compliant. All patients signed an informed consent form before participating in the study. Five benign and five malignant breast lesions were used in this manuscript in the testing stage to demonstrate the feasibility of the ML-PC approach.

3. Born-CGD method

Born-conjugate gradient descent (CGD)-based reconstruction algorithm uses Born-approximation and CGD optimization algorithms for US-guided DOT [37]. Light transport in biological tissue can be approximated by the diffusion Eq. (1) when a medium has a high-albedo $({\mu }_a\ll {\mu }_s)$ and that light scattering in the tissue is nearly isotropic:

$$\frac{\mathrm{\partial }\mathrm{\Phi }(\overrightarrow{r},\omega )}{c\mathrm{\partial }\omega }+{\text{μ}}_{\text{a}}\mathrm{\Phi }(\overrightarrow{r},\omega )-\mathrm{\nabla }\cdot [D\mathrm{\nabla }\mathrm{\Phi }(\overrightarrow{r},\omega )]=S(\overrightarrow{r},\omega ),$$ (1)

where ${\text{μ}}_{\text{a}}$ denotes the absorption coefficient, D denotes the diffusion coefficient, and S denotes the source power density. For monochromatic light, the fluence rate $\mathrm{\Phi }$ can be expressed as a photon density wave $U,U=\frac{\varphi }{ch\upsilon }$, and then (1) can be rewritten as:

$$\frac{\mathrm{\partial }U(\overrightarrow{r},\omega )}{\mathrm{\partial }\omega }+c{\mu }_{a}U(\overrightarrow{r},\omega )-c\mathrm{\nabla }\cdot [D\mathrm{\nabla }U(\overrightarrow{r},\omega )]=q(\overrightarrow{r},\omega ).$$ (2)

Here, c denotes the speed of light and $h\upsilon$ is the photon energy. In frequency domain DOT, the photon density wave $U_{AC}$ consists of a homogeneous part,, and a differential part, $U_{SC}$. The absorption coefficient, ${\mu }_a$, is expressed by its homo$\mathrm{\Delta }{\mu }_a{U}_0$ geneous part, ${\mu }_{a0},$ and a differential part, . If $\mathrm{\Delta }{\mu }_a$ is much less than ${\mu }_{a0}$, the Born approximation, U_S << U₀, is valid. Note that the object to be imaged is assumed to have absorption contrast only because it is much higher than scattering contrast for malignant lesions.

The Born solution of a semi-infinite medium is given as (3):

$$\begin{array}{ll}{U}_{SC}(r_s,r_d)& =\int U_0(r_s,r_v)\cdot O(r_v)\cdot G(r_v,r_d)dr\\ & =W_{Born}\times \mathrm{\Delta }{\mu }_a\end{array}$$ (3)

where $\text{O}(r)=-\frac{\mathrm{\Delta }{\mu }_a(r)}{D}$. Here, G reμ_a0 presents the Green’s function of the semi-infinite medium. Additionally, $r_s$ denotes the sources’ positions, $r_v$ represents the voxels’ positions, $r_d$ denotes the detectors’ positions. $W_{Born}$ is the Born weight matrix and $\mathrm{\Delta }{\mu }_a$ is the difference vector between the true $\overline{{\mu }_a}(r)$ distribution and the background.

For the Born weight matrix calculation, the absorption and scattering coefficients are obtained by fitting reflection measurements acquired from homogeneous media in simulation and phantom studies, and contralateral normal breast in patient data.

4. ML-PC model

Here, we introduce our ML-PC model similar to an auto-encoder structure, which uses two neural networks to solve the forward and inverse problems [38].

The forward model of ML-PC learns the photon transport process (Fig. 4(A)). In a homogeneous medium, the diffusion equation is a well-established model. But the accuracy of the diffusion equation can be compromised in clinical studies, where biological tissue may not satisfy the diffusion approximation due to complex boundary conditions, chest wall underneath the breast tissue, poor probe-tissue contact, etc. Thus, the data-driven neural network is a better choice.

Fig. 4.

Training process of ML-PC model using simulation and part of phantom data. A and B illustrate the training process of forward ML-PC model and inverse ML-PC model, respectively.

The inverse model of ML-PC (Fig. 4(B)) tries to reconstruct the optical properties from measurements. Because the inverse problem is ill-posed and under-determined, it is difficult for the neural network to achieve an accurate reconstruction. The inverse network needs information from the forward model of ML-PC to achieve more accurate reconstruction and this is accomplished during training process illustrated in Fig. 5 with more details given in Section IV and V. Additionally, the co-registered US image can provide anatomical information to improve lesion localization for the inverse model [23].

Fig. 5.

Validation process of ML-PC model.

In the inverse model, the normalized measurement, referred as perturbation and noted as P(measured), was used as the input data. P(measured) is the percentage difference between the target measurements and reference measurements:

$$\text{P}(\text{measured})=\frac{\text{M}(target)-\text{M}(ref)}{\text{M}(ref)}$$ (4)

Both forward and inverse models use fully connected layers. Based on the auto-encoder structure, the forward model has a four-layer structure: an input layer (1792 voxels, 0.25 cm x 0.25 cm x 0.5 cm), two hidden layers (256 neurons), and an output layer (252 neurons). The inverse model also has four layers: an input layer (252 neurons), two hidden layers (256 neurons), and an output layer (1792 voxels). A ReLU activation layer and an Adam optimizer with a learning rate of 1e^-4 are used. Because we use perturbation as input to the inverse model (Eq. (4)), there is no normalization to the input data.

4.1. Training

In training stage (as shown in Fig. 4), we used simulation data and homogeneous phantom data (as shown in Table 2) for which we had ground truth (true ${\mu }_a$ map). We trained the model with mixed simulation and phantom data. With ground truth, we could train both forward and inverse models in a supervised way. For the forward model (Fig. 4(A)), the input was true ${\mu }_a$ map, the output was the predicted perturbation, P(predicted), and the loss function was the mean square error (MSE) of the COMSOL computed forward perturbation measurement, P(measured), and ML-PC predicted perturbation, with L2 regularization for neurons’ weights w.

$$Loss(\text{forward}\text{model})={||\text{P}(\text{predicted})-\text{P}(\text{measured})||}^2+{||w||}^2$$ (5)

Table 2. Data for ML-PC model.

Training (mixed)	Validation (mixed)	Testing
Simulation data	Simulation data	Phantom data (Hemisphere)
Phantom data (Homogeneous)	Phantom data (Non-homogeneous)	Clinical data

For the inverse model (Fig. 4(B)), the input was the P (measured), and the output was the reconstructed ${\tilde{\mu }}_a(r)$ map, and the loss function was the MSE of the true ${\mu }_a(r)$ and the reconstructed ${\tilde{\mu }}_a(r)$, plus L2 regularization for neurons’ weights w.

$$Loss(\text{inverse model})={||{\mu }_a(\text{reconstructed})-{\mu }_a(\text{truth})||}^2+{||w||}^2$$ (6)

The model was implemented in TensorFlow and trained for a total of 200 epochs on an Nvidia 1080 Ti GPU using ADAM optimizer with learning rate of 0.001, beta1 of 0.9, beta2 of 0.999 and epsilon of 1e-08. The batch size 256 was used and the training process was monitored for early stopping with auto mode in TensorFlow.

4.2. Testing and physical constraints for ML-PC

As discussed in Section II, DOT reconstruction is an ill-posed and under-determined problem. If we used the neural network only without any additional information, the reconstruction may be limited by the simulation data. Thus, we added two types of physical constraints to validate ML-PC model to achieve an accurate reconstruction (as shown in Fig. 5). And this stage, we combined the forward and inverse model to build our ML-PC model. Then we used weights from the training stage as an initial estimate, fine-tuned the network with physical constraints.

One physical constraint was the commonly used Born object function (Born constraint), which added physical model information. To test whether the reconstruction approximately follows the Born solution, this constraint used the Born weight multiplied by $\mathrm{\Delta }{\mu }_a(r)$ as a computed perturbation. Then we compared the computed perturbation with the training stage predicted perturbation.

$$\begin{array}{ll}\hfill \mathrm{\Delta }{\mu }_a& =\overline{{\mu }_a}(r)-{\text{μ}}_{\text{a}0}\\ \text{Born}\mathrm{\_}\text{loss}\hfill & =||P(\text{predicted})-W_{Born}\ast \mathrm{\Delta }{\mu }_{\mathit{a}}||.\end{array}$$ (7)

As a second constraint, anatomical information was added to the reconstruction to reconstruct the lesion at the similar location as identified by co-registered US [39]. First, we assumed that the target had an approximately Gaussian distribution which created ball-like shape, and that the target’s ${\mu }_a$ were higher than that of the surroundings. This is because the ${\mu }_a$ of a malignant lesion is typically higher than that of surrounding background. We calculated ${\mu }_{a0}$ and ${\mu }_{s0}^{\mathrm{\prime }}$ based on linear fitting of reference measurements [40].

The anatomical distribution function is shown in Eq. (8), where $\text{r}$ is the target radius measured from the center of the co-registered US image, and ${\text{d}}_{\text{vt}}$ is the distance between each voxel and the target center:

$$\begin{array}{c}\text{anatomical}\mathrm{\_}\text{distribution}=-\frac{1}{\sqrt{\pi }\text{r}}\exp \left(-\frac{{\text{d}}_{\text{vt}}^2}{2r^2}\right).\end{array}$$ (8)

Because the scales of the anatomical distribution and the reconstructed ${\tilde{\mu }}_a(r)$ map were different, we used the total variation (TV) loss of the anatomical constraint and${\tilde{\mu }}_a(r)$ map, as shown in Eq. (9), where v is the voxel value. TV calculates the difference between a voxel and the surrounding voxels, and we used it to calculate the voxel gradients difference of the anatomical distribution and the reconstructed ${\tilde{\mu }}_a(r)$ map. So, it can measure how close the anatomical distribution was to the reconstructed ${\tilde{\mu }}_a(r)$.

$$\begin{array}{c}\text{TV}(v)=\sum _{i,j}\left(\sqrt{{|v_{i+1,j}-v_{i,j}|}^2}+\sqrt{{|v_{i,j+1}-v_{i,j}|}^2}\right)\end{array}$$ (9)

Moreover, we added L2 regularization for neurons’ weights w and biases b to avoid overfitting. The final loss function for validation stage is formulated as in (10).

$$\begin{array}{l}Loss(\text{finetune})=||P(\text{measured})-P(\text{predicted})||+\\ {\gamma }_1||P(\text{predicted})-W_{Born}\ast \mathrm{\Delta }{\mu }_a||+\\ {\gamma }_2||\text{TV}(\text{anatomical}\mathrm{\_}\text{distribution})-\text{TV}({\tilde{\mu }}_a(r))||+{\gamma }_3({||w||}^2+{||b||}^2).\end{array}$$ (10)

Here, ${\gamma }_1,{\gamma }_2$, and ${\gamma }_3$ are small coefficients, 0.1, 0.001, and 0.001, respectively, determined empirically.

5. Quantitative evaluation

The reconstructed target diameter, reconstruction depth profile, and reconstruction of the maximum absorption coefficient were used to quantitatively compare ML-PC with the benchmark Born-CGD method. To compare target shapes, the reconstructed target diameter was calculated from the full width at half maximum (FWHM) of the reconstructed ${\mu }_a$ distribution, ${\tilde{\mu }}_a(r)$.

5.1. Reconstructed depth profile

To compare depth profiles, we implemented a method used by Zhang et al. [20]. The method used the ratio of the sums of each voxel’s absorption coefficient in the first and n-th depth layers to quantitatively evaluate the depth profiles. The contrast of layer n was calculated using (11):

$$\begin{array}{c}{C}^n=\frac{\sum {\tilde{\mu }}_a{(r)}^n}{\sum {\tilde{\mu }}_a{(r)}^1}.\end{array}$$ (11)

The first reconstructed layer is used as the reference layer, so its contrast is always 1.

5.2. Reconstructed maximum absorption coefficient

The reconstructed maximum absorption coefficient was used to quantify the differences between ML-PC and Born-CGD methods for phantom targets with different absorption coefficients.

6. Results

For diagnosis of breast cancer, we have focused on total hemoglobin concentration (HbT) because it is related to tumor angiogenesis. HbT is calculated using absorption coefficients reconstructed from four optical wavelengths. Therefore, we reconstructed absorption coefficient only. Here, 780 nm is used to demonstrate the concept and other wavelengths can be similarly used in ML-PC.

6.1. Phantom reconstruction results

To evaluate the difference between ML-PC with the conventional ML method and the Born-CGD method, we designed a set of experiments with non-homogeneous phantoms. The conventional ML model was the same as our ML-PC model, but without physical constraints. As shown in Fig. 6, the non-homogeneous phantoms were made by forming two half balls with the same 3.2 cm diameter but different absorption coefficients: the white half-ball has a low contrast (0.11 cm⁻¹), but the grey half-ball has a high contrast (0.23 cm⁻¹). In imaging experiments, one phantom (phantom #1) was positioned with the high contrast half ball on top and the low contrast half ball on bottom, while the second phantom (phantom #2) reversed this arrangement. The phantoms were imbedded 2.5 cm deep in an intralipid solution with an absorption coefficient 0.02 cm⁻¹ and a reduced scattering coefficient 7-8 cm⁻¹.

Fig. 6.

Non-homogeneous phantoms. Left (phantom #1) phantom with a high contrast half ball on top; Right (phantom #2) phantom with a low contrast half ball on top.

The reconstruction results from the three methods, ML without physical constraints, Born-CGD, and ML-PC, are shown in Figs. 7 and 8. The ground truth should have 5-layers (from layer 3 to layer 7). We can see that ML method without physical constraints reconstructed the two non-homogeneous phantoms as homogeneous targets. This result indicates that, without physical constraints, the ML method is likely to memorize patterns from training instead of learning the reconstruction process. Thus, obtaining an accurate testing result from the ML method without physical constraints would require a huge training dataset with all types of non-homogeneous and heterogeneous targets, which is not very practical.

Fig. 7.

Reconstruction results of phantom #1. Left: ML method without physical constraints; Middle: Born-CGD method; Right: ML-PC. Each figure consists of 7 slices, from the surface to a 3.5 cm depth, with 0.5 cm spacing. Each slice is 9 cm by 9 cm in the x and y dimensions.

Fig. 8.

Reconstruction results of phantom #2. Left: ML method without physical constraints; Middle: Born-CGD method; Right: ML-PC.

The Born-CGD method provides better results than the ML method without physical constraints. However, due to the light shadowing effect, the reconstructed absorption map does not show absorption contrast in the deeper layers (slices 5 and 6, Fig. 7) in phantom #1. In reflection geometry measurements, photons are absorbed and scattered more by the upper portion of the target, thus detectors detect more photons propagated from the top portion [41]. However, the ML-PC method can learn from data and can reconstruct absorption maps with improved target depth profiles. The reconstructed absorption distribution clearly represents the depth-wise progression of the slices, from high contrast to low in phantom #1 (Fig. 7) and from low contrast to high in phantom #2 (Fig. 8).

By validating the network with physical constraints, ML-PC combines the strengths of the conventional ML method and the Born-CGD method. For example, ML-PC reduces the need for massive training data by using Born loss. The physical constraints can tune the neural network with new data that was not included in the training. Similarly, by using ground truth information, ML-PC learns a better model than the Born model, showing a strong depth-dependent profile in reconstructed absorption maps.

6.2. Phantom quantitative results

6.2.1. Reconstructed target diameters

In Table 3, the ML reconstructed target shapes are much closer to the ground truth, while the Born-CGD reconstructed results are much larger. Also, when the target is deep or has low contrast, the Born-CGD method reconstructs the target shape with much larger dimensions due to the lower number of diffused photons received.

Table 3. Reconstructed diameters; true diameters = 3 cm.

Phantom Configurations		Born-CGD, cm	ML-PC, cm
${\text{μ}}_{\mathrm{a}}$, cm−1	center depth, cm
0.2	2.0	3.48	2.74
0.2	$2.5$	4.36	3.28
0.2	$3.0$	2.94	2.94
0.2	$3.5$	3.04	2.82
0.1	$2.0$	4.06	3.26
0.1	$2.5$	5.02	3.06
0.1	$3.0$	3.44	2.88
0.1	$3.5$	3.60	3.08

6.2.2. Reconstructed target depth profile

For comparison, we used a high contrast, 3.0 cm diameter target, then reconstructed its absorption in five layers. Layer one is the first layer that contains the target.

As Fig. 9 shows, each box describes the contrast of each layer for five target center depths (2.0–4.0 cm, with a 0.5 cm step) and three repeated experiments. When using the Born-CGD method, there are almost no reconstructed signals for layers 2 through 5. On the other hand, the ML-PC method provides stronger contrast values for layers 2 and 3.

Fig. 9.

A, Born-CGD method reconstructed contrast for each layer; B, ML-PC method reconstructed contrast for each layer.

The layer contrast indicates that the ML-PC method can reconstruct deep layers better than the Born-CGD method. ML-PC, by learning from the ground truth and combining physical constraints, achieves better results than either the Born-CGD method or the ML method with no physical constraints.

6.2.3. Reconstruction of the maximum absorption coefficient

Figure 10 shows the maximum ${\text{μ}}_{\text{a}}$ values of reconstructed high contrast (0.23 cm⁻¹) and low contrast (0.11 cm⁻¹) phantom targets using Born-CGD method and ML-PC.

Fig. 10.

Reconstructed maximum ${\mu }_a$ values from the Born-CGD and ML-PC methods for high contrast (0.23 cm⁻¹) and low contrast phantoms (0.11 cm⁻¹).

To quantify the differences, the mean percentage error is used, which is the absolute difference between the ground truth value and the reconstructed value, divided by the ground truth. For a high contrast phantom, the mean percentage error for Born-CGD is 16.41%; for ML-PC it is 13.4%. For a low contrast phantom, the mean percentage error for Born-CGD is 23.42%; for ML-PC it is 9.06%.

Figure 10 shows that the ML-PC method provides more accurate reconstruction than the Born-CGD method. Especially for some very deep phantom targets, Born-CGD does not reconstruct well, because few photons containing the information of the lesion can arrive at the detectors. However, the ML-PC method learns the reconstruction even for scant photons, so there are still some contrasts that indicates a deep target.

6.3. Clinical examples

To further test for our ML-PC method, we have shown results of five malignant and five benign clinical cases. Quantitative results for the maximum absorption coefficients of the ML-PC and Born-CGD methods and their target depth profiles are also provided.

Figure 11 shows reconstruction results of a malignant case (A-C) and a benign case (D-F). Figures 11(B) and (C) show that the ML-PC method can reconstruct deeper than the Born-CGD method for this malignant lesion. Also, the target shape in Fig. 11(C) is closer to the round shape shown in the US image than in Fig. 11(B).

Fig. 11.

Examples from patient studies. A and D are co-registered ultrasound images. B and E are reconstruction results using the Born-CGD method. C and F are reconstruction results using the ML-PC method. In the top row, A, B, and C are images from a 53-year-old women with a high grade invasive ductal carcinoma. Based on co-registered ultrasound, the center of the mass is located at 1.2 cm depth. In the bottom row, D, E, and F are images from a 55-year-old woman with a benign fibroadenoma. The center of the mass is located at 0.9 cm depth.

In Figs. 11(E) and (F), reconstructions from a benign case show that both the ML-PC method and Born-CGD method have similar target depth profiles and absorption coefficients, but the ML-PC method positions the reconstructed lesion in a better location (in the center of the figure).

We also applied the same anatomical constraint used in ML-PC method to Born-CGD to compare the potential improvement of additional constraint to Born-CGD method. The anatomical constraint improved the malignant case with the reconstructed lesion more localized to the center, however, did not significantly change the diffused target map for the benign lesion.

Figure 12 shows the reconstructed maximum absorption coefficients for all the clinical data, using both the Born-CGD method and ML-PC method. Both methods provide similar reconstructions: for malignant cases, the maximum absorption coefficients are near $0.2c{m}^{-1}$, and for benign cases, the maximum absorption coefficients are near $0.1c{m}^{-1}$. However, the ML-PC method has better contrast between malignant and benign cases, based on the p value. Also, the ratio of the medians of the maximum μ_a values for malignant and benign cases is 1.63 using Born-CGD, and 2.22 using ML-PC. Therefore, the ML-PC method can better separate benign and malignant lesions.

Fig. 12.

Clinically reconstructed maximum μ_a values from the Born-CGD and ML-PC methods for malignant (M) and benign (B) cases.

As in the phantom study, the contrasts of each layer for malignant and benign cases were computed using the Born-CGD and ML-PC methods, with the results shown in Fig. 13. The ML-PC method has a better depth profile than the Born-CGD method for both malignant and benign cases. However, in malignant cases, due to the light shadowing effect, the ML-PC contrast in deeper layers is much better than that using the Born-CGD method.

Fig. 13.

A, Contrasts of three reconstructed layers for five malignant cases using the Born-CGD and ML-PC methods; B, contrasts of three reconstructed layers for five benign cases using Born-CGD and ML-PC.

7. Conclusions

In this paper, we proposed a new DOT reconstruction method using machine learning model with physical constraints (ML-PC). The model was based on an auto-encoder and trained with simulation and phantom experiments to perform reconstruction. A fine-tuning or validation strategy was used to further improve the reconstruction. Physical constraints (the Born constraint and an anatomical constraint) were added to the model.

By tuning the network with physical constraints, ML-PC combines the strengths of the ML method (better depth profiles) and the Born-CGD method (more general model). For example, ML-PC reduces the need for massive training data by using Born loss. The physical constraints can tune the neural network with new data that was not included in the training and validation. ML-PC provides a better model than the Born model using complex training data, showing a better depth profile in reconstructed absorption maps. With the physical constraints, ML-PC method has the strengths of the ML model, such as less depth-dependent reconstruction, more accurately reconstructed target sizes and absorption coefficients, and more robustness than provided by the Born-CGD method.

The Born constraint makes the reconstructions follow the physical model. And the anatomical constraint will force the reconstruction to be similar to the US image. So, these constraints will help our ML-PC model to solve ill-posed and under-determined DOT inverse problem.

Phantom experiments and clinical data demonstrate the strength of the ML-PC model. Quantitative results show that the ML-PC method provides more accurate maximum absorption coefficients, target sizes, and depth profiles. The computation cost of reconstructing the μa map with the ML-PC model is at least an order of magnitude faster than with the Born-CGD method.

Although our simulation and phantom data include non-homogeneous targets, differences between breast lesions and simulation or phantom targets remain. ML-PC can be further improved by including more heterogeneous targets in training, and extending the current ranges of target absorption and scattering coefficients in training data. We also plan to add multiple targets in the training data to further improve the ML-PC model. Additionally, the background tissue and chest wall in the simulation and phantom data are homogeneous, so more heterogeneous background tissue will be used to improve the training of ML-PC model. Furthermore, the ML-PC method used Born constraint with Born weights computed from a semi-infinite analytical solution, when a medium is significantly deviated from the semi-infinite assumption, the ML-PC may be limited. Lastly, if a DOT system with different geometry and source-detector pairs, the ML-PC model will need to be re-trained. However, the ML-PC structure can remain the same. Future work will also demonstrate the robustness of the proposed ML-PC method on a large clinical data set.

Funding

National Cancer Institute10.13039/100000054 (CA228047).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.