Deep learning acceleration of iterative model-based light fluence correction for photoacoustic tomography

Abstract

Photoacoustic tomography (PAT) is a promising imaging technique that can visualize the distribution of chromophores within biological tissue. However, the accuracy of PAT imaging is compromised by light fluence (LF), which hinders the quantification of light absorbers. Currently, model-based iterative methods are used for LF correction, but they require extensive computational resources due to repeated LF estimation based on differential light transport models. To improve LF correction efficiency, we propose to use Fourier neural operator (FNO), a neural network specially designed for estimating partial differential equations, to learn the forward projection of light transport in PAT. Trained using paired finite-element-based LF simulation data, our FNO model replaces the traditional computational heavy LF estimator during iterative correction, such that the correction procedure is considerably accelerated. Simulation and experimental results demonstrate that our method achieves comparable LF correction quality to traditional iterative methods while reducing the correction time by over 30 times.

1. Introduction

Photoacoustic tomography (PAT) is a non-invasive imaging modality that combines the benefits of optical and ultrasound imaging. PAT utilizes the transient thermoelastic expansion of biological tissues, excited by light, to generate acoustic waves that are subsequently detected by an array of sensors [1], [2], [3], [4]. A cross-sectional image representing the initial absorbed light energy is obtained using the detected signal. Using multi-wavelength light excitation, multispectral PAT is able to effectively distinguish and quantitatively analyze different biological tissues given their optical absorption at different excitation wavelengths. Profiting from high imaging speed and centimeter-scale imaging depth, multispectral PAT has been widely applied in various preclinical studies [5], [6] and clinical trials [7], [8].

PAT image represents the initial PA pressure after laser excitation, which is proportional to the absorbed energy per unit volume of tissue. The absorbed energy density can be approximated by the product of the light fluence (LF) arriving at a voxel and the absorption coefficient (${\mu }_a$) of the enclosed absorber [9]. Removing LF, the quantitative ${\mu }_a$ map which represents intrinsic optical features of specific absorbers can be obtained [10]. However, since measuring the actual LF distribution of the imaging object is challenging, light transport models such as radiation transfer equation [11], [12], diffusion equation [13], [14], and numerical models based on Monte Carlo method [15], [16] are employed to simulate the deposited LF.

In an ideal case, if the estimated LF distribution is accurate, the quantitative ${\mu }_a$ map can be calculated easily by dividing initial pressure by LF. In practice, however, there is always error and variation during not only LF simulation but initial pressure reconstruction as well. This leads to inaccuracy in the simulation and correction of LF. To solve this problem, model-based iterative LF correction algorithm is proposed [17], [18], [19], [20], [21]. This method does not require the analytic form of the model, instead, it continually updates the unknown parameters through iterative optimization until the output of the solver matches the measured data [9]. For example, Yuan et al. [17] used an iterative nonlinear algorithm based on the diffusion equation and combined Tikonov regularization with space-based regularization to restore the optical properties of PAT images. Cox et al. [18], [19] accelerated the calculation of the functional gradient vector with an adjoint model to achieve greater computational efficiency, but resulting in more iterations to converge. Liu et al. [20] applied an iterative algorithm based on 3D Monte Carlo simulation of light transport to achieve 3D light correction in PAT, but it was limited to simple numerical simulation and phantom experiments. In order to improve the correction accuracy and simplify the calculation process, Zhang et al. [21] proposed a non-segmentation pixel-wise correction method by using a finite-element-based model to iteratively optimize both the ${\mu }_a$ map and LF distribution of target tissues. This method has shown excellent correction effects in simulation, phantom, and animal experiments.

Although these model-based iterative LF correction methods have been well established, they all face a major problem: heavy computational burden. The reason causing such a problem is that during iteration, a numerical LF solver is repeatedly used to update the LF distribution within the object. Whether the solver is based on finite element method (FEM) or Monte Carlo method, the solution of an accurate LF map requires high computational cost. For the acceleration of LF solvers, graphics processing unit (GPU) is used without reducing the total computation amount [22], [23]. To reduce computation time, researchers have also proposed adding tissue segmentation results as prior knowledge to assist correction [24], [25], [26], yet its estimation accuracy is compromised since a uniform distribution of optical coefficient is assumed within each segmented tissue region.

Except for the above model-based correction methods, deep-learning-based (DL-based) approaches have been introduced to LF correction of PAT imaging. Currently, the application of DL is mainly focused on directly reconstructing absorption coefficient maps from original PAT images [27], [28], [29]. For example, Chen et al. [28] proposed a DL-based quantitative photoacoustic imaging method based on U-Net, which obtained the absorption coefficient map directly from the reconstructed initial pressure maps. The effectiveness of their method is proved by simulation experiments based on Monte Carlo LF simulation. By comparing various network structures, Madasamy et al. [29] found that DL-based method could compensate for the nonlinear light fluence distribution more effectively and more efficiently as well. However, due to the need to learn an unknown mapping, DL-based methods usually require a large amount of data and a large-size network for training and testing. Their performance is still inferior to traditional iterative correction approaches. In addition, deep learning also has been used to learn the light transport models and enhance the synthesis of photoacoustic image [30], [31], which is helpful for generating efficient training data for quantitative imaging.

For conventional LF correction techniques based on the diffusion equation, the computational bottleneck is centered on solving the light-transport partial differential equation (PDE). Recently, efficient solvers for PDE have been explored using DL-based techniques, which have shown great promise in speed and accuracy. These DL-based PDE solvers include finite-dimensional operators [32], [33], [34], neural finite element models [35], [36], [37], and neural operators [38], [39], [40], [41]. Compared to their opponents, neural operators overcome the mesh-dependent nature of finite-dimensional approaches by generating a single set of network parameters used with different discretization. They only need to be trained once, and they do not require knowledge of the underlying PDE and are thus suitable for problems where accurate PDEs are difficult to obtain. The recently reported Fourier neural operator (FNO) shares all of the mentioned characteristics of neural operators. By parameterizing the integral kernel in Fourier space, it is able to further reduce the cost of evaluating the integration operator [41].

Considering its potential advantage in LF estimation, neural operators may be used to replace the LF estimator in model-based iterative correction, such that the heavy computational cost of repeated LF calculation can be reduced. Based on this concept, herein we propose a FNO-based accelerated iterative light fluence correction method to speed up iterative LF correction in PAT imaging. Our method is based on the model-based iterative LF correction algorithm, but we couple a trained FNO network model into the iteration process as a forward LF estimator and update both the ${\mu }_a$ map and LF distribution through alternating optimization. Due to its invariance to discretization and minimal network size, FNO is chosen to learn the light transport model, which enables precise LF estimation result with fewer computing resource. With our FNO-based acceleration technique, the iterative LF correction procedure achieves an over 30-fold increase in correction speed compared to traditional methods, as verified by simulation and small animal imaging experiments. Despite this considerably improvement in processing time, the result of LF correction obtained by the proposed method is still comparable to that obtained using traditional iterative correction methods.

2. Methods

2.1. The optical forward process in PAT imaging

During the optical forward process in PAT imaging, the initial pressure distribution $p\left(r\right)$ at a point $r$ within a given biological tissue can be expressed by the following formula:

$$\begin{array}{c}p\left(r\right)=\Gamma {\mu }_a\left(r\right)\varphi \left({\mu }_a\left(r\right),{\mu }_s\left(r\right),g\left(r\right)\right),\end{array}$$ (1)

where $\Gamma$denotes the Gruneisen coefficient, which represents the PA efficiency, measuring the transformation efficiency from thermal energy to pressure. ${\mu }_a\left(r\right)$ and ${\mu }_s\left(r\right)$denote local absorption and scattering coefficients respectively. $\varphi$ represents the deposited light fluence, and $g\left(r\right)$ represents the anisotropic scattering factor at the point $r$. Let us assume that the PAT image is accurately reconstructed and the effect of structural distortion is ignored [42], [43]. Also, in biological soft tissues, the Gruneisen coefficient has a small change, so it is assumed to be constant [25]. In this case, the photoacoustic pressure $p^{\prime }$ in the reconstructed image can be expressed as the product of the absorption coefficient ${\mu }_a$ and the LF distribution $\varphi$:

$$\begin{array}{c}{p}^{\prime }\left(r\right)=\varphi \left({\mu }_a\left(r\right),{\mu }_s^{\prime }\left(r\right)\right)\cdot {\mu }_a\left(r\right),\end{array}$$ (2)

where ${\mu }_s^{\prime }\left(r\right)$ is the reduced scattering coefficient, calculated by ${\mu }_s^{\prime }\left(r\right)={\mu }_s\left(r\right)\left(1-g\left(r\right)\right)$.

2.2. Model-based iterative light fluence correction

In this work, we choose diffusion equation (DE) as our light transport model. DE is the first-order spherical harmonic expansion approximation of the radiative transfer equation. It is given in the frequency domain as:

$$\begin{array}{c}-\nabla \bullet D\left(r\right)\nabla \varphi \left(r,\omega \right)+\left({\mu }_a\left(r\right)+\frac{i\omega }{c}\right)\varphi \left(r,\omega \right)=q(r),\end{array}$$ (3)

where $D=c/(3({\mu }_a+{\mu }_s\prime ))$ is the optical diffusion coefficient, $c$ represents the speed of light. $q$ represents the light source, which can be viewed as constant in both time and space in PAT. It can be seen that the LF distribution $\varphi$ is co-determined by ${\mu }_a$and ${\mu }_s^{\prime }$. Therefore, the solution of (3) can be obtained by finite-element methods given the distribution of ${\mu }_a$and ${\mu }_s^{\prime }$.

In biological tissue, since the variation of tissue scattering coefficient is small, ${\mu }_s^{\prime }$ is often assumed known [10], [25], [44], [45]. Therefore, the forward model (2) can be simplified to:

$$\begin{array}{c}{p}^{\prime }(r)=\varphi \left({\mu }_a(r)\right)\cdot {\mu }_a(r),\end{array}$$ (4)

where $p^{\prime }(r)$ and ${\mu }_a(r)$ are in vector representation, $\varphi$ is LF in sparse matrix representation. Accordingly, the solution ${\mu }_a^s$ of ${\mu }_a$ can be formulated as a least square optimization problem, where $p^{\prime }$ represents a fixed photoacoustic pressure:

$$\begin{array}{c}{\mu }_a^s=\underset{{\mu }_a}{\mathit{argmin}}{∥p^{\prime }-\varphi ({\mu }_a)\cdot {\mu }_a∥}^2.\end{array}$$ (5)

This problem can be solved by the alternating iterative optimization method, as shown in Fig. 1(a). To begin with, the ${\mu }_a$ map is initialized as 0, based on which an initial LF map is obtained accordingly by using a FEM-based DE solver. In each iteration, a new LF matrix ${\varphi }^k$ is first solved by the LF solver given the last updated absorption map ${\mu }_a^{k-1}$. With ${\varphi }^k$ fixed, the optimization problem is converted into a linear model and a new ${\mu }_a^k$ map is obtained by using the gradient descent algorithm. Then the ${\mu }_a^k$ map is used as the input for the next iteration. The iterative procedure continues until the residual error is smaller than a predefined value or the number of iterations exceeds a preset value, and the final ${\mu }_a$ map represents the corrected PAT image.

Fig. 1.

Flow chart of iterative light fluence correction in PAT. (a) Traditional iterative LF correction method. (b) Proposed accelerated LF correction method using FNO.

The above iterative LF correction method combines light transport model with iterative optimization to reconstruct the optical coefficient map of the target. By continuously updating the ${\mu }_a$ map and LF map, the difference between the measurement data and model-generated data is minimized, and a satisfactory ${\mu }_a$ image is obtained.

However, the above iterative correction requires repeated estimation of the LF distribution ${\varphi }^k$, i.e. repeated utilization of the numerical solver for light transport modeling. Since traditional LF estimators are based on either the Monte Carlo method or FEM methods, the computational cost for these methods is relatively high, leading to a long correction time.

2.3. Accelerated iterative light fluence correction by using FNO network

2.3.1. Acceleration algorithm overview

To solve the abovementioned computational bottleneck, as shown in Fig. 1(b), we propose to use a deep-learning-based network model to learn the forward simulation capability of light transport model, and substitute the conventional numerical solvers with the trained model. Because the runtime for a well-trained network model is much faster than a numerical solver, the LF estimation time can be drastically reduced and LF correction can be further speeded up.

Fig. 2 provides a flowchart illustrating the procedure of the proposed FNO-based accelerated LF correction method (FNO-AIC). We adopt the method of alternating iteration to update LF and ${\mu }_a$ [21]. Before the iterative correction begins, some important parameters need to be set, including $\mathit{Iter}1$, $\mathit{Iter}2$, $\epsilon 1$ and $\epsilon 2$. $\mathit{Iter}1$ and $\mathit{Iter}2$ are the iterative parameters which control the maximum number of iterations for LF map and ${\mu }_a$ map, respectively. $\epsilon 1$ and $\epsilon 2$ are the preset minimum residual errors that determine whether the iterations of LF maps and ${\mu }_a$ map continue, respectively. To start the iteration, an initial ${\mu }_a$ map is input into a trained FNO-based LF estimator to generate the corresponding LF map. Then, the conjugate gradient algorithm is used to iteratively update ${\mu }_a$, which is subsequently input into the FNO estimator to produce a new LF map. The above process is repeated until the loss value $\mathit{Err}1$, which represents the mean absolute error between the generated data and the detected data, is less than $\epsilon 1$ or when the number of LF iteration $I$ exceeds $\mathit{Iter}1$. At the end of the iteration process, the resulting ${\mu }_a$ map represents the corrected PAT image.

Fig. 2.

Flow chart of accelerated iterative LF correction by FNO. $\mathit{Iter}1$, $\mathit{Iter}2$ are the max number of iterations to update LF and ${\mu }_a$, respectively. ε1 and ε2 are the preset minimum residual errors that control whether LF and ${\mu }_a$ iterations continue, respectively. Trained FNO model: a trained Fourier neural operator for LF estimation. ${\varphi }^I$: the LF generated in the I-th LF iteration. ${\mu }_a^I$: the absorption coefficient in the I-th LF iteration. ${\mu }_a^{I,J}$: the absorption coefficient generated in the J-th ${\mu }_a$ iteration of the I-th LF iteration.

2.3.2. Fourier neural operator for LF estimation

At the core of our method is the Fourier neural operator, which serves as the efficient LF estimator. FNO provides a promising approach for estimating complicated PDE that is difficult to simulate through physical process. The architecture of the FNO network used in this study is illustrated in Fig. 3. The core component of FNO is the Fourier layer, which learns hidden information about the physical process behind the data. The input is elevated to a higher dimension determined by the hyperparameter called channel by a fully connected layer. Four Fourier layers consecutively update the underlying representation of n-channel, and two convolutional layer projects the output back to the target dimension. The Fourier layer consists of two branches: the first one involves the Fourier transform, which translates feature maps into Fourier space to learn global low-frequency information, and the other branch utilizes convolutional layers to learn high-frequency features in the latent space. The reason for learning global low frequency information in Fourier space consists of two parts. Firstly, every value in the Fourier domain is related to all values in spatial domain for the same object, which means that FNO is better for learning global information in Fourier space. Secondly, low-frequency information in Fourier space is more concentrated than that in spatial domain, thus encouraging FNO to learn helpful low-frequency information more easily. The first branch transforms input to the Fourier space by a two-dimensional Fourier transform and truncated by another hyperparameter, mode, to filter out higher-frequency components and perform linear transformations on low-frequency components. After that, two convolution layers and an activation layer are added to the first branch of the Fourier layer to further enhance low-frequency information learning. The combined output from the two branches transmits to the next layer through an activation layer. These iterative updates can be expressed as:

$$\begin{array}{c}{v}_{t+1}\left(x\right)\mathrm{≔}\sigma (W_{t0}{v}_t\left(x\right)+W_{t2}(\sigma (W_{t1}{\mathcal{F}}^{-1}(\mathcal{F}\left({\kappa }_{\varphi }\right)\bullet \mathcal{F}\left(v_t\right))))(x)),\end{array}$$ (6)

Fig. 3.

(a) The architecture of FNO for LF estimation in PAT. The input $a(x)$ is lifted to a higher dimension using a fully connected layer and then transported into four Fourier layers. Finally, the output $v_4(x)$ is projected back to the target dimension by two 1×1 convolutional layers. (b) The detailed architecture of the Fourier layer. The high-dimension input $v_0(x)$ is transported through two different branches. The upper branch uses the Fourier transform to project the input into the Fourier domain, the linear transform R is on the low mode and filters out the high mode. Then the result projects back into the target space by inverse Fourier transform. Two 1×1 convolutional layers, $W_{t1}$ and $W_{t2}$, further enhance low-frequency information learning, and a 1×1 convolutional layer, $W_{t0}$, is applied in the bottom branch to perform a linear transformation of the input.

where $\sigma$ is a non-linear activation function whose action is defined component-wise, $W_{t0}$, $W_{t1}$, $W_{t2}$, denotes linear transform, ${\kappa }_{\varphi }$ plays the role of a kernel function which is learned from data, parameterized in Fourier space by Fourier transform $\mathcal{F}$. The features learned in the spectral space are transformed back to the spatial domain by inverse Fourier transform ${\mathcal{F}}^{-1}$. As the absorption coefficient varies greatly between different tissues, the ${\mu }_a$ map contains more high-frequency information. For that reason, we choose higher channels to excavate the features of ${\mu }_a$ maps. On the contrary, due to the slow change of diffused photon fluence in biological tissue, the LF distribution contains more low-frequency information, which informs us to choose fewer modes to reduce interference. In this paper, FNO (a, b) represents the FNO with a modes and b channels.

3. Experimental setup

3.1. PAT imaging system

The imaging equipment employed is a commercial small animal multispectral photoacoustic tomography system (MSOT inVision128, iThera Medical, Germany). Five pairs of laser emitters are evenly distributed at 270 degrees to provide 360-degree uniform illumination on the sample surface, forming a width of approximately 8 mm ring illumination, as present in Fig. 4(a). The system has a tunable (660–960 nm) laser with a pulse width of around 5 ns and a repetition frequency of 10 Hz. The ultrasound generated by the excited sample is coupled through water and transmitted to a ring-shaped array transducer consisting of 128 elements covering 270 degrees with a radius of 40.5 mm. During the imaging process, the animal is placed in a specialized holder that facilitates alignment with the central axis of the ring-shaped transducer. The raw data is reconstructed into a 300×300 two-dimensional image using a model-based iterative image reconstruction algorithm [43].

Fig. 4.

(a) Schematic of the PAT system used in this study. (b) Left: Diagram of the simulation setup. Red dots represent the positions of the light sources. Right: absorption coefficient diagram of simulated mouse organ.

3.2. Simulation experiment

We generate simulation datasets using simulated models of healthy mouse organs [21]. To better emulate real-world scenarios, we set the simulation conditions based on the illumination setup of the MSOT imaging system in Toast++ [46], an open-source finite-element-based LF simulation software. The reason of using FEM method rather than Monte Carlo simulation is on the simulation time consumed. Monte Carlo simulation usually needs much longer running time than the FEM. As shown in Fig. 4(b), we create a circular computational mesh with a radius of 40 mm, consisting of 30 sectors and 100 rings, which resulted in 157291 nodes and 311640 elements. Five light sources were evenly distributed along absorption coefficient and scattering coefficient of the background medium, which is simulated to be pure water, are both set to 0.0001 ${\mathit{mm}}^{-1}$. We obtain an uncorrected PAT image by multiplying the simulated LF map with the ideal ${\mu }_a$ map, and further add noise with mean 0 and variance 2×10⁻⁵ to obtain the final uncorrected PAT image. Following this procedure, we generate five datasets at 5 different illumination wavelengths, namely, 700 nm, 730 nm, 760 nm, 800 nm and 850 nm. Each dataset contains 400 pairs of ${\mu }_a$ maps, LF maps, and uncorrected PAT images, of which 80% were used as training datasets, 10% as validating datasets, and 10% as test datasets. Five FNO models are trained using different datasets to learn the corresponding physical processes for different wavelengths. The ideal ${\mu }_a$ maps are used as input, and the simulated LF maps serve as the label for training. The comparison results of FNO trained by dataset with all five illumination wavelengths and that trained by dataset with a single illumination wavelength are given in Supplementary Material.

To compare the performance of our method, we train two U-Net models with the same parameters and network structure by two different methods. The first model is trained using the same training method employed for FNO and serves as an LF-estimator, which results in a U-Net-based accelerated iterative correction model, denoted as U-Net-AIC. The other model is trained to learn a direct mapping from the initial PAT image to the ${\mu }_a$ map, which represents an end-to-end processing by U-Net and is denoted as U-Net-E2E. The reason for choosing U-Net as a comparison is that it is a common architecture for image-to-image learning tasks, especially in medical imaging, and a fully convolutional neural network compared to the FNO. Moreover, U-Net is already used for light transport in tissue [30], [31]. In addition, we also train FNO as an end-to-end processing method, which is denoted as FNO-E2E.

3.3. Animal experiment

The animal experiments are performed on the MSOT imaging system described previously. We reconstruct the raw data into PAT images using a model-based image reconstruction method [43]. As accurate ${\mu }_a$ and LF maps could not be obtained, we use a traditional iterative LF correction method [21] to obtain these maps simultaneously. The reason for this is that this method can obtain pixel-level reconstruction accuracy without pre-segmentation of the object. These reconstructed ${\mu }_a$ and LF maps are utilized as samples for training the network models and as reference for experimental evaluation, resulting in a total of five datasets at five different illumination wavelengths, each containing 500 PAT images, 500 ${\mu }_a$ maps, and 500 LF maps. As in the simulation experiments, the data sets were divided into the training sets and test sets in a ratio of 4:1. Furthermore, two U-Net-based methods and FNO-E2E are also used as in the simulation experiment to compare the performance of FNO-AIC.

3.4. Quantification metrics

Root mean square error (RMSE), peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) are used to evaluate the quality of the results produced by different methods. They are given by:

$$\begin{array}{c}\mathit{RMSE}=\sqrt{\frac{1}{n}\sum _{i=1}^n{(T-M)}^2},\end{array}$$ (7)

$$\begin{array}{c}\mathit{PSNR}=20{\log }_{10}\left(\frac{\mathit{MAX}}{\mathit{RMSE}}\right),\end{array}$$ (8)

$$\begin{array}{c}\mathit{SSIM}=\frac{\left(2{\mu }_M{\mu }_T+C_1\right)\left(2{\sigma }_{\mathit{MT}}+C_2\right)}{\left({\mu }_M^2+{\mu }_T^2+C_1\right)\left({\sigma }_M^2+{\sigma }_T^2+C_2\right)},\end{array}$$ (9)

where $T$ represents the ground truth, $M$ represents the result of the experiment, $n$ represents number of pixels. $\mathit{MAX}$ is the max value of the ground truth. ${\mu }_M$ and ${\mu }_T$ represent the mean of $M$ and $T$, respectively, ${\sigma }_M^2$ and ${\sigma }_T^2$ are the corresponding variance. ${\sigma }_{\mathit{MT}}$ is the cross-covariance for $M$ and $T$ sequentially. The $C_1$ and $C_2$ are given by:

$$\begin{array}{c}{C}_1={\left(0.01\times \mathit{MAX}\right)}^2,C_2={\left(0.03\times \mathit{MAX}\right)}^2.\end{array}$$ (10)

3.5. Implementation

The algorithm development platform utilized in this study is based on a desktop computer with a AMD Ryzen5–5600 CPU and a NVIDIA RTX 3060 GPU. The parameters of the FNO models of FNO-AIC and FNO-E2E are updated based on the mean square error loss and the sum of absolute error loss in 500 epochs respectively. The parameters of the U-Net models are both updated based on the mean square error loss in 500 epochs. All the network models are trained using the Adam optimizer, and the learning rate is decreased by a factor of 0.1 every 25 epochs, starting from an initial value of 0.001. The architecture of U-Net applied in this work can be found in Supplementary Material. For all experiments, $\epsilon 1$ and $\epsilon 2$ are both set to ${10}^{-12}$.

4. Results

4.1. Simulation results: LF estimation

Firstly, the simulation dataset with the illumination wavelength of 700 nm is used to test the forward simulation ability of our FNO-based LF estimator. Fig. 5 depicts the visual comparison of results from different networks. The absolute error maps between reference and the results of neural-network-based estimators are presented at the second line of Fig. 5. As can be seen, the selected networks are all capable of forward simulation, but result in different accuracy. The LF map generated by U-Net displays distinguishable artifacts, as pointed by the black arrow in the figure. In addition, the absolute error maps highlight that all results from FNO have less error compared to those from U-Net. In order to further analyze the forward simulation results of each network, we choose RMSE as the quantitative index to quantitatively compare the results of each network and listed the results at the bottom of Fig. 5, along with the number of parameters of each network. We found that the RMSE of U-Net reaches 6.44E-3, which is much higher than the RMSE of the FNO models. It is also worth noting that except for FNO (15,64), the other three FNO models with fewer parameters can achieve superior forward results than U-Net. To further assess the forward simulation capability of the proposed method, we select FNO (10,64) as the forward network and test its performance on simulation datasets with five illumination wavelengths. The quantitative experimental results are presented in Table 1. Among all five experiments with different illumination wavelengths, the RMSE of FNO is less than that of the corresponding U-Net model. By calculation, the mean value of FNO at five illumination wavelengths is 4.01E-3, while that of U-Net is 6.36E-3, which is about 1.59 times higher than that of FNO.

Fig. 5.

Visual comparison of the results of FNO and U-Net for learning the forward light transport process. Input is the ideal ${\mu }_a$ map and Reference is the simulated LF map. The bottom row shows the absolute error between the LF prediction result and ground truth. FNO (10,64) represents FNO model with 10 modes and 64 channels.

Table 1.

RMSE (mean with standard deviation) between the simulated LF map and the network-generated LF map simulated by FNO and U-Net models on simulation data.

	700 nm	730 nm	760 nm	800 nm	850 nm
FNO (10,64)	3.83E-3 ± 3.12E-3	3.78E-3 ± 2.87E-3	4.17E-3 ± 2.92E-3	3.91E-3 ± 2.77E-3	4.37E-3 ± 3.17E-3
U-Net	6.44E-3 ± 4.25E-3	5.29E-3 ± 2.80E-3	6.52E-3 ± 3.35E-3	6.29E-3 ± 2.99E-3	7.25E-3 ± 3.74E-3

4.2. Simulation results: LF correction

Next, we compare the performance of five different LF correction methods, namely the traditional iterative correction method (TIC), FNO-based accelerated iterative correction (FNO-AIC), end-to-end processing by FNO (FNO-E2E), U-Net-based accelerated iterative correction (U-Net-AIC) and end-to-end processing by U-Net (U-Net-E2E), in terms of correction time and correction quality. In all simulation experiments, for the two accelerated iterative correction methods, Iter1 is set to 30 and Iter2 is set to 80, which are determined through experiments with the dataset of 700 nm illumination wavelength. Fig. 6 shows the visual comparison of correction results at 850 nm illumination wavelength. U-Net-E2E produces the worst result, which is characterized by blurred outlines of different tissue (black arrow) and non-uniform artifacts (yellow arrow). In contrast, results obtained from other four correction methods are much more superior and differ minimally from the ground truth. Notably, TIC produces the best results with almost negligible errors. This is because the accurate LF maps are generated by the FEM-based LF estimator. The FNO-E2E produces the second-best correction results with very little error. The results of the two neural-network-based iterative correction methods both show slight error, yet FNO-AIC produces more accurate correction results compared to U-Net-AIC. This is in consistent with the previous results of forward simulation capability, confirming that the error in the forward LF generation will be passed to the subsequent correction.

Fig. 6.

LF Correction results of different methods in simulation experiment. Ground Truth: ideal ${\mu }_a$ map. PAT: uncorrected PAT image. FNO-AIC: FNO-based accelerated iterative correction method. U-Net-AIC: U-Net-based accelerated iterative correction method. FNO-E2E: end-to-end processing method by FNO. U-Net-E2E: end-to-end processing method by U-Net. Abs. error: absolute error between the correction result and the Ground Truth.

Quantitative experimental results at all five illumination wavelengths are presented in Table 2. As can be seen from the table, TIC achieves the best correction effect among the five methods, but it also consumes the longest processing time, which is more than 80 s on average. On the contrary, both end-to-end methods consume the least time. FNO-E2E achieves the second-best correction results and the correction results of U-Net-E2E are the worst among all experiment. The time required for the network-based accelerated iterative correction varies from 2.25 to 2.63 s, which is more than 30 times faster than TIC. Among the two accelerated methods, FNO-AIC achieves an average 11.69% improvement in PSNR.

Table 2.

Quantitative results (mean with standard deviation) of different LF correction methods on simulation data of five different illumination wavelengths.

Wavelength (nm)	Method	RMSE	PSNR	1-SSIM	Time (s)
700	TIC	0.00054 ± 0.00125	50.657 ± 10.462	0.00415 ± 0.00268	84.78 ± 0.35
	FNO-AIC	0.00079 ± 0.00136	41.767 ± 9.476	0.00637 ± 0.00546	2.40 ± 0.76
	U-Net-AIC	0.00115 ± 0.00212	37.960 ± 8.110	0.00725 ± 0.00480	2.56 ± 0.78
	FNO-E2E	0.00038 ± 0.00045	44.241 ± 7.619	0.00308 ± 0.00557	0.01 ± 0.00
	U-Net-E2E	0.00371 ± 0.00330	22.018 ± 4.063	0.03963 ± 0.02548	0.01 ± 0.00
730	TIC	0.00026 ± 0.00059	51.968 ± 8.232	0.00473 ± 0.00340	82.09 ± 0.39
	FNO-AIC	0.00050 ± 0.00084	43.298 ± 7.921	0.00595 ± 0.00395	2.25 ± 0.80
	U-Net-AIC	0.00067 ± 0.00080	39.488 ± 6.974	0.00651 ± 0.00313	2.34 ± 0.82
	FNO-E2E	0.00044 ± 0.00047	40.443 ± 6.656	0.22700 ± 0.24441	0.01 ± 0.00
	U-Net-E2E	0.00299 ± 0.00261	22.757 ± 4.297	0.03966 ± 0.02321	0.01 ± 0.00
760	TIC	0.00109 ± 0.00225	50.109 ± 13.548	0.00289 ± 0.00176	84.76 ± 0.31
	FNO-AIC	0.00112 ± 0.00189	41.170 ± 9.406	0.00594 ± 0.00809	2.55 ± 0.68
	U-Net-AIC	0.00178 ± 0.00225	36.042 ± 9.384	0.00995 ± 0.01471	2.70 ± 0.67
	FNO-E2E	0.00076 ± 0.00098	40.555 ± 7.348	0.09603 ± 0.17691	0.01 ± 0.00
	U-Net-E2E	0.00449 ± 0.00385	22.624 ± 4.157	0.03678 ± 0.02759	0.01 ± 0.00
800	TIC	0.00090 ± 0.00179	50.617 ± 13.028	0.00266 ± 0.00154	79.50 ± 5.78
	FNO-AIC	0.00100 ± 0.00180	41.991 ± 8.783	0.00527 ± 0.00694	2.43 ± 0.71
	U-Net-AIC	0.00137 ± 0.00178	37.027 ± 7.309	0.00630 ± 0.00758	2.59 ± 0.70
	FNO-E2E	0.00073 ± 0.00077	39.940 ± 6.807	0.01066 ± 0.03070	0.01 ± 0.00
	U-Net-E2E	0.00466 ± 0.00397	22.259 ± 4.306	0.04213 ± 0.02833	0.01 ± 0.00
850	TIC	0.00163 ± 0.00307	50.463 ± 15.412	0.00249 ± 0.00204	86.23 ± 0.32
	FNO-AIC	0.00212 ± 0.00340	39.562 ± 11.161	0.00857 ± 0.01491	2.49 ± 0.66
	U-Net-AIC	0.00226 ± 0.00305	35.520 ± 7.928	0.00716 ± 0.00953	2.65 ± 0.65
	FNO-E2E	0.00087 ± 0.00105	41.148 ± 7.270	0.01840 ± 0.06209	0.01 ± 0.00
	U-Net-E2E	0.00541 ± 0.00439	22.760 ± 4.305	0.03100 ± 0.01951	0.01 ± 0.00

4.3. Simulation results: mesh size

For the same imaging area, traditional LF estimators need a dense computational mesh to provide more precise LF simulation, which corresponds to more computing resources and time cost. Here, we compare the calculation speed and correction quality of the traditional method and the other four deep learning methods under five different mesh sizes. We chose the simulation dataset at 700 nm illumination wavelength as the experimental data. During LF simulation, we keep the radius of the simulation environment and the number of light sources unchanged, but increase the grid parameters by 1, 1.25. 1.5, 1.75 and 2 times. Accordingly, the image size also increases to ensure that the size of the object remains unchanged. For both the two U-Net models, since the original architecture is not adapted to the changed image size, we adjust the networks accordingly while keeping the training parameters and loss function unchanged. For both the two FNO models, because of its invariance to discretization, its hyperparameters, network architecture, training parameters and loss function are kept unchanged.

The quantitative results of the above experiments are presented in Table 3. TIC yields the best correction results but consumes the longest time. Conversely, U-Net-E2E is drastically faster but produces inferior correction results. In particular, in 1.25x and 1.75x, the U-Net model from U-Net-E2E failed to fit to the training data and therefore does not achieve LF correction. With the increase of mesh size, the correction ability of FNO-E2E remains relatively stable. FNO-AIC demonstrates consistent correction precision for different image resolutions, whereas U-Net-AIC shows obvious fluctuation in the five mesh sizes. Compared to TIC, the FNO-AIC achieves acceleration rates of 35.33, 16.37, 19.79, 19 and 28.32 under the five mesh sizes, respectively. It also outperforms the U-Net-AIC in terms of both speed and correction quality. The reason for the superior performance achieved by our FNO-AIC lies in the fact that any discretized data can be transformed into the Fourier space through the application of the Fourier transform. Moreover, FNO does not require any network structure or training parameter adjustment across the five mesh sizes.

Table 3.

Quantitative results (mean with standard deviation) of different correction methods at different mesh sizes.

RMSE
	1X		1.25X		1.5X		1.75X		2X
TIC	0.00054 ± 0.00125		0.00063 ± 0.00144		0.00066 ± 0.00150		0.00069 ± 0.00155		0.00071 ± 0.00159
FNO-AIC	0.00079 ± 0.00136		0.00071 ± 0.00132		0.00085 ± 0.00119		0.00077 ± 0.00122		0.00083 ± 0.00113
U-Net-AIC	0.00115 ± 0.00212		0.00246 ± 0.00494		0.00091 ± 0.00121		0.00249 ± 0.00333		0.00140 ± 0.00165
FNO-E2E	0.00038 ± 0.00045		0.00052 ± 0.00057		0.00063 ± 0.00069		0.00057 ± 0.00056		0.00056 ± 0.00065
U-Net-E2E	0.00371 ± 0.00330		X		0.00369 ± 0.00345		X		0.00420 ± 0.00385
PSNR
	1X		1.25X		1.5X		1.75X		2X
TIC	50.657 ± 10.462		50.066 ± 11.050		49.867 ± 11.233		49.701 ± 11.388		49.567 ± 11.504
FNO-AIC	41.767 ± 9.476		43.679 ± 9.310		41.340 ± 9.731		43.005 ± 9.951		41.944 ± 10.118
U-Net-AIC	37.960 ± 8.110		34.079 ± 8.750		38.227 ± 7.303		29.149 ± 5.293		34.622 ± 8.349
FNO-E2E	44.241 ± 7.619		41.045 ± 6.000		38.359 ± 5.583		38.417 ± 5.523		39.510 ± 5.562
U-Net-E2E	22.018 ± 4.063		X		22.372 ± 4.620		X		21.203 ± 4.520
1-SSIM
		1X		1.25X		1.5X		1.75X		2X
TIC		0.00415 ± 0.00268		0.00422 ± 0.00264		0.00425 ± 0.00264		0.00429 ± 0.00263		0.00431 ± 0.00263
FNO-AIC		0.00637 ± 0.00546		0.00617 ± 0.00559		0.00590 ± 0.00438		0.00629 ± 0.00521		0.00578 ± 0.00459
U-Net-AIC		0.00725 ± 0.00480		0.01241 ± 0.01459		0.00638 ± 0.00324		0.03783 ± 0.03860		0.01121 ± 0.01019
FNO-E2E		0.00308 ± 0.00557		0.14441 ± 0.12840		0.03029 ± 0.11607		0.01416 ± 0.03284		0.01661 ± 0.04389
U-Net-E2E		0.03963 ± 0.02548		X		0.05522 ± 0.03719		X		0.04730 ± 0.02807
Time (s)
	1X		1.25X		1.5X		1.75X		2X
TIC	84.78 ± 0.35		145.75 ± 2.85		195.32 ± 11.18		290.51 ± 16.35		447.38 ± 1.19
FNO-AIC	2.40 ± 0.76		8.90 ± 2.28		9.87 ± 3.29		15.29 ± 3.57		15.80 ± 5.33
U-Net-AIC	2.56 ± 0.78		9.13 ± 2.32		10.69 ± 3.32		15.28 ± 3.91		18.09 ± 5.76
FNO-E2E	0.01 ± 0.00		0.01 ± 0.00		0.01 ± 0.00		0.01 ± 0.00		0.01 ± 0.00
U-Net-E2E	0.01 ± 0.00		X		0.01 ± 0.00		X		0.01 ± 0.00

4.4. Simulation results: spectral un-mixing

In order to further test the effectiveness and precision of five different LF correction methods, we used the linear spectral un-mixing method [47] to calculate the distribution of oxyhemoglobin (HbO₂), de-oxyhemoglobin (Hb) and oxygen saturation (sO₂) from the obtained experimental results. Fig. 7 shows the visual comparison of the un-mixing results. We can see that the error of sO₂ obtained by the end-to-end methods are greater than that of iterative methods. Among the three iterative methods, TIC achieves the best correction result. FNO-AIC is slightly better than U-Net-AIC. Table 4 lists the quantification metrics of sO₂ from the five comparison methods. As with the results of visual comparison, the traditional iterative method achieved the best quantitative results with a mean PSNR up to 49.193 while FNO-AIC achieved the second-best experimental results with a mean PSNR of 44.177. More visual comparison of the result at different position can be found in Supplementary Material.

Fig. 7.

The results of spectral un-mixing from five different LF correction methods. TIC: traditional iterative correction method. FNO-AIC: FNO-based accelerated iterative correction method. U-Net-AIC: U-Net-based accelerated iterative correction method. FNO-E2E: end-to-end processing method by FNO. U-Net-E2E: end-to-end processing method by U-Net.

Table 4.

Quantitative results (mean with standard deviation) of sO₂ from five different correction methods.

	TIC	FNO-AIC	U-Net-AIC	FNO-E2E	U-Net-E2E
RMSE	0.0052 ± 0.0060	0.0089 ± 0.0124	0.0089 ± 0.0093	0.0104 ± 0.0065	0.0376 ± 0.0127
PSNR	49.193 ± 9.993	44.177 ± 8.074	42.070 ± 6.428	39.378 ± 5.305	27.243 ± 3.489
1-SSIM	0.0047 ± 0.0052	0.0112 ± 0.0165	0.0101 ± 0.0129	0.0405 ± 0.0316	0.0707 ± 0.0187

4.5. Small animal imaging experiment result

We conduct small animal experiments to test the effectiveness of FNO-AIC. Due to the unavailability of real ${\mu }_a$ and LF maps, the correction results from TIC are used as reference. In all small animal experiments, the FNO applied is also with 10 modes and 64 channels. In all small animal experiments, for the two accelerated iterative correction methods, Iter1 is set to 30 and Iter2 is set to 20, which are determined through experiments with the dataset of 700 nm illumination wavelength. Fig. 8 presents a visual comparison of the correction results at the liver position with two illumination wavelengths. The results from U-Net-E2E are the worst among all four methods, which show the higher error. In contrast, the other three correction methods generate much superior correction results. As can be seen from the error map, compared with U-Net-AIC and FNO-E2E, the error of FNO-AIC is smaller at both wavelengths, especially at regions with high absorption coefficient (yellow arrows). This proves that FNO-AIC is more accurate than U-Net-AIC and FNO-E2E at different wavelengths.

Fig. 8.

LF correction results at the liver position with two different illumination wavelengths 760 nm and 850 nm. Reference: correction results of traditional iterative correction method. PAT: uncorrected PAT image. FNO-AIC: FNO-based accelerated iterative correction method. U-Net-AIC: U-Net-based accelerated iterative correction method. FNO-E2E: end-to-end processing method by FNO. U-Net-E2E: end-to-end processing method by U-Net. Abs. error: absolute error between the correction result and Reference.

Furthermore, we present the correction results of two different positions at 800 nm illumination wavelength in Fig. 9. The results of U-Net-E2E are still not ideal, which error is the largest and distributed globally. Consistent with the experimental results at different wavelengths, FNO-AIC is still more accurate than U-Net-AIC and FNO-E2E at different positions. As can be seen from the error map, the error of FNO-AIC appears smaller and the absolute value is lower. In the position of high absorption coefficient (yellow arrows), FNO-AIC is also more accurate than U-Net-AIC.

Fig. 9.

LF correction results at two different positions. Reference: correction results of traditional iterative correction method. PAT: uncorrected PAT image. Abs. error: absolute error between the correction result and Reference.

The evaluation metrics of four different correction methods are quantitatively shown in Table 5. At five illumination wavelengths, FNO-AIC reaches a mean RMSE of 0.00131, a mean PSNR of 55.217 and a mean 1-SSIM of 0.00044. U-Net-AIC reaches a mean RMSE of 0.00154, a mean PSNR of 53.202 and a mean 1-SSIM of 0.00050. FNO-E2E reaches a mean RMSE of 0.00146, a mean PSNR of 49.665 and a mean 1-SSIM of 0.00364. U-Net-E2E reaches a mean RMSE of 0.00796, a mean PSNR of 34.929 and a mean 1-SSIM of 0.03974. The quantization results are consistent with the visual comparison results above which indicates that U-Net-E2E produces the worst correction quality at five illumination wavelengths. Among the other three methods, the results of FNO-AIC are best under the three quantitative indicators at five illumination wavelengths, which proves that it has a correction ability closer to the traditional iterative method.

Table 5.

Quantified correction results (mean with standard deviation) of animal imaging experiments at five illumination wavelengths.

Wavelength(nm)	Method	RMSE	PSNR	1-SSIM
700	FNO-AIC	0.0010 ± 0.0022	58.390 ± 6.273	0.00015 ± 0.00029
	U-Net-AIC	0.0013 ± 0.0027	54.514 ± 6.512	0.00021 ± 0.00037
	FNO-E2E	0.0011 ± 0.0021	48.468 ± 3.724	0.00301 ± 0.00594
	U-Net-E2E	0.0058 ± 0.0087	30.901 ± 4.461	0.09731 ± 0.08115
730	FNO-AIC	0.0017 ± 0.0031	53.743 ± 6.671	0.00060 ± 0.00252
	U-Net-AIC	0.0019 ± 0.0020	51.306 ± 5.735	0.00081 ± 0.00370
	FNO-E2E	0.0017 ± 0.0015	49.364 ± 4.586	0.00324 ± 0.00419
	U-Net-E2E	0.0142 ± 0.0221	34.364 ± 5.575	0.03289 ± 0.03276
760	FNO-AIC	0.0018 ± 0.0032	54.665 ± 6.840	0.00070 ± 0.00274
	U-Net-AIC	0.0021 ± 0.0026	52.807 ± 6.124	0.00066 ± 0.00305
	FNO-E2E	0.0019 ± 0.0020	50.606 ± 4.669	0.00250 ± 0.00387
	U-Net-E2E	0.0083 ± 0.0066	37.002 ± 3.752	0.02230 ± 0.02224
800	FNO-AIC	0.0013 ± 0.0012	52.847 ± 5.790	0.00049 ± 0.00202
	U-Net-AIC	0.0015 ± 0.0016	52.377 ± 5.880	0.00051 ± 0.00192
	FNO-E2E	0.0015 ± 0.0011	49.858 ± 4.453	0.00500 ± 0.00501
	U-Net-E2E	0.0065 ± 0.0042	36.281 ± 3.508	0.02214 ± 0.02005
850	FNO-AIC	0.0007 ± 0.0008	56.439 ± 5.598	0.00024 ± 0.00111
	U-Net-AIC	0.0009 ± 0.0010	55.006 ± 6.533	0.00030 ± 0.00106
	FNO-E2E	0.0011 ± 0.0009	50.027 ± 3.574	0.00445 ± 0.00425
	U-Net-E2E	0.0049 ± 0.0029	36.096 ± 3.205	0.02405 ± 0.02043

Table 6 shows the time consumption of processing one PAT image by four different correction methods. The average processing time of the TIC method is 84.09 s at 5 wavelengths, FNO-AIC is 1.05 s, U-Net-AIC is 1.00 s, FNO-E2E is 0.01 s and U-Net-E2E is 0.01 s. The accelerated iterative method is more than 80 times faster than the traditional method.

Table 6.

The LF correction time (s) in animal imaging experiments with five illumination wavelengths.

	700 nm	730 nm	760 nm	800 nm	850 nm
TIC	77.51 ± 13.79	85.66 ± 1.88	88.78 ± 0.40	83.34 ± 1.68	85.15 ± 2.49
FNO-AIC	1.06 ± 0.23	1.05 ± 0.29	1.04 ± 0.30	1.05 ± 0.29	1.06 ± 0.29
U-Net-AIC	1.02 ± 0.10	1.00 ± 0.30	0.99 ± 0.30	1.00 ± 0.30	1.01 ± 0.04
FNO-E2E	0.01 ± 0.00	0.01 ± 0.00	0.01 ± 0.00	0.01 ± 0.00	0.01 ± 0.00
U-Net-E2E	0.01 ± 0.00	0.01 ± 0.00	0.01 ± 0.00	0.01 ± 0.00	0.01 ± 0.00

The reported values are means with standard deviation

5. Discussion

The above results of both simulation and small animal imaging experiments demonstrate that the proposed method can effectively achieve accelerated LF correction for PAT. The obtained correction results achieve comparable accuracy to traditional iterative correction method, yet the processing speed has been considerably enhanced, with at least a 30-fold improvement. Notably, the proposed method demonstrates its suitability for diverse PAT images of varying sizes, without substantial compromise to processing speed. Also, the method proposed is suitable for PAT images of different illumination wavelengths and achieves consistent LF correction results.

As can be found in the simulation experiment of spectral un-mixing, FNO-AIC achieves the most accurate sO₂ map among the DL-based methods, which indicates the effectiveness of our method on LF correction. In small animal imaging experiments, the results of FNO-AIC are the closest to those generated by traditional iterative method. In the same time, DL-based iterative correction methods are more similar to traditional iterative method than end-to-end methods with the same network model. This might be because the network model in iterative correction learns the mapping between tissue absorption coefficient and LF, which is determined by physical light transport theory.

In our method, FNO plays an important role. Due to its ability to estimate the light transport model and rapid calculation, FNO considerably speeds up the process of iterative LF correction by replacing the role of tradition numerical LF solver. The two hyperparameters of FNO, mode and channel, have important effects on its function. The function of mode mainly filters out the high-frequency information of the image after Fourier transform. The main function of channel is to lift the dimension of the input image. These two hyperparameters cooperate with each other to form a complete FNO. A large mode value may make the network learn some unnecessary high-frequency information, such as noise, while a low mode value will affect the generalization result of the entire network. Similarly, a channel that is too high or too low will result in overfitting or underfitting of the neural network, affecting the accuracy of the generated light fluence map and thus the subsequent correction process.

FNO has the advantage of invariance to discretization. As shown in simulation experiment of mesh size, compared with the traditional U-Net model that needs to adjust network structure or training parameters for different input image sizes which may affect the results, FNO can generate results with the same quality across different image resolutions without any adjustment. Moreover, traditional FEM-based LF solver needs a larger mesh to perform more accurate simulations, which means more iteration and computation. In contrast, due to its discrete invariance, FNO can estimate for the LF map under different levels of discretization without changing parameters. At the same time, its calculation speed and solution accuracy will not decrease.

Despite all these superiorities, the proposed work also has limitations and can be further improved in future works. First of all, there is still a gap between the accuracy of FNO and traditional LF solver. This is because the model has not yet fully learned the ideal characteristic of nonlinear light propagation in tissue. This limitation is expected to be improved by further modifying the network structure using recently proposed advanced techniques [48], [49]. Secondly, when the light source is determined, the LF in biological tissues is determined by the absorption coefficient and the scattering coefficient together. In this paper, the relationship between the absorption coefficient and the LF is studied by default when the scattering coefficient is known. However, in practice, the scattering coefficient is unknown, and thus how to solve the absorption coefficient and scattering coefficient at the same time needs further research. Thirdly, although the image size does not affect the processing time of FNO, it has a great impact on the iterative convergence time, which is also the reason why our method consumes more time when increasing image size. Fourthly, the optical parameters of water vary for different wavelengths [50], [51], yet in our simulation we assumed they are the same. Therefore, the realism of our simulations may be limited. Also, in our work, experiments are only applied to five specific wavelengths in our PAT system with ring-shaped illumination and detection. For other imaging configurations, further experiments are needed in order to prove the applicability of our method. Finally, our FNO is learned based on the finite element results. For the learning of other solving methods, such as the Monte Carlo method, further experiments and research are needed.

6. Conclusion

In this study, we propose an iterative PAT light fluence correction method accelerated by the Fourier neural operator. By substituting the conventional light transport model with a well-trained Fourier neural operator and employing an alternating iterative LF correction algorithm, the proposed method achieves precise reconstruction of the absorption coefficient map with exceptional computational efficiency. We tested the proposed method on simulation and small animal imaging experiments, and the results demonstrate the feasibility and adaptability of the proposed method.

Funding

National Natural Science Foundation of China (62371220); Guangdong Basic and Applied Basic Research Foundation (2021A1515012542, 2022A1515011748); Guangdong Pearl River Talented Young Scholar Program (2017GC010282).

CRediT authorship contribution statement

Zhaoyong Liang: Software, Methodology, Investigation, Writing – original draft, Writing – review & editing. Shuangyang Zhang: Software, Resources. Zhichao Liang: Software, Resources. Zongxin Mo: Data curation. Xiaoming Zhang: Data curation. Yutian Zhong: Data curation. Wufan Chen: Resources, Supervision. Li Qi: Writing – review & editing, Supervision, Funding acquisition, Conceptualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Biographies

Zhaoyong Liang received his bachelor’s degree from Southern Medical University in 2022. He is currently pursuing the master’s degree in the school of Biomedical Engineering, Southern Medical University. His research interests include photoacoustic tomography and medical image processing.

Shuangyang Zhang received his Ph.D. degree in 2022 from Southern Medical University. He is currently a postdoctoral fellow at the Guangdong Provincial Key Laboratory of Medial Image Processing, School of Biomedical Engineering, Southern Medical University in Guangzhou, China. His research interests include photoacoustic imaging and multimodal image registration.

Zhichao Liang received his bachelor's degree from Southern Medical University in 2021. He is currently pursuing the master's degree in the school of Biomedical Engineering, Southern Medical University. His research interests include photoacoustic tomography and image segmentation.

Zongxin Mo received his bachelor's degree from Guangzhou Medical University in 2022. He is currently pursuing the master's degree at the School of Biomedical Engineering, Southern Medical University. His research interests include photoacoustic tomography and medical image processing.

Xiaoming Zhang received his bachelor’s degree from Southern Medical University in 2022. He is currently pursuing the master’s degree in the school of Biomedical Engineering, Southern Medical University. His research interests include image restoration based on photoacoustic Tomography and relevant system based on photoacoustic imaging and MRI.

Yutian Zhong received his bachelor’s degree from Southern Medical University in 2022. He is currently pursuing the master’s degree in the school of Biomedical Engineering, Southern Medical University. His research interests include photoacoustic tomography and medical image processing.

Wufan Chen received his B.S. degree in 1975 and M.Sc. degree in 1981, both from Beihang University. He is currently a full professor at the Guangdong Provincial Key Laboratory of Medial Image Processing, School of Biomedical Engineering, Southern Medical University. His research interests include biomedical imaging principle and image processing.

Li Qi received his Ph.D. degree in Optical Engineering in 2016 from Nanjing University. He is currently a lecturer at the Guangdong Provincial Key Laboratory of Medial Image Processing, School of Biomedical Engineering, Southern Medical University in Guangzhou, China. His research interests include photoacoustic imaging and optical coherence tomography.

Appendix A. Supplementary material

Supplementary material

Data availability

Data will be made available on request.