Ultrasound-guided sound speed correction for photoacoustic computed tomography

Abstract

Photoacoustic computed tomography (PACT) reconstructs high-resolution images of various chromophores in deep biological tissue. A key to high-quality reconstruction is accurate compensation for the spatially heterogeneous speed of sound (SoS) in tissue. Existing computational methods often estimate or compensate SoS by tuning it directly in the image domain, for example by optimizing sharpness or contrast of reconstructed PA images. However, because the PA signal-to-noise ratio (SNR) decays rapidly with depth due to optical attenuation, such image-domain cues become less informative in deeper regions, limiting SoS accuracy there. Here, we present a dual-modal deep learning framework to correct the heterogeneous SoS via joint processing co-registered PA and ultrasound (US) images. We estimate the spatially varying SoS map from the US image and then fuse the SoS map with the PA image to compute a reduced-aberration photoacoustic image. This method takes advantages of the rich speckle and high SNR in the co-registered US image – and thus can compensate for SoS with high accuracy and efficiency. We tested this method on numerical and tissue-mimicking phantoms, demonstrating cross-domain generalization. In-vivo results demonstrate that incorporation of the predicted SoS maps significantly improved PA image quality, enhancing structural detail and reducing acoustic artifacts. Via fusing the US and PA images, our method produces high-contrast PA images with significantly reduced SoS distortion and artifacts.

1. Introduction

Photoacoustic (PA) tomography is a non-invasive biomedical imaging technique that can furnish comprehensive insights into the anatomical [1], [2], [3], [4], [5], functional [6], [7], [8], [9], and molecular [10], [11], [12] attributes of deep biology tissues at high resolution. To reconstruct a PA image, photoacoustic computed tomography (PACT) commonly uses universal back-projection (UBP) or related beamforming schemes to combine time series from multiple receive elements under the assumption of a known speed of sound (SoS). Accurate knowledge of the spatially varying SoS is therefore critical for producing high-contrast, artifact-reduced PA images [13], [14], [15].

Direct measurement of the SoS map at each anatomical site is costly, time consuming, and seldom performed in practical PACT systems [13], [16], [17]. A common practice is to use one or a few fixed SoS values to approximate the SoS map [2], [6], [18], and some methods adaptively estimate such global values from PA data [19]. Although these strategies can correct low-order phase errors, they remain suboptimal in acoustically heterogeneous tissue.

Substantial effort has focused on estimating dense SoS maps directly from PA measurements. Model-based reconstruction approaches have explored joint reconstruction techniques to simultaneously estimate SoS and PA initial pressure using PA signals and acoustic wave equations in either frequency domain [20], [21], [22], [23], [24], [25], or time domain [26], [27], [28]. More recently, constrained optimization–based joint reconstruction methods have been proposed to further stabilize the simultaneous estimation of initial pressure and SoS from PA data alone [29]. In parallel, data-driven techniques have been explored to adaptively estimate SoS in complex media [30], [31], [32], [33]. Despite their promise, many such methods are computationally intensive and sensitive to noise, limiting their practicality in real-time or low signal-to-noise ratio (SNR) settings. In addition, k-Wave–based PACT simulation frameworks have been developed to generate realistic in-silico and numerical breast phantoms, providing synthetic datasets for systematically testing reconstruction and quantitative imaging strategies [34], [35], [36].

Building on this foundation, deep learning was introduced to further advance PACT reconstruction. Initial works demonstrated feasibility on in-silico phantoms and simplified setups [37], [38], [39], [40], and later deep neural networks (DNNs) showed promise in mitigating under-sampling and noise in-vivo [41], [42], [43], [44]. More recently, learning-based image reconstruction has been applied to fast 3D transcranial PACT to mitigate skull-induced aberrations and recover vascular structures from distorted measurements [45]. To more directly address SoS-related artifacts, Jeon et al. developed SegU-Net, which reduced artifacts in both simulations and in-vivo experiments [46], yet its performance remains dependent on PA image quality, limiting its broader applicability. Subsequently, Dehner et al. introduced DeepMB, a real-time PA reconstructor that enables interactive adjustment of a single global SoS value for fast, high-quality imaging [47]. However, its reliance on one global SoS value, rather than a spatially varying SoS map, can limit performance in strongly heterogeneous media.

Because ultrasound (US) typically exhibits rich speckle and high SNR, estimating dense SoS maps from US is generally more stable and accurate than from PA alone. Traditional approaches, such as those developed by Jaeger et al., laid the foundation for SoS imaging and aberration correction using multiple ultrasound transmission angles [48], followed by improved forward models and estimation pipelines [49], [50]. Recently, the use of deep learning has further advanced in this field, particularly with synthetic datasets generated through the k-Wave software. These datasets allow for the precise annotation of ultrasound signals with local sound speed information, enabling more accurate SoS predictions [51], [52], [53], [54], supporting both research and potential clinical translation.

Building upon these advancements, Shi et al. developed a DNN model that generates SoS maps from US images and then applies the conventional time-reversal (TR) method to reconstruct PA images [55]. While this compensates heterogeneous SoS, TR remains computationally expensive and can struggle with noise and background clutter. To our knowledge, a unified end-to-end architecture that estimates SoS from US and performs PA reconstruction via deep learning has not been developed.

Here we present an end-to-end deep-learning framework that (i) predicts a spatially varying SoS map from US B-mode and (ii) reconstructs a PA image conditioned on that map—enabling aberration correction without RF data or TR. We acquire co-registered US/PA frames on a clinically compatible PA–US system. A fine-tuned residual encoder–decoder in our network predicts the SoS map from the US image. From the predicted SoS range, we uniformly sample eight SoS values and beamform PA images to form an 8-image stack. The predicted SoS map is then one-hot encoded into 80 bins and fed to squeeze-and-excitation (SE) gates to modulate decoder features [56], while the 8-image PA stack is projected by a 1× 1 convolution into an 80-channel embedding. The two streams are fused to yield the final PA reconstruction. By exploiting the high SNR and rich speckle of US and explicit SoS conditioning, our method corrects phase errors, suppresses sidelobes, and reduces background noise with far lower runtime than TR method. We validate the approach on in-silico, ex-vivo, and in-vivo data, showing consistent gains in full width at half maximum (FWHM), Contrast-to-Noise Ratio (CNR), Peak Signal-to-Noise Ratio (PSNR; reported only where a reference exists—in-silico and ex-vivo phantom), and computational efficiency in heterogeneous media.

2. Methods

2.1. Deep-learning network architecture

As shown in Fig. 1, our deep-learning network has two main tasks: estimating the SoS map from the US data and fusing the SoS map with the photoacoustic image. The network uses ResU-Net as the backbone architecture for both tasks [57], [58], which consists of an encoder and decoder connected by residual connections at each level.

Fig. 1.

Two-task framework: Task 1 (blue) estimates the SoS map from a US stack using a pretrained ResU-Net (ResNet-34 encoder). Task 2 (gray) reconstructs PA images by fusing the predicted SoS (one-hot, 80 ch) with PA data (DAS×8 using single SoS values from the map, then 1×1 conv → 80 ch). Down-sampled SoS features modulate decoder activations via SE blocks (GAP–FC–ReLU–FC–Sigmoid, channel-wise weighting). Inputs/outputs are labeled. ‘ch’ in the figure means ‘channel’.

In the first task, we estimate the SoS map from US images. To better exploit speckle and texture, the network takes a multi-channel input composed of the original B-mode image plus two log-compressed variants with different offsets (“gain”). Here, “gain” is the dB-domain offset used in the log mapping (not electronic amplification) and effectively defines a display window [−G,0] dB that is rescaled to [0,1]. We use two windows [${-G}_{\mathit{low}},0$] and [${-G}_{\mathit{high}},0$] with $G_{\mathit{low}}<G_{\mathit{high}}.$ $G_{\mathit{low}}$ yields a tighter window that accentuates bright structures and edges (sharper boundaries), whereas $G_{\mathit{high}}$uses a wider window that lifts low-intensity echoes and reveals more subtle tissue texture at greater depths. For each group of images that shares similar acquisition and contrast statistics (e.g., one in-silico scene type, the ex-vivo phantom data, or the in-vivo data), we choose a single pair of $\left({\mathrm{G}}_{\text{low}},{\mathrm{G}}_{\text{high}}\right)$by briefly sweeping candidate values to bracket the B-mode intensity histogram. The same pair is then used for all images in that group, and we found the results to be insensitive to the exact choice. The picking mimics clinical gain tuning and exposes the model to complementary views of the same anatomy, improving over prior single-channel or synthetic-delay inputs [27], [51], [53]. Compared to the conventional U-Net [58], our ResU-Net employs a ResNet-34 encoder (stages with 3/4/6/3 residual blocks) pretrained on ImageNet [59]. Residual connections in the network ease optimization and deepen feature extraction; transfer learning provides generic priors on edges and tissue texture that speed convergence and improve accuracy with limited data. Batch normalization and rectified linear units (ReLU) were systematically employed following each convolutional and transposed convolutional layer to promote stable and efficient training, except for the final 1× 1 convolutional layer, to stabilize the training process and introduce non-linearity.

In the second task, we fused the estimated SoS map with the PA data. We adopt an 80-channel ResU-Net variant (U-shaped), chosen for its ability to capture features from both the SoS distribution and PA artifacts. In this configuration, pretrained weights are not used because the modified 80-channel input is incompatible with standard pretrained backbones. To better align the two data modalities, the predicted SoS map was one-hot encoded into 80 discrete channels (details in Supplementary Method 9). In parallel, we beamform eight single-SoS DAS reconstructions using speeds uniformly sampled from the min–max range of the predicted SoS; these eight images are stacked and passed through a 1× 1 convolution to produce an 80-channel feature map, matched to the SoS one-hot dimensionality. The 80-channel SoS map was downsampled to match the spatial resolution at each decoder stage and fed to SE block [56], which generated channel-wise reweighting vectors. Within the SE block, Global Average Pooling (GAP) first “squeezes” each feature map into a per-channel descriptor; this descriptor is then passed through two Fully Connected (FC) layers with a ReLU and a Sigmoid to generate the channel-wise weights. These weights are finally applied to the decoder features via element-wise channel multiplication. This design encourages the network to emphasize acoustically informative channels and strengthens the fusion of SoS information with PA reconstructions.

2.2. Loss functions and training configuration

For both deep-learning tasks—estimating SoS maps and generating high-resolution PA images—the models were trained over 300 epochs with a batch size of two, employing distinct learning rates tailored to each task. Specifically, learning rates were chosen by short log-spaced range tests with validation monitoring. For the first task we tested $\{1\times {10}^{-2},5\times {10}^{-3},2\times {10}^{-3},1\times {10}^{-3},5\times {10}^{-4}\}$; for the second task we examined $\{1\times {10}^{-4},5\times {10}^{-5},2\times {10}^{-5},1\times {10}^{-5},5\times {10}^{-6}\}$. Each candidate was run for 10–15 pilot epochs, and we selected the largest rate that yielded a stable, monotonically decreasing validation loss without divergence: $2{\times 10}^{-3}$ for the first task and $2{\times 10}^{-5}$ for the second task. The lower rate in the second task reflects its greater sensitivity to step size from fusing multimodal inputs and an untrained 80-channel encoder. After selection, learning rates were kept fixed. To prevent overfitting, early stopping was implemented based on the validation loss. Training was optimized using the Adam optimizer [60] with a weight decay of ${1\times 10}^{-8}$, which helped stabilize updates and regularize the models. All models were developed using PyTorch [61] version 1.12 and executed on an NVIDIA GeForce RTX 4090D graphics card. GPU memory usage differed between tasks, with the first task consuming 2.7 GB and the second task 11.8 GB. For both tasks, we optimized performance using the following loss function:

$$L=L_{\mathit{char}}\left(X_S,Y\right)+\mu L_{\mathit{edge}}(X_S,Y)$$ (1)

Here, $X_S$ denotes the predicted image and $Y$ represents the ground-truth image. The coefficient μ, which controls the relative weighting of the two loss components, was fixed at 0.05, following established practices [62]. the term $L_{\mathit{char}}$ is the Charbonnier loss [63]:

$$L_{\mathit{char}}=\sqrt{{‖X_s-Y‖}^2+{\epsilon }^2}$$ (2)

. We set the constant $\epsilon$ to ${10}^{-3}$ across all experiments based on empirical determination. Additionally, $L_{\mathit{edge}}$ is the edge loss, which is formulated as:

$$L_{\mathit{edge}}=\sqrt{{‖{\nabla }^2(X_s)-{\nabla }^2(Y)‖}^2+{\epsilon }^2}$$ (3)

where ${\nabla }^2$ represents the Laplacian operator. To prevent overfitting, we applied early stopping based on validation PSNR between the predicted image $X_s$ and ground-truth $Y$. We saved the checkpoint with the highest validation PSNR and stopped training when this metric failed to improve within a fixed patience window.

2.3. In-silico dataset preparation

A dual-modal PA–US imaging system was simulated using the k-Wave toolbox [52]. Ultrasound RF data were generated from spatial distributions of SoS and density, while PA RF data were generated from scene-specific initial-pressure maps ($p_0$), both modalities were run on the same spatial grid, time step ($∆t$), transducer model, and perfectly matched layer (PML) to ensure identical acoustic settings. Fig. 2 summarizes the distribution of our in-silico scenes. We instantiate six scenario categories: (1) Small Tissue; (2) Various Tissues (large tissue mixed with smaller tissue); (3) Small Tissue + Skin; (4) Big Tissue + Skin; (5) Various Tissues + Skin (skin over mixed large/small tissues); and (6) Multi-layers (stacked skin, big tissue, small tissue, and blood/water). Class-wise acoustic parameters are listed in Table 1 [54], [64], [65], [66], [67], [68], [69], [70], [71], [72] For example, “Big Tissue” is obtained by averaging values for skeletal muscle and heart muscle; “Small Tissue” follows heart-lumen–like parameters; and “Skin” uses typical values reported for human skin (epidermis and dermis) as the superficial soft-tissue layer. In the spatial layout, “Big Tissue” regions are modeled with simplified ellipses to capture realistic bulk mass variation, whereas the placement of “Small Tissue” regions is guided by image structures derived from breast ultrasound datasets [73]. Each scene is generated either with or without a uniform skin layer to maintain consistent boundary conditions when needed. Overall, the six scenarios are constructed from four base tissue classes—skin, small tissue, big tissue, and blood/water—with the skin-covered configurations and “Multi-layers” cases formed by stacking these classes.

Fig. 2.

(a) Data-generation and reconstruction pipeline. PA branch: an initial-pressure map built from vessel masks plus simple synthetic shapes, together with the same SoS map, is supplied to k-Wave to simulate PA RF data; TR reconstruction with the true heterogeneous SoS map (TR+GT) yields the ground-truth PA image. US branch: a sound-speed map and a density map are supplied to k-Wave to simulate US RF data, which are then reconstructed with DAS beamforming to yield US B-mode images. (b) Representative in-silico scenes: Small Tissue, Various Tissues, Small Tissue-Skin, Big Tissue-Skin, Various Tissues-Skin, and Multi-Layers. For each scene we show the sound map (m/s) and the density map (kg/m³) shared by US and PA, and a representative US image; the corresponding PA initial-pressure map is generated as described in (a).

Table 1.

Characteristic parameters for speed of sound and density for different tissues.

Name	${\mathit{\mu }}_{\mathit{c}}$(m/s)	${\mathit{\delta }}_{\mathit{c}}$(m/s)	${\mathit{\mu }}_{\mathit{density}}\mathit{(}\mathit{kg}\mathit{/}{\mathit{m}}^{\mathit{3}}\mathit{)}$	${\mathit{\delta }}_{\mathit{density}}\mathit{(}\mathit{kg}\mathit{/}{\mathit{m}}^{\mathit{3}}\mathit{)}$
Skin	1624.0	91.8	1109.0	14.0
Small Tissues (e.g. Heart Lumen)	1578.2	11.3	1050.0	19.0
Big Tissue (e.g., Muscle, Heart Muscle)	1574.9	19.9	1085.2	29.8
Blood	1549.3	1.3	1020.0	1.0
Water	1482.3	0.0	994.0	0.0

Note: ${\mathit{\mu }}_{\mathit{c}}$ is the mean value of SoS when, ${\mathit{\delta }}_{\mathit{c}}$ is the standard deviation of SoS. ${\mathit{\mu }}_{\mathit{density}}$ is the mean value of density map, and ${\mathit{\delta }}_{\mathit{denstity}}$ is standard deviation of the density map. Units: SoS (m/s), density (kg/m³).

For PA, each scene additionally includes an initial-pressure map $p_0$ built from vessel masks in [74], augmented with simple ellipses and lines to create spatially varying absorbers. We assume homogeneous 1064-nm illumination and assign absorption primarily to the vessel masks (background negligible). At this wavelength, vessel absorption coefficients are sampled from reported ranges for oxygenated and deoxygenated blood, so that different vessels exhibit slightly different $p_0$ amplitudes even after normalization and thus span a range of intrinsic absorber strengths. Under this homogeneous-fluence assumption, depth-dependent optical attenuation inside the object is not modeled. The $p_0$ maps are optionally edge-smoothed and normalized, then propagated in k-Wave on the same SoS and density grids used for US, so US scattering and PA propagation share identical acoustic conditions (see Supplementary Method 3 for the $p_0$ formulation and symbol definitions).

After generating the spatial distributions of density and SoS for US imaging, as well as the $p_0$ for PA imaging, the corresponding RF data were generated using the k-Wave toolbox [52], as illustrated in Fig. 2(a). We denote the RF data matrices by $r_{\mathit{us}}\in {\mathbb{R}}^{T_{\mathit{us}}\times M}$ and $r_{\mathit{pa}}\in {\mathbb{R}}^{T_{\mathit{pa}}\times M}$, where each column is the time trace recorded by one receive element, $M=128$, and $T_{*}$ is the number of time samples per A-line. In our setup, $T_{\mathit{us}}=4301$ and $T_{\mathit{pa}}=2048$. The probe was modeled as a flat 1D linear array of 128 contiguous elements with 0.3-mm lateral pitch and width (4 grid points, zero kerf), placed along the top boundary of the computational grid, centered laterally and aligned with the y-axis, and used for both transmission and reception. Both modalities were simulated under identical acoustic settings—the same spatial grid, medium maps, transducer model, PML, and $\mathrm{\Delta }t$ (thus the same temporal sampling rate, see Supplementary Method 4 for discretization and sampling details). Detailed simulation parameters (Table 2) were as follows.

Table 2.

K-wave parameter values.

k-wave parameter	Parameter value
Grid size	0.075 mm
Simulation region size	30 mm$\times$45 mm$\times$8 mm
Speed of sound	Designed using Ultrasound Dataset [73]
Media Density	Designed using Ultrasound Dataset [73]
Transducer element number	128
Transducer element width	0.3 mm
CFL (Courant–Friedrichs–Lewy) number	0.3
PML thickness	20 grid size

During data generation, PA RF data in all splits (train/val/test) were corrupted with additive white Gaussian noise (AWGN) using k-Wave’s addNoise. For each frame (2048 samples × 128 channels), a target SNR was uniformly sampled between 5 and 15 dB, and addNoise was applied channel-wise to scale the zero-mean noise accordingly, yielding $r_{\mathit{pa}}^{\mathit{noisy}}=r_{\mathit{pa}}+n\in {\mathbb{R}}^{2048\times 128}.$ We did not add noise to US RF data, as its acquisition SNR is typically high and stable under our settings. To support both tasks—SoS estimation from US and PA reconstruction using the estimated SoS—samples from each configuration were proportionally allocated to training, validation, and testing; the per-task distributions are summarized in Table 3.

Table 3.

Sample numbers for training, testing and validation.

	Small Tissue	Various Tissues	Small Tissue-Skin	Big Tissue-Skin	Various Tissues-Skin	Multi-Layers	Total
Training	300	300	900	600	900	600	6000
Validation	50	50	150	100	150	100	600
Testing	150	150	450	300	450	300	1800
All	500	500	1500	1000	1500	1000	6000

2.4. Experimental setup and data acquisition

2.4.1. Dual-modal US/PACT system and common acquisition settings

A co-registered ultrasound and photoacoustic tomography (US/PACT) system was used in this study (Supplementary Fig. 2 and Supplementary Method 2). A Q-switched 1064-nm Nd:YAG laser (Spectra-Physics) operated at a pulse repetition frequency of 20 Hz, and the laser output was delivered via a custom bifurcated fiber bundle to illuminate the target on both sides of the ultrasound probe. The surface fluence was set to 15 mJ/cm², remaining below the ANSI safety limit. Photoacoustic signals were detected by a 128-element linear array transducer (Verasonics L11–4V; center frequency 6.25 MHz, 96 % bandwidth) and acquired using a Verasonics Vantage research platform. For ultrasound imaging, five plane waves were transmitted at angles of −10°, −5°, 0°, 5°, and 10° with 100 μs intervals, followed by laser triggering for PA acquisition. Unless otherwise specified, the same acquisition settings were used across experiments to ensure fair comparisons.

2.4.2. Reference SoS measurement by acoustic transmission test

Material-matched reference sound speeds were measured using an acoustic transmission test with a tungsten-wire PA target, following prior work [13], [75]. Briefly, homogeneous slabs (agar, all-fat, and all-muscle) of known thickness $d$ were placed in a water bath above a tungsten-wire reference. Under identical acquisition settings, PA A-lines from the tungsten wire were acquired with and without the slab in place. The slab sound speed $c_s$ was computed from the additional time-of-flight delay $\mathrm{\Delta }t$ introduced by the slab:

$${\mathrm{c}}_{\mathrm{s}}={(\frac{1}{{\mathrm{c}}_{\mathrm{w}}}-\frac{\mathrm{\Delta t}}{\mathrm{d}})}^{-1}$$ (4)

where $c_w$ denotes the speed of sound in water and $\mathrm{\Delta }t$ is the measured time shift between the tungsten-wire–induced PA peaks acquired with and without the slab. Reference SoS statistics (mean and standard deviation) were computed from repeated acquisitions for each slab material.

2.4.3. In-vivo studies and ethics approvals

In-vivo validation was conducted in both animal model and human volunteer model. For the animal study, we imaged the neck of an 8-week-old male Sprague–Dawley rat (180–220 g). The rat was anesthetized with 1.2 % vaporized isoflurane and positioned supine on a 37 °C heated pad. A membrane-sealed water tank was placed over the neck with ultrasound gel for acoustic coupling. All animal procedures were approved by the Animal Ethics Committee of the City University of Hong Kong. For the human study, in-vivo US/PA imagings were performed on the forearm of healthy adult volunteers under a protocol approved by the Human Research Ethics Committee of the City University of Hong Kong. In some acquisitions, an additional superficial coupling layer was introduced to emulate acoustic heterogeneity (details provided in the corresponding Results section).

2.4.4. ROI definition and quantitative metrics

For quantitative evaluation, we computed CNR and lateral FWHM using consistent ROIs within each dataset. For CNR, a signal ROI was defined as a depth band containing visible vessels, while a background ROI was defined as a vessel-free depth band with the same lateral extent to estimate background fluctuation. The same ROI sizes and lateral span were used for all reconstruction methods for that dataset. When different SoS assumptions led to an apparent axial shift of structures, ROIs were shifted only along depth to remain centered on the same anatomical region, while keeping ROI dimensions unchanged to ensure fair comparison. CNR was computed using the standard ROI-based definition (Supplementary Method 8).

FWHM was used for quantitative evaluation of profile sharpness on a representative target that is visible across reconstructions. For each dataset, we selected a target (e.g., a point target in in-silico phantoms or a representative vessel segment in experiments) and kept the target identity and measurement location consistent across all methods. For each reconstruction, a 1D intensity profile was extracted through the local peak of the target. The profile was normalized by its peak value, and the FWHM was computed as the full width at half maximum. Depending on the experiment, the profile was taken either along the axial direction or the lateral direction. In the in-silico point-target study, we report both axial and lateral FWHM as functions of depth. In ex-vivo and in-vivo vessel experiments, we primarily report lateral FWHM, because vessel orientations and shapes can make axial width less comparable across cases.

In addition to ROI definitions for quantitative metrics, region partitioning is also required for the dual-SoS method, where a piecewise-constant SoS map is constructed for two-layer beamforming. Specifically, the dual-SoS map was constructed as a two-region, piecewise-constant approximation derived from the US image. The split boundary was defined using the most clearly delineated layer interface. For the three-layer phantom in Fig. 4(a), we used the second (most prominent) interface as the split, merging the upper two layers into one region and treating the bottom layer as the second region; the two SoS values were set as the mean SoS within each region.

Fig. 4.

(a) Qualitative results. I: GT SoS map used in k-Wave; II: US B-mode (DAS beamforming from k-Wave-simulated RF data); III: SoS predicted by deep learning. IV: PA image reconstructed with DAS using the single SoS selected by the AUS (optimal c =1526 m/s); V: dual-speed DAS (CF number based$,c_{\mathit{shallow}}/c_{\mathit{deep}}$ = 1520/1480 m/s; three-layer split: second US interface; see Section 2.4.4); VI: deep-learning PA-only fusion using SegU-Net; VII: time-reversal with the predicted SoS; VIII: proposed DL reconstructor using B-mode stacks + predicted SoS (RF-free); IX: reference (time-reversal with ground-truth SoS; green boxes mark the 12 point targets). (b-e) Quantitative analysis: (b) CNR comparison across methods; (c) axial FWHM vs. depth; (d) PSNR vs Reference (Time-Reversal with Ground-Truth SoS); (e) lateral FWHM vs. depth.

3. Results

3.1. In-silico and ex-vivo phantom study of speed of sound estimation

For each ground truth SoS map, we generated 20 independent US-RF realizations by applying small RF-level perturbations while keeping the SoS map fixed, and we report errors as mean absolute error (MAE)$\pm$standard deviation (SD) across these realizations (Table 4).

Table 4.

Category-wise SoS estimation error (MAE, m/s; mean$\pm$SD) and structural similarity (SSIM; mean$\pm$SD) for our network on the in-silico test set. Statistics are computed over 20 US-RF realizations per ground-truth SoS map.

	Category	SoS estimation error (MAE, mean$\mathit{\pm }$SD)	Structural similarity index (SSIM)
Metrics for Our Network when Predicting SoS map from US images	Small Tissue	0.95$\pm$ 0.06	0.81$\pm$ 0.02
	Various Tissues	1.28 $\pm$ 0.11	0.89$\pm$ 0.01
	Small Tissue-Skin	2.27 $\pm$ 0.18	0.78 $\pm$ 0.02
	Big Tissue-Skin	1.95 $\pm$ 0.14	0.98 $\pm$ 0.01
	Various Tissues-Skin	2.82 $\pm$ 0.19	0.88 $\pm$ 0.03
	Multi-Layers	3.90 $\pm$ 0.49	0.79 $\pm$ 0.02

Under this protocol, our fine-tuned ResU-Net delivers accurate SoS estimation across tissue configurations. In simpler cases like “Small Tissue,” ResU-Net we utilized yields minimal error, while more complex scenarios—layered structures or skin interfaces—lead to larger deviations. The highest error is observed in the “Multi Layers” case, reflecting the greater challenge posed by abrupt acoustic transitions. Despite these variations, ResU-Net reliably preserves anatomical boundaries and reconstructs key spatial patterns of SoS, indicating strong generalization to heterogeneous media. To quantify structural fidelity, we additionally computed the structural similarity index (SSIM) between the predicted and ground-truth SoS maps. ResU-Net attains high SSIM values across all categories and outperforms U-Net and SegU-Net (see Table 4 and Supplementary Table S2), indicating that it preserves anatomical boundaries and key spatial patterns of SoS more faithfully than the baseline networks. A detailed comparison with U-Net and SegU-Net is provided in Supplementary Fig. 1, where ResU-Net consistently outperforms the other models in both spatial fidelity and quantitative accuracy.

For quantitative assessment, the mean SoS and its standard deviation were computed from ten repeated acquisitions and compared with material-specific reference SoS values measured on homogeneous slabs using the acoustic transmission test described in Section 2.4.2; these reference values are listed in Supplementary Table 3. As summarized in Fig. 3(b), our network delineates muscle–fat boundaries and accurately recovers the depth-dependent SoS with minimal bias, supporting generalization beyond the synthetic domain. The quantitative comparison is reported in Table 5.

Fig. 3.

(a) In-silico test results across six categories (Small Tissue, Various Tissues, Small-Tissue–Skin, Big-Tissue–Skin, Various-Tissues–Skin, Multi-Layers). For each case the rows show the US B-mode input, the ResU-Net prediction, and the ground-truth SoS map; color bars indicate US intensity (dB) and SoS (m/s). Errors are summarized as MAE$\pm$SD over 20 independent RF realizations per ground-truth SoS map (see Table 4). (b) Ex-vivo layered bovine-steak/agar phantom. Left: specimens; middle: US B-mode; right: predicted SoS map with labeled ROIs (1–5). ROIs 3–1 and 3–2 denote the muscle and fatty regions within the third bovine layer, respectively. ROI-wise reference SoS values are obtained from homogeneous slab measurements using the tungsten-wire method (Supplementary Method 5; Supp. Table 3), and the corresponding MAE$\pm$SD are reported in Table 5.

Table 5.

ROI-wise Speed-of-Sound Evaluation on the Ex-Vivo Layered Phantom (Material-Wise Reference vs. Our Network Estimates). ‘Ground Truth SoS’ denotes the material-wise reference mean$\pm$SD from homogeneous slab measurements (Supp. Table S1). ‘Estimated SoS’ is the segment-wise mean$\pm$SD from the predicted SoS map. ‘Error’ reports bias = (estimated mean − reference mean); the uncertainty reflects the SD of the segment estimate.

Area	Depth (mm)	Material	Ground truth SOS (m/s)	Estimated SOS (ResU-net, m/s)	Error SoS (ResU-net, m/s)
1	20.0 – 24.9	steak(muscle)	1604.6 $\pm$ 6.2	1609.5 $\pm$ 7.9	4.9 $\pm$ 7.9
2	24.9 – 27.6	agar	1538.3 $\pm$ 1.2	1526.0 $\pm$ 9.6	12.3 $\pm$ 9.6
3–1	27.6 – 32.7	steak(muscle)	1604.6$\pm$ 6.2	1593.4 $\pm$ 10.2	11.2 $\pm$ 10.2
3–2	27.6 – 32.7	steak(fat)	1502.4 $\pm$ 3.9	1494.2 $\pm$ 1.5	8.2 $\pm$ 1.5
4	32.7 – 35.4	agar	1538.3 $\pm$ 1.2	1540.0 $\pm$ 9.6	1.7$\pm$ 9.6
5	35.4 – 40.0	steak(fat)	1502.4 $\pm$ 3.9	1514.8 $\pm$ 11.9	12.4 $\pm$ 11.9

3.2. In-silico phantom study of photoacoustic image reconstruction using SoS map

In the in-silico study, we first used k-Wave to synthesize co-registered US/PA data from ground-truth (GT) SoS maps for the six tissue scenarios described in Section 2.2. These data were used to train and quantitatively evaluate the SoS-prediction network. For the reconstruction experiment in Fig. 4, we additionally generated a three-layer in-silico phantom using the same k-Wave configuration but with an extended axial field-of-view (12 cm), containing twelve-point absorbers placed between 2 and 8 cm depth to enable depth-dependent FWHM and CNR analysis. Our network then firstly predicted SoS map from the US images (MAE = 3.4 m/s). We use several reconstructions for comparison: DAS with a single SoS using automatic SoS selection (AUS) via the Brenner focus function [76] (details in Supplementary Method 6.1); dual-speed DAS chosen by CF number [2] (Supplementary Method 6.2); deep-learning PA-only fusion (SegU-Net) [46]; time-reversal with the predicted SoS (TR+pred) and our method [77]. Time-reversal with the ground-truth SoS (TR+GT) served as the reference.

Fig. 4 summarizes these comparisons. As expected, TR+GT provides the sharpest reference reconstruction. Using the predicted SoS in TR (TR+pred) yields axial and lateral FWHM curves that closely track those of TR+GT, confirming that the SoS error of 3.4 m/s is small enough for a highly SoS-sensitive model-based reconstruction. Our learned method achieves resolution that is largely comparable to TR+pred—with slightly broader axial FWHM at some depths but similar or even smaller lateral FWHM in others—while consistently providing higher CNR (Fig. 4b) and PSNR relative to TR+GT (Fig. 4d). All bars are reported as mean$\pm$SD, computed from 10 independent re-simulations in which we add AWGN of the same power to the PA RF data and re-run the k-Wave pipeline (US RF remains noise-free).

Overall, these in-silico results indicate that a reasonably accurate SoS predicted from US is sufficient for effective acoustic correction, and that our method achieves a favorable trade-off: resolution close to TR+pred and TR+GT, but with improved CNR and substantially lower computational cost than physics-based TR (see Section 3.6 and Supplementary Method 8 for an overall runtime comparison), while providing markedly better image quality than classical DAS beamforming.

3.3. Ex-vivo tissue phantom study for enhanced PA reconstruction using SoS map

To further validate our method, we performed an ex-vivo study with a chicken-breast slab covering a 90 $\mu m$ tungsten wire (Fig. 5a). The reference image (Fig. 5i) was acquired after removing the tissue, providing a distortion-free result. The chicken breast SoS was measured as $1539.8\pm 1.47\mathrm{m}/\mathrm{s}$ using the acoustic transmission test in Eq. (4) with six samples with different thicknesses. The predicted SoS map (Fig. 5c) agrees closely with the measured value $\left(1542.8\pm 3.67\mathrm{m}/\mathrm{s}\right)$ using six images, supporting accurate aberration correction in heterogeneous tissue. Fig. 5 compares PA reconstructions (d) single-SoS AUS (1540 m/s), (e) dual-speed (CF based), (f) deep-learning PA-only fusion (SegU-Net), (g) time-reversal with the predicted SoS map, and (h) our proposed method that uses the predicted SoS map. Consistent with the in-silico study, our method (Fig. 5h) best localizes the wire with higher contrast and fewer sidelobes than single/dual-speed beamforming (Fig. 5d–e). The TR+pred result (Fig. 5g) improves focusing but remains noisier, while SegU-Net (Fig. 5f) exhibits poorer axial sharpness and occasional spurious double-peaks.

Fig. 5.

Ex-vivo chicken-breast experiment. (a) Schematic showing the measurement plane. (b) B-mode US image of the region (scale bar: 5 mm). (c) Predicted SoS map (m/s). PA reconstructions with different methods: (d) PA image reconstructed with DAS using the single SoS selected by the AUS (optimal c =1540 m/s). (e) dual-speed based on CF number ($c_{\mathit{shallow}}/c_{\mathit{deep}}$ = 1500/1550 m/s), (f) deep-learning PA-only fusion using SegU-Net, (g) time-reversal using the predicted SoS map, (h) proposed deep-learning method with the predicted SoS map, and (i) reference image acquired after removing the tissue (tungsten-wire only). (j) CNR comparison (dB) of the wire target across methods. (k) gain-normalized PSNR (dB) against the reference wire image within the common ROI. (l) Lateral profiles through the wire (peak-normalized) with FWHM reported in the legend; dashed line denotes half-maximum. Mean$\pm$SD for (j-l) are computed from 6 repeated acquisitions.

For quantitative analysis, we extracted lateral profiles through the wire and computed FWHM (Fig. 5l), CNR (Fig. 5j) and PSNR (Fig. 5k). The mean values and standard deviations were computed from 6 repeated acquisitions. Numerically, lateral profiles through the wire (Fig. 5l) show our method has narrower FWHM than other methods. The CNR bar chart in Fig. 5(j) indicates our method achieves the highest CNR. PSNR in Fig. 5(k) shows only minor differences across methods. Since PSNR is computed in a cropped ROI after only a tiny lateral shift and a single global gain (Supplementary Method 7), it primarily reflects structural MSE around the single wire target; hence the methods yield similar PSNR.

Collectively, in this ex-vivo phantom with a ground-truth reference, our method achieves higher CNR while maintaining FWHM comparable to TR+pred. Compared with SegU-Net, our method produces sharper vessel depiction with comparable CNR. Overall, it provides the best balance of resolution and contrast among the five methods, demonstrating effective correction of SoS-induced aberrations and improved PA image fidelity.

3.4. In-vivo prediction for internal jugular vein (IJV)

The in-vivo study protocols and ethics approvals are described in Section 2.4.3. Fig. 6 summarizes the reconstruction results across comparison methods with our method.

Fig. 6.

(a) Schematic of the in-vivo rat experiment with the imaging plane indicated. (b) B-mode US image of the jugular-vein region. (c) SoS map predicted by the deep-learning network (d) PA image reconstructed with DAS using the single SoS selected by the AUS (optimal c =1514 m/s). (e) Dual-speed DAS reconstruction, where the two speeds are chosen by the CF number ($c_{\mathit{shallow}}/c_{\mathit{deep}}$ = 1485/1580 m/s). (f) deep-learning PA-only fusion using SegU-Net. (g) Time-reversal reconstruction driven by the predicted SoS map (TR+pred). (h) Our method using the predicted SoS map. In (d–h), dashed boxes mark two vessels (Line 1 and Line 2) used for quantitative evaluation; scale bar, 3 mm. (i) CNR comparison across methods (bar chart) measured within the field. (j,k) Lateral FWHM (bar charts) for Line 1 and Line 2, respectively. Error bars in (i-k) denote SD computed from 6 images.

From the B-mode US image (Fig. 6b), our network predicts a spatially varying SoS map (Fig. 6c) that is then fused with the PA image stack. We compared: AUS (Fig. 6d), dual-speed (CF based; Fig. 6e), deep-learning PA-only fusion using (SegU-Net; Fig. 6f), time-reversal driven by the predicted SoS (TR+pred; Fig. 6g), and our method (Fig. 6h). The dashed boxes in Fig. 6(d)–h mark two vessels (Line 1 and Line 2) used for quantitative analysis. The CNR comparison (Fig. 6i) shows our method achieving the highest CNR in the field, with SegU-Net a close second, while AUS and dual-speed are lowest. TR+pred trails well behind our method due to stronger background clutter.

For Line 1 in the dashed box, the upper wall exhibited composite peaks due to near-field interference (most evident with AUS). Therefore, we measured FWHM on the lower wall for Line 1 and the upper wall for Line 2 which is clean. Under this setting, our method produced the smallest FWHM at both Line 1 and Line 2, comparable to TR+pred. Taken together, our method shows sharper vessel profiles while delivering higher vessel-to-background CNR, underscoring the value of injecting a predicted SoS map from US images to enable effective aberration correction in vivo.

3.5. In-vivo imaging of human forearm

To further assess clinical generalizability, we performed in-vivo US/PA imaging on a healthy adult volunteer under an approved protocol (Section 2.4.3). In this experiment, a thin, boneless chicken-breast layer was placed between the probe and the forearm to introduce superficial acoustic heterogeneity (Fig. 7a).

Fig. 7.

(a) Schematic of the forearm experiment; a boneless chicken-breast slab was placed on the forearm to emulate layered soft-tissue propagation. (b) B-mode US image. (c) SoS map predicted by the deep network. (d–h) PA reconstructions: (d) single-SoS DAS selected by an autofocus approach (AUS; optimal c=1668 m/s); (e) dual-speed DAS chosen by the CF number ($c_{\mathit{shallow}}/c_{\mathit{deep}}$ = 1500/1668 m/s). (f) deep-learning PA-only fusion (SegU-Net); (g) time-reversal using the predicted SoS; (h) proposed DL reconstructor conditioned on the predicted SoS. In (d–h), dashed boxes mark two vessels (Line 1 and Line 2) used for quantification and the insets show magnified views; scale bar, 3 mm. (i) CNR comparison for the representative acquisition shown in this figure, within a common ROI after a single global gain match (cohort-level case-by-case CNR is summarized in Supplementary Fig. S3(b)). (j,k) Lateral FWHM (bar charts) for Line 1 and Line 2, respectively. Error bars in (i-k) denote SD computed from 6 repeated images for this acquisition.

From the co-registered US data we estimated a spatially varying SoS map (Fig. 7c) and generate several PA reconstructions: single-SoS DAS with AUS, dual-speed DAS (CF based), deep-learning PA-only fusion (SegU-Net), time-reversal using the predicted SoS (TR+pred), and our method (Fig. 7d–h; US image in Fig. 7b).

Visually, our method yields clearer vessel continuity and boundaries. For quantification, we extracted lateral intensity profiles along two representative vessels (Line 1 and Line 2 in Fig. 7(d); Fig. 7(j), k). To avoid bias, profiles were taken on segments locally orthogonal to the vessel wall and exhibiting a single-peaked line-spread function; we excluded the superficial, obliquely oriented segment of Vessel 2, and the upper wall for Vessel 1. Our method achieved the narrowest lateral FWHM for Line 1 and Line 2 among five methods. Therefore, even under stronger, layered SoS heterogeneity, our method preserves sharper vessels with high CNR, underscoring robust sound aberration correction.

3.6. Overall performance and computational cost

Across the in-silico phantoms, our model attains CNR on par with TR+GT SoS and higher than TR+pred while suppressing sidelobes and artifacts (Fig. 4). On ex-vivo phantom and in-vivo data, our method delivers the highest CNR while maintaining lateral FWHM comparable to TR+pred (Fig. 5, Fig. 6, Fig. 7).

For the six other in-vivo forearm acquisitions, our method achieves a mean CNR of 48.38$\pm$4.23 dB versus 33.94$\pm$1.50 dB for TR+pred, while reducing the per-frame runtime from 37.89$\pm$6.68 s to 2.54$\pm$0.06 s (≈15 × speed-up) like Supplementary Fig. S3.

CNR varies across acquisitions due to differences in probe angle/contact and vessel morphology. Supplementary Fig. S3 summarizes the per-case results across the six acquisitions, with implementation details provided in Supplementary Method 8. Together, these results confirm consistent CNR gains and a substantial reduction in computational cost for our approach compared with TR+pred.

4. Conclusions

In this study, we aim to mitigate a significant challenge of PA imaging posed by SoS aberrations and streak artifacts, which are critical factors degrading the imaging quality. We propose a novel framework that leverages the high signal-to-noise ratio and rich speckle information of co-registered US images to estimate spatially resolved SoS maps. These maps are then incorporated into the deep-learning-based PA reconstruction pipeline, enabling correction of acoustic aberrations and improvement in image fidelity. The proposed deep learning model is capable of accurately estimating SoS distributions and integrating them into the PA reconstruction process. Unlike methods such as that of Shi et al. [55], which rely on computationally intensive TR-based PA reconstructions, our approach offers a more efficient and scalable alternative.

Extensive in-silico, ex-vivo, and in-vivo studies show that, against four methods—AUS, dual-speed DAS (CF based), TR+pred, and an image-domain network without an SoS prior (SegU-Net)—our method yields sharper vascular delineation across depths/tissues, higher CNR, comparable PSNR (in-silico and ex-vivo phantom only) and substantially lower runtime than time-reversal. The integration of US-derived SoS maps successfully enhances the robustness of PA imaging, which is particularly critical for clinical applications where anatomical and acoustic heterogeneity are prevalent.

The present work is intentionally focused on soft-tissue–like heterogeneity, where SoS varies within a moderate range and density transitions are relatively smooth. In addition, our in-silico optical model assumes homogeneous 1064-nm illumination with contrast encoded through vessel-dependent absorption coefficients, without simulating depth-dependent fluence variations inside the object. While our experiments indicate that this simplified fluence model is sufficient to study the impact of SoS heterogeneity on PA reconstruction in this regime, more realistic light-transport modeling will be required to fully capture fluence distortions in highly heterogeneous tissues, such as brain imaging through the skull. Large discontinuities in both SoS and density at bone–soft-tissue interfaces can introduce strong reflections, refraction, mode conversion, and shadowing that are not yet represented in our current training data.

In future work, we plan to exploit the plug-in nature of our framework—US-based SoS estimation plus SoS-conditioned PA reconstruction—to retrain the SoS module for bone-inclusive phantoms and to combine US-driven SoS priors with more advanced acoustic and optical models specifically tailored for transcranial PACT. We expect such extensions to further widen the applicability of SoS-aware PA reconstruction to challenging neurovascular and deep-tissue imaging scenarios.

CRediT authorship contribution statement

Lidai Wang: Writing – review & editing, Supervision, Methodology, Funding acquisition, Conceptualization. Bin Ouyang: Validation, Methodology. Zheng Qu: Writing – review & editing, Visualization, Validation, Methodology, Conceptualization. Xuanhao Zhang: Writing – review & editing, Writing – original draft, Visualization, Validation, Methodology, Investigation, Conceptualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Biographies

Xuanhao Zhang: Xuanhao Zhang is a Ph.D. student at the Department of Biomedical Engineering, City University of Hong Kong. He received Bachelor's degrees from Nottingham University. His research focuses on photoacoustic imaging and ultrasound imaging.

Zheng Qu: Zheng Qu is a Ph.D. student at the Department of Biomedical Engineering, City University of Hong Kong. He received Bachelor's and Master's degrees from Tianjin University. His research focuses on photoacoustic imaging and ultrasound imaging.

Bin Ouyang： Ouyang Bin is a Ph.D. candidate student at the Department of Biomedical Engineering, City University of Hong Kong. He received Bachelor's degrees from SUN YAT-SEN University. His research focuses on photoacoustic imaging and flexible electronics.

Lidai Wang: Lidai Wang received the Bachelor and Master degrees from the Tsinghua University, Beijing, and received the Ph.D. degree from the University of Toronto, Canada. After working as a postdoctoral research fellow in the Prof Lihong Wang's group, he joined the City University of Hong Kong in 2015. His research focuses on biophotonics, biomedical imaging, wavefront engineering, ultrasonically encoded photoacoustic flowgraphy (UE-PAF), instrumentation and their biomedical applications.

Appendix A. Supplementary material

Supplementary material

Data availability

Data will be made available on request.