Latent diffusion-based image reconstruction for near-infrared spectral tomography

Abstract

Near infrared spectral tomography (NIRST) is a promising non-invasive imaging technique that uses near-infrared light to probe the optical properties of biological tissue. However, due to light scattering in tissue, NIRST image reconstruction is inherently ill-posed and highly sensitive to noise. To address this challenge, an image reconstruction algorithm (Diff-NIRST), which integrated a conditional latent diffusion model, an image autoencoding module, and a conditional signal encoder, was developed. The image autoencoding module reduced image dimensionality to ensure compatibility with the latent space, while the conditional signal encoder transformed NIRST measurements into conditioning inputs for the generative process. The conditional latent diffusion model then reconstructed images of chromophore concentrations. Validation was performed through simulations. The method was subsequently applied to data from a patient case. In numerical simulations, 100 phantoms were reconstructed containing one or two inclusions with diameters of 4-14 mm and varying contrast ratios (CR) of total hemoglobin (HbT) and water content between inclusions and background. Diff-NIRST consistently outperformed conventional Tikhonov regularization and two other deep learning approaches, yielding an average improvement >17.1% in peak signal-to-noise ratio (PSNR), >48.3% and >23.5% reduction in CR error and size error (SE) for HbT, respectively. For water, it achieved an average >11% increase in PSNR, >16.7% and >57.1% reduction in CR error and SE, respectively. Application to breast cancer patient data further demonstrated that Diff-NIRST, trained solely on simulation data, successfully reconstructed clinical images, underscoring its translation potential.

1. Introduction

Near infrared spectral tomography (NIRST) is a non-invasive imaging technique that has been studied extensively for clinical applications in breast [1,2] and functional brain imaging [3,4]. NIRST reconstructs images of physiologically relevant chromophore concentrations—such as oxy-hemoglobin (HbO), deoxy-hemoglobin (Hb), and water [5]—from signals measured at the tissue’s surface. However, NIRST image reconstruction is a typical ill-posed estimation problem that is highly sensitive to noise introduced during data acquisition. This sensitivity arises from the diffusive nature of light propagation in biological tissues and the limited number of surface measurements available for reconstruction.

In recent years, deep learning (DL) approaches [6–10] have improved the quality of NIRST images relative to conventional reconstruction algorithms, such as Tikhonov regularization [11,12]. Among DL-based methods, end-to-end reconstruction schemes—which directly learn the mapping between NIRST signals and chromophore distributions from paired training data—have become particularly popular [9,10]. Although these approaches require extensive training, they reduce computational time significantly during inference because they avoid explicit modeling of light propagation. However, the performance of these end-to-end methods depends on the quality and diversity of the training dataset, and their image quality often degrades when applied to unseen data. To address this limitation, Model-CNN [10], a hybrid approach that integrates DL with the physical forward model governed by the diffusion equation, was developed. Despite its improvements, Model-CNN still struggles to suppress artifacts caused by noisy measurements.

Diffusion models when used as generative models have shown promise for their ability to produce high-quality images in image processing [13–16] and photoacoustic tomography (PAT) image reconstruction applications. For example, Guo et al. applied diffusion models to enhance PAT image quality with limited measurements [17], while González et al. developed a conditional diffusion model in which autoencoder-reconstructed images served as conditional inputs for further refinement [18]. These studies suggest that diffusion models provide distinguishing features for generating high-quality images from noisy measurements compared to conventional neural networks.

Inspired by these results, we developed a latent diffusion model-based image reconstruction algorithm for NIRST (Diff-NIRST). The algorithm integrates a conditional latent diffusion model, an image autoencoding module and a conditional signal encoder. The image autoencoding module is designed to reduce image dimensions from a high-dimensional pixel space to a low-dimensional latent space. The signal encoder transforms NIRST signals into conditional inputs to guide the formation of NIRST images, and the conditional latent diffusion model then acts as an image generator to produce images of chromophore concentrations, such as HbO, Hb, and water. By performing the diffusion process within the latent space, Diff-NIRST substantially reduces the computational burden associated with diffusion models. Unlike Model-CNN, which requires modeling light propagation, or end-to-end DL methods that deterministically map input signals to output images, Diff-NIRST generates images through an iterative denoising process that progressively refines a noisy image over many steps. This iterative process improves robustness to measurement noise, making Diff-NIRST especially effective when training data are limited or noisy while still achieving relatively short reconstruction times.

In this study, Diff-NIRST was developed and validated through a progressive series of numerical simulation tests, and then applied to human patient studies. In addition, Diff-NIRST’s performance was compared to Tikhonov regularization and two deep learning-based methods—Model-CNN [10], and FD-UNet [19].

2. Methods

2.1. Background

Light propagation in biological tissue was modeled using the continuous diffusion equation, which is expressed as:

$$-\mathrm{\nabla }\cdot [D(\mathbf{r},\lambda )\mathrm{\nabla }\mathrm{\Phi }(\mathbf{r},\lambda )]+{\mu }_a(\mathbf{r},\lambda )\mathrm{\Phi }(\mathbf{r},\lambda )=q(\mathbf{r},\lambda )$$ (1)

where $\mathrm{\Phi }(\mathbf{r},\lambda )$ denotes the photon fluence rate at wavelength λ, ${\mu }_a(\mathbf{r},\lambda )$ and $D(\mathbf{r},\lambda )={[3({\mu }_a(\mathbf{r},\lambda )+{\mu }_{\mathrm{s}}^{\prime }(\mathbf{r},\lambda ))]}^{-1}$ are the absorption and diffusion coefficients, and $q(\mathbf{r},\lambda )$ is the isotropic source term [12]. The Robin boundary condition was applied to account for the refractive-index mismatch between tissue and air [12,20].

Instead of solving the optical properties at each wavelength, we direct estimate the chromophore concentrations from all wavelengths using measurements of light fluence from the tissue surface. This is a typically inverse problem. And this inversion is achieved using Diff-NIRST.

2.2. Architecture of Diff-NIRST

Figure 1 illustrates the architecture of Diff-NIRST. As a latent diffusion model, Diff-NIRST incorporates NIRST signals as conditions in a compact and low-dimensional latent space to guide the generation of NIRST images, c. The latent space consists of three subnetworks: (i) a conditional latent diffusion subnetwork, (ii) an image autoencoding subnetwork, and (iii) a conditional signal encoding subnetwork, g_θ. The details of each sub-network are described further below.

Fig. 1.

Diff-NIRST architecture through which a conditional latent diffusion model is guided by encoded multi-wavelength NIRST signals.

2.2.1. Conditional latent diffusion subnetwork

The conditional latent diffusion involves forward and reverse diffusion processes. During the forward diffusion process, NIRST image, c, is encoded by Encoder $\mathcal{E}$ to produce ${\mathbf{z}}_0$ (the latent space representation of c), and is then subjected to T diffusion steps to gradually add noise, ϵ, and generate a sequence of intermediate representations: ${\mathbf{z}}_0,\cdots ,{\mathbf{z}}_T$ . The t^th intermediate representation is denoted by ${\mathbf{z}}_t$ , which is expressed as:

$${\mathbf{z}}_t=\sqrt{\overline{{\alpha }_t}}{\mathbf{z}}_0+\sqrt{1-\overline{{\alpha }_t}}\mathrm{ϵ},\mathrm{ϵ}\sim \mathrm{N}(0,I)$$ (2)

where $\overline{{\alpha }_t}={\prod }_{i=1}^t{\alpha }_i$ , ${\alpha }_i$ are hyper-parameters related to variance. The reverse diffusion is modelled by a denoising U-net in the latent space with parameter θ. It accepts the intermediate perturbed feature maps ${\mathbf{z}}_t$ and $\mathbf{y}$ (compressed NIRST signals in the latent space) as inputs to predict a noise level ${\mathrm{ϵ}}_{\theta }({\mathbf{z}}_t,\mathbf{y},t)$ for recovering feature map $\widehat{{\mathbf{z}}_{t-1}}$ from previous steps [21]:

$$\widehat{{\mathbf{z}}_{t-1}}=\sqrt{\overline{{\alpha }_{t-1}}}\left(\frac{{\mathbf{z}}_t-\sqrt{1-\overline{{\alpha }_t}}\cdot {\mathrm{ϵ}}_{\theta }({\mathbf{z}}_t,\mathbf{y},t)}{\sqrt{\overline{{\alpha }_t}}}\right)+\sqrt{1-\overline{{\alpha }_{t-1}}}\cdot {\mathrm{ϵ}}_{\theta }({\mathbf{z}}_t,\mathbf{y},t)$$ (3)

$\widehat{{\mathbf{z}}_0}$ is recovered by repeating (3) T times, and decoding the final feature map through a decoder, $\mathcal{D}$ , to generate NIRST images, $\hat{c}$ .

The network, ${\mathrm{ϵ}}_{\theta }$ , is trained to predict the noise added at each step by using:

$${\mathcal{L}}_E={\mathbb{E}}_{{\mathbf{z}}_0,\mathbf{y},ϵ\sim \mathcal{N}(0,I),t}\left[\Vert ϵ-{ϵ}_{\theta }({\mathbf{z}}_t,\mathbf{y},t)\Vert \right]$$ (4)

where ${\mathrm{ϵ}}_{\theta }({\mathbf{z}}_t,\mathbf{y},t)$ is a time-conditional U-net. It is populated with cross-attention layers to incorporate encoded signals as reconstruction results. The cross-attention layer is designed to be:

$$\text{CrossAttention}(Q,K,V)=\text{softmax}(\frac{Q{K}^T}{\sqrt{d_k}})V$$ (5)

where Q, K, V are given by: $Q=W_Q^{(i)}{\xi }_i({\mathbf{z}}_t)$ , $K=W_K^{(i)}\mathbf{y}$ , $V=W_V^{(i)}\mathbf{y}$ , respectively. Here, ${\xi }_i({\mathbf{z}}_t)$ is the intermediate representation of the denoising network, ${ϵ}_{\theta }$ , and $W_Q^{(i)}\in {\mathbb{R}}^{d\times d_{ϵ}^i}$ , $W_K^{(i)}\in {\mathbb{R}}^{d\times d_{\tau }}$ , and $W_V^{(i)}\in {\mathbb{R}}^{d\times d_{\tau }}$ are the learnable weight matrices [22,23].

2.2.2. Image autoencoding in latent space

Traditional diffusion models are computationally expensive when calculated in pixel space. Diff-NIRST overcomes this limitation by compressing the input with an autoencoder module which consists of image encoding and decoding. Image encoding compresses image features, accomplished by encoder, $\mathcal{E}$ , through which NIRST image $c\in {\mathbb{P}}^{H\times W\times 3}$ (with dimensions H, W in the pixel space and three channels represent HbO, Hb, and water) is encoded into latent representation $\mathbf{z}\in {\mathbb{P}}^{h\times w\times l}$ (with dimensions h, w, l in the latent space) expressed as:

$${\mathbf{z}}_0=\mathcal{E}(c)$$ (6)

The encoder is composed of multiple encoding blocks, each of which contains a convolutional layer, followed by a BatchNorm layer, an ELU layer and a Max-pooling layer. For image decoding, decoder, $\mathcal{D}$ , with structural symmetry to the encoder is applied, but with the max-pooling layer replaced with a deconvolution layer. The decoder maps $\hat{\mathbf{z}}$ in latent space to $\hat{c}$ in pixel space written as:

$$\hat{c}=\mathcal{D}(\hat{\mathbf{z}})$$ (7)

To minimize information loss during the encoding and decoding processes, a loss function that combines perceptual loss [24] and adversarial training [25] is adopted. Each encoding block reduces image dimensions by a factor of 2, while each decoding block increases them by the same factor. The detailed structures of these blocks are presented in Fig. 2.

Fig. 2.

Structure of (a) encoding and (b) decoding blocks.

2.2.3. Conditional signal encoder

NIRST signals were mapped to a denoising U-net in the latent space network to guide noise prediction at each step, t. The mapping was implemented in a conditional encoder, g_θ, in which NIRST signals were first preconditioned. NIRST signal dimensions were adjusted to pixel space through a trained linear block, f, which consisted of two linear layers and an ELU layer. After preconditioning, the condition, y, for guiding NIRST image generation was obtained through g_θ as:

$$\mathit{y}=g_{\theta }\left(f\left(\begin{array}{c}{\mathrm{\Phi }}_1^M\\ ⋮\\ {\mathrm{\Phi }}_{\lambda }^M\\ ⋮\\ {\mathrm{\Phi }}_n^M\end{array}\right)\right),\lambda =1,\dots ,n$$ (8)

where ${\mathrm{\Phi }}_{\lambda }^M$ are the measurements at wavelength λ, and g_θ is an autoencoder with fully dense blocks [19], which consists of four downsampling layers. As shown in Fig. 3(a), each downsampling layer was composed of a dense block, a normalization layer, and a SiLU activation layer. This fully dense autoencoder was optimized using AdamW [26], and mean square error (MSE) was used to calculate reconstruction loss.

Fig. 3.

Structures of (a) conditional signal encoder, g_θ, (b) four-layered dense block, and (c) dense layer.

As depicted in Figs. 3(b) and 3(c), within the dense block, the output of the early convolutional layer was connected to each subsequent convolutional layer via multi-layer short distance skip connections. Each layer contributed additional feature maps to the output, and a 1 × 1 convolutional layer guaranteed computational efficiency. This short distance skip connection is essentially a residual learning method, which has been proven to be effective in restoring artifact-free images. Empirically, four-layer dense blocks were determined to achieve an optimal balance between reconstruction quality and computational efficiency. Output was a concatenation of the inputs and outputs of all convolutional layers within this dense block.

2.3. Performance comparison of Diff-NIRST with other approaches

To evaluate the proposed method, reconstruction results of Diff-NIRST were compared with those obtained with Tikhonov regularization, Model-CNN [10], and FD-Unet [19], as a representative triplet of traditional and newer deep-learning approaches. Assessments of image quality were quantified in terms of peak signal-to-noise ratio (PSNR) [27], structural similarity (SSIM) [28], and contrast ratio (CR) between background and inclusion. PSNR measures differences between ground truth and reconstructed images; higher values indicate better fidelity. SSIM evaluates structural similarity and is more closely aligned with human visual perception; higher values indicate better perceived quality. CR quantifies imaged property differences between inclusion and background regions; higher values indicate clearer inclusion identification.

Additionally, size error (SE) was used to quantify relative size differences between reconstructed and ground truth inclusions [29]. Reconstructed inclusion size was calculated as the full width at 80% of the maximum property value estimated within the inclusion region; lower SE values indicate more accurate recovery of inclusion size. Number of inference steps (NIS) [30] was used to assess trade-offs between computational time and reconstruction quality.

2.4. Dataset for training, validation, and testing

Figure 4 illustrates the simulated phantom configurations used to produce training data, which were similar to those used in patient imaging studies [31]. Synthetic data were generated based on a homogeneous circular phantom with a diameter of 80 mm. Concentrations of HbO, Hb and Water were assigned to be c_HbO = 10 μmol, c_Hb = 10 μmol and c_Water = 40%, respectively. One or two inclusions, with diameters ranging from 2 mm to 15 mm, were randomly placed inside each phantom. Concentrations of HbO and Hb were varied from 20 to 40 μmol, while Water percentage ranged from 45% to 65%. In total, 11100 simulated phantoms were created. Each phantom was discretized into a finite element mesh containing 1785 nodes and 3418 triangle elements. Sixteen (16) source/detector pairs were uniformly placed around the boundary of each phantom. During these computational experiments, one source was activated while all other sources functioned as detectors, yielding 240 (16 × 15) synthetic measurements for each wavelength.

Fig. 4.

Simulated phantom configurations. Each simulated circular phantom (80 mm in diameter) contained one or two inclusions with randomly assigned positions, diameters, and chromophore concentrations. Sixteen source/detector pairs were assumed to be spaced evenly around the phantom circumference. NIRFAST calculated the resulting multi-wavelength signals at each detector for each source illumination covering six wavelengths for each phantom/inclusion geometry/property configuration.

Open-source software, NIRFAST [12], was used to generate simulated data through solving continuous diffusion equation coupled with Robin boundary condition for each phantom at wavelengths of 660, 750, 780, 810, 830, and 850 nm, respectively. Consequently, a total of 1440 data points (6 wavelengths ×16 sources×15 detectors) were computed in each case. The 11100 simulated phantom measurements were randomly divided into 10000 for training, 1000 for validation, and 100 for testing. Five (5%) Gaussian noise was added to the simulated measurements for each training and validation data, while different levels of noise (5%-20%) were added to testing data.

The grid size used in the diffusion model was 128 × 128. Maximum time step, T, of the diffusion model was set to 1000, and noise parameter, β, was linearly interpolated from β₁ = 0.0001 to β_T = 0.02. Both image encoding module and conditional signal encoder were optimized using the AdamW optimizer with α₁ = 0.9 and α₂ = 0.999. Initial training parameters were set as learn rate=0.0001, number of epochs=200, and Batchsize was set to 64. Learning rate was dynamically adjusted using the CosineAnnealingLR mechanism with a decay factor of 0.5.

The denoising U-net ${\mathrm{ϵ}}_{\theta }$ was also optimized by the Adam optimizer with α₁ = 0.9 and α₂ = 0.999. Initial learning rate, number of epochs and batch size were set to 0.0001, 500 and 16, respectively. All DL models were trained on a computer equipped with an Intel Xeon Gold 5218 CPU, 64 GB of RAM, and two NVIDIA RTX 3090 GPUs, each having 24 GB VRAM. Tikhonov regularization was implemented using NIRFAST. Its initial regularization parameter was set to 10, and its maximum number of iterations was 40.

3. Results

3.1. Images of the simulation phantoms

Among the 100 testing data samples, 68 were phantoms with a single inclusion and 32 were phantoms with two inclusions. For the single-inclusion cases, average inclusion diameter was 8.9 ± 4.2 mm. In the two-inclusion cases, average diameters were 10.8 ± 3.4 and 7.01 ± 2.51 mm for the larger and smaller inclusions, respectively. Additionally, average total hemoglobin (HbT, the sum of HbO and Hb) concentrations and water percentages were 55.4 ± 5.2 μmol and 54.5 ± 7.3%, respectively, corresponding to average CRs of 2.77 ± 0.26 for HbT and 1.39 ± 0.18 for water. Gaussian noise varied from 5% to 20% in the synthetic measurements generated from the simulated numerical circular phantoms with one or two inclusions. Representative images (two inclusions only) of HbT and water reconstructed by different algorithms are shown in Fig. 5. In the case shown, the larger inclusion had a diameter of 12 mm with an HbT concentration of 59 μmol (CR = 2.95) and water percentage of 48.5% (CR = 1.21) whereas the smaller inclusion has a diameter of 9 mm with an HbT concentration of 57 μmol (CR = 2.85) and water percentage of 45.9% (CR = 1.15). As shown in Fig. 5(b), at 5% measurement noise level, Tikhonov regularization produced the poorest performance, yielding images with the lowest PSNR (26.4 dB) and CR (1.28), and highest SE (0.93) relative to ground truth (Fig. 5(a)). For Model-CNN (Fig. 5(c)), accurate inclusion locations were recovered in HbT images with less accuracy in inclusion sizes and concentrations and corresponding water images. FD-Unet (Fig. 5(d)) recovered inclusions with smaller sizes and lower HbT concentrations and water percentages relative to ground truth. As shown in Fig. 5(e), Diff-NIRST yielded the best overall performance. It achieved more accurate estimates of inclusion concentrations and size with significantly reduced image artifacts. Even at 20% measurement noise, Diff-NIRST improved PSNR and CR for HbT from baseline values of 24.7 dB and 1.19 (Tikhonov), 26.4 dB and 1.97 (Model-CNN), and 27.5 dB and 1.48 (FD-Unet) to 32.8 dB and 2.65, respectively. Additionally, Diff-NIRST achieved accurate recovery of inclusion size with SE of 6% and 9% for HbT and water images, respectively.

Fig. 5.

Representative reconstructed images of two inclusion simulated phantoms with different algorithms under synthetic measurement noise levels of 5%, 10%, and 20%. Larger inclusion had a diameter of 12 mm, with HbT concentration of 59 μmol (CR = 2.95) and content of 48.5% (CR = 1.21); the smaller inclusion has a diameter of 9 mm, with an HbT concentration of 57 μmol (CR = 2.85) and water content of 45.9% (CR = 1.15). Top three rows show HbT images, while bottom three rows present water images. (a) ground truth images, images reconstructed by (b) Tikhonov regularization, (c) Model-CNN, (d) FD-Unet, and (e) Diff-NIRST. Units for HbT and water are µmol and percentage (%), respectively.

Corresponding statistical results, derived from 100 testing phantoms, are summarized in Table 1. As shown in Table 1, image quality of reconstructed HbT and water images deteriorated as noise levels increased from 5% to 20%, regardless of the algorithms applied. Average PSNR of HbT for Tikhonov regularization decreased from 25.1 ± 1.4 dB (at 5%) to 24.3 ± 1.3 dB (at 10%) and further declined to 23.3 ± 1.3 dB (at 20%). For Model-CNN, as noise level increased to 20%, artifacts became more prominent in the background of reconstructed images, and average PSNR of HbT decreased by 7.7%. For FD-Unet, performance also deteriorated with increasing noise, as indicated by PSNR dropping from 27 dB to 25.1 dB when noise levels increased from 5% to 20%. In contrast, Diff-NIRST maintained relatively stable performance with PSNR of HbT of 31.1 ± 2 dB, and SE of 0.13 ± 0.1 at 20% noise. In terms of CR and SE, Diff-NIRST reconstructed inclusions with higher contrast ratio and lower size error. For example, at 20% noise, Diff-NIRST reduced HbT contrast error and SE by 74.6% and 93.1% compared to Tikhonov, 59.6% and 35% compared to Model-CNN, 73.7% and 40.9% compared to FD-Unet, respectively.

Table 1. Statistical Results of Image-Quality Assessments from 100 Simulated Test Phantoms. Reconstructed Images Were Obtained from Four Algorithms: Traditional Tikhonov, Model-CNN, FD-UNet, and Diff-NIRST (NIS = 20).

Noise	Method	HbT				Water
		PSNR (dB)	SSIM	CR error (%)	SE (%)	PSNR (dB)	SSIM	CR error (%)	SE (%)
5%	Tikhonov	25.1 ± 1.4	0.40 ± 0.08	56 ± 5	113 ± 84	24.5 ± 2.3	0.28 ± 0.05	12 ± 5	147 ± 121
	Model-CNN	27.3 ± 3.3	0.90 ± 0.05	30 ± 10	17 ± 12	33.7 ± 3.3	0.85 ± 0.06	6 ± 4	28 ± 14
	FD-Unet	27.0 ± 0.9	0.55 ± 0.10	54 ± 5	19 ± 21	30.7 ± 2.6	0.51 ± 0.10	8 ± 5	21 ± 16
	Diff-NIRST	33.7 ± 1.9	0.92 ± 0.02	13 ± 7	13 ± 12	37.4 ± 3.6	0.85 ± 0.04	5 ± 3	9 ± 8
10%	Tikhonov	24.3 ± 1.3	0.18 ± 0.07	59 ± 4	176 ± 115	23.9 ± 2.0	0.10 ± 0.05	16 ± 5	211 ± 119
	Model-CNN	26.9 ± 3.2	0.89 ± 0.05	29 ± 10	17 ± 12	33.5 ± 3.2	0.85 ± 0.06	8 ± 5	28 ± 14
	FD-Unet	26.1 ± 1.0	0.51 ± 0.11	54 ± 6	19 ± 12	29.2 ± 3.0	0.49 ± 0.10	9 ± 4	34 ± 11
	Diff-NIRST	31.5 ± 2.4	0.91 ± 0.03	15 ± 8	13 ± 9	37.3 ± 3.0	0.84 ± 0.05	5 ± 3	11 ± 7
20%	Tikhonov	23.3 ± 1.3	0.15 ± 0.06	59 ± 4	188 ± 131	23.8 ± 1.9	0.09 ± 0.03	20 ± 8	229 ± 127
	Model-CNN	25.2 ± 2.6	0.87 ± 0.04	37 ± 8	20 ± 14	32.7 ± 3.0	0.83 ± 0.06	8 ± 5	28 ± 17
	FD-Unet	25.1 ± 1.1	0.46 ± 0.11	57 ± 5	22 ± 21	26.2 ± 2.2	0.49 ± 0.08	12 ± 3	34 ± 18
	Diff-NIRST	31.1 ± 2.0	0.89 ± 0.03	15 ± 8	13 ± 10	36.8 ± 3.4	0.83 ± 0.06	5 ± 4	12 ± 5

3.2. Optimizing reconstruction parameters

3.2.1. Number of inference steps (NIS)

When image size and downsampling factor are fixed, time required to generate the diffusion models depends primarily on the number of inference steps (NIS). While increasing NIS generally enhances image quality, processing time also increases, and improvements in image quality diminish as NIS exceeds a certain threshold. To determine an NIS that balanced reconstructed image quality and computational time, PSNR and SSIM were evaluated for a series of NIS values. Using the PSNR and SSIM at NIS = 1000 as a baseline, and applying a predefined tolerance threshold of 3%, we calculated percentage changes in PSNR, SSIM, and processing time.

Table 2 presents statistical results for 10% and 20% added noise based on the test dataset, with the downsampling factor fixed at 1. As shown, when NIS = 20, PSNR of reconstructed HbT and water images decreased by only 0.32% and 0.79%, respectively, while CR and SE were unchanged. Importantly, these values were still substantially higher than those obtained with Tikhonov regularization or the two other DL-based reconstruction algorithms. Moreover, the processing time at NIS = 20 was less than 1.5% of that for NIS = 1000 and only a fraction of a second. In contrast, Tikhonov and Model-CNN required substantially longer computational times of 41.54 ± 2.36 seconds and 22.41 ± 3.11 seconds, respectively.

Table 2. Statistical Comparisons for Diff-NIRST (With Varied NIS) in Terms of Computation Time and Image Quality Under 10% and 20% Noise, based on 100 Simulated Test Phantoms with the Downsampling Factor Fixed at 1.

Noise	NIS	Time (s)	HbT				Water
			PSNR  (dB)	SSIM	CR error (%)	SE (%)	PSNR (dB)	SSIM	CR error (%)	SE (%)
10%	1	0.14 ± 0.03	17.4 ± 1.6	0.41 ± 0.03	73 ± 61	81 ± 70	19.4 ± 5.2	0.29 ± 0.08	66 ± 41	88 ± 82
	20	2.71 ± 0.22	31.6 ± 2.2	0.91 ± 0.03	15 ± 7	13 ± 6	37.6 ± 2.9	0.85 ± 0.05	5 ± 3	11 ± 6
	100	20.66 ± 0.38	31.7 ± 2.1	0.91 ± 0.02	15 ± 6	13 ± 9	37.9 ± 2.2	0.85 ± 0.05	5 ± 3	11 ± 7
	1000	211.41 ± 0.81	31.7 ± 2.1	0.91 ± 0.03	15 ± 6	13 ± 8	37.9 ± 2.2	0.85 ± 0.06	5 ± 3	11 ± 6
20%	1	0.14 ± 0.02	15.4 ± 3.2	0.37 ± 0.07	75 ± 61	88 ± 71	17.3 ± 3.1	0.27 ± 0.11	66 ± 48	91 ± 86
	20	2.70 ± 0.23	31.2 ± 2.0	0.90 ± 0.02	15 ± 7	13 ± 6	35.0 ± 2.1	0.83 ± 0.05	5 ± 4	12 ± 6
	100	20.65 ± 0.41	31.2 ± 2.0	0.90 ± 0.03	15 ± 7	13 ± 9	35.1 ± 2.3	0.83 ± 0.06	5 ± 4	12 ± 5
	1000	211.33 ± 0.87	31.3 ± 1.7	0.90 ± 0.02	15 ± 7	13 ± 9	35.1 ± 2.2	0.83 ± 0.07	5 ± 4	12 ± 5

3.2.2. Effect of downsampling factor, r

To reduce the image reconstruction time of Diff-NIRST further, we evaluated the impact of downsampling (r = 2^m, with m ∈ N) in the latent-space diffusion models. This parameter affects image quality and computational speed simultaneously. To identify a value that maintained reconstruction performance without significantly compromising efficiency, we assessed PSNR and SSIM across different downsampling factors using the 100 simulated test phantoms. Here, NIS = 20, the value found to improve computational efficiency without compromising image integrity in the previous subsection. Quantitative comparisons are summarized in Table 3. Increasing the downsampling factor to 4 significantly reduced reconstruction time while maintaining the same image quality. Compared to the baseline case (r = 1, i.e. no downsampling), r = 4 reduced processing time from about 2.70 ± 0.23 seconds to approximately 0.31 ± 0.13 seconds. Despite this significant image reconstruction time reduction, PSNR and SSIM values for both HbT and water images decreased less than 1%, while CR and SE remained the same. Notably, the model failed to reconstruct NIRST images when the downsampling rate reached r = 8 or higher, due to excessive information loss preventing successful optical image reconstruction.

Table 3. Quantitative Results for Diff-NIRST Using Different Downsampling Factors (r) With a Fixed Number of Inference Steps (NIS = 20) Based on 100 Simulated Test Phantoms. w/o Represents That Diff-NIRST Was Conducted by Removing the Image Autoencoding Module.

Noise	r	Reconstruction Time (s)	HbT				Water
			PSNR (dB)	SSIM	CR error (%)	SE (%)	PSNR (dB)	SSIM	CR error (%)	SE (%)
10%	1	2.71 ± 0.22	31.6 ± 2.2	0.91 ± 0.03	15 ± 7	13 ± 6	37.6 ± 2.9	0.85 ± 0.05	5 ± 3	11 ± 6
	2	1.12 ± 0.10	31.5 ± 2.0	0.91 ± 0.03	15 ± 7	13 ± 6	37.6 ± 2.9	0.85 ± 0.05	5 ± 3	11 ± 6
	4	0.31 ± 0.14	31.5 ± 1.9	0.91 ± 0.03	15 ± 8	13 ± 9	37.3 ± 3.0	0.84 ± 0.05	5 ± 3	11 ± 7
	w/o	2.70 ± 0.21	31.7 ± 2.3	0.92 ± 0.02	15 ± 7	13 ± 6	37.8 ± 2.4	0.86 ± 0.03	5 ± 2	11 ± 6
20%	1	2.70 ± 0.23	31.2 ± 2.0	0.90 ± 0.02	15 ± 7	13 ± 6	35.0 ± 2.1	0.83 ± 0.05	5 ± 4	12 ± 6
	2	1.13 ± 0.12	31.2 ± 1.9	0.89 ± 0.03	15 ± 7	13 ± 9	35.0 ± 2.1	0.83 ± 0.05	5 ± 4	12 ± 4
	4	0.31 ± 0.13	31.1 ± 2.0	0.89 ± 0.03	15 ± 8	13 ± 10	34.8 ± 3.4	0.83 ± 0.06	5 ± 4	12 ± 5
	w/o	2.70 ± 0.21	31.2 ± 2.2	0.90 ± 0.01	15 ± 7	13 ± 6	35.3 ± 2.3	0.84 ± 0.06	5 ± 3	12 ± 5

Furthermore, the effect of the image autoencoding module on reconstruction quality was analyzed. To this end, the image autoencoding module, including image compression encoder and image restoration decoder, was removed so that the diffusion model operated directly in pixel space. The corresponding results are also presented in Table 3. Compared with Diff-NIRST without the image autoencoding module, the reconstruction quality at a downsampling factor of 4 in the latent space shows only a slight degradation. For example, at a 20% noise level, the PSNR decreases by approximately 0.63% and 1.3% for HbT and H₂O respectively, while the SSIM decreases by approximately 1.1% and 2.3%. Overall, the loss in image quality is less than 2.5%. In contrast, the computational time is significantly reduced from 2.70 ± 0.21 seconds to 0.31 ± 0.14 seconds. Therefore, the image autoencoding module is adapted in Diff-NIRST to achieve a favorable trade-off between reconstruction accuracy and computational efficiency.

3.2.3. Patient study

To investigate performance of Diff-NIRST with experimental data (beyond the synthetic data with simulated noise in the prior sections), a clinical case study involving a breast cancer patient was evaluated. The imaging procedure complied with the Health Insurance Portability and Accountability Act (HIPAA) and was approved by the Institutional Review Board at Dartmouth College (USA). The subject was a 30-year-old woman diagnosed with a 2 cm tumor in her left breast. NIRST data were acquired using our custom NIRST system while the patient was positioned prone [31]. A circular fiber array consisting of 16 fiber bundles arranged in a single plane collected NIRST measurements at five wavelengths: 661, 761, 785, 808, and 826 nm. 240 measurements were acquired at each wavelength.

Figure 6 presents reconstructed images obtained from Tikhonov regularization, Model-CNN, FD-Unet and Diff-NIRST. FD-Unet (Fig. 6(c)) failed to recover HbT and water images. Compared to the images reconstructed by Tikhonov regularization and Model-CNN, Diff-NIRST reduced artifacts. Contrast values of HbT and water between tumor and surrounding normal tissue were 2.30 (Diff-NIRST), 1.08 (Tikhonov), and 1.61 (Model-CNN) for HbT; and 2.37 (Diff-NIRST), 1.09 (Tikhonov), and 1.32 (Model-CNN) for water, respectively. The highest contrast achieved by Diff-NIRST was indicative of malignancy – a finding later confirmed through post-surgical pathological analysis.

Fig. 6.

Reconstructed HbT and water images of a patient with breast cancer obtained by (a) Tikhonov, (b) Model-CNN, (c) FD-Unet and (d) Diff-NIRST.

4. Discussion

In this study, we developed a deep learning inspired diffusion model-based reconstruction algorithm for NIRST. Motivation stemmed from the fact that traditional regularization techniques often introduce artifacts into reconstructed images while end-to-end DL algorithms suffer from poor generalization, especially when applied to data that differ from the training set. Moreover, both approaches are susceptible to noise due to the inherent ill-posedness of NIRST reconstruction. As shown in Figs. 5 and 6, Diff-NIRST demonstrated robustness to noise, consistently delivering NIRST images with precise inclusion localization, clear boundaries, and high reconstruction fidelity. Moreover, results shown in Table 1 indicated Diff-NIRST effectively reduced image artifacts and improved both PSNR and SSIM. Diff-NIRST was based on a conditional latent diffusion model. To enhance its performance, a cross-attention mechanism was integrated into the denoising U-Net, enabling the model to leverage multi-wavelength signals, suppress noise, and improve the fidelity of NIRST image reconstruction. Effectiveness of the approach was validated in a sizeable series of simulated phantom testing data where it consistently outperformed baseline methods, and its potential to achieve similar performance gains with experimental measurements acquired during a clinical patient exam appears promising.

In addition, to explore Diff-NIRST computational efficiency, we conducted experiments on NIS and latent space diffusion model downsampling. As shown in Table 2, Diff-NIRS outperformed baseline methods for NIS = 20. While increasing NIS to 1000 brought marginal gains—e.g., an increase of ∼ 0.1 dB in PSNR for HbT and nearly unchanged SSIM—the inference time grew by over 40-fold. The diminishing performance returns with higher NIS and highlights that NIS = 20 offered a good trade-off between performance and efficiency. Image reconstruction was also performed in a lower-dimensional latent space instead of the higher-dimensional pixel space. With a downsampling factor of 4, computation time decreased from 2.71 seconds to 0.31 seconds—an approximate 9-fold reduction – an improvement likely to be essential for 3D reconstruction tasks.

To investigate the clinical potential of Diff-NIRST, we conducted an in vivo imaging study on a breast cancer patient. The reconstructed images obtained using Diff-NIRST were compared with those from other DL-based algorithms. Notably, performance degradation with experimental data was observed when DL models were trained only on simulated data. For example, the end-to-end FD-Unet exhibited the poorest generalization performance in the patient case, likely due to a domain gap between simulated and experimental measurements. Despite efforts to closely mimic experimental conditions when generating synthetic data, differences persist between simulated and acquired NIRST data, particularly in measurement error/noise and tissue heterogeneity, among other factors. In contrast, Model-CNN and Diff-NIRST, which incorporate light propagation and/or diffusion models, demonstrated greater robustness when applied to unseen data and yielded images with fewer artifacts and enhanced tumor-to-background contrast. Diff-NIRST outperformed Model-CNN in terms of generalization, likely because of its probabilistic sampling and denoising mechanisms that offer improved robustness to domain shifts inherent in experimental NIRST data. Remarkably, Diff-NIRST reconstructed patient images efficiently despite being trained solely on simulated data, without any fine-tuning or domain adaptation. These findings aligned with the final pathological diagnosis of malignancy, supporting the potential of Diff-NIRST as a promising tool for clinical optical imaging.

Although this study involved only one clinical patient case, the single-patient experiment serves as a preliminary proof-of-concept, demonstrating that Diff-NIRST can be successfully applied to real clinical measurements and can reconstruct meaningful chromophore distributions. In addition, Diff-NIRST was extensively evaluated on a large number of simulated datasets with different noise levels and chromophore concentrations, which helps validate its robustness and stability. A larger cohort study is necessary to further establish the clinical reliability and generalizability of Diff-NIRST, and such validation will be pursued in future work.

As shown in Table 1, CRs of HbT and water images recovered by all four algorithms were underestimated relative to their ground truths, regardless of noise level, likely due to the ill-posed nature of NIRST image reconstruction and measurement noise. Compared to the other three approaches, Diff-NIRST achieved lower CR errors and SEs for HbT, within ranges of 15% ± 8% and 13% ± 10%, respectively, compared to 30% ± 10% and 17% ± 10% for the other methods. Moreover, while average CRs of images obtained with the other approaches increased markedly as noise levels rose from 5% to 20%, CRs of images reconstructed with Diff-NIRST remained stable, demonstrating greater robustness to noise.

Furthermore, recovering small inclusions with diffuse optical imaging is inherently challenging, as CR typically decreases with inclusion size [32–35]. In this study, we successfully recovered inclusions with an average size less than 11 mm. To enhance the capability of Diff-NIRST further to detect even smaller inclusions with higher CR, structural data from other clinical imaging modalities (e.g., MRI) may be incorporated into the diffusion model.

The training process for Diff-NIRST remains a significant challenge. As shown by the blue loss curve in Fig. 7, Diff-NIRST optimization failed to converge when trained in an end-to-end manner, with the loss oscillating around a mean value of 3 × 10⁻⁴ with an amplitude of 2.1 × 10⁻⁴. We attribute this non-convergence to a mismatch between the reconstruction objective in the pixel space and the image synthesis objective in the latent space. Specifically, the image autoencoder subnetwork focuses on image compression and reconstruction fidelity, whereas the conditional latent diffusion subnetwork is designed to generate images in the latent space. This inconsistency leads to convergence instability, especially when NIRST measurements are noisy and the training dataset is limited.

Fig. 7.

Training loss as a function of optimization steps for the end-to-end and multi-stage training strategies, respectively.

To address this issue, a multi-stage training strategy was adopted. First, the image autoencoding subnetwork is trained independently to learn a compact and stable latent representation. Next, the conditional signal encoding subnetwork is trained to transform NIRST measurements into conditioning inputs for the diffusion model. After these two subnetworks are well trained, the diffusion model is optimized in the latent space by incorporating conditioning inputs. The red loss curve in Fig. 7 shows that, under this multi-stage training strategy, Diff-NIRST converges successfully, stabilizing around 3.7 × 10⁻⁶ with a fluctuation amplitude less than 3%, in this multi-stage training strategy. In future work, more effective training strategies, such as warm-up schedules [36,37], will be explored to further mitigate gradient instability and reduce conflicts across different network modules.

The use of 2D phantoms in this work serves as a proof-of-concept and methodological validation, providing a computationally efficient environment for validating the proposed Diff-NIRST and systematically analyzing its performance under different noise levels. Extending Diff-NIRST from 2D to 3D is essential for clinical applicability, as 3D modeling of photon migration and tissue characterization in clinical imaging system is more accurate than those in 2D implementation.

However, Diff-NIRST involves iterative denoising sampling, and extending the algorithm directly to 3D would require significantly higher computational cost and memory consumption. In conventional pixel-space diffusion models, computational cost increases rapidly with image dimensionality because the diffusion process operates directly on high-resolution volumetric data. For example, a typical 2D reconstruction of 128 × 128 contains approximately 1.6 × 10⁴ pixels, whereas a moderate 3D volume of 128 × 128 × 64 contains more than 1.0 × 10⁶ voxels, leading to significantly increased memory consumption and sampling time. In contrast, the proposed latent space diffusion model compresses images into compact representations through the image autoencoding module before iterative denoising sampling. When extended to 3D, the autoencoder can reduce volumetric data (e.g., 128 × 128 × 64) to a much smaller latent volume (e.g., 32 × 32 × 16 or lower), reducing computational burden by more than an order of magnitude. Since the diffusion process operates in the latent space, both memory usage and sampling complexity scale much more favorably with 3D data volumes.

Importantly, Diff-NIRST provides a general reconstruction framework that is not inherently limited to 2D scenarios and is compatible with 3D implementation. The three components presented in this study including the conditional latent diffusion mode, image autoencoding subnetwork and the conditional signal encoding subnetwork can be extended to 3D NIRST reconstruction by replacing them with their 3D counterparts. Specifically, the encoder-decoder in image autoencoding subnetwork can be replaced with 3D convolutional autoencoders, the denoising U-net can be extended to a 3D U-Net operating in latent space, and the dense blocks in conditional signal encoding subnetwork can be adapted to 3D convolutional operators. Although the computational cost of 3D Diff-NIRST remains higher than that of the 2D implementation, the latent space approach significantly alleviates scalability challenges and makes 3D implementation feasible. In future work, we plan to extend Diff-NIRST to 3D breast NIRST reconstruction to recover 3D chromophore distributions from clinical NIRST measurements.

To reduce hallucinations in the diffusion model, the conditional signal encoder is designed to transform NIRST measurements into conditioning inputs for the generative process. By integrating these conditioning inputs into the generative process, diffusion model is constrained by maintaining consistency with acquired NIRST data. Therefore, unrealistic artifacts are suppressed and robustness to noise are improved, ultimately reducing hallucinations compared with unconstrained generative approaches.

5. Conclusion

In this paper, a novel NIRST image reconstruction method, Diff-NIRST, was developed based on a conditional latent diffusion model. To leverage the diffusion model framework effectively, we designed image encoding and decoding modules that reduce dimensionality of input images, enabling compatibility with the latent space. Additionally, a conditional signal encoding subnetwork was developed to transform NIRST signals into conditional inputs that guide the image generation process. Effectiveness of Diff-NIRST was validated through an extensive series of simulated phantom studies, and its potential application to clinical data was explored in a patient case. Quantitative results demonstrated that Diff-NIRST outperformed Tikhonov regularization and two DL algorithms in terms of reconstruction accuracy, image quality and recovered inclusion size.

Funding

Natural Science Foundation of Beijing Municipality 10.13039/501100005089 ( L258054, QY25364); National Natural Science Foundation of China 10.13039/501100001809 ( 82171992, 62105010); National Institutes of Health 10.13039/100000002 ( R01 CA176086, R01 EB027098).

Disclosures

The authors have no relevant financial interests in this work.

Data availability

Data underlying the results presented in this paper are not publicly available at this time, but may be obtained from the corresponding authors upon reasonable request.