Deep learning-enabled high-performance multiphoton fluorescence vascular imaging using clinically approved fluorescent probes

Summary

Among various modalities, multiphoton fluorescence imaging (MPFI) stands out for its exceptionally high spatial resolution in deep tissue imaging. Unfortunately, current clinically approved fluorescent probes are not engineered for MPFI, hindering the entry of MPFI into the clinical stage. Although several high-performance customized multiphoton probes have been developed, their biosafety has yet to be corroborated. To address this concern, we developed a deep learning-based method, trained on previously reported MPFI images enabled by aggregation-induced emissions luminogens nanoparticles, for high-performance MPFI using commercial q800 quantum dots and a clinically approved indocyanine green (ICG) probe. Remarkably, the proposed method demonstrated exceptional effectiveness when handling previously unseen data and showed strong optimization performance for MPFI images of cerebral microvasculature. Especially, blood vessels in the hippocampus region become clear and noise free after deep learning processing. Overall, this work offers a valuable strategy to greatly improve the practicality and applicability of MPFI.

Graphical abstract

Highlights

• •The model considers vascular autocorrelation and z axis continuity
• •The model improves visualization in high-noise deep brain regions
• •The model enhances the translational potential of multiphoton fluorescence imaging

Nanoparticles; Optical imaging; Biocomputational method

Introduction

Cerebral microvascular imaging provides detailed information about vascular structures and blood flow velocity,^1,2 which are closely related to a variety of brain conditions, such as cerebral infarction,³ intracerebral hemorrhage,⁴ and brain tumors.^5,6 Among various imaging modalities, multiphoton fluorescence imaging (MPFI) stands out for its exceptional spatial resolution (on the submicron scale) and deep-tissue penetration in living organisms due to its localized excitation effect⁷ and exceptionally long excitation wavelength.^8,9 Such features are extremely valuable in identifying cerebral microvasculature¹⁰ and deciphering the pathogenesis of brain conditions.¹¹ However, this technique has not been advanced clinically. One major problem is that current clinically approved fluorescent probes^12,13 are not engineered for MPFI and, therefore, potentially exhibit suboptimal MPFI performance. While significant progress has been made in material sciences over the past two decades,¹⁴ and various customized multiphoton probes with excellent imaging performance have been developed,^{15,16,17,18,19,20} extensive and time-consuming toxicological testing to ensure biosafety remains a daunting challenge for their clinical translation. Therefore, developing an image enhancement method could be a strategic approach to achieving high-performance MPFI suitable for clinical use,²¹ even with Food and Drug Administration (FDA)-approved commercial dyes that have less-than-optimal optical properties.^22,23,24

Although various algorithms, such as Gaussian filtering, low-pass filtering, wavelet transform denoising,²⁵ and adaptive thresholding based on local variance denoising (ATLV), have been proven to be effective in improving imaging quality, they often require prior knowledge and significant time costs.^26,27 In recent years, with the emergence of convolutional neural networks (CNNs), image enhancement algorithms have gradually shifted to deep learning methods, as they offer greater flexibility and accuracy. However, the effectiveness of these methods is highly dependent on the availability of high-quality target images generated by specific devices and probes.²⁸ Recent advancements in AI have sparked significant interest in deep learning-based image segmentation techniques within the research community, particularly in biomedical imaging.^29,30,31 A notable example is the training of CNN segmentation models using retinal vessel annotations from the Digital Retinal Images for Vessel Extraction dataset, which successfully extracted vascular information from previously unseen images.³² It is thus reasonable to propose that applying deep learning segmentation techniques to extract cerebral microvasculature from raw MPFI images is technically feasible and not limited by the imaging devices and contrast agents used.

In this contribution, a deep learning-based enhancement method was successfully developed and applied to improve the quality of multiphoton brain vascular images enabled by clinically approved probes (Scheme 1). The original imaging data, which formed the training set, originates from our previously reported results using aggregation-induced emission luminogens (AIEgens) nanoparticles (NPs).⁸ Then, we utilized a paired dataset consisting of original images and their corresponding mask images to train the model for vascular semantic segmentation. After training, the network successfully transformed blurry and noisy images into clear and high-quality images. Remarkably, the foreground-denoised segmented images achieved a significant signal-to-noise ratio (SNR), and the generated MPFI images exhibited vascular structures that closely matched the ground-truth images obtained using AIEgens NPs. Additionally, the neural networks showed remarkable effectiveness when handling previously unseen data and demonstrated strong optimization performance for vascular imaging using quantum dots and FDA-approved probe. This work provides a promising method to significantly enhance the practicality and application potential of MPFI, marking substantial progress toward basic medical research and clinical translation.

Scheme 1.

Schematic illustration of the deep learning-based denoising method for MPFI images

Results and discussion

Overview of the deep learning method

This work aims to refine the quality of cerebral microvascular MPFI images based on deep learning image segmentation techniques to eliminate scattering noise and a post-process algorithm to remove the foreground noise. The logic and overall workflow of the proposed deep learning denoising method are presented in Figure 1.

Figure 1.

The overview of the proposed deep learning method

(A) Training methods and procedures for vascular segmentation.

(B) Post-processing algorithm for foreground noise removal. The color bar represents the raw grayscale intensity (0–255) of the 3D reconstruction imaging.

In brief, mask images are used to construct a paired dataset required for supervised learning in deep learning segmentation models. The vascular locations were annotated using LabelMe (Figure S1). In deep learning-based segmentation tasks, various issues within the dataset, such as sample imbalance, edge blurring, and difficult-to-segment regions, significantly affect the performance of the trained model (Figure S2). Different deep learning models and loss functions address these issues with varying focuses. To determine the optimal solution to improve the quality of MPFI images, we implemented a two-stage training strategy involving six deep learning models and two loss functions (Figure 1A). The segmented vascular image (Figure S3) can be obtained by performing a multiplication operation between the vessel mask images predicted by the segmentation model and the original images to remove the background noise completely. Then, we applied a two-step post-processing algorithm to the segmented vascular images for foreground denoising (Figure 1B). It should be noted that the noise levels are extremely unbalanced across the imaging dataset owing to the effect of depth on photon scattering. Thus, to preserve the original vascular information as much as possible, wavelet transform is used for noise assessment. The segmented images are classified based on the noise levels. Only the identified noisy images undergo denoising. This approach minimizes potential damage to fine vascular structures that can occur during the denoising process.

Finally, the deep learning-based segmentation model, which is trained using the approach mentioned above, when combined with post-processing algorithms, can effectively denoise MPFI images. This denoising performance remains effective both below 1,000 μm imaging depth (Figure S4), with relatively low noise levels, and beyond 1,000 μm (Figure S5), where noise interference becomes substantially more pronounced.

Deep learning training stage

The training dataset used in this study originated from our previously published papers.^8,16 Detailed descriptions of the construction of the training dataset can be found in the STAR Methods section. In the segmentation task, our team of specialists annotates MPFI vascular images based on their experience and prior knowledge. The label sets were repeatedly tested and refined through multiple preliminary experiments to ensure accuracy and high confidence. In brief, systematic validation of the label sets was conducted through iterative training cycles using a lightweight segmentation model. Specifically, the segmentation outcomes of each image in the training set were rigorously evaluated across multiple training epochs, and iterative relabeling of the identified outliers was performed with expert manual verification. This protocol ensured both topological consistency and statistical robustness of the annotated dataset.

To obtain a deep learning model with good performance, we conducted a detailed pre-analysis of the training set. In the labels, white pixels (representing vessels) account for only 7.29% of the entire image, while black pixels (background) make up 92.71%, indicating a severe sample imbalance. Additionally, by analyzing the percentage of high-frequency coefficients from wavelet transforms of images at different depths (used to represent the noise intensity), we observed that the noise intensity gradually increases after a depth of 1,500 μm (Figure S6). This rise in noise contributes to increased difficulty in vascular segmentation and results in edge-blurring issues of blood vessels. Therefore, a two-stage training strategy using Dice Loss³³ and Focal Loss³⁴ was employed. In the first stage, Dice Loss emphasizes the overall overlap of regions rather than the accuracy of individual pixel classification. It effectively handles class imbalance and boundary blurring, providing better performance in these areas and ensuring that the model achieves basic performance. As shown in Figure 2A, after 200 epochs of training, the dice loss value approaches a minimum and remains stable, indicating a good overall performance of segmentation. In the second stage, Focal Loss is employed for enhanced training, enabling good segmentation performance in the hard regions. According to Figure 2B, maximum dice is achieved in the test set at epoch 3. In this case, dice values increased from 0.7719 to 0.7863 and from 0.7088 to 0.7478 at two selected hard-segmented regions, suggesting improved segmentation performance (Figure 2C).

Figure 2.

Two rounds of training using two loss functions

(A) Preliminary training using Dice Loss and dice values for hard and easy regions.

(B) Enhanced training using Focal Loss and improvement of segmentation ability in the hard regions.

(C) Improvement in the segmentation of the hard region after enhanced training. Scale bars, 100 μm.

In detail, there were six widely used deep learning models for biomedical image processing, including FCN-Resnet,³⁵ DeepLabV3,³⁶ U-net,³⁷ ResU-net,³⁸ Attention ResU-net,³⁹ and NestedU-net⁴⁰ (Figure S7). They were trained using Dice Loss at first in the preliminary training and then evaluated on the testing set using three metrics, Time, IoU, and Dice.⁴¹ The latter two metrics are commonly used for segmentation evaluation to quantify spatial overlap accuracy and assess pixel-wise agreement between predictions and ground truth, providing robustness against class imbalance-induced metric bias. Computing time is an important metric because it accounts for the time complexity of models. A set of MPFI data typically contains hundreds of images. When processing these images for denoising and segmentation, low algorithmic time efficiency can impede practical application in both fundamental research and clinical translation. Herein, Attention ResU-net is recognized as the optimal model based on its highest composite performance score (Table 1). This model was modified by adding two self-attention layers between the down and upsampling layers of ResU-net. The first self-attention layer flattens the down-sampled data into a one-dimensional sequence, which is then fed into a transformer block, followed by recombination. The second self-attention layer takes the output from the first layer, uses the batch dimension as the sequence input to the transformer block, and then reassembles the output (Figure 3). Through the self-attention mechanism, the transformer can dynamically adjust the level of focus on different regions of the image. This flexibility enables the model to better identify important details and features, as well as better read the correlation of different positions of vascular imaging. Furthermore, a region of interest (ROI) at a vascular depth of 430 μm in the test set was used to analyze the effects of adding self-attention layers to the ResU-net. As shown in Figure S8, with the self-attention layer incorporated, the segmented vascular image more closely aligns with the ground truth.

Table 1.

Metrics for the six models after preliminary training

Model	Backbone	Time (s)	IoU	Dice	Score
FCN-Resnet	Resnet34	21.45	0.9180	0.8257	0.8351
DeepLabV3	Resnet50	45.50	0.9175	0.8293	0.8213
U-net	VGG16	22.76	0.9155	0.8133	0.1999
Nested U-net	VGG16	2,397.14	0.9164	0.8344	0.5440
ResU-net	Resnet34	26.31	0.9174	0.8258	0.7406
Attention ResU-net	Resnet34	28.49	0.9177	0.8323	0.9116

Attention ResU-net is the best-performing model overall.

Figure 3.

Schematic illustration of the structure of Attention ResU-net

(A) Overall structure of Attention ResU-net consisting of a self-attention layer and residual networks.

(B) Structure of two-part self-attention layers in Attention ResU-net.

To further evaluate deep learning models based on segmentation performance, we transformed the images generated by different models into the frequency domain using the fast Fourier Transform and quantified them as spatial frequency profiles (Figure S9A). This approach allows us to analyze the similarity between the segmented images and the ground truth. As shown in Figure S9B, the segmented images generated by ResU-net and Attention ResU-net exhibit the most similar profiles to those of the ground truth, suggesting the optimal deep learning model for the segmentation task.

Post-process to remove foreground noises

Foreground noises could significantly affect the image quality. The foreground noise level is extremely unbalanced across the imaging dataset due to the effect of depth on photon scattering. Therefore, to efficiently remove the foreground noises of MPFI results, the deep learning segmented images need to be classified based on the noise levels. In brief, the discrete wavelet transforms employing Haar wavelet basis decompose images into four coefficients: cA (Approximation), cH (Horizontal), cV (Vertical), and cD (Diagonal). The high-frequency coefficient proportions quantify the noise levels. Maximum between-class variance serves as a thresholding method to differentiate noisy and noise-free images. For example, the image at 1,125 μm was set as ROI to analyze the vascular signal distribution before and after vascular segmentation. As shown in Figure 4A, owing to the low level of foreground noise, the deep learning segmentation is good enough to generate a high-quality image with high accuracy. Additionally, the image at 1,660 μm was set as another ROI to be denoised. As shown in Figure 4B, although the deep learning segmentation can efficiently remove the background noises, the remaining foreground noises are non-negligible and detrimental to the image quality. Exceptionally, after post-processing, the foreground noise was evidently reduced, resulting in smoother signal distribution and good image quality. Such results indicated the necessity of removing foreground noises.

Figure 4.

Post-processing of MPFI images to remove the foreground noise

(A) Changes in vascular signals at the ROI before and after segmentation. Scale bars, 100 μm.

(B) Changes in vascular signals at the ROI before and after segmentation and denoising. Scale bars, 100 μm.

(C) Interactive supplementary video of vascular structure at depths ranging from 1,500 to 1,900 μm in three states: original images, segmented images, and post-process images. Scale bars, 100 μm.

(D) Changes in noise intensity distribution in the test set before and after segmentation and denoising.

Furthermore, since the signal quality always deteriorated significantly in the deep brain and the MPFI results were used to present in a three-dimensional form, to highlight the performance of our proposed deep learning-based denoising method, MPFI results between the depths of 1,500–1,900 μm were processed. As shown in Figure 4C, in the original image, blood vessels were embedded in severe background noise. After segmentation, the background noises were eliminated. Additionally, the post-processing refined the image by reducing the foreground noise. The noise level of the final image was assessed according to the proportion of high-frequency coefficients after the wavelet transform. As shown in Figure 4D, the noise level in the deep brain is significantly reduced from 0.0637 to 0.0190, suggesting the powerful denoising ability of our deep learning model.

It is important to examine the reproducibility of our models across different animals and experimental conditions. In this case, three individual mice were imaged using AIEgen NPs. All the imaging depths are close to 1,700 μm. As shown in Figure S10, the noise distribution of the raw images is presented in Figure S10, which shows relatively large and unstable noise beyond 1,250 μm depth, attributable to inter-animal variability. After processing the data with our proposed model, the noise distribution across all three individual datasets was significantly suppressed and exhibited markedly improved consistency (Figure S11). The mean noise values for the three batches were 6.385, 5.944, and 5.302, respectively. The inter-batch coefficient of variation (CV) was calculated to be 9.31%, which is well below the 20% threshold commonly accepted for high reproducibility in biological experiments. The low value of the CV suggests a high degree of inter-batch consistency, which demonstrates the robustness of our method.

Comparison of optimization effects with traditional denoising techniques

To highlight the advantages of our proposed method, we further compared it with two conventional denoising methods: wavelet and ATLV denoising. These two conventional methods are capable of adaptively determining thresholds based on the image information and removing noise while preserving a certain degree of detail and edge information.

By setting ROIs along the x-z axis of the vascular images between depths of 1,300–1900 μm, we analyzed the vascular signal distribution in the original image, image after wavelet denoising, image after ATLV denoising, and image after segmentation and post-processing (Figure 5A). The signal distribution in the image is much smoother after segmentation and post-processing, and non-vascular signals (background noise) have been more comprehensively eliminated compared to other denoising methods (Figure 5B). The SNR of these four types of images was further analyzed layer by layer to compare the optimization performance of four denoising methods. As shown in Figure 5C, wavelet denoising and ATLV denoising methods can only partially refine the image quality by suppressing the noise level. By contrast, our proposed deep learning method can effectively eliminate the background noise while reducing the foreground noise, thus significantly enhancing the image quality. Furthermore, the image SNR of the overall image sequence was examined to better compare the denoising performance among the three methods mentioned above. As can be seen in Figure 5D, vascular imaging after segmentation and denoising exhibits the highest SNR from an overall distribution perspective, confirming its excellent denoising ability.

Figure 5.

Comparison of optimization effects with traditional denoising techniques

(A) The ROI along the z axis after processing by four different denoising methods. Scale bars, 50 μm. The color bar represents the raw grayscale intensity (0–255) of these images.

(B) Signal changes in the ROI along the z axis after processing with four denoising methods.

(C) Planar optimization effects of four denoising methods.

(D) SNR distribution along the z axis after processing with four denoising methods.

Optimization of MPFI images enabled by commercial probes

To evaluate the generalization ability of the trained deep learning vascular segmentation model, we tested it with MPFI data enabled by different probes. It is worth mentioning that due to the lower contrast and higher noise levels of original images compared to the training set, inputting original images directly into the deep learning segmentation model was not ideal. To enhance the model’s segmentation capability for unseen vascular imaging data, we designed specific preprocessing steps to address these two issues. The preprocessing steps are summarized in the STAR Methods section.

Two commercial probes, quantum dot q800⁴² and indocyanine green (ICG),^43,44 were used for brain microvascular MPFI (Figure S12, Table 2). Consistent with the AIEgen probe training and testing sets, the images for both ICG and q800 also have dimensions of 512 pixels × 512 pixels and are stored in TIFF format. For the ICG probe dataset, a total of 314 images were acquired with an inter-slice spacing of 5 μm, yielding a total imaging depth of 1,570 μm. The ICG probe, with an absorption wavelength at 805 nm, was excited at a wavelength of 1,700 nm. Its two-photon absorption cross-section was determined to be 1.873 × 10⁻⁵¹ cm⁴s/photon¹. For the q800 probe dataset, a total of 314 images were acquired with an inter-slice spacing of 4 μm, resulting in a total depth of 1,256 μm. The q800 probe, with strong absorption below 800 nm, was excited at a wavelength of 2,200 nm, and its three-photon absorption cross-section was measured to be 1.68 × 10⁻⁷⁸ cm⁶s²/photon². The MPFI results enabled by ICG and q800 originated from our previously reported paper.^42,43 The photostability of q800 and ICG was characterized, and the results are demonstrated in Figure S13.

Table 2.

Optical properties of q800 and ICG

Probe	Absorption peak (nm)	Emission peak (nm)	Multiphoton action cross-section
q800	Below 800 nm	785 nm	1.68 × 10−78 cm6s2/photon2 at 2,200 nm
ICG	806 nm	830 nm	1.87 × 10−51 cm4s/photon1 at 1,700 nm

Due to the optical stability of quantum dots, three-photon imaging of cerebral microvasculature using q800 provides good imaging results in shallow layers (Figure 6A). However, in deep brain areas, the contrast decreases and the clarity diminishes. Although detail enhancement algorithms, e.g., Histogram Equalization, can enhance contrast, they amplify background noise along with vascular information at the same time. Building on these preprocessed data, applying the deep learning segmentation model and the post-process algorithm effectively diminishes the influence of background and foreground noise, resulting in high-contrast, noise-free vascular images of cerebral microvasculature using q800 as a probe (Figure S14). In particular, images of the deep brain became easily distinguishable, certifying the application potential of MPFI.

Figure 6.

The deep learning-based method denoises unseen MPFI images enabled by commercial and clinically approved probes

(A) The optimization workflow for three-photon brain microvascular imaging using the q800 probe.

(B) The optimization workflow for two-photon brain microvascular imaging using the ICG probe.

(C) 3D reconstruction of imaging using the ICG probe in the hippocampal region, signal contrast in hippocampal vessels, and the overall image noise in the hippocampus region in three states: original images, segmented images, and post-process images. Scale bar, 100 μm. The color bar represents the raw grayscale intensity (0–255) of these 3D reconstruction images.

Our proposed deep learning method is also applicable to two-photon fluorescence images. When using ICG, an FDA-approved probe with high biocompatibility, two-photon imaging benefits from safety and low side effects in vivo (Figure 6B). However, the overall vascular imaging of ICG was characterized by significant noise and low contrast, hindering the detailed observation of vascular structures. Therefore, appropriate preprocessing is required before segmenting vascular images. During preprocessing, the image is subjected to median filtering, outlier removal, and histogram equalization to achieve basic noise reduction and contrast enhancement while preserving as much vascular information as possible. After processing, the deep learning model was used to remove the substantial residual background noise, and the post-process algorithm was applied to reduce foreground noise, resulting in high-contrast, noise-free vascular images (Figure S15). In another set of two-photon imaging data, both fluorescence and third harmonic generation (THG) signals were captured. The THG signals served to delineate white matter regions, beneath which lies the hippocampus—a deep brain structure. As illustrated in Figure 6C, blood vessels in the hippocampus region become much clearer and easier to distinguish. Notably, vascular imaging at a depth of 1,620 μm demonstrated significant improvements in image quality following segmentation and post-processing, with signal distributions exhibiting a more uniform spatial profile. Furthermore, quantitative analysis revealed that noise levels in ICG-enabled two-photon images were reduced from ∼0.8 to negligible levels. The results mentioned above show the application potential of the combination of the deep learning method and ICG.

Overall, the deep learning-based segmentation model in this work demonstrates great effectiveness and application value in optimizing MPFI data enabled by clinically approved probes. Although processing different MPFI data requires distinct preprocessing methods, once a suitable preprocessing method is confirmed for a particular imaging type, subsequent segmentation becomes much simpler and yields highly effective results of segmentation. Furthermore, to better apply the trained deep learning model to optimize the brain vascular MPFI data, we developed an image processing software based on Python-QT (Figure S16). This software supports the import of Tiff format images, vascular segmentation, as well as pre- and post-processing.

Conclusion

In this study, we developed and implemented a deep learning-based denoising method to improve the quality of MPFI images acquired using commercially available probes. A training dataset was constructed based on our previous work about AIEgens NP-mediated MPFI, followed by the preparation of paired data comprising raw images and their corresponding masks. After training, the deep learning model effectively transformed low-quality, blurry input images into clear, high-resolution outputs with minimal noise. The denoised segmented images achieved a significantly enhanced SNR, while the reconstructed MPFI images exhibited vascular structures that closely aligned with ground-truth images enabled by AIEgens NPs. Notably, the proposed method demonstrated robust generalization capabilities when applied to unseen datasets and showed optimized performance for cerebral microvasculature imaging using commercial quantum dots and FDA-approved ICG. This approach advances the utility of MPFI in biomedical research and clinical translation, offering a practical solution to overcome limitations associated with conventional imaging agents and techniques.

Limitations of the study

MPFI offers exceptional spatial resolution, deep-tissue penetration, and relatively high imaging speed, rendering it a highly promising tool for brain research. Our deep learning approach significantly enhances the imaging performance of suboptimal yet clinically approved ICG, thereby facilitating the clinical translation of MPFI. However, it is important to acknowledge that MPFI remains largely confined to fundamental research. At this stage, its applicability has been demonstrated primarily in relatively straightforward scenarios, such as microvascular imaging, hemodynamic profiling, and neural activity mapping.

From a clinical perspective, MPFI holds potential for addressing clinical needs, including intraoperative vascular navigation, real-time blood-brain barrier assessment, and visualization in neurosurgical decision-making. Nevertheless, generating relevant MPFI data from disease models would require the development and optimization of specialized animal models, necessitating substantial further research. Moreover, considerable challenges remain for the clinical adoption of MPFI. Further technical advancements are essential, particularly in hardware design, including expanded fields of view, reduced equipment costs, and simplified operational workflows. On the computational side, future deep learning models should incorporate more sophisticated image recognition capabilities and establish clearer correlations among vascular morphology, hemodynamic parameters, lesion identification, and pathological signatures. Only through integrated progress in both instrumentation and analytical algorithms can the clinical application of MPFI transition from a conceptual prospect to a practical reality.

In addition, while the proposed deep learning model demonstrates robust performance in enhancing microvascular image quality and structure segmentation, its clinical application must be approached with careful consideration of associated ethical challenges. A significant concern is the risk of misdiagnosis resulting from over-reliance on automated outputs, particularly in cases where the model may generate false-positives or false-negatives. It is, therefore, imperative that all AI-derived results be validated by experienced clinicians and not be used as standalone diagnostic tools. Furthermore, as the model was trained on data acquired under specific experimental conditions, it may inherit biases related to imaging protocols, sample characteristics, or anatomic variability. To mitigate these issues, future efforts should emphasize diversified training datasets and ongoing model calibration. Importantly, AI-assisted imaging should be viewed as a complement to, not a replacement for, human expertise. Continuous professional oversight and interpretability-aware design are essential to ensuring ethical and safe integration of such technologies into clinical practice.

Resource availability

Lead contact

The relevant experimental methods and related data of this study can be obtained by contacting the lead contact, Gaixia Xu (xugaixia@szu.edu.cn).

Materials availability

This study did not generate new unique reagents.

Data and code availability

• •All data reported in this paper will be shared by the lead contact upon request.
• •Original codes used in this study are publicly available on GitHub (https://github.com/lhr314/MPFI_vessels_denoising)
• •Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

Acknowledgments

The authors acknowledge the financial support from the Shenzhen Medical Research Fund (A2403014 and D2401012), National Key Research and Development Project of China (2024YFF0726004), National Natural Science Foundation of China (62205216), Basic and Applied Basic Research Foundation of Guangdong Province (2025A1515010016), Basic Research Foundation of Shenzhen (JCYJ20240813143321028, JCYJ20230808105412024, and JCYJ20220531101814034), Medicine Plus Program of Shenzhen University (2023YG002 and 2024YG009), Scientific Instrument Developing Project of Shenzhen University (2023YQ008), and the Research Grants Council of the Hong Kong Special Administrative Region, China (PolyU 15218923). The authors also thank the Instrument Analysis Center of Shenzhen University for the equipment used.

Author contributions

Conceptualization, Z.X., K.Y., and G.X.; methodology, H.L., S.Y., J.P., Y.Z., and K.W.; investigation, H.L., T.C., and Z.X.; analysis, H.L., S.Y., J.P., and Y.Z,; funding acquisition, Z.X., Y.S., and G.X.; project administration, Z.X., Y.G., and G.X. writing – original draft, H.L. and Z.X.; writing – review & editing, Z.X., K.-T.Y., Y.G., W.-C.L., and G.X.

Declaration of interests

The authors declare that they have no competing interests.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Software and algorithms
ImageJ	Open source	N/A
Python	Python Software Foundation	https://www.python.org/
PyTorch	Meta AI	https://pytorch.org/
scikit-learn	scikit-learn developers	https://scikit-learn.org/
OpenCV	Open source	N/A
Matplotlib	Matplotlib development team	https://matplotlib.org/
PyWavelets	PyWavelets developers	https://pywavelets.readthedocs.io/
LabelMe	MIT Computer Science and Artificial Intelligence Laboratory (CSAIL)	https://pywavelets.readthedocs.io/
Origin	OriginLab	https://www.originlab.com/

Method details

Construction of training and testing set

The imaging sequence with a total depth of 1685 μm was used as the training set (337 images) while the imaging sequence with a deeper depth of 1900 μm was used as the testing set (380 images). Each image in these files has a size of 512 × 512 pixels, with a depth difference of 5 μm between the adjacent images. Data augmentation was performed to increase the training set to 1,348 images through rotation, horizontal flipping, and vertical flipping (Figure S17). Additionally, augmentation operations are applied to the original data while maintaining the overall sequence of the image stack relationship. This procedure enhanced the model’s generalization performance and improved its robustness in handling various imaging scenarios. Additionally, since some models consider the sequence of images stack, augmentation operations are applied to the original data while maintaining the overall sequence of images stack relationship. The newly enhanced data are similarly added to the training set according to the sequence relationship. The quantum yields and multiphoton absorption cross-section of all probes used in this study are summarized in Table S1.

Imaging and data analysis

The wavelet transform calculation of images were carried out using the Pywt (PyWavelets Development Team, Netherlands). All the fluorescence intensity curves were obtained through the normalized gray values of the ROI in the original images. All data analyses and image process were performed using Matplotlib (Matplotlib Development Team, USA), OpenCV (OpenCV Team, USA), skimage (scikit-image Development Team, USA), tifffile (Christoph Gohlke, USA) and ImageJ (NIH, USA). napari (napari contributors, USA) was used to precisely rebuild the 3D volume model of blood vessels for images and display the intuitive visualization before and after segmentation.

Mathematical expressions of dice loss and focal loss

$$DiceLoss=1-Dice=1-\frac{2\left|X\cap Y\right|}{\left|X\right|+\left|Y\right|}$$ (Equation 1)

$$FocalLoss={\alpha \left(1-p_t\right)}^{\gamma }\mathit{log}\left(p_t\right)$$ (Equation 2)

In Dice Loss, ∣X∩Y∣ represents the number of intersecting elements between X and Y, while ∣X∣ and ∣Y∣ denote the number of elements in X and Y, respectively. This loss function focuses on the overlapping portion between predicted results and ground truth labels, providing a good adjustment for the imbalance between positive and negative samples (vessels and background). In Focal Loss, p is the probability of predicting the target class for a pixel. If the pixel of the ground truth label is 1 (vessel), then p is used; if the pixel of the ground truth label is 0 (background), then 1−p is used. This loss function introduces two parameters, α, and γ, into the cross-entropy loss function to adjust the loss weights for positive-negative sample imbalance and difficult-easy sample adjustment.

The calculation of the composite score

The formula for the composite performance score is as follows

$$\text{Score}=0.4\ast \text{Io}{\mathrm{U}}_{\text{norm}}+0.4\ast \text{Dic}{\mathrm{e}}_{\text{norm}}+0.2\ast (1-{\mathrm{T}}_{\text{norm}})$$ (Equation 3)

The weights for these three metrics are 0.4 for Dice, 0.4 for IoU, and 0.2 for Time and these metrics are normalized before calculation. In our task, which involves the segmentation and reconstruction of a set of MPFI vascular images comprising over 300 sub-images, we considered temporal efficiency a crucial factor. Therefore, we incorporated Time (the time required for the model to segment a full set of images) as a weighted metric. The construction of the composite score draws inspiration from challenges such as the EAD Challenge and the ISBI Cell Tracking Challenge.

The inter-batch coefficient of variation equation

The inter-batch coefficient of variation (CV) can be calculated according to the below equation:

$$CV(\%)=\frac{{\sigma }_{means}}{{\mu }_{global}}\times 100\%$$ (Equation 4)

where σ_means represents the standard deviation of the batch means, and μ_global denotes the global mean across all data points.

The preprocessing steps of the testing dataset

q800: Beyond a certain depth, a reduction in contrast and suboptimal observability may occur. To address this, histogram equalization is applied to improve image contrast.

ICG: The overall vascular imaging based on ICG exhibits significant noise and low contrast. Therefore, the following sequential processing steps are implemented: median filtering, removal of outliers filtering, and histogram equalization. The intensity of filtering is carefully controlled within a reasonable range to prevent the loss of fine vascular structures.

The formula for SNR

$$SNR\left(dB\right)=10{\mathit{log}}_{10}\left[\frac{{\sum }_{x=0}^{m-1}{\sum }_{y=0}^{n-1}{\left(f\left(x,y\right)\right)}^2}{{\sum }_{x=0}^{m-1}{\sum }_{y=0}^{n-1}{\left(f\left(x,y\right)-\hat{f}\left(x,y\right)\right)}^2}\right]$$ (Equation 5)

f(x,y) represents the pixel values in the image to be computed, and $\hat{f}\left(x,y\right)$ represents the pixel values in the segmented vascular image based on the labels.

The calculation of the noise level of dataset

The noise level was estimated based on the following formula

$$D=\{D_1^H\left(i,j\right),D_1^V\left(i,j\right),D_1^D\left(i,j\right)\forall i,j\}$$ (Equation 6)

$$Noiselevel=\frac{1}{0.625}·median\left(\left|D-median\left(D\right)\right|\right)$$ (Equation 7)

D represents the set of all first-level wavelet detail coefficients, specifically including the horizontal (H), vertical (V), and diagonal (D) directional components, where $D_1^H\left(i,j\right)$, $D_1^V\left(i,j\right)$, and $D_1^D\left(i,j\right)$ denote the coefficient values at position (i,j) in the horizontal, vertical, and diagonal detail subbands, respectively. This comprehensive set captures the high-frequency information across all spatial orientations: D = {$D_1^H\left(i,j\right)$, $D_1^V\left(i,j\right)$, $D_1^D\left(i,j\right)$ ∀i,j}. The noise level is calculated using a robust statistical estimator based on the Median Absolute Deviation (MAD). The term D-median(D) computes the absolute deviations of all wavelet coefficients from their collective median value, providing a measure of dispersion that is resistant to outliers. The scaling factor 1/0.6745 serves as a normalization constant that converts the MAD estimate to a consistent standard deviation estimator under the assumption of noise in the wavelet domain. The noise level of AIEgen training set is demonstrated in Figure S18.

Quantification and statistical analysis

All statistical analyses and plotting were performed using Python with the Matplotlib library. Detailed descriptions of the image processing and analysis software (e.g., Pywt, OpenCV, skimage) are provided in the “imaging and data analysis” section of the method details. All statistical details for the experiments, including the statistical tests, the exact value of n, and the measures of data presentation, are reported in the Figure Legends. Data Presentation: The variability in image processing times in Figure 5 is attributed to minor fluctuations in hardware temperature and OS environment, hence the use of mean ± Standard Deviation (SD). Sample Size (n): The sample size n represents the number of independent images in the training set (n = 337) or testing set (n = 380). Statistical Metrics: Model performance was evaluated using the Dice coefficient, IoU, and a composite score as defined in the method details.

Published: November 19, 2025