The integrated high-resolution reflection-mode photoacoustic and fluorescence confocal microscopy

Abstract

A dual modality microscopy with the highest imaging resolution reported so far based on reflection-mode photoacoustic and confocal fluorescence is presented in this study. The unique design of the imaging head of the microscope makes it highly convenient for scalable high-resolution imaging by simply switching the optical objectives. The submicron resolution performance of the system is demonstrated via in vivo imaging of zebrafish, normal mouse ear, and a xenograft tumor model inoculated in the mouse ear. The imaging results confirm that the presented dual-modality microscopy imaging system could play a vital role in observing model organism, studying tumor angiogenesis and assessment of antineoplastic drugs.

1. Introduction

The development of multimodal imaging technology is of great significance in the biomedical research field because the merged image provides a comprehensive understanding of the physiological process in biological systems as different imaging modalities extract complementary morphological, functional and molecular information [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]. The combination of photoacoustic and fluorescence imaging is a good example of this multi-modal imaging. Ntziachristos et al. reported a hybrid system integrating multiphoton, second and third harmonic, and photoacoustic mesoscopy and microscopy [1], [2]. The capability of the system to acquire broad information from biological specimens was nicely demonstrated by label-free imaging of a zebrafish larvae and mouse ear ex vivo. The cost of multiphoton microscopy remains generally high at current stage though. Confocal flourescence imaging offers several advantages over conventional optical microscopy, including elimination of out-of-focus light, high spatial resolution and the capability of optical sectioning [12], [13]. It is low-cost and has the flexibility to select different excitation wavelengths [14]. The application of a wide array of new synthetic and endogenous fluorochromes has made it possible to identify cells and sub-cellular microstructures with high specificity [15]. Photoacoustic imaging, on the other hand, is used mostly to image objects that are ultra-sensitive to optical absorption such as hemoglobin, melanin, cytochromes, and lipid [16], [17], [18], [19], [20], [21]. Therefore, by combining photoacoustic and confocal fluorescence imaging, complementary optical absorption and fluorescence information can be derived from the underlying imaging objects [22]. Several attempts have been made to integrate optical-resolution photoacoustic microscopy with confocal fluorescence microscopy (PAM/CFM) [22], [23], [24], [25], [26], [27]. To achieve high imaging resolution in the submicron regime, transmission mode PAM/CFM dual-modality imaging configuration is generally adopted [22], [23], [24]. But transmission-mode system has a limited biological application only to thin samples due to the nature of data transmission and acquisition. Therefore, reflection mode PAM/CFM systems have been proposed and developed [25], [26]. Although submicron resolution reflection mode photoacoustic microscopy has been reported [28], [29], reflection mode PAM/CFM dual-modality imaging system achieving submicron resolution has rarely been seen. This is because in the reflection mode, the imaging resolution is mostly limited to several to tens microns because optical objective with long working distance (consequently, small numerical aperture (NA)) is needed to create enough space between the objective and the imaging sample to accommodate the ultrasound transducer and the optical acoustic coupling module [26].

In our previous study, we proposed a reflection-mode submicron resolution photoacoustic microscopy system by integrating a miniaturized high-frequency ultrasound transducer to the bottom of a high NA water-immersion optical objective [30]. The ultrasound transducer was fixed and placed in close contact with the objective to guarantee high optical throughput and sufficient working space for reflection mode imaging. We successfully demonstrated submicron resolution imaging with this system. In this paper, to achieve photoacoustic and confocal fluorescence dual-modality imaging system, we integrated fluorescence imaging capability to this system. The unique design of the imaging head of the integrated system makes it highly convenient to scale imaging resolutions by simply switching the objectives with different NA. We quantified the resolution of this new imaging system and further applied the system to in vivo mouse and zebrafish imaging. To the best of our knowledge, the submicron resolution presented in this paper using our PAM/CFM dual-modality imaging system is the highest among all the reflection hybrid imaging systems reported so far, which justifies the main novelty of the work.

2. Experimental setup

Fig. 1 shows the schematic diagram of our integrated PAM/CFM system. A 532 nm pulsed laser (GKNQL-532-6-10, Beijing GK Laser Technology) and a continuous wave (CW) 488 nm laser (OBIS 488 nm LS 40 mW; Coherent) were used as the light source for photoacoustic and fluorescence signal excitation, respectively. The 488  nm CW laser source is integrated with a single-mode fiber for laser output, and the divergence beam from the fiber was collimated by a fiber collimator (F260APC-A, Thorlabs). Hence the core of the single-mode fibers served as the first confocal microscope pinhole in CFM. The two laser beams were first combined through a dichroic mirror (DM1, FF505-SDi01, Semrock), then reflected by multiple mirrors, and finally steered by a two-dimensional galvanometer scanner (CTI 6210, Cambridge) for raster scanning based photoacoustic and fluorescence imaging. The raster scanning step size was kept less than 1/5 of the lateral resolution for the resolution measurements in this study. For in vivo imaging, the step size was chosen to be about 3 times that of the lateral resolution to enhance the imaging speed. A telescope system consisting of a scan lens (CLS-SL, Thorlabs) and a tube lens (U-TR30-2, Olympus) was used to deliver the laser beam to the back aperture of a water-immersion objective (LUMPLFLN40XW, 40×/0.8; UMPLFLN10XW, 10×/0.3, Olympus). The magnification ratio of the telescope system was adjusted allowing the laser beam to slightly overfilling the back aperture of the objective and thus creating an optimal diffraction-limited focus spot on the sample. The generated photoacoustic signal was detected using a customized miniature ultrasound transducer (central frequency: 40 MHz; bandwidth; 60%; dimension: 0.6 mm × 0.5 mm × 0.2 mm; Blatek, State College, US) fixed firmly below the center of the objective. The distance from the transducer to the sample (i.e. working distance) in our design is about 1 mm, which is generally smaller compared to most other implementations [23], [31]. This is because we use larger NA objectives. The NA of the objective in most other implementations is around 0.1. The compact optical–acoustic coupling configuration enabled reduced acoustic energy transmission loss compared to that in most conventional optical resolution photoacoustic microscopy systems that use an optical–acoustic coupler for coupling purpose [31]. In addition, coaxial optical excitation and acoustic detection were achieved for high-sensitive photoacoustic imaging. The fluorescence signal passed through an optical filer (ET520/40M, Chroma) to remove the reflected excitation light, and then transmitted through a second objective (M-5X, Newport) and a pinhole (900PH-20, diameter 20 μm, Newport), and was finally collected by a photomultiplier tube (H7421-40, Hamamatsu). Due to the miniature size of the ultrasound transducer, its influence was negligible on the focusing of excitation light and the collection of fluorescent signals. During the imaging process, both the objective and the ultrasound transducer were immersed in a water tank filled with deionized water. The bottom of the water tank was made from polyethylene film layer for efficient optical and acoustic transmission. To acquire the specimen information at different depths, we mounted the imaging objective to a motorized z scanner (MTS25, Thorlabs) for confocal fluorescence depth scanning. The depth information for photoacoustic imaging was obtained based on the flight time of the photoacoustic signals. The acquired photoacoustic, and the fluorescent signals were transferred to the data acquisition (DAQ) card (ATS9325, Alazar) as shown in Fig. 1. The photoacoustic signal was amplified by a low-noise amplifier (ZFL-500LN-BNC+, Mini-Circuits). A National Instruments card (PCIe 6323, National Instruments) was used as the trigger source to synchronize the laser firing, data acquiring, and control of the scanning galvanometer. All photoacoustic images were median filtered with 3*3 pixels window for smoothing.

Fig. 1.

Schematic of the integrated PAM/CFM system. CL, collimator; DM, dichroic mirror; PMT, photomultiplier tube; DAQ, data acquisition card.

3. Results

Two objectives with NA of 0.3 and 0.8 were applied for imaging. Phantom experiments were first carried out to quantify the resolution, imaging depth, and optical sectioning capability of the system. The submicron lateral resolution of photoacoustic imaging was quantified by measuring a blade with a sharp edge when using 0.8 NA objective. Unidirectional B-scan imaging was performed to acquire the spread function at the edge, i.e., the plot of the photoacoustic signal amplitude as a function of the lateral distance across the blade edge. The first order derivative was then applied on the edge spread function to obtain the line spread function. The full width at half maximum (FWHM) of the line spread function was calculated to obtain the measured lateral resolution of the photoacoustic imaging modality. The resolution of confocal fluorescence imaging was quantified by imaging 100 nm fluorescent bead. The lateral resolutions for both 0.8 NA and 0.3 NA objectives, and the axial resolution for 0.8 NA objective were obtained. The fluorescence intensity profile along the lateral and axial direction of the bead was fitted to a Gaussian curve. The full width at half maximum (FWHM) of the curve was calculated to obtain the measured resolution. The imaging depth and optical sectioning capability of confocal fluorescence imaging and photoacoustic imaging when using 0.8 NA objective were characterized by imaging 100 nm fluorescent beads and black ink, respectively, in optical scattering agarose medium. For in vivo measurement, to demonstrate the resolution-scalable characteristic of the imaging system, mouse ear imaging was performed with both 0.3 and 0.8 NA objectives. To demonstrate the high resolution in vivo when using 0.8 NA objective, 1 μm fluorescent beads were injected to the mouse ear and the confocal fluorescence images were obtained and shown in Supplementary Figure 4. Further, the PAM/CFM system was applied for in vivo tumor imaging and whole body zebrafish imaging. The 0.3 NA objective was used to obtain a large field of view. The Green Fluorescent Protein (GFP) transgenic 4T1 breast cancer cells were injected subcutaneously to the nude mouse ear for tumor inoculation. After one week of injection, the mouse was transferred to the imaging stage and immobilized with a custom-made animal holder. During the imaging process, the mouse was anesthetized with 1.5% isoflurane mixed with oxygen. Live GFP transgenic zebrafish was used and mounted in the ultrasound gel and water mixture for immobilization during the imaging process. The laser energy used in this study was 100 nJ per pulse for photoacoustic imaging and 200 μW for fluorescence imaging, both of which were kept within the ANSI safety limit [32]. For both mouse and zebrafish observation, photoacoustic and fluorescence imaging was performed alternatively to reduce the laser power on the sample. Since both photoacoustic and fluorescence imaging shared the same optical delivery path and lateral scanning mechanism, the images from the two modalities were automatically co-registered. The 0.3 NA objective enabled a field of view (FOV) of 800 μm × 800 μm. The sample number is 256 × 256, corresponding to pixel size of 3.12 μm × 3.12 μm. The line scanning rate was 40 Hz for photoacoustic imaging (i.e. 6.4 s to image the FOV) and 200 Hz for confocal fluorescence imaging (i.e. 1.3 s to image the FOV), limited by the laser repetition rate and the raster scanning speed, respectively. All the animal handling and experimental procedures were approved by the Animal Study Committee of Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences.

Fig. 2 shows the resolution quantification results for both confocal fluorescence and photoacoustic imaging modalities. The insets in the figure are the images obtained for the resolution measurements. The lateral resolutions of confocal fluorescence imaging for 0.8 NA and 0.3 NA objectives were measured to be 350 nm (Fig. 2(a)) and 952 nm (Fig. 2(c)), respectively. The axial resolution of confocal fluorescence imaging for 0.8 NA objective was measured to be 1.76 μm (Fig. 2(b)). The lateral resolution of photoacoustic imaging for 0.8 NA objective was measured to be 470 nm (Fig. 2(d)). All the measured resolutions are close to their theoretical values (errors within 16%). Fig. 3 shows the imaging depth and optical sectioning capability of confocal fluorescence imaging and photoacoustic imaging when using 0.8 NA objective to image 100 nm fluorescent beads and black ink, respectively, in optical scattering agarose medium. The fluorescent beads at 100 μm depth can be decently seen in the image. The lateral resolutions of confocal fluorescence imaging at different depths were also evaluated, which are 350 nm, 370 nm, 435 nm, 476 nm, 480 nm and 616 nm, respectively, at 0, 20, 40, 60, 80 and 100 μm depths. The black ink was found to take rod-like shapes. Hence the short edge direction was used for resolution characterization. The lateral resolutions of PAM at different depths were measured to be 525 nm, 666 nm, 733 nm, 800 nm, 925 nm and 1.2 μm, respectively, at 0, 20, 40, 60, 80 and 100 μm depths. The resolution-scalable characteristic of the imaging system is demonstrated in Fig. 4. The blood vasculature of the mouse ear was observed with conventional white light imaging (Fig. 4(a) and (c)) and photoacoustic imaging (Fig. 4(b) and (d)), using 0.3 and 0.8 NA objectives, respectively. As can be seen in this Fig. 4, the photoacoustic images exhibit similar vascular structures like that in the white light images, but with a much-enhanced signal to noise ratio (SNR). Compared to white light imaging, more capillary blood vessels can be observed with the photoacoustic imaging. The motion artifacts in the photoacoustic images are possibly due to the scanning of the galvanometer.

Fig. 2.

Characterization of lateral resolutions and axial resolution of the hybrid system. The lateral ((a) and (c)) and axial (b) resolutions of CFM were characterized by imaging 100 nm fluorescent bead using 0.8 NA ((a) and (b)) and 0.3 NA (c) objectives (see top left insets in each figure). The cross-section profiles and their Gaussian curves are plotted in all the figures. (d) The photoacoustic line spread function (LSF) at the edge of a sharp metallic blade with the 0.8 NA objective. Blue scattered asterisk: original photoacoustic signal; red dashed line: edge spread function (ESF); green solid line: the first-order derivative of the ESF, the LSF along the scanning direction.

Fig. 3.

CFM images of 100 nm fluorescent beads and PAM images of black ink at different depths in agarose when using 0.8 NA objective. Scale bar, 4 μm.

Fig. 4.

In vivo resolution-scalable photoacoustic imaging ((a) and (c)) and white light imaging ((b) and (d)) of a mouse ear, with 0.3 ((a) and (b)) and 0.8 ((c) and (d)) NA objectives. Scale bar is 100 μm.

The photoacoustic, fluorescence and merged imaging results of a zebrafish are shown in Fig. 5(a)–(c). The fluorescence image depicts the overall profile of the zebrafish while the photoacoustic image reflects the vasculature distribution within the body. Compared to the individual image, the merged image provides a more comprehensive understanding of the biological system. The eyes of zebrafish can be visualized in the image. More imaging results of the zebrafish are presented in the supplementary materials (Supplementary Figures 1–3, Supplementary Video). The photoacoustic, fluorescence and merged images of a tumor inoculated in the mouse ear are shown in Fig. 5(d)–(f). In the fluorescence imaging, the GFP transgenic tumor mass can be identified with high specificity and thus the boundary of the tumor can be depicted in the fluorescence image (Fig. 5(d)) with high fidelity. On the other hand, blood vessels feeding the tumor can be clearly visualized in the photoacoustic image as shown in Fig. 5(e). Hence the combined photoacoustic with fluorescence imaging, as shown in Fig. 5(f), provides key information for the study of tumor angiogenesis process without the need of any exogenous labeling of the blood vasculature. Further, we calculated the SNR in both photoacoustic and confocal fluorescence images by dividing the mean intensity of ROI (region of interest) by standard deviation of dark region. The SNR were measured to be 14.40 dB, 20.88 dB, 18.50 dB and 14.08 dB in Fig. 5(a), (b), (d) and (e), respectively. With the complementary capability of photoacoustic and confocal fluorescence imaging, extensive information of tissue microarchitecture in vivo can be obtained, such as blood vasculature and cellular architecture. The dual-modality microscopy could also play a vital role in other biomedical applications such as antineoplastic drug assessment [33], to understand the pharmacological mechanism (e.g., tumor blood vessel normalization [34]) of various drugs for tumor treatment [35].

Fig. 5.

Confocal fluorescence ((a) and (d)), photoacoustic ((b) and (e)) and their merged ((c) and (f)) images of a zebrafish (first row) and a tumor inoculated in the mouse ear (second row), respectively. Dashed circle in (d) indicates the tumor injection site.

4. Conclusions

To conclude, a reflection mode integrated PAM/CFM dual-modality imaging system with the highest resolution compared to existing systems is presented in this study. This system uses optical/acoustic coaxial design by attaching an ultrasound transducer directly below the objective to achieve high imaging sensitivity for the photoacoustic modality. The imaging head of the system remains compact enabling resolution-scalable imaging by simply switching objectives with different NA. The phantom imaging as well as in vivo zebrafish and mouse imaging using PAM/CFM system showed much-improved image resolution while providing complementary information. It is noteworthy that the axial resolution of PAM is worse than that of CFM, which raises challenge when merging the two modality images. There are some works that have been reported to improve the axial resolution of PAM, such as using optical micro-ring detection or based on the Grueneisen relaxation effect [36], [37]. The lateral resolution of the system can be further improved by using an optical objective whose NA is higher than 0.8, but could lead to reduced imaging depth. By using a wavelength tunable pulsed laser source (e.g., OPO laser) and by adding other single wavelength continuous laser sources to the current excitation laser path, the system can be further enhanced to differentiate more biological constituents, such as oxygenated and deoxygenated hemoglobin differentiation for SO₂ quantification [38]. Thus we believe, using our system with enhancements, biomedical applications can be further expanded, such as to the study neurovascular coupling mechanism in the brain [39], [40] and for tumor microenvironment research [35] to uncover the signaling pathway of disease-specific up-regulated molecules.

Conflict of interest

The authors declare that there are no conflicts of interest.

Biographies

Chengbo Liu is an Associate Professor at SIAT, CAS. He received both his Ph.D. and Bachelor degree from Xi’an Jiaotong University, each in 2012 in Biophysics and 2007 in Biomedical Engineering. During his Ph.D. training, he spent two years doing tissue spectroscopy research at Duke University as a visiting scholar. Now he is an associate professor at SIAT, working on multi-scale photoacoustic imaging and its translational research.

Jiuling Liao received his B.S. degree in Photoelectric Information Engineering from Nanjing University of Science and Technology in 2012, and Ph.D. degree in Optical Engineering from Nanjing University of Science and Technology in 2017. Dr. Liao joined SIAT as a Postdoctor in July 2017. His research interest focuses on the application of multimodal microscopy.

Longchao Chen received his Master's degree in Optical Engineering from Fujian Normal University, Ministry of Education Key Laboratory of OptoElectronic Science and Technology for Medicine in 2014. His current interest focuses on designing and researching Laser scanning confocal microscopy and super-resolution microscopy system.

Jianhua Chen received his Master's degree in Electronic Engineering from Tsinghua University on the topic of ultra-high speed optical fiber transmission and signal processing. His current research focuses on developing data acquisition and imaging schemes for AR/OR-Photoacoustic medical devices.

Rubo Ding received his B.S. degree from Zhejiang University. His research interest focuses on developing new model medical equipment.

Xiaojing Gong is an Associate Professor at SIAT, CAS. He received his B.S. degree in Mechatronics from the Southeast University in 2000, and Ph.D. degree in Fine Instruments and Mechanics from the University of Science and Technology of China in 2007. Since joined the IBHE of SIAT in 2007, Dr. Gong focuses in the research of Biomedical Optics Imaging, includes Photoacoustic Tomography/Microscopy, Optical Coherence Tomography/Microscopy, Terahertz biomedical imaging, etc.

Caimei Cui received her B.S. degree in Biotechnology from Xiamen University, and Mater degree in Medical Biochemistry and Molecular Biology Sun Yat their-sen University. Now she focuses on translational clinical research.

Zhiqiang Pang received his Master's degree in Engineering Management from Tsinghua University. He focuses on translating research discoveries from the laboratory into clinical practice to diagnose and treat patients.

Wei Zheng is a professor at SIAT, CAS. He received his B.S. degree in Optical Engineering from Zhejiang University in 2003, his Ph.D. degree in Electronic and Computer Engineering from Hong Kong University of Science and Technology in 2011. His research focuses on biomedical optics, including fluorescence spectroscopy, fluorescence microscopy and nonlinear optical microscopy.

Liang Song is a professor at SIAT, CAS and founding directors of The Research Lab for Biomedical Optics, and The Shenzhen Key Lab for Molecular Imaging. Prior to joining SIAT, he studied at Washington University, St. Louis and received his Ph.D. in Biomedical Engineering. His research focuses on multiple novel photoacoustic imaging technologies.

Appendix A. Supplementary data

The following are the supplementary data to this article: