Integrating photoacoustic microscopy with other imaging technologies for multimodal imaging

Abstract

As a hybrid optical microscopic imaging technology, photoacoustic microscopy images the optical absorption contrasts and takes advantage of low acoustic scattering of biological tissues to achieve high-resolution anatomical and functional imaging. When combined with other imaging modalities, photoacoustic microscopy-based multimodal technologies can provide complementary contrast mechanisms to reveal complementary information of biological tissues. To achieve intrinsically and precisely registered images in a multimodal photoacoustic microscopy imaging system, either the ultrasonic transducer or the light source can be shared among the different imaging modalities. These technologies are the major focus of this minireview. It also covered the progress of the recently developed penta-modal photoacoustic microscopy imaging system featuring a novel dynamic focusing technique enabled by OCT contour scan.

Impact statement

Photoacoustic microscopy-based multimodal imaging technologies can provide complementary contrasts for biomedical imaging with high resolution. These technologies are being developed to significantly improve our capacity for disease research and diagnosis.

Introduction

Photoacoustic microscopy (PAM) is an emerging branch of photoacoustic (PA) imaging, a field of hybrid optical imaging technologies. It has attracted tremendous attention among scientists and researchers in the biomedical optical imaging field since invented more than a decade ago. It has been shown that PAM can image the structure and function of biological tissues with three dimension (3D) spatial resolution by taking advantage of the spatially resolved PA effect. ¹ The PA effect is generated in a sample as a result of transient pressure increase upon illumination of a pulsed/intensity-modulated laser. The pressure increase is caused by conversion of the light energy into heat when absorbed by an optical absorber in the biological sample, typically hemoglobin in the blood or melanin. The locally absorbed light energy together with the sample’s mechanical and thermal properties determines the amplitude of the initial pressure impulse.^2,3 Eventually, ultrasonic waves are generated and detected with an ultrasonic transducer converting into electric signals. The signals are then amplified, digitized, and constructed into image with a computer. The transverse resolution of PAM is determined by either the ultrasonic or the optical focus and accordingly the technology can be categorized as either acoustic- or optical-resolution PAM (AR-PAM vs. OR-PAM). ⁴ Different from AR-PAM, where the light illuminates an area bigger than the ultrasonic focal spot, in OR-PAM the illuminating light is focused into the sample. ⁵ In both versions of PAM, the frequency of the ultrasonic transducer and its bandwidth determines the depth resolution.

PAM offers additional complementary information when integrated with other imaging technologies in a multimodal imaging platform. PAM images the optical absorption contrasts of biological tissues, a unique contrast mechanism different from other imaging modalities. Although all the different optical imaging technologies play significant roles in various biomedical research, they all can provide limited information needed for disease diagnosis and research due to limitations of each contrast mechanism. To fill this gap, multimodal microscopic imaging techniques have been investigated to offer more comprehensive information of biological tissues. PAM was combined with ultrasound (US)^6–13 to provide multi-contrast imaging or to use US as guidance to facilitate PAM image construction/quantification. PAM was integrated with confocal fluorescence microscopy (CFM)^14–20 for dual molecular-contrast imaging, which has been applied in retinal imaging in vivo. Combination of PAM with optical coherence tomography (OCT)-based imaging techniques including optical Doppler tomography (ODT) and OCT angiography (OCTA) has also been demonstrated for in vivo applications.^21–35 It was also demonstrated to integrate PAM with multiphoton and second harmonic generation (SHG) microscopy.^36–38

Among the different strategies to achieve multimodal PAM imaging, two special techniques can acquire intrinsically registered images of the different modalities, one of which is to share the ultrasonic transducer and the other is to share the illuminating light source. This article provides a review of these multimodal PAM imaging techniques, developed by different research groups.

Multimodal PAM imaging techniques with shared ultrasonic transducers

Due to its unique hybrid feature, PAM can be naturally integrated with US to provide both optical absorption and acoustic contrasts for multimodal imaging. A single ultrasonic transducer can be shared to detect the PA signal and to transmit/receive the US pulse echoes to generate co-registered PAM and US images. Several research groups combined either OR-PAM^6,7,10 or AR-PAM^9,11–13 with US imaging using a shared ultrasonic transducer.

The combined US and optical absorption contrasts were applied to identify cell types in blood smear, which can be potentially applied to detect abnormal and diseased cells. Eric M. Strohm et al. ⁶ integrated OR-PAM with US imaging in the transmission mode using an inverted optical microscope. By using a 1000 MHz ultrasonic transducer co-aligned and co-focused with a 20× objective lens, they achieved a transverse spatial resolution of 1 µm. The system can acquire PAM images at illuminating wavelengths of 532 nm and 600 nm. The two illuminating wavelengths were generated by using the Raman effect in a single mode optical fiber (SMF). US and PAM images were acquired sequentially, US imaging first followed by 532 nm then 600 nm PAM, by scanning the specimen using the microscope translation stage (step size: 0.33 µm, field-of-view: 20 × 20 µm). By combing features of PAM and the US images, different cell type in blood smear such as neutrophils, lymphocytes, and monocytes can be identified. To improve the signal-to-noise ratio (SNR), the signals were averaged 100 times. Thus, the imaging speed should be slow although parameters like the pulse repetition rate (PRR) of the laser and the sample scanning speed were not given in the paper.

In a PAM + US dual-modal technique, the integrated US biomicroscopy was used to image the skull and detect its thickness in rodents.^7,8 By comparing the A-lines of the US and PAM and using an angle-corrected homogeneous model for the skull of rodent, the PA signals of the calvarian vasculature can be segmented from that of the cerebral. This technique thus provided a method for removing the effect of the skull in PAM brain images and enabled PAM imaging of the calvarian vasculature. Hector Estrada et al.^7,8 developed a dual modal OR-PAM and US imaging system with a customized focused ultrasonic transducer. The ultrasonic transducer has an opening at the center to mount a GRIN lens, which focused the illuminating laser light of 578 nm onto the ultrasound focus. The imaging head was mechanically scanned in two-dimensions (2D) by using a fast piezo stage and a linear translational stage in the x and y axes, respectively. The combined 3D PAM-US images were acquired within 90 s and a lateral resolution of better than 20 µm was achieved.

In addition to complementary anatomical imaging, the integrated US system can also provide Doppler ultrasound (DUS) imaging. Yan Jiang et al. ⁹ combined AR-PAM with DUS imaging by using a shared 25-MHz ultrasound transducer and confocal dark-field illumination. The shared ultrasonic transducer was responsible for both PAM signal detection and US excitation/detection. The dual modal imaging system was tested on flow phantom. They have demonstrated that DUS had advantages for imaging large vessels with high flow speeds, while PAM is more effective for imaging small vessels with slow speeds. The combined imaging can improve estimates of fractional blood volume.

Multimodal PAM imaging platforms with shared light sources

By using the same light source, OR-PAM can be combined with other imaging modalities such as CFM^14–16,20 and OCT^23–25,27 to achieve multimodal imaging.

PAM and CFM

The feasibility of mapping two different molecules simultaneously by imaging the absorption and fluorescence contrasts using a single pulsed light source was first reported in 2010. ¹⁴ The technique was demonstrated by imaging excised ocular tissues. Xiangyang Zhang et al. ¹⁵ further integrated photoacoustic ophthalmoscopy (PAOM), a PAM technology for imaging the retina in vivo, with autofluorescence (AF) imaging. The technique was able to image melanin and lipofuscin in the retinal pigment epithelium (RPE) cells of rat eyes simultaneously in vivo with a single light source of 532 nm. The PAM-AF imaging system was also integrated with a spectral-domain OCT, which used a near-infrared broadband light source centered at 830 nm. The OCT system provides guidance to PAOM and can help reduce the visible laser exposure to the eye. The pulsed laser light was scanned and delivered to the eye by an X-Y galvanometer scanner in combination with a pair of relay and ocular lenses. For PAM signal detection, a single element unfocused 30 MHz needle ultrasound transducer placed on the eyelid using ultrasonic coupling gel was used. The fluorescence signal was detected with an avalanche photodiode through a 50 µm multimode optical fiber acting as a pinhole. With a 24 kHz PRR, the system can acquire 3D images of the retina within 2.7 s. A schematic of the imaging system and the imaging results are shown in Figure 1.

Figure 1.

Schematic of the multimodal PAOM system (a) and sample images acquired from pigmented rats (b). ND: neutral-density filter; PD: photodiode; I: iris; DM: dichroic mirror; LF: long pass Filter; LRF: laser rejection filter; MMF: multimode optical fiber; APD: avalanche photodetector; UT: ultrasonic transducer; SLD: Superluminescent diode; PC: polarization controller. In (b), a1 and a2: AF image; b1 and b2: PAOM image; a1 and b1: 10 weeks old; a2 and b2: 18 weeks old. ¹⁵ (A color version of this figure is available in the online journal.)

Dual modal OR-PAM and CFM using a single excitation laser was also demonstrated by Yu Wang et al. ¹⁶ for in-vivo mouse ear imaging to examine tumor angiogenesis, vasculature, and microenvironments. The sample was illuminated by two wavelengths, 570 nm and 593 nm, which were applied sequentially for each B-scan. The PAM mode was thus able to provide oxygen saturation imaging of the blood vessels. With a 75 MHz ultrasonic transducer, the depth and transverse resolutions were quantified, 17 µm and 3.9 µm for PAM, 38 µm and 3.9 µm for CFM. By combining PAM angiography and fluorescence lymphangiography, they successfully demonstrated that the dual modal PAM-CFM system was able to provide comprehensive information for monitoring the growth and treatment effect of tumor in small animals.

PAM and OCT

Combining PAM with OCT was first introduced in 2009 for in vivo animal imaging, ²² by using two different light sources for PAM and OCT. Xiangyang Zhang et al. ²³ reported the first dual modal PAM-OCT technology in 2012 using a shared broadband pulsed light source to achieve PAM and OCT imaging simultaneously. The PAM signal was detected with an unfocused ultrasonic transducer, while the OCT interference signal was detected with a Michelson interferometer in the spectral domain. The technology was termed optical coherence photoacoustic microscopy (OC-PAM). Following that, several research groups published similar techniques.^23–25,27

The first reported OC-PAM ²³ used a pulsed broadband dye-laser (center wavelength: 580 nm) for both PAM and OCT. The OCT Michelson interferometer was built in free space. Each laser pulse generated one depth-scan for both imaging modalities. The speed of the PAM and OCT was determined by the PRR of the dye laser (Maximum PRR: 5 KHz). The system was demonstrated by imaging the vasculature of a mouse ear in vivo (see Figure 2). Changho Lee et al. ²⁴ also combined PAM with OCT in 2013 using a supercontinuum laser source centered at 1064 nm; however, no in vivo imaging study was reported in their work.

Figure 2.

Schematic of the experimental system of a free-space OC-PAM (a) and images simultaneously acquired from a mouse ear in vivo (b). L1–L4: lens; BS: beam splitter; SMF: single mode fiber; PD: photodiode; UT: ultrasonic transducer; M1, M2: mirror. In (b), (a) OCT B-scan image; (b) PAM B-scan image; (c) maximum-amplitude-projection (MAP) of the 3D PAM dataset. Bar: 100 µm. ²³ (A color version of this figure is available in the online journal.)

Xiaojing Liu et al. ²⁵ reported an OC-PAM technology in 2015 to provide multimodal in vivo imaging of the retina. The system used an ultrafast Ti:sapphire laser amplifier as the illuminating light source for both PAM and OCT (Maximum PRR: 10 KHz, center wavelength: 800 nm, bandwidth: 30 nm). A 2 × 2 single-mode optical fiber coupler was used to build the Michelson interferometer for OCT. A custom-built needle ultrasonic transducer was used to detect the PAM signal. The transducer (center frequency: 30 MHz, active element diameter: 0.4 mm) was placed in contact with the eyelid coupled with ultrasound gel. Pigmented rat retina was successfully imaged, in which the retinal layer structures were imaged by OCT, while the melanin distribution in the retina was imaged by PAM (see Figure 3).

Figure 3.

Schematic of the experimental system (a) and simultaneously acquired OCT and PAM B-scan images (b) of a fiber-based OC-PAM. L1, L2: lens; PC: polarization controller; UT: ultrasonic transducer; M1, M2: mirror; C1, C2: collimator. Bar: 100 µm. ²⁵ (A color version of this figure is available in the online journal.)

OC-PAM with an intensity-modulated continuous-wave light source was also demonstrated. ²⁶ The technique used an 840 nm superluminescent diode (SLD). The output light was modulated by modulating the SLD driving current. Compared to pulsed OC-PAM, the advantage of this technique is that the OCT image quality can be as good as conventional spectral-domain OCT with similar light intensity and bandwidth. The technique was tested by imaging biological samples in vivo and ex vivo. Gold nanorods were used as contrast agents for the PAM mode when the vasculature of mouse ear was imaged.

Magalie Bondu et al. ²⁷ reported a dual modal OCT-PAM technology with a configuration similar to Zhang et al. ²³ except that a supercontinuum light source was used. Although the same light source was used, the OCT and PAM was generated by different bands of the supercontinuum output: OCT used the band centered at 1310 nm, while PAM used the band of 500–840 nm. Since the center wavelength and bandwidth of the light source can be independently selected, the system can perform multispectral PAM (MPAM). A customized ultrasonic transducer (center frequency: 40.3 MHz, active element: φ0.4 mm) was used to detect the PA signals. The system achieved imaging speed of 20 B-scans/second for both PAM and OCT. The system was tested on phantoms and in vitro samples using dyes as contrast agent.

Latest progress for OCT-guided penta-modal PAM imaging technique

Recently, PAM was integrated with OCT, ODT, and CFM for quadruple-modal imaging (see Figure 4) using either mechanical scanning, which scans the sample, or combined mechanical and optical scanning. In the system, a single Q-switched 532 nm laser was shared for PAM and CFM.^39,40

Figure 4.

A schematic of the PAM/OCT/ODT/CFM multimodal imaging system. SLD: superluminescent diode; FC: 2 × 2 fiber coupler; PC: polarization controller; L1–L4: Lens; M1–M3: mirror; DM1, DM2: dichroic mirror; C1–C3: collimator; ND: neutral-density filter; F1–F3: optical filter; PBS: pellicle beam splitter; PD: photodiode. ⁴⁰ (A color version of this figure is available in the online journal.)

By adding OCTA function, the system also achieved penta-modal imaging as shown in Figure 5. ⁴¹ Generated by the same photons, the images of PAM and CFM are precisely registered in the lateral directions, while the OCT, ODT, and OCTA images are also registered. Registration among the PAM and OCT-based images was achieved by light alignment and synchronization control.

Figure 5.

PAM (a: maximum amplitude projection), CFM (c), OCT (d: en face view, e: B-scan), and OCTA (b: en face view, f: cross-sectional) images acquired in the same ROI of a mouse ear. The locations of the cross-sectional images are marked in the corresponding en face views by solid lines. Bar: 200 mm. ³⁹ (A color version of this figure is available in the online journal.)

The imaging system also features OCT-guided dynamic focusing, a technique for adjusting objective lens focus to follow the contour of the sample surface. It is necessary to apply dynamic focusing for a sample having uneven surface to help achieve uniform spatial resolution and SNR across the region of interest (ROI). Dynamic focusing was enabled by the OCT depth information of the sample surface. ³⁹ Dynamic focusing using PAM or US for contour scanning was reported by other groups.^42,43 The advantage of OCT-guided dynamic focusing is its better guiding accuracy and faster guiding speed thanks to the better performance of OCT.

There are two different strategies for dynamic focusing using OCT guidance, point by point and aerial, depending on which scanning mechanism was used. When mechanical scanning was employed for multimodal imaging, ³⁹ point by point dynamic focusing can be applied. The focus of the objective lens can be adjusted for each A-line of PAM. However, this strategy is relatively slow. When fast imaging speed is desired, this method can be a bottleneck. To solve this problem, the aerial dynamic focusing strategy was developed for multimodal PAM imaging system using the combination of optical and mechanical scanning. ⁴⁰ This approach adjusts the objective-lens focus according to the depth of surface at the center of each fast-optical scan area. This strategy is suitable for achieving dynamic focusing in large field-of-view (FOV) with fast-speed imaging. The results of dynamic focusing with the two different methods are shown in Figures 6 and 7.

Figure 6.

PAM (a: maximum amplitude projection), CFM (d), OCT (b: B-scan, e: en face view) and ODT (c) images of a mouse ear simultaneously acquired with dynamic focusing. (f) Fused PAM, CFM, and OCT images; SG: sebaceous glands. The locations of the cross-sectional images are marked in the corresponding en face views by solid lines. Bar: 100 µm. ³⁹ (A color version of this figure is available in the online journal.)

Figure 7.

PAM (c and d: with dynamic focusing, g and h: no dynamic focusing) and CFM (a and b: with dynamic focusing, e and f: no dynamic focusing) images of a purple queen leaf for testing the OCT-guided dynamic focusing. The enlarged images are marked by a dashed box in the corresponding images. (i) A photo of the imaged leaf; (j) fluorescence signal intensity with and without dynamic focusing, respectively; (k) photoacoustic signal intensity with and without dynamic focusing, respectively; bar: 1 mm and 200 µm for large and small figures, respectively. ⁴⁰ . (A color version of this figure is available in the online journal.)

Summary and future directions

We mainly reviewed the multimodal PAM imaging techniques that either shared the ultrasonic transducer or the light source to acquire co-registered images of biological samples. The major advantage of sharing the ultrasound transducer or the illumination light source is that intrinsically and precisely registered images of the different modalities can be acquired, thus enabling correlative study of different optical contrasts. The possible downside, however, is the limited wavelength, thus limited fluorophores, that can be selected when the light source is shared for different imaging modalities.

These technologies have been demonstrated to be able to image complementary contrasts of a sample either ex vivo or in vivo. Taking the penta-modal imaging system as an example, it images the contrasts that are based on optical absorption, optical scattering, blood flow velocity, motion of the scatters, and fluorescence. Further development of these technologies will need to address the limitations that the current technologies have for clinical research and diagnosis. One limiting factor for the application of the PAM multimodal imaging technologies is how to extract critical information quantitatively from these images for diagnosis and clinical research. Further investigation is needed before these technologies can make a significant impact in practical applications. One achievable improvement is to include the function of imaging blood oxygenation by using multiple illuminating wavelengths. This will add more value to the multimodal imaging technologies.

Another limiting factor for these multimodal imaging technologies in clinical applications is speed of the PAM imaging, which is limited by the scanning mechanism and the PRR of the laser. Thus, a high PRR laser with stable pulse energy is always desired for improving the imaging speed. Fast scanning technique is essential for achieving fast imaging speed. Current optical scanning can achieve fast imaging speed in a very limited FOV. The FOV of current optical scanning OR-PAM is limited by the FOV of the unfocused ultrasonic transducer, which also limits the SNR of PAM. Thus, future development of the multimodal imaging technologies highly relies on the performance and design of the ultrasonic detection techniques.

In conclusion, PAM-based multimodal imaging technologies are still in the early stages of development. It relies on the advancement of laser, scanning mechanisms, and ultrasound detection techniques to make significant impact on disease diagnosis and research.