High-Speed Integrated Endoscopic Photoacoustic and Ultrasound Imaging System

Abstract

Endoscopic integrated photoacoustic and ultrasound imaging has the potential for early detection of the cancer in the gastrointestinal tract. Currently, slow imaging speed is one of the limitations for clinical translation. Here, we developed a high speed integrated endoscopic PA and US imaging system, which is able to perform PA and US imaging simultaneously up to 50 frames per second. Using this system, the architectural morphology and vasculature of the rectum wall were visualized from a Sprague Dawley rat in-vivo.

I. INTRODUCTION

Colorectal cancer, the third most common type of cancer globally, has ~1.4 million new cases and 694,000 deaths annually[1]. Currently, gastroenterologists routinely utilize visible light endoscopy to visualize the rectum wall for diagnosing various diseases. Due to the lack of depth resolved information and limited vascular lesion sensitivity, they are not able to clearly visualize early epithelial dysplastic changes that may lead to the development of cancer [2]. In recent decades, there has been substantial development in endoscopic imaging technologies [3–6] that are capable of visualizing the sub-surface tissue morphology and vascular network to provide necessary diagnostic information. For example, endoscopic ultrasound (US) imaging allows a clinician to obtain images of the gastrointestinal (GI) tract and the surrounding tissue/organs with a large penetration depth [5, 7]. Endoscopic optical coherence tomography (OCT) is capable of providing high-resolution cross-sectional images [6, 8–10], which is often used to image sub-layered architecture. Both OCT and US provide the morphology of biological tissue but lack molecular information, which is often insufficient for accurate diagnosis since both structure and the pattern of vasculature are also highly relevant to GI disease. Endoscopic photoacoustic (PA) imaging is a non-invasive imaging modality that provides molecular contrast with depth resolved information [11–14]. Integrated with ultrasound (US) imaging, this multimodal endoscopic PA/US imaging technology is able to provide both structural and chemical compositions of colorectal walls for diagnosis of GI cancer at an early stage. Several groups have reported different designs of an integrated endoscopic PA/US imaging system [15–19] that represent a significant step forward for the characterization of GI cancer. However, these imaging systems are still not adequate for in-vivo clinical translation due to insufficient field-of-view, large probe diameters, and slow imaging speed. For example, the systems [18, 20] reported by Xing et al. and Li et al. are limited due to oversized probes which are incompatible with a clinical endoscope. Yang et al. developed a series of endoscopic photoacoustic imaging systems [16, 21] based on a distal scanning method with much smaller catheters. However, only part of the cross-sectional images could be obtained due to the partial blocking of the view from the electric wires of the micro motor. In addition, the probes were rigid and had slow imaging speeds (<10 Frame/s), which limited the clinical applications.

In this study, we demonstrate an integrated endoscopic PA and US imaging system. Utilizing a high repetition rate pulsed laser, optimized rotary joint as well as proximal scanning method, this integrated imaging system is able to obtain morphological tissue information and vasculature of the GI tract simultaneously at a high imaging speed up to 50 frames/s (the fastest speed reported to date). We conducted in-vivo animal studies to demonstrate the performance of our imaging system for evaluating the GI tract.

II. METHODS

One of the key determining factors of clinical translation is the imaging speed. To acquire quality images for accurate disease detection, high speed imaging is essential as it can minimize the motion artifact caused by breathing and rectal peristalsis. In addition, increasing imaging speed hence improving imaging area helps physicians visualize a larger sections of GI tract in a shorter period of time. Currently, the imaging speed of an endoscopic PA/US system is what limits the translation to clinical application. In this study, several improvements of an endoscopic PA/US imaging system were made to achieve a higher imaging speed with good imaging quality. Most current endoscopic probes for GI application utilize distal scanning with a micromotor, which is difficult to achieve a high rotation speed in water [16][21]. To address this issue, a proximal scanning method utilizing torque coil to transmit the torque from a rotary motor was applied to drive the imaging probe, providing a rotation speed up to 100 revolutions per second in water. A 10 W pulsed laser with a repetition rate up to 300 kHz was used as the excitation source to perform photoacoustic imaging. In consideration of the laser energy loss caused by high speed rotation, we customized the optical rotary joint to maintain a high transmission efficiency of laser energy for a rotation speed up to 50 revolutions per second. High speed rotation also generated a higher noise level that degraded the sensitivity of the imaging system, hence the slip ring, the motor driver, and the motor were covered by magnetic shielding foils to enhance electromagnetic shielding. Additionally, instead of the conventional B-scan averaging which greatly decreases the imaging speed, an algorithm was developed for residual electrical noise removal. Lastly, a gradient index (GRIN) lens was used to collimate the illumination light, providing improved image resolution and system sensitivity.

Figure 1 illustrates the overall setup of the integrated endoscopic PA/US imaging system (a), schematic (b), and photograph (c) of our imaging probe. In the system, a 532-nm nanosecond laser (DCH-532–10, Photonics Industries International Inc.) with a repetition rate up to 300 kHz is utilized for PA signal excitation. The output laser beam is focused by a condenser lens into the multimode fiber (MMF) of the imaging probe. The MMF is used to deliver the laser energy. A custom-made, single-element transducer (0.7 × 0.7 × 0.5 mm³ with an active element area of 0.5 × 0.5 mm², 45 MHz center frequency) is used to detect the photoacoustic and ultrasound signals from the biological tissue. The trigger signal from the pulsed laser is used as the main trigger to synchronize data acquisition and laser emission. Simultaneously, the main trigger signal is delayed by 5 μs to trigger the ultrasound pulser/receiver (DPR500, JSR Ultrasonics) to emit acoustic waves for ultrasound imaging. The generated PA and US signals are band-pass filtered, amplified, and digitized with a data acquisition (DAQ) card (ATS9350, Alazar Technologies Inc). in a personal computer. In order to obtain cross-sectional images (B-scans), we applied a proximal scanning method in which the imaging probe is rotated through a rotary joint. The rotary joint is assembled with a custom-made electric slip ring (Hangzhou Prosper Electric Co.,Ltd.), a fiber optic rotary joint (Princetel, Inc.) and a rotary motor (MicroMo Electronics, Inc.) which allow the laser beam and electrical signal to pass across rotating interfaces. In consideration of increased electrical noise and laser energy loss caused by high speed scanning, we customized a fiber optic rotary joint which is able to keep high transmission efficiency of laser energy with a rotation speed up to 50 revolutions per second. Furthermore, we have made improvements to enhance electromagnetic shielding and developed an algorithm to remove electrical noise instead of taking an average which greatly decreases the imaging speed. The algorithm separates the noise and the signal by correlating two adjacent B-scan images, in which the noise is differentiated by its randomness and thus the signal can be extracted from the original data. In addition, spiral three-dimensional (3D) images can be obtained by a pull-back imaging probe using a translation stage. The software is written entirely in C++ for data acquisition, image processing, and display in real-time using a graphics processing unit.

Fig. 1.

The setup of integrated imaging system (a), schematic (b), and photograph (c) of our imaging probe. 3D scanner consists of fiber optic rotary joint, slip ring, motor, and pull-back translation stage.

In the probe, the laser beam propagates through the MMF, collimated by a 1-mm GRIN lens (Aviation Magneto Optical Sensor Corp.), and reflected by a rod mirror (Aviation Magneto Optical Sensor Corp.) with a diameter of 1 mm at an angle of 45° towards the tissue surface. The laser pulse energy emitted from the imaging probe is maintained to be ~30 μJ throughout the study. In consideration of water absorption and astigmatism caused by the sheath, the corresponding fluence on the rectum is 15 mJ/cm², which is well within the American National Standard Institute (ANSI) safety standard (20 mJ/cm²) in the visible spectrum (400–700 nm) [22]. A miniature custom made single-element ultrasonic transducer is used to detect the PA waves from the sample as well as to perform pulse-echo US imaging. Both transducer and rod mirror are tilted at a small angle in order to obtain optimized overlap between optical and acoustic beams. The outer diameter and rigid length of the imaging probe are 1.5 mm and 11mm, respectively. The length of imaging probe is 50 cm. A double-wrapped torque coil (ID: 0.4 mm, OD: 0.8 mm, Asahi Intecc USA, Inc.) is connected to the distal end of the imaging probe to transmit the torque from the rotary motor to perform cross-sectional images (B-scans) with high imaging speed up to 50 revolutions per second (RPS). Compared with the distal scanning method that applies a micro motor to drive the mirror, the proximal scanning method has full field of view imaging, improved flexibility and high imaging speed for the endoscopic PA/US system.

III. EXPERIMENTS AND RESULTS

In order to demonstrate the performance of our integrated endoscopic PA/US imaging system, we conducted an in-vivo experiment to image the rectum of a SD rat. The rat was placed under general anesthesia by IP injection of ketamine hydrochloride (87mg/kg) and xylazine (10mg/kg) through a 29G needle. After the rat was anesthetized, we performed enemas to clean the rectum and then inserted our imaging probe with sheath for in-vivo imaging. All methods were carried out in accordance with the University of California, Irvine (UCI) Institutional Review Board (IRB) and the Institutional Biosafety Committee (IBC). All experimental protocols were approved by the UCI IBC under protocol #2016–3198.

Figure 2 shows the representative PA and US images. The transverse resolution of the PA imaging is ~250 μm, which is determined by the optical beam size. For the US imaging, the transverse resolution is ~ 300 μm, which is mainly governed by the ultrasound transducer size. The axial resolutions of the PA and the US systems both depend on the bandwidth of the ultrasound transduce and are approximately 50 μm. The imaging depth is ~4 mm, determined by the overlapping range between the optical beam and the acoustic wave. At the optimum imaging depth in which the optical beam and acoustic wave are fully overlapped, the signal to noise ratio (SNR) of the PA and US systems are ~45 dB and ~42 dB, respectively. The detailed methods for measuring these parameters are described previously [23]. We acquired ~ 500 B-scan images with a pullback speed of 0.5 mm/s. For groups I, II, and III, a 20 frame per second (FPS) was used to perform B-scan imaging. For group IV, a 50 FPS was applied to perform imaging. From the US images of the four groups [Figure 2 (Ia), (IIa), (IIIa), and (IVa)], the typical layered architecture indicated by the white dashed box and seminal vesicles that correspond to low echo in US images indicated by the white arrow can be identified. From the PA images [Figure 2 (Ib), (IIb), (IIIb), and (IVb)] of the four groups, the signal of blood vessels present in different layers can be found. Fused PA and US images [Figure 2 (Ic), (IIc), (IIIc), and (IVc)] provide the co-registration images, which are advantageous over either modality alone to supplement lesion evaluation. These results demonstrate that this integrated endoscopic imaging system has the capability to visualize the layered architecture and vasculature of the rectum wall simultaneously. A video in the Supplementary Information shows the PA, US, and fused B-scan images while pulling back the imaging probe.

Fig. 2.

Cross-sectional PA, US, and fused images with different locations along pullback direction. (a) US images. (b) PA images. (c) Fused images. White dashed box: typical layers of rectum wall. Group I, II, and III were obtained with 20 frames per second (FPS). Group IV was obtained with 50 frames per second. White arrow: surrounding organ. Scale bar: 1 mm (see Visualizations 1 and 2).

Figures 3 (I), 3 (II), and 3 (III) show representative 3D PA, US, and fused images, respectively, of the rectum. Figure 4 shows an unwrapped image from Figure 3(e); the pattern of vasculature can be observed. The entire process of imaging only takes ~ 10 seconds, and no averaging was applied. From the 3D US images [Figures 3 (IIa), (IIb) and Figures 4 (IIa), (IIb)], morphology of the rectum wall and surrounding organ can be identified. The white dashed box indicates the typical layered architecture, and the white arrow indicates the seminal vesicles. From 3D PA images [Figures 3 (Ia), (Ib) and Figures 4 (Ia), (Ib)], vasculature of the rectum wall was found.

Fig. 3.

3D endoscopic PA, US, and fused images of the rat rectum. (Ia) and (Ib) PA images. (IIa) and (IIb) US images. (IIIa) and (IIIb) Fused images. White arrow: surrounding organ. Scale bar: 1 mm.

Fig. 4.

Unwrapped 3D endoscopic PA, US, and fused images of the rat rectum. (I) PA images. (II) US images. (III) Fused images. White arrow: surrounding organ. Scale bar: 1 mm.

IV. SUMMARY

Endoscopic integrating PA and US imaging is a minimally invasive non-ionizing imaging technology that has the potential for the diagnosis and classification of GI disease. Here, we reported an integrated endoscopic PA/US imaging system which is able to provide information of tissue structure and vasculature of GI tissues simultaneously. Utilizing a high repetition rate pulsed laser, optimized rotary joint as well as a proximal scanning method, a high speed integrated endoscopic PA/US imaging system was obtained. The outer diameter of the imaging probe is around 1.5 mm, which is accessible through the accessory channel of the commercial endoscope. The results obtained from the in-vivo rat experiment demonstrated that the typical layered architecture and vasculature can be identified by this integrated imaging system. While our PA/US imaging system has laid the groundwork for clinical imaging, several challenges still need to be addressed for clinical integration. (1) Resolution: to visualize the microvasculature of the rectal wall, the transverse resolution of the PA imaging needs to be improved. Furthermore, to accurately demarcate the tissue layers, axial resolution of both modalities has to be improved as well, which may be achieved by employing a higher frequency acoustic transducer. (2) Probe form factor: for deeper GI tract imaging (e.g., small intestine), the diameter and the rigid length of the imaging probe need to be further minimized to ensure a smooth insertion. This may be achieved by using a GRIN fiber with a better flexibility and a smaller diameter to focus the optical beam. (3) Sensitivity: for higher speed imaging (> 50 RPS), the performance of a fiber optic rotary joint and the slip ring will also need to be further optimized. In addition, the overlap between the optical beam and the acoustic wave can be further improved to enhance the detection efficiency through the entire imaging depth, and this may be accomplished by employing coaxial imaging. Lastly, a diseased animal model with in-vivo imaging is needed for further verification.

Biographies

Yan Li received her B.S and M.S degrees from the Tianjin University and Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Science, China, in 2011 and 2014, respectively. Currently, she is a Ph.D. candidate working under the guidance of Dr. Zhongping Chen. Her interests include multi-modality endoscopic imaging by combining ultrasonic and optical techniques, such as ultrasound, optical coherence tomography, photoacoustic imaging, and fluorescence.

Zhikai Zhu is a senior student in South University of Science and Technology, China. He is currently an undergraduate researcher in the Beckman Laser Institute and the Department of Biomedical Engineering at University of California, Irvine (UCI). His current research interests include multi-modality endoscopic imaging system such as optical coherence tomography and photoacoustic imaging.

Joseph Jing received his Ph. D. degree from the Department of Biomedical Engineering at the University of California, Irvine in 2016. He is currently a Postdoctoral Scholar in the Medical Engineering Department at Caltech. His research interests include the development of novel high speed imaging technologies for in vivo biomedical applications.

Jason J. Chen received his bachelor’s degree from the University of California, Irvine in 2013. He worked as a research specialist for Dr. Brian J.F. Wong at Beckman Laser Institute until he started his PhD program in 2017. His main research focus is optical coherence tomography, including optical system design, software analysis, and applications in otolaryngology and ophthalmology.

Emon Heidari received his M.S. degree in Biomedical Engineering from the University of California, Irvine in 2016. He is now finishing up his Ph.D. in Biomedical Engineering at the University of California Irvine. His research interest includes biomedical imaging and medical device designed to address clinical unmet needs in head and neck cancer as well as interventional pulmonology.

Youmin He received the B.S and M.S degrees from the School of Electronics and Information Engineering, Beijing Jiaotong University, Beijing, China, in 2011 and2014, respectively. In 2017, he received the M.S. degree in biomedical engineering from the University of California, Irvine, CA, USA and is currently a Ph.D. candidate working under the guidance of Dr. Z. Chen. His research interests include fast image segmentation, parallel computing for real time OCT imaging, high resolution OCT system for airway imaging, and ARF-OCE technology development and application on ophthalmology and cardiology.

Jiang Zhu received the Ph.D. degrees in biology from Tsinghua University in 2009. Currently, he is now a postdoctoral researcher in Beckman Laser Institute. His interests include optical coherence elastography and Doppler optical coherence tomography.

Teng Ma received his B.S.E. degree in 2011 from the University of Michigan, Ann Arbor, MI, majoring in biomedical engineering. He received his Ph.D. degree in biomedical engineering from the University of Southern California, Los Angeles, CA, in 2016. He is currently a Professor at Shenzhen Institutes of Advanced Technology. His research interests include medical ultrasound technology and multi-modality intravascular imaging by combining ultrasonic and optical techniques, such as intravascular ultrasound (IVUS), intravascular optical coherence tomography (IV-OCT), intravascular photoacoustic imaging (IVPA), and acoustic radiation force optical coherence elastography (ARF-OCE).

Mingyue Yu received a B.E. degree from Tianjin University, China, in 2009. She received her Ph.D. degree in biomedical engineering from the University of Southern California, Los Angeles, CA, in 2017. Her interest include high frequency ultrasonic transducer technology and intravascular ultrasonic imaging.

Qifa Zhou received his Ph. D. degree from the Department of Electronic Materials and Engineering of Xi’an Jiaotong University, China in 1993. He is currently a Research Professor at the NIH Resource on Medical Ultrasonic Transducer Technology and the Department of Biomedical Engineering and Industry & System Engineering at the University of Southern California (USC), Los Angeles, CA. Before joining USC in 2002, he worked in the Department of Physics at Zhongshan University in China, the Department of Applied Physics, Hong Kong Polytechnic University, and the Materials Research Laboratory, Pennsylvania State University. Dr. Zhou is a fellow of International Society for Optics and Photonics (SPIE) and American Institute for Medical and Biological Engineering (AIMBE).He is also a senior member of the IEEE Ultrasonics, Ferroelectrics, and Frequency Control (UFFC) Society and a member of the UFFC Society’s Ferroelectric Committee. He is a member of the Technical Program Committee of the IEEE International Ultrasonics Symposium. He is an Associate Editor of the IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control. His current research interests include the development of ferroelectric thin films, MEMS technology, nano-composites, and modeling and fabrication of high-frequency ultrasound transducers and arrays for medical imaging applications, such as photoacoustic imaging and intravascular imaging. He has published more than 130 journal papers in this area.

Zhongping Chen received the B.S. degree in applied physics from Shanghai Jiao Tong University, Shanghai, China, in 1982, the M.S. degree in electrical engineering from Cornell University, NY, USA, in 1987, and the Ph.D. degree in applied physics from Cornell University in 1993. He is currently a Professor of biomedical engineering and the Director of F-OCT Laboratory at the University of California, Irvine, CA, USA. He is a cofounder and the Board Chairman of OCT Medical Imaging, Inc. His research interests encompass the areas of biomedical photonics, microfabrication, biomaterials, and biosensors. His research group has pioneered the development of functional optical coherence tomography, which simultaneously provides high-resolution 3-D images of tissue structure, blood flow, and birefringence. He has published more than 220 peer-reviewed papers and review articles and holds a number of patents in the fields of biomaterials, biosensors, and biomedical imaging. Dr. Chen is a Fellow of the American Institute of Medical and Biological Engineering (AIMBE), a Fellow of SPIE, and a Fellow of the Optical Society of America.