Real-time, label-free, intraoperative visualization of peripheral nerves and micro-vasculatures using multimodal optical imaging techniques

Abstract

Accurate, real-time identification and display of critical anatomic structures, such as the nerve and vasculature structures, are critical for reducing complications and improving surgical outcomes. Human vision is frequently limited in clearly distinguishing and contrasting these structures. We present a novel imaging system, which enables noninvasive visualization of critical anatomic structures during surgical dissection. Peripheral nerves are visualized by a snapshot polarimetry that calculates the anisotropic optical properties. Vascular structures, both venous and arterial, are identified and monitored in real-time using a near-infrared laser-speckle-contrast imaging. We evaluate the system by performing in vivo animal studies with qualitative comparison by contrast-agent-aided fluorescence imaging.

1. Introduction

Accurate identification, precise dissection, and careful preservation of critical structures, such as nerves and blood vessels, are key to successful surgical outcomes [1]. Unintended and/or unrecognized injuries to critical structures result in debilitating short- and long- term morbidity, avoidable mortality, and considerable socioeconomic and healthcare burdens [2–4]. These tissue injuries occur most frequently intraoperatively, and are mainly attributable to human factors such as technical and/or cognitive competence in executing surgical tasks in real-time. Although surgeons can look up detailed information during surgery for complex structures and visual aids are often available, identification of relevant and critical anatomic structures and pathologies can be further limited by significant variations in individual surgeon’s training, experience, and clinical volume [5–7].

Nerve injury is a significant complication associated with many surgical procedures, including prostatectomy, thyroidectomy, hernia repair, and breast cancer surgery, and can affect up to 16.4% of patients [1–4, 8–12]. For example, in the case of gynecologic surgery, nerve injuries are a common complication, occurring in 1.1–1.9% of cases [13]. Although the majority of neuropathies resolve with conservative management and physiotherapy [14], however severe injury cases such as axonotmesis can sustain long-term morbidity requiring prolonged rehabilitation or additional reparative surgery [15]. Other traumatic neuropathies and surgical nerve reconstructive techniques can also be found in [16–18]. These complications, leading to loss of function, sensation, muscular atrophy, and/or chronic neuropathy, considerably impair patient quality of life [2, 19–22]. Intraoperative nerve injuries are often caused by surgeon’s limited ability to identify and distinguish nerve fibers from the surrounding tissues. Nerves often appear as a network of iridescence rather than distinct anatomical structures, or distorted or displaced by adjacent pathology such as malignancy. Surgeons usually recognize damage to nerves of interests post facto (unintended injury) [12]. Intraoperative placements of neural probes are time-consuming and disrupt workflow, and lack anatomic and physiologic accuracy [23]. Current nerve-sparing techniques rely primarily upon anatomic landmark identification by surgeons, which is highly dependent on each surgeon's memory based on experience and training, or intraoperative electrical stimulation devices with unknown precision [24–27]. Thus, the development of new imaging tools, that quickly identify and clearly display peripheral nerves in real-time during surgery, is of paramount importance.

Recently, there have been interesting studies to identify nerves using label-free optical imaging techniques such as polarization sensitive optical coherence tomography [28] and collimated polarized light imaging (CPLi) [29]. Islam et al. extracts the subsurface structural features of ‘Bands-of-Fontana’ from rat sciatic nerve using a polarization-sensitive spectral domain optical coherence tomography. Although PS-OCT has a limited image volume and short working distance, the authors successfully demonstrated a 3D imaging of nerve structural deformation under stretched conditions. In contrast, Chin et al. used a collimated polarized light to differentiate anisotropic optical properties of nerve structures and highlighted a real time intraoperative nerve imaging capability with wide field of view and longer working distance, which permits surgical interventions into the deep tissue areas. However, this work has limited to macroscopic observations of human cadaver tissue, dropping the opportunity to recognize unique optical signatures of peripheral nerves, ‘Bands-of-Fontana.’, that can be an indicator for the nerve injury [30].

Misidentification of blood vessels occur in all specialties, and have potential life threatening outcomes [31]. Anatomical variations and local pathologic conditions, such as inflammation and tumor infiltration, limit vessel-visualization and lead to injuries. Although repairs to damaged blood vessels are performed immediately, and redundancies in blood supply can often compensate for sequelae of acute ischemia, complications requiring additional treatment due to hemorrhaging, necrosis, abscesses, and strictures are not always avoidable [31–33]. Pre-operative “road-maps” using complex medical imaging devices, such as computed tomography (CT) and magnetic resonance imaging (MRI), can act as intraoperative guides, however, the time latency between imaging and surgery increases the likelihood of inaccuracies [34–37]. Therefore, accurate identification and clear, precise display of blood flow and tissue perfusion in real-time during surgical procedures would tremendously benefit surgeons by facilitating quick decision-making, reducing operative times, and improving surgical outcomes.

Intraoperative identification of critical structures, such as vessels, bile ducts, ureters, nerves, and lymph nodes, has been successfully demonstrated using fluorescence image-guided surgery, both in preclinical and clinical studies [38, 39]. Optical contrast agents are usually coupled with optical imaging instruments to provide real-time visualization and to further increase the contrast of otherwise indistinguishable anatomic features. However, there are currently only a few fluorescent dyes approved for clinical use [40, 41]. The use of current intraoperative dyes is further complicated by the need for preparation times, costs, dose constraints, short half-cycles, allergic reactions, injection timings, and the inability to conduct multiple-use cases [42–45]. Dye-free, intraoperative visualization of critical hidden, sub-surface structures in real-time obviate the above disadvantages of dye based visualization and would be ideal most surgical procedures [46–49].

In this study, we introduce a new multimodal optical imaging device capable of highlighting peripheral nerves and micro-vessels in real-time on the screen of surgical fields-of-view, without any labeling dye but can also extend the capability of fluorescence imaging. We integrate the anisotropic optical properties of peripheral nerves, captured and processed by a snapshot polarization imager, and real-time processing and visualization of microvascular morphology and tissue perfusion in vivo by a graphics processing unit (GPU)-accelerated invisible laser-speckle-contrast imaging (LSCI). We present the optical layouts, preclinical validation studies, and performance evaluation (using rodent and swine models) of the imaging system. Finally, we discuss the benefits, limitations, and potential applications of the current design and technology in various surgeries.

2. Materials and methods

2.1. System design

Surgical microscopes utilizing LSCI and fluorescence have been previously reported [50–52]. To further enable multimodal imaging, and its intraoperative use, we used an existing surgical OPMI S5 microscope (Carl Zeiss, Germany) with additional camera ports (as shown in Fig. 1 ). The microscope includes a 250-mm focal length main objective (obj.) lens and 2 additional camera ports, located on the left and right arms, split by a virtual beam-splitter (BS1) connected to the main body of the microscope. The optical path of the left arm was divided into two, for CAM1 (NIR LSCI, GS3-U3-41C6NIR-C, FLIR, U.S.A.) and CAM2 (HD Color Vision, GS3-U3-41C6C-C, FLIR, U.S.A.), by a 50:50 beam-splitter coupler (BS2; Carl Zeiss, Germany); CAM1 operates at 90 frames per second with a 2048 x 2048 pixel-image-size and CAM2 acquires images at 30 frames per second, with a 2048 x 2048 pixel-image-size. CAM1 also contains a laser-line-filter (830 nm cut) to pass the laser light wavelength only, and a near-infrared linear-polarization-filter (LP2, Thorlabs, New Jersey, U.S.A.) to handle and minimize Fresnel surface reflections from wet tissues. The right arm included an additional observance arm to split the beam path into two, for CAM3 (NIR Fluorescence, GS3-U3-41C6NIR-C, FLIR, U.S.A.) and CAM4 (PolarCAM M, 4D technologies, U.S.A.), by video couplers (Quintus Zeiss adapter, Oprtornics, U.S.A.). CAM3 operated at 30 frames per second with a 2048 x 2048 pixel-image-size, including both long-pass (cut-off wavelength: 800 nm) and band-pass (790/30 nm) filters to pass only emission-fluorescence signals. CAM4 was a snapshot micropolarizer camera operating at 35 frames per second with a 1500 x 1200 pixel-image-size, with 4 micro-patterned linear-polarization-filters on the charge coupled device (CCD) camera sensor, enabling real-time polarization calculations and imaging of birefringence [53].

Fig. 1.

Optical layout and implementation of the proposed system using a surgical microscope. a. Photo of the system implementation. b. Optical design. BS1, BS2, BS3: virtual beam-splitters; E1&2: eye pieces; L1: plano-concave lens; LF: laser line filter; LP&BP: long-pass filter and band-pass filter; LP1&2: linear polarizers; OA: observation arm; Obj.: main objective lens.

For illumination, we used a built-in microscope illuminator (Halogen bulb) with a short-pass filter (cut-off wavelength: 800 nm) to avoid cross-talk issues, and a near-infrared laser diode (P = 5 mW, 830 nm; Wuhan, China) equipped with a plano-concave lens (LC1054-B, Thorlabs, New Jersey, U.S.A.) and linear polarizer (LP1, Thorlabs, New Jersey, U.S.A.) to illuminate the surgical area. In addition, UV/NIR LED arrays (Center wavelengths: 380 nm and 750 nm; Larson Electronics, Texas, U.S.A.) were selectively used for generating fluorescence-image-control-data.

2.2. Image processing for real-time nerve and vasculature imaging

Polarization light can reveal diagnostic information on tissue morphology by enabling subsurface imaging [49]. For nerve identification, we used a camera employing a micropolarizer array, with linear polarizers oriented at 0, 45, 90, and 135 degrees [53]. The pixelated polarization camera acquires four polarization orientations in a single video frame, which enables instantaneous measurements of linear-Stokes-parameters [54]. The Stokes vector is a convenient method for describing the polarization state of a light beam, as each element can be conveniently measured. Combined with the Muller matrix, which describes the transfer function of an optical device, one can easily compute the complete polarization characterization of a medium; which can be further decomposed to enable interpretation of depolarization, de-attenuation, and retardance, including birefringence and optical rotation [55]. In this study, we used phase retardance (δ) to calculate birefringence imaging and differentiate the peripheral nerves from surrounding tissues (Eq. (1)), where the ${\mathit{M}}_{\mathit{R}}$ indicates retardance matrices (full analysis is available in [48, 56]). Peripheral nervous system (PNS) contain anisotropic properties of myelin sheaths and internal Bands-of-Fontana structures which can be detected by depth-selective polarization imaging [30, 56].

$$\delta ={\text{COS}}^{-1}(\sqrt{{\left({\mathit{M}}_{\mathit{R}}22+{\mathit{M}}_{\mathit{R}}33\right)}^2+{\left({\mathit{M}}_{\mathit{R}}23-{\mathit{M}}_{\mathit{R}}32\right)}^2}-1)$$ (1)

where M_Rij indicates the element of Muller matrix, i and j are each column and row of the matrix. For real-time vasculature imaging, a general-purpose graphics processing unit (GPGPU –Titan X, provided by NVIDIA) was used to accelerate the processing speed through parallelization. In most previous research works, LSCI technique has been performed off-line due to the heavy computation required for high-resolution image processing and the serial processing of central processing unit (CPU) has insufficient power to accomplish this in real-time. However, for clinical applications, real-time processing and visualization are critical. Thanks to advancement of new technology, GPU now allows a parallel processing and memory latency can be mitigated by employing multiple processing core units. In our implementation, the raw laser-speckle data acquired from CAM1 was transferred, as a normalized array-bound texture with 32-bit floating point values, into the shared GPGPU memory. A spatial contrast value was then computed for each pixel of the array, using a sliding window of variable size (either 5 x 5 or 7 x 7 pixels). The following results were performed on our PC equipped with Intel^® Core i7-8700K Processor, 16G main memory and one GeForce Titan X graphics card. In 2048 x 2048 pixels size, 16 bit-depths image processing, with the spatial window size of 5 x 5, the GPU processing speed (11.13-ms per image) was approximately 67.2 times faster than CPU (748-ms per image). To better visualize the newly computed array of spatial contrast values, heat-mapping was applied based on the normalized spatial contrast value. This heat-map was stored as a 32-bit packed integer, representing red, green, blue, and alpha channels (RGBA). Following synchronization of all parallel processing cores, the final RGBA heat-map was copied back to the host machine, and stored as memory in a dynamic buffer (Fig. 2 ). This dynamic buffer provides sufficient capacity to store up to the 30 most-recently computed heat-maps. To mitigate noise and extract greater signal fidelity in the time domain, temporal blending of these buffered frames was applied, using a fixed and equal weight, to the alpha channel of each stored frame. The final blended image was composited and displayed to the user via an OpenGL-based frontend user interface.

Fig. 2.

The overall processing framework of image analysis based on a graphics processing unit. CPU: central Processing Unit; CUDA: parallel computing platform; GPU: graphics processing unit; NIR: near-infrared.

2.3. Software development for graphical user-interface

Graphical user-interface (GUI) bindings and controls for the camera, data recording, and processing parameters were custom developed in native POSIX C + + , using OpenGL as the primary backend for rendering and compositing both the user interface and processed image data. The frontend software (‘STARcontrol’) was targeted for 64-bit Linux/UNIX systems. STARcontrol also served as a hub for inter-process communication (IPC) between the many different software modules required for the entire processing stack to run (e.g., taking data from the hardware layer – cameras, applying preprocessing, passing data to LSCI module, etc.). This also provides the user with the ability to adjust any operational parameter dynamically and easily, as all were bound to specific and easy-to-navigate elements in the GUI (Fig. 3 ).

Fig. 3.

Graphical user interface (imaging mode options: color, LSCI, polarization, and fluorescence; image field of view; camera selection and control; multimodal image alignment using the openGL function). LSCI: laser-speckle-contrast-imaging

The OpenGL backend also provided the ability to easily apply morphological transformations to the imaging data in real-time (e.g., rotation, translation, zoom, etc.), enabling rapid alignment of different image planes, captured from different sources and sensors. In addition to morphological changes, STARcontrol also provided the user with the ability to change global alpha, blending attributes, and color overlays in each of the imaging planes. These features collectively provided the user with the capacity to tailor the visualization of the scene to their specific needs, applying different visual weights to various imaging modes to highlight specific features (e.g., subsurface vasculature extracted via LSCI over surface features).

2.4. Animal procedures and fluorescence imaging

2.4.1. Nerve, vein, and artery imaging in ex vivo porcine tissue

We acquired various sized porcine nerves, tendons, veins, and arteries and dissected the tissue region of interests using a sharp scalpel; we then conducted imaging of the regions using our system. Fresh samples were obtained from a local abattoir and dissected into tissue segments of interest. The sample was moistened with physiological saline and preserved at 4°C for up to 30 h from the time of slaughter until imaging. Before imaging, different segments of the porcine tissues were dissected using forceps and dissectors. During the measurements, the remaining samples were preserved in saline in sealed sterile containers for hydration maintenance, for up to 30 min.

2.4.2. Rat anesthesia and exposure for sciatic and femoral nerves, arteries, and veins

All procedures were performed in the animal research facility under institutional animal care and use committee guidelines (IACUC) #30597. Male and female 250 - 350 g Sprague-Dawley rats (n = 4) from Charles River Laboratories (Wilmington, Massachusetts, U.S.A.) were used for this experiment. A 3-min inhalation of 4% Isoflurane was used for sedation and restraint. Anesthesia was maintained using intramuscular injections of 2 mg/kg Xylaxine and 75 mg/kg Ketamine. After ensuring sterile conditions, femoral nerve and vessels were exposed at their junction with the inguinal ligament, and imaged with our vision system. For small bowel imaging, a mid-line laparotomy was performed and small part of the bowels were exposed and imaged under our imaging system.

2.4.3. Pig anesthesia and exposure for liver hilum

Our institutional animal care and use committee approved our protocol (IACUC #30591). All procedures were performed in the animal research facility. Female Yorkshire 10 kg pigs (n = 2) from Archer Farms (Darling, Maryland, U.S.A.) were used for this experiment. As the peripheral nervous system in the body is similar between female and male, we used available animals at the time of our experiments. Pigs were sedated by intramuscular injection of xylazine-ketamine anesthetic. A 3-min inhalation of 4% Isoflurane was used and maintained for anesthesia. After ensuring sterile conditions, midline laparotomies were performed. Part of the liver was exposed and imaged with our imaging system.

2.4.4. Fluorescent dye injections in rat

For rats, as a control test and comparison study, a 24-G catheter was placed in the tail vein to visualize nerves and vessels by intravenously injected fluorescent dyes. To visualize sciatic and femoral nerves in rats (n = 2), an 8-mg/kg injection of bis-benzenamine (BMB) was performed at the beginning of the procedure and fluorescence was recorded 2 hours later [8]. For vasculature imaging (n = 2), a 0.15-mg/kg injection of indocyanine green (ICG) (MP Biomedical, LLC, Solon, Ohio, U.S.A.) was used for fluorescence angiography and immediately imaged after injection for 5 minutes using an NIR-camera [52].

3. Experimental results

3.1. Ex vivo and in vivo imaging of peripheral nerves

To test and calibrate the polarimetric imaging setup, we first tested excised ex vivo porcine nerves, tendons, and vessels. To demonstrate the advantage, we selected and dissected pelvic nerves, veins, and arteries, and placed them together for comparison and contrast. Polarimetric imaging enables real-time identification of the pelvic nerve, and highlights the subsurface structures (as shown in Fig. 4 ). The pelvic nerve shows the intrinsic structure of ‘Bands-of-Fontana’ and we can use this optical signature of spiral patterns for feature extraction, and to distinguish the nerves from other surrounding tissue types. As highlighted in Fig. 4(b), individual fascicles inside the nerve tissue were identified by their ‘zig-zag’ spiral pattern, but we were unable to see the subsurface fascicles due to the limited sensitivity of depth-imaging.

Fig. 4.

Polarimetric imaging for subsurface pattern extraction. (a) Color photo taken with the color camera. (b) Representative Retaradance map from birefringence imaging; pelvic nerve branches, with individual fascicles, are highlighted by the intrinsic zig-zag spiral patterns of ‘Bands-of-Fontana’. Please note that sub-surface fascicles within pelvic nerve are identified and differentiated. White bars: 1 cm.

Sciatic nerves were visualized and distinguished using our polarization imaging tool using live rats (n = 2), and the result was compared to fluorescence imaging using BMB dye (n = 2) (Fig. 5(b) ). BMB dye highlights not only peripheral nerve structures but also surrounding tissues, such as fat. By contrast, Fig. 5(c) shows that label-free polarimetric imaging and processing enables the extraction of hidden optical signatures from sciatic nerves, distinguishing them from surrounding tissues. This was achieved by phase retardation calculation and image processing kernel including feature extraction and image segmentation.

Fig. 5.

Imaging of sciatic nerve in rats. (a) Color image of sciatic nerve in a rat. The spiral pattern is barely seen. (b) BMB injected fluorescence image. (c) Polarimetric kernel processed image highlighting ‘Bands-of-Fontana’. (d) Overlay image. White bars: 5 mm.

3.2. Small bowel vasculature imaging in a rat model

To demonstrate if microvasculature can be visualized in real-time, we imaged small intestine using rat models. We first tested bowel perfusion using the developed system and compared with dye-based fluorescence imaging. As shown in Fig. 6 [Visualization 1 ^{(9.7MB, mp4)} ], the LSCI module enables instantaneous visualization of both arteries and veins, whilst fluorescence imaging using ICG injection only permits temporary visualization of arteries and veins (ICG half cycle: 3–4 seconds) [39, 44]. With regards to temporal resolution, LSCI enables seamless visualization of blood flow, both in veins and arteries, whilst fluorescence imaging is only available for 3–5 seconds after systemic injections. LSCI permitted the visualization of fine details, both from mesentery and small arcades, throughout the bowel tissues; however, the contrast of fluorescence imaging was blurry due to the diffused fluorescent light emitted from the tissue and it was difficult to identify fine vasculatures.

Fig. 6.

Real-time perfusion imaging of the small bowel in a rat. (a) A color photo taken from our imaging system. (b) A comparative LSCI image. (c) Overlay image onto color image. (d) Intravenous injection of ICG and fluorescence image overlay onto color image. (e) A still image of artery visualization with fluorescence signals. (f) A still image of vein visualization. (see Visualization 1 ^{(9.7MB, mp4)} ). All white bars: 2 cm.

3.3. Hepatic artery visualization in a pig model

We further tested our imaging capability in a large animal using the same imaging system; the target tissue was the liver hilum. As shown in Fig. 7 , we monitored hidden, subsurface hepatic arteries located in the liver hilum (1–3 mm deep from the surface peritoneum) and their branches extended to liver lobes in real-time from a live pig during partial liver resection surgery. In this experiment, we used a 5-mW power, 830-nm near-infrared laser light to create a speckle pattern and GPU accelerated ‘spatial speckle contrast processing’ with a 5 x 5-pixel sliding window, enabling an instantaneous display of blood flow (exposure time: 10 ms) in hepatic arteries. By overlaying the live feed of blood flow over real color video streams, blood vessels could be simply displayed and hidden key-branches of hepatic arteries identified; moreover, real-time blood flows could be monitored during surgical dissection.

Fig. 7.

Subsurface blood flow monitoring using an LSCI. (a) A color photo of the liver hilum in a live pig. (b) Real-time visualization of subsurface hepatic arteries. Hidden branches are highlighted by arrows and circles. (see Visualization 2 ^{(8MB, mp4)} ). White bars: 3 cm.

3.4. Simultaneous femoral nerve, and vessel imaging in a rat model

Our multimodal imaging system also proved to be useful for simultaneously identifying and visualizing peripheral nerves and micro-vasculatures together in vivo. Figure 8 (Visualization 3 ^{(8.1MB, mp4)} ) depicts a representative image of femoral nerve, artery, and vein in a live rat. The polarimetric imaging module enables real-time identification of femoral nerve, highlighting the structures in green, whilst laser speckle contrast imaging visualizes the blood flow in both veins and arteries in red. Interestingly, fat tissues were also highlighted during live-imaging.

Fig. 8.

Femoral nerve, vein, and artery in a rat. (a) A standard color photo of femoral area in live rat. (b) LSCI imaging mode highlighting femoral vasculatures colored in red. Please note that unseen vessels were also highlighted and femoral nerve is not visible (c) Polarimetric processed image highlighting femoral nerve. (see Visualization 3 ^{(8.1MB, mp4)} for real time multimodal imaging). Asterisk indicates femoral nerve, yellow arrow heads indicate femoral vein and white arrows are vasculatures. All white bars: 1 cm.

4. Discussion

4.1. Multimodality optical imaging integration into an operating microscope

We report a combination of polarimetry and LSCI imaging during surgery with live animals. This is achieved and demonstrated by integrating polarimetric and LSCI technologies into one system and it enabled real-time, in situ visualization of both peripheral nerve and blood flow across the inguinal regions, with no dissection of perivascular tissues and minimal mechanical disruption to the vessels. Incorporating multimodality imaging to one existing operating microscope has several advantages. Each camera shares a common optical pathway; thus, alignment of multiple sensors is relatively easy. This imaging modality can be modular; therefore, the field-of-view, zoom, and focus can be changed simultaneously, enabling the same surgical field-of-view magnifications and demagnifications by the user. In our setup, a 250-mm working distance is sufficient for surgeons to perform the animal procedures, in both small and large live animals. Our snapshot polarimetry is distinctive from the previous label-free nerve imaging techniques of PS-OCT and CPLi as we were able to differentiate individual fascicles in real time, which can play a crucial role for nerve sparing techniques. Both IR-based blood vessel (superficial and deep tissue) identification and nerve imaging (superficial) using the anisotropic optical properties are two known optical techniques which can be easily combined. However, one of the main contribution of the current study is the proposed system can also be used for fluorescence imaging (superficial and deep-tissue). In the real operation room, surgery requires situational awareness of surrounding tissues and relevant anatomy in the operative field. Our multimodal optical imaging system permits a real-time identification of critical structures without labeling but is also fully capable of fluorescence imaging, that can be complementary. For example, surgeon can use our label free imaging techniques to identify nerves and vessels in the beginning of procedures and finally confirm these anatomic structures again using a fluorescence. This alternative option is of potential benefit for surgeons, and will enable individual imaging-module-selection depending on the surgery type and imaging mode preferences.

4.2. Polarimetric imaging

Our system qualitatively demonstrated that peripheral nerve identification can be possible based on polarimetry and subsurface feature extraction of unique spiral patterns ‘Bands-of-Fontana’. Although it is clear in our first pilot demonstration using live rat model, additional works will be needed to validate the usefulness of this technology by setting up a survey or testing groups to let surgeons choose which one is nerve among other tissues in their color images, fluorescence images, and our polarimetry extracted images. Ultimately, in the near future, machine learning could be employed to computationally segment each tissue and it may increase the sensitivity. Our study is well aligned with the recent works by Chin et al. [29, 57], where the authors recently demonstrated a real-time nerve identification in the human hand. Regarding the issue of fat tissue false identification in our results, it can be caused by the polarization imaging setup. Chin et al. used a linearly polarized imaging scheme with two perpendicular linear polarizers to reject ‘Fresnel reflection from tissue surface’ whilst we used a circularly polarized illumination for the birefringence calculation, in nerve identification mode, thus failing to avoid direct reflections from fat tissue at given wavelength of 830 nm. Any birefringent part of the tissue introduces a phase delay in their spatial electric field and it creates an ellipticity in the polarization state. The illumination through perpendicular polarizers disappears when the polarizers are aligned with the major or minor axis of the birefringent part of tissues such as fat, which simply measures the birefringence and highlights nerve tissues well, as demonstrated in Chin et al. However, to find the true direction of alignment requires a rotation through 0 to 180 degrees to solve the ambiguity of which rotation angle corresponds to the major and the minor axes. We alternatively used a circularly polarized light as an input to eliminate rotation of illumination light and calculated birefringence using a snapshot polarimetry to enable real time calculation without any rotation angle changes. The tendon and collagen tissues have been well studied using a circularly polarization light to measure their retardance changes under stress [58]. On the other hand, Qi et al. also experienced the same observation in fat tissue where they speculated this as the fat tissue also demonstrates distinguished heterogeneity in the ratiometric depolarization images, which might be affected by different scattering properties originated from the fluctuating superficial structures of the fat [48]. Thus, misidentification of our current study might be associated to the wavelength dependency on incident light (absorption coefficient of fat tissue is low at 830nm). However, our imaging scheme permits individual fascicles can be differentiated, potentially critical for nerve sparing and/or placing peripheral nerve stimulation devices [24, 59]. Further image-processing and tissue-identification improvements using deep learning algorithms will be helpful.

4.3. Laser speckle contrast imaging

In our rat studies, LSCI could detect both veins and arteries in real-time by GPU accelerated processing, whilst fluorescence imaging only permitted very short display durations in each vessel at one time. Watson et al. demonstrated augmented microscopy using near-infrared fluorescence images [60]; their study required exogenous injections of fluorescent dye to visualize target structures, and temporal imaging resolutions were limited. This highlights a significant advantage of our label-free imaging method over conventional fluorescent-agent-aided-surgical-guidance and/or traditional surgery relying on surgeons’ visual abilities. Another advantage is that the method is non-contact (i.e. does not require any interaction with the tissue). Therefore, one can easily turn the technology on and off, at any time. Recently, Feng et al. proposed an off-line method that separates the arteries and veins simultaneously in cerebral blood flow imaging by the relative temporal minimum reflectance analysis of laser speckle contrast images, and the same technique can be used in our application for real-time automatic artery-vein separation by combining this method with our current setup [61]. It should also be noted that the current system and processing method cannot provide absolute blood-speed values, because LSCI signals are based on average velocities and red blood cell concentrations that require further improvement using multi exposure calculations to obtain quantitative baseline flow measures [62].

4.4. Potential applications and future works

As we have demonstrated the usefulness of our snapshot polarimetry in nerve identification in this study, the same optical technique can be benefited for other biomedical applications such as skin diagnosis, collagen and tendon imaging [58, 63]. However, these anatomic structures could further hamper the identification of nerve structures given the condition that they exhibit clear anisotropic optical behaviors. Although we have not demonstrated the full quantitative analysis in the current study but we instead introduced the feature extraction strategy of intrinsic ‘zig-zag’ spiral pattern of peripheral nerves in ex vivo sample imaging to avoid false identification thus can enhance the specificity. Our future work will focus on this study plan for the quantitative metrics in vivo. Furthermore, the real-time nerve identification technique will pave a new way for mapping peripheral nervous system and guiding surgical interventions such as placement of therapeutic and modulatory probes to relieve conditions [64]. Determining borders between viable and ischemic tissues could aid surgeons in the identification of adequate locations for anastomoses during surgery [65]. LSCI has recently gained interest in the assessment of intraoperative cerebral microcirculation, where local cerebral blood flow is measured after revascularization surgery [51, 66]. LSCI has also proven useful in the assessment of hepatic microcirculation during liver resection surgery in mice model [50, 67]. However, to the best of our knowledge, real-time visualization of sub-surface hepatic arteries has not been demonstrated in large animals such as porcine model, which are similar in physiology and anatomy to those of humans. Furthermore, full-field perfusion maps can support intravenous fluid and vasoactive drugs intraoperatively. In the future, these maps could be applied to the liver hilum, gastric vessels, and bowel ischemia, all of which are well studied using fluorescence angiography [39]. Lastly, these optical techniques rely on highly adaptable light sources; thus, they can easily be extrapolated to laparoscopes among other applications, which rely on imaging and display for minimally invasive surgery [68].

5. Conclusion

We presented a customized operating microscope that integrates the advantages of multiple optical imaging techniques into a single platform. The main benefits of polarimetric imaging combined with laser-speckle-contrast imaging in one commercially available microscope include non-invasive, real-time imaging (of critical anatomic structures), without the need for exogenous dye injections. We validated our imaging tool through multiple in vivo studies and demonstrated that label-free imaging performs comparable to conventional fluorescence imaging. In future studies, this technology should be applied for minimally invasive approaches to enhance surgical application.

Funding

Sheikh Zayed Institute for Pediatric Surgical Innovation.

Disclosures

The authors declare that there are no conflicts of interest related to this article.