Characterization and in-vivo evaluation of a multi-resolution foveated laparoscope for minimally invasive surgery

Abstract

The state-of-the-art laparoscope lacks the ability to capture high-magnification and wide-angle images simultaneously, which introduces challenges when both close- up views for details and wide-angle overviews for orientation are required in clinical practice. A multi-resolution foveated laparoscope (MRFL) which can provide the surgeon both high-magnification close-up and wide-angle images was proposed to address the limitations of the state-of-art surgical laparoscopes. In this paper, we present the overall system design from both clinical and optical system perspectives along with a set of experiments to characterize the optical performances of our prototype system and describe our preliminary in-vivo evaluation of the prototype with a pig model. The experimental results demonstrate that at the optimum working distance of 120mm, the high-magnification probe has a resolution of 6.35lp/mm and image a surgical area of 53 × 40mm²; the wide-angle probe provides a surgical area coverage of 160 × 120mm² with a resolution of 2.83lp/mm. The in-vivo evaluation demonstrates that MRFL has great potential in clinical applications for improving the safety and efficiency of the laparoscopic surgery.

1. Introduction

Laparoscopy has been established as the most successful means of providing minimally invasive surgery (MIS) due to a number of well-recognized advantages compared to the conventional open surgery, such as reduced pain, shorter recovery time and low infection rate [1]. Statistics shows that 96% of 1.06 million cases of cholecystectomy and 75% of 359,000 cases of appendectomy were performed by MIS in the United States in 2011 [2]. It has become a standard clinical procedure for cholecystectomy, appendectomy, and splenectomy.

However, the state-of-the-art laparoscopic technology suffers from several significant limitations, one of which is a tradeoff of limited instantaneous field of view (FOV) for high spatial resolution versus wide FOV for situational awareness but with diminished resolution [2]. Standard laparoscopes (SL) lack the ability to acquire both wide-angle and high-resolution images simultaneously through a single scope. With a standard laparoscope, in order to see fine details of a surgical field, laparoscopic procedures must usually be carried out at a highly zoomed-in view, where the scope is moved in to operate at a short working distance (WD), typically less than 50mm. A highly zoomed-in view leads to the loss of peripheral vision and awareness of situations occurring outside the immediate focus area of the laparoscope. One example occurs when a non-insulated laparoscopic instrument is in inadvertent and unrecognized contact with an energized instrument resulting in spread of electric current being applied to unintended structures, a situation known as “direct coupling” [3]. Insulation failures in energized instruments themselves can also directly lead to injury to bowel, vascular, and other structures. Several other serious complications of the laparoscopic surgery may occur partially due to the loss of situational awareness, including bile duct injury, bile leaks, bleeding and bowel injury. These injuries often remain unrecognized if they occur on the part of the surgical instrument which is not within the keyhole field of view (FOV) of the laparoscope [4]. At present, this limitation is clinically addressed by manually moving the entire laparoscope in and out of the camera port to obtain either close-up or wide-angle views, respectively. In addition, all of this maneuvering to adjust the laparoscope’s position requires a trained assistant to hold, move, and manipulate the laparoscope almost constantly.

The practice of frequently maneuvering the laparoscope by a trained assistant can lead to poor or awkward ergonomic scenarios; for example, having to work with hands in a cross-over position between the surgeon and the assistant holding the camera [5]. This type of ergonomic conflicts imposes inherent technical challenges to laparoscopic procedures, and it is further aggravated with the introduction of single port access (SPA) techniques to laparoscopic surgery. SPA uses one combined surgical port for all instruments instead of using multiple ports in the abdominal wall. The grouping of port access in SPA procedures, however, raises a number of challenges, including tunnel vision due to the in-line arrangement of instruments, poor triangulation of instruments, requiring crossing of instruments to obtain proper retraction, and increased risk of instrument collision due to the close proximity to other surgical devices. It has been suggested that varying the optical magnification and increasing the working distance of laparoscope can eliminate this limitation [6].

Aiming to address these limitations, several different types of laparoscopic technologies have been developed to effectively change the FOV and magnification [7,8] or to control viewing direction. For instance, some of the latest commercial laparoscopes are equipped with optical zoom capability [www.stryker.com, www.karlstorz.com] or provide different viewing directions by offering a flexible tip end [medical.olympusamerica.com]. Alternatively, miniature cameras for laparoscopic surgery have been introduced for single-port-access surgery [9]. These laparoscopic systems, however, are not able to simultaneously capture the high-magnification view and the wide-angle view. The enhanced vision laparoscopic system by Tamadazte et al can concurrently provide different views, but the high-magnification view is fixed and thus maneuvering the laparoscope during the surgery is inevitably required [10]. The dual-view endoscope prototype demonstrated by Yamauchi et al provides the ability to capture a zoomed-in view and a wide-angle view simultaneously through an image-shifting prim, but the prototype is limited by its low resolution and low light throughput [11].

Recently, we reported the development of a multi-resolution foveated laparoscope (MRFL) to address the limitations of those existing technologies. A MRFL prototype was demonstrated with a large working distance and the ability to simultaneously capture both high-magnification and wide-angle images in real-time in a fully integrated system. Additionally, the high-magnification probe can be optically scanned toward and engaged at any subfield within the wide-angle field [12]. Following up our initial prototype demonstration in [12], in this paper the overall system design of a MRFL prototype system is presented from both clinical and optical system perspectives (Section 2). In addition, we will present two sets of experiments performed recently with the prototype. The first set of experiments aims to characterize the optical performances of our prototype system (Section 3) and the second set of experiments aims to validate the clinical use of the instrument through our preliminary in-vivo evaluation of the prototype with a live porcine model (Section 4).

2. MRFL system design

From a clinical application perspective, Fig. 1 shows the conceptual layout of a MRFL with low length profile in MIS surgery. The MRFL consists of two fully-integrated imaging probes in a single laparoscope, a wide-angle probe and a high-magnification probe. The wide-angle probe with relatively low magnification captures a wide-angle overview of the abdominal cavity for orientation and situational awareness, while concurrently the high-magnification probe with narrow but adequate FOV obtains images of a sub-region of the wide-angle field at much higher resolution for accurate surgical operation. The instrument therefore provides the ability to survey a large surgical field and to visualize a targeted area in high spatial resolvability for surgical treatment. Concurrent access to both imaging scales in real time offers un-compromised context and resolution, which is expected to offer improved situational awareness and therefore better patient safety and surgical outcome.

Fig. 1.

Conceptual idea for operation of MRFL in laparoscopic surgery.

Additionally, a two-dimensional optical scanner integrated within the system can steer and engage the high-magnification probe to any region of interest (ROI) within the wide-angle field. Therefore, the MRFL system can be secured at a fixed location on the abdominal wall and no physical advancing or withdrawing the MRFL scope is needed to obtain different views. Such arrangement will not only allow MIS procedures to be performed without requiring a dedicated camera assistant or robotic arm, but will also reduce physical interference with other surgical instrument and awkward ergonomic conflicts.

Thirdly, the foveated high-magnification scope provides much more improved spatial resolution than a standard laparoscope and enables highly resolvable visualization of tissues and thus enhances intro-operative surgical decision making. It is thus expected to enable enhanced surgical technique, accuracy, and potentially reduce operation time.

Finally, it is worth noting that the MRFL is optimized to maintain a much longer working distance and low-length profile than those of a standard laparoscope. As illustrated in Fig. 1, the designed WD of a MRFL is about 120mm or larger, while the typical WD of a standard laparoscope is about 50mm for operation. The length of a low-profile MRFL is about 150mm while a SL has a typical insertion length of more than 350mm. With a longer working distance, the surgical area captured by the MRFL can be effectively increased, and the instrument can be positioned at a further distance from the surgical site to mitigate physical interferences with other surgical instrument in the abdominal cavity. The low-length profile characteristics of the MRFL system further helps to reduce instrument crowdedness.

These features discussed above on an MRFL scope are highly desirable in the SPA procedures which suffer from severe instruments crowdedness though one single trocar and the keyhole tunnel vision. Since the MRFL is secured at a large distance away from the surgical area and no physical movement is needed, it reduces the interference between the laparoscope and other surgical instruments. In addition, the multi-resolution foveated capability eliminated the keyhole tunnel vision of the standard laparoscope used in the SPA procedure.

From an optical system design perspective, Fig. 2 illustrates the schematic layout of a low-length profile MRFL system, while Fig. 3 shows the optical layout of our MRFL prototype. The optical system consists of a shared objective lens group, two shared rod lens relay groups, a scanning lens group, a high-magnification imaging probe, a wide-angle imaging probe and a beamsplitter separating the paths of the two imaging probe. The objective lens captures the entire surgical area and forms an intermediate image at its back focal plane. The intermediate image is then relayed twice by the relay lens groups to the front focal plane of the scanning lens group. The scanning lens collimates the beams and relays the pupil onto a 2D scanning mirror. A polarization beam splitter (PBS) along with a quarter wave plate (QWP) is inserted between the scanning lens and the scanning mirror for splitting the light paths for the wide-angle and high-magnification probes. The PBS reflects s-polarized light toward the wide-angle image probe and transmits p-polarized light toward the QWP and scanning mirror. The fast axis of the QWP is oriented at a 45° angle with the p-polarization axis which allows the effective conversion of incident p-polarization into s-polarization following the double pass of the QWP. The PBS then reflects the converted s-polarized light toward the high-magnification imaging probe.

Fig. 2.

Schematic layout of a dual-resolution, foveated laparoscope for minimally invasive surgery. The scope consists of a wide-angle imaging probe and a high-magnification probe. The two probes share the same objective lens, relay lens groups, and scanning lens groups.

Fig. 3.

Optical system layout of a MRFL prototype with a low-length profile, consisting of an objective lens group (a), 2 telecentric rod lens relay groups (b), a scanning lens group (c), a wide-angle probe (d), and a high-magnification probe (e).

Figure 4 demonstrates two integrated MRFL prototypes of different length profiles: a normal profile and a low profile, compared against a commercially available Karl Storz 26033AP standard laparoscope. The first-order optical design specifications of our MRFL prototype systems are summarized in Table 1. The overall optical system was optimized for a working distance of 120mm. The shared objective lens and relay lens groups were designed to capture an overall 80° FOV, with a 0.8mm entrance pupil diameter (EPD) and a 120mm working distance. The relay lens group has an optical magnification of −1. The wide-angle probe captures the entire 80° FOV imaged by the objective-relay groups, while the high-magnification probe captures a 26° FOV of interest. The ratio of the optical power of the two probes is 3 to achieve high-magnification for the foveated probe. In both imaging probes of the prototypes, a 1/3” CCD sensor (DR2-13S2C-CS by PointGrey), with 1280 × 960 pixels and pixel size of 3.75um, was utilized. A motorized 2D optical scanner (Zaber T-OMG series) was integrated to enable the ability to steer the FOV of high-magnification probe across the entire surgical field in less than 0.1 seconds in a positional accuracy of 0.1mm. The anticipated beamsplitting effect on light attenuation was partially compensated by the fact that the F/2.5 objective lens has nearly 5.76 times light collection capability as much as a typical F/6 objective lens in most SLs.

Fig. 4.

MRFL prototypes in comparison with a commercially available standard laparoscope.

Table 1. First-order optical design specifications of a MRFL prototype system.

Parameter		Value
Overall system specifications
	Working distance	120mm
	Depth of field	80-180mm
	Field of view	80°
	Visual field size at 120mm working distance
Shared objective and relay lenses
	FOV	80°
	Entrance pupil diameter	0.8 mm
	F/#	2.5
	Lens diameter	< 6 mm
	Overall packaging diameter with illumination fiber bundle	10mm
High-magnification imaging probe
	FOV	26°
	Visual field size at 120mm working distance	53 × 40 mm2
	Spatial resolution in object space at 120mm working distance	80 um or 6.25lp/mm
	Imaging sensor	Point Grey Dragonfly II, 1/3”, 1280 × 960 pixels
	2D scanner	Zaber T-OMG series, < 100ms scan across the entire field
Wide-angle imaging probe
	FOV	80°
	Visual field size at 120mm working distance	160 × 120 mm2
	Spatial resolution in object space at 120mm working distance	240 um or 2.1lp/mm
	Imaging sensor	Point Grey Dragonfly II, 1/3”, 1280 × 960 pixels

At a 120mm working distance, the wide-angle probe captures a surgical area of 160 × 120mm² which is over 9 times of the typical surgical area imaged by a standard laparoscope operating at a 50mm working distance, while it provides a spatial resolution limit of 240um or theoretical spatial frequency limit of 2.1 lp/mm which is equivalent to that of a standard laparoscope. At a 120mm working distance, the high-magnification probe captures a surgical area of 53 × 40mm² which is equivalent to the field coverage of a standard laparoscope operating at a 50mm working distance, but it provides a spatial resolution limit of 80um or theoretical spatial frequency limit of 6.25 lp/mm which is three-times as good as that of a standard laparoscope [13].

Two other aspects of the system specifications are that both the F/2.5 objective lens and the relay lens groups are required to be telecentric in the image space and the lens diameter needs to be smaller than 5mm such that they can be assembled in a standard 10mm diameter rigid laparoscope package. Adopting a standard packaging not only allows using the standard fiber illumination bundle and light sources for standard laparoscopes, but also offers the ability to assess our prototypes through a standard laparoscope trocar port and similar procedures. The telecentricity aspect further enables the ability to concatenate a flexible number of relay lens groups of limited diameter without causing noticeable image quality degradation in order to implement prototypes of different length profiles. For instance two groups of relays are needed to build the low-length profile prototype in Fig. 4 with a total insertion length of about 150mm while 4 groups of relays allow extending the insertion length to over 300mm for the normal-profile MRFL prototype in the same figure.

3. Characterization of optical performance

The optics of the MRFL prototypes was custom-designed and built and the optical system was assembled in our lab. It was optimized with a 120mm optimal WD and a working range of 80~180mm, within which the modulation transfer function (MTF) values of both the wide-angle and high-resolution probes are greater than 0.2 at the Nyquist frequency of the image sensors. This section will describe a set of experiments performed to characterize the optical performance of the prototypes, including the obtained resolution of both imaging probes at 120mm working distance (section 3.1 and section 3.2), the throughput analysis (Section 3.3), and the MTF (section 3.4). The results obtained through the experiments are summarized in Table 2.

Table 2. Optical performance characterization of a low-length profile MRFL prototype at 120mm working distance.

	Wide-angle imaging probe		High-magnification imaging probe
Field	Center	Corner	Center	Corner
Field of view (mm2)	160 × 120		53 × 40
Limiting resolution (um)	176.68	280.90	78.74	140.45
Limiting spatial frequency (lps/mm)	2.83	1.78	6.35	3.56
Throughput (mm2sr)	0.3351		0.0372

3.1 Resolution measurement of high-magnification probe at 120mm working distance

A 1951 USAF glass slide resolution target (Groups 0-3) was used to measure the limiting resolution of both the high-magnification probe and the wide-angle probe. A flat-panel LED source was used as the backlight to uniformly illuminate the target slide within a large area. A holographic diffuser with an 80° diffusing angle was placed behind the resolution target which enables the diffused light from the target across the entire field can fulfill the entrance pupil of the MRFL.

Figure 5(a) shows the captured image of the resolution target with the high-magnification probe at 120mm working distances orienting at the center field, along with the intensity profiles of three horizontal target bars in Group 2 (element 4 through 6). Figure 5(b) shows the intensity profiles of three horizontal bars in Group 2 (element 2 through 4). The contrasts of the group 2 bars in both horizontal and vertical directions are listed in Table 3. It demonstrates that when the high-magnification probe orients at the center field, the group 2 element 5 bar (6.35 lp/mm or 78.74um) can be resolved in the vertical direction, and the group 2 element 4 bar (5.66 lp/mm or 88.34um) is resolvable in the horizontal direction.

Fig. 5.

Images and intensity profiles of the resolution target of the high-magnification probe at 120mm working distance orienting at the center field, (a) in vertical direction; (b) in horizontal direction.

Table 3. Image contrast of the high-magnification probe orienting at the center field.

Contrast	G2 E6 (7.13 lp/mm)	G2 E5 (6.35 lp/mm)	G2 E4 (5.66 lp/mm)	G2 E3 (5.04 lp/mm)	G2 E2 (4.49 lp/mm)
Vertical	0.0806	0.1616	0.2760	0.3362	0.4530
Horizontal	0	0.1103	0.1429	0.24	0.3333

Figures 6(a) and 6(b) demonstrate the captured image and intensity profiles of the resolution target with the high-magnification probe at 120mm working distances orienting at the corner field in the vertical and horizontal directions, respectively. The contrasts of group 1 bars in both horizontal and vertical directions are listed in Table 4. It demonstrates that the high-magnification probe is able to resolve the group 1 element 6 bar (3.56 lp/mm or 140.45um) in both directions when it orients at the corner field.

Fig. 6.

Images and intensity profiles of the resolution target of the high-magnification probe at 120mm working distance orienting at the corner field, (a) in vertical direction; (b) in horizontal direction.

Table 4. Image contrast of the high-magnification probe orienting at the corner field.

Contrast	G1 E6 3.56 lp/mm	G1 E5 3.17 lp/mm	G1 E4 2.83 lp/mm	G1 E3 2.52 lp/mm	G1 E2 2.24 lp/mm	G1 E1 2.00 lp/mm
Vertical	0.1288	0.1371	0.1504	0.2308	0.2526	0.3666
Horizontal	0.1152	0.1965	0.2132	0.2557	0.2779	0.3424

Compared with the resolution of the high-magnification probe orienting at the center field, the resolution is lower when it orients at the corner field. One reason is that the MRFL is designed for a flat field, when the high-magnification probe orients at the corner field, the actual working distance is equivalent to 134mm, substantially larger than 120mm, and thus the spatial resolution is expected to decrease. Another reason is the distortion of the optical system, which is a magnification error related to field position. Alike a typical wide-field of view imaging system, the corner field of the MRFL optics is subject to a substantial amount of barrel distortion (15% at diagonal 80 degrees). Consequently, the actual magnification of the corner field is smaller than the center field, which leads to a lower spatial resolution. Off-axis aberrations such as astigmatism and lateral chromatic aberration can cause the performance drop as well. As shown in Fig. 5, the image contrast in the vertical direction is higher than that in the horizontal direction; and lateral color is somewhat noticeable in Fig. 6 as well.

3.2 Resolution measurement of wide-angle probe at 120mm working distance

Figure 7 shows the resolution of the center field of the wide-angle probe at 120mm working distances. In the center field, the group 1 element 4 bar can be resolved, which corresponds to a spatial frequency of 2.83lp/mm or a limiting resolution of 176um. The contrast of each resolution target is shown in Table 5.

Fig. 7.

Images and intensity profiles of the resolution target of the center field of the wide-angle probe at 120mm working distance, (a) in vertical direction; (b) in horizontal direction.

Table 5. Image contrast of the center field of the wide-angle probe.

Contrast	G1 E4 2.83 lp/mm	G1 E3 2.52 lp/mm	G1 E2 2.24 lp/mm	G1 E1 2.00 lp/mm
Vertical	0.3260	0.4637	0.5610	0.6014
Horizontal	0.1813	0.3035	0.4875	0.5935

Figure 8 demonstrates the resolution of the corner field of the wide-angle probe at 120mm working distance. It is shown that the group 0 element 5 can be resolved, which corresponds to a spatial frequency of 1.59lp/mm or a limiting resolution of 314um. The contrast of the bar target is listed in Table 6.

Fig. 8.

Images and intensity profiles of the resolution target of the center field of the wide-angle probe at 120mm working distance, (a) in vertical direction; (b) in horizontal direction

Table 6. Image contrast of the corner field of the wide-angle probe.

Contrast	G0 E4 1.59 lp/mm	G0 E3 1.41 lp/mm	G0 E2 1.26 lp/mm	G0 E1 1.12 lp/mm
Vertical	0.1706	0.2693	0.2562	0.3869
Horizontal	0.1604	0.3059	0.4969	0.5951

Comparing Fig. 7 and Fig. 8, it can be found that in the corner field, certain amount of distortion and lateral color are observable. These aberrations degrade the performance of the corner field. However, it is still acceptable since the wide-angle probe is used for orientation and situational awareness, the performance is not as critical as that of the high-magnification probe.

3.3 Throughput analysis

The throughput of an optical system determines the light collection capability. When a digital imaging sensor such as CCD or CMOS is used, the throughput will affect the frame rate and signal to noise ratio. If the throughput is insufficient, the signal to noise ratio will decrease, thus the image will appear noisy and the contrast will also be reduced. The throughput of an imaging probe in the MRFL system, is defined by

$$\Phi =\epsilon A\Omega$$ (1)

where ε is the light transmission efficiency of the imaging probe, A is the area captured by the probe, and Ω is the solid angle of the entrance pupil. ε is mainly affected by the beamsplitting ratio between the wide-angle and high-magnification imaging paths and it is 50% for both imaging probes in our prototype implementation. The solid angle Ω can be calculated by $\Omega =2\pi \left(1-\cos {\theta }_0\right)$ where ${\theta }_0$ is defined as ${\theta }_0={\tan }^{-1}\left(EPD/2WD\right)$. The throughput of the wide-angle probe and the high-magnification probe at 120mm working distance is 0.3351mm²sr and 0.0372mm²sr, respectively. Similarly, at 80mm working distance, the throughput of the wide-angle probe and high-magnification probe is 0.2945 mm²sr and 0.0327 mm²sr, respectively. The throughput of the high-magnification probe is only 1/9 of the wide-angle probe.

When the surgical area is inadequately illuminated, the much lower throughput of the high-magnification probe inevitably leads to lower signal to noise ratio and lower image contrast of the its images than that of the wide-angle image. To balance the effects of throughput difference between the two imaging probes, we plan to implement a different beam splitting ratio along with an improved illumination source for future MRFL developments.

3.4 MTF measurement

To further verify the optical performance of the high-magnification probe, we adopted the slanted edge method to measure the MTF of the high-magnification probe orienting at the center field [Imatest LLC]. The MTF measurements in both horizontal and vertical directions are carried out. As shown in Fig. 9, the dashed black line is the diffraction limited MTF, the red curve is the MTF of the designed high-magnification probe, the solid blue curve is the measured MTF in horizontal direction, and the dashed blue curve is the MTF in vertical direction. In the horizontal direction, about 10% MTF drop was observed across the measured spatial frequency range (up to 70lps/mm), while in the vertical direction, about 20% MTF drop was observed. These performance drop may be attributed to lens manufacturing errors and system assembling errors.

Fig. 9.

MTF measurement

It shall be noted that the MTF curves in Fig. 11 were obtained in the image space. To better characterize the optical performance of the MRFL system, the MTF in the object space need to be calculated. This can be done by calculating the optical magnification of the high-magnification probe, which is given by

$$m_{high-mag}=\frac{f_{obj}}{WD}\times m_{relay}^n\frac{f_{high-mag}}{f_{scan}}$$ (2)

Where the focal length of the objective, scanning lens and high-magnification probe are 2mm, 14mm and 90mm, respectively, the magnification of each relay lens group, m_relay, −1, and n is the number of relay groups . In our current MRFL prototype, the magnification of the high-magnification probe is determined to 0.107 for a working distance of 120mm.

Driven by the limited dynamic range of the imaging sensor, hereby 15% was selected to be the threshold MTF value and based on Fig. 9 60lp/mm was determined to be the maximum resolvable spatial frequency in the image space. The corresponding spatial frequency ${\xi }_{\max }$in the object space can be calculated by ${\xi }_{\max }=60\times m_{high-mag}=6.42lp/mm$, which agrees with the limiting resolution measured in Section 3.1.

4. Biological model evaluation

Following the successful fabrication and optical performance characterization of the MRFL prototypes, the MRFL prototypes was evaluated on a live porcine model at the live animal lab in the Keck School of Medicine at the University of Southern California. The porcine was placed under general anesthesia with its abdomen incised with a CO₂-filled cavity. As shown in Fig. 10, three incisions were made on the abdominal wall of the pig with three standard trocars for laparoscopic procedures in place. During the test, one of the trocars was utilized for positioning a MRFL prototype while a standard laparoscope for comparison or a laparoscopic grabber or scissor may be inserted through the other trocars. Figure 10 demonstrated the setup with a normal-length MRFL prototype inserted through the bottom trocar. The images shown in Fig. 11 and Fig. 12 were captured with this setup where only about half of the normal-length tube was inserted into the trocar to ensure a working distance round 120mm.

Fig. 10.

In-vivo animal test

Fig. 11.

MRFL in-vivo evaluation with a porcine model at an approximately 120mm working distance from the surgical cite: (a) high-magnification image of the spleen; (b) wide-angle image of the spleen; (c) high-magnification image of the gallbladder; (d) wide-angle image of the gallbladder. The high-magnification and wide-angle images were acquired simultaneously through the MRFL prototype.

Fig. 12.

Demonstration of situational awareness. (a) (c) high-magnification images of the gallbladder captured by the high-magnification probe of the MRFL system failed to show the insertion and close approach of another surgical instrument; (b) (d) wide-angle views, corresponding to (a) and (c), respectively, by the wide-angle probe shows the overview of the abdominal cavity with a clear visualization of the surgical instruments as well as their insertion path, preventing accidental collision or injuries.

Figures 11(a) and 11(b) demonstrate the high-magnification image and wide-angle image of the spleen captured simultaneously by the two imaging probes of the MRFL prototype, where the splenectomy was performed. Figures 11(c) and 11(d) demonstrate images of the gallbladder for cholecystectomy. The cyan-boxes in Figs. 11(b) and 11(d) marks the corresponding regions of interest captured the high-magnification foveated probe. The surgical areas displayed in Figs. 11(a) and 11(c) by the high-magnification probe are similar to those by the standard laparoscope used for comparison. These pictures further demonstrate that the high-magnification probe’s capability of capturing the adequate fine structures of the spleen and the gallbladder for surgical procedures. Figures 11(b) and 11(d) demonstrate the surgical fields of the wide-angle probe which is substantially larger than that by the standard laparoscope in comparison. The wide-angle views can guide the manipulation of other instruments without collision. In addition, Fig. 11(d) shows the position of a standard laparoscope used for comparison, which suggest that the working distance of the standard laparoscope is much smaller than that of the MRFL. In order to get the close-up view similar to that by the high-magnification probe of the MRFL system or wide-angle view similar to that by the wide-angle probe, the standard laparoscope needs to move forward or withdraw backward from the surgical site.

Figure 12 demonstrates the capabilities offered by an MRFL system in terms of maintaining situational awareness and preventing accidental contact of the instruments with unintended organs or structures. Along with the MRFL inserted through one of the three trocars, the surgeon grabbed the gallbladder with a surgical grabber on his left hand and inserted a surgical scissor through the third trocar with his right hand. As shown in Figs. 12(a) and 12(c), the high-magnification probe captured the needed details for surgical procedure, equivalent to the close-up view of a standard laparoscope, but failed to make aware of the path of the scissor and its close approach to the other organs. However, without the need of retracting the scope, the wide-angle probe of the MRFL system simultaneously captured the overview of the abdominal cavity shown in Fig. 12(b) and 12(d), which helped the surgeon fully aware of the insertion path of the scissor in the abdominal cavity and effectively prevented the accidental contact of the instruments with other organs. In the current MIS surgery, when a standard laparoscope is used at a zoomed-in view for surgical operation, the surgeon loses his or her vision outside the immediate focus of the laparoscope and thus loses awareness of any injuries occurring outside the close-up view, which can cause fatal problems in some extreme case such as massive hemorrhage. The capability of simultaneously capturing close-up and wide-angle views in a MRFL system can effectively address such limitation.

5. Conclusion

In summary, this paper presented the overall system design of a MRFL prototype, described the optical performance of the prototype obtained through a set of characterization experiments, and validated the clinical use of the prototype through a preliminary evaluation with a live porcine model. The results demonstrated that the MRFL has the capability of capturing both high-magnification and wide-angle views simultaneously through an integrated system, providing adequate details for surgical procedures and large field coverage without the need of moving forward or withdrawing the laparoscope. The results further demonstrated that the MRFL system can provide surgeons superior situational awareness, and can effectively prevent accidental injuries on unintended structures and reduce the complications. Last but not the least, having a large working distance with the MRFL system can effectively reduce instruments collision inside the abdominal cavity, which is anticipated to improve the safety and efficiency of MIS procedures.

For the future work, we will focus on solving the throughput difference between the high-magnification probe and wide-angle probe to improve the image quality, and carrying out more clinical studies to evaluate the outcome of MIS surgery using the MRFL.