Robotic Endoscopy

Abstract

Endoscopes extend the eyes of the physician into the patient's body. They are widely used in gastrointestinal (GI) diagnostics and minimally invasive surgery. Endoscopes can be classified into 3 types: rigid, flexible, and capsule endoscopes. Rigid and flexible endoscopes are traditionally held and manipulated by the physician to visualize the region of interest, while capsule endoscopes move passively along with the GI peristalsis. With the advancement of technology, robotic endoscopy has been increasingly developed and accepted. In this work, robotic endoscopy from 3 categories (robot-assisted rigid endoscopy, robot-assisted flexible endoscopy, and active GI endoscopy including active flexible colonoscopy and active capsule endoscopy) is reviewed by PubMed search with the criteria (‘Robotics’ OR ‘Robot’) and (‘Endoscopy’ OR ‘Endoscope’).

Introduction

Endoscopes extend the eyes of the physician into the patient's body to visualize internal organs and guide surgical operations. This has made it possible for clinicians to inspect the inside of structures such as the urethra and the GI tract including the esophagus, stomach, small intestine, and colon. In addition, it enables surgeons to perform keyhole surgery instead of making large incisions. The history of endoscopy can be dated back to the 10th century when the Arabian physician Albukasim (936–1013 AD) used reflected light to inspect the cervix [1]. 7 centuries later, in 1806, Frankfurt-born physician Philip Bozzini developed a rigid tubular device called the ‘Lichtleiter’ to inspect the female urethra and vagina with cervix. In 1868, Adolf Kussmaul performed the first direct esophagoscopy in a sword swallower [2]. However, only in recent decades, with the technological advancements of flexible fiber-optic instruments in 1957, charge-coupled devices (CCD) in 1969, and the video computer chip in 1986, endoscopy has become widely used in general surgery and for GI inspection. Since then, endoscopes have undergone rapid development: since 1993, the endoscope enables 3-dimensional vision [3], and since 2000, the endoscope can pass through the entire GI tract for inspection purposes [4].

Currently, endoscopes fall into 3 categories: the rigid endoscope, the flexible endoscope, and the capsule endoscope. Rigid endoscopes are mostly used for guiding surgical operations; flexible endoscopes are mainly used for inspecting the biological duct, and capsule endoscopes are used for inspecting the GI tract. Rigid and flexible endoscopes are traditionally held and manipulated by an assisting physician. To master the manipulation and communicate well with the surgeon, extensive training and experience are required. In addition, the physiological hand tremor can make the view unstable, and holding the endoscope is labor-intensive, especially during long operations. These problems can be solved by use of a robot which outperforms humans in terms of accuracy and is free of fatigue. These benefits are well demonstrated in robotic endoscope holders, such as the Automated Endoscopic System for Optimal Positioning (AESOP®, Computer Motion, Inc., Goleta, CA, USA) [5]. Also, early capsule endoscopes passed through the GI tract without active locomotion but were of limited therapeutic function and efficacy. By combining them with robotic technology, capsule endoscopes can now move actively inside the GI tract, laying the foundation for therapeutic capsule endoscopes.

In this work, we review robotic endoscopy of the 3 categories, i.e., robot-assisted rigid endoscopy, robot-assisted flexible endoscopy, and active GI endoscopy including active flexible colonoscopy and active capsule endoscopy. Endoscopes imbedded in robot systems with therapeutic functions, such as the DaVinci® surgical system (Intuitive Surgical, Sunnyvale, CA, USA), single-port access robots, and natural orifice transluminal endoscopy surgical robots, are not included in this review.

Methods

By using the criteria (‘Robotics’ OR ‘Robot’) and (‘Endoscopy’ OR ‘Endoscope’) in ‘All Fields’, a total of 6,950 publications were found in PubMed for the period of January 01, 1900 to December 31, 2017. In contrast, by using the criteria (‘Robotics’ OR ‘Robot’) and (‘Endoscopy’ OR ‘Endoscope’) in ‘Title/Abstract’, we found 294 publications. Figure 1 shows the annual distribution of these papers. From the 264 robotic endoscopy-related publications together with other well-known articles, the literature on robotic endoscopy was summarized for 3 categories: The first category is robot-assisted rigid endoscopy for minimally invasive surgery, i.e., the rigid endoscope is held and manipulated by a robot arm. The second category is robot-assisted flexible endoscopy for GI tract inspection, i.e., the flexible endoscope is held and manipulated by a robotic device. The third category is active GI endoscopy, which includes active flexible colonoscopy and active capsule endoscopy.

Fig. 1.

a Number of PubMed articles using the criteria (‘Robotics’ OR ‘Robot’) and (‘Endoscopy’ OR ‘Endoscope’) in All Fields; b number of PubMed articles using the criteria (‘Robotics’ OR ‘Robot’) and (‘Endoscopy’ OR ‘Endoscope’) in Title/Abstract.

Robot-Assisted Rigid Endoscopy

The robot-assisted rigid endoscope represents the first combination of an endoscope with a robot. In 1994, Computer Motion, Inc. developed the AESOP for holding and maneuvering rigid endoscopes [6] (fig. 2a). This system could be controlled by hand or foot switches or by using vocal commands (AESOP 2000). Its major benefit was that the robot could hold the endoscope for a long time without fatigue or tremor, thereby advancing a steady view of the surgical field throughout the operation. Meanwhile, the view is directly controlled by the physician, thus eliminating the communication between the physician and the endoscope holder. This is especially beneficial when the scope holder is inexperienced. The robotic arm in AESOP uses a 3-degrees-of-freedom SCARA-type architecture; therefore, the invariant point of motion at the trocar insertion site is not mechanically guaranteed. In addition, this system does not have a built-in mechanism to stop the endoscope if the voice is not recognized. Another example is the TISKA Endoarm developed by the Karlsruhe Research Center, Germany, and KARL STORZ GmbH & Co., Tuttlingen, Germany (fig. 2b) [7]. It utilized a special parallelogram mechanism to keep the point at the trocar insertion invariant (remote center of motion (RCM)). The system can be repositioned with one hand and uses magnetic brakes to hold the position. A compact endoscope manipulator was developed in France (fig. 2c) and later on named the ViKY robotic scope holder (Endocontrol, Grenoble, France) [8, 9]. The ViKY system is controlled by the surgeon using a pedal or vocal commands, allowing for basic displacement with large/small amplitude and automatic return to the memorized positions. The system received its CE mark in 2007 and was Food and Drug Administration (FDA) approved in 2008. One distinct feature of the ViKY system is that it is body-mounted which makes it very compact. However, this also increases the interference at the trocar as part of the space is occupied by the actuation unit. The SOLOASSIST™ developed by AKTORmed GmbH, Barbing, Germany [10], is a table-mounted, voice-controlled endoscope (fig. 2d). It is compatible with joystick control. The RCM is manually registered in the setup phase by means of a metal sphere. The AutoLap™ developed by Medical Surgical Technologies (MST, Sydney, Australia) [11] utilizes an image-guided system to control the endoscope (fig. 2e). The surgeon can virtually tag the desired position using the tip of the surgical tool, and the robotic arm then guides the endoscope to the tagged point along an optimal trajectory. The AutoLap is also compatible with joystick and manual control. There are many more similar systems, such as EndoAssist™ (Prosurgics, Guilford, UK) [12], Navio™ (Smith&Nephew, London, UK) [13], and the CUHK endoscope manipulator (The Chinese University of Hong Kong, SAR, China) (fig. 2f) [14].

Fig. 2.

Examples of robot-assisted rigid endoscopy: a AESOP [6]; b TISKA Endoarm [7]; c ViKY robotic scope holder [8]; d SOLOASSIST [10]; e AutoLap [11]; and f CUHK endoscope manipulator [14].

The human-robot interface (HRI) is important in robot-assisted rigid endoscopy. In the current systems, voice control, joystick control, and foot pedal control are the prevailing control modes. Voice control is natural and effective. It suffers from the error of voice recognition, which may impair the effectiveness of endoscope control and raise safety concerns. In recent systems, such as SOLOASSIST, the recognition rate is around 95%. Other issues with voice control are that: i) it is slow compared to human assistance; ii) it does not allow effective fine adjustment of the view; and iii) it can distract other support staff in the operating room. Joystick control (including hand/finger-mounted, foot-mounted, and head-mounted) is another effective method. Compared with voice control, it is more straightforward; however, it is less intuitive and requires involvement of the hand, foot, or head and neck during the operation. Other HRI methods increasingly used in robotic endoscopes include eye gaze control as used in the AESOP [15] and CUHK endoscope [16] and image tracking as employed in the AutoLap [11]. These control methods further smooth the control of the endoscope and enable operating in an intuitive and automatic way. Of note, it is advisable not to rely on a single HRI method, and a hybrid approach is more practical as shown in the commercial products.

Robot-Assisted Flexible Endoscopy

Flexible endoscopy is the current gold standard of GI tract inspection and treatment. Conventionally, all steps, including insertion, holding, and maneuvering of the endoscope, are performed manually. The EndoDrive® (ECE Medical Products, Erlangen, Germany) [17] is the first commercially available system for electro-mechanical support of shaft insertion of flexible endoscopes (fig. 3a). The system allows positioning and driving of the endoscope shaft with a foot pedal and leaves both hands free for maneuvering the tip-bending section and instruments. Rotation of the shaft is also done manually. Kume et al. [18] and Keeichiro et al. [19] developed the master-slave Endoscopic Operation Robot (EOR). It allows steering, advancing, rotating, and stabilizing of a standard colonoscope by 2 joysticks. In the latest version, force sensing is installed, and the endoscope can be manipulated by one hand [19].

Fig. 3.

Examples of robot-assisted flexible endoscopy: flexible endoscope holding principle of a EndoDrive [17] and b the gastroscope intervention mechanism (GIM) by Yang et al. [22].

In 2013, Ruiter et al. [20] developed a robotic system to steer the flexible endoscope and evaluated its effectiveness. 2 setups with robot assistance were evaluated. In the first setup, the endoscope was steered by the master device by a single hand, and the instrument, e.g. biopsy needle, was controlled by the other hand. In the second setup, only the tip of the endoscope was steered by the master and the shaft was maneuvered manually (insertion and rotation). The instrument was operated by an assistant. A trial with 12 participants showed that tasks were completed faster than in a conventional manual operation, and 11 participants preferred the single-hand setup.

In 2017, Woo et al. [21] in Korea also developed a master-slave robot with force feedback for colonoscopy. The master robot contains a tilting device for controlling the tip angulation and an insertion/rotation device. The slave robot allows for gripping the conventional colonoscope. The robot allows 1.5 m insertion length, 360 degrees rotation, and a ±180 degree tilting angle. The robot was evaluated with 4 subjects comprising experienced endoscopists and novices. Results showed that insertion time for experienced endoscopists and novices were 17.4 ± 2.1 min and 39.0 ± 5.3 min, respectively.

In 2017, Yang et al. [22] developed a robotic system for holding a gastroscope, using a gastroscope intervention mechanism (GIM) with pneumatic-driven clamping function (fig. 3b). Compared to the EndoDrive and the EOR that use rigid roller pairs to hold and drive the flexible endoscope, the GIM affords uniform pneumatic pressure around the gastroscope. This can better protect the flexible endoscope from being over-clamped. Animal tests show that by using the GIM, the insertion speed is significantly slower compared to manual insertion (269 vs. 118 s).

Flexible endoscope manipulations common to most systems are shaft insertion and tip steering. The only system that incorporates shaft rotation is the EOR. In other systems, rotation is performed manually. None of the systems consider movements such as jiggling, which is common in manual control. These complex movements add difficulty to developing robot-assisted flexible endoscopy compared to rigid endoscopy.

Besides the attempts in robot-assisted maneuvering, other efforts have been made to enhance the therapeutic functions of endoscopes by adding operating arms at the tip of the endoscope (e.g., ISIS-Scope/STRAS™ (KARL STORZ/IRCAD, Strasbourg, France), MASTER™ (National University/Nanyang Technological University, Singapore), Endomina™ (Endo Tools Therapeutics, Gosselies, Belgium)). However, these are not within the scope of this review (refer to [23] for more details).

Active Endoscopy

Active Flexible Colonoscopy

With robot assistance, the physician can now operate the colonoscope away from the patient. However, the endoscope still has to be pushed forward. To reduce the risk of perforation and improve patient tolerance, the colonoscope needs to have active locomotion or conform to the shape of the colon [24, 25, 26, 27, 28, 29, 30].

The Aer-O-Scope™ (GI View Ltd., Ramat Gan, Israel) is a pneumatically actuated disposable colonoscope (fig. 4a). Its active locomotion is achieved by 2 balloons and pneumatic pressure. The first balloon carries the camera and is mobile, while the second balloon is fixed to the overtube. Both balloons are inserted into the rectum. By inflating the 2 balloons, the colon section in between the balloons is sealed. When CO₂ is insufflated between the balloons, the pneumatic force pushes the mobile balloon forward. Once the mobile balloon reaches the cecum, the CO₂ between the balloons is vented and CO₂ is insufflated between the cecum and the mobile balloon, the pneumatic force pushes the mobile balloon backward. The omni-directional CMOS (complementary metal-oxide semiconductor) camera is carried by the mobile balloon to inspect the colon. To protect the intestine, the operating pressure is monitored to not exceed 54 mbar [24]. Studies show that the Aer-O-Scope has a cecum intubation rate of 98.2% and polyp detection rate of 87.5% compared with conventional colonoscopy [25].

Fig. 4.

Examples of self-propelled colonoscopies: a Aer-O-Scope [29]; b NeoGuide [26]; c Endotics Endoscopy System [27]; d invendoscope [30]; and e soft earthworm endoscope robot [31, 32].

The NeoGuide™ Endoscopy System (NES) (NeoGuide Systems, Inc., Los Gatos, CA, USA) is intended for colon diagnostics and therapy. It incorporates a tip position sensor to measure the tip steering and an external position sensor to measure the insertion depth. The NES is distinct from conventional colonoscopes in that it contains multiple electromechanically controlled segments. During insertion, the segments follow the movement of the tip, thus conforming to the shape of the colon [26]. As a result, even if the NES is pushed at the back, the discomfort during the inspection is reduced. A clinical trial of the NES was performed with 11 patients, and the cecum was reached in 10 patients [26].

The Endotics® Endoscopy System (EES) (Era Endoscopy SRL, Peccioli, Italy) is a pneumatically-driven flexible endoscope for colonoscopies. It has a steerable tip, a flexible body, and a thin tail. It mimics the caterpillar locomotion by using 2 mucosal clamping devices and an extension/retraction mechanism [27]. A clinical trial with 71 patients showed a cecum intubation rate of 81.6% versus 94.3% with conventional colonoscopy. The examination time using the EES is longer (45.1 ± 18.5 min vs. 23.7 ± 7.2 min), and the polyp detection rate is lower. However, with the EES, patient discomfort is much lower, and none of the patients required sedation during the examination.

The invendoscope™ (Invendo Medical, Kissing, Germany) is a single-use, hand-held, motor-controlled colonoscope. The 10-mm endoscope contains an inverted sleeve and a driving unit with 8 wheels. A hand-held joystick is used to steer the endoscope. During insertion, the driving wheels grip onto the inner side of the inverted sleeve and propel the endoscope forward or backward while the sleeve unrolls. The invendoscope was first tested on 34 patients, and the cecum intubation rate was 82%; a later clinical trial with the Invendo SC20 on 61 patients showed that the cecum can be reached in 98.4% within 15 min [28].

Many other projects on active colonoscopies are ongoing; examples include the CUHK double balloon endoscope [31] and the soft earthworm endoscope robot [32, 33]. Both systems employ a locomotion principle similar to that of the caterpillar/inchworm/earthworm. One distinct advantage of the soft earthworm endoscope robot is that the endoscope body is made purely out of soft material with a rigidity equivalent to that of the colon tissue. With this it is expected that patient discomfort is minimized during the inspection. However, so far, the robot has only been tested in an ex vivo pig intestine model. A clinical trial is needed to validate the efficacy of the soft robot.

Active Capsule Endoscope

One distinct feature of the capsule endoscope is that it is swallowable and can pass through the entire GI tract making it an ideal platform for GI tract screening. However, conventional capsule endoscopes do not have active locomotion and cannot provide therapeutic functions. This greatly limits their efficacy and widespread use. Extensive research into how to add active locomotion and therapeutic functions to the capsule endoscopes has been carried out [34, 35, 36]. Active locomotion concepts include: self-propelled (e.g., crawling with legs, crawling with paddles, crawling with treads, swimming with propellers, and swimming with flapping tails) and externally propelled by magnetic force [37]. Figure 5 shows some examples. Currently, self-propelled capsule endoscopes are only in the research labs and none of them have been tried clinically. Major challenges include limited onboard power, weak propulsion force, and difficulty to adapt to the significant morphologic variations of the GI tract. External propulsion is more realistic and was implemented on the PillCam™ (Given Imaging, Yokneam Illit, Israel) [38], Olympus capsule endoscope (Olympus Medical Systems, Shinjuku, Tokyo, Japan) [39], OMOM® capsule endoscope (Jinshan Science & Technology, Chongqing, China) [40], and NaviCam® system (Ankon Technologies, Wuhan, China) [41, 42]. The magnetically actuated PillCam and OMOM capsule both use a manual handle with permanent magnets to steer the capsule endoscope for inspecting the stomach. Olympus Medical Systems in collaboration with Siemens Healthcare (Erlangen, Germany) developed the magnetically navigated video capsule endoscope. The Olympus capsule endoscope is steered by the Siemens guidance system to inspect the stomach using a controlled electromagnetic field. This system has a footprint of 1 × 2 m. The relevant trial resulted in 1 case of failure of the technique, and in the remaining 52 cases, the visualization rates of the antrum, body, fundus, and cardia were 98, 96, 73, and 75%, respectively. In the NaviCam system, the capsule endoscope is held and steered by a robot using electromagnetic force for stomach inspection. The physician uses the joystick to control the movements of the robot. In a clinical trial with 8 patients, the cardia, fundus, greater and lesser curvature, anterior and posterior wall, antrum, and pylorus were all visualized. The mean duration of examination was 25 ± 7 mins [41]. In another clinical trial with 350 patients, the overall diagnostic accuracy was 93.4% (95% confidence interval 90.83–96.02) compared to conventional gastroscopy [42].

Fig. 5.

Examples of active capsule endoscope and in vivo miniature robots: a Active capsule endoscope with legged locomotion [43]; b micro-intestinal robot with wireless power transmission [44, 45]; c propeller-based swimming capsule endoscope for stomach [46]; and d the Ankon magnetically actuated capsule endoscope system [42].

Conclusion and Discussion

In this work, the application of robotics in endoscopy is reviewed from the aspect of endoscope manipulation including active locomotion of capsule endoscopes. Robotic techniques that help to add therapeutic functions, such as the MASTER system, are not included. The distinct benefits of robot-assisted rigid/flexible endoscopy are that the robot helps to reduce the labor intensity of the procedure, provides a steadier view, and facilitates the communication between the surgeon and the endoscope. Key issues in manipulating rigid endoscopes include the RCM and the HRI. The RCM helps to maintain the position at the incision point thereby avoiding tearing of the surgical incision. Most existing systems rely on the control system for the RCM so as to simplify the robot structure. Common HRI solutions include voice control and joystick control (by hand/finger, foot, and head/neck). Voice control is natural and suffers from the error of voice recognition, low control speed, and being potentially distracting for support staff in the operating room. Joystick control is effective and straightforward; however, the surgeon is distracted when manipulating the endoscope. More recent HRI methods include eye gaze control and image-guided automatic tracking. By integrating these HRI methods, the surgeon's workload when operating the endoscope would be minimized to achieve endoscopic solo surgery. Manipulating a flexible endoscope is much more complex compared to a rigid endoscope. It involves holding of the shaft, shaft insertion, shaft rotation, tip steering, and other complex movements such as jiggling. Current systems can mostly help with insertion and tip steering; however, without a full set of movements, the flexible endoscope cannot be manipulated effectively. In this sense, robot-assisted flexible endoscopy is still in its infancy. Once the flexible endoscope can be held and manipulated by a robot like the rigid endoscope, therapeutic endoscopists could also perform endoscopic solo surgery. At the same time, with other robotics techniques, endoscopes (particularly colonoscopes and capsule endoscopes) can move forward by means of growing, crawling, or swimming [43, 44, 45, 46], therefore improving patient tolerability and reducing the risks of endoscopy. Many other roles exist for robots in minimally invasive surgery (MIS) visualization and GI treatment. For example, miniature wireless endoscopes could be inserted into the surgical cavity and manipulated by a robotic device from the outside, thereby reducing instrument fencing in MIS [47, 48]. In therapeutic endoscopes, robotic systems (e.g. MASTER) could help to enhance distal arm maneuverability and facilitate the surgical operation [23, 49].

In surgery, robotic rigid endoscopy is being increasingly used, and surgeons can now perform solo surgery. Thus, the robot is replacing the role of the endoscopy assistant. One may ask, could the robot take over the entire endoscopy procedure? As in the area of diagnostic endoscopy, the experience of the endoscopist remains crucial in surgery also [50]. The current systems for flexible endoscopy and active capsule endoscopy are still in the experimental stage, and much more groundwork has to be done before clinical use can be achieved, let alone autonomy. With advancements in technology, especially artificial intelligence and imaging processing, the level of autonomy [51] in endoscopy will certainly increase. However, even if the technology is ready, ethical issues are another hurdle. Will we be so lucky as to witness fully autonomous endoscopy in our lifetime?

Funding

This work is supported by the Hong Kong General Research Fund (Project No. 14212316 and 14207017), Chow Yuk Ho Technology Centre for Innovative Medicine Seed Grant (Project No. TIMSG-15/16-2), and CUHK Direct Grant (Project No. 2015.2.011 and 2016.108).

Disclosure Statement

The authors have no conflict of interest.