A review on tele-manipulators for remote diagnostic procedures and surgery

Abstract

With modern medicine and healthcare services improving in leaps and bounds, the integration of telemedicine has helped in expanding these specialised healthcare services to remote locations. Healthcare telerobotic systems form a component of telemedicine, which allows medical intervention from a distance. It has been nearly 40 years since a robotic technology, PUMA 560, was introduced to perform a stereotaxic biopsy in the brain. The use of telemanipulators for remote surgical procedures began around 1995, with the Aesop, the Zeus, and the da Vinci robotic surgery systems. Since then, the utilisation of robots has steadily increased in diverse healthcare disciplines, from clinical diagnosis to telesurgery. The telemanipulator system functions in a master–slave protocol mode, with the doctor operating the master system, aided by audio-visual and haptic feedback. Based on the control commands from the master, the slave system, a remote manipulator, interacts directly with the patient. It eliminates the requirement for the doctor to be physically present in the spatial vicinity of the patient by virtually bringing expert-guided medical services to them. Post the Covid-19 pandemic, an exponential surge in the utilisation of telerobotic systems has been observed. This study aims to present an organised review of the state-of-the-art telemanipulators used for remote diagnostic procedures and surgeries, highlighting their challenges and scope for future research and development.

Introduction

In recent years, the demand for the deployment of robots in healthcare systems has grown many fold. Shortage of healthcare professionals, cost of healthcare services, and population aging are some of the reasons. Many researchers have worked rigorously to develop robots that can work autonomously alongside present healthcare professionals and assist them. In this article, a few such robots which are currently deployed are discussed.

The term telemanipulator is formed by joining two words, i.e., tele and manipulator. Tele stands for telecommunication and a manipulator is a device, comprised of links connected by joints, used to handle materials by the user without directly touching the material. Such manipulators which are operated from a far distance (operator not in the vicinity of the manipulator) are called telemanipulators. In recent years, the use of telemanipulators in the field of medicine has experienced a significant surge. Telemanipulators used in medicine lie under a bigger umbrella called telemedicine (initially defined by Dr. Kenneth Bird in 1971 [1]). Two of the important areas in medicine are diagnosis and surgery. In this article, a detailed review of telemanipulators used for diagnostics and surgery is illustrated. Section 2 elaborates robot systems used for performing remote surgery. A state-of-the-art review of telemeanipulators deployed in medical diagnosis is presented in Sect. 3. Major conclusions from the review are highlighted in section 4.

Telemanipulators in surgery

The process of controlling a manipulator from a remote place to perform surgical operations is termed telesurgery. The manipulators used for this purpose are termed telemanipulators. The advantages of these systems are the accessibility to high-quality surgery for patients at remote locations, allowing collaboration between experts geographically separated and enhancing surgical precision. Telesurgery is performed by two groups of experts, i.e., one of the groups in proximity to the patient, and the other group, comprising surgical experts, controlling the manipulator from a remote location, to operate on the patient. The group in proximity to the patients, sets up the surgical robot. This group is also responsible to terminate the tele assisted robotic surgery in case of an emergency and to continue the surgical process as appropriate. Using haptic (touch) sensors, surgeon at remote location generates the input control signals. These signals are then transmitted using wireless communication protocol to the telemanipulator at patient end to perform a specific procedure in the surgery. During this process, real-time visual feedback signals are sent to the surgical experts at the remote end to have exact information about the environment where the patient is being operated on. With the advancement of technology in the field of telecommunication, such as 5 G networks, a surge in the demand for telesurgery is observed. During the recent COVID-19 pandemic when travel was restricted there was a use case for requirement for remotely guided procedures.

Since the first known material manipulator, developed in 1949 by Raymond C. Goertz [2] to handle hazardous radioactive materials, robotic manipulators have evolved considerably. In 1961, manipulators for industrial applications was introduced by a company called Unimation. The robot was called Unimate [3]. Although these manipulators were able to replace some of the human works, the true dexterity of a human arm was achieved when in 1978 Programmable Universal Manipulation Arm (PUMA) was introduced. This led to the use of manipulators in surgical robots, demonstrated by Kwoh et. al. [4] in 1988 to perform brain biopsy. Many researchers have worked rigorously to develop efficient surgical robots. This is described below.

PROBOT

In late 1980, a robot was developed at the Imperial College in London to assist transurethral prostatectomies. The robot was called PROBOT [5–7]. Although transurethral prostatectomies are not difficult, robots assisting the surgery reduces risk of human error and time. Transurethral prostatectomy is a process to remove the extra-growth of tissue in the prostate gland. The clinical problem is impaired flow, a uncommon issue in elderly men. By the time this research was initiated, advanced 6-axis PUMA robots were already popular for industrial application. Initial experiments were performed with the PUMA robot. Due to the large workspace and unconstrained motion of such a robot, the laboratory developed its own robot. This new robot was designed to have a much smaller working environment limited to the size of the prostate. The robot had four axes of movement. i.e., 3 translational axes and 1 additional degree of freedom (DOF) for moving the cutter. The rotational speed of the cutter was set to 40,000 rpm. An endoscope was also attached to the probe to have live image feedback during the surgical procedure. Using the input from the surgeon about the resection areas on the 3-D model of the prostrate the robot calculated the trajectories and autonomously performed the surgical procedure. This helped reduced fatigue in the surgeon and improved cutting accuracy of the tissues on the prostate gland.

ROBODOC

The ROBODOC robotic system was developed by Integrated Surgical Systems, Sacramento, CA, USA in 1992, to assist in performing a cementless total hip replacement [8–10]. The system was developed to minimize potential human error in performing the surgery. It comprised of a computer workstation, called ORTHODOC, used to perform preoperative planning and a 5-axes robot in which a high-speed milling tool is attached as an end effector. During the initial introduction of cementless hip replacement surgery, many patients complained about thigh pain, intraoperative fracture, and failure of bony ingrowth. It was then realized that the accurate shaping of the femur bone is equally important as proper fitting of the bone. Since the equipment used to perform cementless surgery was the same as cemented surgery, carving the bones created rough surfaces resulting in gaps when implants were placed in cementless surgery. This called for the development of precision bone-carving robots. Using the 3-D model developed by CT scans, all movements required for carving the bone were precalculated and bone carving was performed autonomously. After introducing robotic hip replacement surgery, the discomforts related to post-surgical effects were reduced subsequently.

AESOP

In the field of robots in medicine, the next important invention was introduced as the Automated Endoscopic System for Optimal Positioning (AESOP) by Computer Motion, Santa Barbara, CA, USA [11, 12]. AESOP was the first laparoscopic camera holder which obtained FDA approval. Later AESOP was upgraded to a voice-controlled system with seven degrees of freedom manipulator to mimic a human hand. Traditionally, during laparoscopic surgery, the laparoscopic camera was handheld and operated by a human assistant. Although the time required to perform surgery using the traditional method and AESOP had no significant difference, the stability of video feedback using AESOP robots had improved significantly [13–15]. It also replaced an assistant required for such surgery.

ZEUS

The same company which developed AESOP introduced the concept of telerobotics surgery with ZEUS in 1998 [16–20]. The system worked in a principle of master–slave configuration, in which the slave robot performs the surgery as per the command generated at the master system by a specialist. While operating the system, the master robot which is at a far distance from the patient connects to the slave robot through telecommunication. Long distance telesurgery between New York, USA and Strasbourg, France using fiber optic cables was achieved through ZEUS robotic system. Three arms of the slave in ZEUS robotic system, two for performing the surgery and the other as AESOP, are attached to a table. The two surgical arms have four degrees of freedom. The master robot consists of a video monitor and two handles. The video monitor receives video feedback from the environment of the patient and the two handles are used to manipulate the two surgical arms. Although telesurgery was successfully demonstrated by the ZEUS system, the system had its own limitations. The system was too big to be operated on in a smaller space and to view the 3-D image, the surgeon needs to use a special glass which ocasionally caused motion sickness to the surgeon.

Da Vinci Robot

In 1999, the first standard Da Vinci surgical system was introduced by Intuitive Surgical Inc. which was founded in 1995 [21]. The system consists of three major devices i.e., patient cart, vision cart, and surgeon console [22]. Initially, the system was introduced with three robotic arms on the patient cart, however, in an upgrade in 2005 an additional robotic arm was integrated resulting in four robotic arms on the patient cart. The Da Vinci surgical robots operate on the same master–slave principle as the ZEUS. In the patient cart, three of the arms are used to perform surgery and the fourth is used for high-definition visual feedback. In the version released in 2009, the system is equipped with another surgical console to incorporate collaborative surgeries. As of now, Da Vinci robot is one of the most sophisticated teleoperated laparoscopic surgical technological systems available which can efficiently perform complex kidney, prostate, ureter, and pelvic surgeries.

Telemanipulators in Medical Diagnosis

The idea behind integrating robotic telemanipulators in medical diagnostic procedures is to bring the expertise of specialists to regions where there are no specialists. The expense and inconvenience to patients travelling long distances for regular checkups could reduce. From an ergonomic standpoint, the telerobotic systems further aid in the comfort of the operators. Holding the probe at different uncomfortable angles can lead to long-term musculoskeletal problems. Incorporating these several additions [23, 24] aim to improve operator experience.

Ultrasound imaging has become an indispensable diagnostic imaging technology used in several medical emergencies and to aid surgical decisions. Ultrasonography however is strongly operator dependent. Telemanipulators for ultrasound imaging therefore is of immense importance..

A typical telerobotic ultrasound system consists of three components. At the remote sonographer site, an experienced sonographer moves around a haptic-feedback-enabled mock probe. At the other end, the patient site, a robotic manipulator operates the scanning ultrasound probe and interacts directly with the patient. Finally, a telecommunication system connects the docto/sonographerr to the patient, enabling seamless diagnostic procedures. Figure 1. illustrates the key components of a typical telerobotic ultrasound system. By bringing innovations to these three components, several variations have been reported in the literature. A few of these have been commercialised. The following section reviews the state-of-the-art technologies reported in this area.

Fig. 1.

Schematic of a tele-manipulator operating at the patient site remotely controlled by a physician

One of the earliest works was done by Gourdon et al. in the SYRTECH project [25]. They illustrated a 3 DOF cage-like robotic system that interacted with the patient and was remotely manipulated by a sonographer using a joystick controller. The robot did not have any active translational degrees of freedom and had to be positioned manually at the area of scanning, for example, abdominal scans. The wrist inside the cage-like structure could actively rotate in 3D space, displaying the orientations as intended by the sonographer. The system has been successfully deployed in performing remote ultrasonography of the liver and the heart.

Following this, a series of pioneering works by Vieyres et al. and supported by the European Space Agency (ESA): TERESA [26], OTELO [27, 28], ESTELE [29, 30] and PROSIT [29], laid the foundation for the now commercialised MELODY system [31, 32]. The MELODY system consists of three active rotational degrees of freedom and three passive translational degrees of freedom. A human assistant at the patient site passively controls the positioning of the ultrasonographic probe. And its orientation is remotely controlled by the expert sonographer. Now, for the remote control, instead of a mouse or a joystick, the system deviates from its precursors by equipping a haptic feedback-enabled dummy probe. The system is made commercially available by AdEchoTEch, France, and has successfully been deployed in over three hundred patients for cardiac [33], abdominal [34, 35], obstetric [36] and pelvic [37] ultrasound scans.

Wang et al. at King’s College London led a series of developments in robot-assisted ultrasound scan procedures under the intelligent Fetal Imaging and Diagnosis (iFIND) project [38]. They started with a bulky 7-DOF cartesian robot in iFIND-v1 and moved on to iFIND-v2, which had a more compact and lightweight design with better safety management techniques [39]. In iFIND-v3 they shifted from the single-arm robot approaches of the predecessors to incorporating a 17-DOF dual-probe robot system [40]. In more recent versions, they proposed integration of soft robotic end-effector [41] and self-adaptive parallel manipulators (SAPM) [42]. These conforms to the contours of the surface in contact and adds to patient safety. The project aims at improving pregnancy screenings by bringing innovative technologies in obstetric and abdominal ultrasound scan procedures [38].

Szczęśniak-Stańczyk et al. developed an unified robotic system, Remote Medical Diagnostician (ReMeDi) [43] It consists of a kinematically redundant 7-DOF manipulator with an integrated video-conferencing system, enabling the doctor to remotely interact with the patient. Unlike its predecessors, this does not require the presence of a human assistant at the patient site and provides the provision for complete teleoperation. Besides ultrasonographic scan procedures, the system allows for patient discussions, medical assessment, auscultation and palpation. Further demonstrations have been successfully carried out to illustrate its efficacy for remote cardiac examinations [44].

The MGIUS-R3 system [45] is a commercially available telerobotic system for ultrasonography, developed by MGITech Co. It consists of a 6-DOF robotic manipulator with an integrated force-feedback sensory system, handling the ultrasonic probe in contact with the patient’s body. It has been used for conducting ultrasonography procedures for cardiopulmonary tests in real time [46]. Table 1 summarizes all the systems discussed above.

Table 1.

Summary of custom-made tele-manipulators in medical diagnosis

System	Year	Design	Application	Reference
SYRTECH	1999	3DOF system (modified wrist for rotation)	Cardiac and liver USG	[25]
TERESA	2003	3DOF spherical wrist for rotation and platform with 1 translational DOF	Universal	[26]
OTELO	2006	6 DOF system	Universal	[27, 28]
ESTELE	2012	4 DOF system (3 rotation + 1 translation)	Universal	[29, 30]
PROSIT	2012	4 DOF system (3 rotation + 1 translation)	Universal	[29]
MELODY	2016–2018	3 active rotational DOFs and 3 passive translational DOFs	Cardiac, abdominal, obstetric, pelvic USG	[31–37]
iFIND	2019	v1: 7-DOF cartesian robot v2: 8-DOF system (5DOF wrist + 2DOF parallel arm + 1DOF global rotation) v3: 17-DOF dual robot system	Abdominal and obstetric USG	[38–42]
ReMeDi	2016–2020	7DOF system with integrated video-conferencing system	Cardiac and abdominal USG	[43, 44]
MGIUS-R3	2020	6DOF manipulator	Cardiac USG	[45, 46]

Apart from these robots, which are specifically designed to perform ultrasonography, researchers have also tried to extend the usage of commercially available robots in this domain, which will make the development of the overall system more affordable. Chatelain et al. [47] used the Viper s650 robot by Adept Technology Inc., USA [48] and integrated a 6-axes force/torque sensor to build their system. For their remote trauma assessment study, Mathur et al. [49] developed a robotic system with the 7-DOF KUKA LWR arm (KUKA AG, Augsburg Germany) [50]. The remote robotic ultrasound system proposed by Huang et al. [51] for 3D scanning of human tissues used the ProSix C4 robot by Epson [52]. UR5 collaborative robot arm by Universal Robots Corporation, Denmark, has been one of the most widely used commercially available robot manipulators because of its low cost and compliance with ISO 10218-1:2006 [53]. Mathiassen et al. [24] used the 6DOF UR5 system to interact with the patient and the Phantom Omni system as the remote controller. Geng et al. [54] developed a master–slave system for robot-assisted ultrasound imaging by integrating the slave: UR5 robotic arm with the master: a haptic device, Touch by 3D System Corporation, America [55]. Another popularly available commercial robotic arm is Panda from FRANKA EMIKA [56]. It has similar advantages as that of the UR5 robot, except for a smaller work volume and lower torque limit, making it safer for patients in case of accidents. Kaminski et al. [57] and Sandoval et al. [58] have used this Panda arm in their setups for tele-ultrasonography respectively. Thus, utilizing commercially available robotic systems, besides making the development more feasible, helps make the final product more affordable and robust. A summary of all such system developed using commercial robots, discussed here, is presented in Table 2.

Table 2.

Summary of tele-manipulators in medical diagnosis developed using commercially available systems

Robot	Distributer	Design	Application	Reference
Viper s650 [48]	Adept Technology Inc., USA	6 DOF robot arm	Robotic tele-echography	[47]
KUKA LWR [50]	KUKA, Augsburg Germany	7 DOF robot arm	Focused Assessment with Sonography for Trauma (FAST)	[49]
ProSix C4 [52]	Epson, Japan	6 DOF robot arm	3D imaging of human tissues	[51]
UR5 [59]	Universal Robots Corporation, Denmark	6 DOF robot arm	Universal	[54]
PANDA [56]	Franka Emika, Germany	7 DOF robot arm	Thyroid USG	[57, 58]

Ultrasound imaging, with its highly safe diagnosis principle, has been deployed to provide images and examine a wide array of areas in the human body: heart, liver, kidneys, gallbladder, breasts, uterus, major blood vessels and even eyes. Echocardiography is the use of ultrasound waves for cardiac imaging [60]. A tele-echocardiography system, TER, is reported in [61]. The device is equipped with a force feedback system, allowing the operator to apply variable pressures on the patient’s body. Masuda et al. [62] also developed a wireless tele-echography system-assisted EDR (echographic diagnosis robot). It has also been used to diagnose tumours and guide biopsies. Prostate ultrasound is deployed to generate images of the rectum and assist in biopsy for further diagnosis. Pisani et al. [63] presented a robotic system with the 4 DOF SCARA (Selective, Compliance, Arm, Robotized, Assembly) robotic arm to conduct a biopsy of the prostate aided by an ultrasound probe localisation system. The high-precision system has the potential to be controlled both locally and remotely from a distance for teleoperation and was demonstrated on over five hundred tests.

Conclusions

In this article, a review of existing telemanipulators used for diagnostics and surgery is discussed. The use of manipulators in medicine has improved efficiency and accuracy of surgical procedures. It has helped reduce fatigue in physicians, while performing long surgical procedures. Since manipulators can perform precision movements repeatedly, the use of such systems may increase. With evolving technologies, such as fifth-generation communications and Smart material-based compact actuators, the development of cost-efficient and compact telemanipulators may be envisaged, in the near future.