Haptic Feedback in Robot-Assisted Minimally Invasive Surgery

Abstract

Purpose of Review

Robot-assisted minimally invasive surgery (RMIS) holds great promise for improving the accuracy and dexterity of a surgeon while minimizing trauma to the patient. However, widespread clinical success with RMIS has been marginal. It is hypothesized that the lack of haptic (force and tactile) feedback presented to the surgeon is a limiting factor. This review explains the technical challenges of creating haptic feedback for robot-assisted surgery and provides recent results that evaluate the effectiveness of haptic feedback in mock surgical tasks.

Recent Findings

Haptic feedback systems for RMIS are still under development and evaluation. Most provide only force feedback, with limited fidelity. The major challenge at this time is sensing forces applied to the patient. A few tactile feedback systems for RMIS have been created, but their practicality for clinical implementation needs to be shown. It is particularly difficult to sense and display spatially distributed tactile information. The cost-benefit ratio for haptic feedback in RMIS has not been established.

Summary

The designs of existing commercial RMIS systems are not conducive for force feedback, and creative solutions are needed to create compelling tactile feedback systems. Surgeons, engineers, and neuroscientists should work together to develop effective solutions for haptic feedback in RMIS.

Introduction

Haptics generally describes touch feedback, which may include kinesthetic (force) and cutaneous (tactile) feedback. In manual minimally invasive surgery (MIS), surgeons feel the interaction of the instrument with the patient via a long shaft, which eliminates tactile cues and masks force cues. Some studies have linked the lack of significant haptic feedback in MIS to increased intra-operative injury [1]. In teleoperated robot-assisted minimally invasive surgery (RMIS), all natural haptic feedback is eliminated because the surgeon no longer manipulates the instrument directly. The lack of effective haptic feedback is often reported by surgeons and robotics researchers alike to be a major limitation to current RMIS systems.

Haptic Technology

In the robotics and virtual reality literature, haptics is broadly defined as real and simulated touch interactions between robots, humans, and real, remote, or simulated environments, in various combinations. The goal of haptic technology in robot-assisted minimally invasive surgery is to provide “transparency”, in which the surgeon does not feel as if he is operating a remote mechanism, but rather that his own hands are contacting the patient. This requires artificial haptic sensors on the patient-side robot to acquire haptic information, and haptic displays to convey the information to the surgeon (Figure 1). We categorize haptics as kinesthetic (related to forces and positions of the muscles and joints ) and/or cutaneous (tactile; related to the skin) in nature. Haptics includes force, distributed pressure, temperature, vibrations, and texture, which are in some cases difficult to model and quantify, let alone acquire and display. To provide myriad haptic information to the surgeon without sacrificing the maneuverability and dexterity afforded by the RMIS system is a major technical challenge. Simultaneously, the robot components, particularly disposable instruments, must remain low cost and robust.

Figure 1.

The main components of a teleoperated robot for minimally invasive surgery with multimodal haptic feedback. Both force and tactile feedback are included in the model, and graphical display (one method of sensory substitution) is shown as a possible alternative to direct haptic feedback.

As a technical field, haptics research has been active for several decades. In the 1990s, haptics research expanded significantly with the availability of high-fidelity, commercially available force feedback systems from companies such as SensAble Technologies, Inc. (Woburn, MA, USA) and Immersion, Inc. (San Jose, CA, USA). Currently, much of the force feedback research focuses on developing practical systems for application in fields such as entertainment, education, training, medicine and dentistry, and rehabilitation. While researchers have studied tactile feedback for many years, there is currently no commercially available tactile display system that provides distributed information to the skin in a compact package feasible for applications. One aspect of tactile feedback that has proven effective in applications such as video games is vibration feedback, in which a single vibrating actuator is typically used to provide information about events such as making and breaking contact. Further reading about haptic technology includes books [2][3][4], tutorials [5][6][7], and research reviews [8][9][10]. Recent reviews of haptics in surgery are [11][12].

Force Feedback

Kinesthetic or force feedback systems for RMIS typically measure or estimate the forces applied to the patient by the surgical instrument, and provide resolved forces to the hand via a force feedback device. Commercially available force sensors are very effective for measuring forces and torques in many teleoperation applications, but the surgical environment places severe constraints on size, geometry, cost, biocompatibility, and sterilizability. Although it is difficult to add force sensors to existing robotic instruments that were not designed with force sensing in mind, some researchers have had success on this front by creating specialized grippers that can attach to the jaws of existing instruments [13][14]. Another approach is to re-think the design of surgical instruments. The design of the force sensor can be integrated with that of the dexterous instrument [15][16][17], as shown in Figure 2. For reasons of cost, biocompatibility, and sterilizability, the forces applied to the patient would ideally be estimated without using force sensors. For patient-side robots designed with low inertia and friction, the difference between the desired and actual position of the patient-side robot (where the desired position is that of the master manipulator) is an indication for forces being applied to the environment. However, the fidelity of such systems are limited since there are dynamic forces present in most robots that are difficult to account for and often mask the relatively minute forces of interacting with the patient [18].

Figure 2.

A robotic surgery system for two-hand manipulation with integrated force feedback and 3D vision, designed by researchers at DLR, Germany. The system consists of a specially designed dexterous force-sensing instrument, robotic arms and teleoperation controller, and haptic device commercially available from Force Dimension, Inc. (Lausanne, Switzerland) as the master manipulator. Original figures used with permission from B. Kuebler, DLR.

Direct force feedback to the surgeon's hands can use conventional force display technology, in which the motors of the master manipulator are programmed to recreate the forces sensed by the patient-side robot. A dexterous surgical robot typically has seven degrees of freedom of motion, including translational, rotational, and gripping. However, not all of those degrees of freedom are actuated on the master. That is, the system cannot provide force feedback in certain directions. The effects may be negligible or detrimental, depending on the directions of force feedback lost [19][20]. The dynamics of the master manipulator can also limit the accuracy of the force display, but a more fundamental limitation is the trade-off between system stability and transparency for force feedback. A perfectly transparent telemanipulator is not possible because it would require perfect models of the master and patient-side robot dynamics, zero time delays from computer processing and information transmission, and perfect environment force sensing or estimation. As one pushes toward the limit of transparency, small errors and delays in the system can cause uncontrollable oscillations in a “closed loop” teleoperator – this is known as instability and would be unacceptable in surgery. An alternative approach is to use sensory substitution to display force, including audio feedback [21], graphical feedback [22][23], or other forms of haptic feedback such as vibrotactile display [24]. Substantial information about environment properties and forces can be acquired by simply observing visually how the patient's tissue and materials such as suture respond to motions of the surgical instruments. A design guideline is that sensory substitution through graphical overlays should not distract from the surgeon's view of the patient via the endoscopic camera(s) [25].

In the last few years, several research groups have used force sensing and feedback techniques described above to test the effectiveness of haptic feedback on surgeon performance and “outcomes” in phantom patients. All the experiments to date are preclinical. (Current commercial systems that use haptic feedback include those of Hansen Medical and MAKO Surgical Corp; however, no data exits demonstrating the relative effectiveness of those systems with and without haptic feedback.) Ortmaier, et al. [17] found that haptic feedback reduced unintentional injuries during a dissection task. However, operating time was longer than that of a manual intervention. Wagner and Howe [13] found that force feedback reduces potential tissue damage (as measured by the level of applied force) for both surgeons and non-surgeons, but only surgically trained individuals improve performance without a significant increase in trial time.

They hypothesize that this is due to the interaction between visual-spatial motor abilities and the information contained in the mechanical interaction forces. Cao et al. [26] used a virtual environment to demonstrate the surgeons performed a TransferPlace task faster and more accurately with haptics than without, even under cognitive load.

Mahvash et al. [27] used a modified da Vinci Surgical System to demonstrate that, in a palpation task, direct force feedback is superior to graphical force displays. Due to the limited fidelity of force feedback of their system (which did not use force sensors), users’ identification of hard lumps in artificial tissue was only correct for models with significantly different mechanical properties between the lumps and surrounding tissue. Zhou et al. [28] did a study of MIS that showed that with trocar friction, one of the undesirable forces that also arises in RMIS, surgeons’ force perception was degraded and the time to detect contact was longer. When friction was present, experienced surgeons detected contact with tissue faster than novices. Compared to no force feedback, Reiley et al. [25] found that graphical displays of applied force during a knot-tying task reduced suture breakage and overall applied forces, while increasing consistency of applied forces for inexperienced robot-assisted surgeons. In contrast to the direct force feedback results from [26], the results of Reiley, et al. suggest that graphical force feedback primarily benefits novices, with diminishing benefits among experienced surgeons. In a simple grasping task, Tholey, et al. [Tholey 2005] found that providing both vision and force feedback leads to better tissue characterization than only vision or force feedback alone.

One would expect that better performance is achieved with direct force feedback than graphical feedback; sensory substitution systems are unnatural and thus have a longer learning curve, and direct force feedback provides physical constraints that helps a surgeon make the correct motions simply due to dynamic force balance [29]. There is an alternative to force feedback from the environment that provides such useful physical constraints: virtual fixtures. These are software-generated force and position signals applied to human operators in order to improve the safety, accuracy, and speed of robot-assisted manipulation tasks [30]. For example, a virtual “wall” may be placed around a delicate anatomical structure to keep the surgical instruments from contacting it.

Although this article focuses on haptic feedback in actual surgeries, it is worth noting that the role of force feedback in training is a topic of current research. Virtual reality simulators have proven highly effective in developing laparoscopic minimally invasive surgery (MIS) skills, especially when force feedback is provided in early training [31][32]. However, accurate modeling of tissue-instrument interaction that gives rise to motions and forces relevant to outcomes is not yet achievable at computation rates that allow real-time interaction.

4. Tactile Feedback

Compared to force feedback, there has been relatively little work done in the area of tactile feedback for RMIS. In many surgical procedures, such as suture knot tying, force feedback is sufficient; the addition of contact location, finger pad deformation, and pressure distribution information may not be necessary [33]. However, palpation is one task for which deformation of the fingerpad seems to be particularly relevant [34][27], motivating the need for tactile feedback.

As in force feedback, tactile feedback systems require both a sensor and a display. The goal of tactile sensing in RMIS can be to detect local mechanical properties of tissue such as compliance, viscosity, and surface texture – all indications of the health of the tissue – or to obtain information that be used directly for feedback to a human operator, such as pressure distribution or deformation over a contact area [35]. Constraints in sensor design include cost, size, geometry (for example, to fit within a laparoscopic grasper), biocompatibility, and surface finish to allow grasping. Many sensors require some deformation of the sensor in order to measure distributed information; this necessitates flexible coverings, which also remove detailed, local information. In addition, data recording from tactile sensors is difficult because they often include many individual sensing elements; wireless communications are possible, but power must still be cabled to the instrument tip. Examples of tactile sensors include arrays of capacitive sensors [36]and force-sensitive resistors [37], instrumented membranes [38], and micromachined piezoelectric arrays [39]. Companies that sell tactile array systems include Pressure Profile Systems, Inc. (Los Angeles, CA, USA) and TekScan, Inc. (South Boston, MA, USA). Data relevant to tactile information can also be obtained through other means, such as laparoscopic ultrasound [40].

Tactile displays attempt to create the perception that the surgeon's fingertip is directly contacting the patient or surgical material such as suture. The most literal type of tactile display is an array of pins that are individually actuated (for example, [41]), so that their position commands are easily mapped from data from an array-type tactile sensor. Making such array-type displays for RMIS is very challenging due to size and weight constraints. The display must sit at the end of the master manipulator and not impede its maneuverability. Such pin displays developed for MIS and RMIS are actuated using shape-memory alloys [42], micromotors [43], and pneumatic systems [44][45]. The latter method allows the most lightweight display to be attached to the master manipulator, but requires infrastructure for air pressure, which can be noisy, and has limited resolution. Little work has been done to combine kinesthetic and tactile information for surgery, but one study demonstrates that the ability to maintain an appropriate force in the remote environment is necessary for the surgeon to take full advantage of the spatially distributed force information from the tactile sensor [46]. Graphical displays of tactile data can also be very compelling, especially for diagnosis applications [47][48][49]. Most of the tactile sensors and displays developed have not been tested in RMIS systems. Due to the complexity of integrating distributed tactile information into RMIS, it may be useful in the future to consider clever “tactile illusions” [50] and other display methods recently developed in the haptics research community.

6. Conclusion

Haptic feedback for RMIS is currently being developed and evaluated in engineering labs, and further development is required before these techniques are ready for clinical testing. Because the fidelity of haptic feedback and surgical scenario (e.g. degrees of freedom and type of surgery) of each research system is different, the results regarding the effectiveness of haptic feedback in the literature are not completely consistent. Contributions by neuroscientists to our understanding of how we perceive force and tactile information may affect how we design haptic displays. Promising new developments in the haptic technology and neuroscience fields may yield more efficient, practical force and tactile displays in the future. To accomplish these goals, surgeons, engineers, and neuroscientists need to work together to develop and test effective haptic displays for RMIS.