Force Feedback Plays a Significant Role in Minimally Invasive Surgery

Abstract

Objective:

To evaluate the role of force feedback with applications to minimally invasive surgery (MIS). Two research hypotheses were tested using our automated laparoscopic grasper.

Summary Background Data:

Conventional laparoscopic tools do not have the ability of providing force feedback to a surgeon when in use with or without robotic surgical systems. Loss of haptic (force and tactile) feedback in MIS procedures is a disadvantage to surgeons since they are conventionally used to palpating tissues to diagnose tissues as normal or abnormal. Therefore, the need exists to incorporate force feedback into laparoscopic tools.

Methods:

We have developed an automated laparoscopic grasper with force feedback capability to help surgeons differentiate tissue stiffness through a haptic interface device. We tested our system with 20 human subjects (10 surgeons and 10 nonsurgeons) using our grasper to evaluate the role of force feedback to characterize tissues and answer 2 research hypotheses.

Results:

Our experiments confirmed 1 of our 2 research hypotheses, namely, providing both vision and force feedback leads to better tissue characterization than only vision feedback or only force feedback.

Conclusions:

We have validated 1 of our 2 research hypotheses regarding incorporating force feedback with vision feedback to characterize tissues of varying stiffness.

This paper focuses on evaluating the role of force feedback in minimally invasive surgery using an automated laparoscopic grasper developed in our laboratory. Several experiments with human subjects were performed to characterize various tissue samples using only vision feedback, only force feedback, both vision and force feedback, and direct exploration.

Minimally invasive surgical (MIS) procedures using long instruments have profoundly influenced modern surgery by decreasing invasiveness, therefore minimizing patient recovery time and cost. However, surgical procedures using long tools inserted through small ports on the body deprive surgeons of the depth perception, dexterity, sense of touch, and straightforward hand-eye coordination that they are accustomed to in open procedures. Surgeons have traditionally relied on the combination of their tactile and visual abilities to diagnose tissue as normal or abnormal.¹

The use of laparoscopic tools capable of providing force feedback is lacking in current robot-assisted surgical systems. Current robotic surgical systems, while providing excellent visual feedback, are incapable of providing force feedback. Surgeons who operate using these robotic assistants do not have force and tactile feedback from the operative site as they are now indirectly in contact with the surgical site instrumented tools. Therefore, the need to incorporate force feedback capabilities into MIS procedures, especially robotically assisted procedures, provides an excellent opportunity to improve the quality of surgical procedures. Improvements in MIS systems will lead to significant societal impacts through better patient care, reduced morbidity, shorter hospital stays, reduced trauma, faster recovery times, and lower health care costs.

While research has been done on developing laparoscopic tools capable of providing force feedback, their use in surgical practice is limited by the size of these instrumented laparoscopic tools and the ease of use in surgical procedures. As a result of these shortcomings, our goal was to (a) design and develop a modular laparoscopic tool, which would provide force feedback for tissue characterization; and (b) using the developed laparoscopic grasper, we wanted to evaluate the role of force feedback in MIS. In this paper, we will focus primarily on the second goal, namely, the evaluation of the role of force feedback in MIS. We have developed an interface to aid the surgeons in characterizing artificial tissues of varying stiffness (prepared using hydrogels), which are a reasonable approximation to the tissues encountered by surgeons in liver tumor resection. The soft tissues in our experiments will approximately represent normal liver tissue, medium tissues will approximately represent a tumor in the formation stage, and hard tissues will approximately represent a fully developed tumor. These tissue samples were tested by palpation by our surgeon collaborator to make a realistic choice of artificial tissue samples in our experimental studies.

Several researchers have already proposed solutions to incorporate force feedback into current laparoscopic tools.^2–8 Many of these solutions involve adding force sensors, such as strain gauges, to record forces at the tool tip or developing a new robotic manipulator used to operate laparoscopic tools and obtain force feedback. Additional work has been done in creating new laparoscopic tools with force feedback incorporated into their designs.^9–12 However, there still exist many problems within the designs of laparoscopic tools and their use in robotic surgery.^13,14 Schurr et al¹⁵ have developed a 2-arm, master-slave manipulator called ARTEMIS for use by surgeons. Dario et al¹⁶ have developed a miniature robotic surgical system for colonoscopy. Several master-slave robotic surgical systems have also been commercially developed. However, these systems do not provide force feedback to the surgeon. Current research within the area of force feedback has led to the use of cable-driven mechanisms to reduce backlash and play in the mechanism.^17–19

MATERIALS AND METHODS

Prototype Development

In this section, we briefly describe the laparoscopic grasper that was designed and developed in our laboratory for evaluating the role of force and vision feedback in surgery. Our design for the laparoscopic grasper with force feedback capability was guided by constraints associated with current laparoscopic tools. The constraints were a shaft diameter of 15 mm, a 90-degree angle between the jaws at the maximum open position, and accurate positioning of the jaws in relation to the handle of the tool. Using these constraints, we developed a cable-driven pulley mechanism controlled by a DC motor for the actuation of the jaws (see Fig. 1a). The entire assembly comprised of the laparoscopic grasper and motor was mounted on an aluminum plate that could be attached to a robot arm for teleoperation (see Fig. 2). Each steel cable, connected to a pulley mounted on the motor, traveled through the main shaft to a corresponding pulley for each jaw. At each jaw pulley, the cable wrapped around the pulley in the opposite direction compared with the other (one clockwise, one counterclockwise). This provided the opposing motion for the opening and closing of the jaws and also allowed for accurate positioning of the jaws. The initial prototype was designed as a grasper and contained 2 serrated jaws to facilitate grasping of tissue. A tensioning device for the steel cables was also added to allow for easy adjustments in the mechanism. The novelty of our design is that it can also incorporate tissue-cutting blades (as shown schematically in Fig. 1b) instead of the grasper jaws (and they are readily interchangeable) due to the quick-change feature in our prototype. This was achieved by mounting the jaws on the outside of the shaft by the use of a screw. This change was included as concurrent research in our laboratory consists of studying cutting and dissection tasks for reality-based modeling of tool and tissue interaction forces in MIS.

FIGURE 2. The laparoscopic grasper attached to the Mitsubishi robot arm for tissue manipulation.

Characterization of Tissue Samples

The first experiment was to determine the effectiveness of our automated grasper to correctly characterize tissues of varying hardness through a haptic- and vision-based display before proceeding to experimental studies with human subjects on evaluating the role of force feedback in MIS. We tested soft, medium, and hard tissue samples. The soft tissue samples approximately represented normal liver tissue, while the medium tissue samples approximately represented a tumor in the formation stage, and the hard samples approximately represented a fully developed tumor. Softer samples required less force to deform, while harder samples required more force for the same deformation. To successfully complete this experiment, we developed several artificial tissue samples from hydrogel material. The hydrogel was created using a combination of polyvinyl alcohol (PVA) and polyvinyl pyrrolidone (PVP) in the consistency of 99% PVA and 1% PVP in a 10% concentration. This solution was then cast into molds and subjected to several freezing/thawing cycles. Six cycles were performed, and after each cycle a sample was removed. Sample 1, which was removed first, represented the softest tissue, while sample 6, which was removed last, represented the hardest tissue. We performed compression tests on the 6 artificial tissue samples on an Instron machine to verify that there was a marked difference in the tissue stiffness prior to using the tissue samples in our research hypothesis testing. We experimentally observed a significant difference in the tissue hardness for all the 6 samples.

We chose 3 different tissue samples for the tissue characterization experiment and used the automated laparoscopic grasper developed in our laboratory for this experiment. The samples were grasped in the laparoscopic tool, and measurements of the grasping force and angular displacement of the jaws were recorded. The grasping force was calculated by the amount of current applied to the motor. Figure 3 shows a plot of the grasping force as felt by the user while grasping tissues of varying hardness. As seen from the figure, for the same jaw displacement, tissue sample 6 had a higher grasping force than samples 3 and 1. This confirmed that the automated grasper was capable of providing force feedback, which varied with the tissue hardness.

FIGURE 3. Tissue characterization graph for the three artificial tissue samples showing force vs. angular position of the jaw.

Experimental Setup and Research Protocol for Tissue Characterization

We were interested in evaluating the role of force feedback in tissue characterization. For this we used only vision feedback, only force feedback, a combination of vision and force feedback, and direct exploration with the tissue samples. To prevent any learning, we gave the subjects adequate time to practice with the system before the experimentation began (details in Results section).

Vision (V) Feedback Protocol

For the first part of the experiment, the subject was presented with 3 random tissue samples of varying consistency to be grasped with the prototype. The first test consisted of using only vision feedback from the remote site to rank the samples from softest to hardest. The subject was able to view the grasping of the samples through a video screen to view the deformation of the sample. For this experiment, we used a standard CCD video camera that was approximately 1 foot in the horizontal position and approximately 6 inches in the vertical, from the sample, therefore giving an isometric view of the grasper jaws and the tissue sample. The jaws closed on the tissue to a desired angle to show the subject the deformation of the tissue. One sample at a time was grasped, and the subject could view the grasping on a video screen. Once the 3 samples (soft, medium, and hard) had been presented, the subject was asked to rank them from softest to hardest based on their observations. It should be noted that during each trial, the subject could request a replay of 1 or more of the samples before ranking the samples, therefore reducing error due to random guessing or forgetting which sample was soft, medium, and hard. The subject then ranked the 3 samples, and the process was repeated for a total of 5 trials. Each trial was repeated with the same samples randomly arrayed to prevent any memorization of the samples.

Force (F) Feedback Protocol

The second test consisted of the same method with the exception of using force feedback instead of vision feedback to characterize the tissue. For this experiment, the subject interacted with the PHANToM (haptic interface device manufactured by Sensable Technologies, Woburn, MA) by inserting their finger in a thimble attached to the arm of the PHANToM (see Fig. 3). The direction of the force feedback within the PHANToM was vertical and acting upwards. Therefore, the subject was required to hold the edge of the desk with their thumb under the desk and other 3 fingers resting on top of the desk. Through the keyboard, an operator opened and closed the jaws while the subject received force feedback from each sample. This setup was representative of palpation (using the thumb and forefinger) done by surgeons (although virtual) where arm dynamics do not affect the results. The grasper was set to close the jaws to the same angle for each sample to keep the deformation constant. Therefore, the only variable that existed was the amount of force required for the deformation, and this force was based on the tissue's stiffness. After the closure of the jaws to the prespecified angle (same as in the vision feedback experiment), the subject could manipulate the PHANToM with their index finger to characterize the amount of grasping force for that particular sample. This process was repeated for the 3 tissue samples in the trial, and the subject was given the option to feel any or all of the samples again. The subject then ranked the 3 samples in order from softest to hardest. This process was also repeated for a total of 5 trials.

Vision and Force (V + F) Feedback Protocol

The third test consisted of using both force feedback and vision feedback to differentiate between the 3 tissue samples. This test was performed in the same way as the force test with the exception of having video feedback of the grasping. The subject was able to view the deformation of the tissue while also receiving force feedback from the PHANToM. Again, the subject was presented with the 3 samples and was asked to rank them from softest to hardest. An opportunity for a replay of any or all samples was given during each trial. A total of 5 trials were performed for this part.

Direct Exploration (DE) Protocol

The final test of this experiment was to ask the subject to rank the 3 tissue samples from softest to hardest based on direct exploration of the samples with their fingers. Only 1 trial was required for this protocol.

Data Collection and Analysis

The experiment was performed by a total of 20 subjects for a total of 900 trials (5 trials each for V, F, V + F, and 3 tissue samples to be characterized; hence 45 trials by each subject). Ten subjects were surgeons who had experience in MIS, and the other 10 were nonsurgeons who had no surgical or medical background. The data were collected in a qualitative fashion with responses characterized as either “true” or “false” and denoted by a value of 1 or 0, respectively. For a particular trial of all 3-tissue samples to be characterized as true, the subject was required to identify all 3-tissue samples in their correct order of stiffness. The data were then analyzed using a single-factor analysis of variance (ANOVA) method. ANOVA generates a P value (probability) for the null hypothesis (H₀) and thus a probability for the research hypothesis (H₁) to be tested. The lower the P value, the smaller the probability for the null hypothesis to be true and consequently higher is the probability that there is a significant statistical difference between the data sets (or the research hypothesis H₁ is true). The level of significance (α value) for our statistical analysis was chosen to be 0.05, meaning that a hypothesis would be considered true if P < α. In addition, Tukey's method was used as the post hoc ANOVA test. Tukey's method is used to test all pairwise mean comparisons after an ANOVA is performed on the data set. The ANOVA analysis can only be used if the dependent variance is the same in each population, which was not the case in our experimental observations. Therefore, Tukey's method must be used for accurate determination of statistical significance. This method will declare that 2 means are significantly different if the absolute value of the difference of the means exceeds:

where α = level of significance (0.05), a = number of populations, f = number of samples taken − number of populations, MSE = mean square error, and n = number of samples per population. The resulting value (T_α) represents the number at which the level of significance is α.

RESULTS

Research Hypotheses

We performed the above experiments to test the validity of 2 research hypotheses that have direct clinical relevance. These are:

1. Providing only force feedback leads to better tissue characterization for all 3 tissue samples compared with using only vision feedback for all 3 tissue samples (F_a > V_a).

2. Providing vision and force feedback simultaneously for all 3 tissue samples is better than only vision feedback or only force feedback for all 3 tissue samples ((V + F)_a > V_a, (V + F)_a > F_a).

The “>” sign denotes “is better than” in the above hypotheses. Subscripts “s,” “h,” and “a” refer to soft tissue, hard tissue, and all tissue samples, respectively.

Evaluating the Role of Force-Feedback in Tissue Characterization

The effect of learning in the experiment was closely examined to prevent any tainting of the data. As explained above, the tissue samples were randomly presented in each trial, and subjects were also given an opportunity for a replay to eliminate any random guesses. Subjects were also given sufficient time to experiment with a sample tissue (not used in the actual experiment) to become acquainted with the setup and the PHANToM. Each subject spent at least 10 minutes practicing until they felt comfortable to begin the experiment. The order of the experiments (vision feedback alone, force feedback alone, simultaneous vision and force feedback, and direct exploration) was limited by the type of feedback and therefore could not be randomized. Vision feedback alone and force feedback alone must be tested before simultaneous vision + force feedback and direct exploration because of the presence of more than 1 sensory clue. If the above approach is not followed, the subjects may link a force felt by a particular sample to the visual cues from that sample and this may contribute to a learning effect. The trials were thus presented in the following order: vision feedback alone, force feedback alone, simultaneous vision + force feedback, and direct exploration.

To verify that there was indeed no learning effect in our experiments, we examined the data after the experiments were completed with all the subjects. As shown in Figure 4, for each type of feedback (ie, vision feedback alone, force feedback alone, and simultaneous vision + force feedback), there was no steady increase in the percentage correct response as the trials progressed. It can thus be concluded that the subjects did not exhibit a learning effect as the trials progressed. The individual results for each subject testing the 3 tissue samples in each of the 4 methods, namely, V, F, V + F, and DE, are shown is Figure 5 based on the protocol outlined in Experimental Setup and Research Protocol for Tissue Characterization. Overall, the average correct response using only vision feedback was 52%, while the average correct using only force feedback was 67%. Providing both vision feedback and force feedback produced an average of 83% and direct exploration produced an average of 100% correct. Excluding direct exploration, our results confirm that on the average, subjects are more comfortable characterizing tissues when both vision and force feedback were provided (ratio of the average correct response to the corresponding standard deviation was the highest with V + F compared with only V or only F). An ANOVA analysis was performed to evaluate whether there was a significant difference between the 3 data sets (V, F, V + F). The P value obtained when comparing all 3 data sets was 0.00001325, leading to a probability of greater than 99.99% that there was a significant statistical difference between the data sets. However, further analysis is needed to determine which data sets are significantly different, which will be addressed in each of the hypotheses below.

FIGURE 5. Percentage of successful tissue characterizations for each individual using each of the four methods. The histogram for each subject is always presented in the order V, F, V + F, and DE as shown in the figure for subject 1.

Hypothesis 1

Providing only force feedback leads to better tissue characterization for all 3 samples compared with using only vision feedback for all 3 samples (F_a > V_a).

In our first hypothesis, the subjects were asked to characterize all 3 tissue samples correctly in any given trial. As shown in Figure 6 (a), providing vision feedback alone resulted in differentiating the tissue samples correctly in 50% of the trials for all subjects, while using only force feedback resulted in 67% of the trials being correct. Tukey's method was applied to the vision feedback and force feedback data sets using a level of significance of 0.05. The value of T_0.05 was calculated to be 0.1525, and the difference between the means of the data sets was 0.15. Therefore, since the difference of the means was less than the value of T_0.05 (although marginal), we cannot say with confidence that there is a statistical difference between only vision feedback and only force feedback. However, the values are very close, and significance of the null hypothesis was calculated to be 0.052, which is just above the level of significance (α = 0.05).

Individually, surgeons were correct in 62% and 72% of the trials for only vision feedback and only force feedback, respectively, while nonsurgeons were correct in 42% of the trials using only vision feedback and 62% of the trials using only force feedback (see Figs. 6b and c) for identifying all 3 tissue samples correctly. We analyzed the statistical difference between V_a and F_a for the surgeons’ data set and the nonsurgeons’ data set. Using Tukey's method, the calculated value of T_0.05 was 0.2095 and 0.2194 for surgeons and nonsurgeons, respectively. The difference in the means for vision feedback and force feedback for surgeons and nonsurgeons was 0.10 and 0.20, respectively. As shown, both of these values are less than the calculated T_0.05, and the significances calculated for these were 0.495 and 0.081 for surgeons and nonsurgeons, respectively. Therefore, we cannot conclude that surgeons will perform better tissue characterizations with only force feedback than with only vision feedback. However, there is a significant difference between the surgeons and nonsurgeons using the 2 methods. This is because Tukey's method is a measure of the difference between the 2 data sets (vision feedback and force feedback). The surgeons had a smaller difference (10%) than the nonsurgeons (20%) because of their experience using vision to characterize tissues in laparoscopic procedures, which leads to a higher significance of the null hypothesis (0.495) for surgeons compared with nonsurgeons (0.081). However, surgeons’ overall percent correct was higher than that of nonsurgeons.

Hypothesis 2

Providing vision and force feedback simultaneously for all 3 samples is better than only vision feedback or only force feedback for all 3 samples ((V + F)_a > V_a, (V + F)_a > F_a).

Our second hypothesis stated that (V + F)_a was better than V_a or F_a for tissue characterization. As shown in Figure 7 (a), the percent correct for vision feedback, force feedback, and simultaneous vision and force feedback was 52%, 67%, and 83%, respectively, for identifying all 3 tissue samples correctly in any given trial. For direct exploration method, the subjects characterized the 3 tissue samples correctly 100% of the time. Tukey's method was used to analyze the entire data set comparing (V + F)_a to V_a and (V + F)_a to F_a. As shown in the first hypothesis, T_0.05 was calculated to be 0.1525 for all subjects. The difference of the means between (V + F)_a to V_a was 0.31 and the difference between (V + F)_a to F_a was 0.16. Since both of these differences are higher than T_0.05, there is a significant statistical difference between the data sets compared. Also, the corresponding significance of H₀ for comparing (V + F)_a to V_a was less than 0.001 and for comparing (V + F)_a to F_a was 0.035, leading to confidence levels of 99.99% and 96.5%, respectively, for the research hypothesis H₁ to be true. Therefore, there is a significant statistical difference between (V + F)_a and either V_a or F_a.

Individually, the surgeons’ performance was 62%, 72%, and 84% for V_a, F_a, and (V + F)_a, respectively, and nonsurgeons’ performance was 42%, 62%, and 82% for V_a, F_a, and (V + F)_a, respectively (see Figs. 7b and c) for identifying all 3 tissue samples. Tukey's method was used for the data set containing only surgeons and the data set containing only nonsurgeons. As shown in hypothesis 1, the value of T_0.05 for surgeons and nonsurgeons was 0.2095 and 0.2194, respectively. The difference of the means for the surgeons’ data sets was 0.22 for (V + F)_a versus V_a, and for the nonsurgeons’ data sets, the difference of the means was 0.40 for (V + F)_a versus V_a. For surgeons, the difference of the means for (V + F)_a versus V_a (0.22) is greater than T_0.05 (0.2095); therefore, we can conclude that providing simultaneous vision and force feedback to the surgeon is better than just vision feedback. For nonsurgeons, the difference of the means for (V + F)_a versus V_a (0.40) is greater than T_0.05 (0.2194). Therefore for nonsurgeons also, providing simultaneous vision and force feedback is better than just vision feedback. Individually, the calculated significances of the null hypothesis for (V + F)_a versus V_a was 0.036 for surgeons and less than 0.001 for nonsurgeons. Hence, we can conclude on an individual basis also that surgeons and nonsurgeons performed better with simultaneous vision and force feedback than only vision feedback. Thus, our claim that current MIS procedures can greatly benefit from the addition of force feedback to vision feedback is validated.

DISCUSSION

We have developed the initial prototype of our automated laparoscopic grasper with force feedback capability along with an information-enhanced display to provide vision and force feedback to the user while manipulating tissues. This tool incorporates a quick-change feature for easy conversion between grasper, cutter, and dissector, as well as accurate positioning of the jaws. These improvements over conventional tools have made it easier for us to use it with robotic systems while still retaining advantages of conventional laparoscopic surgery. The experimental work has demonstrated these advantages by showing that an operator can easily differentiate between tissue samples of varying stiffness. Our results confirm that subjects are more comfortable and more accurate at characterizing tissues with simultaneous vision and force feedback compared with vision feedback alone or force feedback alone. Overall, considering all 20 subjects, while hypothesis 1 (F_a > V_a) was not proven correct, hypothesis 2 ((V + F)_a > V_a, (V + F)_a > F_a) was validated through our experimental results. Though hypothesis 1 (F_a > V_a) was not validated, the significance (0.052) was very close to the 95% cutoff limit. The statistical analysis has also shown that nonsurgeons performed significantly better with simultaneous vision and force feedback interface than with vision feedback alone or force feedback alone. This demonstrated that an inexperienced person could easily characterize different tissue samples if they are provided with both vision and force feedback. The performance of the nonsurgeons can be attributed to the high difference in the percentage correct between the various types of feedback, which causes a lower P value, as this is a measurement of the difference between data sets. The surgeons had higher P values since their difference in the percentage correct was much lower than nonsurgeons because of their training and experience in characterizing tissues using only vision feedback in conventional laparoscopic surgery. The results of this study will lead to a better understanding of the design of automated laparoscopic tools that can provide force feedback in surgical procedures, simulation, and training.

Although we have successfully proved that force feedback plays a significant role in MIS, there were limitations in our experiments as compared with a surgical setting. First, we have only done studies on grasping, rather than palpation. This was primarily due to the limitation of the hardware. Future prototypes will be designed to overcome this limitation. Second, the visual field in our setup was restricted by a stationary camera as opposed to a movable endoscope that would be used in a laparoscopic procedure. The vision feedback part of the experiment also restricts the feedback to pure vision, while in conventional laparoscopic procedures the surgeon has some force feedback through the surgical tool (primarily through the reaction force of the tool contacting an organ or tissue). However, this force feedback, though limited in laparoscopic procedures, is completely lost in robotically assisted laparoscopic procedures. While an experienced surgeon trains their perceptual faculties to infer the interaction of the laparoscopic tool with the tissue through primarily vision feedback (from our experiments, some surgeons had all correct responses with vision feedback alone), it is reasonable to assume (based on our experimental data) that over time, the perceptual ability to combine vision and force information received from automated tools is possible. Third, another simplification we have made in our studies, compared with an actual clinical setting, is that the subject could only fully open or fully close the automated grasper, whereas a surgeon has more control in a surgical setting to allow for palpation of the tissue by controlling the “angle of closing” of the grasper jaws.

In addition to the experimental setup limitations, there are several issues with the apparatus that we need to address in future research. Future versions of our laparoscopic tool must be scaled down further to incorporate this tool in all commonly used laparoscopic procedures. This can be accomplished through small changes in the design, which will allow for a smaller shaft and jaws. Currently, we use motor current to estimate the grasping force at the jaws. In future versions, we envision using small piezoelectric sensors placed on the jaws to record the actual forces exerted on the tissue. This would allow for a very accurate characterization of the tissue sample via a haptic interface device than using the current-based indirect estimation method as in the present prototype. In conclusion, the experimental results and the analysis presented in this work validates our claim that force feedback added to vision feedback leads to an improvement in tissue characterization than using only vision feedback for MIS.

ACKNOWLEDGMENTS

We would like to acknowledge the support of National Science Foundation grants EIA-0079830, EIA-0312709, and CAREER Award IIS-0133471 for this work.