Toward a novel soft robotic system for minimally invasive interventions

Abstract

Purpose

During minimally invasive surgery, surgeons maneuver tools through complex anatomies, which is difficult without the ability to control the position of the tools inside the body. A potential solution for a substantial portion of these procedures is the efficient design and control of a pneumatically actuated soft robot system.

Methods

We designed and evaluated a system to control a steerable catheter tip. A macroscale 3D printed catheter tip was designed to have two separately pressurized channels to induce bending in two directions. A motorized hand controller was developed to allow users to control the bending angle while manually inserting the steerable tip. Preliminary characterization of two catheter tip prototypes was performed and used to map desired angle inputs into pressure commands.

Results

The integrated robotic system allowed both a novice and a skilled surgeon to position the steerable catheter tip at the location of cylindrical targets with sub-millimeter accuracy. The novice was able to reach each target within ten seconds and the skilled surgeon within five seconds on average.

Conclusion

This soft robotic system enables its user to simultaneously insert and bend the pneumatically actuated catheter tip with high accuracy and in a short amount of time. These results show promise concerning the development of a soft robotic system that can improve outcomes in minimally invasive interventions.

Introduction

Minimally invasive surgery (MIS) involves using flexible tools to navigate through complex and confined anatomical spaces for diagnosis and treatment [1]. Tools that offer the ability to control and steer their position while inside the body are especially important for overcoming procedural challenges, such as maintaining position despite the dynamics of a beating heart [2], accessing lung nodules [3], or navigating the brain’s tortuous vasculature to treat aneurysms [4]. Specifically, major arteries in which aneurysms are commonly found in the heart and brain have diameters as small as 4mm and 2mm, respectively [5–7]. An effective robotic system, therefore, should enable positioning that is accurate enough to navigate through branch vessels and aneurysm necks of these dimensions. To achieve these ends, soft robots are often utilized, since they are composed of flexible materials and are inherently safe and compliant, as opposed to traditional robots that have rigid links controlled by electromechanical motors [8].

A variety of actuation mechanisms and control schemes are used for soft robots in MIS. Intuitive Surgical’s Ion System [9] and Johnson & Johnson’s Ethicon Monarch System [10] offer flexible cable-driven tools for robotic bronchoscopy and a remote suite with joysticks and buttons to control tool movement. Other cable-driven systems include Merit Medical’s SwiftNINJA microcatheter for neurovascular surgery [11] which has a dial at its proximal end to adjust the bending angle, and [12], in which authors use thermal drawing to create tools from multiple different materials. Although cable-driven systems are commonly used in commercial applications and have robust kinematics models, friction forces make the distal end difficult to control through adjustments at the proximal end, especially when the tool has traversed a long and tortuous path [13]. Mechanisms such as magnetic and fluid-driven actuation circumvent this issue since the actuation of the tip is dependent on an applied magnetic field or pressure and is not hindered by the configuration of the tool across its full length.

As an alternative to cable-driven systems, researchers have explored magnetic actuation strategies to actively control the steering directions of soft catheters. By integrating permanent magnets at the tip of the catheter, external magnets can be utilized to induce magnetic forces capable of bending the catheter tip [14, 15]. Unfortunately, the magnetic force exponentially decreases as the size of catheter magnets decreases, and the manipulation distance of the internal and external magnets increases [16]. In addition, the permanent magnets need to be precisely relocated and reoriented to control the magnetic fields, thereby necessitating the use of robotic manipulators which can impose a potential workspace conflict with other surgery equipment such as C-arm scanners [17].

Compared to magnetic actuation, fluid-driven soft catheters demonstrate the potential for achieving linearized control with simpler external control systems. For example, Gopesh et al., in an in vivo study, demonstrated a microcatheter with a soft robotic tip that used individual channels to induce bending in a particular direction when pressurized [18]; however, channels are actuated individually by hand such that the surgeon is required to pause and tune the angle at key points while traversing the anatomy, which is less efficient than remote steering. Similarly, Li et al. demonstrated a soft robotic catheter that extends from its tip when pressurized [19], but the angle of extension is built into the catheter at certain points along its length, preventing steering the tip to arbitrary angles during the procedure. Ikuta et al. developed a closed-loop pressure controller for a hydraulic device with series bending segments along the length of the catheter [20], but do not address how to build an interface for the system that would allow intuitive operation of the catheter during a procedure. In addition, considerable effort has been put toward designing and characterizing novel soft pneumatic robot designs [21–29], developing accurate models of soft pneumatic robots [30], and characterizing the strain behavior of common soft robot materials [31, 32]. While these efforts toward accurate characterization of soft robots serve as the foundation of our work and will inform future design iterations, they do not address how such devices can be integrated into the clinical workflow for MIS.

A critical need for active catheter navigation is controllers that can be integrated into the clinical workflow, in which traditionally surgeons insert and rotate the catheter at the entry point. Commercial platforms give surgeons full remote control through the use of joysticks, trackballs, sliders, and other similar devices in a radiation-shielded suite. Similarly, several studies have focused on developing control platforms for catheter manipulation [33–35], but they rely on robotic systems that control both tip insertion and tip angle. Given their greater complexity, these systems can be slower to implement and fail to replicate the entire clinical workflow experience, which includes haptic feedback the surgeon would traditionally experience [36]. We intend to supplement the clinical workflow by allowing the surgeon to manually insert the catheter while simultaneously adjusting the tip angle as they see fit.

The contribution of this work is the development of a soft robotic system that: (i) utilizes a pneumatic actuation approach to simplify both the mechanical and control architecture and (ii) provides an on-demand means of adjusting the tip angle that allows surgeons to intuitively control both bending and insertion without the cumbersome task of manually pressurizing individual channels. Further, (iii) we validate the performance of the system in a positioning task with a human in-the-loop with both a novice and skilled surgeon.

Methods

Scope and concept

In this study, we develop a small segment for a steerable catheter tip. To build an entire catheter, the actuated catheter tip can be placed at the distal end to control the bending or placed in series with other actuators to produce multiple active segments as shown in Fig. 1. In our prior work, we have shown the ability to 3D print microscale fluidic structures [37] and actuators [38]; however, in this study, we utilize a macroscale prototype as an exemplar with which to demonstrate the control system concept. In addition, the prototype is not designed directly for surgical intervention, i.e., there is no inner lumen to pass auxiliary devices.

Fig. 1.

Potential implementations of our soft robot actuator. a A single actuator placed at the distal end of the catheter to selectively traverse bifurcations b Multiple appended actuators allowing more complex shapes

Actuator design

We developed a bidirectional system of actuators, with each actuator operated by a separate pressure input. A bellowed design is effective for pneumatic soft robots since most of the bellow surface area is perpendicular to the actuator’s length and thus pressure acting normal to this surface primarily causes extension (therefore bending) rather than radial expansion. Previously, researchers have utilized additive manufacturing to fabricate systems of soft robotic actuators [39, 40]. We designed our actuators to be fabricated using a liquid crystal display (LCD) 3D printing process as shown in Fig. 2a. LCD printing provides an inexpensive and accessible method to print elastic resins. Both actuators have a wall thickness of 225 μm and contain 4 bellows. The bellows each have a diameter of 4.7 mm and a height of 1.44 mm. A clearing hole was included at the top of each actuator to facilitate the removal of uncured resin following the printing process. Bidirectional actuation is achieved by applying pressure to each actuator individually as shown in Fig. 2b.

Fig. 2.

Manufacturing and operation of soft robotic catheter tip. a Printing process for the soft robot actuator b Pressurizing the leftmost channel inflates the bellows and bends the catheter to the right, and pressurizing the rightmost channel bends the catheter to the left

Manufacturing

The actuators for the catheter tip were modeled using computer-aided design (CAD) software, SolidWorks (Dassault Systems, France), and then exported as an STL file. This file was subsequently imported into a slicing software, ChituBox (ChituBox, China) for generation of printing layers and relevant parameters for the Elegoo Mars 3 LCD 3D printer (Elegoo, China). The actuators were printed with RESIONE F80 Elastic 3D Printer Resin (Dongguan Godsaid Technology Co., China) with a layer height of 50 μm. Following the LCD printing process, the actuators were rinsed with isopropyl alcohol (IPA) to remove uncured resin on the surface of the actuators. Each actuator was perfused with IPA, then dried using pressurized nitrogen (N2) gas to facilitate the removal of the remaining uncured resin within the actuators. The perfusion and drying process was repeated as needed to remove all uncured resin, and the actuators were post-cured under UV light for 3 min. After curing, the clearing holes at the top of the actuators were sealed using UV glue, and tubing was connected to the actuators using a waterproof epoxy.

Control architecture

The control system architecture is shown in Fig. 4. A personal computer (PC) is used as the central controller, connecting directly to the three peripherals: a myRIO-1900 real-time embedded evaluation board (National Instruments, USA) to control the hand controller, an Elveflow OB1-MK4 microfluidic flow controller (Elveflow, France) to actuate the catheter tip, and a Basler ace 2 USB3 area scan camera (Basler, Germany) to record catheter motion. The hand controller is shown in Fig. 3 and consists of a Maxon RE30 Brushed Motor (Maxon, Germany), E5 Optical Encoder (US Digital, USA) enclosed in a 11 cm × 11 cm × 9.5 cm 3D printed housing in addition to an ESCON 50/5 DC motor controller (Maxon, Germany). System software is developed in LabVIEW (National Instruments, USA).

Fig. 4.

Control system flow of information. The PC acts as a central controller, receiving the desired angle from the myRIO and mapping it into a pressure command for the flow controller to output to the catheter tip. The PC receives and logs the pressure measurement, command, video recording, and desired angle

Fig. 3.

Hand controller operation. a A counter-clockwise rotation of the hand controller bends the catheter tip to the left b Top view of the hand controller, consisting of a dial for the user to twist with their hand along with an angle marking plate and indicator to show the desired angle being sent to the flow controller c. A clockwise rotation of the hand controller bends the catheter to the right

After observing and studying neurointerventionalists performing MIS procedures, it became apparent that the controller needs to be placed in proximity to the catheter insertion site and control needs to be accomplished with small hand motions in order to enable intuitive and coordinated insertion and steering. To control catheter motion, the user turns the controller dial with their hand. The dial can be turned with one hand (even a few fingers), freeing up the user to also insert the catheter. The controller is equipped with a motor to enable haptic effects during both position and velocity control. During position control, the angle that the user turns the dial is set as the desired angle for the catheter tip. Simultaneously, the motor exerts a torque on the user that is proportional to the velocity of rotation, thereby creating a virtual damping field that prevents the user from moving the dial too fast and reduces the effects of inadvertent movements. During velocity control, the angle of the controller dial is set as the desired velocity of catheter tip. Given this desired velocity, Euler integration gives the desired catheter tip angle. The applied motor torque during velocity control corresponds to a virtual spring-damper system, which pulls the dial back to the zero-position similar to a typical joystick.

The myRIO and PC LabVIEW virtual instruments (VIs) communicate with one another through a network-published shared variable containing the desired catheter angle. The PC reads this value, computes the necessary pressure command using the device-specific calibration, and sends this command to the flow controller. The flow controller inner-loop achieves the desired pressure output to each channel of the catheter. In parallel, the PC records video of the catheter’s movement and other diagnostics such as commanded and actual pressure, and desired angle. During calibration, the video is post-processed to extract the tip angle of the catheter.

Experiments and results

Characterization

The calibration procedure is shown conceptually in Fig. 5. The catheter tip was held in the plane of the camera with a 3D printed attachment fixed to the camera mount. To achieve angle-recognition during post-processing, four distinct spots were colored in blue on the catheter tip: two at the base and two at the distal end. The center of each of these spots serves as endpoints for two vectors. The angle between these two vectors is equivalent to the angle between the two ends of the catheter tip and is used to describe the configuration of the catheter tip.

Fig. 5.

a Pressure command used to calibrate the catheter. Pressure is increased in eight equal-sized steps to the maximum pressure and decreased back to zero b. The angle between the base and the distal end of the catheter tip is calculated using OpenCV in Python (with colored markers) c. Each pressure is associated with its corresponding steady-state pressure to create a pressure-angle mapping

To calibrate, one channel’s pressure was increased discretely in eight equal-sized steps lasting five seconds each, up to the channel’s maximum pressure. This process was repeated ten times with a five second break in between repetitions and then again for the second channel after a ten minute break. An average of the last ten pressure values (which covers the last half second) on each step gives the steady-state angle value.

The pressure-angle mapping for two catheter tip prototypes is shown in Fig. 6. The figure shows the first load alongside the remaining loads averaged, with the standard deviation at each set point as error bars. Interestingly, the pressure-angle relationships of each mapping are strikingly different. Although both catheter tips have a similar angle range—83 degrees for catheter tip 1 and 70 degrees for catheter tip 2—catheter tip 1 reaches the edge of the range at 400 kPa in comparison to 200 kPa for catheter tip 2. The angle uncertainty is approximately ±1.2 degrees, corresponding to half of the maximum deviation in angle calculation over five seconds when the tips are not pressurized or moving. The angle resolution can be approximated by 0.003 degrees, corresponding to the minimum difference between two angles in a given run. Although both catheter tips exhibit hysteresis and relaxation after the first loading, both of these effects are much more pronounced in channel 1 of tip 1 as compared to the other channels. These discrepancies show the need for device-specific calibration to develop an effective controller. The final mapping is obtained by averaging the steady-state angle values at each point along the curve across load cycles, excluding the first load cycle. For control in the following experiment, it is first determined based on the current and last desired angles whether we want to load or unload, and then the corresponding pressure command is interpolated from the loading or unloading curve.

Fig. 6.

Pressure-angle mapping for the catheter tips. The pressure in the given channel is increased to maximum and back to zero ten successive times. The last nine loads are averaged and the angle standard deviation at each pressure set point is shown as error bars a Mapping for channel 1 of catheter tip 1. b Mapping for channel 2 of catheter tip 1 c Mapping for channel 1 of catheter tip 2 d Mapping for channel 2 of catheter tip 2

Experimental setup

To evaluate the performance of the catheter tip and control system, we investigate how accurately we can position the catheter tip by simultaneously inserting the catheter tip manually and turning the controller dial to adjust the angle. This test allows us to determine whether our robotic system will enable the surgeon to efficiently and accurately select branch vessels and areas of interest during surgical navigation. The procedure is shown in Fig. 7. For this task, a block with five 2mm diameter cylindrical targets, with centers equally spaced 2.55 mm apart was placed underneath the catheter tip and pneumatic tubes. The experimenter moves the catheter tip from the starting point forward while turning the dial of the hand controller until a mark on the distal end catheter tip is touching a given target, then presses a button on the PC to signal completion of that target. Then, they straighten the catheter tip and retract it back to the starting position, ready to approach the next target. Once all targets were reached, a post-processing script measures the distance between center of the distal end of the catheter tip and the closest point on the target. Four sequences of five random targets were generated. Each of these sequences is performed twice by a user with no surgical training, but familiar with surgical robotic systems, while in position control mode, then twice in velocity control mode, totaling 80 trials (where one trial is an attempt to reach a single target). Subsequently, the same sequences were performed once each by a skilled surgeon using position and velocity control mode, totaling 40 trials. Both users were never completed the task before, but the novice user had used the system for approximately one hour before performing the task and the skilled user was given ten warm-up trials.

Fig. 7.

Experimental Procedure. a The catheter tip begins at the starting position, aligned with the ends of the side-walls b The user moves the catheter tip forward while adjusting the angle to reach the target. The experimenter presses a button to signal they have reached the target, and the positioning error is calculated and recorded. c Once a target is reached, the experimenter moves the catheter tip back to the starting position in preparation to reach the next target

Experimental results

For the novice user, the average (± standard deviation) position error was 0.8±0.3 mm (8.4±3.1% of workspace) for the 40 position control mode trials, and 0.7±0.3 mm (7.4±4.2% of workspace) for the 40 velocity control mode trials. For the skilled user, the average position error was 0.7±0.4 mm (7.4±4.2% of workspace) for the 20 position trials, and 0.8±0.4 mm (8.4±4.2% of workspace) for the 20 velocity trials. This workspace was obtained by calculating the distance traveled by the tip during a calibration trial, and for this tip was 9.5 mm. All errors are well within the limit of 2mm obtained from vessel and aneurysm neck diameters in the brain [5–7]. The resolution of distance measurements is 0.045 mm, as this is the distance across one pixel as determined by the resolution and focal distance of the optical setup. The uncertainty in the distance calculations is approximately ±0.14 mm, as derived from the camera noise of around ±3 pixels. The error appears to have a dependence on the target and is plotted in Fig. 8a and c. For the novice user, the error at targets one and five is slightly higher than for targets two through four. This is somewhat expected, as these targets require actuating to a greater angle, and so require more catheter tip motion. However, we do not see the same trend for the skilled user. Rather, we observe higher errors for velocity control rather than position control. In addition to the average error for each target, we observed whether the error changes over time (as the user gains more experience). This result, shown in Fig. 8b and d, highlights that the error remains relatively constant (close to the average) for both users and modes. It should be noted that the error is already low to begin with, on the order of one tenth of the width of the catheter, so there is not much room to improve.

Fig. 8.

Positioning error during the experimental task. a Novice user: average error for each target number during the position and velocity control tasks. Target numbers are ordered from left to right, 1 being the leftmost and 5 the rightmost. b Novice user: error during each trial, during position and velocity control modes c Skilled user: average error versus target number. d Skilled user: error during each trial

In addition to the error, the time taken to reach each position is recorded as a supplemental metric to compare between modes, users, and targets. The absolute value of time taken to reach each target primarily shows that accurate positioning was achieved in just five seconds for a skilled surgeon and less than ten seconds for a novice. Figure 9a shows the average time versus target number. This time is measured as the time between when the user crosses the start position and when they stop moving at the target. For the novice user, we observe some learning effect. This effect is most pronounced in the position mode trials (see Fig. 9b), which makes sense given that this control mode was performed before the velocity control trials. Another potential influence on the time taken to reach certain targets is the process of learning the task. As trials progress, it is expected that the user reaches the target in the same or less time as the previous trials, provided the error also stays the same or decreases. Interestingly, the learning effect is not evident in the time data for the skilled user—this is potentially because this user has much more experience with dexterous manipulation tasks. As shown in Fig. 9b, the time to reach the target for the novice in both control modes appears to level off after the first twelve trials. Therefore, when computing the average times for the novice the first twelve trials are omitted. The resulting averages for the novice user are 7.8±3.4 s in position control mode and 7.7±2.6 s in velocity control mode. The averages for the skilled user are 3.1±0.7 s in position control mode and 3.9±1.3 s in velocity control mode.

Fig. 9.

Time taken to reach the targets. a Novice user: Average time to reach each target in position and velocity control modes. Target numbers are ordered from left to right, 1 being the leftmost and 5 the rightmost b Novice user: Time taken to reach the target in each trial, during position and velocity control modes c Skilled user: average time versus target d Skilled user: time over trials

For the velocity control mode for the novice user in Fig. 9b, we see a similar general trend as in the average error results, in that the middle target is the quickest to reach. The difference between the two outside targets on either side, however, is not very noticeable. This may be due to the fact the smaller angles are not necessarily quicker to reach than the larger ones, as most of the time during the task is taken when the catheter is close to the target, and the user is fine-tuning the position. For target three, the user often will not have to actuate the catheter at all since it is straight ahead of the starting position, thus decreasing the time it takes to reach. Interestingly, the position control mode result takes noticeably longer for targets one and two as compared to symmetric targets four and five, although those sets of targets would be expected to take roughly the same time to reach. A potential explanation is that although the targets are symmetric, the user does not perform the same exact movements to reach each side-rotating the hand clockwise and counter-clockwise exercises different muscles. This also would explain why the same asymmetry is not present in the velocity control results for the novice user-with velocity control, the user does not need to significantly rotate or readjust their hand to reach high angles, they are able to rotate as much as is comfortable and wait as the tip adjusts. The skilled user does not exhibit this same effect, potentially due to their improved dexterous manipulation skills.

Discussion

The proposed soft robotic system allows its user to reach targets with millimeter-level accuracy, which is around one tenth of the catheter tip’s workspace. The preliminary data suggests that the interface is intuitive and easy to learn, as the novice user was able to reach targets in roughly eight seconds on average after the initial learning curve, and the skilled user was able to reach targets in around three to four seconds. The skilled user’s data does not reveal the same trends as the novice users, such as asymmetry and learning effect. While the advanced dexterous manipulation skills of the skilled user are a potential reason, data from more users would help clarify this. The similarity in results between position control mode and velocity control mode suggest that both are viable options and may depend on user preference and prior experience (as the skilled is slightly faster in position control). The calibration results indicate the magnitude of variance that is possible between two prototypes, although they have the same design and material, and were manufactured by the same process. Actuator geometry and material properties are sensitive to slight differences in the post-processing steps such as removing uncured resin and the time spent under UV light. These differences between prototypes can significantly alter the flow dynamics. Although these dynamics are not extensively explored in this work, we expect that this variance can be reduced in future iterations. This highlights the need for device-specific calibration, although the calibration and control method can remain the same. In addition, recent advances in direct laser writing (DLW) printers enable printing large batches of micron-resolution parts, due to increases in scan speeds, voxel tuning algorithms, and larger print fields. Further, mass producing catheters is not necessarily the only option, since 3D printing patient-specific catheters could help overcome the challenges posed by certain unique anatomies.

The actuators used in this study are ten millimeters wide and thus not suitable for many surgical interventions, as standard catheters for the heart and brain are often less than a few millimeters in diameter. Despite these limitations, the user is able to position the catheter tip with sub-millimeter accuracy, less than the typical diameter of the smallest major artery and aneurysm neck in the brain [5, 6]. It should be noted that the size and spacing of targets is not intended to represent a real clinical requirement, but rather to place sufficient visible targets within the tip’s workspace. Beyond positioning accuracy, further design optimization and construction of a full-length catheter would allow additional clinically relevant experiments tasking users to traverse through an anatomical model, as is done for the tools developed in [3, 19, 27]. These investigations would provide useful insight on the environment interaction forces the catheter experiences, which are difficult to estimate at our current scale given that tube stiffness is expected to approximately scale with the fourth power of its radius. However, since several smaller prototypes have similar pressure operating ranges to ours [18–20] and we likely can adjust the visualization to “zoom in” on the tip, we can reasonably expect that the placement accuracy observed here can be maintained on smaller scales. In terms of this architecture, the soft robot is difficult to control in an open-loop configuration due to the effects of hysteresis. The dynamic response of the robot to pressure inputs as well as external forces during real surgery will further complicate the control. There have been several attempts to develop control algorithms to overcome these challenges [41] such as integrating image-guidance to account for the un-modeled forces [42, 43]. Finally, the soft robots have only been controlled with single-channel actuation, rather than allowing both to be actuated at the same time. Actuating channels simultaneously allows the robot to extend as well as bend, increasing its workspace.

Conclusion

A novel robotic system that integrates a soft pneumatic robot and control interface has been designed and evaluated. Initial results show the potential to position the robot with millimeter-level accuracy using an intuitive controller that allows the user to insert and steer the robot simultaneously. In future works, our “ex situ DLW (esDLW)” strategies for printing microfluidic technologies directly atop mesoscale fluidic tubing [37, 44] in combination with our recent developments for DLW-based soft micro-robotic actuators [38, 45] could be leveraged to scale down the system reported here to sub-millimeter scales. In combination, these results serve as a important first step toward future investigations into the most intuitive and effective control methods of soft pneumatic surgical devices.