Surgeon Design Interface for Patient-Specific Concentric Tube Robots

Abstract

Concentric tube robots have potential for use in a wide variety of surgical procedures due to their small size, dexterity, and ability to move in highly curved paths. Unlike most existing clinical robots, the design of these robots can be developed and manufactured on a patient- and procedure-specific basis. The design of concentric tube robots typically requires significant computation and optimization, and it remains unclear how the surgeon should be involved. We propose to use a virtual reality-based design environment for surgeons to easily and intuitively visualize and design a set of concentric tube robots for a specific patient and procedure. In this paper, we describe a novel patient-specific design process in the context of the virtual reality interface. We also show a resulting concentric tube robot design, created by a pediatric urologist to access a kidney stone in a pediatric patient.

I. INTRODUCTION

Unlike traditional robot-assisted minimally invasive surgical systems, which can be limited by their need to accommodate a wide range of procedures and patients, concentric tube robots can be designed on a patient- and procedure-specific basis. One potential advantage of personalized robot design is that it allows for the optimization and fine-tuning of medical procedures. This optimization is especially important for specialized patient groups, including pediatric and obese patients, since it may prove too difficult to properly operate standard systems on their distinct anatomy. Another potential advantage is that the surgeon can be more involved in not just the planning of the procedure, but also in the design of the physical instruments that he or she will use.

A. Background

1) Concentric tube robots

Concentric tube robots are highly customizable and have therefore been proposed for patient- and procedure-specific fabrication. They consist of hollow, precurved elastic tubes, each nested concentrically inside the next [1], [2]. As the tubes are rotated and inserted with respect to each other, the interaction between overlapping tubes enables the entire robot to change shape through either free space or tissues. Their small size and ability to move in highly curved paths allow access to hard-to-reach places within the body. These robots have previously been fabricated out of Nitinol, which can sustain recoverable strains of around 8–11% [3], using a carefully refined heat treatment process. However, researchers have recently proposed other fabrication methods, such as 3D printing, which expands the range of materials to include various thermoplastics and increases the design space [4], [5]. The overall design of concentric tube robots is complicated not only because of the large number of tunable parameters, but also because of the complex kinematics. Rotating or inserting a tube can affect the shape of the entire robot, making it difficult to describe as a set of kinematically independent links without adding specific design constraints.

2) Optimization algorithms for concentric tube robot design

Research on the design of concentric tube robots has focused on optimization algorithms mainly based on anatomical constraints and desired tasks. Bedell et al. presented an optimization framework for an algorithm that given the constraining anatomy, tip workspace, and number of fixed and variable curvature sections, solves for a design that meets anatomical constraints, possesses the desired workspace, and minimizes the curvatures and lengths of all sections [6]. Burgner et al. proposed an algorithm that seeks to maximize the coverage of the required surgical workspace [7], and Torres et al. proposed a task-oriented design method that incorporates motion planning [8].

Although each of these algorithms differs in their exact inputs and objective functions, they all use little to no surgeon input to the design. In some algorithms, a surgeon or an experienced researcher may help define an initial design for the algorithm. For example, Anor et al. created a MATLAB graphical user interface that allows the user to load a geometric model of a surface and select start points, end points, directions, and constraints for the initial configuration [9]. Once the starting configuration has been specified, the rest is left to the optimization algorithm. Other algorithms do not include a user at all, but rather, rely on a randomly generated design to start [8]. Because of the surgeon’s vast knowledge of the patient anatomy and procedure, we propose to enable him or her to give more input into the design. In addition, there may be nuances that are too difficult to detect and account for by using any single optimization algorithm. These currently unaddressed needs motivate the development of our design method, which leverages the expertise of the surgeon in order to design patient- and procedure-specific concentric tube robots.

3) Virtual reality and its uses in surgery and interventional radiology

The use of virtual reality in a medical context is rapidly growing, especially in the areas of medical education and surgical simulation [10]. In medical education and training, researchers have used virtual reality as a means to explore the interrelationship of anatomical structures, as well as a means to train and provide objective assessment to new doctors [11]. For surgeries, one of the main applications of virtual reality has been simulators that allow surgeons to rehearse complex procedures before actually performing them. In addition, virtual reality has been used for preoperative planning, which can help with visualizing potential outcomes, and has been used to design custom implants [12]. Despite this growing number of applications, we believe that an unaddressed, large impact usage of virtual reality in a medical context is the design of surgical instruments.

B. Contributions

Our contributions are as follows: (1) We present an intuitive interface for designing a set of concentric tube robots for a specific patient and procedure. The user sees a 3D model of the anatomy of interest and can initialize a design based on a number of via points. The interface also enables the user to alter individual tube parameters and see how the changes affect the final configuration of the concentric tube robot. Although previous groups have presented design optimizations, this interface leverages the expertise of the surgeon and puts him or her in the design loop. In addition, this work is the first we are aware of that proposes using virtual reality for the design of surgical instruments. (2) We present an example of a surgeon-designed concentric tube robot. The surgeon was given the task of designing a concentric tube robot to reach an upper pole kidney stone. After an introduction to the various features and interaction modes of the interface, the surgeon was able to design two different robots in less than six minutes and three minutes, respectively. This demonstration shows the feasibility of teaching a surgeon to use such an interface to design patient- and procedure-specific concentric tube robots.

II. USER DESIGN INTERFACE: SETUP AND INPUTS

The user interface consists of an initialization step, followed by an iterative design process with three main modes shown in Figure 2. The final output of the interface is the parameters of a patient-specific concentric tube robot design, which can then be manufactured. In order to begin using the interface, the physical system must be setup, and the two main system inputs, the 3D reconstructed patient anatomy and select surgeon defined tube parameters, must be created and defined.

Fig. 2.

User interface workflow, starting from inputs of the reconstructed patient anatomy and select surgeon defined tube parameters. The design interface can then be used to initialize and iterate through various designs before the parameters of a final patient-specific concentric tube robot design are output.

A. Physical setup of design interface

The design interface is built using Chai3D (www.chai3d.org), an open source framework for haptics, visualization, and interactive real-time simulation. Chai3d also allows for the integration of the three main components of the design interface: an Oculus Rift Development Kit 2 (DK2; Oculus VR), a PHANTOM Omni (Geomagic, Inc.), and a set of da Vinci surgical system foot pedals (Intuitive Surgical, Inc.).

The DK2 is a virtual reality head-mounted display that can be used to create an immersive environment for the user. A major benefit of using such a display is the ability to render the patient anatomy and concentric tube robot designs in 3D. Leveraging the head tracking capabilities, the surgeon can view the 3D model from different perspectives. In order to display this 3D environment, a separate stereo image is rendered for each eye.

The surgeon uses a PHANTOM Omni in order to interact with the virtual environment, which consists of a set of static, non deformable 3D models of anatomical structures. The Omni is a six degree-of-freedom haptic device that includes two buttons, which we will refer to as Button 1 and Button 2. The Omni is used to render haptic feedback to the surgeon, allowing him or her to feel the virtual environment. There are two main forms of haptic feedback used. The first is a reaction force felt when in contact with the surface of the virtual anatomical model and is based on the stiffness value, set here to be 30 Nm⁻¹. This stiffness is large enough for the surgeon to easily tell when the tool contacts the model, but small enough to prevent any instabilities. Similarly, the tip, base, and backbone of the concentric tube robot have stiffnesses of 80 Nm⁻¹, 80 Nm⁻¹, and 20 Nm⁻¹, respectively. The higher stiffnesses of the ends of the concentric tube robot enable the surgeon to more easily grasp and move them to accomplish tasks explained in detail in Section III. In addition to rendering surface stiffnesses, there are cases explained in Section III, where it is useful to render a force to pull the user to a desired location. Lastly, the Omni is also used as an input device for the system, enabling the surgeon to grab or drag objects and reorient the view. The position of the Omni is rendered in the virtual environment as a sphere, whose color can also be modulated depending on the interaction.

The final component of the design interface is the set of da Vinci foot pedals used to switch between program operation modes. An Arduino Uno is used to read the digital outputs from the foot pedals and serially write that data to the main C++ program. The program remains in ‘Design Mode’ (explained in detail in Section IIIB) unless one of the following mode-switching foot pedals is pressed: camera, design mode two, and simulation. Once in camera mode, the ‘+’ and ‘−’ pedals can be used to zoom in and out. And the final pedal, the ‘clutch’ pedal, is used when workspace limits have been reached and the end effector of the Omni needs to be moved without affecting the position of the tool in the virtual environment.

B. Development of patient-specific model

The first step in developing a patient-specific model is to identify the important organs in the area of interest. A 3D model of each of these is then reconstructed and placed inside the virtual environment as illustrated in Figure 3. To begin, CT scans are imported into an open source software called 3D Slicer (www.slicer.org), where each organ is segmented either through manual outlining and/or thresholding. The CT scans used here are deidentified scans from a pediatric patient with kidney stones. Once each of the CT scans has been segmented, a 3D model is reconstructed and exported as an STL file. This model is then imported into another open source software tool called MeshLab (meshlab.sourceforge.net), which is used to manually clean and simplify the model, mainly through vertex reduction. The final step is to use Solidworks to convert the model into an OBJ file, which can then be imported through the Chai3D interface into the virtual environment.

Fig. 3.

Pipeline from initial CT scans to final 3D model of a single organ/tissue that is ready to import into the virtual environment.

C. Preselected Parameters

The number of different parameters that can be tuned when designing a concentric tube robot creates a very large overall design space. In order to define a reasonable range of options, the surgeon must select several parameters prior to using the interface. As shown in Table I, parameters must be selected based on the specific patient and procedure and are entered in as inputs to the interface, as shown in Figure 2. Other parameters can be adjusted within the design interface, as shown in Table I and Figure 4. At the end of the design process, the surgeon can create another design by entering new initial inputs and repeating the process.

TABLE I.

Design Parameters

	Symbol	Description
Preselected	n	number of tubes
	ODi	outer diameter
	IDi	inner diameter
	Lsi	length of straight section
Adjustable	κi	curvature
	Lci	length of curved section
	Ei	Young’s modulus (based on material)
	αi	rotation actuator angle
	βi	final insertion actuator distance

Fig. 4.

Tube parameters that can be modified with the design interface. These parameters include individual tube curvature (κ_i), curved length (L_ci), Young’s modulus (E_i), rotation actuator angle (α_i), and the final insertion actuator distance (β_i). The variable s is the length along the backbone of the concentric tube.

The first design choice is the number of tubes (n) that will comprise the concentric tube robot. Based on the obstacles that must be avoided and the targets that must be reached, the surgeon should be able to approximate a curved path. The number of curves, and therefore the number of tubes needed, can then be approximated based on that path. Second, it is necessary to set the outer diameter of the outermost tube (OD_n−1). The size of the patient will put a limit on the maximum size of the overall set of concentric tubes, thus putting an upper limit on OD_n−1. Third, the size of the tool to be passed through the concentric tube robot will determine the minimum possible inner diameter of the innermost tube (ID₀). Using the limits on these two extremes, along with the material limitations on minimum wall thicknesses, the remaining OD_i and ID_i can be determined. In the future these parameters could also be tuned within the design interface; here we will leave them as initial decisions for the surgeon.

III. PATIENT-SPECIFIC CONCENTRIC TUBE ROBOT DESIGN PROCESS

Once the surgeon has determined the values of the preselected parameters (as described in Section II.C), he or she can manipulate the initial design in the virtual environment. We propose an iterative process consisting of four main “modes”: Camera Mode, Initialization Mode, Design Mode, and Playback Mode.

A. Explore the environment (Camera Mode)

Camera Mode allows the user to explore the patient anatomy in an intuitive way. Unlike 2D medical images, which can be difficult to interpret when designing a 3D device, this interface is built to enable the user to easily view the anatomy in the area of interest. It also allows the user to easily reorient the view based on the task and to visualize the concentric tube robot in the environment.

The user can enter and exit Camera Mode by pressing the “Camera” foot pedal. Upon entering this mode, a transparent sphere appears around the patient anatomy as shown in Figure 5(a). Using the Omni, the user can grab the sphere and drag it to any desired orientation as shown in Figure 5(b). The rotation is achieved by computing the vector from the center of the sphere to the contact point of the tool with the sphere, ${\overrightarrow{v}}_k$, where k is the iteration through the haptic loop since contact was made with the sphere. The normal vector, ${\overrightarrow{v}}_{\mathit{normal}}$, between ${\overrightarrow{v}}_k$ and ${\overrightarrow{v}}_{k-1}$ is then computed as

$${\overrightarrow{v}}_{\mathit{normal}}={\overrightarrow{v}}_{k-1}\times {\overrightarrow{v}}_k.$$ (1)

And the angle between these same two vectors is,

$$\theta ={\cos }^{-1}\left(\frac{{\overrightarrow{v}}_{k-1}\cdot {\overrightarrow{v}}_k}{\Vert {\overrightarrow{v}}_{k-1}\Vert \Vert {\overrightarrow{v}}_k\Vert }\right).$$ (2)

The objects in the environment are then rotated about ${\overrightarrow{v}}_{\mathit{normal}}$ by θ, as long as the tool remains in contact with the sphere.

Fig. 5.

In camera mode (a), a transparent sphere appears around the patient anatomy. The user can then interact with this sphere (b) by dragging it to any desired orientation. The user can also jump to various default views, including (c), (d), and (e) by pressing a button.

In addition, there are six “default” views, a few of which are shown in Figure 5(c), 5(d), and 5(e), that the user can switch between by pressing Button 1. These default views were selected in order to give the user a quick way of jumping to sagittal, coronal, and transverse views from two different directions. The user can also press the “+” and “−” foot pedals in order to zoom in and out.

B. Initialize design (Initialization Mode)

The first task in Design Mode is to select initial via points for the concentric tube robot as shown in Figure 6(a). The number of via points placed is equal to one more than the number of tubes (n) that the user specifies prior to running the interface software. The examples shown here use three tubes (n = 3). In addition to placing these via points, the user can orient the vector representing the straight portion of the concentric tube robot. The goal is to determine a set of concentric tubes that will pass as close to those via points as possible. To do this there are three main steps: determine approximate curvatures, back-calculate tube parameters, and compute the forward kinematics.

Fig. 6.

Screen shots of (a) initial points placed by the user and the initial tangent vector, as well as the initial concentric tube robot design (b) with and (c) without the initial points shown.

In order to determine approximate curvatures, a circle with center $\left(\overrightarrow{c}\right)$ and radius (r) is fit using two points and a tangent vector. We start with the point ${\overrightarrow{a}}_1$, closest to the base of the concentric tube robot, and we work our way towards ${\overrightarrow{a}}_{n+1}$, closest to the tip. We also start with a vector $\left({\overrightarrow{v}}_1\right)$ tangent to ${\overrightarrow{a}}_1$. We then solve the following system of equations:

$$\Vert {\overrightarrow{a}}_1-\overrightarrow{c}\Vert -r=0,$$ (3)

$$\Vert {\overrightarrow{a}}_2-\overrightarrow{c}\Vert -r=0,$$ (4)

$$\left[\left({\overrightarrow{a}}_1-\overrightarrow{c}\right)\times \left({\overrightarrow{a}}_2-\overrightarrow{c}\right)\right]\times \left[{\overrightarrow{v}}_1\times \left({\overrightarrow{a}}_1-\overrightarrow{c}\right)\right]=0,$$ (5)

$${\overrightarrow{v}}_1\cdot \left({\overrightarrow{a}}_1-\overrightarrow{c}\right)=0,$$ (6)

where Equations 3 and 4 specify that the distance from each of the points of interest $\left({\overrightarrow{a}}_1,{\overrightarrow{a}}_2\right)$ to the center of the circle $\left(\overrightarrow{c}\right)$ equals the radius (r) of the circle, Equation 5 ensures that the center of the circle $\left(\overrightarrow{c}\right)$ lies in the same plane as the specified points $\left({\overrightarrow{a}}_1,{\overrightarrow{a}}_2\right)$ and vector $\left({\overrightarrow{v}}_1\right)$, and Equation 6 makes the specified vector $\left({\overrightarrow{v}}_1\right)$ tangent to the circle. Solving the above equations gives the center and radius of a circle that passes through ${\overrightarrow{a}}_1$ and ${\overrightarrow{a}}_2$ and is tangent to ${\overrightarrow{v}}_1$. The vector tangent to this circle at ${\overrightarrow{a}}_2$ can then be calculated and used to fit a circle through ${\overrightarrow{a}}_2$ and the next via point, ${\overrightarrow{a}}_3$. This process can be repeated for each consecutive pair of via points, until we have a spline consisting of constant curvature sections as shown in Figure 7(a).

Fig. 7.

Plots showing (a) the circles fit to the initial points placed by the user as well as the initial tangent vector and (b) the curved portions of the concentric tube robot after the kinematics have been calculated, both before and after it has been reoriented to align with the initially placed points.

The next step is to use these circle parameters to back-calculate individual tube parameters. We do this by assuming that the final curved section, between points ${\overrightarrow{a}}_1$ and ${\overrightarrow{a}}_{n+1}\left({\overrightarrow{a}}_3\text{and}{\overrightarrow{a}}_4\text{here}\right)$, represents a section of a concentric tube robot that consists solely of the innermost tube. This assumption means that the radius previously calculated (r₃) gives us the radius of curvature of tube i = 0. Therefore, ${\kappa }_0=\frac{1}{r_3}$. Working our way towards the base of the concentric tube robot, we look at the curved section between points ${\overrightarrow{a}}_2$ and ${\overrightarrow{a}}_3$, that has a previously calculated radius r₂. We assume that this section consists of two overlapping tubes that when combined, result in an equilibrium curvature (κ_eq) that lies somewhere between the individual precurvatures. Therefore, ${\kappa }_{eq}=\frac{1}{r_2}$. We then use the following to relate this equilibrium curvature (κ_eq) to the individual precurvatures for the planar case,

$${\kappa }_{eq}=\frac{E_0{I}_0{\kappa }_0+E_1{I}_1{\kappa }_1}{E_0{I}_0+E_1{I}_1}\text{or}{\kappa }_{eq}=\frac{{\displaystyle \sum _{i=0}^{m-1}{E}_i{I}_i{\kappa }_i}}{{\displaystyle \sum _{i=0}^{m-1}{E}_i{I}_i}}$$ (7)

where I_i and E_i are the cross-sectional moment of inertia of tube i and Young’s modulus of tube i, respectively [1], [3], and m is the number of overlapping tubes. Equation 7 is rearranged and solved to find κ₁. This process can be repeated for each subsequent curved section until all tube precurvatures have been calculated.

Next, the lengths of each curved section (L_ci) must be computed. We estimate the initial curved length of each tube to be equal to the length of the corresponding curved segment. For the curved segment between ${\overrightarrow{a}}_i$ and ${\overrightarrow{a}}_{i+1}$ we evaluate the following,

$$L_{ci}=r_i{\cos }^{-1}\left(\frac{\left({\overrightarrow{a}}_i-{\overrightarrow{c}}_i\right)\cdot \left({\overrightarrow{a}}_{i+1}-{\overrightarrow{c}}_i\right)}{\Vert {\overrightarrow{a}}_i-{\overrightarrow{c}}_i\Vert \Vert {\overrightarrow{a}}_{i+1}-{\overrightarrow{c}}_i\Vert }\right).$$ (8)

Finally, we compute the actuator angles (α_i) and distances (β_i) of each tube. We first define the actuator angle of the outermost tube (α_n−1) to be zero. We create a right-handed coordinate system defined by the initial tangent vector $\left({\overrightarrow{v}}_1\right)$, the normal to this curve $\left({\overrightarrow{n}}_1\right)$, and a third basis vector we will call ${\overrightarrow{w}}_1$. The remaining actuator angles can then be calculated by projecting the vector normal to each subsequent curve onto the plane made by ${\overrightarrow{n}}_1$ and ${\overrightarrow{w}}_1$, and computing the angle between this projection and ${\overrightarrow{n}}_1$. The β_i values are set such that in the final configuration the entire curved length of each tube will be fully inserted.

Once all of the tube parameters have been estimated, the forward kinematics can be computed [13], [14]. Simply drawing the computed configuration of the concentric tube robot would result in an unaligned tube configuration similar to the green line shown in Figure 7(b). In order to align the concentric tube robot configuration with the initial via points, Procrustes analysis is used. We do this by creating two matrices X₁, $X_2\in {\mathrm{ℝ}}^{3\times n}$, that store n points, centered at ${\overrightarrow{a}}_1$, that correspond between the two 3D shapes that we want to align. In this case X₁ stores the points along the backbone of the concentric tube robot determined based on the kinematics, and X₂ stores the points along the initial spline of constant curvature sections that was computed based on the initial via points $\left({\overrightarrow{a}}_1\right)$ placed. We then expect that for each column, ${\overrightarrow{x}}_{1i}$ of X₁ and ${\overrightarrow{x}}_{2i}$ of X₂,

$$R{\overrightarrow{x}}_{1i}={\overrightarrow{x}}_{2i},$$ (9)

where R is an orthogonal matrix. Then by optimizing for R, we can solve the orthogonal Procrustes problem, which aims to minimize

$${\Vert R{X}_1-X_2\Vert }_{\text{Fro}}^2.$$ (10)

The solution to this orthogonal Procrustes problem is

$$R=V{U}^{\mathrm{T}}$$ (11)

where the SVD is used to decompose $X_1{X}_2^{\mathrm{T}}=U\sum V^{\mathrm{T}}$. We then use R to rotate the points in X₁ about the via point ${\overrightarrow{a}}_1$, giving us the blue line shown in Figure 7(b).

C. Alter tube parameters (Design Mode)

Once an initial concentric tube robot design has been generated and drawn, the user can alter the curvature, curved length, actuator angle, and material of each individual tube. Table II shows examples of how the user can interact with a concentric tube robot design to alter these parameters, along with images of the design before and after the interaction.

TABLE II.

Variable Tube Parameters

	κi	Lci	αi	material
Initial View
Interaction
Final View

To change the curvature of tube i, the user presses Button 1 to grab onto the white line projecting from the middle of tube i. The user can then drag the line perpendicular to the tangent of the curve at the midpoint. When the user releases the line, the new radius of curvature is calculated by fitting a circle based on three points: the first point of tube i that is visible beyond the tip of tube i + 1, the last point of tube i, and the end point of the white line that the user dragged. The radius of the fit circle becomes the new radius of curvature of that tube.

To change the length of the curved section of tube i, the user presses Button 2 to grab onto the white line projecting from the middle of tube i. When the line has been successfully grabbed, a force is applied to the Omni to pull the cursor to the tip of that tube. The user can then pull the cursor along the length of the concentric tube robot to either increase or decrease the length of that curved section. As the cursor moves, the colors of the concentric tube robot change to match the length change. While the user is dragging the cursor along the concentric tube robot, a force proportional to the distance they are away from the original tip of that tube is applied in the opposite direction. Once the user releases Button 2, the actual kinematics are computed based off of the new curved length, and the actual configuration is redrawn.

The actuator angle (α_i) and material of a tube can be changed when in the second design mode, which is entered using the foot pedals. The user can then press Button 2 to grab onto any part of tube i. Then by rotating only the final link of the Omni, the user can increase or decrease the amount by which they want to change α_i of that tube. Again, when the user releases the button, the kinematics are calculated and the new configuration is drawn. When in the second design mode, the user can instead come into contact with tube i and use Button 1 to click through the different material options. Current material options are Nitinol, PEBA2301 (a polyether block amide 3D printed using selective laser sintering), and Accura 25 (a polypropylene-like/ABS-like plastic 3D printed using stereolithography). When the user selects a different tube material, the relevant mechanical properties, Young’s Modulus (E_i) and Poisson’s ratio (ν_i) are updated, and the kinematics are recomputed.

The final parameters that can be changed are the position and orientation of the concentric tube robot with respect to the patient anatomy. The user can use Button 1 to grab onto the sphere located at the proximal end of the concentric tube robot. Once this proximal sphere has been grabbed, its position will be set to the position of the tool as the user moves it around. Setting the position requires a frame transformation from the world frame to that of the proximal sphere as follows,

$${ }^{\mathrm{P}}p={}^{\mathrm{P}}{T}^{\text{WW}}p$$ (12)

where ^Wp is the position of the tool in the world frame, ^PT ^W is the homogeneous transformation from the world frame to the frame of the proximal sphere, and ^Pp is the position of the tool in the frame of the proximal sphere.

There are two ways to change the orientation of the concentric tube robot in order to accommodate the following situations: keeping the proximal point constant while dragging the distal end to the desired location, and keeping both the proximal and distal locations constant while rotating the entire concentric tube robot about its axis. In the first case, the user grabs the sphere located at the distal end of the concentric tube robot. The proximal sphere, and thus the base of the concentric tube robot, remains in place as the user drags the distal sphere to the desired location of the tip of the robot. This reorientation is achieved by calculating the vector between the base of the robot and the tool position and comparing this to the same vector during the previous iteration through the loop. The concentric tube robot can then be rotated about the normal to these two vectors by an amount equal to the angle between them. The second way to reorient the concentric tube robot is to grab the proximal sphere and rotate the final link of the Omni in order to rotate the entire concentric tube robot about the axis going from the base to the tip.

D. Watch simulated procedure (Playback Mode)

Throughout the design process, it is helpful for the user to be able to visualize the designed concentric tube robot in motion. We currently limit the simulation to an approximate follow-the-leader deployment sequence [15], where the backbone follows the path through space traced by the tip.

This sequence is approximate since constant curvature tubes can only follow the leader exactly when the tube curvatures lie in the same plane, and here we do not limit the user to designing concentric tube robots that lie in a single plane. In this deployment sequence, the actuator angles are set and held at a constant α_i set by the user. All the tubes must start in a fully overlapping configuration, and all subsequent insertions of tube pairs must be done by inserting all overlapping tubes together, followed by inserting only one tube while holding the other fixed. Because the angular displacement between tubes is not equal to nπ (they do not lie in the same plane), there will be some deviation from perfect follow-the-leader behavior. Using this deployment sequence, we compute and store the configuration of the concentric tube robot at discrete insertion distances (β_i). We then draw these configurations one after the next such that the user perceives the motion as a continuous insertion.

Besides watching the entire simulation at once, the user can choose to pause the motion at any given point by pressing Button 2. This gives the user the opportunity to view the concentric tube robot configuration at different stages during deployment, making it easier to analyze its position relative to the anatomy at given points in time. The user can then restart the simulation from where it left off by pressing the same button.

IV. DEMONSTRATION OF SURGEON DESIGN INTERFACE

A. Surgeon Task

In order to demonstrate the use of the interface, we presented a pediatric urologist with the task of designing a concentric tube robot to access a kidney stone in a pediatric patient. Although access to the kidney in adults is relatively easy to accomplish using a straight needle punctured through the skin, this procedure is more difficult and risky in pediatric patients due to their smaller body surface area. Because of the high risk of damaging nearby tissue, we are interested in the design of a tool that can enter below the 12th rib, snake up through the renal pelvis, and curve towards the upper pole of the kidney [16].

As shown in Figure 8, a 3D model of the patient anatomy, including all relevant organs, was reconstructed and placed inside the virtual environment of the interface. Based on the anatomy, it was determined that the concentric tube robot should have three tubes. The inner diameter of the innermost tube has to be greater than 1.5 mm so that a flexible needle or probe can fit through the inside. And the outer diameter of the outermost tube has to be less than 10 mm at the very greatest, based on surgeon feedback for the maximum puncture size that can comfortably be made in a pediatric patient.

Fig. 8.

3D model of patient anatomy in the area of interest including the skin, ribs, left and right kidneys, renal pelvis, and the kidney stone.

B. Results

An explanation of the interface, its features, and how to perform the tasks in each mode was given to the surgeon prior to immersing him in the virtual environment. After this initial explanation, he put on the DK2 and took control with the PHANTOM Omni. He was then walked through the process of testing each feature in the various modes, in order to both familiarize him with what design parameters were available for him to change, as well as what interactions were actually needed to make these changes. He then spent around five minutes freely exploring the different modes and features until he felt comfortable enough to begin designing a set of concentric tubes to actually perform the previously described task.

He performed this design task twice – the first time spending just under six minutes and the second time spending just under three. During his second iteration, he started by placing four via points in order to obtain a gentle s-curve. Because he seemed comfortable with the general shape needed to achieve the given task, he began by placing these points outside the anatomy and moving the set of tubes into the desired position later. Once the initial configuration had been computed, he decided that the innermost tube did not have the desired curvature, and as seen in Figure 9(a), he began to change this curvature by grabbing the white flag. After the kinematics of the new tube design was computed, he was satisfied with the shape and began to move the position of the concentric tube robot towards the anatomy as seen in Figure 9(b). He then switched into Camera Mode to change the view and zoom in closer (Figure 9(c)) until he had the orientation needed to see the target location for the tip of the robot. At that point, he reoriented and repositioned the concentric tube robot (Figure 9(d)) until he was satisfied with the final design and relative orientation and position with respect to the anatomy (Figure 9(e)).

Fig. 9.

Snapshots from the surgeon design task including (a) changing the curvature of the innermost tube, (b) repositioning the concentric tube robot, (c) changing the view by rotating and zooming, (d) reorienting and repositioning the concentric tube robot, and (e) view of the design relative to the anatomy.

This first use scenario shows that the speed and ease with which the surgeon learned to use the interface demonstrates that it is intuitive. This is a key feature since a surgeon is more likely to adopt use of the interface if the learning curve is low and he or she can see immediate results. In addition, this demonstration shows the benefit of having the surgeon in the design loop, and we can see how he naturally uses his medical expertise during the design process.

V. CONCLUSIONS

This paper proposes a framework for putting the surgeon in the loop when designing a set of patient- and procedure-specific concentric tubes, and demonstrates a novel interface for this design process. Results demonstrate that a surgeon can use the interface to design a concentric tube robot to access a kidney stone in a pediatric patient.

There are many additional features that could be added to the design interface. A few examples include giving the user control over the insertion sequence, offering more material options, sending the user feedback if the robot is going to contact tissue, and allowing the user to add or subtract via points at any point during the design process. In addition, there are several tests that should be performed on the output designs before using them in a clinical setting. First, a quantitative assessment of the surgeon interaction with the interface, as well as of the surgeon’s design, should be developed. Next, designs output from the interface should be physically manufactured and tested in a geometrically accurate environment. Once these tests have been completed, this interface could play an integral role in the design of patient- and procedure-specific concentric tube robots. It could also be expanded to include the design of steerable needles or other flexible, customizable surgical tools.

Fig. 1.

Flow of patient-specific design process. Starting with CT scans of a specific patient, a 3D model can be created and placed into a virtual reality design interface. A surgeon can then iterate through various concentric tube robot designs, 3D print the final desired design, and use them for a particular procedure.