Design of Patient-Specific Concentric Tube Robots Using Path Planning from 3-D Ultrasound

Abstract

Percutaneous techniques and robot-assisted surgical systems have enabled minimally invasive procedures that offer reduced scarring, recovery time, and complications compared to traditional open surgeries. Despite these improvements, access to diseased sites using the standard, straight needle-based percutaneous techniques is still limited for certain procedures due to intervening tissues. These limitations can be further exacerbated in specific patient groups, particularly pediatric patients, whose anatomy does not fit the traditional tools and systems. We therefore propose a patient-specific paradigm to design and fabricate dexterous, robotic tools based on the patient’s preoperative images. In this paper, we present the main steps of our proposed paradigm – image-based path planning, robot design, and fabrication – along with an example case that focuses on a class of dexterous, snake-like tools called concentric tube robots. We demonstrate planning a safe path using a patient’s preoperative ultrasound images. We then determine the concentric tube robot parameters needed to achieve this path, and finally, we use 3-D printing to fabricate the patient-specific robot.

I. Introduction

Minimally invasive surgical procedures are performed whenever possible due to the reduction in complications and co-morbidity. However, when the surgical target is at a difficult anatomical site, minimally invasive procedures with straight instruments may not be safe. The risks of serious complications may be even higher in pediatric patients, whose anatomy is more compact, and in other specialized patient groups whose anatomy may prove current generalized systems inadequate. To address these limitations, we propose a patient-specific design paradigm. The primary contribution of this work is to demonstrate this entire personalized design process including preoperative 3-D ultrasound imaging, segmentation of surrounding organs, safe path planning, design of a flexible robot, and finally robot fabrication (Figure 1).

Fig. 1.

Patient-specific design workflow includes imaging, segmentation, path planning, concentric tube robot design, and fabrication.

A. Medical Application

To ground this work in a particular clinical application, we focus here on accessing kidney stones in pediatric patients through a percutaneous subcostal approach. In a large international multi-institution study of 5,803 patients undergoing percutaneous nephrolithotomy (minimally-invasive procedure to remove stones by puncturing the skin), a complication rate of 21.5% was reported [1]. In a review by Stephan et al., the complication rate was up to 83% [2]. With better planning and dexterity, the number of such injuries and complications could be reduced. Our goal is to avoid damaging nearby tissue by using a subcostal approach that enters below the 12th rib and snakes through the renal pelvis towards the target. Here we focus on the case of an 8-year-old female patient with a 3mm renal calculus located in the interpolar region of the right kidney. Although focused here on pediatric stone removal, the aim is to demonstrate the development of a paradigm that can be applied to other procedures and patient groups in the future.

B. Concentric Tube Robots

Concentric tube robots are snake-like, dexterous robots, consisting of hollow, precurved tubes that nest concentrically, one inside the next [3], [4]. Each tube can be individually rotated and inserted with respect to the others, and the interaction between overlapping tubes of different curvatures enables the robot’s overall shape to change. There are several parameters, including diameters, curvatures, and lengths, that can be adjusted to change the dexterity and accessible workspace, making concentric tube robots a natural choice for personalization. Concentric tube robots are also small and dexterous, since their bending does not rely on external forces, but rather, is inherent in the elastic interaction of the tubes. There are numerous possible sequences of insertions and rotations to deploy a concentric tube robot to a given configuration. In this work, a plan is created for a “follow-the-leader” deployment sequence [5], where the backbone exactly follows the tip. The surgeon can teleoperate the robot with haptic feedback that constrains movement to the desired path, or the system can allow the surgeon to implement other deployment sequences.

II. Image Segmentation and Path Planning

A. Imaging

Abdominal ultrasound is a safe and simple imaging exam that allows diagnosing and monitoring pediatric conditions. For path planning, we acquired three-dimensional (3-D) B-mode ultrasound data to facilitate the visualization and quantification of the kidney and its surrounding region, including a partial view of the liver (for the right kidney). Images were acquired with a Philips iU22 system (Philips Healthcare) with an X6–1 xMATRIX array transducer. The volume size was 496 × 410 × 219 voxels, with a voxel resolution of 0:27 × 0:21 × 0:38 mm.

B. Segmentation

Based on the preoperative 3-D ultrasound scans of the patient, we identified and delineated the renal structures, including the renal parenchyma (S_K) and collecting system (S_CS ), and the surrounding organs at risk using semi-automatic segmentation. This facilitated the visualization and study of the patient-specific anatomy, and the design of a safe path for the intervention.

For the segmentation of the kidney and collecting system, we used the framework recently presented in [6]. The kidney was first segmented using an active shape model [7] variant, tailored to deal with the high intensity variability and inhomogeneity of ultrasound images. The anatomical coherence of the model is guaranteed via the statistical shape model of the kidney built from 39 pediatric cases. To compensate for the image dependency on the propagation direction of the ultrasound wavefront, the model incorporates an orientation-based landmark weighting model that gives high confidence to the edges normal to the wavefront, and low confidence to features tangential to the sound wave. The model is completed with a new ultrasound appearance descriptor based on multi-scale omnidirectional Gabor filters. ${\widehat{S}}_K$ represents this initial segmentation of the kidney.

Segmentation of the renal calyces and pelvis are critical to design a minimally invasive access path, since factors including the angle of the access path to the renal calix directly affects how invasive the procedure will be. In particular, we used the active contour-based formulation introduced in [6], which includes the renal fat as a patient-specific anatomical constraint to control the contour evolution process, mimicking the propagation of fluid inside the kidney previously segmented. ${\widehat{S}}_{CS}$ represents the resulting collecting system segmentation.

Once the renal structures were segmented, a first estimation of the surrounding organs at risk were obtained using a simplified version of the multi-organ statistical shape model described in [8]. For this case study involving the right kidney, the right lung (S_RL) and liver (S_L) were included in the model. Given the multi-organ statistical shape model, and using ${\widehat{S}}_K$ as a predictor, ${\widehat{S}}_{RL}$ and ${\widehat{S}}_L$ were estimated via a weighted point distribution model [8], where the weights corresponding to S_RL and S_L were set to zero, while the weights of S_K were set to 1. Figure 2(a) shows a 2D view of the resulting ${\widehat{S}}_K$ and the estimated ${\widehat{S}}_L$.

Fig. 2.

(a) View of the 3-D ultrasound of the patient, showing the control points selected by the surgeon for lateral access (a₀ to a₄), the closest organ of risk (${\widehat{S}}_L$), and the straight (Γ_str) and curved path (Γ_opt) to access the stone (a₄). (b) Colormap representing the distance to critical surrounding organs. Note that the straight path would perforate the liver.

C. Path Planning

The estimated patient-specific anatomical map, (${\widehat{S}}_K$, ${\widehat{S}}_{CS}$, ${\widehat{S}}_{RL}$, ${\widehat{S}}_L$), along with the 3-D ultrasound volume (for manual correction if necessary) are presented to the specialist (i.e., surgeon or interventional radiologist). The specialist then manually selects the critical points of the curved path by mouse clicks, defining the insertion point on the skin surface (a₀), the target point (a_N ), as well as any additional control points if necessary (e.g. to control the angle of the instrument to access the calix). Let (a_j, j = 0, … , N ) represent the set of selected points. In this case, a total of 5 points were defined, as shown in Figure 2(a). The optimal path, Γ_opt, is defined as

$${\mathrm{\Gamma }}_{opt}=\underset{\mathrm{\Gamma }}{\text{argmax}}\left(\sum _{i=\left\{RL,L\right\}}d\left({\mathrm{\Gamma }}_{a_0,a_N},{\widehat{S}}_i\right)|d\left({\mathrm{\Gamma }}_{a_0,a_N},a_j\right)=0\right),$$ (1)

where ${\mathrm{\Gamma }}_{a_0,a_N}$ is a curved path with C¹ continuity connecting the access (a₀) and the target (a_N) points, and $d\left({\mathrm{\Gamma }}_{a_0,a_N},{\widehat{S}}_i\right)$ and $d\left({\mathrm{\Gamma }}_{a_0,a_N},a_j\right)$ represent the shortest distance of this path to the segmented organs, ${\widehat{S}}_i$, and the control points, a_j, respectively. Equation 1 can be solved as a constrained spline problem [9] using the distance map defined by the set of segmented organs. Figure 2(b) shows the distance map of the resulting path to the surrounding organs.

III. Concentric Tube Robot Design and Fabrication

The next step in the patient-specific design process is to design and fabricate a set of concentric tubes that can most closely achieve the desired curved path.

A. Tube Design

1). Curve Fitting:

There are several parameters to select for a given concentric tube robot design. First, the number of tubes (n) must be selected based on the curved path. Here we have chosen n = 3. The next step is to fit regions of constant curvature to the given path. A circle with center ($\overrightarrow{c}$) and radius (r) is fit to each segment using two points and a tangent vector as in [10]. We fit one straight region, from ${\overrightarrow{p}}_0$ to ${\overrightarrow{p}}_1$, and three regions of constant curvature, constrained to be tangent to the previous segment (Figure 3(a)).

Fig. 3.

(a) Optimal path (gray) based on image segmentation. Starting and ending points (p₀ to p₄) of the piece-wise segments are shown in red, along with the curve fits for each segment. (b) Concentric tube robot configuration based on the tube parameters backed out from the curve fits.

In contrast to past work [10], our optimal path does not have defined via points (${\overrightarrow{p}}_0\text{ to }{\overrightarrow{p}}_{n+1}$) through which the curves should pass. Instead, we iterate through possible combinations of starting and ending points for the segments. To narrow down the range of combinations, we set ${\overrightarrow{p}}_0$ as the robot base, ${\overrightarrow{p}}_1$ as the end of the straight region as fit by eye, and ${\overrightarrow{p}}_4$ as the robot tip. We then iterated through various ${\overrightarrow{p}}_2$ and ${\overrightarrow{p}}_3$ positions, calculating the fits of the resulting segments. The mean square error of the fits for each segment were calculated and weighted by the number of points in the segment. The curvature fits for curves 0, 1, and 2 were 0.0632, 0.0676, and 0.0373 mm⁻¹, respectively (Figure 3(a)).

2). Parameter Selection:

This piece-wise constant curvature path can then be used to determine the curvatures and lengths of each of the concentric tubes. To start, we assume that the final curved section, between points ${\overrightarrow{p}}_3$ and ${\overrightarrow{p}}_4$, represents a section of the robot that consists solely of the innermost tube. Therefore the curvature of the innermost tube (κ₀) is equal to the inverse of the radius of curvature fit to this segment. Working back towards the base of the concentric tube robot, each curved segment is assumed to consist of i+1 overlapping tubes, where i is the tube number, starting with i = 0 for the innermost tube. The curvature of such segments lies somewhere in between that of the original precurvatures, and this new equlibrium curvature (k_eq) can be described for the planar case as [3], [11],

$$k_{eq}=\frac{\sum _{i=0}^{m-1}{E}_i{I}_i{k}_i}{\sum _{i=0}^{m-1}{E}_i{I}_i},$$ (2)

where I_i and E_i are the cross-sectional moment of inertia of tube i and Young’s modulus of tube i, respectively, and m is the number of overlapping tubes. Equation 2 is used to solve for κ_i of the remaining curved segments. Finally, the curved length of each tube (L_ci) is set equal to the length of the corresponding segment of the piece-wise curved path.

The final tube parameters were selected based on the particular patient and procedure. The inner diameter of the smallest tube (ID₀) has a lower bound based on the diameter of the flexible tool that must be passed through the concentric tube robot, and the outer diameter of the outermost tube (OD₂) has an upper bound based on the size of the patient. The remaining diameters are determined based on the wall thickness required by the fabrication method, as well as the gap between tubes needed for smooth relative motion. The last parameter, the straight length of each tube (L_si), is determined based on the dimensions of the robot driving the concentric tubes. The final designed parameters are shown in Table I.

TABLE I.

Tube Parameters

Tube #	OD [mm]	ID [mm]	Lc [mm]	Ls [mm]	κ [mm−1]
0	2.4	1.0	14.9	200.1	0.0632
1	3.8	3.0	21.6	98.6	0.0847
2	5.4	4.4	8.8	39.7	0.0499

B. Tube Fabrication

1). Material and Method:

The designed tube parameters are then used to generate a CAD model, which is subsequently 3-D printed. In contrast to traditional methods patient-specific design methodology [12], [13]. A MakerBot Replicator 2X was used to print polycaprolactone (PCL) (from MakerBot Industries). PCL is a biodegradable polyester, often used for sutures [14]. Once printed, we can insert a flexible inner wire with a sharp tip through the robot to help cut through tissue, and the the entire concentric tube robot can be attached to the driving robot, as shown in Figure 4, in order to assess the final configuration achieved. The robot we have developed to drive the concentric tubes is compact, modular, and can independently control both insertion and rotation of each of the three tubes [15].

Fig. 4.

3-D printed concentric tube robot attached to the 6-d.o.f. driving robot that controls both insertion and rotation of each tube individually.

2). Results:

The individual tubes are shown in Figure 5(a). Due to printer resolution and tolerances, there were slight variations between the designed and measured dimensions (Table II). Calipers were used to measure the diameters of the tubes at several points, and minimum and maximum percent errors are given based on these. Curvature measurements were made by comparing the tubes against lines of known constant curvature.

Fig. 5.

(a) Set of tubes 3-D printed with polycaprolactone (PCL). (b) Overhead view of 3-D printed concentric tube robot in desired configuration with curve fits overlaid on each of the curved segments.

TABLE II.

Tube Parameter Comparison: Showing mean percent error (and minimum and maximum percent error where relevant) between the designed and measured tube parameters.

	mean % error (min % error to max % error)
Tube #	OD	ID	κ
0	0.4% (−2.9 to 3.8)	2.0% (−4.0 to 8.0)	0.4%
1	−2.0% (−4.2 to 0.3)	−0.8% (−5.0 to 3.3)	−1.6%
2	0.4% (−0.2 to 0.9)	−0.6% (−2.3 to 1.1)	−1.0%

To assess the overall shape of the concentric tube robot, the tubes were arranged concentrically, with the desired relative insertion and rotation. Once arranged in the desired configuration, that would be reached via follow-the-leader deployment, an overhead image was taken in order to analyze the curvatures. As shown in Figure 5(b), the image was converted into a binary image, where the concentric tube robot is shown in white. A centerline through the tubes was then found and segmented by eye into three regions. Constant curvature segments were then individually fit to the centerline in each region, and the resulting fits are shown in Figure 5(b). The radii of curvatures were converted from pixels to millimeters based on a reference ruler (not shown in the image here). The curvatures were found to be 0.078, 0.0571, and 0.0296 mm⁻¹ for curves 0, 1, and 2, respectively. These correspond to deltas of −0.0148, 0.0105, and 0.0078 mm⁻¹ from the desired curvatures. Slight deviations from the exact path are acceptable as long as the concentric tube robot remains within the safe workspace and does not puncture critical organs. We leave further evaluation to future work.

IV. Conclusion

This paper proposes the development of a patient-specific design paradigm and demonstrates the process for planning, designing, and fabricating tools to access a pediatric kidney stone. Results demonstrate the feasibility in 3-D printing a set of concentric tubes based on a desired path planned using a patient’s preoperative ultrasound images. Future work includes a complete demonstration of using these patient-specific tools in a phantom patient model. In addition, we plan to integrate information from the surgeon’s planned path into the control scheme for driving the concentric tubes during a procedure. Finally, this work could be extended to other patient groups and procedures.