Sensorless Motion Planning for Medical Needle Insertion in Deformable Tissues

Abstract

Minimally invasive medical procedures such as biopsies, anesthesia drug injections, and brachytherapy cancer treatments require inserting a needle to a specific target inside soft tissues. This is difficult because needle insertion displaces and deforms the surrounding soft tissues causing the target to move during the procedure. To facilitate physician training and preoperative planning for these procedures, we develop a needle insertion motion planning system based on an interactive simulation of needle insertion in deformable tissues and numerical optimization to reduce placement error. We describe a 2-D physically based, dynamic simulation of needle insertion that uses a finite-element model of deformable soft tissues and models needle cutting and frictional forces along the needle shaft. The simulation offers guarantees on simulation stability for mesh modications and achieves interactive, real-time performance on a standard PC. Using texture mapping, the simulation provides visualization comparable to ultrasound images that the physician would see during the procedure. We use the simulation as a component of a sensorless planning algorithm that uses numerical optimization to compute needle insertion offsets that compensate for tissue deformations. We apply the method to radioactive seed implantation during permanent seed prostate brachytherapy to minimize seed placement error.

I. INTRODUCTION

Needle insertion is a key component of many minimally invasive medical procedures such as brachytherapy cancer treatment, tissue biopsies, and anesthesia drug injections. Accurately guiding the tip of a needle to a specific target inside soft tissue is crucial for the success of these procedures. However, significant errors are common in current practice. For brachytherapy prostate cancer treatment, an experienced physician implanting radioactive seeds in 20 patients using needles achieved average placement errors of 0.63cm, a substantial error of over 15% of average prostate diameter [1]. A key source of this error is tissue deformations. Inserting and retracting a needle into soft tissue exerts forces causing the surrounding soft tissue to displace and deform: ignoring these deformations can result in substantial placement error, as illustrated in Fig. 1 left column.

Fig. 1.

Four vertical frames illustrate brachytherapy needle insertion based on deforming ultrasound images of the human prostate using simulation. The left column shows results without planning, producing substantial placement error. The right column shows results with the sensorless plan, with minimal placement error. The target implant location is indicated in all frames with a cross fixed in the world frame. Frame (a) outlines the undeformed prostate. In Frame (b), the needle is inserted and the radioactive seed (small square) is released at the needle tip. In Frame (c), the needle is retracted. Frame (d) indicates the resulting placement error, the distance between the target and resulting actual seed location. Without planning, placement error is substantial: 26% of the prostate diameter, resulting in damage to healthy tissue and failure to kill cancerous cells. With sensorless planning, placement error is negligible.

We are developing a sensorless planning system that uses a physically-based, dynamic simulation of needle insertion and numeric optimization to reduce placement error. To facilitate physician training, the simulation can be run interactively using different patient anatomy and tissue properties. For patient-specific pre-operative planning, our system uses the simulation to compute a needle offset that compensates for tissue deformations to reach a given target location. In this paper, we describe our current 2D physically-based, dynamic simulation that uses a finite element model of soft tissue and a reduced set of scalar parameters such as tissue stiffness and compressibility and needle friction, sharpness, and velocity. As the needle is inserted into the tissue, the tissue mesh is updated to maintain element boundaries along the needle shaft, and the tissue is deformed due to cutting and frictional forces. The planner uses optimization to intelligently test different insertion locations and depths to compute the optimal needle offset: a sensorless motion plan as illustrated in Fig. 1 right column greatly reduced placement error in simulation.

We apply the planning system to permanent seed prostate brachytherapy, a minimally invasive medical procedure where physicians use needles to permanently implant inside the prostate radioactive seeds that irradiate surrounding tissue over several months, as shown in Fig. 2. The success of this procedure depends on the accurate placement of radioactive seeds within the prostate gland to ensure that a high dose is delivered to the cancer cells and a low dose is delivered to the surrounding healthy tissues [2], [3]. We define placement error as the Euclidean distance between the desired location for the seed specified by the dosimetric plan (the target) and the actual implanted seed location after needle retraction.

Fig. 2.

During permanent seed prostate brachytherapy (a), needles carrying radioactive seeds are inserted transperineally into the patient, who is lying on his back [3]. Intra-operative transrectal ultrasound can be used for imaging (b), but these images do not provide sufficient quality signal to track tissue deformations.

Tissue deformations during needle insertion and retraction contribute to seed placement error during brachytherapy [3], [1]. Although real-time ultrasound imaging is available during the procedure, it is used to verify the depth of the needle tip in the world frame and cannot effectively be used to track soft tissue deformations because the tissue is composed of large homogeneous regions and there are no crisp markers with known positions inside the soft tissues.

Our method simulates tissue deformations in a 2D imaging plane. Taschereau et al. found that placement error along insertion depth (z-axis) averages 4.7mm, error along insertion height (y-axis) averages 2.2mm, and error between planes (x-axis) averages 2mm for permanent seed brachytherapy [1]. Our 2D planning method uses a simulation that computes tissue deformations along the two axes with largest error, y and z.

We formulate the planning problem in Sec. III. We describe our needle insertion simulation in Sec. IV, which can be used interactively for physician training or offline for procedure planning. In Sec. V, we use the simulation as a component of the planning system that computes needle insertion offsets to compensate for the effect of tissue deformations. In Sec. VI, we apply the simulator and planner to minimize the placement error of radioactive seed implants for prostate brachytherapy. Throughout this paper, we use needle insertion terminology from brachytherapy for simplicity. However, out methods are generally applicable to many needle insertion procedures; at the release point, the physician could implant a radioactive seed for brachytherapy, inject a drug for anesthesia, or extract a tissue sample for a biopsy.

II. RELATED WORK

In robotics, sensorless planning algorithms, pioneered by Mason and Erdmann in the 1980’s [4], have been developed to position and orient mechanical parts using parallel jaws [5], [6], vibrating surfaces [7], single joint robots over conveyor belts [8], and squeeze and roll primitives [9]. For seed placement planning using rigid needles, our goal is to model and compensate for mechanical response before actions are performed.

To apply sensorless planning to needle insertion, we require a fast and accurate simulation of the procedure. Abolhassani et al. provide a survey of models and simulations of needle insertion [10]. DiMaio and Salcudean performed pioneering work in modeling the forces applied to tissue during needle insertion, and used the measured force distribution and 2D quasi-static FEM for simulation [11]. The force distribution, which was modeled with a parameterized surface, was determined for an artificial tissue phantom and may be difficult to measure in vivo. Alterovitz et al. introduced a simulation based on a reduced set of scalar parameters such as needle friction, sharpness, and velocity [12], [13]. These FEM simulations require that needle cutting and frictional forces be applied at nodes of the finite element mesh. DiMaio and Salcudean relied on node snapping, which moves the closest mesh node to the needle path in the world frame [11]. Nienhuys et al. proposed mesh refinement to mitigate the discretization error caused by node snapping [14]. Alterovitz et al. use mesh modification so no extra elements need to be created and the path cut by the needle is directly encoded within the reference mesh [12], [13].

Recent work has begun exploring 3D simulation of needle insertion which enables more accurate representation of anatomy and the simulation of needle insertion outside a single imaging plane. Nienhuys et al. used 3D mesh refinement [14] and Goksel et al. used mesh modification [15] in order to apply needle insertion forces at mesh nodes. However, current 3D simulations do not perform at interactive or real-time rates; Nienhuys et al. acknowledge the method cannot be used interactively and the method of Goksel et al. includes a mesh modification step that requires up to 0.5 seconds per frame for a mesh composed of 600 nodes. Also, the method of Goksel et al. does not offer guarantees on mesh stability throughout the simulation, and no previous 3D method provides visualization capabilities useful for physician training. The simulation method we present in this paper, an expanded version of our prior work [12], [13], [16], considers tissue deformations in 2D, providing graphical visualization via texture-mapping and offering guarantees on simulation speed and stability that are necessary for physician training and efficient automated planning. We will discuss 3D simulation extensions in future work.

Accurately setting needle/tissue parameters is important for realistic simulation. We use results from Krouskop et al., who estimated the elastic modulus for prostate and breast tissue using ultrasonic elastography [17]. Physical needle insertion experiments in animal tissues have been used to measure and model tip and frictional forces in canine prostate [18], forces in ex vivo bovine liver [19], and the effect of needle insertion velocity in porcine heart [20]. Recent work has also provided physical measurements and models of the bending of needles during insertion [21], an aspect of needle insertion we plan to integrate into 3D simulation and planning in future work.

Medical needle insertion procedures may benefit from the more precise control of needle position and velocity made possible through robotic surgical assistants [22]. Dedicated robotic needle hardware is being developed for different types of needle insertion procedures [23], [24], [25].

When real-time sensor data such as MR imaging is available during a needle insertion procedure and the target and relevant obstacles are all discernible in the images, robotic control algorithms can be used to guide the needle to the desired target. Shi et al. developed an image-guided system that uses real-time imaging to track the target and repeatedly updates the needle insertion direction to a straight line path to the target [26]. When real-time sensor data is unavailable or unreliable, sensorless planning based on pre-operatively predicting the effects of tissue deformations can be applied. Recent work has addressed planning local trajectories in deformable tissue for flexible needles with symmetric tips [27], [28] and bevel tips [29]. In this paper we explicitly use simulation of insertion of rigid needles into deformable tissues to plan needle procedures.

III. PROBLEM FORMULATION

We consider a 2D slice of tissue in the yz plane. At time t = 0, the tissue is at rest (undeformed). The target is denoted by a point g = (y_g, z_g) in the world frame at time t = 0.

A needle motion plan is defined by a control vector X. We define X = (y_r, z_r) where y_r is the “insertion height” and z_r is the “insertion depth.” A needle insertion procedure consists of inserting the needle at height y_r to a depth z_r, implanting a radioactive seed at this release point, retracting the needle, and waiting for steady-state. For simplicity, we assume that the needle moves parallel to the z-axis and that the coordinate system of the needle and the coordinate system of the tissue are identical. The location of the seed in the world frame after retraction is denoted by p = (y_p, z_p).

Due to the effects of tissue deformation, X ≠ p. We measure seed placement error using the Euclidean distance between the final seed location p and the target location g:

$$\epsilon =\Vert \mathbf{p}-\mathbf{g}\Vert .$$

The physically-based simulation is essentially a function whose input is a motion plan X and whose output is the final seed placement p. Hence, we specify the simulation as a function S(X) that returns the final seed placement:

$$\mathbf{p}=S(\mathbf{X}).$$

For a given target point g inside soft tissue, the motion planning problem is to compute a plan X that minimizes placement error. To compute an optimal motion plan X*, we use the simulation as a function in the optimization:

$${\mathbf{X}}^{*}=\underset{\mathbf{X}}{\text{arg min}(\epsilon )}=\underset{\mathbf{X}}{\text{arg min}}(\Vert S(\mathbf{X})-\mathbf{g}\Vert ).$$ (1)

During planning, we restrict the range of the parameters of the motion plan X to clinically feasible values. We restrict the y_r to the region of skin where the needle can be feasibly and safely inserted, y_r ∈ (y_min,y_max). We define z_max as the maximum medically feasible needle insertion depth.

IV. NEEDLE INSERTION SIMULATION

We simulate the insertion and retraction of a thin, rigid, symmetric tip needle in a 2D slice of soft tissue. We use a finite element method (FEM) to compute the deformations of soft tissues when forces are applied by the needle. Rather than calculating only static deformations, we simulate the dynamic behavior of soft tissues by solving for the acceleration, velocity, and displacement of each node for every time step to produce a history-dependent simulation.

As in related work [30], [11], we approximate soft tissues as linearly elastic, isotropic materials (Cauchy strain). Tissue may be inhomogeneous but must be fully connected with no gaps between different tissue types. We do not model slip between tissue types or physiological changes that result from needle insertion, such as edema (tissue swelling). As computation speed improves and biomechanics experiments provide more nonlinear tissue properties, we plan to extend the simulation to incorporate more complex tissue models.

A. Simulation Input

We represent the anatomy geometry using a finite element mesh. The input required for our geometric model includes a bitmap image of a 2D slice of tissue and a segmentation of the tissue types in the image using polygons. Based on the polygonal segmentation, we automatically generate a finite element reference mesh 𝒢 composed of n nodes and m triangular elements in a regular right triangle mesh or using the constrained Delaunay triangulation program Triangle [31]. Each node’s coordinate is stored in the node coordinate vector x. In 2D, each node has 2 degrees of freedom (DOF) so x is of dimension d = 2n.

To compute tissue deformations, the model must also include tissue material properties, boundary conditions for the finite element mesh, and needle properties. For each segmented tissue type, the model requires as input the tissue material properties (i.e. the Young’s modulus and Poisson ratio for linearly elastic materials). Each element in the mesh may be assigned unique material properties, which allows for the simulation of multiple tissue types in one mesh. We discuss a particular anatomy model, for the prostate, and estimation of simulation parameters in Sec. VI-A.

B. Simulation Output

The simulation computes mesh deformations that estimate the tissue’s response to the needle over time. The deformation is defined by a displacement vector u, which specifies the displacement of each node in mesh 𝒢. The deformed mesh 𝒢′ is constructed in the world frame using the displaced node coordinates x + u. The simulation computes the displacement u_i as a function of time step i. Using a fixed time step duration h, we obtain simulated deformations for times t = hi, i ≥ 0.

C. Computing Tissue Deformations

For a 2D mesh composed of 3-node triangular elements, the dynamic FEM problem is defined by a system of d = 2n linear equations:

$${\mathbf{\text{Ma}}}_i+{\mathbf{\text{Cv}}}_i+{\mathbf{\text{Ku}}}_i={\mathbf{f}}_i$$ (2)

where M is the mass matrix, C is the damping matrix, K is the stiffness matrix, f_i is the external force vector, a_i is the nodal acceleration vector, v_i is the nodal velocity vector, and u_i is the nodal displacement vector at time step i. These matrices are constructed using the finite element method [32]. The vector f_i represents the forces exerted by the needle on the tissue, as will be described in Sec. IV-D. The matrices M, C, and K are defined using the material properties of the elements in the mesh defining the deformable object, which include stiffness, compressibility, Rayleigh damping coefficients, and mass density [32]. Since they are constructed by superimposing the element mass, damping, and stiffness matrices, the number of non-zero entries in each of these matrices is O(d). When a node in the reference mesh is moved or constrained, these matrices must be updated, a process that takes constant time for each DOF.

To solve for u_i from its time derivatives v_i and a_i in the system of equations 2, we integrate over time for each time step i. We use the Newmark method [33], which translates the differential system into a linear system of equations. By taking advantage of the sparsity of the matrices, we solve the system in O(d²) time per time step [13]. With mass lumping [34], [35], we achieve interactive, real-time performance with O(d) time per time step [13].

D. Simulating Needle Procedures

Rather than modeling the needle as a distinct meshed object, we instead model the needle implicitly by applying needle insertion forces to the surrounding soft tissue. The needle exerts force on the tissue at the needle tip, where the needle is displacing and cutting the tissue, and along the shaft due to friction. We model these forces and apply them as the force vector f_i, which we update at every time step of the simulation. This implicit method for representing the needle facilitates real-time interactive performance since no expensive collision detection between the needle and soft tissue is required.

We apply the needle insertion forces as boundary conditions on elements in the mesh. Since the needle may be inserted at any location, it is necessary to modify the reference mesh in real-time to ensure that element boundaries are present where the tip and friction forces must be applied. To apply the tip force, a node is maintained at the needle tip location during insertion. To apply the friction forces, a list of nodes along the needle shaft is maintained and these shaft nodes are constrained to only move along the z-axis.

1) Cutting at the needle tip

By default, inserting the needle causes the tip to push tissue but not cut it. This corresponds to a displacement of the needle tip node c in the world frame, but no change in the reference mesh. At every time step, we measure the force f_c applied by the tissue onto the needle tip. We define f_b as the magnitude of the force required to cut a length b of tissue where fb depends on the sharpness of the needle. For each time step in which f_c ≥ f_b, we modify the reference mesh to move the needle tip node a distance b.

We illustrate this mesh modification in Fig. 3. Let point p be the location of the needle tip node c in the reference mesh. The needle tip at node c = i is moving horizontally to the left in the world frame as shown by the vector r′ in Fig. 3(a). This vector is linearly transformed [32] to the reference mesh in Fig. 3(b) and is denoted by r. We move the needle tip node a distance b along r in the reference mesh to a new point p + br. After each time step in which the needle cuts tissue, p moves closer to q where q is the projection of the vector p + br onto the opposite segment (j, k). To maintain a planar mesh with non-overlapping elements, we periodically select a new tip node. When the Euclidean distance from node l (the first node on the needle shaft behind the tip node) to node i is more than twice the distance from node i to point q, node i is added to the needle shaft: the z-component of node i is freed and returned to its original value and the node is constrained to lie on the needle axis by fixing its y-component degree of freedom. The closer of node j or k is moved to p + br and is defined as the new tip node c.

Fig. 3.

The needle is in the interior of the mesh with needle tip node c = i at point p′ moving in direction r′ in the world frame (a). Mesh modification is performed in the reference mesh (b) to represent the path cut by the needle through the tissue. Vector r′ is transformed to r in the reference mesh and the tip node is moved along r a cut distance b as described in Sec. IV-D.

To maintain simulation stability, it is necessary to maintain a topologically valid planar mesh in which all elements have strictly positive area. Using the mesh modification above on a sparse mesh, the tip node may move such that triangle (i, l, h) has negative area, as discussed in [13]. For this to occur, the y-component of r must change sign twice over the span of just 2 element edges. We can avoid this situation by using a finer mesh that prevents the necessary conditions for negative area triangles from occurring, or by using the method proposed by Nienhuys and van der Stappen [36] to efficiently modify the mesh using local edge flips to maintain a valid Delaunay triangulation. We used a sufficiently fine mesh such that local re-meshing was never required and the simulation was sufficiently fast for interactive performance.

2) Friction along the needle shaft

Our stick-slip approach to modeling static and kinetic friction between the needle shaft and the tissue is based on the friction model of Baraff and Witkin [37]. When the tangential velocity of a node along the needle shaft and the velocity of the needle are equal to within a small epsilon threshold, then static friction is applied: the node is attached to the needle and moves at the same velocity along the z-axis. When the tangential force required to attach the node to the needle exceeds a slip force threshold, then the node is freed to slide along the needle shaft and a dissipative force is applied.

3) Seed implantation

At any time during needle insertion, a seed can be implanted at the location of the needle tip s = p. We assume that the seed does not cut tissue, so, after it is implanted, the seed moves in the world frame with the deforming tissue that surrounds it but its coordinate in the reference mesh remains fixed. To maintain the seed at a fixed position in the reference mesh as the mesh is modified, we store in memory the mesh element e containing s. When any node j of element e is moved in the reference mesh during the simulation, we update e by examining each triangle containing node j and checking if point s is in that triangle using the zero-winding rule [38]. By storing the surrounding element of s in the reference mesh, we can efficiently compute the location s′ of the seed in the world frame using the shape functions of e for the deformed mesh [32].

4) Needle retraction

During needle retraction, friction is applied on all the shaft nodes as during insertion, but a tip force is not applied. When the needle retracts past a node on the shaft, that node is removed from the shaft node list and no tip node is maintained.

E. Simulation Visualization

The visual feedback of the simulation is intended to mimic the experience of a physician performing a needle insertion procedure with ultrasound image guidance [12]. We use a static 2D image of the prostate as input. As the user of the simulation inserts the needle, we deform the input image to match the deformations computed in the mesh. We implemented this visualization using texture-mapping [38], where the deformed image is constructed by using mesh 𝒢 to obtain texture map coordinates for 𝒢′. Because 2D texture-mapping is implemented in hardware on all modern graphics cards, using this method does not substantially penalize the speed of the simulation.

In addition to displaying the deformed soft tissue, the simulation also allows the user to selectively overlay clinically relevant information, such as organ outlines, the target location, the needle, and implanted seeds in the deformable tissue. The needle is overlaid on the deformed tissue image by drawing a line between the tip position and the needle entry location in the world frame. This line passes along the element edge boundaries that are created along the needle shaft as it cuts through the tissue. Implanted seeds are displayed by drawing a rectangle (the shape of the seed) at the seed location s in deformed mesh 𝒢. A screen capture from the simulation for the prostate brachytherapy application is shown in Fig. 4.

Fig. 4.

The simulation user interface, which is based on an ultrasound image, is intended to mimic the experience of a physician performing brachytherapy. The physician interactively guides the needle using a mouse and implants seeds (small squares). Tissue deformations and seed locations are predicted and displayed. The implantation error is the distance between the seed and its target (cross) after needle retraction.

V. NEEDLE INSERTION PLANNING

Given a target point g, the goal of needle insertion planning is to find an optimal motion plan X* that minimizes placement error ε = ‖p − g‖, where the final seed implant location p is a function of the plan X. Because the relationship between p and X cannot be defined as a closed-form equation, the optimal X* cannot be computed analytically. Our algorithm efficiently uses the simulation as a function in an optimization algorithm to compute the optimal motion plan X*.

A. Planning Problem Formulation

The planning algorithm’s inputs and outputs are defined by:

Input

• Needle insertion simulator S (as defined in Sec. IV)

• g: Target coordinate in the tissue

• (y_min, y_max): Range of feasible insertion heights

• z_max: Maximum feasible insertion depth

• υ: Needle speed during insertion and retraction

• h: Simulation time step

Output

• X*: Optimal motion plan that minimizes placement error

A naïve planner that ignores tissue deformations would set X = g. If tissue deformations occur, the naïve plan will not reach the specified target, as shown in simulation in Fig. 1 left column.

To estimate the optimal plan X*, the planner computes an offset from g for both the insertion depth and height. The offset for needle insertion depth is necessary because tissue in front of the needle tip is compressed during insertion; the needle must be inserted deeper than z_g to compensate for this compression. The offset for insertion height is necessary since organs or glands (such as the prostate) may rotate during needle insertion. For example, if the needle is inserted near the bottom of the prostate, the gland will rotate clockwise because it is composed of a stiffer tissue than the surrounding soft tissue, as shown in Fig. 5. Hence, the needle must be inserted higher to compensate for its deflected path through the prostate.

Fig. 5.

When the needle pushes against the lower half of the prostate from the right, the prostate rotates clockwise because it is stiffer than the surrounding tissue. This rotation can lead to significant changes in the optimal needle insertion height.

B. Planning Algorithm

We formulate the motion planning problem as an optimization problem, as given in equation 1, where X has 2 degrees of freedom, y_r and z_r. To computationally accelerate the optimization, we consider two one-dimensional problems. First, we implement an algorithm that, given an insertion height y_r, computes the optimal insertion depth z_r:

$$z_r^{*}(y_r)=\underset{z_r}{\text{arg}\text{min}}(\Vert (S((y_r,z_r))-\mathbf{g}\Vert ).$$ (3)

Then, we implement an algorithm that optimizes y_r and uses the first algorithm to implicitly compute $z_r^{*}$ for each candidate y_r:

$$y_r^{*}=\underset{y_r}{\text{arg}\text{min}}(\Vert (S((y_r,z_r^{*}(y_r)))-\mathbf{g}\Vert ).$$ (4)

Equation 3 can be solved efficiently by noting that it is not necessary to fully simulate needle retraction for each candidate plan X. Let k be the node at the needle tip at the time of seed implantation. Since we model tissues as elastic, the displacement u_kj from system 2 will be 0 for all iterations j after the needle has been retracted and steady state is reached. Hence, the location in the world frame of the release point X = (y_r, z_r) after needle retraction will be x_k + u_kj = x_k. Since we assumed that seeds do not cut tissue, the final seed location is p = x_k and the placement error is ε = ‖x_k − g‖, where x_k is the reference mesh coordinate of the node k at the needle tip when it reaches the release point X = (y_r, z_r) in simulation. An implication of this is that we can compute the optimal $z_r^{*}$ in equation 3 by running a single simulation of needle insertion from z_r ≤ 0 until z_r = z_max. At each time step we compute ∊ in O(1) computation time and record $z_r^{*}$ for the lowest ∊. This method is guaranteed to find the optimal $z_r^{*}$ (within the resolution of the time steps) regardless of the convexity properties of equation 3.

Solving equation 3 using this approach requires computing z_max/(υh) simulation time steps, each requiring O(d) time (or slower if a more accurate FEM model or solver is used) as described in Sec. IV. Since the needle tip will move a distance υh each time step, the resolution of $z_r^{*}$ is υh. A small time step h is desirable to improve the resolution of $z_r^{*}$, but the number of time steps required to compute the optimal insertion depth $z_r^{*}$ grows as h decreases.

Solving equation 4 is difficult because derivative values are not available and the function is not guaranteed to be convex. In general, an approximate minimum can be found using a grid search over y_r ∈ (y_min, y_max). However, equation 4 will be unimodal (strictly quasiconvex) near the minimum in cases for which it is not possible to insert the needle at different heights and still reach the same point in the reference mesh of the tissue. Although this property is not guaranteed, it holds for most feasible targets in our simulation that are not adjacent to a tissue type boundary. In such cases, we use a line search method, golden section search [39], to find the optimal $y_r^{*}$. Golden section search, a variant of the Fibonacci search that requires fewer function evaluations, does not require derivative information (which is not available in the simulation) and convergence is guaranteed.

VI. APPLICATION TO PROSTATE BRACHYTHERAPY

During permanent seed prostate brachytherapy, physicians insert into the prostate approximately 20 stiff, hollow needles loaded with radioactive seeds. Physicians use metal guides during insertion to ensure that the needles are all inserted parallel to the z-axis. The needles are inserted, one at a time, to pre-specified targets and the seeds are released. In the clinical workflow, the motion planner can be be executed after a dosimetric plan is generated and a 2D image is obtained in the y,z plane using standard transrectal ultrasound (as shown in Fig. 2) or by extracting a slice from a 3D image such as MRI. The planner will provide clinicians with information to estimate the optimal needle insertion coordinate and depth prior to inserting the needle bearing radioactive seeds.

A. Simulation Implementation

We implemented the simulator in C++ using OpenGL for visualization and tested on a 750MHz Pentium III laptop PC with 256MB RAM with an Intel 815EM graphics chip with 11 MB video SDRAM. When executed in interactive simulation mode, a physician can guide the needle and implant seeds using a mouse, as shown in Fig. 4. For a model with 1250 triangular elements the simulator responds at the rate of 24 frames per second, sufficient for visual feedback (but not fast enough for haptic control). When executed in planning mode, we assume the needle is inserted at a constant velocity of 0.5cm/sec and use a fixed simulation time step of h=1/30 seconds. The simulation runs on standard PC’s running Windows 2000 or XP.

Our anatomy model of the prostate is based on data obtained in the operating room at the UCSF Comprehensive Cancer Center from a patient undergoing brachytherapy treatment for prostate cancer. An ultrasound video was recorded using an ultrasound probe in the sagittal plane, as shown in Fig. 2. The first frame of the ultrasound video was manually segmented by a physician from the UCSF Comprehensive Cancer Center. The segmentation was used to manually generate a mesh composed of n = 676 nodes and m = 1250 triangular elements for a 3.5cm diameter prostate and surrounding fatty tissue. The ultrasound image also served as the texture map image for the simulator. The boundary of the mesh is defined by a square for which the right face (where the needle is inserted) is free, the bottom face corresponding the transrectal ultrasound probe is rigid, and the other two faces are also marked rigid. We previously performed analysis of the sensitivity of simulation results to frictional parameters [13] and also determined the the finite element method is not sensitive to changing the mesh density [40] between 500 and 1000 nodes.

The Young’s modulus E and Poisson ratio ν are set based on the results of Krouskop et al. to E = 60kPa and ν = 0.49 for the prostate and E = 30kPa and ν = 0.49 for the surrounding fatty tissue [17]. Needle properties, including friction parameters, are treated as variables that can be set in the user interface of the simulation. To set default values, we compared the output of the simulation with the ultrasound video and set unknown simulation parameters so that the simulation output closely matched the ultrasound video. UCSF clinicians comparing the two image sequences judged them as highly similar.

B. Sensorless Planner Results

To test planner performance, we selected 12 sample points inside the prostate, shown by the crosses in the Fig. 6. We applied golden section search in the range y_r ∈ (y_g − 0.2cm, y_g + 0.2cm) with tolerance 0.01cm for each target. Without planning, the average error was 0.59cm (17% of prostate diameter) with a standard deviation of 0.10cm. Using our planner, the average error was reduced in simulation to 0.002cm (0.06% of prostate diameter) with a standard deviation of 0.004cm. The average time to compute the optimal plan X* per target was 98 seconds.

Fig. 6.

Twelve sample points were selected as targets marked “+” inside the prostate. Actual seed placements using simulation are marked “•”. Lack of planning results in major placement errors averaging 17% of the prostate diameter (a), which will lead to a poor radioactive dose distribution. Seed placement error was negligible using the planner (b).

We examine in detail the planner results for the example in Fig. 1. The target is located at g = (1.50cm, 3.00cm). Using the “default” plan X = g with the zero offsets, the seed is implanted at p = S(g) = (1.41cm, 2.21cm), a placement error of ε = 0.79cm (23% of the prostate diameter).

In Fig. 7(a), we plot the placement error ε(X = (y_g, z_r)) for insertion at the target height y_r = y_g = 1.5cm. The placement error for the “default” plan can be seen at z_r = z_g = 3.0cm. The error in the depth coordinate is caused primarily because the tissue in front of the needle tip is being compressed before it is cut. Inserting the needle deeper than the target depth decreases the error. If insertion height is held constant at y_r = y_g, placement error can be reduced by 82% to only ε = 0.14cm (4% of prostate diameter) by inserting to a depth of $z_r^{*}=3.84\text{cm}$.

Fig. 7.

(a) Needles should generally be inserted deeper than the target depth to compensate for tissue deformations and minimize placement error. The bold portion of the line denotes feasible seed placements inside the prostate. (b) The motion planner computes the optimal insertion height y_r and corresponding optimal depth $z_r^{*}$ to minimize placement error ∊. Placement error is minimized for (y_r, z_r) = (1.59cm; 3.80cm).

In Fig. 7(b), we plot the optimal surface $\epsilon (\mathbf{X}=(y_r,z_r^{*}(y_r))$. The golden section search described in Sec. V-B efficiently finds the minimum of this surface to determine X* with ε* = 0.003cm (0.09% of prostate diameter) by inserting at height $y_r^{*}=1.59\text{cm}$ to a depth $z_r^{*}=3.80\text{cm}$.

VII. CONCLUSION

To facilitate physician training and pre-operative planning for medical needle insertion procedures, we developed a needle insertion motion planning system based on an interactive simulation of the insertion of rigid needles into soft tissue.

The first component of this system is a physically-based, dynamic simulation of needle insertion in soft tissues. In this paper, we introduced a simulation that uses a 2D finite element model of soft tissue and a reduced set of scalar parameters such as needle friction, sharpness, and velocity. The simulation modifies the mesh to maintain element boundaries along the needle shaft and models the effects of cutting at the needle tip and frictional forces along the needle shaft. The simulation achieves real-time, interactive performance on a standard PC. Using texture-mapping, the simulation provides visualization comparable to ultrasound images that the physician would see during the procedure. We hope that needle insertion simulation can be used for physician training as an alternative to the limited and expensive mechanical models currently in use.

The second component, a sensorless planning algorithm, uses the first component and numerical optimization to compute needle insertion offsets that compensate for tissue deformations. The sensorless planner uses the simulation as a function to estimate placement error. We apply the method to radioactive seed implantation during permanent seed prostate brachytherapy to minimize seed placement error. The effectiveness of the planner in vivo will be dependent on the accuracy of the simulation of needle insertion and tissue deformations for a specific patient.

In future work, we will extend our current 2D simulation to consider a 3D representation of anatomy and model nonlinear and anisotropic tissue properties, slip between tissues, and needle bending. This extension will enable explicit reduction of placement error along the x axis and will potentially improve the quality of tissue deformation prediction. The extension will require 3D tetrahedral or hexahedral mesh generation, 3D FEM, visualization of 3D deformations, acquisition of nonlinear tissue parameters, and an extension of the optimization algorithm to consider an extra degree of freedom. We hope to achieve interactive real-time performance by parallelizing the FEM and optimization algorithms across multiple cores of modern CPU’s. We also plan to acquire MR imaging data of needle insertion procedures at the UCSF Comprehensive Cancer Center to numerically validate the simulations across multiple patients and improve simulation accuracy. This will also enable us to evaluate the trade-off between computation time and improved accuracy for each of the proposed extensions to determine the model that is clinically most effective. We also plan to extend the simulation and planner to new medical procedures.

Past work on patient-specific image-guided needle procedures uses local control to compensate for errors induced by tissue deformation but do not pre-operatively consider these effects [26]. Conversely, our sensorless planner searches for a globally optimal insertion plan but does not consider anomalies that may occur during execution. In the long run, we will combine these approaches to create a pre-operative plan that is optimal under uncertainty and then use information from real-time imaging, when available, to correct deviations from the pre-operative plan.

Biographies

Ron Alterovitz (S’03M’09) received the B.S. degree in engineering and applied science (computer science) (with honors) from Caltech, Pasadena, CA, in 2001, and the Ph.D. degree in industrial engineering and operations research from the University of California, Berkeley, in 2006.

He is currently an Assistant Professor of computer science at the University of North Carolina at Chapel Hill. He was previously engaged in research at the University of California, San Francisco (UCSF) Comprehensive Cancer Center, the Department of Electrical Engineering and Computer Sciences, University of California, and the Robotics and Artificial Intelligence (AI) Group, Laboratory for Analysis and Architecture of Systems-National Center for Scientic Research (LAAS-CNRS), Toulouse, France. His research interests include developing motion planning algorithms and physically based simulations for medical applications, including treatment planning, device control, image registration, and physician training.

Dr. Alterovitz is the recipient of the National Institutes of Health (NIH) Ruth L. Kirschstein National Research Service Award, the National Defense Science and Engineering Graduate Fellowship, the National Science Foundation (NSF) Graduate Research Fellowship, and the IEEE International Conference on Intelligent Robots and Systems (IROS) Best Paper Finalist Award.

Kenneth Y. Goldberg (S’84-M’90-SM’98-F’05) received the Ph.D. degree in computer science from Carnegie Mellon University (CMU), Pittsburgh, PA, in 1990.

He is a Professor of industrial engineering and operations research at the University of California, Berkeley, with secondary appointments in electrical engineering and computer science in the School of Information. He also studied at the University of Pennsylvania, Edinburgh University, and the Technion. From 1991 to 1995, he taught at the University of Southern California, and in fall 2000 was visiting faculty at the Massachusetts Institute of Technology (MIT) Media Laboratory. His current research interests include algorithmic automation, medical robotics, and networked telerobotics.

Prof. Goldberg is the Founding Chair of the Advisory Board to the IEEE Transactions on Automation Science and Engineering. He received the National Science Foundation Young Investigator Award in 1994, the National Science Foundation (NSF) Presidential Faculty Fellowship in 1995, the Joseph Engelberger Award for Robotics Education in 2000, and the IEEE Major Educational Innovation Award in 2001. He is currently the Vice President of Technical Activities for the IEEE Robotics and Automation Society and the Director of the Berkeley Center for New Media.

Jean Pouliot received the Ph.D. degree in nuclear physics from Laval University, Quebec, QC, Canada, in 1986.

He is currently the Head of the Physics Division, the Vice Chair, and a Professor of radiation oncology at the University of California, San Francisco (UCSF), with a joint appointment with the Graduate Bioengineering University of California Berkeley UCSF program. His current research interests include development and the clinical integration of dose-guided radiation therapy with megavoltage cone-beam computed tomography (CT) for patient verication, organ motion and tumor evolution studies during cancer irradiation, and inverse planning simulated annealing (IPSA) for dose distribution optimization of image-guided high dose rate and permanent prostate implant brachytherapy. He is the author or coauthor of more than 140 peer-reviewed publications.

Prof. Pouliot was voted among the Top 25 Innovators in the U.S. Health Imaging and IT in June 2006 for his pioneering research on megavoltage cone-beam CT imaging.

Dr. I-Chow Hsu received the Medical degree from Johns Hopkins School of Medicine, Baltimore, MD. He completed his residency in radiation oncology at Columbia-Presbyterian Medical Center. In 1997, he joined the faculty at the University of California, San Francisco, where he is currently an Associate Professor and a Vice Chair in the Department of Radiation Oncology. He is interested in the development of minimally invasive implant procedures and precision delivery of conformal high dose rate brachytherapy and hyperthermia. He is an expert in brachytherapy for genitourinary tumor, gynecological tumor, head and neck tumor, and soft tissue tumor.

Dr. Hsu is an active member of several professional societies including the American Society for Therapeutic Radiology and Oncology and the Radiation Therapy Oncology Group.