Experimental Evaluation of Ultrasound-Guided 3D Needle Steering in Biological Tissue

Abstract

Purpose

In this paper, we present a system capable of automatically steering bevel-tip flexible needles under ultrasound guidance towards stationary and moving targets in gelatin phantoms and biological tissue while avoiding stationary and moving obstacles. We use three-dimensional (3D) ultrasound to track the needle tip during the procedure.

Methods

Our system uses a fast sampling-based path planner to compute and periodically update a feasible path to the target that avoids obstacles. We then use a novel control algorithm to steer the needle along the path in a manner that reduces the number of needle rotations, thus reducing tissue damage. We present experimental results for needle insertion procedures for both stationary and moving targets and obstacles for up to 90 mm of needle insertion.

Results

We obtained a mean targeting error of 0.32 ± 0.10 mm and 0.38 ± 0.19 mm in gelatin-based phantom and biological tissue, respectively.

Conclusions

The achieved submillimeter accuracy suggests that our approach is sufficient to target the smallest lesions (ϕ2 mm) that can be detected using state-of-the-art ultrasound imaging systems.

1 Introduction

Needle insertion into soft-tissue is a minimally invasive procedure used for diagnostic and therapeutic purposes such as biopsy and brachytherapy, respectively. Examples of diagnostic needle insertion procedures are liver and lung biopsies to detect tumors [1, 2]. Therapeutic applications of needle insertion include brachytherapy of cervical, prostate and breast cancers [3]. Imaging modalities such as ultrasound, magnetic resonance (MR), and computed tomography (CT) are often used during needle insertion procedures to determine the positions of the needle and target for accurate needle tip placement [4]. Inaccurate placement may result in misdiagnosis and unsuccessful treatment during biopsy and brachytherapy, respectively. The needles usually used in such procedures are rigid. Such needles do not provide the clinician with sufficient steering capabilities that allow the needle to avoid certain obstacles and reach the intended target [5].

The steerability of the needle is improved by introducing flexible needles. Such needles can be used to steer around sensitive and hard tissue such as blood vessels and bones, respectively [6–8]. The flexible needles fabricated with an asymmetric tip (bevel tip) naturally deflect during insertion into soft-tissue (Fig. 1) [9,10]. The needle deflection due to its tip-asymmetry is used to steer the needle to reach a certain target position [5,7]. The needle is assumed to deflect along a circular path during insertion. This assumption is used in various studies to model the needle deflection during insertion [7, 9–11].

Fig. 1.

The experimental setup shows the needle insertion device and the transducer control device. The upper inset depicts biological tissue (chicken breast) embedded in a gelatin phantom. The lower inset shows the needle bevel tip.

The deflection of a needle with a bevel tip can be controlled using duty-cycling of the needle during insertion [12, 13]. This algorithm can vary the needle curvature by changing the ratio between period of needle insertion with spinning to the total period of insertion. The main disadvantage of the duty-cycling approach is that it requires excessive number of rotations of the needle inside the tissue that might increase tissue damage, and subsequently patient trauma [14]. In recent studies, control algorithms were developed for needle steering in two-dimensional (2D) space. These control algorithms enhanced the needle targeting accuracy compared to manual needle control but some improvements are required to bring these methods to clinical practice such as considering the physiological motion, tissue inhomogeneity and fluid flow. DiMaio and Salcudean presented a path planning and control algorithm that related the needle motion at the base (outside the soft-tissue phantom) to the tip motion inside the tissue [15]. Glozman and Shoham, and Neubach and Shoham developed an image-guided closed-loop control algorithm for steering flexible needles using fluoroscopic and ultrasound images, respectively [16,17]. They solved forward and inverse kinematics of the needle for 2D path planning. Abayazid et al. presented a 2D ultrasound image-guided steering algorithm, and a three-dimensional (3D) steering algorithm where they used Fiber Bragg Grating sensors for feedback [18, 19]. Chatelain et al. developed a real-time needle tracking method by servoing images obtained from a 3D ultrasound probe [20]. Reed et al. integrated a path planner and stabilizing controller for needle steering on a 2D plane [21]. Seiler et al. developed a planning method for correcting a path using Lie group symmetries [22]. Hauser et al. developed a 3D feedback controller that steers the needle along a helical path, although results were evaluated in simulation without physical experiments [23].

Several studies presented 2D path planning algorithms for steering flexible needles, but our focus in this paper is on 3D steering [15, 24–26]. Duindam et al. presented fast 3D path planning algorithms based on inverse kinematics and optimization, although these methods do not offer any completeness guarantees, i.e., they may fail to return a solution for problems with obstacles [27,28]. Park et al. developed a path-of-probability algorithm that considers uncertainty in needle motion using diffusion-based error propagation, but the presence of obstacles affects the completeness of the planner [29]. Several 3D path planning algorithms have been introduced that are based on Rapidly-exploring Random Trees (RRTs) [30, 31]. Our approach integrates ideas from Patil et al. to quickly compute feasible, collision-free paths in 3D that solves the problem of failure in providing the path during presence of obstacles [31].

The proposed system, depicted in Fig. 2, is a step forward to achieve a clinically viable robotic needle steering system. The anatomical regions of interest in the patient are acquired pre-operatively using ultrasound images. Based on the images, the clinician identifies the target location and sensitive structures such as glands or blood vessels and other obstacles such as bones. The path planning algorithm generates a needle trajectory to avoid obstacles and reach the target. The planner generates new paths intra-operatively based on the updated needle tip position (obtained from ultrasound images) and target position during insertion. The needle insertion procedure is autonomous under supervision of the clinician.

Fig. 2.

The workflow presents a clinically viable robotic needle steering system. The needle insertion device controls the direction of insertion inside the patient’s soft tissue. Needle tip tracking and path planning are performed intra-operatively to provide control algorithm and the clinician with data required to control the insertion device.

In the current study, we integrate the presented 3D tracking, path planning and control algorithms to steer a bevel-tipped flexible needle to reach a target in 3D space while avoiding obstacles. The proposed control algorithm provides a reduced number of needle rotations to reach the target location to minimize tissue damage. The algorithms are validated by conducting insertion experiments into a soft-tissue phantom and biological tissue (chicken breast) while avoiding virtual and real obstacles. The contributions of this work include:

• The use of ultrasound-based 3D needle tracking combined with 3D real-time path planning for avoiding real obstacles.

• 3D steering and path planning for needle insertion into biological tissue.

• Experimental evaluation of needle steering towards a moving target while avoiding more than one moving obstacle.

In the following section, we describe the ultrasound-based needle tip tracking algorithm. We then describe the path planning method and the control algorithm, which reduces the number of needle rotations inside soft tissue to reduce patient trauma. Finally, we present our results in soft-tissue phantoms and biological tissue.

2 Three-Dimensional Needle Tracking

We use a high resolution 2D ultrasound transducer to obtain the needle tip pose during insertion. The resolution of the ultrasound image is 0.12 mm per pixel. The ultrasound transducer is placed to visualize the tip, and orientated perpendicular to the needle insertion direction (X-axis of frame (Ψ₀)) as shown in Fig 3. The resulting ultrasound image shows a radial cross-sectional view of the needle. The cross-section of the needle does not look circular in the ultrasound image due to reverberation artifacts [32]. These artifacts occur due to bouncing of the ultrasound waves between materials of different acoustic impedance such as the needle and the surrounding tissue. The resulting artifact visible in ultrasound images has a tail-shaped structure of equally spaced echoes along the sound wave. The length of the tail-shaped structure depends on the bouncing echoes that are received by the transducer. The reverberation artifact is often referred to as the comet tail artifact (CTA) [33]. An image processing algorithm is used to determine the centroid of the needle in the ultrasound images. The needle in ultrasound images is enhanced by a series of basic image processing techniques, including median filtering, thresholding, and erosion and dilation, as shown in Fig. 3. Some extra processing steps are performed to remove artifacts that appear in ultrasound images while scanning biological tissue (chicken breast). The ultrasound image is filtered to eliminate the speckles that look similar to the needle tip. This is achieved by applying an additional erosion step and by reducing the image intensity gain using the ultrasound device settings.

Fig. 3.

Overview of the various coordinate systems and the image processing techniques used to evaluate the needle tip pose with respect to the fixed reference frame (Ψ₀). The fixed reference frame (Ψ₀) is located at the entry point of the needle in the soft-tissue phantom. Frame (Ψ_n) is fixed at the needle insertion device end-effector. Frame (Ψ_p) is fixed at the ultrasound transducer end-effector. Frame (Ψ_t) is located at the needle tip, while frame (Ψ_t̂) is located at the needle tip as determined by the tracking algorithm. The aberration in transducer position along the insertion axis (x-axis of frame (Ψ₀)) is given by ±λ. The perpendicular placed 2D ultrasound transducer provides a radial cross-sectional view of the needle which is affected by the comet tail artifact (CTA). An image processing methodology is used to evaluate the needle centroid location in the ultrasound image frame (Ψ_u). (a) A median filter is applied to suppress speckle in the ultrasound image. (b) Thresholding is performed to obtain a binary image. (c) Erosion and subsequently dilation is applied to remove the remaining speckle. (d) A feature extraction algorithm based on Hough transform is applied to determine a line segment denoted $\overline{AB}$ to describe the needle with CTA. (e) The needle centroid (y_c, z_c) is evaluated as the red circle, from A which represents a point on the surface of the needle in the direction of B at a distance equal to the radius of the needle.

The 2D ultrasound transducer needs to compensate for needle tip motion along the x-axis of frame (Ψ₀). A positioning device is used to control the ultrasound transducer. The positioning device moves the transducer corresponding to the needle motion to provide ultrasound images of the needle tip during insertion. This allows the needle tip pose to be expressed in the fixed reference frame using a series of coordinate transformations between frames (Ψ_u, Ψ_p and Ψ₀). Further details regarding coordinate transformations and control of the transducer motion are presented in the work of Vrooijink et al. [34]. The tracking algorithm is evaluated in gelatin phantoms and the mean errors of the needle tip position along X-, Y - and Z-axes (frame (Ψ₀)) are 0.64 mm, 0.25 mm and 0.27 mm, respectively, using insertion velocities between 1 – 5 mm/s.

3 Three-Dimensional Needle Path Planning and Control

The tracking algorithm determines the needle tip location for feedback to the control system. The control system incorporates a path planning algorithm to generate the optimal needle trajectory towards the target. In the current section, we describe the 3D path planning and control algorithms.

3.1 Path Planning

We use a 3D path planning algorithm to enable the needle to reach a target while avoiding obstacles in a 3D environment [31]. Using feedback from ultrasound imaging, the system steers the needle to approximately track the planned path using the control algorithm described in Sec. 3.2.

For path planning, the system uses a customized RRT, a sampling-based method for path planning [35]. The main advantage of using an RRT is that our implementation is fast enough for real-time path planning during insertion if the needle is inserted with the insertion velocities used in clinical applications (0.4–10 mm/s) [36]. To enable fast performance, our path planner makes use of reachability-guided sampling for efficient expansion of the rapidly-exploring search tree [37]. We also relax the constraint of constant-curvature needle trajectories by assuming that the controller can realize bounded-curvature needle trajectories by alternating the bevel tip direction. These customizations help us to reduce the computational time compared to prior sampling-based planners and make the path planner suitable for closed-loop needle steering [30]. We refer the reader to Patil et al. for additional details on the planning algorithm [31].

Given pre-operative medical images, the clinician can specify the insertion location, the target location, and the geometry of obstacles, which can include sensitive structures such as glands or blood vessels as well as impenetrable structures such as bones. After specifying the entire environment, the path planner computes a path that (1) reaches the target, and (2) is feasible, i.e., avoids obstacles. The output of the path planning algorithm is a sequence of milestones along the path defined at 6 mm intervals. The control algorithm discussed in Sec. 3.2 begins by steering the needle toward the first milestone along the path. As soon as a milestone is reached, the control algorithm steers the needle toward the next milestone along the path.

Since the obstacles or target may move during the procedure, the system operates in a closed-loop fashion by replanning every second. At each replanning step, a path is computed from the needle tip pose that is estimated by the needle tip tracking algorithm. The path planner also uses the actual positions of the target and the obstacles at each replanning step. After a new plan is computed, the method updates the milestones used by the control algorithm.

3.2 Control Algorithm

We assume that the needle moves along a circular path during insertion based on the bevel direction [9,10]. Axially rotating the needle about its insertion axis adjusts the tip orientation to control the direction of insertion. This rotation enables the control algorithm to direct the tip towards a target.

The control algorithm guides the needle toward the planned path’s next milestone, which we refer to in this paragraph as the target of the control algorithm. The frame (Ψ_t) is attached to the needle tip (Fig. 4(a)). Unless otherwise stated all variables are expressed in frame (Ψ_t). The needle tip position ( ${\mathbf{p}}_{\text{tip}}^0\in {ℝ}^{3\times 1}$) and orientation ( ${\mathbf{R}}_{\text{tip}}^0\in {ℝ}^{3\times 3}$) with respect to the global coordinate frame (Ψ₀) are obtained using the needle tip tracking algorithm (Sec. 2). The target position is set to be a static or a moving point in 3D space. The target position ( ${\mathbf{p}}_{\text{tar}}^{\text{tip}}\in {ℝ}^{3\times 1}$) with respect to frame (Ψ_t) is ${\mathbf{p}}_{\text{tar}}^{\text{tip}}={[p_{x_{\text{tar}}}^{\text{tip}}{p}_{y_{\text{tar}}}^{\text{tip}}{p}_{z_{\text{tar}}}^{\text{tip}}]}^{\mathrm{T}}$, where $p_{x_{\text{tar}}}^{\text{tip}},p_{y_{\text{tar}}}^{\text{tip}}$ and $p_{z_{\text{tar}}}^{\text{tip}}$ represent the target positions along the X-, Y - and Z-axes, respectively. In Fig. 4(a), the conical shape represents the region that the needle can reach during insertion. The plane containing the control circle with center (c_con) intersects the target and lies on the plane perpendicular to the X-axis. The radius (r_con) of the control circle is calculated using $r_{\text{con}}=r_{\text{cur}}-\sqrt{r_{\text{cur}}^2-{(p_{x_{\text{tar}}}^{\text{tip}})}^2}$, where r_cur is the radius of curvature of the needle path (Fig. 4(b)), and it is obtained experimentally. The distance between c_con and the target position (d_tar) in Y Z-plane (Fig. 4(c)) is determined from trigonometry. As the needle moves towards the target during insertion, the radius (r_con) decreases. The needle will rotate about its axis if the target intersects the circumference of the control circle (d_tar ≥ r_con) to keep the needle in the reachable region. The needle rotates by the angle (θ) to direct the needle tip towards the target (Fig. 4(c)). The control algorithm updates the value of θ every 40 ms. Additional details concerning the control algorithm are presented in the work of Abayazid et al. [19]. The control algorithm is validated experimentally, as demonstrated in the following section.

Fig. 4.

(a) The path planning algorithm generates a feasible path by exploring the state space using a rapidly exploring random tree. The path planner generates milestones along the path, and the control algorithm steers the needle from milestone to milestone to reach the target. (b) In the control algorithm, the region the needle tip can reach is represented by a three-dimensional conical shape. The frame (Ψ_t) is attached to the needle tip, and the needle insertion starts in the X-direction. The area of the control circle (with center (c_con)) intersects the target and is perpendicular to the X-axis (frame (Ψ_t)). (c) The radius (r_con) is determined using the radius of curvature (r_cur) of the needle and the distance ( $p_{x_{\text{tar}}}^{\text{tip}}$) between the tip and target along the X-axis. (d) The needle rotates about its axis by angle (θ) if the distance (d_tar) between c_con and target is larger than or equal to r_con.

4 Experiments

In this section, we present the experimental setup used to insert the needle into the soft-tissue, the experimental plan, and the results.

4.1 Setup

The experimental setup is divided into two parts. First, the insertion device allows the needle to be inserted and rotated about its axis. The details of the needle insertion device are presented in previous work [18]. Second, a transducer control device that permits the ultrasound transducer to move in three degrees of freedom, as shown in Fig. 1. The 18 MHz transducer (18L6 HD with a mean ultrasound beam width of 0.4 mm) is connected to a Siemens Acuson S2000 ultrasound machine (Siemens AG, Erlangen, Germany). Additional information about the transducer control device is presented by Vrooijink et al. [34].

The needle is inserted into a soft-tissue phantom made up of a gelatin mixture [19]. Silica powder is added to the mixture to mimic the acoustic scattering of human tissue. The flexible needle is made of Nitinol alloy (nickel and titanium). The Nitinol needle has a diameter of 0.5 mm with a bevel angle (at the tip) of 30°.

4.2 Results

In the current section, different experimental scenarios are conducted to evaluate the performance of the proposed needle tracking, path planning and control algorithms. The needle radius of curvature in the phantom is determined empirically (270 mm) [19]. A safety margin is added to the needle curvature value to compensate for variations or disturbances that may take place during insertion. The needle is inserted with a velocity of 1 mm/s. Each experimental case is performed five times. The experimental cases are depicted in Fig. 5.

Fig. 5.

The experimental cases. (a) Case 1: the needle is steered towards a stationary virtual target using a random path. (b) Case 2: the needle is steered towards a moving virtual target using a random path. (c) Case 3: the needle moves along a planned path to avoid virtual obstacles and reach a stationary virtual target in a gelatin phantom. (d) Case 4: the needle moves along a planned path to avoid real obstacles and reach a stationary virtual target. (e) Case 5: the needle moves along a planned path to avoid obstacles and reach a stationary virtual target in biological tissue (chicken breast). (f) Case 6: the needle moves along a planned path to avoid two moving obstacles and reach a moving virtual target. The mean targeting error (absolute distance between the needle tip and the target position at the end of insertion) of Case i is e_μi, where i = 1, 2, 3, 4,5 and 6. The planned path is updated every second. Please refer to the accompanying video that demonstrates the experiments.

In Case 1 and Case 2, the steering algorithm controls the needle to reach a stationary and a moving virtual target, respectively (no path planning is applied) (Fig. 5(a) and (b)). In Case 3 and Case 4, path planning is applied pre-operatively to generate the optimal trajectory between the needle tip and the target. In Case 3, virtual obstacles used while in Case 4, real obstacles are embedded into the gelatin phantom. The real obstacles are 3D-printed plastic shapes (Fig. 5(d)). The phantom is scanned pre-operatively to localize the real obstacles in the soft-tissue phantom. The obstacles appear dark in the ultrasound image frames. The images are inverted and a threshold is set to obtain a binary image. The location of the obstacle is determined by calculating the centroid of the white region in each image frame (obstacle after inversion) and then along the frames that include the obstacle. The obtained obstacle location is exported to the path planning algorithm. Our system assumes that the shape of obstacles are recognized by the planner, which requires segmenting obstacles in pre-operative medical imaging. The segmentations can be produced manually by a physician (for fixed obstacles obstacles) or automatically using segmentation software. (We note that automatic segmentation is a challenging problem that is actively being studied and is beyond the scope of this work.) The steering algorithm moves the needle along the generated path to avoid the obstacles and reach the target using milestones (Fig. 5(c) and (d)). In Case 5, the needle is steered towards a virtual target in biological tissue embedded in a gelatin phantom while avoiding virtual obstacles as shown in the lower inset in Fig. 1 and Fig. 5(e). In Case 6, the virtual target moves away from the needle tip in the direction of the needle orientation with a velocity of 0.125 mm/s. This results in a target motion of 10 mm. The target moves in the direction of needle insertion to simulate the effect of tissue deformation caused by the needle compression on the surrounding tissue. The path is updated every second to avoid the moving obstacles and reach the moving target. The obstacles move in the direction of the needle path with a velocity of 0.06 mm/s (Fig. 5(f)). The targeting error is the absolute distance between the target position that is pre-defined and needle tip position obtained from the needle tracking algorithm described in Section 2. The mean targeting errors for all experimental cases are provided in Fig. 5. Please refer to the accompanying video that demonstrates the experiments.

5 Discussion

This study combines a 3D real-time ultrasound-based needle tracking with path planning and control algorithms. These algorithms are used to accurately steer bevel-tipped flexible needles towards stationary and moving targets while avoiding virtual and real obstacles. The main advantage of the proposed control algorithm is that the needle rotates only when a change of the direction of insertion is required. This reduces the number of full rotations of the needle, and thus has the potential to reduce patient trauma [14]. In the implementation of the control algorithm, the needle can rotate in both directions to reduce the angle of rotation. The reduction of rotation angle suppresses the effect of torsion along the needle shaft which reduces the error between its orientation at the tip and base. Experiments were also performed using duty-cycling algorithm and compared to the proposed control algorithm to estimate its influence on the number of needle rotations (tissue damage). For the same insertion distance and path planner settings, the duty-cycling control algorithm required 51 complete rotations of the needle, while the proposed algorithm performed the procedure with just 11 complete rotations.

Experiments are performed to evaluate the targeting accuracy of the proposed system. Six experimental cases are performed to validate the tracking, path planning and control algorithms. The needle is steered in gelatin phantom and biological tissue. The needle visibility in ultrasound images is deteriorated due to shadows surrounding the solid obstacles during insertion, and this affects the targeting accuracy in Case 4. The targeting error increases while steering in biological tissue (Case 5) due to tissue inhomogeneity. This causes variation in the needle behavior during insertion. The experimental results show that the mean targeting error ranges between 0.24 ± 0.09 mm and 0.38 ± 0.19 mm. An extra experiment is conducted to validate the proposed system using a real ϕ 6 mm target made of an aqueous solution of 20 wt.% polyvinyl alcohol (PVA) (SigmaAldrich Chemie B.V., Zwijndrecht, The Netherlands). This experiment is performed five times, and the insertion distance ranges between 86 and 102 mm. The target and the obstacle are stationary, and their positions are determined using a pre-operative ultrasound scan. The needle tip reaches the target in each experimental trial. Fig. 6 shows a representative ultrasound image of the cross-section of the target penetrated by the needle tip.

Fig. 6.

The ultrasound image shows a cross-section of the target (ϕ 6 mm) embedded in a soft-tissue phantom at the end of the needle insertion, and the tip penetrating the target.

The needle insertions performed in the current study are conducted in an experimental environment where the needle is inserted into a static phantom that contains two types of materials (gelatin and chicken breast tissue). In a clinical environment, we expect more variables that may reduce the targeting accuracy such as physiological motion, fluid flow and tissue inhomogeneity. Further improvements are required to bring the system to the clinical practice. In future work, the ultrasound needle tracking device will be adapted to track the needle tip while scanning curved surfaces. A technique should also be developed for 3D reconstruction of the shape of targets and obstacles preoperatively and then tracking of real targets and obstacles in real-time during insertion into biological tissue in order to improve the targeting accuracy. The steering system can be extended to detect the patient movements that occur during needle insertion such as respiration and fluid flow. A model should be developed to estimate the needle curvature in different heterogeneous tissue for accurate targeting. Real-time shared control between the steering algorithm and the operator will be established to achieve a practical system for clinical operations.