STEERABLE NEEDLE TRAJECTORY FOLLOWING IN THE LUNG: TORSIONAL DEADBAND COMPENSATION AND FULL POSE ESTIMATION WITH 5DOF FEEDBACK FOR NEEDLES PASSING THROUGH FLEXIBLE ENDOSCOPES

Abstract

Bronchoscopic diagnosis and intervention in the lung is a new frontier for steerable needles, where they have the potential to enable minimally invasive, accurate access to small nodules that cannot be reliably accessed today. However, the curved, flexible bronchoscope requires a much longer needle than prior work has considered, with complex interactions between the needle and bronchoscope channel, introducing new challenges in steerable needle control. In particular, friction between the working channel and needle causes torsional windup along the bronchoscope, the effects of which cannot be directly measured at the tip of thin needles embedded with 5 degree-of-freedom magnetic tracking coils. To compensate for these effects, we propose a new torsional deadband-aware Extended Kalman Filter to estimate the full needle tip pose including the axial angle, which defines its steering direction. We use the Kalman Filter estimates with an established sliding mode controller to steer along desired trajectories in lung tissue. We demonstrate that this simple torsional deadband model is sufficient to account for the complex interactions between the needle and endoscope channel for control purposes. We measure mean final targeting error of 1.36 mm in phantom tissue and 1.84 mm in ex-vivo porcine lung, with mean trajectory following error of 1.28 mm and 1.10 mm, respectively.

INTRODUCTION

The development of bevel tip steerable needles [1] has been motivated by their potential to steer around obstacles and correct for factors like tissue deformation, needle bending, and other effects that cause traditional needles to miss their desired targets (see [2] and [3] for reviews). Steerable needles have been envisioned in applications including the liver [4], lungs [5], kidneys [4], prostate [6], and brain [7].

Interventions in the lung [5] are particularly promising, given the large number of potentially cancerous small nodules for which biopsy is desired [8], but for which complication rates are high for percutaneous approaches [9]. Similarly, existing bronchoscope-delivered needles either cannot reach these nodules (due to lack of an airway in close proximity), or cannot reach them with sufficient accuracy, leading to low yields [10]. Bevel tip steerable needles have the potential to enhance accuracy and steer to targets far from airways large enough to admit bronchoscopes. However, controlling the needle once delivered into the lung through a bronchoscope is challenging, for several reasons.

First, the bronchoscope requires an insertion pathway (bronchoscope channel) that is much longer than has previously been considered for percutaneous needle insertion. Second, it is desirable to use small-diameter needles for reducing required bronchoscope port diameter (and hence potentially facilitating the use of smaller diameter bronchoscopes, which can reach deeper into the lung). Another reason a small diameter is desirable is that it reduces needle bending stiffness, which reduces the loads required on bronchoscope tendons to deflect the tip of the bronchoscope with the needle loaded inside of it. Yet another motivation for smaller diameter needles is reduction of invasiveness for the patient; smaller needles require less tissue disruption during insertion [11]. Using 5 degree-of-freedom (DOF) tracking coils (0.41 mm diameter), rather than the larger 6-DOF versions (0.8 mm diameter) [12], for feedback enables smaller diameter needles.

Both long needle length and small diameter contribute to torsional windup along the needle in the presence of needle-port friction. Torsional friction acting along the long flexible needle shaft causes the needle tip rotation (about the needle axis) to be different than the needle base rotation. In general, it is challenging to model the interaction of a flexible metal cylinder passing through a larger tube (see e.g. [13]). Furthermore, the one DOF lost in moving from a larger 6-DOF magnetic tracking coil to a smaller 5-DOF coil is precisely the one we most care about – the axial angle at the needle’s tip, which defines the bevel’s direction, and hence the direction of forward progression of the needle as it is inserted.

The torsional friction challenge has been studied previously for steerable needles in the context of torsion induced by needle-tissue interactions [14-17]. These results assume continuous contact with tissue, as well as a length of unsupported needle outside the tissue, i.e. the context one would expect in a percutaneous insertion. Thus, these results do not apply directly to the non-uniform contact conditions along the interior of a bronchoscope port. In this paper, rather than model this complex interaction explicitly, we instead assume a simplified model with Coulomb friction which induces a torsional deadband (i.e. a range of values for which the tip does not axially rotate when one changes the direction of base axial rotation), with the goal of determining whether such a model is sufficient for closed-loop control.

Various forms of observer models, including Extended Kalman Filters (EKFs) have been used in a variety of ways to estimate state in both 2D and 3D needle steering. Needle-tissue interaction forces [18, 19] and needle deflection [20] have both been estimated using this method. In another study, an EKF was used to approximate belief dynamics in belief space planning for steerable needles [21]. EKFs have also been used to estimate a steerable needle’s planar, single-dimensional orientation from the needle’s two-dimensional position, sensed using a vision system [22, 23]. This concept has also been applied to full 6-DOF pose estimation from a sensed sequence of 3D needle tip positions [24, 25]. Observers have also been used previously to estimate the axial orientation of a steerable needle while sensing the position using a vision system [23, 26].

Our approach in this paper is inspired by these prior papers and adapts their EKF paradigm to the context of the torsional deadband mentioned previously. In terms of control, a wide variety of controllers have been developed for bevel-tip steerable needles (see [27] for a review). In this paper, we use the established sliding-mode controller proposed by Rucker et al. [28]. The primary contribution of our paper is to determine whether a simple Coulomb friction model can be used to effectively control a trans-bronchoscopic steerable needle with Kalman Filter-based roll estimation. We find that this simple modeling and control approach produces excellent accuracy when steering in phantom tissues and in inflated ex-vivo porcine lung tissue.

METHODS

In this section, we explain the integration of the torsional deadband into the steerable needle model, the Extended Kalman Filter estimation, and the needle steering controller. All vectors are written in bold lower case letters, and matrices are denoted with bold capital letters. The steerable needle model is illustrated in Figure 1, and the needle state estimation and control scheme is shown in Figure 2.

Figure 1.

Steerable needle kinematics with the actuation variables at the base (υ₁ and υ₂) and the tip of the needle (u₁ and u₂).

Figure 2.

Overall control scheme for the proposed torsional deadband aware needle pose estimator and steering controller. The pose estimator components are shown in the Extended Kalman Filter box (in red). The controller takes the current needle pose estimate $\widehat{\mathit{x}}$ and determines the actuator commands at the needle base (υ₁, υ₂) to reach the target point p_d. The feedback from the physical needle tip is obtained with the 5-DOF magnetic tracker, and sent to the EKF pose estimator.

Review of Steerable Needle Kinematics

We use the nonholonomic steerable needle kinematic model described in [1]. In this model, the steerable needle is assumed to follow a constant curvature path when inserted, as shown in Figure 1. The needle curves in the direction of its bevel, and axial rotation changes the plane in which the needle curves. The needle motion is described by

$$\begin{gathered} \stackrel{.}{\mathit{p}}=\mathit{R}\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 0\hfill \\ \hfill u_1\hfill \end{array}\right] \\ \stackrel{.}{\mathit{R}}=\mathit{R}[\omega {]}_{\times }=\mathit{R}=\left[\begin{array}{ccc}\hfill 0\hfill & \hfill -u_2\hfill & \hfill 0\hfill \\ \hfill u_2\hfill & \hfill 0\hfill & \hfill -\kappa u_1\hfill \\ \hfill 0\hfill & \hfill \kappa u_1\hfill & \hfill 0\hfill \end{array}\right], \end{gathered}$$ (1)

where the position vector of the needle tip is denoted by p, and the orientation of the needle tip is denoted by the rotation matrix R. Actuation variables for the needle are the insertion and rotation speeds, denoted by u₁ and u₂, respectively. The angular velocity of the needle tip expressed in the body frame is given by ω = [κu₁ 0 u₂]^T. The operator [ ]_× represents the transformation to a skew-symmetric matrix. κ denotes the curvature of the needle.

The needle tip frame rotation can also be expressed using quaternions as

$$\begin{gathered} \left[\begin{array}{c}\hfill 0\hfill \\ \hfill \stackrel{.}{\mathit{p}}\hfill \end{array}\right]=\mathit{q}\ast \left[\begin{array}{c}\hfill 0\hfill \\ \hfill 0\hfill \\ \hfill 0\hfill \\ \hfill u_1\hfill \end{array}\right]\ast {\mathit{q}}^{-1} \\ \stackrel{.}{\mathit{q}}=\frac{1}{2}\mathit{q}\ast \left[\begin{array}{c}\hfill 0\hfill \\ \hfill \omega \hfill \end{array}\right]=\frac{1}{2}\mathit{q}\ast \left[\begin{array}{c}\hfill 0\hfill \\ \hfill \kappa u_1\hfill \\ \hfill 0\hfill \\ \hfill u_2\hfill \end{array}\right], \end{gathered}$$ (2)

where the orientation of the needle tip is denoted by the unit quaternion q (using the convention that the scalar is the first component), and the * represents the quaternion multiplication operator. We will adopt this representation for the rest of our paper, and use it in our Extended Kalman Filter implementation.

Torsional Deadband-Aware Needle Model

In the needle model (2), the inputs u₁ and u₂ are the linear insertion speed and axial rotation speed of the needle tip, respectively. These are along and about the tip-frame z-axis, respectively, as shown in Figure 1. In the past, the needle has been assumed to be infinitely stiff axially and torsionally, meaning that velocities applied at the needle base are directly transmitted to the needle tip. Here, we continue to assume that the needle is axially stiff, such that the insertion speed at the base υ₁ is transferred exactly to the needle tip (u₁ = υ₁). However, we relax the assumption that the needle is infinitely torsionally stiff, and allow the needle’s tip angle to lag behind its base angle, to account for torsional frictional effects. In other words, we assume tip rotation and base rotation speeds, u₂ and υ₂ respectively, can differ. As mentioned in the introduction, such torsional dynamics have been previously modeled for steerable needles embedded in tissue [14, 16] using complex interaction models.

In this paper, we seek to determine whether the complex interactions between the needle and bronchoscope port, as well as needle-tissue interactions beyond the bronchoscope tip, can be modeled well enough for closed-loop control using a simple Coulomb friction model. Such a model implies a torsional deadband of fixed angular magnitude. Figure 3 illustrates the relationship between the base and tip of the needle, where the needle base rotation angle (as a function of time t) is denoted by θ_b(t), and the needle tip rotation angle is denoted by θ_t(t). These angles are given by integrating the rotation speeds at their corresponding locations:

$$\begin{gathered} {\theta }_b(t)={\int }_0^t{\upsilon }_2(\tau )d\tau \\ {\theta }_t(t)={\int }_0^t{u}_2(\tau )d\tau . \end{gathered}$$ (3)

Figure 3.

The deadband model with fixed magnitude 2c that incorporates the Coulomb friction model is presented. An example hysteresis curve relating the base rotation θ_b to the tip rotation θ_t around the deadband range is shown by the path following counterclockwise direction. The shaded region represents the entire deadband range, and extends to infinity.

In this deadband model, the input u₂ in the kinematic model (2) is a function of the tip angle θ_t(t), the actuator angle θ_b(t) and its derivative ${\upsilon }_2={\stackrel{.}{\theta }}_b$ as follows:

$$u_2={\upsilon }_{2}C({\theta }_t,{\theta }_b,{\upsilon }_2),$$ (4)

where the function C(θ_t, θ_b, υ₂) checks the conditions for deadband and returns 1 or 0 as described below:

$$C({\theta }_t,{\theta }_b,{\upsilon }_2)=\{\begin{array}{cc}1\hfill & \text{if}{\theta }_t\le {\theta }_b-c\wedge {\upsilon }_2>0\hfill \\ 1\hfill & \text{if}{\theta }_t\ge {\theta }_b+c\wedge {\upsilon }_2<0\hfill \\ 0\hfill & \text{otherwise}.\hfill \end{array}\phantom{\}}$$ (5)

We define 2c as the total width of the rotational deadband which depends on the frictional forces on the needle. We assume c to be a constant based on our Coulomb friction model, and it can be determined prior to steering.

Extended Kalman Filter with 5-DOF Magnetic Tracker

Our Extended Kalman Filter uses magnetic tracker information from an embedded 5-DOF coil at the needle tip, in combination with the torsional deadband-aware needle model described in the previous section. We use the following notation: ^ˆ denotes an estimated state, ^′ denotes a predicted state, and ^~ denotes a sensed quantity. The full 6-DOF pose of the needle tip is estimated by the EKF as $\widehat{\mathit{x}}=[{\widehat{\mathit{p}}}^T{\widehat{\mathit{q}}}^T{]}^T$, where $\widehat{\mathit{p}}$ denotes the estimated tip position and $\widehat{\mathit{q}}$ denotes the estimated unit quaternion representing the tip orientation. The actuation variables are taken as υ₁ (insertion) and υ₂ (rotation speed at the base, which affects u₂ through the deadband model). The torsion-aware needle model is used to predict the needle state based on the commanded actuation. The EKF prediction step is written as

$$\begin{gathered} {\mathit{x}}_n^{\prime }=f({\widehat{\mathit{x}}}_{n-1},{\upsilon }_1,{\upsilon }_2,\Delta t) \\ {\mathit{P}}_n^{\prime }={\mathit{J}}_{n-1}{\mathit{P}}_{n-1}{\mathit{J}}_{n-1}^T+\mathit{Q}, \end{gathered}$$ (6)

where ${\mathit{x}}_n^{\prime }$ and ${\mathit{P}}_n^{\prime }$ are the new predicted state and state covariance matrix, respectively. The variable n is used for discrete time. Δt is the time step between each prediction. f(·) is the torsional deadband aware needle model function which is computed by discretely integrating (2) using a single forward-Euler step over Δt, with u₁ = υ₁ and u₂ computed by the deadband model in (4) and (5). J is the 7x7 Jacobian matrix for the needle model, given by ${\phantom{\mid }{\mathit{J}}_{n-1}=\frac{\partial f}{\partial \mathit{x}}\mid }_{\mathit{x}={\widehat{\mathit{x}}}_{n-1}}$. The matrix Q is the covariance of the process noise.

Sensor information is then used to update the estimate. The 5-DOF information retrieved from the electromagnetic tracker consists of the position, ${\tilde{\mathit{p}}}_n$, and the needle bearing, which is the unit vector along the z-axis, ${\tilde{\mathit{v}}}_{z,n}$. The discrete derivative of this needle bearing vector provides a vector indicating the direction in which the needle is curving, which is along the negative y-axis of the needle as shown in Figure 1. With normalization of the derivative, y-axis unit vector is given by

$${\tilde{\mathit{v}}}_{y,n}=-\frac{{\tilde{\mathit{v}}}_{z,n}-{\tilde{\mathit{v}}}_{z,n-1}}{\mid {\tilde{\mathit{v}}}_{z,n}-{\tilde{\mathit{v}}}_{z,n-1}\mid }.$$ (7)

The sensing vector is then constructed as ${\tilde{\mathit{y}}}_n=[{\tilde{p}}_n^T{\tilde{\mathit{v}}}_{z,n}^T{\tilde{\mathit{v}}}_{y,n}^T{]}^T$. This is compared to the predicted value ${\mathit{y}}_n^{\prime }=h({\mathit{x}}_n^{\prime })$ to calculate the residuals as

$${\mathit{r}}_n={\tilde{\mathit{y}}}_n-{\mathit{y}}_n^{\prime },$$ (8)

where h(·) is the function relating the state variables to the measurement. The following Kalman equations are used to update the estimate based on the residuals:

$$\begin{gathered} {\mathit{S}}_n={\mathit{H}}_n{\mathit{P}}_n^{\prime }{\mathit{H}}_n^T+\mathit{Z} \\ {\mathit{K}}_n={\mathit{P}}_n^{\prime }{\mathit{H}}_n^T{\mathit{S}}_n^{-1} \\ {\widehat{\mathit{x}}}_n={\mathit{x}}_n^{\prime }+{\mathit{K}}_n{\mathit{r}}_n \\ {\mathit{P}}_n=(\mathit{I}-{\mathit{K}}_n{\mathit{H}}_n){\mathit{P}}_n^{\prime }, \end{gathered}$$ (9)

where S_n is the innovation covariance, K_n is a matrix containing the Kalman gains, and I is the identity matrix. H is the 9x7 Jacobian matrix for measurement function, defined by ${\phantom{\mid }{\mathit{H}}_n=\frac{\partial h}{\partial \mathit{x}}\mid }_{\mathit{x}={\mathit{x}}_n^{\prime }}$. The matrix Z is the sensor noise covariance matrix. The brand new state estimate ${\widehat{\mathit{x}}}_n$ is used to inform the controller of the needle tip position.

Needle Steering Control

We used the basic Sliding Mode Controller from [28] for needle steering. The current needle tip state estimate ${\widehat{\mathit{x}}}_n$ from the EKF and the desired point, p_d, are used to calculate the error vector in the needle tip frame, e:

$$\left[\begin{array}{c}\hfill 0\hfill \\ \hfill \mathit{e}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill 0\hfill \\ \hfill e_x\hfill \\ \hfill e_y\hfill \\ \hfill e_z\hfill \end{array}\right]={\widehat{\mathit{q}}}_n^{-1}\ast \left[\begin{array}{c}\hfill 0\hfill \\ \hfill {\mathit{p}}_d-{\widehat{\mathit{p}}}_n\hfill \end{array}\right]\ast {\widehat{\mathit{q}}}_n.$$ (10)

Using the components of e, angular error can be determined as

$$\sigma =\text{atan}2(e_x,-e_y),$$ (11)

where σ is the angular error between the desired curving direction and current estimated curving direction. Finally, the base insertion and rotation speeds are determined as

$$\begin{gathered} {\upsilon }_1={\lambda }_1 \\ {\upsilon }_2={\lambda }_2\text{sign}(\sigma ), \end{gathered}$$ (12)

where λ₁ and λ₂ are the pre-selected sliding mode controller parameters for insertion speed and rotation speed, respectively. The function sign() is the basic signum function that returns −1 when the input is negative, +1 when the input is positive, and 0 when the input is 0.

Note that this control strategy can be used to steer toward a single target point as well as along a desired trajectory. To follow trajectories, the desired point p_d for the controller is moved along the trajectory at the insertion speed. A short lead distance is used to ensure the desired point is ahead of the needle along the trajectory.

EXPERIMENTS & RESULTS

We validated our proposed method by performing needle steering experiments in gelatin and ex-vivo porcine lung tissue. For all experiments, we used the transbronchial robotic needle insertion system shown in Figure 4 which is described in [5], with an updated actuation unit described in [29], connected to a flexible bronchoscope (Ambu aScope™ 4, Ambu, Inc., USA) with an 81 cm long working channel.

Figure 4.

The robotic actuation unit, coupled with an Ambu aScope™4 bronchoscope.

The needle we used in the experiments was utilizing a flexure tip concept, originally proposed in [30], with a cutout design for the flexure element [31, 32]. The head of the needle was made from a superelastic Nitinol tube with an inner diameter (ID) of 0.61 mm and an outer diameter (OD) of 0.86 mm. We machined the flexure hinge and the bevel into a short length of this tube. The finished needle head was then glued to the needle shaft, a Nitinol tube with 0.41 mm ID and 0.53 mm OD. We incorporated a 5-DOF magnetic tracking sensor (Northern Digital Inc., Canada) through the inner lumen of the needle for real-time feedback during steering. The sensor coil was placed distal to the needle hinge and fixed with glue. We measured the distance between the glued sensor location and the actual needle tip, and the offset distance was accounted for in software. The needle was deployed into the tissue for steering through a plastic introduction tube that was passed through the bronchoscope. Note that there is some ambiguity in experiments associated with determining the initial torsional state within the deadband range, since the tip angle remains constant as the base angle begins to rotate, in that case. To overcome this in a practical setting, we simply rotated the needle in one direction before each insertion, to wind it up torsionally and cause it to start in a known torsional state.

The proposed state estimator and the controller were implemented using ROS (Robot Operating System). The control loop is operated at 40 Hz, which is the maximum update rate of the magnetic tracking system. The controller gains were selected to be λ₁ = 5.0 mm/s and λ₂ = 90 °/s. A lead distance of 3 mm was used for moving the desired point along trajectories in the controller. The sensor noise matrix for the Kalman Filter was chosen to be Z = diag([0.12 o_1x3 0.01 o_1x3 0.5 o_1x3]) based on the sensor data sheet and the observed noise from the sensor with our system, where o is a vector full of ones. The process noise was selected as Q = diag([0.001 o_1x3 0.0005 o_1x4]).

Steering in Phantom Tissue

As a preliminary validation of our proposed method, we performed steering experiments in 10% by weight Knox gelatin (Kraft Foods Global Inc., IL). The experimental setup for this experiment is shown in Figure 5.

Figure 5.

The experimental setup for gel experiments.

Prior to the experiments, the width of the deadband was calibrated as 2c = 30°. This was done by observing the needle tip rotation with a glued 6-DOF tracking coil at the tip, and comparing it to the base rotation while the needle was in the introduction tube deployed through the bronchoscope. A standard 5 French flexible medical catheter (Cordis, USA) was used as the introduction tube. We also calibrated the radius of curvature of the needle in gelatin as 1 /κ = 120 mm. The same gelatin recipe mentioned above was used for this procedure as well. All calibrated values were incorporated into our EKF and needle model.

We generated eight 60 mm circular trajectories leading to arbitrarily selected target points within the needle’s workspace to test targeting and trajectory following accuracy. The needle was steered to these target points, and the Kalman Filter state estimate data was recorded during the insertions, as shown in Figure 6. Raw 5-DOF magnetic tracker sensor data was also recorded during these insertions to compare with the EKF accuracy. As can be seen in Figure 6, the needle successfully steered to the desired target points with 1.36 mm final mean targeting error, shown in Table 1. The needle was able to follow the desired trajectories with an average deviation of 1.28 mm based on the EKF. Table 1 also shows that the tip accuracy based on EKF is agreement with the raw sensor accuracy, indicating that the EKF state estimates remain close to the physical needle tip positions.

Figure 6.

Needle tip paths and the desired trajectories in the gel experiments. Kalman filter tip positions are plotted.

Table 1.

Accuracy results for gel steering experiments. Both EKF and raw magnetic tracker sensor based needle tip positions were recorded for each insertion, and they are used to calculate the steering errors. The distance between the target and the needle tip position at the end of the insertion is given as the final targeting error. The trajectory error is calculated as the average of the needle tip distances to the desired trajectory throughout the steer.

Insertion Number	Final Targeting Error (mm)		Mean Trajectory Following Error (mm)
	EKF Estimate	Raw Sensor	EKF Estimate	Raw Sensor
1	1.17	0.53	0.98	1.05
2	0.44	0.34	1.13	1.12
3	1.17	2.05	1.33	1.39
4	1.34	1.38	1.65	1.76
5	1.95	1.91	1.00	1.26
6	1.88	1.54	1.71	1.75
7	1.95	2.08	1.35	1.36
8	0.94	1.44	1.10	1.10
Mean	1.36	1.41	1.28	1.35

Ex-vivo Lung Steering

To further test our methods in a clinical scenario, we performed steering experiments in ex-vivo porcine lungs with the experimental setup shown in Figure 7. The ex-vivo lungs were statically inflated with an air compressor. A pre-operative Computed Tomography (CT) scan (xCAT, Xoran Technologies, USA) was taken at the beginning of the experiments to identify lung anatomy and to determine target points. Airways and vasculature were segmented using the method of [33]. Fiducial markers were used to register the electromagnetic tracker to the CT scanner using point-based registration.

Figure 7.

The experimental setup for ex-vivo lung experiments.

After performing registration, we deployed the bronchoscope into the airways and used a 21 gauge trans-bronchial needle (eXcelon, Boston Scientific Inc.) to pierce through the wall of the airway. We then slid a plastic sheath over this needle, and then removed the needle from the sheath. Next, we deployed our steerable needle through the plastic sheath. The steerable needle consisted of a coaxial solid flexure tip needle surrounded by a nitinol cannula, which supported the needle between the bronchoscope tip and the bronchial wall.

Once the needle was in place, we selected a target point in the needle’s workspace. We then used a motion planner to generate a trajectory for the steerable needle from its current pose to this target while avoiding the obstacles identified by the segmentation, including vasculature, airways, and the pleura [33, 34]. Needle steering was initiated along this trajectory, and stopped when the needle reached the target point. In order to rapidly pass through the deadband during insertions, the needle’s insertion velocity was reduced by a factor of 2 and the needle’s axial rotation velocity was increased by a factor of 2.

The targeting results in ex-vivo lung are shown in Table 2. Figure 8 shows one of the insertions. As can be seen from these results, the needle was able to follow the desired trajectories with good accuracy.

Table 2.

Accuracy results for ex-vivo lung experiments. The needle tip positions obtained from our EKF and raw magnetic tracker sensor readings were both recorded for each insertion, and they were used to calculate the errors. The Final Targeting Errors are the distances between the needle tip position and the target, at the end of the insertion. The average of the needle tip distances to the desired trajectory throughout the insertion are reported as the Mean Trajectory Following Error.

Insertion Number	Final Targeting Error (mm)		Mean Trajectory Following Error (mm)
	EKF Estimate	Raw Sensor	EKF Estimate	Raw Sensor
1	0.34	0.69	1.00	1.61
2	3.33	3.58	1.20	1.30
Mean	1.84	2.14	1.10	1.46

Figure 8.

An example needle insertion from an ex-vivo experiment. The recorded needle tip position estimates and the planned trajectory are shown with respect to segmented anatomy. The airways are highlighted in blue, with major blood vessels in green.

CONCLUSION

This paper focuses on the challenge of controlling the path of a long, thin steerable needle delivered through the working channel in a bronchoscope. We sought to determine whether a simple Coulomb friction model (implying a torsional deadband) can be used to model the complex interactions between the needle and the bronchoscope port sufficiently for closed-loop control, acting on only 5-DOF electromagnetic tracking feedback. We found that, in combination with an Extended Kalman Filter and an established sliding mode controller, this approach enables accurate steering along desired trajectories to specified targets in phantom tissue and in inflated ex-vivo porcine lung. In future work, we plan to perform additional ex-vivo experiments and eventually use this approach in-vivo in porcine lungs. As we move toward these experiments we will explore whether the coaxial cannula used in our ex-vivo experiments is necessary or whether the needle’s overall sized can be reduced by omitting it. We will also explore the best nominal speeds for insertion and axial rotation, and whether speed variation is needed within the deadband. Furthermore, as we move toward in-vivo experiments, the system will require accurate image registration approaches as well as the potential need to compensation for respiratory motion. However, these challenges have been overcome in prior image guidance and robotic needle insertion studies so they do not seem insurmountable. Thus, this approach paves the way for accurate control of smaller diameter needles that do not hinder the bending capabilities of the tendon actuated bronchoscope tip, and reduce invasiveness, compared to the larger diameters used in the past to accommodate 6-DOF tracking coils.