Real-time Needle Steering in Response to Rolling Vein Deformation by a 9-DOF Image-Guided Autonomous Venipuncture Robot

Abstract

Venipuncture is the most common invasive medical procedure performed in the United States and the number one cause of hospital injury. Failure rates are particularly high in pediatric and elderly patients, whose veins tend to deform, move, or roll as the needle is introduced. To improve venipuncture accuracy in challenging patient populations, we have developed a portable device that autonomously servos a needle into a suitable vein under image guidance. The device operates in real time, combining near-infrared and ultrasound imaging, computer vision software, and a 9 degrees-of-freedom robot that servos the needle. In this paper, we present the kinematic and mechanical design of the latest generation robot. We then investigate in silico and in vitro the mechanics of vessel rolling and deformation in response to needle insertions performed by the robot. Finally, we demonstrate how the robot can make real-time adjustments under ultrasound image guidance to compensate for subtle vessel motions during venipuncture.

I. Introduction

The process of placing a cannula in a peripheral forearm vein to withdraw a blood sample or deliver intravenous (IV) fluid is the most ubiquitous clinical routine practiced in medicine [1]. Today, venipuncture remains a manual procedure, where clinicians visually locate and feel for a suitable vein (e.g. in the medial antecubital region of the inner elbow) and then insert a needle aiming to reach the center of the vein. Oftentimes it is challenging to visualize a suitable venipuncture site, particularly in patients with dark skin or a high body mass index. After a vein is identified, the process of inserting the cannula into the vessel can also be difficult. Failed needlestick attempts frequently occur due to inaccurate estimations of vein depth that cause the needle tip to pierce through the back of the vein wall.

Failed attempts are also common in pediatric and elderly patients with small and fragile veins [2]. When introducing the needle into such veins, the mechanical forces generated by the needle tip may cause the vein to deform, move, or roll, and this motion may lead to missed insertion attempts. In such cases, the clinician is required to make subtle real-time adjustments to the needle orientation. However, this manual needle steering process requires complex visuomotor skills and clinical experience. Poorly introduced needles may result in complications such as increased pain, internal bleeding, or leakage of IV fluids into the extravascular tissue.

In recent years, various imaging technologies based on near-infrared (NIR) light or ultrasound (US) have been introduced to aid clinicians in visualizing veins. However, recent studies have indicated that these devices do not improve first-stick success rates significantly compared to manual techniques [3]. Meanwhile, surgical robots that automate needle insertion procedures (e.g. brachytherapy, biopsies, and ablation therapy) are comparatively large and expensive, and these systems have not been designed for venous access [4].

To improve the rate of venipuncture insertion accuracy in difficult patients, our group has developed a portable device that autonomously servos a needle into a suitable peripheral forearm vein under image- and force-based guidance [5], [6]. The device combines NIR and US imaging to provide clear visualization of the vasculature. The device then performs a sequence of image analysis procedures, including vessel segmentation, reconstruction, and tracking, to continuously locate the 3D spatial coordinates of a selected vein in real-time. Finally, the 3D coordinates are directed to a robotic system, which guides a cannula into the center of the vein under real-time closed-loop US image feedback. Previous generations of the device have been validated in human imaging trials and in vitro bench tests. In these studies, the device has been shown to significantly improve the visualization of subcutaneous veins compared to manual techniques such as mechanical palpation or landmark-based identification; additionally, improved cannulation accuracy and procedure completion time has been demonstrated in commercial phlebotomy models and gelatin phantoms.

Nevertheless, while the earlier generations of the device were successful in demonstrating the feasibility of automated venous access, three important robotic limitations were observed. First, the previous devices lacked the ability to align both the needle manipulator and the US transducer along the longitudinal axis of the vein. Thus, while experiments could be conducted on test models because the veins were in line with the manipulator, successful cannulation of human peripheral vasculature would have been difficult. Secondly, the previous devices lacked a radial degree of rotation, which prevented lateral veins at the sides of the forearm to be reached. Instead, the patient would have had to reorient their arm so that the lateral veins faced the device. Third, because of the missing degrees of rotation described above, the previous devices were unable to make fine adjustments to the orientation of the needle, which is necessary to compensate for vessel rolling and tissue deformation.

This paper describes the most recent design of the automated venipuncture device, which addresses the above limitations by incorporating added degrees-of-freedom (DOF) and improved kinematic controls. These changes significantly extend the operating workspace of the device and enable the device to perform real-time adjustments of the needle pose in response to vessel motions during cannulation. The paper is organized as follows: Sec. II discusses the key sub-systems of the device with a focus on the mechanical and kinematic design of the robot. Sec. III describes the analytic and numerical models implemented to study the vessel mechanical responses to needle insertion. The results of the computational approach is then compared to those of in vitro robotic cannulation experiments performed on phantoms. Finally, Sec. IV concludes the paper and outlines future work.

II. Device Design & Development

A. Robotic Venipuncture Process Flow

Several key components of the system design, including the robot registration relative to the NIR and US imaging coordinate frames, the real-time robotic servoing under US-guided feedback, and the overall workflow in the clinical setting, have been previously described [5], [6]. To perform a venous blood draw using the device, the clinical protocol begins with the practitioner swabbing the patient’s forearm with alcohol, applying a tourniquet proximal to the elbow, and placing the arm within the device. The device identifies suitable cannulation sites by coarsely scanning the forearm using 940 nm NIR imaging, which enhances the contrast of blood vessels due to increased differential hemoglobin absorption and reduced optical scatter [5]. The 3D geometry of the vessels is reconstructed in real-time using GPU-accelerated stereovision software. The vessels are then segmented, branch points in the vasculature are detected, and potential cannulation sites are scored based on vessel length, diameter, and continuity. The final cannulation site is chosen by the clinician, and the 3D spatial coordinates of the selected site are directed to the robot component.

To insert the needle, the robot uses a 6-DOF base positioning system to bring a Doppler US transducer and needle manipulator over the chosen site, aligning the transducer and manipulator with the vein in the longitudinal and radial directions (α and ϕ respectively as shown in Fig. 1) based on inverse kinematics. US imaging provides a high-magnification, longitudinal view of the vessel and confirms the presence of blood flow. The US system is further used to locate and visualize the cannula as it pierces the vein, confirming that the needle tip reaches the center of the vessel for blood collection. The US image coordinates are registered to the robot coordinates by pre-calibrating the US image using a fudicial phantom [6].

Fig. 1.

(a) CAD render of the 9-DOF venipuncture robot, showing the 6-DOF base positioning system, 3-DOF needle manipulator, and the NIR/US imaging systems; (b) detailed CAD model of the needle manipulator.

Furthermore, the force sensor on the needle manipulator detects when the cannula has pierced the top skin layer as well as the vein wall, thus providing another mechanism for computing the position of the needle tip relative to the target vein. After the procedure, the robot withdraws and disposes the cannula. Throughout the procedure, the practitioner has no contact with exposed sharps, thus eliminating the risk of accidental needle stick injuries.

B. Robotic Design

The two major robotic sub-systems were designed as modular entities and are described below:

1) Base Positioning System

The base positioning system includes 6-DOF—three prismatic joints and three revolute joints coupled together via aluminum brackets. The prismatic joints form a Cartesian positioner (x_r, y_r, z_r as depicted in Fig. 1) actuated by precision lead screw linear stages (LSM Series, Zaber). The three revolute joints (α, ϕ, and β), which allow the US transducer and needle manipulator to be aligned with the vessel orientation, are actuated by anti-backlash, worm gear-based rotary stages (3MR and 4MR, National Aperture Inc.). The lead screw linear stages and worm gear rotary stages were selected over other common power transmission gearing like timing belts and pulleys because of their high accuracy and repeatability. Additionally, when the power is off, the lead screws and worm gears ensure that all joints self-lock, maintaining safety for both the patient and clinician.

2) Needle Manipulator

The needle manipulator’s function is to guide the cannula into the center of the vein, with the ability to make real-time adjustments to the needle pose and trajectory. Previously, the manipulator consisted of 4-DOF serial arm [6]. Although the serial arm had a large operating workspace, joint stability was compromised due to the use of direct drive power transmission and due to the large distance (i.e. >10 cm) between the manipulator’s base joint and the insertion target. Furthermore, the insertion angle could not be varied and instead was dictated by the inverse kinematics. Finally, force sensing and needle attachment mechanisms had not been incorporated in the previous system. To address these problems, we revised our design criteria for the needle manipulator as follows:

1. The manipulator should have the accuracy to cannulate ∅2–4 mm veins.

2. The manipulator should be able to adjust the insertion angle of the needle from 0–30°.

3. The manipulator should be able to insert the needle with >5 N of force, with force sensing capabilities.

4. The manipulator should have an attach and release mechanism to automate the needle handling process.

The resulting design consists of a highly miniaturized 3-DOF manipulator (Fig. 1b) with enhanced stability, force sensing, and the ability to vary the cannula insertion angle. Here, a micro lead screw based linear stage (3M-FOS, National Aperture Inc) was implemented to adjust the height of the needle. When combined with a DC brushed servo (RE-10, Maxon Motors) that sets the insertion angle and a linear spindle drive (RE-8, Maxon Motors), the manipulator is able to precisely cannulate the vessel of interest at depths ranging from 1–20 mm with the ability to set the injection angle at each height.

An electromagnetic within the end-effector allows the manipulator to automatically load and release the needle without human intervention. Here, the needle is pre-packaged with a plastic needle clip containing a 1 cm diameter steel target. Before the procedure, the needle and pre-attached clip are loaded onto the device. The manipulator is then able to position itself over the clip and secure the needle by activating the electromagnet. After the procedure, the needle is disposed directly into a sharps container by deactivating the electromagnet.

C. Robot Kinematics and Workspace Analysis

The forward and inverse kinematic equations for the 6-DOF base positioning system and the 3-DOF needle manipulator were derived using a standard Denavit-Hartenberg (DH) convention [7] (Table I). For the base positioning system, the parameters specified in the DH table link the origin frame at the base of the system to the wrist, which is coupled to the US transducer. Meanwhile, the DH parameters for the needle manipulator link the manipulator origin frame, with a constant offset from the US transducer frame, to the distal end of the end-effector. The needle tip position is calculated using a wrist-to-tool transform.

TABLE I.

DH Parameters of the 9-DOF robotic system. Joints: 1–7 for the base; joints: 8–11 for the manipulator.

i	Type	αi−1	ai−1	di	θi
1	Revolute, β	0	0	0	0
2	Prismatic, zr	90°	0	0	0
3	Prismatic, xr	90°	0	0	0
4	Prismatic, yr	90°	0	0	0
5	Revolute, ϕ	−90°	0	0	0
6	Revolute, α	−90°	0	40 mm	0
7	US probe offset, p	0	0	95 mm	0
8	Prismatic, zm	90°	0	0	0
9	Revolute, θ	90°	0	0	0
10	Prismatic, xm	0	0	25 mm	0
11	Needle, n	0	0	−56 mm	0

To ensure that the volume of reachable workspace was sufficient for the needle insertion task (≥175 cm³ as governed by a clinical survey [5]), we computed the overall operating work envelope of the 9-DOF robot based on the kinematic equations and the limits of motion for each link. Briefly, the travel range of the x_r, y_r, and z_r prismatic joints provided a base rectangular work volume of 10×10×7.5 cm. Meanwhile, the rotational range of the revolute joints (α: ±60° ϕ: ±45° and β: ±90°) provided an additional hemispherical envelope of ~90 cm³ from any given position within the base work volume. Finally, the needle manipulator added a planar ~11.5 cm² range of motion lying parallel with the axis of insertion. In total, the operating workspace allows the robot to cannulate any vein in the anterior forearm (Fig. 2), spanning from the wrist to the antecubital fossa region of the elbow. Additionally, unlike earlier generations, the current robot is able to align with significantly angled or varicose veins. The system is also able to reach lateral veins located along the sides of the forearm.

Fig. 2.

(a) Dexterous hemispherical workspace (~90 cm³) of the needle manipulator highlighted in gray over the forearm. The manipulator workspace may be positioned anywhere within a 10×10×7.5 cm base rectangular work volume. (b) isometric, (c) top, and (d) side view of the US probe-needle unit trajectory (blue line) during the robotic venipuncture.

III. Experimental Methods & Results

A. Computational Models of Vessel Rolling and Deformation

To simulate the mechanics of needle insertion into a blood vessel, we first modeled the vein as a thin-walled cylinder beam (elastic modulus, E_v = 10 MPa; outer diameter, d_o = 3.2 mm, inner diameter d_i = 2.0 mm, length l_v = 80 mm), and calculated displacements based on applying a point load on-axis to cause beam deflection (Fig. 3a). Supporting the vein underneath is a distributed load representing the surrounding tissue. Because we were interested in the behavior of the vein prior to the needle piercing the wall, the load was maintained below the minimum puncture force for venous access. Specifically, an applied load of P = 0.5 N was introduced at an angle of 15°. Accounting for the distributed load underneath (ω = 0.004 N/mm), the resulting point load in the axial direction was 0.2 N. Using (1),

Fig. 3.

(a) Theoretical beam bending model, (b) SolidWorks simulation, and (c) experimental phantom cannulation setup.

$$I_{yy}=I_{zz}=\pi \left(d_o^4-d_i^4\right)/64$$ (1)

where I is the area moment of inertia in y and z, the resulting maximum displacement d_max can be calculated from (2),

$${\delta }_{\mathit{max}}=\frac{P{l}^3}{48EI}$$ (2)

Next, to account for the nonlinear behavior of the tissue caused by on- and off-axis loads during needle insertion [8], we implemented finite element (FE) models of a typical adult median cubital vein and a dermis/hypodermis support layer (8×3×1 cm rectangular volume). Table II summarizes the mechanical properties of the FE models, which were based on literature references of human skin and vessel mechanical properties [9], [10]. Three different simulations were implemented the SolidWorks Simulation software package (Fig. 3b). In the first simulation, the needle was aligned with the medial axis of the vein. Here, only downward vessel displacements were expected. This condition is representative of an ideal venipuncture. In the second simulation, the needle was positioned off-axis. Slip was allowed at the contact interface between the needle and the vein. This condition, where the vein slides away from the needle tip immediately upon contact, is similarly observed in patients with poorly anchored or rolling veins. The needle was likewise positioned off-axis in the third simulation. However, a no-slip condition was enforced between the needle and the vein. In this condition, which may be observed in patients with hardened veins, high strains are exerted on the vessel wall that may lead to significant deformation or vessel rupture [11]. In all simulations, the needle gauge was set at 23G, the angle of insertion at 15°, and the maximum insertion distance at 10 mm. As in the beam deflection model, the applied loads were maintained below the minimum puncture force.

TABLE II.

Mechanical properties used for vein modeling and simulations. Values based on literature references of human tissue mechanical properties [9], [10]

Tissue Layer	Material Parameter	Unit	Finite Element Model	In Vitro Phantom Model
Vein	Material	–	Elastin	Silicone
	Outer vein diameter, dv	mm	3.2	3.2
	Wall thickness, tv	mm	0.55	0.6
	Vein length, lv	mm	80	80
	Elastic modulus, Ev	MPa	10	9.5
	Shear modulus, Gv	MPa	5.2	–
	Poisson ratio, νv	–	0.48	0.48
	Compression strength, σcv	MPa	20	–
	Yield strength, σyv	MPa	3.1	3.8
	Tensile strength, σtv	MPa	9	9.3
Dermis, Hypodermis	Material	–	Adipose	Gelatin
	Dimensions, lt, wt, ht	cm	8×3×1	8×3×1
	Elastic modulus, Et	MPa	1	1
	Shear modulus, Gt	MPa	0.5	–
	Poisson ratio, νt	–	0.48	0.49
	Compression strength, σct	MPa	0.75	1
	Yield strength, σyt	MPa	–	–
	Tensile strength, σtt	MPa	–	–

B. In Vitro Evaluation of Vessel Rolling and Deformation

In a final set of experiments, robot cannulation trials were conducted in vitro to investigate vessel deformation and device accuracy. Here, phantom models with mechanical properties similar to those of the human forearm were utilized (Fig. 3c). To keep consistent with the geometry of the FE models, we fabricated each phantom within an 8×3×1 cm container. Like the FE models, each phantom comprised a surrogate vein (Dow-Corning Silastic^® silicone tubing) with the bottom half embedded in support tissue. The support tissue was formed from porcine gelatin, whose mechanical properties can be controlled to resemble those of the dermis and hypodermis. For example, a support tissue with elastic modulus E ≅ 50 kPa (concentration 20% m/v) may be used to model mechanically rigid tissues such as dermis. In contrast, the phantoms used in this study (Table II) were significantly more compliant (E ≅ 0.5 kPa, concentration 2.5% m/v) to mimick adipose tissue. Here, rather than provide support, the tissue would more likely deform upon needle contact and thereby allow the vein to roll.

Phantoms were constructed to investigate the three needle insertion conditions simulated earlier, namely: (1) the on-axis condition; (2) the off-axis slip condition; and (3) the off-axis no-slip condition. In a first set of trials, vessel deformation patterns predicted by the simulations were compared with experimental observations of the three conditions. To ensure that the mechanical properties of the phantom would be affected only by the applied load of the needle, we did not utilize US imaging here. Instead, the robot kinematics were computed directly from the 3D coordinates provided by the stereo NIR imaging. Furthermore, the computational and experimental insertion parameters were kept consistent. That is, the needle was inserted at a constant 15° angle with a 10 mm travel distance and speed of 10 mm/s. Once the 3D coordinate of the vein target was identified, the robotic joints were locked; as such, the linear needle trajectory was unchanged after the cannulation was initiated.

In contrast to the comparative studies, the second set of experiments assessed the real-time image-guided kinematic controls of the robot during insertion. Here, the US imaging system was fully utilized to locate and track the center of the vein in each image frame. Additionally, the robot was allowed to adjust the position and orientation of the needle tip based on the image feedback. In this way, we were able to evaluate the robot’s ability to guide the needle into the vein in the presence of vein movement and deformation.

C. Results

Fig. 4 shows time-lapse images comparing the FE simulations and experimentally observed vessel deformation patterns for the on-axis, off-axis slip, and off-axis no-slip cannulation conditions. The vessel deflection predicted by the beam bending model for the on-axis applied load is also shown (Fig. 4a, solid black line) with maximum displacement d_max = 12.95 mm. For the on-axis FE and experimental trials, (Fig. 4a), vessel displacement was observed to occur primarily in the longitudinal (x) and vertical (z) directions, whereas minimal vessel displacement was observed in the lateral (y) direction. On the contrary, significant lateral (y) vessel displacement was observed when the needle was introduced off-axis. In the slip condition (Fig. 4b), the asymmetrically distributed forces applied to the vein side wall caused the vein to roll under and away from the axis of insertion shortly after initial contact with the vein. Finally, in the no-slip simulation, significant deformations occurred along the distal portions of the vein. In the simulations, the applied stress on the vein side wall was found to exceed the material yield stress after the needle was inserted 5 mm (Fig. 4c). Similarly, in the phantom experiment, the vessel side wall was observed to rupture shortly after 5 mm of insertion.

Fig. 4.

Time-lapse images comparing the simulated (left) and experimentally observed (right) vessel deformation patterns for (a) on-axis, (b) off-axis slip, and (c) off-axis no-slip cannulation conditions. Red arrows indicate the needle tip position and total needle insertion distance along the axis of cannulation. Vessel displacement d_v (mm) is also shown via the color map. Insertion parameters between the simulations and experiments were kept consistent, e.g. the needle was inserted at a constant 15° angle with a 10 mm maximum travel distance and speed of 10 mm/s.

Longitudinal B-mode US images of needle insertion and vessel deformation are shown in Fig. 5. The adjustment of needle orientation in response to the moving vein target can be observed. Here, insertion distance within the field-of-view of the US image was 5 mm (about half the total insertion distance). Likewise, the duration in which the insertion was under US guidance was 0.5 s. In that time, the insertion angle was lowered from its starting angle of θ = 25° (Fig. 4a) to θ = 18° upon first contact with the vessel wall (Fig. 4b), and finally to θ = 10° to allow the needle to be guided into the vessel lumen (Fig. 4c) following an exponential decay trajectory curve. Starting with a steeper angle reduces the amount of dermal tissue punctured by the needle and reduces the axial insertion force at the needle tip. And ending with a shallow angle reduces the likelihood that the needle tip pierces through the other side of the vein. In these studies, the compressive force of the US transducer against the phantoms helped to reduce the extent of vein movement, suggesting that it may be possible to utilize the US tranducer to anchor the vein during the cannulation.

Fig. 5.

Real-time longitudinal B-mode US images showing robotic needle guidance and vessel deformation in a phantom. The adjustment of the needle orientation in response to the moving vein target can be observed. The insertion angle was lowered from its starting angle of θ = 25° (a) to θ = 18° upon first contact with the vessel wall (b), and finally to θ = 10° to allow the needle to be guided into the vessel lumen (c).

IV. Conclusion

In this paper, we presented the development of a 9-DOF robot for automated venipuncture and investigated vessel rolling and deformation during robotic needle insertion. We also demonstrated how the robot can compensate for vessel motion by steering the needle in real-time under image guidance. Future work will focus on refining the FE and phantom models to simulate varying human physiological and demographic characteristics.

The improved models may then be used to study other factors that affect venipuncture accuracy, such as insertion angle, insertion speed, vein depth, and vein diameter. In addition, the FE models will be expanded to simulate the actual penetration of the tissue by the needle. This will allow us to compare experimentally observed force profiles with theoretical solutions, and to validate the force sensing component of the robot.