Force-based Safe Vein Cannulation in Robot-assisted Retinal Surgery: A Preliminary Study

Abstract

Retinal vein cannulation (RVC) is a potential treatment for retinal vein occlusion (RVO). Manual surgery has limitations in RVC due to extremely small vessels and instruments involved, as well as the presence of physiological hand tremor. Robot-assisted retinal surgery may be a better approach to smooth and accurate instrument manipulation during this procedure. Motion of the retina and cornea related to heartbeat may be associated with unexpected forces between the tool and eyeball. In this paper, we propose a force-based control strategy to automatically compensate for the movement of the retina maintaining the tip force and sclera force in a predetermined small range. A dual force-sensing tool is used to monitor the tip force, sclera force and tool insertion depth, which will be used to derive a desired joint velocity for the robot via a modified admittance controller. Then the tool is manipulated to compensate for the movement of the retina as well as reduce the tip force and sclera force. Quantitative experiments are conducted to verify the efficacy of the control strategy and a user study is also conducted by a retinal surgeon to demonstrate the advantages of our automatic compensation approach.

I. Introduction

Retinal vein occlusion (RVO) is a blockage of the retinal veins resulting in visual impairment generally in middle-aged and elderly people [1]. More than 16 million adults are estimated to be affected [2]. Retinal vein cannulation (RVC) is proposed as a treatment method in [3]. In RVC, a surgeon inserts a hollow needle (also named as cannula) into the occluded vessel and then injects clot-dissolving agents (see Figure 1). The dimensions of the vessels (< ϕ 130 μm) and needles (ϕ 30-70 μm) are extremely small in such a procedure [4]. In addition, physiological tremor and patient movement during retinal microsurgery are measured over 100 μm [5], which severely hamper precise needle insertion and steady holding the cannula during the injection.

Fig. 1:

In RVC, a cannula is inserted via the scleral port into the occluded retinal vein and then a clot-disolving drug is injected to dissolve the blockage causing RVO.

Teleoperated, handheld and cooperatively controlled robots have been developed in the last decades [6] to overcome the aforementioned limitations. The stereotaxical microtelemanipulator for ocular surgery (SMOS) is the first teleoperated system to adopt a circular-track remote-center-of-motion (RCM) mechanism with linear and rotational motion for the instrument [7], [8]. PRECEYES surgical system is the most mature robot adopting a linkage-based RCM mechanism developed by TU Eindhoven [9], [10]. The system from TU Munich implemented the RCM in software based on piezoelectric stick-slip actuators that had a very compact structure [11], [12]. Micron is a handheld robot system from CMU that uses a Stewart-platform parallel mechanism driven by piezoelectric motors [13], [14]. All unintentional motion including the hand’s tremor can be canceled via the control system. An example of a cooperatively controlled robot is the Steady-Hand Eye Robot (SHER) developed here at JHU [15], [16]. In SHER, the force applied on the tool-handle is measured by a force sensor that is used to control the velocity of the robot. The first in-vivo robot-assisted RVC interventions on humans have been done by a cooperatively controlled system of KU Leuven [17]. Moreover, the Fiber Brag Grating (FBG)-based force-sensing tool was further developed for this robotic system which can detect two different interaction forces: 1) the force between the retina and the tool tip and, 2) the force between the sclerotomy and tool shaft [18].

Smooth and accurate instrument manipulation is achieved via the aforementioned robotic systems. However, micron level motion of the retina and cornea related to heartbeat [19] may generate unexpected forces between the instrument and eyeball (both the tip and sclera force). Auditory and haptic feedbacks have been deployed to help surgeons enhance awareness of forces exceeding safe boundaries [20], [21]. The limitations of the audio feedback based studies are that the efficacy of reducing the force is highly dependent on the surgeons’ reaction and the extra feedback may diminish the concentration effort of surgeons during surgery. In the recent studies, a RNN-based control framework [22] and a adaptive control framework [23], [24] were developed to actively reduce the interaction force. However, only the sclera force was concerned in these methods.

For safe robot-assisted retinal surgery, detecting these motions and allowing automatic compensation to avoid or at least reduce potential tissue damage is desired. In this paper, we consider the case of the movable retina with stationary sclera. A control algorithm based on active compensation of the needle tip position is proposed to minimize the interaction force between the tool tip and retina, while the sclera force is monitored and kept in a predetermined small range. When the force at the tool tip is over the threshold, the robot produces translational velocities along the x and y directions of the tip frame to compensate the force and to also produce translational velocities along related axes of the sclera frame to keep the sclera force in a predetermined small range. This control system will be verified on the SHER.

The rest of the paper is organized as follows: In Section II, we describe the control strategy of automatic force compensation. In Section III, we set experiments to verify the efficacy of our control strategy. Conclusions and future work are presented in Section IV.

II. Control Strategy

In this section, we describe the control strategy adopted to automatically minimize the tip force and keep the sclera force (see Fig. 2) in a predetermined small range. Our control algorithm is implemented on SHER 2.1 (see Fig. 2 (a)) and a dual-force-sensing cannula (see Fig. 2 (b)) is used to measure the tip and sclera force.

Fig. 2:

Johns Hopkins Steady-Hand Eye Robot [16]. (a) Overall view of SHER 2.1. (b) The description of the x and y components of the sclera force and tip force in the sclera frame {s} and tip frame {t}. The tool insertion depth is denoted as d.

SHER 2.1 is the latest generation of retinal robot assistant developed at the Johns Hopkins University. This robot has 5 degrees of freedom (DoF) and consists of: 1) XYZ linear stages, 2) a rotary stage and 3) a symmetric tilting mechanism with mechanical RCM. A 6 DoF force/torque sensor is mounted between the tilting mechanism and the tool holder that is used to produce input signals for robot manipulation. The dual force-sensing tool (cannula) can be mounted on a tool holder via a quick-release mechanism. Three Fiber Bragg Grating (FBG) fibers are integrated on the flexible tool shaft to measure the x and y components of both the sclera force (i.e. F_sx and F_sy) and tip force (i.e. F_tx and F_ty), and also the insertion depth d of the tool [18]. We assume that the torque at the sclera and friction force along the tool axis at the incision/cannula are negligible. The norm of the sclera force and the tip force are denoted as $F_s=\sqrt{{F_{\mathit{sx}}}^2+{F_{\mathit{sy}}}^2}$ and $F_t=\sqrt{{F_{\mathit{tx}}}^2+{F_{\mathit{ty}}}^2}$.

The sclera frame {s} is attached on the shaft of the cannula and its origin is coincident with the scleral port, and the tip frame {t} is attached on the microneedle and its origin is coincident with the insertion port of the occluded vessel (see Fig. 2 (b)). Then the x and y components of the sclera force and tip force are defined along the related axes of the sclera frame {s} and the tip frame {t}. The tool insertion depth d is the distance between the origin of the frame {s} and {t}. Our controller is designed based on the admittance control law and uses a sinusoidal curve as a mapping function between the force and the velocity to ensure that the velocity could change smoothly and continuously with respect to the force.

First, we set a controller to define the desired tip velocity aimed at compensating for the movement of the vessel and then reduce the interaction force. In this paper we mainly focus on the movement of the retina on the horizontal plane, so compensations are applied along the x and y directions of the tip frame and sclera frame. The desired x and y tip velocities in the tool tip frame {t} are set to

$$v_{ti}=\{\begin{array}{l}sign\left(F_{ti}\right)k_{t1}sin\left(k_{t2}\delta F_{ti}\right),0\le \delta F_{ti}<\widehat{\delta F_{ti}},\\ sign\left(F_{ti}\right)\widehat{\delta F_{ti}},\delta F_{ti}\ge \widehat{\delta F_{ti}},\end{array}$$ (1)

$$\delta F_{ti}=\{\begin{array}{l}\left|F_{ti}\right|-F_{tith},\left|F_{ti}\right|>F_{tith},\\ 0,\left|F_{ti}\right|\le F_{tith}\end{array}$$ (2)

in order to compensate the exceeding tip forces, where i = x, y. F_ti is the i component of the tip force, k_t1 and k_t2 are the coefficients of the mapping function between the tip force and the tip velocity, δF_ti occurs only when the applied force is larger than the predetermined threshold F_tith (see (2)), and $\widehat{\delta F_{ti}}$ is the maximum value of δF_ti over which the velocity will be a constant value. k_t1 is set as $\pi /\left(2\widehat{\delta F_{ti}}\right)$ and k_t2 is set as $\widehat{\delta F_{ti}}$. Within this paper, the force unit is mN and the velocity unit is mm/s.

Secondly, in order to keep the sclera force in a predetermined small range, x and y components of the sclera velocity expressed in the sclera frame {s} are set to be produced based on the sclera force components as follows:

$$v_{si}=\{\begin{array}{l}sign\left(F_{sj}\right)k_{s1}sin\left(k_{s2}\delta F_{sj}\right),0\le \delta F_{sj}<\widehat{\delta F_{sj}},\\ sign\left(F_{sj}\right)\widehat{\delta F_{sj}},\delta F_{sj}\ge \widehat{\delta F_{sj}},\end{array}$$ (3)

$$\delta F_{sj}=\{\begin{array}{l}\left|F_{sj}\right|-F_{sjth},\left|F_{sj}\right|>F_{sjth},\\ 0,\left|F_{sj}\right|\le F_{sjth},\end{array}$$ (4)

where j = x, y. F_sj is the j component of the sclera force, k_s1 and k_s2 are the coefficients of the mapping function between the sclera force and the sclera desired velocity, δF_sj occurs only when the applied force is larger than the predetermined threshold F_sjth (see (4)), and $\widehat{\delta F_{sj}}$ is the maximum value of δF_sj over which the velocity will be a constant value. k_s1 is set as $\pi /\left(2\widehat{\delta F_{sj}}\right)$ and k_s2 is set as $\widehat{\delta F_{sj}}$.

To let the cannula rotate around the sceral port (i.e. RCM), these tip velocities v_sx and v_sy are transmitted to the angular velocity on the sclera frame {s} (see Fig. 4), which are shown as follows:

$$w_{sx}=\frac{\left(v_{sy}-v_{ty}\right)}{d},$$ (5)

$$w_{sy}=\frac{\left(v_{sx}-v_{tx}\right)}{d}.$$ (6)

Within this paper, the angular velocity unit is rad/s.

Fig. 4:

Velocities of the cannula expressed in the sclera frame {s} and tip frame {t}.

$V_d^s={\left[v_{sx},v_{sy},v_{sz},w_{sx},w_{sy},w_{sz}\right]}^T$ is the desired sclera velocity expressed in the sclera frame {s}. In this paper we only focus on the automatic compensation for the vessel motion, the needle is already inside the vessel which means we do not need to use the handle force sensor to control the velocity along the tool shaft, i.e. v_sz is regarded as zero here. w_sz is also set as zero since the tool does not rotate around its own axis.. Based on the forward kinematics of the robot, the velocity vector $V_d^s$ is mapped to the base frame {b} expressed as $V_d^b.$ Then the desired joint velocity is derived as follows:

$${\dot{q}}_d=J^{†}\left(q\right)V_d^b,$$ (7)

where ${\dot{q}}_d\in {\mathbb{R}}^{5\times 1}$ is the deisred joint velocity, $J^{†}\left(q\right)\in {\mathbb{R}}^{5\times 6}$ is the pseudo inverse of the jacobian derived from the forward kinematics of the robot.

Fig. 3 shows the block diagram of the control framework to automatically compensate for the movement of the retina and keep the magnitude of the interaction force in a predetermined small range. The dual force-sensing tool measures the tip force F_t and sclera force F_s generated from the interaction with the retina and sclera, as well as the insertion depth d. Based on the aforementioned admittance control law, $\dot{q}$ is derived from F_t, F_s and d which is then fed into the built-in velocity controller to control the movement of the SHER 2.1. Correspondingly, the robot manipulates the tool to compensate for the movement of the retina and reduce the sclera and tip forces to maintain the needle tip inside the vessel.

Fig. 3:

Block diagram of the control framework.

III. Experimental Setup and Results

In this section, we present the quantitative experiments and user studies to verify the efficacy of the control strategy to actively compensate for the movement of the retina.

A. Experimental Setup

The experimental setup is shown in Fig. 5 and consists of: robotic systems (SHER 2.0 and SHER 2.1), an FBG interrogator (sm130-700, MicronOptics Inc., Atlanta, GA), an E-stop, a retina phantom, a sclera phantom and a dual force-sensing tool. SHER 2.0 was used to move the retina phantom. The tool was then attached to the SHER 2.1 and the explained control strategies were implemented on it in order to compensate the sclera forces and tip forces which would be induced by SHER 2.0 motion. These two robots were equipped with a low-level embedded control system (Galil 4088, Galil, Rocklin, CA). FBG interrogator collected the data from the dual force-sensing tool and then sent it to the computer to derive the tip force, the sclera force, and insertion depth based on the calibration matrices delineated by [18]. A retina phantom consisted of a 3D-printed plastic socket and a rubber membrane with some soft silicon tubes (ϕ = 0.5 mm) served as retinal vessels [25]. A rubber ring with a hole at its center was used as the sclera phantom and was fixed in space using a plastic support. Then, the microneedle attached to the end of force-sensing tool was inserted to the vessels on the retina membrane.

Fig. 5:

Experiment setup. (a) SHER 2.0 for moving the retina phantom, SHER 2.1 for conducting the control algorithm, FBG Interrogator for the dual force-sensing tool and E-stop; (b) 3D-printed retina phantom made of PLA, fixed sclera phantom and dual force-sensing tool; (c) Red retina membrane with vessels, microneedle inside the vessel.

The robot control system ran at 2000 Hz and the data sampling frequency was 200 Hz. All the data was collected and analyzed through the software framework developed based on CISST framework [26].

To reduce influencing factors, SHER 2.0 was programmed to move parallel to the Y axis of SHER 2.1 as shown in Fig. 5 (a). To mimic the movement of the retina, the motion of the SHER 2.0 was set as $S=A\cos \left(\frac{2\pi }{T}t\right),$ where A is the amplitude, T is the period, the frequency f of the curve is then set as 1/T.

B. Experiments with Various Amplitude

Since [19] shows that the amplitude of the retina movement is about 30 μm and the frequency is about 1 or 2 Hz, we first set the amplitude A of the movement of SHER 2.0 as 30 μm with f = 1 Hz. The data was recorded for 20 seconds.

Fig. 6 shows the first 5 seconds of the movement of the retina phantom (see Fig. 6 (a)), the linear velocity and angular velocity of SHER 2.1 (see Fig. 6 (b), (c)). Velocities were all zero in the above experiment denoting that the amplitude were too small and the contact force was not detected by the force-sensing tool.

Fig. 6:

Experimental results with the amplitude of A = 30 μm and frequency f = 1 Hz. (a) Movement of the retina phantom; (b) Linear velocity of the robot expressed in the sclera frame {s}, all of them value zero; (c) Angular velocity of the robot expressed in the sclera frame {s}, all of them value zero.

Experiments with larger amplitudes were also conducted, with A = 60 μm, 120 μm, 240 μm, and the frequency was fixed as 1 Hz. The data was recorded for 20 seconds. Fig. 7 shows the first 5 seconds of the movement of SHER 2.0 with the amplitude A = 60 μm (see Fig. 7 (a)), the first 5 seconds of selected velocity (see Fig. 7 (b)) and the whole 20 seconds of tip force F_t, sclera force F_s with active compensation and tip force ${\tilde{F}}_t$, sclera force ${\tilde{F}}_s$ without compensation (see Fig. 7 (c)). Only the force exceeding the threshold (F_tyth=6 mN) was used to compute values for the velocity. Since the data collected by the software system was smoothed internally, the shown curves were slightly different from the real situation. The following figures in this subsection show the data in the same way.

Fig. 7:

Experimental results with the amplitude A = 60 μm and frequency f = 1 Hz. (a) Movement of the retina phantom; (b) Selected force profile F_ty and velocity profile w_sx; (c) Boxplot of the tip force F_t, sclera force F_s with active compensation and tip force ${\tilde{F}}_t$, sclera force ${\tilde{F}}_s$ without compensation.

For no compensation experiments, the maximum and median values of F_t and maximum and median values of F_s are 16.30 mN, 5.08 mN, 18.25 mN, and 6.02 mN, respectively. Using the active compensation control algorithm, these forces are decreased to 11.48 mN, 4.43 mN, 14.01 mN, and 5.61 mN, respectively. The median values of the F_t are kept below the threshold (6 mN) and also the median values of the F_s are kept below the threshold (10 mN). Moreover, Fig. 8 and Fig. 9 show the experiments with the amplitude A = 120 μm and A = 240 μm respectively. For both these two experiments, the median values of the F_t are kept below the threshold (6 mN) and the median values of the F_s are also kept below the threshold (10 mN).

Fig. 8:

Experimental results with the amplitude A = 120 μm and frequency f = 1 Hz. (a) Movement of the retina phantom; (b) Selected force profile F_ty and velocity profile w_sx; (c) Boxplot of the tip force F_t, sclera force F_s with active compensation and tip force ${\tilde{F}}_t$, sclera force ${\tilde{F}}_s$ without compensation.

Fig. 9:

Experimental results with the amplitude A = 240 μm and frequency f = 1 Hz. (a) Movement of the retina phantom; (b) Selected force profile F_ty and velocity profile w_sx; (c) Boxplot of the tip force F_t, sclera force F_s with active compensation and tip force ${\tilde{F}}_t$, sclera force ${\tilde{F}}_s$ without compensation.

C. Experiments with Various Frequency

In this subsection, we conducted the experiments with the frequency of 2 Hz and the amplitude of 60 μm.

The results are shown in Fig. 10 including the movement of the retina phantom with A = 120 μm and f = 2 Hz (see Fig. 10 (a)), and the boxplot of the tip force F_t, sclera force F_s with f = 1 Hz and tip force F_t, sclera force F_s with f = 2 Hz (see Fig. 10 (b)). As mentioned above, for f = 2 Hz, the maximum and median values of F_t and maximum and median values of F_s are 11.48 mN, 4.43 mN, 14.01 mN, and 5.61 mN, respectively. However, for f = 2 Hz, these forces are significantly increased to 238.98 mN, 138.56 mN, 323.23 mN, and 194.41 mN, respectively.

Fig. 10:

Experimental results with the amplitude A = 60 μm, frequency f = 1 Hz and f = 2 Hz. (a) Movement of the retina phantom; (b) Boxplot of the tip force F_t, sclera force F_s with with f = 1 Hz and tip force F_t, sclera force F_s with f = 2 Hz.

D. User Study

In the user study, a surgeon was asked to manually manipulate the tool (still through the robot) to compensate for the retina movement that could be seen through a microscope. This user study was conducted to compare the effect of the above active compensation (AC) and the effect of the manual compensation (MC). The setup included the same hardware as the previous experiments. We conducted user studies for three amplitudes A = 60 μm, A = 120 μm and A = 240 μm. The data from eight manual compensation (recording time was about 16 seconds) trials were collected for each amplitude.

Tip and sclera force results relatively to the three amplitudes are shown in Fig. 12 (a) indicating the advantages of the control algorithm in compensating the movement of the retina and reducing the tip force and sclera force compared to the results of the manual compensation. Fig. 12 (b) shows that the manual compensation cannot follow the movement of the retina (tip force could roughly represent this movement). The surgeon also reported that it was difficult to manually compensate for such small movements.

Fig. 12:

User study results. (a) Boxplot of the tip force and sclera force; 6AT means the tip force from the active compensation experiments with the amplitude of 60 μm, 12MS means the sclera force from the manual compensation experiments with the amplitude of 120 μm and so on. (b) Selected force profile F_ty and velocity profile w_sx of the manual compensation with the amplitude of 60 μm.

IV. Discussion

The results of the first set of experiments indicate that the minimum amplitude that the robot is able to detect is about 60 μm. This limitation may occur due to the resolution of the force-sensing tool. For higher amplitudes, the results indicate the effectiveness of our control algorithm in increasing the safety level of vein cannulation in robot-assisted retinal surgery by automatically compensating for the movement of the retina and keep the tip force and sclera force in a predetermined small range.

However, the second set of experiments indicate that the robot is unable to follow the frequency at 2 Hz. The control algorithm implemented on the robot may on the contrary increase the tip force and sclera force. These results may be caused due to the limitation of the hardware or the communication speed of the software that renders the robot to be unable to follow the speed of the reciprocating motion.

User study results reported here indicate that the robot control algorithm conveys a compensation advantage for retinal movements and also reduces the tip force and sclera force as compared to manual compensation. Manual compensation is not effective in manipulations on this micron scale.

V. Conclusion and Future Work

In this paper, we proposed a force-based control strategy to automatically compensate for a possible retinal movement while maintaining the tip and sclera forces within a predetermined small range. using the tip and sclera force and tool insertion depth, the desired joint velocity for the robot was derived via a sinusoidal admittance control algorithm. Quantitative experiments were conducted in this work. The median tip and sclera forces were all kept below predetermined thresholds. These results demonstrated the efficacy of our control algorithm in providing safe force control during robot-assisted retinal vein cannulation. During the experiments, the microneedle was always inside the vessel assuring surgical success. The user study further demonstrated the advantages of robot control algorithm compared to the manual compensation for retinal movement.

However, these experiments also showed the limitations of our present system in that the robot was unable to detect retinal movement with the amplitude below 60 μm. The robot was also unable to follow the retinal movement at or above a frequency of 2 Hz. Moreover, the dimensions of the vessels and microneedle used in these experiments were larger than the average reported in humans. These limitations may influence the current clinical utility of this algorithm.

In the future, we will conduct similar experiments in real eyeball to evaluate the contact force that could damage the retina. We will also improve the force-sensing sensitivity of the tool and the communication speed of the robotic system. The motions of whole eye will be considered and active compensation will be carried out on both, the tool tip and sclera interaction forces. The present control algorithm and the robot itself could be implemented in the settings of other microsurgeries such as those being conducted in brain and other organs with microvascular pathology.

Fig. 11:

User study experimental setup including a microscope, SHER 2.0, SHER 2.1, a dual force-sensing tool, a fixed sclera phantom and a retina phantom.