Towards securing the sclera against patient involuntary head movement in robotic retinal surgery

Abstract

Retinal surgery involves manipulating very delicate tissues within the confined area of eyeball. In such demanding practices, patient involuntary head movement might abruptly raise tool-to-eyeball interaction forces which would be detrimental to eye. This study is aimed at implementing different force control strategies and evaluating how they contribute to attaining sclera force safety while patient head drift is present. To simulate patient head movement, a piezoelectric-actuated linear stage is used to produce random motions in a single direction in random time intervals. Having an eye phantom attached to the linear stage then an experienced eye surgeon is asked to manipulate the eye and repeat a mock surgical task both with and without the assist of the Steady-Hand Eye Robot. For the freehand case, warning sounds were provided to the surgeon as auditory feedback to alert him about excessive slclra forces. For the robot-assisted experiments two variants of an adaptive sclera force control and a virtual fixture method were deployed to see how they can maintain eye safety under head drift circumstances. The results indicate that the developed robot control strategies are able to compensate for head drift and keep the sclera forces under safe levels as well as the free hand operation.

I. INTRODUCTION

Retinal surgery continues to be one of the most demanding surgical practices entailing very sensitive tissue manipulation [1]. Reducing surgeon hand tremor and consequently enhancing tool tip precision would be of principal importance which have been successfully fulfilled by advancements in robot-assisted eye surgery [2]. The surgeon has to insert the surgical tool into the eye through the white portion of the eye (sclera) and execute the necessary steps (Fig. 1). Of highly required features during retinal surgery is the limitation of tool-to-eye interaction forces (sclera forces) and keeping them in safe ranges. This is even more desirable in robot-assisted eye surgery since the robotic assist diminishes the force information that used to be sensed by the surgeon in a freehand manipulation. In other words, the robot inertia and stiffness does not allow the tiny sclera forces to be transferred to surgeon hand such that they can react appropriately. This will result in larger sclera forces in robot-assisted eye manipulation [3]–[5]. On the other hand, patients under anaesthesia might have abrupt head movement leading in unexpected and sudden increase in the sclera forces. The situation would be even exacerbated in robot-assisted eye surgery if the abrupt rise in the sclera force is not accompanied by appropriate reaction from the robot. The intraoperative head drift during the surgery is among the under addressed challenges in retinal surgery [6].

Fig. 1.

Eyeball manipulation with the SHER. Force-sensing tool is grabbed by the dominant hand. The eyeball is attached to the piezoelectric-actuated linear stage producing random step motion in horizontal direction.

The Stedy-Hand Eye Robot (SHER) shown in Fig. 1 developed at the Johns Hopkins University is a collaborative robot for helping surgeon for obtaining better accuracy in tool positioning [7]. There have been several other robots developed including [8]–[12] as collaborative robots and [13]–[15] as tele-manipulated robots. Of note, first in-human robot-assisted eye surgeries have been performed recently by [16], [17].

In recent studies methods for enhancing safety of tool to eye interaction forces in either robotic and freehand eye surgery have been evaluated. He et al. used neural networks to predict the unsafe sclera forces during robot-assisted eye surgery [18]. Cutler et al. assessed the effect of auditory feedback in limiting tool tip forces within safe levels in robot-assisted phantom membrane peeling task [19]. Ebrahimi et al. [20] evaluated the advantage of providing haptic force feedback and audio feedback for restricting the sclera forces. Matinfar et al. ussed a novel sonification method to convey useful information during ophthalmic procedures [21]. Moreover in our recent study we showed the effectiveness of a novel adaptive sclera force control method [22].

To the best of our knowledge, in all of the above mentioned studies it is assumed that patient’s head is kept stationary without any motion during the surgery. In this paper we intend to evaluate the methods for increasing sclera force safety under more realistic circumstances where the patient’s head have random lateral motions during anesthesia which is a common happening in a surgery practice. To simulate this lateral movement we used a piezoelectric-actuated linear stage which produces random motions. An experienced eye surgeon was then asked to perform a mock surgical task on a phantom eyeball which is attached to the linear stage. Then, the freehand with and without the auditory substitution experiments were conducted to assess how well the surgeon is capable of dealing with the head motion. The robotic experiments were also conducted using the SHER with the same surgeon with four different conditions. These conditions consisted of robot-assisted without any sclera force control, with a virtual fixture control, with adaptive control for sclera force component, and adaptive control for sclera force norm. The results were analyzed to see which method better helps the user to handle the safety under head drift circumstances and conclusion were drawn using the user study results.

II. MATERIALS AND METHODS

In this section the patient head movement simulator, and the strategies for counteracting the excessive sclera forces in freehand and robot-assisted eye surgery generated from head motion are discussed.

A. Patient head motion simulator

In order to create a mock patient’s head motion during retinal surgery, we utilized a piezoelectric-actuated linear stage (Q-Motion Stages, PI Motion and Positioning, MA, USA) which is depicted in Fig. 1. The position-controlled stage receives position commands and has three degrees of freedom (DoF) in Cartesian space. Each of the degrees of freedom has a motion range of ±6 mm. The mean lateral head drift during cataract surgery was reported to be 2.9 mm [6]. In some extreme cases the operating microscope had to be adjusted for to accommodate the head drift [6]. To simulate the lateral head motion we programmed the linear stage to make sporadic random motions in one Cartesian direction (Fig. 1). The stage was set to generate step random motions with an amplitude between 1 mm and 3 mm and then go back to its zero position. This motion range firstly includes the mean lateral motion of 2.9 mm which was reported by [6] and secondly does not require to adjust the microscope position during the experiments.

The intervals between two successive instances of head motion during the surgery can also be modeled as a random number based on the feedback we obtained from our surgical lead. For this reason, the stage was made to initiate a motion after a time T_start which is random variable with uniform distribution over the period of 5 to 10 seconds (T_start ~ U[5, 10]) as shown in Fig. 2. Then a random position value having a uniform distribution over [−3, −1] ∪ [1, 3] is produced and sent to the stage embedded motor controller. After reaching the desired position a random variable T_wait having a uniform distribution over 1–3 seconds is generated for the amount of time the stage will stay in the commanded position before moving back to the zero position. A graph showing a typical motion of the stage is shown in Fig. 2.

Fig. 2.

A typical motion generated by the stage in lateral direction to simulate patient’s head motion. The blue and black arrows represent T_start and T_wait, respectively. The solid red lines only connect the discrete commanded positions (hollow circles) and do not carry any other meaning.

B. Sclera Force Safety for Freehand

In order to assist the surgeon to deal with the unsafe sclera forces (Fig. 3) stemming from patient’s head motion during freehand manipulation, we produced warning sounds as audio feedback proportional to the level of sclera forces. After being alerted of the excessive forces, the surgeon should react accordingly and reduce the sclera force by intuitively reorienting the tool toward correct directions. Auditory feedback was proved beneficial in increasing tip force safety and also sclera force safety as reported by [19] and [20], respectively. In our recent study [20], we came up with the upper safe bound of 120 mN for the magnitude of sclera force. This value was obtained by averaging the sclera force data recorded from our expert surgeon manipulating the eyeball with freehand. In other words, this is a hypothetical value which we are using in experiments as an upper safe value for scelra force and does not necessarily mean that this value will harm the actual sclera tissue. However, the methods that are discussed within this paper can be easily adjusted to account for any new thresholds for unsafe sclera force.

Fig. 3.

Left: The environment compliance λ_x and λ_y along the x and y components of sclera force, F_sx and F_sy. Right: A close-up view of the FBG strain sensors attached along the force-sensing tool shaft.

As a precaution, the warning beeps are triggered when the magnitude of sclera force reaches 80 mN. This first level of auditory feedback has a low-volume and low-frequency sound and as soon as the sclera force oversteps 100 mN the sound volume goes higher to better alert the surgeon about approaching the upper safe level. When the sclera force reaches the unsafe level of 120 mN, a continuous high-volume beep is produced for the surgeon.

C. Sclera Force Safety Integrated with SHER

In order to conduct robot-assisted experiments we used the SHER. The robot is a 5-DoF robot including three translational motions attributed to the robot base. The two rotary DoFs of the robot corresponds to the robot end-effector pitch and roll motions as depicted in Fig. 4. In order to move the robot, the surgeon should hold the tool handle which is attached to the robot and then move the robot around (Fig. 1). The robot then obeys the forces and torques exerted by the user F_h and moves accordingly to proper direction and orientation thanks to the impedance control of the robot. Furthermore, the SHER is a velocity-controlled robot which means it receives desired velocity setpoints (${\dot{X}}_d\in {ℝ}^6$ which consists of the linear and angular velocities of the end-effector frame) [22]. It is noteworthy to say that all vectors in this paper are expressed in the end-effector coordinate frame as shown in Fig. 4 which is attached to a fix point on the tool shaft.

Fig. 4.

A close-up view of the SHER’s end-effector. The vector F_h is shown with the dashed arrow.

The wrench vector $F_h\in {ℝ}^6$ (shown in Fig. 4) includes the forces (first three elements of F_h) and torques (last three elements of F_h) exerted by the surgeon expressed in the end-effector frame. The robot normal impedance controller works based on the proportional control law written in (1).

$${\dot{X}}_d=\mathbb{K}{F}_h$$ (1)

Where $\mathbb{K}$ is a constant diagonal gain matrix. This will in turn produce an intuitive motion of the robot where it obeys the way the user wishes to move. As it can be seen this control law does not include any feedback from the sclera force components (F_sx and F_sy depicted in Fig. 3-left). Because in robot-assisted eye surgery the surgeon does not perceive the sclera forces as they used to feel in freehand manipulation, this control law might put the eye at high risk of injury specially when there is sudden patient head movement. Thus, the following sclera force control methods can be examined for safety enhancement when patient head movement is present. The first method is a virtual fixture method which can be considered as a passive method since it only blocks the surgeon from moving toward unsafe directions. The second and third methods are two variants of an adaptive sclera force control method which was developed in [22]. In addition, as it was noted for the auditory feedback control the warning alarms were commenced at 80 mN and the volume would increase when the sclera force reaches 100 and 120 mN. The average of these three numbers would be 100 which is the value used to trigger the other control methods for the robot-assisted conditions to have a fair comparison.

1). Virtual fixture method

The idea of this method is to create a virtual wall along either x or y directions of the end-effector frame depending on which component of sclera force has reached the threshold of 100 mN. For instance, if the magnitude of the component F_sx becomes more than 100 mN, the surgeon can no longer move the robot along the positive x axis of the end-effector frame since it leads to further increase of F_sx. However, the surgeon can move along the negative x axis which has an opposite sign as of the value of sclera force at that instant of time. This in turn makes the surgeon well aware of the unsafe forces and by producing a virtual wall informs the surgeon about the unsafe directions as well. The algorithm when F_sx > 100 can be summarized as written in (2). The notation ${\dot{X}}_d[i]$ denotes the ith element of the vector ${\dot{X}}_d$. A similar scenario can be considered for F_sy as well.

$$\begin{gathered} \text{If}{F}_{hx}>0,\text{then}{\dot{X}}_d[1]=0 \\ \text{If}{F}_{hx}<0,\text{then}\text{use}(1) \end{gathered}$$ (2)

2). Adaptive control for sclera force component

In this control method, the robot tries to actively compensate the exceeded sclera forces generated by the patient head movement. This details of the adaptive control for sclera force can be found in [22] where it was proved to be reliable in providing safe sclera force manipulation in stationary head situations. The adaptive control laws which are applied for the first and/or second components of ${\dot{X}}_d$ will make the components of sclera force to converge to desired trajectories (F_dx and F_dy ). Thus, we can define safe references trajectories for F_sx and F_sy and consequently the robot tries to reduce the forces whenever the adaptive control for that specific component is triggered. The adaptive force control works based on an online estimation of the environment compliance along the x or y directions of the end-effector frame (λ_x and λ_y, Fig. 3-left). By updating these estimations using the adaptation law in (3) the robot finds out how to move to make the components of sclera force converge to the desired trajectories. The adaptive force control and the adaptation law for the x direction would be as following, respectively:

$$\begin{gathered} {\dot{X}}_d[1]={\lambda }_x{\dot{F}}_{dx}-k_x\left(F_{sx}-F_{dx}\right) \\ {\dot{\lambda }}_x=-C_x{\dot{F}}_{dx}\mathrm{\Delta }{F}_x \end{gathered}$$ (3)

where the ${\dot{F}}_{dx}$ is the first derivatives of F_dx. The term ΔF_x is the force tracking error F_sx − F_dx. C_x and k_x are constant positive control gains which were chosen based on try and error and set to be 0.2 and 2 × 10⁻⁶, respectively. In this method, we trigger the adaptive control for the relevant component of sclera force (x or y) independently. For example, if |F_sx| exceeds 100 mN, the first component of ${\dot{X}}_d$ switches to be produced using (3) instead of (1) while other five components of ${\dot{X}}_d$ will continue to be generated using (1). This makes the robot to still obey the commands of the surgeon and not block her/his manipulation. The safe reference trajectory for F_sx is considered as following [22]:

$$F_{dx}=\frac{F_{sx}^0}{2}\left(e^{-\left(t-t_0\right)}+1\right)$$ (4)

where t₀ is the time when F_sx reaches the triggering point of 100 mN and The scalar $F_{sx}^0$ is the value of F_sx at time t = t₀. The element ${\dot{X}}_d[1]$ is again switched back to be produced based on (1) when F_sx is reduced to less than 70 mN [22]. For the y component of sclera force also a similar scenario can be imagined.

3). Adaptive control for sclera force norm

For this part we still abide by the the control laws explained in section II–C.2. However, instead of initiating the adaptive control laws for F_sx and F_sy independently, we trigger (4) for F_sx and F_sy concurrently as soon as the magnitude of sclera force $F_s=\sqrt{F_{sx}^2+F_{sy}^2}$ reaches 100 mN. We speculate this control would contribute more to increasing sclera force safety once head drift is present since both components of sclera force are contributing to decrease the sclera force magnitude which in turn reduces the robot reaction time to unsafe happenings.

III. EXPERIEMTNAL SETUP AND USER STUDY

In order to conduct the experiments and implement the safety methods explained in the previous sections two force sensors are required to measure the user interaction forces and torques vector F_h and also the sclera forces F_sx and F_sy. The ATI force sensor shown in Fig. 4 gives the vector F_h. Moreover, Fiber Bragg grating (FBG) optical strain sensors were used as explained by [23] to build a force-sensing tool (Fig. 3-right) to measure sclera forces F_sx and F_sy. The FBG optical fibers are very sensitive to strain which makes them a suitable choice for eye surgery applications requiring mN-order sclera force measurements. Three optical fibers were attached along a 25-gauge Nitinol shaft to build a custom force-sensing tool. The tool was then calibrated and validated using the methods given by [23]. The FBG fibers are connected to an optical sensing interrogator (SI155-Hyperion from Micron Optics Inc., Atlanta, GA) for measuring the change in the reflected optical wavelength as shown in Fig. 5.

Fig. 5.

Experimental setup including the SHER, the FBG-equipped force-sensing tool, the FBG interrogator, the eye phantom attached to the linear stage, the speaker and a microscope.

The experimental setup for conducting the user study is represented in Fig. 5. It consists of the SHER, linear stage, force-sensing tool, interrogator, eye phantom attached to the linear stage and the optical microscope for vision. The way that the surgeon should perform the robot-assisted experiments is depicted in Fig. 1. For the freehand experiments the robot will be put aside and the surgeon manipulates the eyeball without using the robot. There is a secondary tool which is not a force-sensing tool that the surgeon uses in left hand as shown in Fig. 1 for ease of manipulation.

To run the user study six sets of experiments were conducted with an experienced surgeon. In each experiment set the performance of a single control method was investigated to see how it can handle the patient head movement during robot-assisted eye surgery. The linear stage was used to simulate the patient head movement for all experiment sets as explained in section II–A. There are phantom vessels which are painted in four different colors (Fig.3-a) and attached to the posterior portion of the eye. For each experiment set the surgeon was asked to follow the colored vessels in random sequences with the tool tip while looking through the microscope. Each experiment set was repeated in ten trials. The experiment sets can be enumerated as following:

1. Freehand with no auditory feedback

2. Freehand with auditory feedback (section II–C.1)

3. Robot-assisted with no sclera force control (solely control (1))

4. Robot-assisted with virtual fixture control

5. Robot-assisted with adaptive component control (section II–C.2)

6. Robot-assisted with adaptive norm control (section II–C.3)

IV. RESULTS AND DISCUSSION

After conducting the user study all of the results were collected using the software packages developed for SHER control, the C++ CISST-SAW libraries [19]. The sclera force magnitude during each experiment trial as well as the time required to finish each task are among the data information we are interested in. In addition to the total time, we measured the time spent within the region of unsafe sclera force (F_s > 120 mN) to see how safe that trial has been conducted. In other words our primary measure of safety is to see how well the slcra forces are kept below the unsafe limit when the random patient head motion is present. The results averaged over all 10 trials for each experiments set are summarized in Table I. In addition, a typical trial for the freehand and robot-assisted experiments are shown in Figs. 6 and 7, respectively.

TABLE I:

The sclera force magnitude and time information averaged over all ten trials for each experiment set.

	Sclera force control	Mean of sclera force norm (mN)	Average total time (s)	Time spent over 120 mN (s)	Percent of time spent over 120 mN
Freehand	No sclera control	67.0	25.5	2.2	9 %
	Audio feedback	78.3	28.2	4.7	17 %
Robot-assisted	No sclera control	133	28.37	12.7	45 %
	Virtual fixture	109.5	25.1	8.8	35 %
	Adaptive component	83.1	27.4	2.7	10 %
	Adaptive norm	70.54	27.1	0.44	2 %

Fig. 6.

A typical trial for the freehand experiments.

Fig. 7.

A typical trial for the robot-assisted experiments.

The first conclusion that comes to mind after looking into the last column of Table I is the fact that the adaptive norm control and the adaptive component control of sclera force are able to handle the patient head motion as safe as the freehand manipulation which can be considered as a ground truth for safe manipulation. Furthermore, the adaptive norm control of sclera force even surpasses the freehand manipulation and is best helping the surgeon in handling the patient’s head movement and keeping the sclera forces in safe ranges. This is the direct consequence of proper and fast action of the robot when the head motion appears. By looking into Fig. 7 a similar conclusion can be made.

Another observation from Table I is that the average total time required to finish each experiment is not changing significantly. In other words, although different control strategies are used the manipulation speed is maintained constant and the control methods do not slow the surgeon down.

By considering Fig. 7, it can be seen that the robot-assisted manipulation without any sclera force control cannot handle the increased sclera forces due to the patient’s head movement since the sclera forces are growing as large as 400 mN and the surgeon cannot react properly.

In contrary to being beneficial in eye manipulation with stationary head [20], the audio feedback is not helping an experienced surgeon since it is increasing the time spent on forces over 120 mN (based on Table I) for the freehand manipulation. The difference is that in stationary head manipulation the surgeon is not confronted with sudden head movement. In that case the sclera forces grow gradually and the auditory feedback would be helpful. However, when sudden head movements occurs the audio feedback will just fluster the surgeon while the surgeon can handle the incident well with her/his experience.

The same scenario as the auditory feedback holds for the virtual fixture case. While virtual fixture can be useful in stationery head cases, it cannot handle the patient head motion because it is a passive method and to counteract the patient abrupt head motion in robot-assisted manipulation an active method is required. In other words, when the patient head moves the sclera forces increase and go above the unsafe limit and just blocking the surgeon motion along the unsafe direction (virtual fixture method) does not help bring back the force into safe regions (Fig. 7). Based on the feedback obtained from the surgeon after manipulation, none of the control methods hindered the eye manipulation. The user was completely satisfied with the robot action since it helped keep the tool straight by reducing the sclera forces autonomously which facilitates targeting in robot-assisted eye surgery.

V. CONCLUSION

In this study we conducted a preliminary experiment with an experienced retinal surgeon to see which control methods can better help the surgeon deal with the patient head movement in either freehand or robot-assisted manipulation. Among the robot-assisted methods the adaptive component control and the adaptive norm control of sclera force showed promising performances which can be further investigated. Furthermore, in the freehand manipulation the auditory feedback did not help in the moving head condition. Our future goal would be conducting experiments with multiple retinal surgeons under moving head circumstances. At the end we can conclude that considering the current results and to limit the time for our upcoming multi-user experiments (under one hour to efficiently use the clinicians’ time), we will only use three conditions including: the robot-assisted with adaptive force control for sclera force components, robot-assisted with adaptive force control for sclera force norm and the freehand manipulation.