Abstract
Retinal microsurgery requires steady and precise manipulation of delicate eye tissues in a very small space. Physiological hand tremor and lack of force sensing are among the main technical challenges, limiting surgical performance. We present a system that consists of the cooperatively controlled Steady-Hand Eye Robot and a miniaturized 3-DOF force sensing instrument to address these limitations. While the robot can effectively suppress hand tremor, enable steady and precise tissue manipulation, the force sensing instrument can provide three dimensional force measurements at the tool tip with submillinewton resolution. Auditory sensory substitution is used to give the user real time force information. Evaluation experiments are conducted using artificial and biological membrane peeling phantoms. Experimental results show that the robotic assistance and force-to-audio sensory substitution can effectively control the magnitude of the tool-to-tissue force. The direction profiles of the membrane peeling forces reflect the different delaminating strategies for different membrane phantoms.
I. INTRODUCTION
Retinal microsurgery treats a wide range of eye conditions and diseases. Common disorders include epiretinal membrane (ERM), macular hole, diabetic retinopathy, retinal detachment, and retinal vein occlusion. In retinal microsurgery, the surgeon places a surgical microscope above the patient’s head to gain a magnified visualization inside the patient eye. Long, thin ophthalmic tools (e.g., micro-pick and forceps with a length of about 30 mm and a diameter of less than 0.9 mm) are inserted through a sclerotomy to perform delicate surgical tasks at the back of the eye. One of the typical surgical procedures is ERM peeling. An ERM is a thin layer of semitransparent scar tissue formed on the surface of the retina (thickness about 60 μm [1]). The contraction of an ERM over time can result in the distortion of the underlying retina. ERM peeling removes the scar tissue to relieve retinal traction and thus restore patient’s vision. Another example is internal limiting membrane (ILM) peeling, where the innermost layer of the neurosensory retina is incised a few microns, and carefully delaminated from the rest of the retina. Compared to ERM, ILM peeling is even more challenging, because ILM is transparent and thin, measuring only from 0.5 to 2.5 μm [2]. In both procedures, the surgical motion needs to be very steady and precise to enable safe and effective membrane peeling. However, the physiological hand tremor of retinal surgeons is about 100 μm [3], significantly larger than the thickness of the membranes. In addition, the forces exerted between the tool tip and the eye tissue during retinal microsurgery are mostly imperceptible to the surgeons. Gupta et al. [4] reported that 75% of forces are less than 7.5 mN in magnitude during in vitro retinal manipulation in porcine cadaver eyes. Among the events at this force level, only 19% could be felt by the surgeons. Large forces increase the risk of injuring the retina, potentially leading to irreversible retinal damage. In current practice, retinal surgeons mainly rely on visual feedback, estimate the force magnitude from the tissue deformation, and control the tool motion with low velocity. A robotic surgical system augmented with smart force sensing tools has the potential to eliminate hand tremor, to enable steady and precise surgical motion, and to provide real time force feedback.
This paper presents a system that consists of the Steady-Hand Eye Robot and the three degrees of freedom (DOF) force sensing pick instrument to address limitations in retinal microsurgery. To our knowledge, this is the first robotic system for retinal microsurgery with integrated intraocular 3-DOF force sensing capability at the tool tip. We evaluate this system in simulated membrane peeling tasks using artificial and biological phantoms.
II. Materials and Methods
A. Steady-Hand Eye Robot
The Steady-Hand Eye Robot 2.1 [5] is a high precision, non-back drivable robot, successor of the previous version [6]. Fig. 1 shows the robot CAD model. The hands-on cooperative control enables stable and precise manipulation with the transparency and immediacy of handheld tools. The robot has five DOFs in total, three translational and two rotational DOFs. The rotation about tool axis is free motion. The translational and rotational resolutions are 3 μm and 0.0005°, respectively. A symmetric mechanical remote center-of-motion (RCM) mechanism enables a ±45° rotation range for both left and right hand users.
Fig. 1.
The Steady-Hand Eye Robot with the 3-DOF force sensing pick instrument firmly attached in the tool holder. The tool coordinate axes are aligned with the robot when the robot is at the home position. The 3D force vector is described using azimuth (α) and elevation (β) angles in the tool coordinate frame, as shown in the close-up view of the tool tip. The micro-pick is pointing toward the positive X-direction.
B. 3-DOF Force Sensing Pick Instrument
We have developed a 3-DOF force sensing pick instrument [7] with integrated fiber Bragg grating (FBG) sensors. FBG sensors detect the strain change and modulate it into reflected narrow spectrum spike, termed as Bragg wavelength. They provide high sensitivity strain measurement with biocompatibility and immunity against electromagnetic noises. A superelastic nitinol flexure is designed to reduce the axial stiffness and improve the axial force sensitivity. The 3-DOF force sensing micro-pick preserves the form factor of a standard ophthalmic instrument. The tool diameter is 0.9 mm, while the length of the distal force sensing segment is about 12 mm. This tool provides a force resolution of 0.5 mN and 0.1 mN in axial and transverse direction, respectively. The accuracy is 0.7 mN and 0.2 mN for axial and transverse force sensing, respectively. The direction of three dimensional (3D) force can be described using azimuth (α) and elevation (β) angles in the tool coordinate frame. Fig. 1 illustrates the tool coordinate frame aligned with the robot axes at home position.
C. Membrane Peeling Phantoms
In order to simulate the membrane peeling task, two types of phantoms are used in the experiments: an artificial phantom, created with bandage strips, and a biological phantom, the inner shell membrane of a raw chicken egg.
1) Bandage Strips
A previous study [8] found that sticky tabs from 19 mm Clear Bandages (Rite Aid Corp., Camp Hill, PA) exhibit consistent delaminating behavior. The sticky tab is sliced into strips with a width of 2 mm and attached to an acrylic base using double-sided tape, as shown in Fig. 2(b) and (d). The tab backing can be then peeled multiple times with good consistency. The force required to delaminate is close to the previously reported force level in retinal microsurgery [4].
Fig. 2.
Experimental setup: membrane peeling with robotic assistance using bandage strip phantom (a), free hand membrane peeling using bandage strip phantom (b), membrane peeling with robotic assistance using ISM phantom (c), close-up view of the 3-DOF force sensing micro-pick delaminating the bandage strip phantom (d), and close-up view of the micro-pick peeling off the ISM (e). Blue arrows show the direction of the peeling motion.
2) Inner Shell Membranes of Raw Chicken Eggs
While the bandage strip phantom produces repeatable peeling results, a biological phantom can provide more realistic tissue characteristics. The inner shell membranes (ISM) of raw chicken eggs were used in [9]. The larger end of the egg has an air cell between the inner and outer shell membrane. The egg is cracked in half to remove the yolk and the albumen. The part of the ISM above the air cell is used for membrane peeling, as shown in Fig. 2(c) and (e). Similar to the clinical practice [2], circular movement is applied to remove the desired area of the ISM.
D. Force-to-Audio Sensory Substitution
Previous investigation [8], [10] have shown the effectiveness of force-to-audio sensory substitution. The audio feedback scheme in [8] is used to provide the user awareness of the force exerted at the tool tip. The audio feedback modulates “beeps” played at different frequencies that are dependent on the force magnitude measured by the 3-DOF force sensing tool. When the force is less than 1 mN, the audio is silent. When the force is between 1 and 3.5 mN, “beeps” at a constant slow frequency are played, indicating a “safe zone” of operating force. A “cautious zone” follows with proportionally increasing “beeps” frequency to reflect forces from 3.5 to 7.5 mN. For forces larger than 7.5 mN, a constant fast beeping warns the user of the “dangerous zone.”
III. EXPERIMENTS AND RESULTS
A single non-surgeon subject study is carried out as preliminary evaluation of the robotic assistant with integrated 3-DOF force sensing micro-pick. Previous 2-DOF force sensing tools are desired to be held perpendicular to the membrane to ensure that forces are only exerted in the transverse plane, resulting in suboptimal ergonomics [9]. Utilizing the capability of measuring force in all three directions, the new tool is not required to move under certain motion constraints.
A. Experimental Setup and Tasks
The following four conditions are tested on both phantom types: freehand with/without audio feedback (FH/FHA), and robot-assisted with/without audio feedback (R/RA), as shown in Table I. For each condition, the subject is first trained with 10 practice trials, and then performs 10 recorded experimental trials. In all robot-assisted trials, the 3-DOF force sensing tool is taped firmly in the robot tool holder to prevent rotation about the tool axis, because this degree of freedom is neither actuated nor encoded. The coordinate axes of the 3-DOF force sensing tool are aligned with those of the robot, as shown in Fig. 1. Two adjustable platforms are used to change the relative height of the membrane phantom and to provide wrist support to the user, respectively. No microscope is used for visual magnification.
TABLE I.
Experimental Conditions
| Robot-assisted peeling | Force-to-Audio feedback | ||
|---|---|---|---|
| 1 | FH | × | × |
| 2 | FHA | × | ✓ |
| 3 | R | ✓ | × |
| 4 | RA | ✓ | ✓ |
The membrane peeling task using the artificial phantom requires the subject to delaminate the bandage strip with a length of about 10 mm. Fig. 2(a), (b) and (d) illustrate the experimental setups and the close-up view of the micro-pick pulling off the bandage strip membrane phantom, respectively. In the ISM peeling experiment, the subject is asked to delaminate the ISM for at least 5 mm. Fig. 2(c) and (e) show the experimental setup with ISM and the close-up view of the micro-pick peeling the ISM.
An optical sensing interrogator (sm130–700, Micron Optics, Atlanta, GA) is used for the FBG data acquisition at 2 kHz. The force calculation algorithm is integrated into the robot C++ software with 200 Hz loop rate. The tool tip force, robot position and velocity, as well as the time stamp are recorded and analyzed.
B. Results
The 3-DOF force sensing instrument can measure the force components along X-, Y-, and Z-axes. Fig. 3 illustrates these force components, as well as the force magnitude of two example membrane peeling trials with bandage strip and ISM phantoms. The transverse force measurements, Fx and Fy, exhibit lower noise than the axial force measurement, Fz, because the force sensing tool has better force resolution and accuracy in the transverse directions than those in the axial direction. The force magnitude and direction characteristics are analyzed using the 3D force measurements.
Fig. 3.
Force plots of example experimental trials with bandage strip (a) and ISM (b) phantoms. Experimental condition is robot-assisted with audio feedback (RA). Red curve shows the force magnitude, while green, brown, and blue curves are the force components along X-, Y-, and Z-axes in the tool coordinate frame, respectively. The transverse force measurements, Fx and Fy, exhibit lower noise than the axial force measurement, Fz, because the force sensing tool has better force resolution and accuracy in the transverse directions than those in the axial direction.
1) Force Magnitude Characteristics
The force magnitude is particularly important for limiting the tool-to-tissue force below the safety threshold. The mean and maximum of the force magnitude, as well as the percentage of time with force greater than 7.5 mN are summarized in Table II. The force-to-auditory sensory substitution not only demonstrates significant reduction of both mean and maximum force magnitudes, but also shortens the percentage of time that force greater than the 7.5 mN threshold is applied. This is consistent with previous studies with 2-DOF force sensing tools [8]. In general, robotic assistance enables steady and precise motion control. Although robotic assistance does not exhibit notable improvement over freehand when no audio feedback is provided, it does show the effectiveness of shortening the time percentage with force greater than 7.5 mN when audio feedback is present to provide the cue for motion correction. For both membrane phantom types, robotic assistance with audio feedback yields the smallest time percentage with force greater than 7.5 mN.
TABLE II.
Experimental Results of the Membrane Peeling Experiments
| Bandage Strips | ISM | |||||||
|---|---|---|---|---|---|---|---|---|
| FH | FHA | R | RA | FH | FHA | R | RA | |
| Mean force (mN) | 6.52±0.91 | 5.23±0.56 | 5.88±0.74 | 4.83±0.48 | 4.23±1.03 | 3.83±1.19 | 4.99±2.20 | 3.30±0.88 |
| Max. force (mN) | 14.95±2.13 | 10.18±0.77 | 13.62±3.40 | 12.02±3.52 | 13.95±3.89 | 15.39±4.74 | 17.68±6.90 | 11.20±4.39 |
| Time%(F >7.5 mN) | 34.66±15.40 | 6.82±5.03 | 28.15±15.74 | 3.95±2.82 | 16.22±10.48 | 13.88±13.83 | 23.37±18.07 | 8.29±8.11 |
2) Force Direction Characteristics
While the bandage strip phantom requires a linear peeling trajectory, the ISM phantom is delaminated with a circular movement. Fig. 4 illustrates two examples of 3D force direction histogram to visualize the different peeling strategies using the bandage strip and ISM phantoms. The horizontal axes of Fig. 4(a) and (c) are respectively the azimuth (α) and elevation (β) angles, which describe the force direction as defined in Fig. 1. The vertical axis of Fig. 4(a) and (c) shows the normalized frequency of the forces exerted in the corresponding direction. Fig. 4(b) and (d) show the normalized force direction frequency on a unit sphere. As shown in Fig. 4(a) and (b), forces are concentrated in one direction (α ≈ 180°, β ≈ −45°) during membrane peeling with bandage strip phantom. This dominate direction is the result of combined pulling and lifting of bandage strip, because it is opposite of the micro-pick direction in the X–Y plane, and points 45° downwards. As comparison, the forces exerted in ISM peeling distribute in several major directions, see Fig 4(c) and (d). This is attributed to the circular delaminating motion in the ISM peeling.
Fig. 4.
The normalized force direction histograms are plotted on the α-β angle grid with forces recorded in the bandage strip (a) and ISM (c) peeling. Vertical axis shows the normalized frequency of the corresponding force direction. Heat maps on a unit sphere show the normalized force direction frequency of the forces recorded in the bandage strip (b) and ISM (d) peeling. While delaminating bandage strip phantom concentrates force in one direction consistently, the ISM peeling exerts more distributed forces due to the circular movement. Both are robotically assisted with audio feedback (RA).
IV. DISCUSSION
The experiment setup does not simulate the sclerotomy constraint which limits the tool motion to 4 DOFs, i.e., three rotational DOFs about the sclerotomy site and one translational DOF along the tool axis. This constrained 4-DOF motion corresponds to the RCM concept in medical robotics [11]. In practice, the RCM constraint could result in different characteristics of the tool-to-tissue force as that described in Section III. Future investigation will use more realistic phantoms under a simulated sclerotomy constraint. Additional force sensing at the sclerotomy [12] can be integrated to update the RCM constraint in real time.
To our best knowledge, this is the first robotic system with integrated intraocular 3-DOF force sensing with submillinewton resolution. The results of the preliminary evaluation (Section III) show that the 3-DOF force sensing tool can provide robust force measurements during membrane peeling using bandage strips and ISMs of raw chicken eggs. Previous investigations [6], [8] have developed robot control algorithms based on the tool tip force. However, the implementations and experiments were limited because only 2-DOF force sensing tools were available. The system presented in this paper can eliminate these barriers and enable development and validation of new methods for robot force control.
The real-time 3-DOF force sensing capability can be used not only to improve robot force control, but also to characterize tissue properties. Fig. 5 illustrates two examples of the tool tip trajectory with overlay of force vectors along the trajectory. The tool tip position is logged by the robot, while the 3D force is measured by the 3-DOF force sensing instrument. The tissue deformation can be associated with the force applied. Incorporating additional information, e.g., stereo microscopic video and optical coherence tomography (OCT), the tissue characteristics can be analyzed and modeled, and in turn can be used to develop more sophisticated robot control algorithms to perform desired surgical tasks.
Fig. 5.
Tool tip trajectory (black curves) with overlay of 3D force vectors (red arrows) shown in two examples of bandage strip membrane peeling (a) and ISM peeling (b). Both are robotically assisted with audio feedback (RA).
V. CONCLUSIONS
Retinal microsurgery is one of the most technically challenging microsurgical disciplines. Two main limitations are the physiological hand tremor and the lack of force sensing. The Steady-Hand Eye Robot was designed to enable stable and precise surgical motion, while the 3-DOF force sensing tool was developed to provide real time 3-DOF force measurement with submillinewton resolution. This paper presents the first evaluation of the Steady-Hand Eye Robot augmented with high sensitivity 3-DOF force sensing at the tool tip. Membrane peeling experiments are conducted using bandage strip and ISM phantoms, in order to simulate typical procedures of retinal microsurgery, e.g., ERM and ILM peeling. Four conditions are tested: freehand with/without audio feedback, and robot-assisted with/without audio feedback. Experimental results show that the robotic assistance and force-to-auditory sensory substitution can effectively control the magnitude of the tool-to-tissue force. The direction profiles of the membrane peeling force reflect the different delaminating strategies for different membrane phantoms. Future work includes evaluation of the current system using the Chick Chorioallantoic Membrane [13], and further analysis and modeling on tissue deformation, tool velocity, and tool-to-tissue interaction force during live animal surgery.
Footnotes
Research supported in part by NIH BRP grant 1 R01 EB 007969, in part by NIH grant R01 EB 000526, in part by Wilmer Eye Institute’s Research to Prevent Blindness, and in part by Johns Hopkins University internal funds. Other equipment and systems infrastructure support were developed within the CISST ERC under NSF grant EEC9731748.
References
- 1.Wilkins JR, Puliafito CA, Hee MR, Duker JS, Reichel E, Coker JG, Schuman JS, Swanson EA, Fujimoto JG. Characterization of epiretinal membranes using optical coherence tomography. Ophthalmology. 1996;103(12):2142–2151. doi: 10.1016/s0161-6420(96)30377-1. [DOI] [PubMed] [Google Scholar]
- 2.Almony A, Nudleman E, Shah GK, Blinder KJ, Eliott DB, Mittra RA, Tewari A. Techniques, rationale, and outcomes of internal limiting membrane peeling. Retina (Philadelphia, Pa) 2012 May;32(5):877–891. doi: 10.1097/IAE.0b013e318227ab39. [DOI] [PubMed] [Google Scholar]
- 3.Singhy S, Riviere C. Physiological tremor amplitude during retinal microsurgery. IEEE Northeast Bioengineering Conference. IEEE; 2002; pp. 171–172. [Google Scholar]
- 4.Gupta P, Jensen P, de Juan E. Surgical forces and tactile perception during retinal microsurgery. International Conference on Medical Image Computing and Computer Assisted Intervention. 1999;1679:1218–1225. [Google Scholar]
- 5.He X, Roppenecker D, Gierlach D, Balicki M, Olds K, Gehlbach P, Handa J, Taylor R, Iordachita I. Toward Clinically Applicable Steady-Hand Eye Robot for Vitreoretinal Surgery. ASME 2012 International Mechanical Engineering Congress and Exposition; 2012; pp. 145–153. [Google Scholar]
- 6.Uneri A, Balicki MA, Handa J, Gehlbach P, Taylor RH, Iordachita I. New Steady-Hand Eye Robot with micro-force sensing for vitreoretinal surgery. IEEE International Conference on Biomedical Robotics and Biomechatronics; 2010; pp. 814–819. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.He X, Handa J, Gehlbach P, Taylor R, Iordachita I. A submillimetric 3-dof force sensing instrument with integrated fiber bragg grating for retinal microsurgery. IEEE Transactions on Biomedical Engineering. 2014;61(2):522–534. doi: 10.1109/TBME.2013.2283501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Balicki M, Uneri A, Iordachita I, Handa J, Gehlbach P, Taylor R. Micro-force sensing in robot assisted membrane peeling for vitreoretinal surgery. International Conference on Medical Image Computing and Computer Assisted Intervention; Jan. 2010; pp. 303–310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sunshine S, Balicki M, He X, Olds K, Kang JU, Gehlbach P, Taylor R, Iordachita I, Handa JT. A force-sensing microsurgical instrument that detects forces below human tactile sensation. Retina (Philadelphia, Pa) 2013 Jan;33(1):200–206. doi: 10.1097/IAE.0b013e3182625d2b. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cutler N, Balicki M, Finkelstein M, Wang J, Gehlbach P, Mc-Gready J, Iordachita I, Taylor R, Handa JT. Auditory force feedback substitution improves surgical precision during simulated ophthalmic surgery. Investigative Ophthalmology & Visual Science. 2013 Feb;54(2):1316–1324. doi: 10.1167/iovs.12-11136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Taylor RH, Funda J, Grossman DD, Karidis JP, LaRose DA. Remote center-of-motion robot for surgery. 5,397,323. US Patent. 1995 Mar 14;
- 12.He X, Balicki M, Gehlbach P, Handa J, Taylor R, Iordachita I. A multi-function force sensing instrument for variable admittance robot control in retinal microsurgery. IEEE International Conference on Robotics and Automation; 2014; in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Leng T, Miller JM, Bilbao KV, Palanker DV, Huie P, Blumenkranz MS. The chick chorioallantoic membrane as a model tissue for surgical retinal research and simulation. Retina (Philadelphia, Pa) 2004 Jun;24(3):427–34. doi: 10.1097/00006982-200406000-00014. [DOI] [PubMed] [Google Scholar]