A Surgical Robotic System for Osteoporotic Hip Augmentation: System Development and Experimental Evaluation

Abstract

Minimally-invasive Osteoporotic Hip Augmentation (OHA) by injecting bone cement is a potential treatment option to reduce the risk of hip fracture. This treatment can significantly benefit from computer-assisted planning and execution system to optimize the pattern of cement injection. We present a novel robotic system for the execution of OHA that consists of a 6-DOF robotic arm and integrated drilling and injection component. The minimally-invasive procedure is performed by registering the robot and preoperative images to the surgical scene using multiview image-based 2D/3D registration with no external fiducial attached to the body. The performance of the system is evaluated through experimental sawbone studies as well as cadaveric experiments with intact soft tissues. In the cadaver experiments, distance errors of 3.28mm and 2.64mm for entry and target points and orientation error of 2.30° are calculated. Moreover, the mean surface distance error of 2.13mm with translational error of 4.47mm is reported between injected and planned cement profiles. The experimental results demonstrate the first application of the proposed Robot-Assisted combined Drilling and Injection System (RADIS), incorporating biomechanical planning and intraoperative fiducial-less 2D/3D registration on human cadavers with intact soft tissues.

I. Introduction

THE worldwide occurrence of hip fractures due to osteoporosis is expected to increase from 1.7 million in 1990 to 6.3 million in 2050 [1]. The overall mortality rate within one year of these fractures is approximately 20% [2], [3]. Furthermore, only 60% of the patients who suffer from an osteoporotic hip fracture recover to their previous level of mobility [4]. Biomechanical studies have shown that Osteoporotic Hip Augmentation (OHA) by injecting acrylic bone cement can serve as an immediate preventive approach to reduce the risk of such injuries in contrast to existing drugs that gradually improve the bone density due to osteoporosis. OHA increases the bone strength and energy to failure of the proximal femur and can be performed minimally-invasive with less surgical complications [5], [6], [7], [8], [9]. Beckmann et al. studied the effect of the augmentation using Polymethylmethacrylate (PMMA) as the bone cement with different single and double drill holes and showed the significant improvements in the fracture strength and mechanical stability of the femur with the single central and centrodorsal augmentation approaches [5]. Fliri et al. showed that cement augmentation carries potential to prevent second hip fractures [7]. Basafa et al. developed computer assisted biomechanical planning paradigm to optimize the cement volume and trajectory of cement injection using combined finite element and hydrodynamics models. He successfully executed the plan on isolated cadaveric femurs using navigation system and showed 33% (P < 0.05) improvement in yield load with an average 9.5 ml of injection [10], [11], [8]. In a recent computational study, our group introduced a new subject-specific planning paradigm for OHA with more efficient injection strategy. The navigated execution of the plan in six pairs of isolated cadaveric femurs showed 42%, (P < 0.001) improvement in yield load with injection volume of 9.1 ml on average. [12], [9].

Implementation of the subject-specific biomechanical planning, however, requires a surgical system in which bone is registered to the pre-operative Computed Tomography (CT) scans used for the optimization. Moreover, the system requires localization of a hand-held drill within the same frame of reference. In the first attempt at biomechanical planning and navigated execution of OHA, our group utilized an integrated system that allowed for real-time visualization of a hand-held tool (drill bit) with respect to the femur specimen as well as the optimal pose and orientation of the drill path. Using in-house navigation system, the user manually aligns the drill to the desired entry point of the drill trajectory and subsequently drilling is performed manually with the aid of real-time visualization of the distance and angle errors of the drill path [8], [13] (Fig. 1). Furthermore, we designed and fabricated an automatic injection device for cement delivery after drilling that provides substantial forces needed to inject highly viscous bone cement [14]. While the proposed approach increased the bone strength of the cadaveric femora, manual movement of the drill and injection device results in an average translational error of 6.2 mm between the planned and injected volume of bone cement [9].

Fig. 1.

Manual drilling using navigated hand-drill. The in-house navigation system guides the user to align the drill to the desired planned location and perform the drilling.

The mentioned surgical planning and execution system were commonly performed on isolated femora where optically tracked navigation system was used for femur registration using Iterative Closest Point (ICP) algorithm [15]. However, in real surgical scenario with soft tissues intact, use of optical tracker may not be applicable to perform minimally-invasive augmentation of the osteoporotic hip. The main reason is that, to perform ICP using optical tracker, large section of the tissues needs to be dissected so that the bone surface becomes visible. On the other hand, point-based registration using anatomical features may not be accurate. Our group previously showed that using image-based multi-view 2D/3D registration; as an alternative approach for femoral bone registration with intact soft tissues, reduces registration time and associated errors [13], [16]. The approach requires a tracking fiducial to be attahed to the bone; however, the attachment of fiducial with external pins to the bone with severe osteoporosis may not be an ideal option [17].

While few studies have investigated the biomechanical planning and navigated execution of OHA [8], [9], [13], [18], use of a robotic system for OHA can enhance the drill accuracy, minimizing cementation errors, and contribute to the overall safety and efficacy of the procedure for patients. To our knowledge; however, the literature does not document any study on the development of a robotic system for OHA. In recent years, we developed a fiducial-free 2D/3D registration pipeline with the use of multiple view fluoroscopic images to perform anatomy and tool co-registration [17]. Instead of using external fiducials, this approach uses pelvis; with more distinct anatomical features, as an internal fiducial to register femurs [17], [19]. The proposed method was evaluated through comprehensive simulations and cadaver studies which showed the superior accuracy of using image-guided registration in the presence of soft tissues that can be integrated in robot-assisted femoroplasty.

Robot-assisted drilling in orthopedics surgeries has been examined in numerous studies [20], [21], [22], [23]. Lee et al. introduced a robotic system that drills the bone with a contact drilling force and can automatically stop drilling at the moment of breaking through [21]. However, the drilling rate was faster than the clinically accepted safety criteria (0.26 mm/s). A variety of strategies have been proposed for intraoperative guidance of drilling and screw placement in spinal surgery, including use of fluoroscopy [24], optical navigation [25] and robotic-assistance [26], [27]. In the study of Ortmaier et al., an optically tracked navigation system was integrated with an impedance-controlled light weight robot holding the surgical instrument for pedicle screw placement. The system was tested in both artificial and bovine spine and resulted in average errors of 0.6mm and 0.5° for the 2D position and orientation of the drilling [26]. Currently, there are two FDA-approved robotic systems for spinal surgery in the United States: Mazor^® (Medtronic, Minneapolis, MN) and the Excelsius GPS^® (Globus Medical, Inc., Audubon, PA). Both are “semi-active” systems that focus primarily on drilling pedicle trajectory guidance and rely on a rigid sleeve connected to a positioning robotic arm to constrain the trajectory through which the surgeon can pass tools such as high-speed drills to perforate the pedicle and guide the screw safely into the vertebral body. However, none of these robotic systems are capable of injection and address cementation during the procedure.

In addition, few studies have investigated the use of robotic systems for Percutaneous vertebroplasty (PVP) [28], [29], [30], [31]. Onogi et al, introduced a needle insertion robotic system consisting of a puncture robot with X-ray-translucent end-effector and a passive failsafe mechanism and optical tracking device. The robotic system was evaluated through phantom studies and root mean square errors of 1.7mm and 1.6° between the planned and actual trajectories were reported [29]. In recent studies of robot-assisted vertebroplasty, Neumann et al. introduced a robotic system combining a cold passive exchanger to slow down the cement curing and an active exchanger controlling the injected cement temperature [30]. Opfermann et al. proposed a cannula mounted robot and investigated the feasibility of using image guided vertebral augmentation [31]. The aforementioned studies have commonly shown the higher accuracy of planned cementation with the use of robotic systems.

In this paper, we present a novel image-guided, robot-assisted surgical system for the execution of the OHA that integrates combined bone drilling and cement injection capabilities. The system consists of a 6-DOF robot arm, a custom-designed Drilling and Injection component (DI) capable of both drilling and injection, optical tracking system and C-arm (Fig. 2). Our intraoperative computer-assisted workstation consists of a PC-based user interface, which allows for the execution of the graphical visualization, Injection Navigation, and 2D/3D registration through the use of 3D slicer [32]. The workstation software is built using the in-house “Surgical Assistant Workstation (SAW)” architecture and software libraries [33], providing an open-source environment that allows for the swift integration of pre-existing software and other available infrastructure. The performance of the robot-assisted combined drilling and injection using DI is evaluated through sawbone and cadaveric experiments. An optically tracked navigation system is used in pilot sawbone experiments to register the femur to its CT volume. However, in the cadaver experiments, to account for the presence of soft tissues, we use a fluoroscopic image-based 2D/3D registration method for the femur and DI co-registration [17].

Fig. 2.

Overview of the Robot-Assisted combined Drilling and Injection System (RADIS) for orthopedic applications including OHA. The system consists of a 6-DOF positioning robot, integrated drilling and injection component (DI), optical tracking system and C-arm.

To the best of our knowledge, this is the first study on a surgical robotic system for execution of OHA that is evaluated through cadaveric experiments with intact soft tissues. Contributions and novelties of this study can be summarized as follows:

• Development of an image-guided, robot-assisted surgical system towards minimally-invasive execution of the femoral bone augmentation.

• Cadaveric studies with intact soft tissues for evaluating robot-assisted bone augmentation system with integration of multi-view X-ray based 2D/3D registration.

• Design and fabrication of a single component capable of drilling and controlled injection of the bone cement for OHA.

II. ROBOT-ASSISTED SYSTEM OVERVIEW

A. Rigid link positioning Robot

A 6-DOF rigid link positioning robot (UR10, Universal Robot, Odense, Denmark) is used to position the DI to the entry point of the specific trajectory that is determined based on the patient-specific biomechanical planning using pre-operative CT scan of the femur specimens [8], [9]. The close-loop position control is implemented to control the manipulator pose. Using Denavit-Hartenberg parametrization, the forward kinematics of UR10 is derived and robot Jacobian (J_R) for 6 revolute joints can be expressed as

$$J_R=\left[J_{r_1}{J}_{r_2}\dots J_{r_6}\right].$$ (1)

In the sawbone experiments, based on the system coordinate frames (Fig. 5), the DI tip position $x_{tip}^{DI}$ in the robot base frame can be found as

$$x_{tip}^{DI}=T_{rob}·T_{DI}·T_{jig}^{-1}·T_s·T_{cs}^{-1}·p_{off},$$ (2)

where p_off is the offset point (1-2 mm) from the goal entry point of the drill trajectory in the CT coordinate.

Fig. 5.

Chain of rigid transformation of the RADIS in pilot sawbone experiments. To perform hand-eye calibration, the custom designed optical tracker jig is mounted on the DI guide and T_ej is calculated. Sawbone-to-CT registration, T_cs is computed with a point-cloud to surface registration using ICP algorithm followed by initial registration using three predetermined anatomical landmarks. Three predetermined landmarks for initial registration are shown in black points on the the sawbone and surface landmarks for ICP registration are shown in red dash-lines.

However, due to the weight of the DI attached to the end-effector of the robot and given the relatively large end-effector moment arm (approximately 1m away from the base of the robot), the forward kinematics of the UR10 may not be accurate to reach to the entry goal point of the drill/injection trajectory. To eliminate measurement inaccuracies in the cadaveric experiments, we mount optical tracker reflective jigs on the base of the UR10 (a custom-designed 4 markers jig 2 in Fig. 7) and DI body for direct measurement of the robot pose. The DI tip position $x_{tip}^{DI}$ here can be formulated as

$$x_{tip}^{DI}={{{}^{Carm}T}_{DI}}^{-1}·{{}^{Carm}T}_{femur}·p_{off}.$$

We implement resolved rate motion control [34] for closed-loop position control of the RADIS.

Fig. 7.

Illustration of system calibration and multi-view image-based 2D/3D registration. Key frames are annotated in red cross arrows. (a) Hand-eye calibration scheme for X-ray image-based navigation. ^CarmT_DI is the pose estimation using 2D/3D registration. ^J1T_SJ is the pose estimation of the DI jig (jig 1) frame derived from the optical tracker observations. ^J1T_DI is the hand-eye calibration matrix. (b) Multi-view 2D/3D registration scheme. Example multi-view C-arm image geometries are illustrated using green source-to-detector projection lines. Rigid pose estimations ^CarmT_femur, ^CarmT_DI are presented in dark blue arrows.

B. Injection Actuation System

The injection actuation unit of the DI has been adapted from the automatic injection device, developed by Kutzer et al. [14]. Like the latter, DI also uses a non-captive linear actuation system driven by a 5V bipolar stepper motor (Haydon Kerk Motion Solutions, Waterbury, CT) with integrated quadrature encoder. The actuator is capable of providing up to 890N of continuous force at a pulse rate greater than 620 steps per second which is greater than 4.92 mm/s with a step resolution of 0.0079mm linear travel per step. The actuator is used to push a 1,112N capacity miniature compression load cell (OMEGA LCGD-250, OMEGA Engineering Inc., Stamford, CT) with a resolution better than ±1.33N (±20% FSO, hysteresis ±10% FSO repeatability). The load cell on the switch is pushed to contact the syringe plunger plate to inject the augmentation material into the bone while the contact force between the load cell and syringe plunger is monitored. The contact force and relative position and velocity of the syringe plunger is estimated from encoder data. A programmable motor controller and stepper motor driver (Performax PMX-2ED-SA, Arcus Technologies Inc., Fremont, CA) are used to control the injection using USB2.0 communication.

C. Drilling and Injection Component

To successfully execute OHA using a robot arm, we designed and fabricated an integrated Drilling and Injection component (DI) that is capable of both guiding the drill trajectory and performing controlled injection of the cement using a single registration. This component represents the cutting-edge design of our robot-assisted system, which allows for precise and efficient execution of both guided drilling and injection, thus optimizing the overall outcome of orthopedic bone augmentation procedures. The system design requirements of the DI are as follows: i) Minimizing the time needed to change from drilling to injection configuration. ii) Minimizing registration errors associated with assembling two different tools for drilling and injection. iii) Ensuring that the drill bit and injection cannula pass through the same path, thus allowing for easy insertion of the cannula into the drilled tunnel with a single registration, and iv) Ensuring that the depth of the drilled hole that is determined through biomechanical planning, is controllable.

Fig. 3(a) shows the integrated Drilling and Injection Component (DI) which consists of a supporter shell, a linear stage, a load cell, a gauge with a cylinder guide and a switch block to interchange between drilling and injection. A 3D printed shell attached to the end-effector of the robot is used as a supporter for other parts of the DI. A linear stage provides the push force on the plunger of the syringe and serves as a hard stop for drilling. The push force is measured by a load cell on the linear stage and maintains an injection rate of 0.1 ml/s, which corresponds to 1.05 mm/s of plunger advancement velocity [14]. A gauge made of stainless steel is the supporter of the syringe and the slender cylinder of the gauge is used as a guide for both drill bit and cannula (both are 4 mm). The length of the guide is 75 mm and it is placed on the surface of the tissue of the femur. This length keeps a safe distance from the DI shell to human tissue accounting for changes in the soft tissue thickness.

Fig. 3.

(a) Integrated Drilling and Injection Component (DI), (b) Guided drilling with a cordless hand drill, (c) Cement injection with inserted syringe, (d) three position of the DI switch, (e) Measurement of the hole depth through a linear stage.

In order to interchange between drilling and injection, a switch block is attached to the linear stage. The load cell is fixed on one side of the switch as shown in Fig. 3(a), and the other side of the switch is notched to fix the orientation of the drill bit. The switch can be adjusted in 3 positions as shown in Fig. 3(d), A for drilling, B for changing tools and C for injection. One or two quick-release pins are used to fix the switch positions.

To perform drilling, the switch is fixed at Position A using one quick-release pin. A 4 mm drill bit passes through the notch and a hand-held drill is held by the user. Once the end of the guide touches the anatomy, the depth of the drilled hole can be measured as shown in Fig. 3(e). Since the length of the drill (L_drill) and gauge (L_guage) are constant, the depth of the drilled hole decreases with the linear stage pull back. The user then carries out the drilling with a hand-held drill, continuing until the desired depth is achieved.

After drilling, the switch is turned to position B to clear a space for syringe attachment and subsequently it is turned to position C for cement injection. Two 6 mm quick-release pins are used to maintain a shear force. With the forward motion of the linear stage, the load cell on the switch is pushed to contact the syringe plunger plate and inject the augmentation material into the bone. The stroke of the linear stage is 100 mm, which is sufficient for a 20 ml syringe to inject all the cement inside the femoral head and neck.

III. PRE-OPERATIVE PLANNING

We perform two sets of experiments; sawbone and cadaver experiments, to evaluate the performance of the RADIS. Prior to the sawbone experiments, the potential drill trajectory is identified on the 3D models of the sawbone femurs that are generated from segmented CT scans of sawbones. The predetermined trajectory is defined by insertion point that is the entry point of the drill line on the greater trochanter surface of the femurs and target point that is the starting point of the injection inside the femoral neck.

In the cadaver experiments, the drill path is determined based on the patient-specific biomechanical planning that is described in detail in [8], [9]. For this purpose, pre-operative CT scans of two lower torso specimens are acquired on a Toshiba Aquilian One (Canon, In., Tochigi, Japan) with slice thickness of less than 2 mm, which is resampled to have 1 mm isotropic voxel spacing. An automated method described in [35] is used to segment each femur and pelvis. We then create a Finite element (FE) model of the femur following the procedure described in [10]. Inhomogeneous material properties are assigned to each element of the femur based on the bone density observed from CT scan using density phantom. For this purpose, the bone is divided to the upper region as mostly trabecular bone (90 mm upper part of the proximal femur) and the lower region as mostly cortical bone. The boundary condition simulating a fall to the side on the greater trochanter is applied in the FE model. Following the procedure described in [10], [36], we use the maximum principal strain criterion to determine the volume of the failed elements, we define the yield load as the load at which the volume of the failed elements reaches 1% of the total volume of the femur. From each pair, one femur with the lower initial yield load w selected for augmentation.

The optimized cement distribution profile that increases the predicted yield load of the specimen to twice of its original value is then determined using modified Bi-directional Evolutionary Optimization (BESO) [11]. Next, using geometric optimization, the FE-optimized injection pattern is approximated with 2 or 3 injection blobs in cylindrical shapes with end caps (spheroids). The gradient-descent optimization algorithm optimizes the size and location of the realistic injection spheroids with total volume of no more than 10ml and align them on the single path of the injection to find the closest match between the FE injection profile and realistic cement injection blobs [37]. In this step, in addition to the lengths and radii of the injection blobs, the target point of the injection and orientation of the injection path is determined which also identify the location and orientation of the drill trajectory. Finally, the diffusion of PMMA inside the bone as a porous media is predicted using a modified smoothed particle hydrodynamic (SPH) simulation [37]. The steps of the biomechanical planning of the augmented right femur of the lower torso specimen are shown in Fig. 4

Fig. 4.

Biomechanical Planning steps of the augmented right femur of lower torso specimen. The drill/injection line is determined through the biomechanical planning.

IV. NAVIGATION SYSTEM AND REGISTRATION

A. Sawbone Experiments

We use an in-house optically tracked navigation system that was previously evaluated in [16] to register sawbones to their CT volumes. For this purpose, we attach a tracking rigid body with reflective markers (NDI, Waterloo, ON, Canada) to the sawbones . Three distinctive anatomical landmarks, i.e., center of the femoral head, lateral most point on the greater trochanter, and the protrusion point of the lesser trochanter are selected on the segmented sawbone models in 3D Slicer software [32]. The corresponding landmarks on the sawbone models are manually identified using an optical tracked digitizer (Passive Probe, NDI, Waterloo. ON, Canada) and an optical tracker (Polaris, NDI, Waterloo, ON, Canada). The rigid transformation between two sets of landmarks, provide an initial guess for registration between the tracking camera and the CT coordinates. Using the initial registration, we then digitize the patches of surface landmarks on the proximal metaphysis, physis and epiphysis of the sawbones to perform a point-cloud-to-surface registration using ICP algorithm.

In order to compute the transformation between UR10’s end-effector and the tip of the DI guide, we attach a custom-designed 4 markers jig (jig 1 in Fig. 5) to the DI guide to perform a hand-eye calibration [38] (Fig. 6(a,b)). To solve the hand-eye calibration problem formulated as [39]

$$T_{rob}{T}_{DI}=T_{base}{T}_{jig},$$ (3)

the UR10 robot is moved arbitrary to a sequence of poses and T_rob, forward kinematics of the robot and T_jig, the transformation between the jig and optical tracker are stored in each pose (Fig. 5). Using initial poses, T_b can be written with respect to T_DI as

$$T_{\mathit{\text{base}}}=T_{rob,0}{T}_{DI}{T}_{jig,0}^{-1}.$$ (4)

Substituting (4) in (3) and rearranging it, gives a linear system of equations AX = XB to solve for X = T_DI, where

$$\begin{gathered} A=T_{rob,0}^{-1}{T}_{rob,k} \\ B=T_{jig,0}^{-1}{T}_{jig,k} \end{gathered}$$ (5)

for k = 0, …, N poses. Substituting the calculated T_DI in (3) solves for T_base. By computing T_DI, the chain of transformations is closed and UR10 navigates the DI to the insertion point of the desired drill trajectory.

Fig. 6.

RADIS for sawbone experiments. (a,b) Hand-eye Calibration, (c) Guided drilling, (d) Cement Injection.

B. Cadaveric Experiments

Due to the fact that the bone surface is not exposed during the minimally-invasive procedure with soft tissues intact, performing ICP as described in previous section will not provide an accurate registration. Therefore, the navigation system based on fiducial-free image-based 2D/3D registration proposed by Gao et al [17] is used during the cadaver experiments. The intra-operative poses of the DI and femur are estimated using multi-view C-arm X-ray images, which are then transformed to the pre-operative planned injection trajectory for positioning the DI. To achieve accurate navigation, an optical tracking system is used for closed-loop robot control (Fig. 7(a)). The optical tracker constantly tracks the poses of jig frames; Jig 1 (J1) mounted on the DI body, ${}^{\mathit{\text{tracker}}}{T}_{J1}$ and a static jig frame, jig 2 (SJ), ${}^{\mathit{\text{tracker}}}{T}_{SJ}$. We perform hand-eye calibration to compute the transformation of the DI frame (DI) to the J1, ${}^{J1}{T}_{DI}$. Pose of J1 relative to SJ is computed using ${}^{SJ}{T}_{J1}={\left({}^{\mathit{\text{tracker}}}{T}_{SJ}\right)}^{-1}{·}^{\mathit{\text{tracker}}}{T}_{J1}$. Similar to equation 3, the hand-eye calibration problem here is formulated as:

$${}^{SJ}{T}_{\text{Carm}}{·}^{Carm}{T}_{DI}{=}^{SJ}{T}_{J1}{·}^{J1}{T}_{DI}$$ (6)

A sequence of ^CarmT_DI and ^SJT_J1 poses are collected by moving the UR10 to a variant of positions. The hand-eye calibration matrix, ^J1T_DI is solved using the similar derivation as equations 4 and 5.

The intra-operative poses of the femur and DI are estimated using intensity-based 2D/3D registration. Pre-operative CT scans of the specimens are acquired. The femur volumes are segmented using the method described in [17]. Intensity-based 2D/3D registration is performed by optimizing a similarity score between the target image and a digitally reconstructed radiograph (DRR) simulated from the 3D model of the DI or CT segmentation. The target C-arm images are taken from multiple views to improve single-view ambiguity. The two side C-arm views are separate in a range of 50 to 60 degrees with respect to the C-arm rotation center. The multi-view geometry is estimated using the DI registration. Given X-ray images taken from multiple C-arm views I_m, m ∈ {1, …, M} (M is the total number of views), the 3D volumetric data V, V ∈ {femur, DI}, a DRR operator (𝒫), a similarity metric (S), the 2D/3D registration solves the pose $T_{\text{Carm}}^V$ by optimizing:

$$\underset{T_{\mathit{\text{Carm}}}^V\in SE(3)}{\min }\sum _{m=1}^M𝒮\left(I_m,𝒫\left(V;T_{\mathit{\text{Carm}}}^V\right)\right).$$ (7)

We use patch-based normalized gradient cross correlation (Grad-NCC) as the similarity metric (𝒮) [40]. The optimization strategy is selected as “Covariance Matrix Adaptation: Evolutionary Search” (CMA-ES) due to its robustness to local minima [41]. Relative pose of the DI and femur is computed using ^femurT_DI = (^CarmT_femur)⁻¹ · ^CarmT_DI. By computing ^J1T_DI and ^femurT_DI, the robot is able to navigate the DI to the planned insertion and target points of the planned trajectory following the chain of transformations.

V. Experiments

A. Sawbone Drilling Experiments

To investigate the drilling performance of the RADIS, we first conduct a series of pilot sawbones experiments. For this purpose, five sawbone femur models, manufactured by Pacific Research Laboratories, Inc. (Vshon, Washington, United States) are used. The sawbones have a hard cortical bone and a soft two-part structure of cancellous bone to mimic the structure and function of the human femoral bone. We perform a total of eight drilling experiments to evaluate the drilling accuracy of the RADIS (Fig. 6).

The manipulator positions the DI guide 1-2 mm from the insertion point of the planned injection trajectory to prevent movement of the sawbones. After drilling, post-operative (post-op) CT scan with the drill bit inside the trajectory is taken. We then perform automatic registration using rigid transformation followed by initial alignment of the post-op CT scans of the drilled sawbones to their pre-operative counterparts. We calculate the distance errors of the insertion and target points that is a norm of translation vector between planned and measured drill trajectory. Furthermore, the angle between two drill lines represents the orientation error of the drilling.

B. Sawbone Injection Experiment

One of the major contributing factors to the outcome of the OHA is the difference between the shape of injected cement and the planned profile. To evaluate this metric and investigate the feasibility of the cement injection of the RADIS, we perform a pilot experiment in which a total PMMA volume of 3 cm³ is injected in a sawbone sample #1. This injection starts at the target point at the rate of 0.1 cm³/s, while the injection needle is retracted 33 mm distally from the target along the drill path. A spheroid shape injection blob is expected as a result of this planned trajectory as shown in Fig. 14(a) (red spheroid).

Fig. 14.

Overlay of the cement isosurfaces (a) before and (b) after ICP registration, (c) Hausdorff distance heatmap in registered injected cement blob, compared with planned injection spheroids. Red indicates the planned cement profile and blue indicates the segmented injected blob.

Following the injection, a post-op CT scan of the injected sawbone is collected and registered to the pre-op scans. A threshold-based segmentation is done in 3D Slicer to extract the geometry of the injected PMMA. Fig. 14(a) shows the segmentation of the injected blob in blue. Next, the two injection blobs (planned and injected) are compared in MeshLab; an open-source mesh processing tool [42], by calculating the Hausdorff distance between the two meshes.

C. Cadaveric Drilling Experiment

We carry out a cadaveric experiment on a male specimen including the lower torso, pelvis, and femurs to further evaluate the drilling performance of the RADIS with the use of fluoroscopic image-based 2D/3D registration (Fig. 8). We use a Siemens CIOS Fusion C-Arm to take fluoroscopic images. Three separate X-ray images are taken for femur and tool (DI) registration.

Fig. 8.

Cadaveric drilling experiment. (a) X-ray based registration in RADIS, (b) Guided drilling.

Based on the biomechanical planning, the drill path is determined on the segmented 3D model of the right femur. Similar to sawbone experiments, we command the manipulator to position the DI guide 1-2 mm from the insertion point. After drilling, we place a 4 mm drill bit inside the trajectory and take a lower torso post-op CT scan. The insertion and target points along the drill trajectory are marked on the post-op CT. We manually annotate a 3D point cloud on the femoral head in the post-op CT scan. Next, we perform ICP registration from the 3D point cloud to pre-op segmented femur surface to register the post-op CT to pre-op CT coordinates and subsequently the marked insertion and target points are transformed to the pre-op CT coordinates.

We report l₂ distance error of the insertion and target points and δθ, orientation error compared to the planning using

$$l_2^i=\parallel p_{\mathit{\text{plan}}}^i-p_{\mathit{\text{postop}}}^i{\parallel }_2,i\in \{\mathit{\text{insertion}},\mathit{\text{target}}\}$$ (8)

$$\begin{gathered} v_j=p^{\mathit{\text{target}}}-p^{\mathit{\text{insertion}}},j\in \{\mathit{\text{plan}},\mathit{\text{postop}}\} \\ \delta \theta =\mathit{arccos}\left(\frac{v_{\mathit{\text{plan}}}·v_{\mathit{\text{postop}}}}{\parallel v_{\mathit{\text{plan}}}\parallel ·\parallel v_{\mathit{\text{postop}}}\parallel }\right) \end{gathered}$$ (9)

D. Cadaveric Integrated Drilling and Injection Experiment

We evaluate the performance of our image-guided, integrated robot-assisted combined drilling and injection system through a cadaveric experiment on a 75 years old female specimen including lower torso, pelvis and femurs (Fig. 9). We first command the manipulator to place the DI guide 12 mm from the planned insertion point with the aid of X-ray based 2D/3D registration. We fix the DI switch at Position A for drilling. A 4 mm drill bit passes through the notch of the switch and reaches to the desired location (insertion point) on the surface of the femur. We keep drilling into the bone until the desired depth of the injection line is reached. The drill path is determined based on the pre-operative biomechanical planning on the segmented 3D model of the osteoporotic right femur with a Neck T-score of −2.6.

Fig. 9.

Cadaveric integrated drilling and injection experiment (a) Guided drilling (b,c) Cement injection.

Following drilling, we proceed to cement preparation by mixing 15 gr of radiopaque surgical Simplex P (Stryker, Kalamazoo, MI, united States) bone cement powder with 13.5 ml of the monomer liquid for about 60 s. We fill a 20 ml syringe with the bone cement and attach a 15 cm, 8G Cannula (Scientific Commodities Inc., Lake Havasu, AZ, United Sates) to the syringe. Next, we turn the switch to position B to mount the syringe on the DI and then position C for cement injection. After removing the air and waiting for 6 minutes from the start of mixing the powder and liquid, we estimate the viscosity of the bone cement by ejecting 0.5 ml of cement at the rate of 0.1 ml/s while averaging the syringe pressure during the last second. The remaining time before the cement reaches the desired viscosity of 200 Pa.s. is calculated [37], [8], [9] and it allows us to position the manipulator to the target point of the injection inside the femoral neck.

The subject-specific FE optimization simulate two cement blobs for the lower-torso specimen, to match the BESO results (Fig. 4). Therefore, we command the manipulator to retract the DI from the start of the first injection blob (target point) toward the end of the second injection blob, while cement is automatically injected at the controlled rate of 0.1 ml/s with the forward motion of the linear stage mounted on the DI. Note that the robot’s speed for retraction varies depending on the length and volume of each blob. For this purpose, we set two waypoints for the start and end of each injection blob and command the robot to move linearly between the two waypoints at a specified tool speed. After reaching to the end point of the second blob, the UR10 is retracted to a safe position and the cannula is ejected through the bone.

Following the injection, we obtain a lower torso post-op CT scan from the augmented specimen. After automated femur segmentation [35], we register post-op CT to pre-op CT coordinates by performing ICP registration from the annotated 3D point cloud on the post-op femoral head to pre-op femur surface. We then perform FE simulation on the augmented specimen using acquired post-op CT scan and determine the post yield load. To evaluate how closely the cement distributions from post-op injection matches that of the planned injection, we estimate the translational and rotational errors between the isosurfaces of the injected and planned cement blobs, using the technique described in [8]. In this method, since the drill line is not visible after the injection, the errors are calculated by evaluating the ICP registration that transforms the post-op injected cement surface to a new pose where it better fits the simulated one [8]. We also calculate the mean shape error between the registered, segmented injected PMMA and planned injection blobs by calculating the Hausdorff distance between the two segmented meshes in MeshLab.

VI. Results

A. Sawbone Drilling Experiments

The error results of the sawbone specimens are summarized in Table I. The sawbone-to-CT registration error using an optical tracking system is on average 0.33(±0.04) mm for five sawbones. However, the distance average errors between planned and measured drill trajectories are reported as 3.64(±0.67)mm for insertion point and 3.78(±0.68)mm for target point and the orientation average error is 1.77°(±0.85) for eight drilling experiments. Moreover, the time needed to interchange between drilling and injection is measured as 50 s in which cannula insertion time is 8s. Fig. 10(a,b) compares the measured and planned trajectories in specimen sample #1.

TABLE I.

Summary of distance and orientation errors for drill trajectory in sawbone experiments

Experiment number	Distance insertion point error (mm)	Distance target point error (mm)	orientation error (degree)
1	3.71	3.82	3.76
2	2.96	2.63	1.79
3	4.33	3.83	0.97
4	4.39	4.05	2.02
5	2.59	3.25	1.25
6	3.51	3.41	0.91
7	3.39	4.55	2.04
8	4.28	4.72	1.46
Mean±SD	3.64±0.67	3.78±0.68	1.77±0.85

Fig. 10.

Measured vs. planned trajectory in (a) sawbone sample #1 and (c) drilled lower torso specimen. (b) X-ray image of the drill trajectory in sawbone sample #1 with 4mm drill bit. (d) X-ray image of the guided drilling using DI in the cadaveric drilling experiment.

To compare robot-assisted drilling with manual navigated drilling, we also compute the drill trajectory errors for seven cadaveric isolated femora that were previously drilled manually and augmented using our computer-assisted planning paradigm and in-house navigation system [8], [9]. The distance average errors were 6.21(±3.31)mm and 6.64(±2.81)mm for insertion and target points, respectively and the average orientation error was 6.23°(±2.25) for seven specimens. A paired t-test is performed (P < 0.05) to compare the drill trajectory errors using robot-assisted drilling and manual navigated drilling. The differences are found to be significant for insertion point error (P=0.04), target point error (P=0.03), and orientation error (P=0.01).

B. Sawbone Injection Experiment

The Hausdorff distance heatmap representing the surface distance error between the injected PMMA and planned injection spheroid in sawbone sample #1, is shown in Fig. 14(b). The root mean square surface distance error is calculated as 1.00 mm with the maximum Hausdorff distance of 3.37 mm.

C. Cadaveric Drilling Experiment

The comparison between the measured and planned trajectories in drilled lower torso specimen is shown in Fig. 10(c). The l₂ distance errors between planned and measured insertion and target points in drilled lower torso specimen are calculated as 3.28 mm and 2.64 mm, respectively and the drill line orientation error, δθ is 2.30°.

The biomechanical simulation predicts a 33% increase of yield load for the right femur of the lower torso specimen with the FE-optimized cement injection pattern, simulating a side-way fall on the greater trochanter. Based on the measured target point of the drill trajectory from the post-op CT scan, we shift the cement injection blobs inferior and perform the SPH simulation. FE simulation predicts a 26% increase of the yield load with the injection pattern of the post-op trajectory.

D. Cadaveric Integrated Drilling and Injection Experiment

We present the intra-operative 2D/3D registration overlay images for the augmented lower torso specimen in Fig. 12. C-arm X-ray images are taken from three different views, which are used to jointly register the femur and injection device. The precise match of the DRR edges and X-rays in multi-view geometries indicate an accurate pose estimation in 3D. The X-ray images of drilling and few intermediates of the injection in augmented lower torso specimen are illustrated in Fig. 13. The ICP registration that aligns post-op cement surface onto the simulated one yields the numbers of Ψ = 0.90°, Θ = 6.50° and Φ = 0.24° (X-Y-Z Euler rotation angles) which denote the rotational errors and δd = 4.47 mm, the translational error. Fig. 14(a,b) illustrates the segmented cement isosurfaces before and after ICP registration. The Haudroff distance heatmap indicating the surface distance error between the registered, segmented injected PMMA and planned injection blob is also shown in Fig. 14(c). The mean surface distance error is reported as 2.13 mm with the maximum Hausdorff distance of 7.20 mm. To verify this, we also calculate the mean surface distance error between two isosurfaces of the injected and planned cement as 2.46 mm using the method describe in [8], by measuring the distances between each vertex of one surface to the closest on the other one.

Fig. 12.

Overlay illustrations of registration convergence in the cadaveric integrated drilling and injection experiment. Multi-view cadaver X-ray images are placed as background and DRR-derived edges are overlaid in green. Top row shows the femur registration, and the bottom row shows the tool (DI) registration.

Fig. 13.

X-ray images of (a) the guided drilling, (b) the drill trajectory with 4 mm drill bit, (c,d,e) three intermediates of the cement injection and (f) cement profile in the cadaveric integrated drilling and injection experiment.

FE simulation predicts a 39% increase of the yield load for the osteoporotic right femur of the lower torso specimen (from 1420 N to 1975 N) with 9.97 ml of cement injection in the superior and inferior aspect of the neck, as well as supero-posterior aspect of the greater trochanter. Furthermore, the predicted yield load of the augmented specimen based on the post-injection CT scan is calculated as 1965 N, which is similar to the predicted value based on the plan (1975 N).

VII. Discussion

Augmentation of the proximal femur using PMMA (femoroplasty) has been identified as an effective preventive approach to reduce the risk of the fracture by improving the yield load and yield energy of the proximal femur [5], [6], [8], [9]. We have previously developed a computer-assisted biomechanical planning and surgical workstation containing an optically tracked navigation system, a navigated hand-held drill and an automatic injection device for navigated execution of femoroplasty. The intraoperative execution of the pre-operative biomechanical planning involves bone registration followed by navigation of a cordless hand drill using the tracking rigid body, mounted to the hand drill. The navigation system visually guides the user to manually align the drill to the desired insertion point of the drill trajectory by providing real-time feedback of the distance and angle errors of the drill alignment defined by the pre-operative plan [8], [9].

In the recent surgical implementation of patient-specific biomechanical planning paradigm, the translational error of 6.2 mm was reported between the planned and injected volume of the PMMA [9]. Moreover, the distance error of < 7 mm and rotation error of < 6° in cannula placement and navigation and cement shape error of < 2 mm were reported in [8]. One of the sources of these errors was manual alignment of the drill to the planned trajectory and subsequently manual placement of the cannula tip that is mounted in the injection device. The other source of the error was manual drilling task due to the potential inaccuracies caused by human visual feedback and human hand instability.

With the main goal of improving dexterity and accuracy, this paper presents the first robot-assisted surgical intervention for augmentation of the proximal femur. The system includes a positioning robot and an integrated drilling and injection component (DI), which is designed and fabricated to perform both bone drilling and cement injection tasks with one-time simultaneous registration of the anatomy and tool. This reduces errors caused by changing tools (e.g., tool calibration and registration); moreover, using a single component, the time needed for switching between tasks and inserting cannula is substantially reduced, leading to shorter overall surgery time.

In sawbone pilot experiments, similar to previous manual drilling [8], [9], we use optically tracked navigation system for the femur and DI registration. The distance average errors between planned and measured insertion and target points of the drill trajectory have been decreased by 41% and 43%, respectively with the use of robot-assisted drilling compared to the manual navigated drilling. The orientation average error has been reduced by 72%. In addition, the analysis of differences between the planned and injected PMMA volumes demonstrate the decrease of about 56% of surface distance error between the two volumes. Although the accuracy of drilling and cement injection have been increased substantially, the errors reported in this study (3.64 mm for insertion, 3.78 mm for target, 1.77° for orientation and 1.00 mm for cement shape error) are mainly due to the kinematics inaccuracy of the positioning robot. However, in the cadaver experiments, to remedy this error, we implement optical tracker feedback control for more precise robot positioning.

In real surgical scenario of femoroplasty, due to the existence of the soft tissue, X-ray based 2D/3D registration was proposed for anatomy and tool co-registration [17], [16]. In this study, we successfully integrate the image-based 2D/3D registration pipeline into the surgical robotic system and evaluate it in a cadaver experiments. In the drilling experiment, the entry point error (3.28 mm) is slightly higher than the cadaveric registration tip error reported in [17] (2.64 mm). The potential reasons include systematic errors introduced during hand-eye calibration; optical tracker jig frame detection error; difference between the simulated DRR and real X-ray images caused by imperfect bone CT segmentation, and X-ray spectrum and exposure simulations. On the other hand, the target point error is slightly lower (2.64 mm). Moreover, the translational error between the post-op and simulated cement surfaces (4.47 mm), in the injection experiment, is mainly due to the aforementioned sources of the error, however; the translational error is 35% lower than the error reported in [8]. The orientation error (2.30°) between planned and measured drill lines (the drilling experiment) and the corrective rotation (ϴ) along Y axis (6.50°) that aligns post-injection cement surface to the simulated one, (the injection experiment) are likely due to the femur registration error. Because the femur surface is smooth and featureless, it is challenging to accurately estimate the axial rotation of the femur. Another potential source of error is the movement of the cadaver during drilling procedure. To mitigate this, in future, we can consider implementing multiple iterations of 2D/3D registration throughout the procedure as our registration method is relatively fast and can be automated [19], [17]. This will help in reducing the errors caused by potential cadaver motion. On the other hand, two additional corrective rotations (Ψ) and (Φ) along X and Z axes, have been substantially decreased by 85% and 95%, respectively with the use of RADIS compared to the rotational errors with the use of manual drilling and automatic injection device [8]. Furthermore, the cement shape discrepancy reported in this study (2.13 mm) is correlated to the error reported in [8]. The potential factors that contribute to the cement shape error include the slightly lower volume of the injected cement compared to the plan volume and model simplifications for the cement viscosity [37]. For future experiments, the viscosity-time calibration curve for the available ”Simplex P” bone cement, needs to be further examined.

We previously have shown that measured yield loads after the augmentation correlate well with those of simulations (R² = 0.77) [8], [9]. To evaluate the impact of the drill and cementation accuracies on the fracture-related biomechanical outcome of femoroplasty, we implement FE analysis on the drilled and augmented specimens, developed from post-op CT scans. For this purpose, we perform hydrodynamics simulation on the shifted cement profile based on the post-op trajectory measurements in the drilled specimen. As a result, the estimation of the yield load improvement has been reduced only 7% compared to the original planned trajectory. Moreover, the comparison between the predicted yield load of the augmented specimen, calculated using Post-Op CT scan and the predicted yield load based on the biomechanical plan (1965 N compared to 1975 N), demonstrate the sufficient accuracy and superior performance of the RADIS for execution of osteoporotic hip augmentation. In addition, improving the registration accuracy will consequently enhance the post-operative biomechanical outcome of femoroplasty.

The current study represents the first cadaver studies with intact soft tissues, investigating the use of our novel robot-assisted bone augmentation surgical workstation for combined drilling and injection. The results of the robot-assisted drilling demonstrate a promising improvement over the navigated user drilling with visual feedback. In addition, in the study of Farvadin et al., it was demonstrated that utilizing curved patterns for cementation reduces the volume of cement injection to an average of 7.2 ml while substantially increasing the yield load of the femur by 69% [43]. Future work involves using curved drilling technique with a custom designed continuum manipulator that was previously evaluated in [23], [44], [45] for optimal curved pattern of cement injection. Furthermore, the error analysis on the resulting cement profile of the augmented lower torso specimen, shows the increased accuracy of the cementation. In addition, the increased ease of use of robot-assisted integrated drilling and injection system compared to the manual drilling and automatic injection device, will contribute in successful application of femoroplasty surgery as a routine clinical procedure. We note that in the future, our image guided, robot-assisted combined drilling and injection system need to be further investigated through additional cadaver experiments, followed by mechanical testing to failure simulating a fall to the side on the greater trochanter to experimentally evaluate the yield load after the augmentation.

The image-based navigation of our robotic system requires exposing the patient to fluoroscopic radiation. The proposed approach takes about ten X-ray images for 2D/3D registration. Because fluoroscopy is commonly used in orthopedic surgeries to verify the pose of the surgical tools, this amount of radiation on older patients with osteoporotic bone is acceptable.

VIII. Conclusion

In this paper, we present a surgical system for robot-assisted femoroplasty with integrated drilling and injection approach for execution of osteoporotic hip augmentation, where the trajectory is planned based on the pre-operative robust biomechanical planning. The performance of the RADIS is evaluated through cadaver experiments with intact soft tissues to mimic the real surgical scenario using multi-view X-ray based 2D/3D registration. The results suggest the superior accuracy and efficacy of image-guided, robot-assisted bone augmentation system, which not only enhances the overall safety of the augmentation procedure, but also has a broad application in minimally invasive orthopedic interventions where drilling is combined with injection or insertion of the medical implants.

Fig. 11.

(a) Overlay of the planned injection spheroid (red) and segmented injected PMMA (blue). (b) Heatmap of Hausdorff distance in injected cement blob. The injected mesh is sampled and for each sample, the closest point over the planned injection mesh (grey spheroid) is found. Warmer colors represents greater differences between two meshes.