Fluoroscopic Guidance of a Surgical Robot: Pre-clinical Evaluation in Pelvic Guidewire Placement

Abstract

Purpose:

A method and prototype for a fluoroscopically-guided surgical robot is reported for assisting pelvic fracture fixation. The approach extends the compatibility of existing guidance methods with C-arms that are in mainstream use (without prior geometric calibration) using an online calibration of the C-arm geometry automated via registration to patient anatomy. We report the first preclinical studies of this method in cadaver for evaluation of geometric accuracy.

Methods:

The robot is placed over the patient within the imaging field-of-view and radiographs are acquired as the robot rotates an attached instrument. The radiographs are then used to perform an online geometric calibration via 3D-2D image registration, which solves for the intrinsic and extrinsic parameters of the C-arm imaging system with respect to the patient. The solved projective geometry is then be used to register the robot to the patient and drive the robot to planned trajectories. This method is applied to a robotic system consisting of a drill guide instrument for guidewire placement and evaluated in experiments using a cadaver specimen.

Results:

Robotic drill guide alignment to trajectories defined in the cadaver pelvis were accurate within 2 mm and 1° (on average) using the calibration-free approach. Conformance of trajectories within bone corridors was confirmed in cadaver by extrapolating the aligned drill guide trajectory into the cadaver pelvis.

Conclusion:

This study demonstrates the accuracy of image-guided robotic positioning without prior calibration of the C-arm gantry, facilitating the use of surgical robots with simpler imaging devices that cannot establish or maintain an offline calibration. Future work includes testing of the system in a clinical setting with trained orthopaedic surgeons and residents.

I. INTRODUCTION

Pelvic fracture fixation involves the insertion of Kirschner guidewires (K-wires) and cannulated screws to stabilize the pelvic anatomy following fracture reduction. Challenges stem from the complex 3D shape of the pelvis, difficulties in identifying trajectories from 2D radiographic imaging,¹ and the low tolerance for cortical breach in narrow bone corridors with close proximity to vasculature and nerves.² Robotic-assistance systems have the potential to assist with fracture fixation by providing a drill guide through which guidewires can be delivered.³ This task can be achieved by registering the robot to the patient (e.g., most commonly using an optical surgical tracker), and then driving the robot to a preoperatively defined trajectory plan in the patient CT.⁴

The use of robotic assistance in pelvic fracture fixation is expected to improve the accuracy and efficiency of the procedure, while also reducing radiation exposure to the patient and surgeon that is typical of the current standard of care (viz., intraoperative fluoroscopy).³ However, existing robotic assistants rely on optical tracking to register the robot to the patient, requiring lengthy setup of external markers and unobscured line-of-sight between the camera and markers.^5,6,7 These workflow challenges are exacerbated in the pelvic trauma OR, since the complex shape of the pelvis demands diverse approaches and patient positioning for instrument placement. For these reasons, optical tracking – and robotic systems that use optical tracking – has not seen broad adoption in pelvic trauma surgery, despite emerging use of robot assistance in other related procedures, such as in implanting cranial neuroelectrodes and placing spinal pedicle screws.^3,8

Image-guided robots provide an alternative to optical tracking, relying instead on imaging that is routinely acquired throughout the surgical workflow, to register the robot with the patient.⁴ In the context of pelvic trauma surgery and intraoperative fluoroscopy, this may be achieved using 3D-2D registration of 3D models of the patient and robot (a preoperative CT and CAD model, respectively) to 2D radiographs capturing the patient and robot. A previously developed solution performs the patient and robot registrations without movement of the C-arm gantry (viz., “single-view” registration).⁹ The method takes advantage of the large size and complex shape of the pelvis to register the patient from a single radiographic perspective and uses encoded robot motion within the imaging FOV to obtain disparate views of the robot instead of using encoded C-arm gantry motion. Despite avoiding motion of the gantry, this method still required prior calibration of the projective geometry, selected based on the encoded angular position of the C-arm.

Presented work develops a “calibration-free” approach that offers to extend image-guided robotics to simpler C-arm imaging devices that are in mainstream use, which commonly are not encoded and/or not mechanically stable to allow prior calibration. The method performs an online calibration¹⁰ of the C-arm, using the patient (and their preoperative image) as the calibration object. The established projective geometry is then used to perform robot registration. In this work, we present the calibration-free approach and demonstrate its feasibility in preclinical cadaver studies simulating pelvic guidewire placement in comparison to the performance of the calibrated approach.

II. METHODS

2.1. Single-View Image-guided Robotic Positioning

Robotic alignment of a surgical instrument with a target trajectory planned in the patient preoperative CT involves: (1) acquiring projections of the patient and the robot (specifically, the attached instrument); (2) registering the patient and robot to the C-arm imaging device; (3) calculating the target pose; and (4) servoing the robot such that the instrument aligns with the target. A workflow for achieving (1) was previously presented in which the C-arm gantry remains stationary (“single view”),⁹ and projections of the patient and robot are acquired as the robot precisely moves the attached instrument between projections. The method takes advantage of the large and complex shape of the pelvis, allowing registration of the patient CT to the C-arm imaging device despite having only a single perspective of the pelvis in the projections. The patient registration is solved by optimizing:

$$T_{\text{carm}}^{\text{ct}}=\underset{T}{\mathrm{arg\; max}}\sum _{i=1}^N\text{GO}\left[p_i,{\widehat{p}}_i\left(V_{\mathrm{ct}},T\right)\right]$$ (1)

where N is the number of projection images (p_i) acquired. Digitally-reconstructed radiographs (DRRs, ${\widehat{p}}_i$) are generated by rigidly transforming the preoperative CT, V_ct, by T and computing the line integrals for each projection. The gradient orientation (GO) similarity metric is chosen due to its robustness to “content mismatch” between the CT and projections.¹¹ The objective function is optimized using the covariance matrix adaptation evolution strategy (CMA-ES) in a multi-resolution framework.

In comparison to the pelvis, robotically held instruments are typically too small and simple in shape to be registered accurately in 3D from a single perspective; instead, encoded motion of the robot in between projections allows the acquisition of disparate views of the instrument. The disparity between the pose of the instruments is captured in the C-arm coordinate frame via the projections and can be related to the robot coordinate frame and used to optimize an objective function:

$$T_{\text{carm}}^{\text{inst}}=\underset{T}{\mathrm{arg\; max}}\sum _{i=1}^N\text{GC}\left[p_i,{\widehat{p}}_i\left({\kappa }_{\text{inst}},T{\left(T_{\text{end}}^{\text{inst}}\right)}^{-1}{\left(T_{0_{\mathrm{base}}}^{\text{end}}\right)}^{-1}{T}_{i_{\mathrm{base}}}^{\text{end}}{T}_{\text{end}}^{\text{inst}}\right)\right]$$ (2)

where p_i is the projection taken at robot pose i, ${\widehat{p}}_i$ is the DRR corresponding to robot pose $T_{i_{\mathrm{base}}}^{\text{end}}$, $T_{0_{\mathrm{base}}}^{\text{end}}$ is the first robot pose, and $T_{\text{end}}^{\text{inst}}$ is an offline calibration that relates the 3D instrument model (κ_inst) coordinate frame to the end effector coordinate frame of the robot. The gradient correlation (GC) similarity metric is chosen since it favors high-intensity gradients that are characteristic of metallic surgical instruments.¹² With the pose of the patient and robot resolved in the C-arm coordinate frame (2), the target robot pose can be calculated as follows:

$${\widehat{T}}_{\text{base}}^{\text{end}}=T_{\text{base}}^{\text{end}}{T}_{\text{end}}^{\text{inst}}{\left(T_{\text{carm}}^{\text{inst}}\right)}^{-1}{T}_{\text{carm}}^{\text{ct}}{T}_{\text{ct}}^{\text{plan}}{T}_{\text{plan}}^{\text{inst}}{\left(T_{\text{end}}^{\text{inst}}\right)}^{-1}$$ (3)

where $T_{\text{ct}}^{\text{plan}}$ is the preoperative trajectory plan defined in patient preoperative CT. Figure 1 summarizes the single-view algorithm for driving the robot to a target trajectory.

Figure 1.

Single-view image-guided robotic positioning. The robot is placed over the patient and radiographs are acquired as the robotically-held instrument rotates over the patient. These radiographs of the patient and robot are used to localize one with respect to the other. The robot is then servoed to a preoperatively planned trajectory by following the transformation tree.

2.2. Calibration-Free Robotic Positioning

Equations (1) and (2) involve a forward projection of the 3D patient preoperative CT and 3D instrument CAD model respectively, in order to generate DRRs for registration. This forward projection is generated using projective geometry of the C-arm that is obtained through a prior, offline geometric calibration of the system. By using this calibration, Equations (1) and (2) optimize the rigid parameters (6 degrees of freedom [DoF]) that describe the pose of the patient preoperative CT and instrument CAD model (respectively). However, to calibrate the system, the C-arm must be encoded and repeatable in its motion profile – the gantry should not flex or bend at different C-arm poses in an inconsistent fashion.

When such a C-arm is not available, a novel method has been developed to extend the usability of image-guided robotic positioning for non-encoded C-arms or C-arms that cannot be calibrated. The method first performs an online calibration of the C-arm from a single radiograph of the patient, giving the C-arm projective geometry in relation to the patient. The projective geometry can be described in the form of a projection matrix PM,¹⁰ which can be decomposed in terms of the 9 DoF describing the source position (T_s = [T_s,x, T_s,y, T_s,z]^T), detector position (T_d = [T_d,x, T_d,y, T_d,z]^T), and detector rotation (R_d = [R_d,x, R_d,y, R_d,z]^T) as illustrated in Figure 2.

Figure 2.

Illustration of geometry and transforms for (a) calibrated and (b) calibration-free systems. (A) Calibrated registration, which solves for the rigid parameters (6 DOF) of the object o (e.g., patient preoperative CT) with respect to the calibrated C-arm imaging system. (B) Calibration-free registration, which solves for the intrinsic (T_s) and extrinsic (T_d, R_d) parameters of the C-arm.

The projection matrix PM is composed of an intrinsic matrix PM_int that describes the source position, and extrinsic matrix PM_ext that describes the detector position:

$$\mathit{P}{\mathit{M}}_{\mathbf{\text{int}}}=\left[\begin{array}{cccc}{T}_{s,z}& 0& T_{s,x}& 0\\ 0& T_{s,z}& T_{s,y}& 0\\ 0& 0& 1& 0\end{array}\right]$$ (4a)

$$\mathit{P}{\mathit{M}}_{\mathbf{\text{ext}}}=\left[\begin{array}{cc}& T_{d,x}\\ {\mathit{R}}_{\mathbf{3}\mathit{x}\mathbf{3}}\left(R_{d,x},R_{d,y},R_{d,z}\right)& T_{d,y}\\ & T_{d,z}\\ 000& 1\end{array}\right]$$ (4b)

$$\mathit{P}\mathit{M}=\mathit{P}{\mathit{M}}_{\mathbf{\text{int}}}\mathit{P}{\mathit{M}}_{\mathbf{\text{ext}}}$$ (4c)

To solve for the calibrated projection matrix $\widetilde{\mathit{P}\mathit{M}}$, the following function is optimized:

$$\widetilde{\mathit{P}\mathit{M}}=\underset{\mathit{P}\mathit{M}}{\mathrm{arg\; max}}\text{GO}\left[p_i,{\widehat{p}}_i\left(V_{\text{ct}},{\widehat{T}}_{\text{carm}}^{\text{ct}},\mathit{P}\mathit{M}\right)\right]$$ (5)

where ${\widehat{T}}_{\text{carm}}^{\text{ct}}$ is an initial volume transform for the patient preoperative CT. After $\widetilde{\mathit{P}\mathit{M}}$ is calculated, the pose of the patient with respect to the C-arm can be calculated as:

$$T_{\text{carm}}^{\text{ct}}={{\widehat{\mathit{PM}}}_{\mathbf{\text{ext}}}}^{-1}{\widetilde{\mathit{PM}}}_{\mathbf{\text{ext}}}{\widehat{T}}_{\text{carm}}^{\text{ct}}$$ (6)

where ${\widehat{\mathit{P}\mathit{M}}}_{\text{ext}}$ is the initial extrinsic matrix used during the optimization of Equation (5). To solve for the robot registration, Equation (2) is used with the obtained calibration PM.

2.3. Preclinical Evaluation of Drill Guide Alignment

The Cios Spin 3D C-arm (Siemens Healthineers, Erlangen, Germany) was used for experiments involving image-guided robotic positioning. The system has an isocentric gantry and is capable of acquiring 3D CBCT reconstructions for ground truth pose definition of delivered instruments, but for calibration-free registration, the encoders and system calibration were not used. Experiments used a UR5 general purpose robot (Universal Robots, Odense, Denmark) as a general platform for preclinical studies, offering a 6 DoF manipulator with a maximum payload of 5 kg, consisting of a base, shoulder, elbow, and 3 wrist joints. The instrument used in the studies was a standard surgical drill guide through which guidewires and screws can be delivered.

2.3.1. Data Collection

A cadaver torso was used to assess the workflow and performance of the system in a realistic operating setting. Six preoperative fixation plans (and their associated bone corridors) were designated in the preoperative CT of the pelvis. For each planned trajectory, the cadaver was positioned such that the entry point of the trajectory was approximately at isocenter and the robot was positioned over the patient in the FOV. A cone-beam CT (CBCT) containing 400 projections was acquired at 0.3 mAs/projection for the purposes of ground-truth localization of the patient and robot. Additionally, five fluoroscopy images were acquired at 0.3 mAs, with 5 rotations of the robot end effector at 30° increments, providing images for robot registration as well as calibration-free registration. For each workflow, patient and robot registration was performed using the acquired data and used to drive the robot according to Equation (3). After driving, a CBCT of the aligned drill guide was acquired for ground-truth definition.

2.3.2. Outcome Metrics

For each trajectory, ground-truth $\left({\mathbb{T}}_{\mathit{carm}}^{\mathit{ct}}\right)$ for patient pose was defined from 3D-2D registrations of CT to a large number of projections from the projection datasets. Registration error was estimated by calculating the difference from ground truth.

$${\delta }_x={\Vert T_{\mathit{carm}}^{\mathit{ct}}{\left({\mathbb{T}}_{\mathit{carm}}^{\mathit{ct}}\right)}^{-1}\Vert }_4$$ (7)

To evaluate the geometric accuracy of drill guide alignment, each CBCT of the aligned drill guide was registered to the patient preoperative CT. Two points were manually designated to delineate the trajectory of the drill guide in CBCT (as shown in Figure 3) and its intersection with the cortical bone, surface. This point was defined as (t₁) and the trajectory plan entry point was defined as (p₀) and this delineated trajectory was defined as x_t, and entry point error was defined as:

$${ϵ}_x=\Vert p_0-t_1\Vert$$ (8)

Figure 3.

Pre-clinical testing of calibration-free, image-guided robotic positioning. (a) Preclinical system for image-guided robotic positioning of a drill guide. (b) Calculation of outcome metrics from post-driving CBCT of aligned drill guide.

The angular deviation ϵ_θ between the unit vector trajectory axis $(\stackrel{\rightharpoonup }{t})$ and unit vector planned trajectory axis $(\stackrel{\rightharpoonup }{p})$ was evaluated as:

$${ϵ}_{\theta }={\cos }^{-1}(\stackrel{\rightharpoonup }{t}·\stackrel{\rightharpoonup }{p})$$ (9)

Additionally, each delineated trajectory was extrapolated through a surface mesh representation of the patient pelvis, and the distance of the extrapolated trajectory to the cortical bone surface at discrete points along the trajectory was evaluated to determine whether a K-wire delivered through the aligned drill guide would breach the cortex.

III. RESULTS

3.1. Geometric Accuracy of Instrument Alignment

The geometric accuracy of drill guide alignment was evaluated by determining the patient and robot registration errors, followed by the tip point and axis alignment error of the drill guide. Figure 4a shows the patient registration and robot registration errors δ_x for all trajectories. The results demonstrated the accuracy of registering a robot and patient with no prior calibration of the C-arm compared to a calibrated approach, with patient registration error of 1.2 ± 0.2 mm for the calibration-free approach compared to 0.7 ± 0.2 mm for the calibrated approach, and robot registration of 1.2 ± 0.5 mm compared to 1.0 ± 0.71 mm. The accuracy of drill guide alignment was evaluated for the calibrated C-arm approach and calibration-free C-arm approach, as seen in Figures 4b and 4c. Drill guide tip and axis alignment error for the calibration-free approach was 1.8 ± 0.8 mm and 0.7 ± 0.2°, compared to 1.4 ± 0.7 mm and 0.5 ± 0.3° for the calibrated approach. Both methods achieve average drill guide tip error < 2 mm and axis alignment error < 1°.

Figure 4.

Geometric accuracy of robotic positioning with and without C-arm calibration. (A) Patient and robot registration accuracy for the calibrated and calibration-free approaches. (B,C) Drill guide tip and axis alignment error for both approaches.

Figure 4 shows that while there are marginal increases in error for patient registration, robot registration, and drill guide alignment, all errors remained low (< 2 mm and 1°). The modest increases in error can be attributed to the fact that the calibration-free approach is designed for simple, non-encoded C-arms, and performs an online calibration for such a system using the patient as a calibration object (cf. a more extensive calibration method such as Cho calibration¹³ involving a calibration-specific phantom). It is likely that resulting calibration therefore has more error, and patient and robot registration errors will be higher and this error will propagate into the drill guide tip and drill guide axis alignment.

3.2. Conformance of Aligned Instrument Trajectory Within Bone

Figure 5 shows that for all 6 trajectories, a simulated K-wire extrapolated from the aligned drill guide did not breach the pelvis, confirming the performance of the calibration-free approach compared to the calibrated approach. The results suggest that despite the marginal increases in registration and drill guide placement error when using the calibration-free approach, these errors do not cause clinically significant breach of the cortical bone.

Figure 5.

Distance of simulated K-wires from the cortical bone surface. A K-wire mesh model was propagated along the aligned drill guide and the distance of the K-wire mesh model to a mesh model of the cadaver pelvis was evaluated at discrete positions along the length of the K-wire model. Values below the red line (y=0) indicate breaches of the simulated K-wire out of the cortical bone. Positive values indicate conformance of the simulated K-wire within the bone corridor. Six trajectories with various approaches in the pelvis were planned and executed. The black curve shows the preoperative trajectory plan and its distance to the cortical bone surface along the length of the trajectory. The light blue curve shows the calibrated workflow and its cortical bone distance, while the gray curve shows the calibration-free workflow.

IV. DISCUSSION AND CONCLUSIONS

The experiments performed in this work assessed the accuracy of image-guided robotic positioning in a cadaver specimen while assuming no prior calibration of the C-arm, achieving 3D guidance by first calibrating the system via a registration of patient anatomy and then registering the robot using the obtained projective geometry. This approach facilitates use of commonly used C-arms that cannot maintain a prior offline geometric calibration to register the robot to the patient. The results demonstrated that this approach can accurately align a robotic drill guide to target trajectory plans in realistic preclinical setting in cadaver, achieving drill guide tip error and axis alignment error of 1.8 ± 0.8 mm and 0.7 ± 0.2°. Current work on robot pose and path planning are also being implemented to avoid collision of the drill guide with the patient or OR equipment. This work establishes essential preclinical validation, with future work prior to clinical translation involving a cohort of residents and orthopaedic surgeons to more fully establish the safety of the system, obtain user feedback on interaction with the surgical interface, and measure effects that impact workflow, such as radiation dose and runtime.