Robotic drill guide positioning using known-component 3D–2D image registration

Abstract.

A method for x-ray image-guided robotic instrument positioning is reported and evaluated in preclinical studies of spinal pedicle screw placement with the aim of improving delivery of transpedicle K-wires and screws. The known-component (KC) registration algorithm was used to register the three-dimensional patient CT and drill guide surface model to intraoperative two-dimensional radiographs. Resulting transformations, combined with offline hand–eye calibration, drive the robotically held drill guide to target trajectories defined in the preoperative CT. The method was assessed in comparison with a more conventional tracker-based approach, and robustness to clinically realistic errors was tested in phantom and cadaver. Deviations from planned trajectories were analyzed in terms of target registration error (TRE) at the tooltip (mm) and approach angle (deg). In phantom studies, the KC approach resulted in $\mathrm{TRE}=1.51\pm 0.51\mathrm{mm}$ and $1.01\mathrm{deg}\pm 0.92\mathrm{deg}$, comparable with accuracy in tracker-based approach. In cadaver studies with realistic anatomical deformation, the KC approach yielded $\mathrm{TRE}=2.31\pm 1.05\mathrm{mm}$ and $0.66\mathrm{deg}\pm 0.62\mathrm{deg}$, with statistically significant improvement versus tracker ($\mathrm{TRE}=6.09\pm 1.22\mathrm{mm}$ and $1.06\mathrm{deg}\pm 0.90\mathrm{deg}$). Robustness to deformation is attributed to relatively local rigidity of anatomy in radiographic views. X-ray guidance offered accurate robotic positioning and could fit naturally within clinical workflow of fluoroscopically guided procedures.

1. Introduction

Accurate screw placement within the vertebrae is essential in spinal fixation, an increasingly common spine surgical procedure with a reported growth of 70% in the last decade.¹ Pedicle screws form a stable foundation for various surgical constructs by providing rigid anchorage to the bone. Screw placement involves targeting a narrow bone corridor within the vertebral body (pedicle), in proximity to critical structures, repeated for multiple vertebral levels. Successful placement is achieved when a screw is secured entirely within the pedicle. Screw malplacement, commonly indicated by breaches of $>2\mathrm{mm}$ outside the bony anatomy, has been reported to be 5% to 41% in the lumbar spine and 3% to 55% in the thoracic spine.² Although a small fraction of such malplacements are associated with significant comorbidity, due to the potential for neurovascular damage³ and costly revision surgeries,⁴ multiple approaches have been explored to improve placement accuracy.^5,6

Manual placement of pedicle screws often involves x-ray fluoroscopy guidance for identification of anatomical features and assessment of screw location and depth from two-dimensional (2D) projection views. More recently, CT-based navigation (using preoperative CT, intraoperative CT, or intraoperative cone-beam CT) has enabled higher levels of precision in pedicle screw placement.⁶ In particular, meta-analysis reviews report higher rates of screws fully contained within pedicles for navigation-assisted placements (mean 95.2%) versus placement without navigation (mean 90.3%).⁷ Conversely, while surgical tracking technologies for navigation—e.g., in cranial neurosurgery—are well established, their adoption has been low in procedures that use fluoroscopy—e.g., in spinal neurosurgery or pelvic trauma—in part due to complexity compared with existing clinical workflow, the requirement for additional equipment, line-of-sight, and potential errors induced by anatomical deformation.^8,9 Such limitations motivate development of alternative forms of guidance that could be used even in the absence of additional tracking systems to improve performance, enable broader adoption, and streamline the use of other advanced systems—such as surgical robots—in a manner that better integrates with the existing clinical workflow.

Motivated by the requirement for high accuracy in targeting the pedicle and procedural repetitiveness in fusing multiple vertebrae, this work proposes a process for robotic positioning of a pedicle drill guide via x-ray guidance, thus presenting the surgeon with an accurately placed port through which other surgical instruments (e.g., K-wires or screws) may be delivered. The proposed approach builds on a 3D–2D registration algorithm that incorporates information on device shape [referred to as a known component (KC)] to solve for its three-dimensional (3D) pose from 2D radiographs. In addition to using images already acquired in existing fluoroscopic workflow, the robotic approach has the extra benefit of keeping surgeons’ hands outside the x-ray field of view (FoV) during imaging.¹⁰ The proposed method was evaluated in phantom and cadaver studies, and the geometric accuracy was quantified in comparison with a more conventional approach in which robot positioning is guided by an optical tracker.

2. Methods

2.1. Known-Component Registration

The primary means for guidance is a recently developed algorithm [known-component registration (KC-Reg)], which uses a two-step process to perform a 3D–2D registration of the patient anatomy (represented by a preoperative CT volume, $v$) to 2 to 3 radiographs (e.g., AP, lateral, and oblique), followed by the registration of a device of known design—i.e., the KC.^11,12 Registration of the anatomy uses the gradient orientation (GO) similarity metric,¹³ which has been shown to be robust against information that appears in one image but not the other (“content mismatch,” such as the presence of the drill guide or pedicle screws in the radiograph but not in the CT). GO is defined by the gradients in the fixed ($f$) radiograph and moving ($m$) digitally reconstructed radiograph (DRR) images as

$$\begin{gathered} \mathrm{GO}(f,m)=\frac{1}{N}\sum _{\nabla f>t_f,\nabla m>t_m,\alpha >t_{\alpha }}1-\frac{\ln [|\alpha (\nabla f,\nabla m)|+1]}{2}, \\ \text{such that},\alpha (f,m)={\cos }^{-1}\frac{f·m}{\Vert f\Vert \Vert m\Vert }, \end{gathered}$$ (1)

where the thresholds $t_f$ and $t_m$ are selected as the median value of the respective image gradients, whereas $t_{\alpha }=45\mathrm{deg}$ ensures that only well-aligned gradients contribute to the metric. The DRRs are simulated from the patient CT volume ($v$) by line integrals (transmission/attenuation) determined by a rigid transform ($T$)

$$\mathrm{DRR}(v,T)=\underset{\overrightarrow{r}}{\int }v(T)\mathrm{d}\overrightarrow{r}.$$ (2)

Equations (1) and (2) define an objective function as follows:

$$T_c^v=\underset{T}{\arg \max }\sum _{\theta }\mathrm{GO}[R_{\theta },\mathrm{DRR}(v,T)].$$ (3)

The covariance matrix adaptation evolution strategy, a stochastic, derivative-free optimization method, was used to iteratively solve for the patient pose ($T_c^v$) in the coordinate frame of the imaging device (C-arm, $c$), chosen due to its robust convergence properties and amenability to parallelization.¹⁴

2.1.1. Drill guide as a known component

A simple pedicle drill guide tool shown in Fig. 1(a) was used to create the component computer-aided design (CAD) model ($\kappa$) in Figs. 1(b) and 1(c), where the tooltip was defined as the centroid of the distal aperture of the drill guide and the central axis aligned with the cannula. A small portion of the drill guide handle was also included in the model to account for the symmetric nature of the cannula and ensure a proper resolution of the 3D pose. Registration of the component uses the gradient correlation (GC) similarity metric,¹⁵ defined by the sum of normalized cross correlation (NCC) of orthogonal gradients in $f$ and $m$

$$\begin{gathered} \mathrm{GC}(f,m)=\frac{1}{2}\{\mathrm{NCC}({\nabla }_{x}f,{\nabla }_{x}m)+\mathrm{NCC}({\nabla }_{y}f,{\nabla }_{y}m)\} \\ \text{such that},\mathrm{NCC}(f,m)=\frac{{\sum }_i(f_i-\overline{f})(m_i-\overline{m})}{\sqrt{{\sum }_i{(f_i-\overline{f})}^2}\sqrt{{\sum }_i{(m_i-\overline{m})}^2}}. \end{gathered}$$ (4)

In this manner, the GC metric includes the high-intensity KC gradients while retaining robustness against changes in image gradients from anatomical structures (e.g., vertebrae). The DRRs for the device component (robot end-effector/drill guide) are simulated from a voxelized model generated from the CAD model, $\kappa$. Substituting $\kappa$ for $v$ in Eq. (3) and GC for GO, the pose of the component is solved for with respect to the imaging device ($T_c^{\kappa }$) by optimizing the similarity between the acquired radiographs and component DRRs

$$T_c^{\kappa }=\underset{T}{\arg \max }\sum _{\theta }\mathrm{GC}[R_{\theta },\mathrm{DRR}(\kappa ,T)].$$ (5)

Fig. 1.

(a) Simple handheld pedicle drill guide, (b) measured instrument dimensions, (c) KC-Reg component CAD model, (d) optical rigid-body marker attachment, and (e and f) sphere-in-circle attachment for pivot calibration.

Combined with the former step, a solution for the drill guide (component) pose in the coordinates of the preoperative CT scan can be computed as $T_v^{\kappa }={(T_c^v)}^{-1}{T}_c^{\kappa }$.

2.1.2. Drill guide as a tracked tool

An optical tracker (Polaris Vicra, NDI, Waterloo, Ontario) was used as a conventional alternative for guiding the robot and as a basis for comparison with the KC image-based approach. To that end, the drill guide was modified to accommodate an optical rigid-body marker, $m$, as shown in Fig. 1(d). Tooltip calibration was performed using a spherical attachment to obtain the translational offset $t_m^{\tau }$, followed by an axis calibration to obtain the rotational alignment $R_m^{\tau }$. The resulting pose of the tracked tool ($\tau$) in the coordinate frame of the optical tracker ($o$) is, therefore, $T_o^{\tau }=T_o^m[R_m^{\tau },t_m^{\tau }]$—analogous to $T_c^{\kappa }$ of the KC approach.

2.2. Hand–Eye Calibration

Not unlike the hand–eye coordination of a surgeon, an additional calibration is needed to relate the robot (UR5, Universal Robots, Odense, Denmark) end-effector to both the known ($\kappa$) and tracked ($\tau$) components. Formulation of this problem uses measurements of a target tool at multiple poses from two fixed reference coordinate frames defined as the hand $H$ and eye $E$ for the two frames, respectively. For any two sets of measurements $(i,j)$

$$\begin{gathered} H_{i}X{E}_i^{-1}=H_{j}X{E}_j^{-1} \\ (H_j^{-1}{H}_i)X=X(E_j^{-1}{E}_i), \end{gathered}$$ (6)

such that $H_i$ is a robot pose, specifically a transform from the robot base to the end-effector ($T_b^e$), and $E_i$ is a corresponding pose $T_c^{\kappa }$ or $T_o^{\tau }$, for KC and tracker guidance, respectively. Defining $A_k=H_j^{-1}{H}_i$ and $B_k=E_j^{-1}{E}_i$, the hand–eye calibration ($X$) reduces to the solution of the well-known $AX=XB$ problem.¹⁶

In the experiments below, 32 sets of randomized end-effector poses were measured within a $5\times 5\times 5{\mathrm{cm}}^3$ volume. Since time and dose are not of concern at this stage of offline calibration, the tracker measurements were averaged to reduce potential jitter, and the KC measurements used eight radially distributed projections (more than the minimum two required) to reliably solve for the component pose. A leave-one-out analysis was used to assess the performance in terms of translational deviation from the solution

$${\epsilon }_x=\{\Vert {[A_k{X}^{\prime }{(X^{\prime }{B}_k)}^{-1}]}_4\Vert |\forall k\},$$ (7)

where $A_k$, $B_k$ were not used in computing $X^{\prime }$ and ${[·]}_4$ denotes the last column (translation) of a transformation matrix. Five independent trials were obtained for each hand–eye calibration method to evaluate overall consistency of the solution relative to measurement noise/pose selection.

2.3. Robotic Alignment of the Drill Guide

2.3.1. KC guidance

Given that all prior steps described above are performed once in advance of the procedure, the remaining intraoperative task is to drive the robot to the planned trajectory $T_v^p$, defined within the preoperative CT. Using three radiographs (e.g., AP, lateral, and oblique) to simultaneously solve for $T_c^v$ and $T_c^{\kappa }$ and following the chain of transforms shown in Fig. 2 (also listed in Table 1), the desired robot end-effector pose (${\hat{T}}_b^e$) can be derived from an initial pose ($T_b^e$) as

$${\hat{T}}_b^e=T_b^e{X}_e^{\kappa }{(T_c^{\kappa })}^{-1}{T}_c^v{T}_v^p{(X_e^{\kappa })}^{-1}.$$ (8)

Fig. 2.

Illustration of [(b) blue] the transformation trees for the proposed KC and [(a) red] the more conventional tracker guidance for robotic drill guide positioning.

Table 1.

Glossary of symbols in coordinate transformations.

$v$	Preoperative image volume (CT)
$T_i^j$	Transform representing the pose of frame $j$ with respect to $i$
$c$	X-ray imaging device (Siemens Cios alpha C-arm)
$\kappa$	Known component (tip of drill guide in KC guidance)
$\tau$	Tracked component (tip of drill guide in tracker guidance)
$X$	Hand–eye calibration
$p$	Pedicle screw plan
$o$	Optical tracker (NDI Polaris Vicra)
$m$	Drill guide optical tracking markers
$e$	Robot end-effector
$b$	Robot base

To reach the target pose defined by Eq. (8), the robot traverses a path intended to provide a safe approach to the entry point. To do so, the robot assumes an intermediate pose that is aligned with the target trajectory but is offset along the trajectory by 5 to 10 cm in the direction away from the patient. From this intermediate pose, the robot is only allowed to move along the target trajectory (for both approach and withdrawal).

2.3.2. Tracker guidance

An analogous expression can be similarly defined for the tracker guidance scenario

$${\hat{T}}_b^e=T_b^e{X}_e^{\tau }{(T_o^{\tau })}^{-1}{T}_o^r{T}_r^v{T}_v^p{(X_e^{\tau })}^{-1},$$ (9)

where $T_o^r$ is the pose of a rigidly attached reference marker with respect to the optical tracker ($o$) and $T_r^v$ is a fiducial registration of the tracker to the preoperative CT image. Note that these two transforms are simply replaced by $T_c^v$ in KC guidance, which is updated with each new radiographic image acquisition and registration.

2.4. System Integration

Figure 3 shows the TREK surgical navigation platform that was used to coordinate the individual components in providing both the KC and tracker-guided robotic drill guide positioning.¹⁷ Individual modules summarized below were developed to handle tasks associated with each component (e.g., imaging device, robotic arm, 3D–2D registration, and tracking).

Fig. 3.

Screenshot of the custom surgical navigation solution based on TREK taken during cadaver experiments. Dedicated modules are shown on the left side.

The Pedicle Planner module allows trajectories to be planned within the preoperative CT as shown in green color in Fig. 3. The 3D–2D Registration module uses GPU-accelerated routines for fast computation of patient and component registrations. The C-Arm module handles communication with the imaging system (Cios Alpha, Siemens Healthineers, Erlangen, Germany), which automatically reads x-ray images as they are acquired and queries the C-arm encoder positions in real time. Similarly, the Tracking module queries transform measurements from the tracker in real time. Fiducial registration of the tracker can be readily performed via functionality that allowed measurements of a tracked pointer tool to be taken at each fiducial divot (manually presegmented in $v$). Finally, communication with the robotic arm is achieved via the UR5 module, shown as active in Fig. 3. The event-driven communication between modules yielded a streamlined and automated execution of the task, which aided in performing the experiments described below.

2.5. Experiments in Phantom and Cadaver

The feasibility and accuracy of the proposed approach were first evaluated in phantom experiments, targeting 17 pedicle trajectories spanning T1–L5 (Fig. 4). The pedicle plans were defined manually in CT images such that each trajectory had an anatomically realistic entry point and traversed the center of the pedicle corridor. Automation of trajectory planning is the subject of other work to further streamline the process.^18–23

Fig. 4.

Phantom and cadaver studies. (a) Sawbones spine phantom experimental setup, (b) zoom-in of phantom region of interest, (c) zoom-in of cadaver region of interest, and (d) cadaver experimental setup. Tracker is not shown.

For each trajectory, the robotically held drill guide was placed close to the target plan, such that both the bony anatomy and the component were captured within the x-ray FoV. Following acquisition of radiographs and registration via KC-Reg, the robot was driven to the planned trajectory via KC guidance and a cone-beam CT (CBCT) scan was acquired at the target for offline error analysis. Returning to the same initial pose, the same steps were repeated for tracker guidance. For cadaver experiments, the lumbar spine was exposed, and six trajectories spanning L3–L5 were targeted. Note that while AP, lateral, and oblique radiographs were acquired for convenience, exact views are not required and a separation of 15 deg between each view was shown to be as accurate in prior studies.²⁴ Trajectory planning and truth definition were as described above for the phantom study.

The accuracy of the system was quantified in terms of the target registration error (TRE) at the drill guide tip (${\mathrm{TRE}}_x$) and in aligning the principal component axis to target plan (${\mathrm{TRE}}_{\varphi }$)

$$\begin{gathered} {\mathrm{TRE}}_x=\Vert {[{({\mathbb{T}}_c^v)}^{-1}{\mathbb{T}}_c^{\kappa }]}_4-{[T_v^p]}_4\Vert \\ {\mathrm{TRE}}_{\varphi }={\cos }^{-1}\{{[{({\mathbb{T}}_c^v)}^{-1}{\mathbb{T}}_c^{\kappa }]}_3·{[T_v^p]}_3\}, \end{gathered}$$ (10)

where the last column (${[·]}_4$) of the transformation matrix is the translation of the tooltip, and the third column (or last column of rotation matrix) defines the principal ($z$) axis of the component. The true poses of the patient and component were established by independent 3D–3D registrations of the CT volume (${\mathbb{T}}_c^v$) and component model (${\mathbb{T}}_c^{\kappa }$) to the CBCT image reconstruction, respectively. The error windows in Figs. 7(c) and 8(c) show the screw placement range about the planned trajectory determined by the mean ${\mathrm{TRE}}_x$ and ${\mathrm{TRE}}_{\varphi }$ of drill guide alignment. The window was offset by ${\mathrm{TRE}}_x$ in any respective direction and rotationally swept by ${\mathrm{TRE}}_{\varphi }$ around the screw entry point.

Fig. 7.

Geometric accuracy of robotic guidance in the phantom study: (a) ${\mathrm{TRE}}_x$ and (b) ${\mathrm{TRE}}_{\varphi }$ for KC and tracker guided approaches, (c) error windows (computed as described in Sec. 2.5) in L4 relative to the planned trajectory of a $6\times 45\mathrm{mm}$ screw (green) based on mean errors for KC (cyan) and tracker (magenta) guidance. The planned screw entry point is marked with a dot.

Fig. 8.

Geometric accuracy of robotic positioning in the cadaver study: (a) ${\mathrm{TRE}}_x$ and (b) ${\mathrm{TRE}}_{\varphi }$ for KC and tracker guided approaches, (c) error windows in L4 relative to the planned trajectory of a $6\times 45\mathrm{mm}$ screw (green) based on mean errors for KC (cyan) and tracker (magenta) guidance. The planned screw entry point is marked with a dot.

3. Results

3.1. Assessment of Hand–Eye Calibration

Although hand–eye calibration is well established for robotic control using an optical tracker, performing it within the context of image-based guidance warrants additional study. Figure 5(a) summarizes evaluation of different solvers for the $AX=XB$ problem^25–32 in terms of the distribution of the translational component of leave-one-out errors (${\epsilon }_x$) computed as described in Sec. 2.2. The results suggest that separable solvers provided superior performance in this regard. Using the solver by Park et al., ${\epsilon }_x$ was observed to be $0.60\pm 0.25\mathrm{mm}$ [Fig. 5(b)]. A cross validation across repeat trials in Fig. 5(c) shows reduced variability in translational difference (${\mathrm{\Delta }}_x=\Vert {[X_i{X}_j^{-1}]}_4\Vert$) between hand–eye calibrations for the KC approach ($0.42\pm 0.27\mathrm{mm}$) compared with the tracker ($3.30\pm 1.59\mathrm{mm}$) approach, highlighting the improved solution consistency for the former. A possible explanation for the improved consistency of the KC approach might be its ability to capture a larger variety of drill guide poses, cf. the passive markers that were visible only from the side that faces the tracker.

Fig. 5.

(a) Distribution of translational leave-one-out errors for $AX=XB$ solvers by (from left to right) Tsai, Horaud, Park, Chou, Andreff, Daniilidis, Shiu, and Wang et al.,^25–32 (b) distribution of error (${\epsilon }_x$) for the KC approach hand–eye calibration shown as a violin plot, where the shaded region is an estimation of the probability density function of underlying sample points, and (c) translational differences between hand–eye calibration trials for the tracker (upper left) and KC (lower right) approaches.

3.2. Phantom Study: Locality of the Solution

With the optical reference marker positioned on the phantom near L5 (also common in clinical practice), an increasing trend in TRE was expected for tracker guidance for trajectories farther from the reference marker (i.e., from L5). Figure Fig. 6(a) shows such degradation in the translational component of TRE for the tracker at increasing distances from the lumbar vertebrae. KC guidance is shown to be robust to such effects due to updated registration of the patient with new acquisitions.

Fig. 6.

(a) ${\mathrm{TRE}}_x$ and (b) ${\mathrm{TRE}}_{\varphi }$ relative to the target vertebrae in spine phantom for two forms of guidance. The KC approach is observed to perform consistently across multiple vertebral levels in achieving both the target position and the approach angle. The shaded regions represent the 95% confidence interval about the regression lines.

In the spine phantom experiments, KC guidance yielded $1.51\pm 0.51\mathrm{mm}$ ${\mathrm{TRE}}_x$ and $1.01\mathrm{deg}\pm 0.92\mathrm{deg}$ ${\mathrm{TRE}}_{\varphi }$, whereas tracker guidance resulted in $2.17\pm 0.89\mathrm{mm}$ ${\mathrm{TRE}}_x$ and $0.94\mathrm{deg}\pm 0.48\mathrm{deg}$ ${\mathrm{TRE}}_{\varphi }$ as seen in Figs. 7(a) and 7(b). Although the results were comparable overall, a paired $t$-test suggests a statistically significant ($p<0.05$) error reduction in ${\mathrm{TRE}}_x$ with KC guidance. A likely reason is the (intentional) spatial distribution of registration fiducials around the lumbar region (only), as such limited coverage is fairly common in normal practice. Still, both approaches offered acceptable error (theoretical placement within pedicle cortex) as shown in Fig. 7(c).

3.3. Cadaver Study: Robustness to Deformation

The cadaver underwent realistic gross anatomical deformation in handling and transport between the CT scanner and the C-arm/operating table—positioned prone in each case. KC guidance of the robot in the cadaver resulted in $2.31\pm 1.05\mathrm{mm}$ ${\mathrm{TRE}}_x$ and $0.66\mathrm{deg}\pm 0.62\mathrm{deg}$ ${\mathrm{TRE}}_{\varphi }$ as seen in Figs. 8(a) and 8(b), slightly higher than TRE in the phantom. Although the gradients produced by the deforming soft tissue and the reduced image quality due to higher attenuation (especially in the lateral view) presented challenges, the approach successfully maintained trajectories within the bony cortex as shown in Fig. 8(c). Despite care in transformation and placement of the specimen on the table, the induced deformation from the time of preoperative CT scan and the experiment resulted in significantly ($p<0.05$) higher errors when guiding with the tracker which presented ${\mathrm{TRE}}_x$ of $6.09\pm 1.22\mathrm{mm}$ and ${\mathrm{TRE}}_{\varphi }$ $1.06\mathrm{deg}\pm 0.90\mathrm{deg}$. It is worth noting that the registration fiducials were also subject to the deformation between the preoperative CT and the time of experiment, compounding the error in tracker registration. In comparison, the KC approach remained robust to such changes, as it relies on structures that are relatively rigid, at least within the local region of the radiographic FoV (three to four vertebrae captured within a given radiographic view).

4. Conclusion

A method for robotic positioning of a pedicle drill guide via KC-Reg was presented, using fluoroscopic views already acquired within typical surgical workflow in combination with prior knowledge on patient anatomy (preoperative CT) and instrument shape (known-component model). Common practice in fluoroscopically guided orthopaedic or neuro-spine surgery involves multiplanar radiographic views that are qualitatively interpreted by the surgeon to assess 3D placement of surgical devices (e.g., to identify pedicle breach)—for example, PA and LAT views to assess position and depth in the pedicle, respectively. The KC approach accomplishes this from the same multiplanar views—or from views with as little as $\sim 15\mathrm{deg}$ angular separation²⁴—providing guidance and quality assurance that is accurate, quantitative, reproducible, and less subject to an individual surgeon’s expertise and qualitative image interpretation.

In phantom and cadaver experiments emulating pedicle screw placement, KC guidance of the robot was shown to perform comparably and, under conditions of gross anatomical deformation, better than tracker guidance. The implication is that KC guidance is less susceptible to the effects of anatomical deformation than conventional tracker-based guidance due to its use of local image features in the fluoroscopic FoV. However, a shortfall is recognized in the inability to account for patient movement between radiographic views; even with individual registrations per trajectory as in the experiments performed here, patient movement following registration is unaccounted when driving the robot. Furthermore, the time required for imaging and registration suggests a spot-check/step-and-shoot feedback in guidance rather than real-time guidance. For example, as in the current workflow of fluoroscopically guided procedures, the surgeon could acquire radiographic views at particular steps on each trans-pedicular path—e.g., just after entry, at midpoint, and/or at terminus of the trajectory. In contrast, optical trackers provide real-time feedback and account for rigid-body motion via an optical reference marker, but they are susceptible to anatomical deformation imparted over the course of surgery. The findings presented in this work, therefore, suggest the possibility of combining the KC guidance method with optical trackers to leverage the advantages of both.

One potential approach would use KC-Reg to update the tracker registration. This would require the tracked tool and KC to be in the same frame of reference, such that $T_{\kappa }^{\tau }=I$. Since the tracked tool calibration only accounts for the tooltip and the direction of cannula, the rotation about the cannula would be arbitrarily defined for tracker. A straightforward way to account for the missing degree of freedom could use the carefully obtained hand–eye calibrations, such that $T_{\kappa }^{\tau }={(X_e^{\kappa })}^{-1}{X}_e^{\tau }$. The difference between the reported tool poses for the KC and tracker approaches can then be attributed to patient deformation and/or reference marker motion, such that $T_v^{\tilde{v}}=T_v^{\tau }{(T_{\tilde{v}}^{\kappa }{T}_{\kappa }^{\tau })}^{-1}$, which describes the locally rigid offset between $v$ (used during tracker registration, prior to surgery) to current $\tilde{v}$, expected to include deformation. The offset could be used to detect anatomical deformation and alert the surgeon that the tracker registration is outdated, and even applied to update it as $T_r^{\tilde{v}}=T_r^v{T}_v^{\tilde{v}}$, which can then be substituted into Eq. (9) to drive the robot as before.

KC guidance of robotic positioning appears promising, is consistent with fluoroscopic workflow, and could help reduce intraoperative radiation dose to the patient (fewer fluoroscopic views acquired in trial and error/qualitative interpretation) and to the surgical staff (with the robot end-effector—not the surgeon’s hands—in proximity to the radiographic FoV). A variety of areas for improvement in future work should be acknowledged. Differences between the physical and virtual models compound error during both the KC hand–eye calibration process and the KC-Reg computation of drill guide pose. A higher level of agreement could be achieved either through better modeling of the component—for example, using laser scanning or vendor-provided specifications—or by custom manufacturing using 3D printing. The hand–eye calibration solvers assessed in this work include a subset of the separable and simultaneous classes; however, iterative solvers could offer lower error.¹⁶ Advanced path planning approaches,³³ collision detection,³⁴ and virtual fixtures³⁵ could be implemented to improve safety during robot motion. Finally, to further streamline workflow, the 3D–2D registration step runtime could be improved. The current implementation is such that the initial patient registration (CT-to-fluoroscopy) requires $\sim 100$ to 120 s to compute, addressing a broad search space and multistarts to ensure accurate registration of the anatomy.³⁶ The subsequent KC registrations currently do not take advantage of this initialization, and require a similar $\sim 160$ to 180 s, including the additional component registration step. The workflow implications are that the initial registration is performed once (or infrequently) and that only KC registrations are required so often as to update the robot regarding potential shifts—e.g., once every other vertebral level. KC registration runtimes can, therefore, be improved by better utilization of this broad initialization, use of alternative (local) optimizers with faster convergence time,³⁷ and utilization of more performant (or multiple) GPUs.

An alternative approach to the known-component registration used in this work is to outfit the effector with a dedicated radiopaque fiducial marker (also of known configuration) rigidly affixed to the surgical instrument.^38,39 Such an approach may be advantageous in that the fiducial marker can be optimized for the task of 3D–2D registration, potentially increasing its conspicuity, the ability to localize in a single view, and the speed of registration. However, the KC approach offers several benefits by operating without additional markers and using the robot end-effector directly. First, it avoids the need for additional tools and requisite costs of disposal or sterilization. Second, it maintains the effector’s original intended design (e.g., that of a drill guide) and is accordingly compact and reduces risk of collision. Finally, the KC approach reduces the number of calibration steps to that of the hand–eye calibration of the robot to its end-effector, whereas a radiopaque fiducial approach would require an additional (fiducial-to-effector) tooltip calibration.

Although this study focused on robotic assistance in spine surgery, KC guidance can be envisioned for robotic positioning in other procedures as well. For example, the KC guidance of stereo EEG placement could benefit not only robotic positioning⁴⁰ but also placement of the EEG leads themselves (as KCs). Similarly, the method could broaden the application of robotic assistance in procedures that currently rely heavily on fluoroscopic guidance and require a high degree of expertise on the part of the surgeon to place K-wire in complex anatomical shapes—e.g., pelvic trauma surgery.

Biographies

Thomas Yi is a research assistant at the Johns Hopkins Medical Institutes. He received his BS degree in biomedical engineering and computer science from Johns Hopkins University in 2017. His research interests include image-guided surgery and surgical robotics.

Vignesh Ramchandran is a master’s student at Johns Hopkins University in applied mathematics and statistics. He received his BS degree in biomedical engineering from Johns Hopkins University in 2017. His research interests include machine learning, compressed sensing, and medical image analysis.

Jeffrey H. Siewerdsen received his PhD in physics from the University of Michigan in 1998. He is a professor of biomedical engineering, computer science, radiology, and neurosurgery and director of the Carnegie Center for Surgical Innovation at Johns Hopkins University. His research focuses on the physics of image quality in digital x-ray, CT, and cone-beam CT and the development of systems to improve precision, accuracy, and safety in image-guided interventions.

Ali Uneri received his PhD in computer science from Johns Hopkins University in 2017. He is a postdoctoral fellow at the Biomedical Engineering Department of Johns Hopkins University. His research focuses on image registration, 3D image reconstruction, and surgical robotics for image-guided interventions.

Disclosures

This research was supported by NIH Grant No. R01-EB-017226, NIH Training Grant No. 5 T32 AR067708-03, and academic–industry partnership with Siemens Healthineers (Erlangen Germany). No conflicts of interest, financial or otherwise, are declared by the authors.