Known-component 3D-2D registration for quality assurance of spine surgery pedicle screw placement

Abstract

Purpose

A 3D-2D image registration method is presented that exploits knowledge of interventional devices (e.g., K-wires or spine screws – referred to as “known components”) to extend the functionality of intraoperative radiography/fluoroscopy by providing quantitative measurement and quality assurance (QA) of the surgical product.

Methods

The known-component registration (KC-Reg) algorithm uses robust 3D-2D registration combined with 3D component models of surgical devices known to be present in intraoperative 2D radiographs. Component models were investigated that vary in fidelity from simple parametric models (e.g., approximation of a screw as a simple cylinder, referred to as “parametrically-known” component [pKC] registration) to precise models based on device-specific CAD drawings (referred to as “exactly-known” component [eKC] registration). 3D-2D registration from three intraoperative radiographs was solved using the covariance matrix adaptation evolution strategy (CMA-ES) to maximize image-gradient similarity, relating device placement relative to 3D preoperative CT of the patient. Spine phantom and cadaver studies were conducted to evaluate registration accuracy and demonstrate QA of the surgical product by verification of the type of devices delivered and conformance within the “acceptance window” of the spinal pedicle.

Results

Pedicle screws were successfully registered to radiographs acquired from a mobile C-arm, providing TRE 1–4 mm and <5° using simple parametric (pKC) models, further improved to <1 mm and <1° using eKC registration. Using advanced pKC models, screws that did not match the device models specified in the surgical plan were detected with an accuracy of >99%. Visualization of registered devices relative to surgical planning and the pedicle acceptance window provided potentially valuable QA of the surgical product and reliable detection of pedicle screw breach.

Conclusions

3D-2D registration combined with 3D models of known surgical devices offers a novel method for intraoperative QA. The method provides a near-real-time independent check against pedicle breach, facilitating revision within the same procedure if necessary and providing more rigorous verification of the surgical product.

A Introduction

Accurate delivery of surgical devices is essential to safe, effective surgery. In spine surgery, for example, screws placed in the vertebral body through the spinal pedicle provide a foundation for spine fixation in treatment of trauma, scoliosis, spondylolisthesis, and degenerative instability, where correct placement is critical to patient safety and the biomechanical integrity of the implant construct. An estimated 488,000 spinal fusion cases are performed annually in the U.S., with 70% growth in the last decade (Weiss AJ, Elixhauser A 2014). Screw malplacement, defined as perforation (i.e., breach) of the cortex and impingement on adjacent critical structures, is observed in approximately 28–40% of surgeries (Gertzbein and Robbins 1990, Laine et al 2000). Undesirable clinical outcomes of screw malplacement include the risk of damage to nerve roots, the dural sac, vascular structures, and pleura (Attar et al 2001). Prevention and treatment of these injuries often require a secondary revision surgery, which account for 1–2% of all screw placement cases (Gautschi et al 2011), resulting in additional treatment cost and morbidity. Furthermore, the challenging nature of pedicle screw placement is evidenced by the reported maximum permissible error of 1 mm and 5° (Rampersaud et al 2001). Assessing the quality of the surgical product during the procedure is therefore integral to maximizing patient safety and minimizing the frequency of revision surgery.

The current standard of care for quality assurance (QA) of surgical screw placement involves multiple techniques, often used in combination to circumvent individual limitations. Electromyography (EMG) is a common first line of detection of neural impingement; however, EMG only detects certain types of screw malplacements that cause immediate damage to the spinal cord and is reported to have poor detection sensitivity (Kim et al 2004). Intraoperative radiography and fluoroscopy provide near-realtime feedback and allow the surgeon to correct gross malplacements during the same surgical procedure. While these intraoperative imaging techniques are commonly used for guiding screws/needles, qualitative interpretation of the surgical construct using 2D images can be difficult, subjective, and require carefully acquired views that capture relevant anatomical landmarks. (Weinstein et al 1988). Postoperative CT and/or MR has the highest reported accuracy in detecting breaches (Lehman et al 2007). However, performance is often limited by image artifacts (viz., metal artifacts in MRI and/or CT) that impede diagnostic quality (Yoo et al 1997). Moreover, postoperative QA amounts to post hoc verification wherein positive breach findings – if deemed clinically significant – are an indication that the patient undergo a secondary (revision) surgery.

Recognizing the value of accurate 3D guidance has motivated the implementation of a broad arsenal of technologies, including intraoperative 3D imaging and surgical navigation systems, which have demonstrated improved surgical accuracy and reduced complications (Laine et al 2000). Intraoperative imaging using cone-beam CT (CBCT) (Siewerdsen et al 2005, Silbermann et al 2011), helical CT (Zausinger et al 2009), or MRI (Woodard et al 2001) allows 3D visualization of the surgical product, but carry disadvantages of cost, time, patient access, workflow, and radiation dose that challenge their broad utility. Such modalities also suffer from susceptibility to the same artifacts as in their postoperative counterparts. Modern navigation systems such as surgical trackers, on the other hand, allow real-time guidance and visualization, but require additional equipment in the operating room, external markers affixed to the tools and patient, preoperative placement of extrinsic fiducials, calibration of tracked tools, manual/semi-automated patient registration, and additional intraoperative constraints such as line-of-sight or metal interference. In addition, despite their utility in image-guidance, the deterioration of accuracy over the course of surgery (e.g., due to anatomical deformation and motion of markers) reduces their reliability in verifying the final surgical product.

An alternative approach is 3D–2D registration, which provides 3D localization from intraoperatively acquired 2D images via incorporation of preoperative 3D images along with planning information (Markelj et al 2012). Such methods have recently been used in combination with statistical models of patient anatomy (Sadowsky et al 2007) and device implant models (Jaramaz and Eckman 2006). In a broad spectrum of spinal procedures, these methods can extend the functionality of intraoperative 2D imaging that already exists within the surgical arsenal, thus absolving the aforementioned difficulties associated with 3D imaging and tracking systems, and naturally integrating into existing surgical workflow. Previous work in fluoroscopic guidance demonstrated 3D registration with target registration error (TRE) <2 mm using only two fluoroscopic views (Uneri et al 2014a), acquired with as little as 1/10 the dose of standard radiographic views (Uneri et al 2014b). In the proposed approach, we combine prior knowledge of surgical tools (e.g., fixation hardware, joint prostheses, guide wires, needles, and screws) in the registration framework along with intraoperative radiographic views and preoperative CT images. Analogous use of so-called “known components” was recently proposed in the context of statistical 3D image reconstruction – namely, known-component (KC) reconstruction – in which the implant pose computed from the projection data is incorporated within a model-based iterative reconstruction method to improve 3D image quality by reducing metal artifacts (Stayman et al 2012).

In this study, a new algorithm – namely, known-component registration (KC-Reg) – is presented that combines a 3D-2D image registration method with models of known components, and the method is applied to quantitative evaluation and QA of the surgical product in spinal pedicle screw placement. The proposed method was evaluated using three model varieties of increasing fidelity in component shape: 1) a simple parametric model with a single component; 2) an advanced parametric model with multiple sub-components; and 3) an exact component model obtained from vendor-specific computer-aided design (CAD) drawings. Experiments were performed in an anthropomorphic phantom and a cadaveric torso, each emulating pedicle screw placement at different levels of the spine. The KC-Reg algorithm was assessed in terms of three tasks associated with QA of the surgical product: 1) assessment of the geometric accuracy of the surgical construct; 2) verification of the type of the implanted component; and 3) assurance that the implanted device is within acceptable error margins.

B Methods

B. 1 Registration algorithm

The proposed registration scheme extends a previously reported method (Otake et al 2012, 2013) for robust, intensity-based, rigid 3D-2D registration composed of two distinct stages as depicted in Figure 1. First, the patient's preoperative 3D and intraoperative 2D images are registered as in Figure 1a, wherein patient anatomy and the accompanying surgical plan that contain intended screw trajectories are registered to intraoperative radiographs. This is achieved by an iterative search for patient pose parameters that maximize image similarity between the acquired 2D images and digitally reconstructed radiographs (DRR) computed from the CT image. The method is the same as in (Uneri et al 2014a) and has demonstrated TRE <2 mm for two radiographic views separated by 15° or more, where 90° separation implies orthogonal biplane acquisition. Second, the surgical device (e.g., a screw, referred to generically as the “component”) is registered as in Figure 1b (detailed further in B.2.3), extending the 3D-2D registration framework to solve for the relative pose and/or model parameters of components known to be within the image. Both stages use the same radiograph(s) as input, and the output pose parameters can be composed to visualize the components in the context of either the 3D preoperative image (as in conventional surgical navigation) or as overlays on the 2D radiographs (Figure 1c).

Figure 1.

Flowchart of the KC-Reg algorithm. (a) Patient registration of the preoperative CT and intraoperative radiographs. (b) Component registration of the known component model and intraoperative radiographs. (c) Depiction of registered tools within the CT and/or radiograph allows guidance and QA of device placement relative to image and planning data.

The similarity metric employed in this work is based on pixel-wise correspondence of gradient intensity. Gradient-based measures have the inherent advantage of filtering out low spatial frequency differences, focusing instead on the boundaries of rigid bony anatomy and implanted components. Gradient correlation (GC) by (Penney et al 1998) was used in part due to its robustness against the absolute value of gradient magnitude, an important feature for images with gradients varying strongly in dynamic range (e.g., bone anatomy versus metal devices). The GC metric is defined as the average normalized cross correlation (CC) of the directional gradients of the fixed radiograph f and moving DRR m:

$$\text{GC}(f,m)=\frac{1}{2}(\text{CC}({\nabla }_{x}f,{\nabla }_{x}m)+\text{CC}({\nabla }_{y}f,{\nabla }_{y}m))$$ (1)

$$\text{CC}(f,m)=\frac{{\sum }_{i,j}(f_{i,j}-\overline{f})(m_{i,j}-\overline{m})}{\sqrt{{\sum }_{i,j}{(f_{i,j}-\overline{f})}^2}\sqrt{{\sum }_{i,j}{(m_{i,j}-\overline{m})}^2}}$$ (2)

where ∇ denotes the gradient operator, x and y are orthogonal directions in the 2D image plane, and (i, j) denote 2D image pixel coordinates.

The similarity metric was iteratively optimized using the covariance matrix adaptation evolution strategy (CMA-ES) – a derivative-free, evolutionary algorithm with robust convergence properties when optimizing rugged, non-convex objective functions (Hansen and Ostermeier 2001). The optimization problem is defined as:

$$\widehat{\mathrm{\Phi }}=\underset{\mathrm{\Phi }}{argmax}\sum _{\theta }\text{GC}(f_{\theta },m_{\theta }(\mathrm{\Phi }))$$ (3)

such that the solution is estimated as the vector of parameters Φ̂ that maximize the total GC across all projection views θ. The moving DRR m is computed as:

$$m_{\theta }(\mathrm{\Phi })={\mathcal{P}}_{\theta }\mathcal{T}(\mathrm{\Phi })\circ M$$ (4)

where the 3D image M is projected (∘) using parameters Φ, and for a given view θ. To avoid interpolation of the moving image with the current pose estimate (Φ), the extrinsic parameters of the 3D-2D projective transform _θ are modified instead. The projective transforms _θ may either come from encoded angulations θ of a calibrated C-arm, or could be solved as a parameter of the registration itself (Otake et al 2015). The slow convergence of CMA-ES was mitigated by GPU-based parallelized implementation of the projector and metric computation. The ability to simultaneously perform a large number of function evaluations per iteration (200 at ∼1440 evaluations/s) improves optimizer robustness while limiting the registration runtimes to a few seconds.

B. 2 Registration using known components

The KC-Reg method utilizes information on known components within the 3D-2D registration framework to solve for the unknown parameters that determine each component's 3D pose, shape and/or composition. In practice, the exact component information may not be available, in which case a spectrum of component model representations may be considered ranging in fidelity from simple, single-component, parametric shape models (e.g., a cylinder), to higher-order parametric models constituting multiple sub-components, to the best-case scenario with exact geometric models (e.g., obtained from manufacturer CAD models).

B. 2.1 Parametrically known components (pKC)

In cases where only the approximate shape of surgical devices is known, the component models may be represented using multiple shape parameters. For a spine pedicle screw, a simple cylinder parameterization may be defined using the length and diameter of the screw shaft. This simple component model, referred to as pKC₁ was represented as a cylinder (Figure 2b) with 2 degrees-of-freedom (DoF) determining its shape and 5 DoF determining its 3D pose (ignoring the DoF associated with rotation about its principal axis), aggregating to 7 DoF in total.

Figure 2.

Component models. (a) Photograph of an example known component - a pedicle screw. (b) Simple parameterization of the screw as a cylinder of bounded length and diameter (pKC₁). (c) Higher-order parameterization that includes a screw tip and a rotating polyaxial cap (pKC₂). (d) Exactly-known CAD model of the screw and its cap (eKC). (e-f) DRR of pKC₂ and eKC. (g–h) Gradient magnitudes of the DRRs.

Higher fidelity models may be represented as a combination of multiple sub-components. We implemented a higher-order representation, denoted pKC₂, (Figure 2c) in which a tapered tip was added to the cylindrical shaft model using an ellipsoid dome of variable length. The pKC₂ model also incorporated the polyaxial cap as a larger cylinder with a lower bound in diameter (d), $\underset{\_}{d_{\text{cap}}}>\overline{d_{\text{shaft}}}$, and an upper bound in length (l), $\overline{l_{\text{cap}}}<\underset{\_}{l_{\text{shaft}}}$, to distinguish between otherwise interchangeable cylinders and prevent accidental overlap with the shaft. This higher-order model carries additional DoF, including tip length (+1 DoF), cap shape (+2 DoF), and the rotational joint between the shaft and the cap (+2 DoF), aggregating to 12 DoF in total for pKC₂. A detailed comparison and rationale for pKC₂ as a function of its particular subparts was investigated in earlier work (Uneri et al 2015).

To accommodate various shapes, pKC models were represented as unstructured grids, implemented as triangular meshes of closed surfaces. This sparse 2D manifold representation of the 3D object allowed for a small memory footprint and low-cost manipulation. For a single model, multiple realizations using different parameter values could be stacked together and reshaped via simple matrix operations through translating/scaling the underlying triangles.

B. 2.2 Exactly known components (eKC)

In certain cases, exact component specifications may be available from manufacturer-specific CAD models or precise measurements of tool shape and composition. In the experiments below, the CAD models for the shaft and cap were provided by DePuy Synthes Spine (Raynham, MA USA). The exactly known component (eKC) models were represented as structured grids, voxelized onto a fine image grid with a nominal voxel size of 50×50×50 μm³ to ensure minimal loss of information in the representation without reaching the GPU memory capacity. The eKC model included a shaft (+6 DoF) with a rotating polyaxial cap (+3 DoF). A single translational DoF (p_cap) was also included to account for the variable center of cap rotation, totaling 10 DoF.

B. 2.3 Known component registration (KC-Reg)

The unknown parameters of the device components, modeled as in the previous section, were solved from 2D images acquired in the context of intraoperative patient anatomy using the registration framework described in section B.1. Certain considerations and key advances detailed below were necessary to accommodate these components within the registration framework.

Image Acquisition

Earlier work showed that two radiographic views acquired at >15° separation is adequate for accurately registering patient anatomy (Uneri et al 2014a). The extended anatomy presents a large number of unique gradients, and differences in magnification (with respect to x-ray source-object distance) allows accurate registration even from single views (Uneri et al 2014b), none of which necessarily hold for device components of much smaller size. More importantly, the roughly cylindrical shape of such devices provide minimal information when imaged in an end-on view. Although the acquired views could be optimized based on the planned trajectory, such would assume close adherence to the plan and require unique/custom projections for each screw. As an alternative, we used three views defined simply relative to patient anatomy – AP, oblique, and lateral – as with radiographic views commonly acquired in the OR. To recover from these degenerate views, repeat registrations using $\left(\begin{array}{c}3\\ 2\end{array}\right)$ view combinations were performed in parallel. If the majority of registrations found the screw to be aligned with an end-on view (within a predetermined angular threshold), the image was excluded from the registration, and a complementary oblique view (oriented 90° from the original oblique) was acquired instead.

Parameter Initialization

The result of the patient registration stage was used to initialize the component registration stage from the plan (dotted line in Figure 1a–b), given a surgical plan containing intended screw trajectories and target locations. This initial guess was also used to define a region of support in the radiographs, such that it constrained the component, thereby reducing the memory footprint and limiting search space.

3D-2D Projection

To generate DRRs of the model components, the pKC model projections were formed using a novel mesh projection approach based on a fast ray-triangle intersection algorithm (Möller and Trumbore 1997) as described in (Uneri et al 2015). The eKC model projections used a simple tri-linearly interpolated ray-tracing approach, the same as that used to project a DRR from a CT in patient registration.

Parameter Optimization

The optimization function was redefined to support additional component shape parameters and models composed of multiple subcomponents as follows:

$$m_{\theta }(\mathrm{\Phi })=\sum _i{\mathcal{P}}_{\theta }\mathcal{T}({\mathrm{\Phi }}_{i,1\dots M})\circ C_i({{\mathrm{\Phi }}^{\ast }}_{i,M\dots N})$$ (5)

where, for each subcomponent C_i, the 3D pose is computed from the corresponding subset of parameters, and (for pKC models) the component shape is updated. KC-Reg imposes lower and upper bounds to such model parameters, both to avoid singular configurations (e.g., zero or negative shaft length), as well as to enforce relations between subparts (e.g., cap rotation center of motion in eKC). A sinusoidal mapping was applied on such parameters to apply lower (Φ̲) and upper (Φ̅) bounds as follows:

$${\mathrm{\Phi }}^{\ast }=\underset{\_}{\mathrm{\Phi }}+(\overline{\mathrm{\Phi }}-\underset{\_}{\mathrm{\Phi }})\frac{1-cos\left(\mathrm{\Phi }\frac{\pi }{\omega }\right)}{2}$$ (6)

where Φ* is within these bounds for any value of Φ, while ω = 10 marks the region 0 ≤ Φ ≤ ω such that it is a strictly increasing bijective mapping into [Φ̲, Φ̅].

B. 3 Phantom and cadaver experiments

Experiments were performed using an anthropomorphic body phantom and a human torso cadaver to evaluate the performance of the KC-Reg algorithm. Intraoperative radiographs were obtained from a mobile C-arm incorporating a flat-panel detector (Varian PaxScan 3030+, Palo Alto CA) operated in dual-gain mode (Schmidgunst et al 2007), and a motorized orbit (Siewerdsen et al 2005). For each target region containing the implanted screws, 200 images were acquired over a 178° orbit, each projection having 768×768 pixels with 0.388×388 mm² isotropic pixel size. Projections were corrected for signal uniformity (gain and offset corrections), and no distortion correction was performed, since flat-panel detectors do not suffer pincushion / magnetic field distortion effects evident in some image intensifier systems. Of the 200 projections, only the AP, lateral, and oblique were used in the 3D-2D registration process; the full projection set was used for complementary visualization in CBCT reconstructions not intrinsic to the proposed approach.

The C-arm projection geometry was calibrated using a spiral BB phantom (Navab 1996), and the view angles (θ) were obtained from built-in gantry angle encoders. Alternatively, if a calibration is not available – such as for mobile radiography systems – it is possible to solve for the complete 9 DoF pose of the imaging system view angles (Otake et al 2015). The AP, oblique, and lateral views were selected from three sets containing ∼10 projections each within ±5° angle separation in acquisitions. Input projections were randomly sampled from these sets to obtain three views for each registration.

Planning data was generated for all datasets in the form of simple linear trajectories down the pedicle. Each registration was randomly initialized within 6±2 mm and 3±1° of the actual pose, large enough to simulate potential errors in planning while containing the screw model within the target vertebra. Input 2D radiographs were log corrected, converting detector intensities (I) to line integral of linear attenuation coefficient using ∫ μ dx = −ln(I/I₀), where I₀ is the bare beam intensity. The input 3D CT image (used in the patient registration stage) was converted from Hounsfield units to approximate linear attenuation coefficients assuming μ_water = 0.02 mm^-1. The linear attenuation coefficient of titanium components was assumed to be 0.34 mm⁻¹ (at ∼60 keV); however, the GC similarity metric is relatively insensitive to mismatch in resulting dynamic range (Hubbell and Seltzer 1995) and therefore robust against the linear differences in the intensity scales.

B.3.1 Anthropomorphic torso phantom

The first set of experiments was performed to establish feasibility within a rigid anatomical context using a custom anthropomorphic torso phantom (The Phantom Laboratory, Greenwich, NY). As shown in Figure 3d, a total of 5 pedicle screws were implanted in vertebral levels ranging from L5 to T11. C-arm projection images were acquired at 100 kVp, 1.1 mAs, after placement of each screw. The screws varied in size (#435, #545 #645) as shown in Figure 3a, where the screw number conveys basic shape information (e.g., #435 implies ∼4 mm diameter, ∼35 mm length). Such information was used to initialize the pKC model shapes. The CAD models were provided by the manufacturer (DePuy Synthes Spine, Raynham MA) for use in eKC models.

Figure 3.

Experimental setup. (a) CAD renderings of the three types of pedicle screws (#435, #545, and #645) used in the experiments. (b) Illustration of the mobile C-arm and rendering of the anthropomorphic body phantom. (c) Placement of pedicle screws in the cadaver torso by a trainee supervised by a fellowship-trained spine surgeon. (d–e) Coronal CT images of the torso phantom and cadaver, respectively, showing target screw locations.

B.3.2 Human torso cadaver

A second set of experiments was conducted to test performance under realistic conditions using a cadaveric human torso (81 year old male). A preoperative CT scan was acquired prior to the experiment with the cadaver secured to a carbon fiber tabletop. A total of 8, #645 screws were implanted, targeting 5 vertebrae ranging from L5 to T9 (Figure 3e). C-arm projection images were acquired at 100 kVp, 2.3 mAs per projection.

B.4 Quality assurance (QA)

KC-Reg performance was evaluated with respect to tasks of intraoperative QA as described above. For pedicle screw placement, the relevant QA measures were: 1) geometric accuracy of the registration result; 2) verification that the device delivered in the patient matches that specified in planning; and 3) visualization of the registered component with respect to clinically acceptable error margins as a means of breach detection.

B.4.1 Geometric accuracy

The geometric accuracy of the registration results were evaluated in 13 pedicle screws delivered in phantom and cadaver experiments. The TRE was calculated separately in terms of the translational (x) and rotational (ϕ) components (Figure 4a), using screw tip locations and screw principle axes, respectively. Such decomposition of the metric provided more direct comparison to the clinically desired tolerance – 1 mm and 5° as suggested by (Rampersaud et al 2001). Given a registration estimate (R̂, t̂) composed of a rotation and translation element, the positional and angular TRE metrics were:

Figure 4.

Figures of merit. (a) TRE associated with 3D translation (TRE_x) and 3D rotation (TRE_ϕ). (b) Acceptance window derived from surgical planning in 3D preoperative images. The acceptance window demarks a cylindrical corridor interior to the cortex of the spinal pedicle and an ellipsoidal volume internal to the anterior vertebral body.

$${\text{TRE}}_x={\Vert \widehat{t}-t\Vert }_2$$ (7)

$${\text{TRE}}_{\varphi }={cos}^{-1}\frac{{\widehat{R}}_z\cdot R_z}{\Vert {\widehat{R}}_z\cdot R_z\Vert }$$ (8)

where R_z (third column of the rotation matrix) is the principal axis of the target screw. TRE_x and TRE_ϕ quantify the deviation from intended location (mm) and trajectory (degrees), respectively. The true poses of the screws were computed from 3D-2D registrations using all available views (200 projections acquired over a semicircular arc using the mobile C-arm), excluding the AP, lateral, and oblique views used in 3D-2D registration to avoid bias.

B.4.2 Device verification

During pedicle screw placement, the type of screw delivered in the patient could deviate from the intended plan either purposely (e.g., selection of a larger or longer screw for increased purchase in unanticipated osteoporotic bone) or unintentionally (e.g., inadvertent selection of a suboptimal screw). In either case, the ability to detect deviations from the planned construct is important to QA and verification of the surgical product. We therefore extended the KC-Reg methodology to detect instances in which the device evident in the 2D intraoperative image differs from that specified in planning. Such verification is relevant in cases where only partial information on component shape is known and/or used beforehand (i.e., the two pKC models), since the complete knowledge of device information (i.e., the eKC model) would inherently assume the type of the screw to be known. We used the output shape parameters computed from pKC registration to solve a multi-class learning-based classification problem. The number of classes was equal to the total number of screw types available (N), with N = 3 in the experiments below corresponding to the 3 available sizes of screw.

For training of the classifier, the output parameters of registration solutions corresponding to component shapes (length and diameter of the screw) were used as input to the k-nearest neighbor algorithm for classification of different screw types. The training data set consisted of 200 data samples, and the number of neighbors (k) was varied across a range of 10–40 to empirically test the sensitivity to k. The shape parameters – screw length and diameter – were used as the feature vector in each data sample and were normalized to 0–1 using optimizer bounds. The k-nearest neighbor algorithm provided decision boundaries to perform a multi-class classification test to identify the correct screw type. To assess the performance of device verification, the leave-one-out cross-validation method was used where the particular data sample being tested was excluded during the training phase. Performance of device validation was quantified in terms of classic binary hypothesis metrics of sensitivity (TPF), false negative fraction (FNF), positive predictive value (PPV), and accuracy (ACC):

$$\begin{array}{c}\mathit{\text{TPF}}=\mathit{\text{TP}}/P\\ \mathit{\text{FNF}}=\mathit{\text{FN}}/P\\ \mathit{\text{PPV}}=\mathit{\text{TP}}/(\mathit{\text{TP}}+\mathit{\text{FP}})\\ \mathit{\text{ACC}}=(\mathit{\text{TP}}+\mathit{\text{TN}})/(P+N)\end{array}$$ (9)

where P denotes a positive (i.e., wrong/unintended) screw type, N denotes a negative (i.e., correct/intended) screw type. In the context of spine screw placement, correct detection of unintended screw types (true positive, TP) and failure to detect unintended screw types (false negative, FN) are of specific interest.

B. 4.3 Visualization relative to the surgical plan and pedicle acceptance window

As a final element of intraoperative QA, we used the KC-Reg result to assess if screws were within clinically relevant error margins with respect to their shape and pose. An “acceptance window” for the implanted screw was defined to delineate the spatial window within which screw placement can vary without compromising the clinical objective – bounded by the cortical margins of the pedicle and the anterior aspect of the vertebral body. This definition was formalized based on the planned screw trajectory, modeling the pedicle corridor with a cylinder bounded by the pedicle cortex, and expanding an oblate spheroid about the desired target point (screw tip) within the cortex of the anterior vertebral body (Figure 4b). Such an acceptance window presents a binary evaluation of the surgical product (acceptable/within window versus unacceptable/outside window), which could be more practical in clinical implementation than a continuous quantitative error metric that may invite overcorrection or unnecessary fine tuning of device placement.

In addition to evaluation of the surgical product relative to the acceptance window, the registration method allowed visualization of the results both in the 3D (preoperative) image context and 2D (intraoperative) images via overlay of the registered components. In such visualization, the 3D volumes were resliced along the principal device axis to yield oblique-axial and oblique-sagittal slices that better depict the orientation of a screw to surrounding anatomy. Quantitative assessment of conformance to the surgical plan for each screw was also computed as deviation from the planned target location (Δ_x) and trajectory (Δ_ϕ) using Equations (7) and (8) for TRE evaluation.

C Results

C.1 Geometric accuracy

TRE values from 40 repeat registrations (per target screw) are summarized in Figure 5a–b in terms of the median and interquartile range (IQR). In the phantom experiments (Figure 5a), both TRE_x and TRE_ϕ improved with the degree of sophistication of the component models, achieving median TRE_x <1 mm and TRE_ϕ<1° for the pKC₂ and eKC methods. In the cadaver experiments, TRE_x improved from 2.4±1.9 mm in pKC₁ to 0.6±0.2 mm in pKC₂ and achieved 0.2±0.1 mm in eKC. TRE_ϕ in cadaver was 3.2±4.1° in pKC₁, 1.4±0.7° in pKC₂, and 0.2±0.1° in eKC. The higher errors observed in cadaver than in phantom are likely attributable to more complex gradients presented in real anatomy and deforming soft tissue. Pooling results from the phantom and cadaver studies, the eKC method demonstrated median TRE_x and TRE_ϕ of 0.2 mm and 0.2°, respectively, IQR in TRE_x and TRE_ϕ of 0.12–0.29 mm and 0.13–0.3°, respectively, and 92% of the registrations were within the target accuracy levels of TRE_x <1 mm and TRE_ϕ <5°.

Figure 5.

Geometric accuracy of KC registration: (a) TRE_ϕ vs. TRE_x measured in an anthropomorhic spine phantom and (b) in a cadaver torso. Each boxplot shows rectangles marking the IQR, intersected at the median values. The dashed whiskers extend to 1.5×IQR, and the number of outliers is annotated as the percentage of the total number of sample points.

When compared to pKC₁, the additional modeling of the tapered tip in pKC₂ was found to improve the convergence of the optimizer by providing a small but measurable increase in metric value when the tips matched. Similarly, modeling of the polyaxial cap resolved numerous outliers resulting from over-fitting the cylinder shaft to the cap in radiographs, which was observed to be problematic when the cap was rotated with respect to the shaft. For the component models evaluated, an overall trend of improvement in geometric accuracy and precision with increased model fidelity was observed, with eKC₁ achieving the best performance.

Using three anatomically defined views provided robustness to the presence of multiple components by preventing ambiguity for cases in which multiple screws overlapped in the radiographic views, such as the lateral view for bilateral screw placement. The most challenging cases involved instances in which one of the views aligned closely with the end-on view of the screw. Using the combinatorial voting scheme described in Section B.2.3 with a threshold angle of 20° allowed detection of such views with >80% accuracy, providing a means by which such views could be rejected in favor of an opposing oblique view. As a result, the mean TRE_x improved from 4.9 mm when using two views, to 1.4 mm, while the number of outliers was reduced from ∼20% to 1–2% for the pKC₂ model evaluated in phantom.

C.2 Device verification

The decision boundaries for k-nearest neighbor classification for the three screw classes in both pKC models are shown in Figure 6a–b. For the test cases using the simple pKC₁ model, predictions for screw length and diameters were observed to be dispersed across the feature space, particularly for the smaller sized #435 screw. The pKC₂ model resulted in more repeatable predictions, leading to clustered samples and facilitating the classification problem. Figure 6c-d shows the results for each of the performance measures in the sensitivity/specificity analysis. The pKC₂ model demonstrated accurate classification with high sensitivity and a low rate of false positives. The high variability exhibited by the pKC₁ model classifications were caused by increased sensitivity to the number of nearest-neighbors k. Using the more advanced pKC₂ model, the detection accuracy improved from 92.9% to 99.3%, with consistent performance across a range in k from 10 to 40 neighbors. Including the screw tip in the pKC₂ model specifically improved clustering of the shaft length parameter, and the inclusion of the cap prevented the previously noted susceptibility to error in screw length estimation.

Figure 6.

Verification of the implanted device. Classification using the (a) pKC₁ and (b) pKC₂ models. The feature space is partitioned according to the three screw types tested, and each data point represents a registration result using the marked screw (circle – 435, triangle – 535, and square – 645). For example, a circle in the 435 partition implies correct identification of a 435 screw, whereas a circle in the 545 or 645 partition implies a misidentification. Performance of screw classification using the (c) pKC₁, and (d) pKC₂ models.

C.3 Visualization relative to the surgical plan and pedicle acceptance window

Using the KC-Reg results for the cadaver experiment, example visualization suitable to QA of the surgical product is illustrated in Figure 7. Each row represents a separate screw placement. The first two columns illustrate visualization of screw placement relative to acceptance windows (green or red regions) within the preoperative CT image, and the second two columns illustrate overlay on the intraoperative AP and LAT radiographs relative to the planned trajectory (green or red line and circular endpoint). Of the 8 screws delivered, 6 were correctly placed, and 2 were intentionally malplaced, and each was correctly identified as such by KC-Reg, demarking the image in green or red, respectively, to connote screw delivery within or outside the acceptance window. Such visualization was automatically generated following KC-Reg as the basis of a QA report in which the surgeon can visually and quantitatively assess the quality of surgical product. Clinical assessment of such a reporting tool is the subject of future work.

Figure 7.

Visualization of the registration result in a form suitable to QA of device delivery. The location of screws as indicated by the registration result is shown in yellow: (left) in axial and sagittal slices of the preoperative CT; and (right) in AP and LAT intraoperative radiographs. Acceptance windows and trajectories are also overlaid, depicted in green to indicate that the screw is contained within the window, or red to indicate a breach. Intraoperative The values (Δ_x, Δ_ϕ) are the measured deviations in the tip position and shaft orientation, respectively, relative to the preoperative plan.

D Discussion

Preclinical experiments conducted in phantom and cadaver demonstrated the ability of the KC-Reg method to register pedicle screw models to intraoperative radiographs acquired from AP, oblique, and LAT views and provide a basis for quantitative measurement and QA of the surgical product. This approach could offer a simple image registration solution for intraoperative guidance and quality assurance without (or in complement to) additional hardware tracking systems or intraoperative 3D imaging. For fluoroscopically guided screw placement in which such views are regularly acquired, the KC-Reg system operates within existing workflow and could aid surgeons by means of a near-real-time, independent check on pedicle screw breach. For freehand and/or navigated screw placement without fluoroscopic guidance, the KC-Reg system could be applied to radiographs acquired at the conclusion of a case (e.g., mobile x-ray images regularly acquired as a check on retained foreign bodies) to provide intraoperative QA and permit revision of suboptimal device placement within the same procedure.

The eKC model yielded the best registration accuracy, giving median TRE_x 0.2 mm and TRE_ϕ 0.2° in the phantom and cadaver studies. The simple cylindrical parametric model (pKC₁) yielded median TRE_x 2.5 mm and TRE_ϕ 1.8° with a large number of outliers (∼20%) owing to the overly simplistic model representation of the screw. The more sophisticated pKC₂ model (cylinder with tip and cap) improved performance to TRE_x 0.6 mm and TRE_ϕ 0.7°. Therefore, both eKC (if manufacturer CAD models are available) and pKC₂ (if they are not) satisfied the desired level of accuracy of <1 mm and <5° previously reported in literature (Rampersaud et al 2001). The superior performance of eKC models motivates the use of even higher fidelity parametric models in the future, tailored to specific categories of components – e.g., parameterization of the screw pitch, number of threads, shape of the polyaxial cap. Furthermore, the parametric models could be used to detect and classify deviations from the intended device type with up to 99.3% accuracy in preliminary studies. Finally, the registration results can be summarized in the form of a structured QA report on each implant showing its orientation relative to preoperative CT or intraoperative radiographs, the surgical plan, and acceptable margins.

In the experiments reported, a calibrated C-arm was used which provided the relationship between multiple views as it was rotated in an orbital motion. Alternatively, mobile radiography systems such calibration due to the unconstrained positioning of the detector with respect to the x-ray source. A potential solution may be adapted from (Otake et al 2015), whereby the projection geometry is solved using the full 9 DoF of the imaging system with respect to independently oriented radiographic views. Assuming a fixed patient pose during image acquisition, multiple views can be correlated to each other in the patient registration process (Figure 1), which can then be used simultaneously in component registration. Acquiring multiple views not only improved 3D registration accuracy but also was necessary to accommodate end-on views of the screw, which provide very little information. The solution described above for detecting such degenerate views proved sufficient for cylindrical shapes, but it may not extend generally to other component types – e.g., fixation plates, knee and hip prosthetic implants – in which case a more generalized solution that can adapt to arbitrary shapes would be more practical.

The present work was limited to rigid pose estimation and simple parametric manipulation, although extension to deformable models is possible using methods similar to those in (Stayman et al 2011). The use of triangular meshes is particularly suitable for this application: for example, the vertices could be used as control points for modeling the deformation using b-splines. One potential drawback is the complexity of the mesh projection detailed in (Uneri et al 2015), which requires each projected ray to test against all triangles for a potential intersection. To address this, a similar solution in ray-casting may be adapted to partition the triangles in octrees, thus reducing the search space. Current implementation requires individual registration of screws, and future work is planned for supporting simultaneous registration of multiple components, using optimizer penalties for detecting and addressing component collisions and overlaps. In addition to parallelizing the registration across multiple components, this approach has the potential to better represent multi-level constructs and assure the quality of its configuration/alignment during surgery.

The preclinical studies presented above were performed in phantom and cadaver specimens, and further evaluation in clinical images is the subject of future work pending institutional review board (IRB) approval. Work is underway for integration of KC-Reg with an in-house image guidance and navigation platform (Uneri et al 2012) to facilitate and streamline clinical evaluation in a broader variety of procedures (e.g., joint replacement, femoral nailing, and maxillofacilal reconstruction) and diversity of instrumentation present in radiographic views.