Hand-Held Stereovision System for Image Updating in Open Spine Surgery

Abstract

BACKGROUND

Image guidance in open spinal surgery is compromised by changes in spinal alignment between preoperative images and surgical positioning. We evaluated registration of stereo-views of the surgical field to compensate for vertebral alignment changes.

OBJECTIVE

To assess accuracy and efficiency of an optically tracked hand-held stereovision (HHS) system to acquire images of the exposed spine during surgery.

METHODS

Standard midline posterior approach exposed L1 to L6 in 6 cadaver porcine spines. Fiducial markers were placed on each vertebra as “ground truth” locations. Spines were positioned supine with accentuated lordosis, and preoperative computed tomography (pCT) was acquired. Spines were re-positioned in a neutral prone posture, and locations of fiducials were acquired with a tracked stylus. Intraoperative stereovision (iSV) images were acquired and 3-dimensional (3D) surfaces of the exposed spine were reconstructed. HHS accuracy was assessed in terms of distances between reconstructed fiducial marker locations and their tracked counterparts. Level-wise registrations aligned pCT with iSV to account for changes in spine posture. Accuracy of updated computed tomography (uCT) was assessed using fiducial markers and other landmarks.

RESULTS

Acquisition time for each image pair was <1 s. Mean reconstruction time was <1 s for each image pair using batch processing, and mean accuracy was 1.2 ± 0.6 mm across 6 cases. Mean errors of uCT were 3.1 ± 0.7 and 2.0 ± 0.5 mm on the dorsal and ventral sides, respectively.

CONCLUSION

Results suggest that a portable HHS system offers potential to acquire accurate image data from the surgical field to facilitate surgical navigation during open spine surgery.

ABBREVIATIONS

3D

3 dimensional

FOV

field of view

fps

frames per second

HD

high-definition

HHS

hand-held stereovision

IACUC

Institutional Animal Care and Use Committee

iCT

intraoperative computed tomography

iSV

intraoperative stereovision

OR

operating room

pCT

preoperative computed tomography

uCT

updated computed tomography

VI

virtual instrument

Intraoperative image guidance improves surgical outcomes across various specialties, eg, in cranial, liver, orthopedic, and spine surgery.^1,2 More than 300 000 of open posterior lumbar decompression and fusion surgery are performed each year in the USA, and navigation can improve placement of pedicle screws and cages. However, image guidance based on preoperative data encounters additional challenges from intervertebral motion between supine preoperative scans and prone surgical positioning. Additionally, vertebral manipulation during surgery, such as decompression, pedicle screw placement, and deformity correction, creates additional misalignment and further degrades registration. Techniques have been developed to maintain accuracy of intraoperative navigation.³ For example, O-arm (Medtronic, Dublin, Ireland) and intraoperative computed tomography (iCT) are performed usually with prone positioning and provide up-to-date images, but they expose the patient and surgical team to ionizing radiation, and often increase operative time,^4,5 especially when used iteratively as surgery progresses. Another solution exploits anatomic landmarks to register the surgical field with preoperative computed tomography (pCT) scans (eg, 7D Surgical, Toronto, Canada), but this approach limits navigation to one vertebral segment at a time, and requires manual selection by the surgeon.

In previous studies, we demonstrated that patient registration can be achieved by matching pCT with the patient's prone position through dual cameras (intraoperative stereovision, iSV).^6,7 We have also developed a novel image updating system to deform diagnostic or pCT images to generate updated computed tomography (uCT) that compensates for intervertebral motion using iSV images of the surgical field.⁸ This image updating technique is an accurate, radiation free, low cost, and efficient alternative to navigation using pCT or intraoperative imaging. The iSV system used in previous studies was mounted to a surgical microscope, which is not widely used in lumbar fusion. We have now developed a hand-held stereovision (HHS) system for intraoperative reconstruction of the exposed spine to facilitate image guidance. In this study, we assess HHS accuracy using implanted fiducial markers, and infer the validity of HHS by using iSV surfaces to generate uCT and assess uCT accuracy based on fiducial markers and iCT.

METHODS

HHS System

A HHS system (Figure 1) was developed to acquire iSV images of the surgical field. It consisted of a custom-designed handle, 2 high-definition (HD) cameras (C920, Logitech, Lausanne, Switzerland), an active tracker (Medtronic), and 11 adjustable LEDs (Shenzhen Ustellar Technology Ltd, Shenzhen, Guangdong, China) to illuminate the surgical scene. Cameras were rigidly attached to the handle and angled to maximize the overlapping field of view (FOV) between 2 images at an acquisition distance of ∼150 mm between HHS and the surgical field, which is sufficiently long to capture the depth of a typical lumbar fusion exposure. Both cameras were operated with the same parameters, at an image resolution of 1920 1080 pixels, speed of 30 frames per second (fps), and at a focal distance of ∼150 mm. An active tracker was rigidly attached to the handle, and transformed reconstructed iSV surfaces into coordinates defined by the navigation system (StealthStation S7, Medtronic). A one-time calibration was performed, the details of which have been published.⁹

FIGURE 1.

Hand-held stereovision system: front A and back B views. The black arrow points to the active tracker, the white arrow identifies the LED lights, and the yellow box shows the 2 HD cameras.

Surgical Procedures

Institutional Animal Care and Use Committee (IACUC) approval was obtained for porcine investigation of spinal image guidance. Image data from 6 cadaver specimens were obtained. Spines varied in size, but were comparable to human anatomy. Each specimen was positioned in prone posture for surgical exposure (L1-L6), including spinous and transverse processes. Registration with HHS only requires a small visual field, but we used a long and wide surgical exposure to facilitate accuracy evaluation across the entire lumbar spine. Ligamentous structures remained intact. Three fiducial markers were implanted on each level (1.5 mm in diameter, Leibinger Universal Neuro Fixation System, Stryker Corporation, Kalamazoo, Michigan), one on the spinous and one on each transverse process. The specimen was closed and positioned supine to accentuate lordosis at L3, and pCT (pixel spacing: 0.21 × 0.21-0.35 × 0.35 mm, slice thickness: 0.6 mm) was acquired. The specimen was re-positioned in a neutral prone posture with deformity corrected, and the surgical site was re-opened to expose the vertebrae. A reference frame tracked by StealthStation was attached rigidly to the ilium. Figure 2A shows a photograph of a typical experiment. An iCT (pixel spacing: 0.21 × 0.21-0.40 × 0.40 mm, slice thickness: 0.6 mm) was acquired in the same posture.

FIGURE 2.

Typical porcine spine experimental setting. A shows an exposed spine with fiducial markers implanted and a reference frame attached. The HHS system was held above the surgical field during iSV acquisition. B shows the user interface for image and tracking data acquisition.

Data Acquisition

Image and tracking data were obtained with Labview (National Instruments, Austin, Texas) through virtual instruments (VI) interface (Figure 2B). During each experiment, the VI obtained image and tracking data continuously at a data rate of ∼10 fps. During acquisition, a snapshot was taken, and images were stored with corresponding tracking data. HHS was introduced at time of image acquisition, but was otherwise set aside. HHS motion was estimated from the unit's current location relative to its previous position obtained by the VI (within ∼0.1 s), and was displayed in real time. Operators maintained steady-hand position during acquisition of each snapshot and images were re-acquired if hand motion was >1 mm. Acquisition time for each image pair was <1 s.

Four (4) operators acquired data across 6 specimens. During each experiment, 54 iSV pairs were acquired—3 from each level (L1-L6): a snapshot with spinous process centered in the FOV, and 2 angled snapshots with left and right transverse processes in the FOV center, respectively, repeated by 3 operators (54 image pairs = 3 images on each level 6 levels 3 operators). Images were recorded at various acquisition distances. After image acquisition by each operator, 18 fiducial markers were digitized with a tracked stylus to obtain their ground truth locations in StealthStation coordinates.

Surface reconstruction (iSV) is illustrated in Figure 3. First, image rectification is performed on both left and right images (Figure 3A and 3B)—features appear along the same horizontal line in the rectified data (Figure 3C and 3D). Optical flow identified stereo correspondence between left and right rectified images and generated a disparity map in the horizontal direction (Figure 3E). A 3D surface was produced (Figure 3F) based on the disparity map.

FIGURE 3.

Reconstruction of iSV surface. A and B, Raw left and right images. C and D, Rectified left and right images. E, Disparity in the horizontal direction. F, Reconstructed surface of the spine with tracked fiducial marker locations (green points) and locations recovered from iSV (red points).

Image Updating

pCT was registered with iSV surfaces to generate uCT using level-wise registration, the technical details of which have been described previously.⁸ A flowchart of the image updating process is shown in Figure 4. Briefly, pCT was first segmented to extract bony spine surfaces. During surgery, iSV data were recorded using HHS after exposure. Images (pCT) were registered level by level to match iSV, and uCT was generated and uploaded to StealthStation. Functions and tools available on the navigation system such as trajectories were available with uCT for image guidance.

FIGURE 4.

Flowchart of the image updating process.

Data Analysis

To quantify iSV accuracy, the center of each fiducial marker visible in each iSV surface was identified as a measurement point. Error was calculated as distance between a fiducial marker location reconstructed from iSV (red points in Figure 3F) and its corresponding tracked location from StealthStation (Medtronic; green points in Figure 3F).

In addition, we performed statistical analyses to investigate relationships between magnitudes of iSV errors and (1) their locations in 2D image space, (2) the amount of hand motion at time of acquisition, and (3) their acquisition distances. These variables were defined as: (1) Dist_P2C—distance from each fiducial marker to image center in 2D image space; (2) M, the magnitude of hand motion—distance between the location of HHS at time of acquisition and its most immediate, previous location (within ∼0.1 s); and (3) Dist_P2HHS—distance between each measurement point and HHS. For each variable, a Spearman rank correlation coefficient was calculated relative to iSV error, and a P-value (threshold: .05) tested a no-correlation hypothesis. A fit was performed between each variable and iSV error to estimate coefficients in a linear relationship.

Accuracy of uCT was assessed on the dorsal and ventral sides. Fiducial markers were implanted on the dorsal surface prior to pCT acquisition, and therefore, were also available in uCT as uCT was produced by deforming pCT. To assess dorsal accuracy, the center of each fiducial marker in its deformed location (in uCT) was identified, and the accuracy was calculated as distance between each tracked fiducial marker location in operating room (OR) space and its counterpart as transformed from uCT to OR space. To assess ventral accuracy, iCT was co-registered with uCT on the StealthStation using a rigid registration, and a landmark was identified on the ventral side of each vertebral body in both iCT and uCT. Accuracy was calculated as distance between a landmark location in uCT and its corresponding location in iCT.

All computations were performed on a Dell Alienware 17 R5 laptop (Dell, Round Rock, Texas), equipped with Intel Core i9-8950HK (6-core), 32GB DDR4 RAM, and a graphical processing unit NVIDIA GeForce GTX1080 (Nvidia Corporation, Santa Clara, California). Data analysis including statistical analyses was performed in MATLAB (MathWorks, Natick, Massachusetts).

RESULTS

Results are summarized in Table 1 and Figure 5. Total number of images, number of measurement points, and their mean error and standard deviation appear in Table 1 (rows 2-7), and distributions of errors are plotted in Figure 5A-5F. Overall iSV error from all 6 cases was 1.2 ± 0.6 mm (Table 1, row 8), and the distribution is shown in Figure 5G. Boxplots of iSV errors are provided in Figure 5H. Columns 1 to 6 show cases 1 to 6, respectively, and column 7 indicates overall error from all cases. Performance was similar across cases in terms of mean error, standard deviation, and positive-skew distribution of errors (Figure 5A-5F).

TABLE 1.

Summary of iSV Errors

Case	Number of images	Number of measurement points	Mean ± standard deviation (mm)
1	54	337	1.0 ± 0.4
2	54	390	1.2 ± 0.6
3	54	459	1.2 ± 0.5
4	54	439	1.3 ± 0.6
5	54	478	1.2 ± 0.7
6	54	450	1.4 ± 0.7
Overall	324	2553	1.2 ± 0.6

iSV: intraoperative stereovision.

FIGURE 5.

Plots of iSV errors from 6 cases. A-F, Histograms of errors from cases 1 to 6, respectively. G, Histogram of combined data from all cases. H, Boxplots of errors from cases 1 to 6 in columns 1 to 6, respectively, and boxplot of combined data from all cases in column 7. In each box plot, the red line corresponds to median value, edges of the box indicate 25th (Q1) and 75th (Q3) percentiles, and whiskers extend to the most extreme data points that were not considered outliers. The outliers were calculated as data points greater than Q3 + w × (Q3 – Q1) or less than Q1 – w × (Q3 – Q1), where w = 1.5, and are plotted individually using the “+” symbol.

Results from statistical analyses are listed in Table 2. Scatter plots and linear fits appear in Figure 6A-6C for Dist_P2C, M, and Dist_P2HHS, respectively. Dist_P2C ranged from 15 to 1005 pixels. Assessed pixels covered nearly the entire FOV in 2D image space (rectified image size: 2043 1391 pixels). Spearman's correlation shows iSV error had little relationship with 2D point location (Dist_P2Cρ was near zero and P > .05). The red line in Figure 6A represents the linear fit, which is nearly horizontal with B₀ = 1.3, close to the mean error (1.2 mm), and indicates that the entire FOV was equally accurate. Hand motion, M, ranged from 0.01 to 0.9 mm (Figure 6B). Average error of each iSV surface (range: 0.5-3.3 mm) was used because M is associated with each acquisition (not individual measurement points). Correlation results suggest that iSV error was weakly associated with hand motion (ρ = 0.17, P < .05), and the linear fit (red line in Figure 6B) has a slightly positive slope with B₀ = 1.1. Dist_P2HHS ranged from 95 to 225 mm and iSV error had a positive linear relationship with 3D location of the point (Dist_P2HHS), with ρ = 0.45 and P < .001.

TABLE 2.

Results of Statistical Analyses

	Spearman's rank correlation		Linear fit
Variable	ρ	P	B1	B0
DistP2C	–0.038	.06	–0.0001	1.3
M	0.17	.003	0.30	1.1
DistP2HHS	0.45	<.001	0.013	–0.77

FIGURE 6.

Scatter plots (blue points) and linear fits (red lines) of iSV error vs 3 variables. A, 2D distance from measurement point to image center (Dist_P2C). B, Hand motion (M). C, 3D distance from measurement point to HHS (Dist_P2HHS). The horizontal dotted green line shows a clinically acceptable error range of 2 mm, and the vertical dotted green line shows the acquisition distance of 150 mm.

Computational times of major steps in iSV reconstruction are listed in Table 3. For technical reasons, computational time was longer for the first image pair but significantly shorter for the rest of the data when multiple image pairs were batch processed. Therefore, we report 2 different times for image rectification (column 2) and overall computational time (column 5). The first number (longer time) indicates the computational cost for the first image pair, and the second number (shorter time) indicates the average computational effort involved for processing each of the rest of the image data. Correspondence computation (column 4) varied mainly due to spine size (different parameters were used), but difference was small (within 0.5 s). Overall computational times for image rectification, correspondence computation, and 3D surface reconstruction were 1.47(first image)/0.03, 0.53, and 0.20 s, respectively. The total computational time was ∼2 s for the first surface and <1 s for the rest.

TABLE 3.

Computational Times of Major Steps in iSV Surface Reconstruction

Case	Image rectification (s)	Correspondence computation (s)	3D surface reconstruction (s)	Total (s)
1	1.46/0.03	0.65	0.19	2.32/0.86
2	1.59/0.03	0.32	0.20	2.12/0.55
3	1.42/0.03	0.35	0.20	2.07/0.57
4	1.59/0.04	0.51	0.20	2.57/0.71
5	1.49/0.03	0.61	0.19	2.40/0.82
6	1.37/0.03	0.76	0.20	2.40/0.97
Overall	1.47/0.03	0.53	0.20	2.34/0.74

iSV: intraoperative stereovision.

Accuracy of uCT is reported in Table 4. Dorsal accuracy was 3.1 ± 0.7 mm (range: 2.1-3.9 mm), and ventral accuracy was 2.0 ± 0.5 mm (range: 1.5-2.7 mm) across the 6 cases. These results suggest that uCT was well aligned with independently tracked fiducial markers as well as iCT, and indicate the validity of iSV-based registration.

TABLE 4.

Accuracy of Updated CT

Case	Dorsal (mm)	Ventral (mm)
1	2.8 ± 1.0	2.7 ± 1.3
2	3.9 ± 1.4	1.6 ± 0.5
3	2.8 ± 0.8	1.5 ± 1.1
4	3.1 ± 2.1	2.1 ± 1.1
5	3.7 ± 2.5	1.5 ± 0.9
6	2.1 ± 1.2	2.5 ± 1.2
Average	3.1 ± 0.7	2.0 ± 0.5

DISCUSSION

We consider 2 mm as clinically acceptable for iSV, and the overall HHS error of 1.2 ± 0.6 mm meets this criterion. During pedicle screw placement, perforation of <2 mm is considered safe,¹⁰ and a study by Lien et al¹¹ suggested that distance between the cortex of pedicle and neural structures was 1.7 to 2.0 mm in the medial direction, and 2.4(L5)-9.6(L1) mm in the lateral direction in the lumbar spine. When accuracy of image guidance is within 2 mm, perforation is likely to be smaller than 2 mm because additional error is allowed within the pedicle.¹² Overall image guidance error arises from multiple sources (including but not limited to iSV error), and exceeds 2 mm when any contribution exceeds 2 mm. In this study, iSV error arose from contributions associated with calibration, tracking, correspondence computation, and 3D reconstruction, and is modulated by errors in stylus tracking and fiducial marker localization in both physical and image spaces. Experimental setup (eg, specimen size, distance from tracked instruments to the tracking system, and operators) contributed to discrepancies between cases; nevertheless, error histograms were similar, indicating consistent HHS performance.

Furthermore, accuracy was dependent on acquisition distance. Data showed that image quality was visually acceptable from various distances (95-225 mm) with 2 to 3 lumbar segments in the FOV, and optical flow correspondence computation was successful on all image pairs. In addition, HHS behavior was explored at shorter (<90 mm) and longer (>250 mm) distances, and performance degraded considerably. At short distance, FOV was reduced significantly (to 1-2 levels), and optical flow sometimes failed to compute correspondences correctly as features (spine) appeared much larger in pixels, and images were blurred. A shorter distance is likely to be clinically irrelevant given muscle thickness in humans. At longer distance, although FOV was larger, iSV accuracy was not clinically acceptable. In fact, case 5 included more points with longer distances because one operator held HHS at a farther location (>200 mm). As a result, a second error peak appeared in the histogram (at ∼3 mm, Figure 5E), and more outliers were evident in the boxplot (red crosses in Figure 5H, column 5). Figure 6C shows the majority of points with higher error (>2 mm) was due to acquisition distance—points in the upper-right quadrant (error > 2 mm, acquisition distance > 150 mm) were denser than in the upper-left quadrant (error > 2 mm, acquisition distance < 150 mm). Higher errors at short acquisition distances were likely due to error accumulation from multiple sources. Although maximum errors were 3 to 4 mm, they were measured at individual points and did not represent the overall accuracy of the entire surface and are unlikely to affect subsequent image registration. For example, specular reflection at one fiducial marker can increase local error, but does not affect accuracy of the rest of the surface. Figure 6B shows the average error of each surface, and only 6% had errors >2 mm with all acquisition distances included. Results from linear fits (error = B₀ + B₁ Dist_P2HHS) indicate the expected error is 1.2 mm when images are acquired at 150 mm. Among 2553 measurement points at varying acquisition distances, 89% were within 2 mm. Rates of 95% and 90% can be achieved with acquisition distances within 170 and 195 mm, respectively. Accordingly, we added 2 laser diodes facing the center of the FOV that converge to a single focal point at an acquisition distance of 150 mm to guide the operator.

To assess the validity of iSV data further, uCT results were evaluated. First, dorsal accuracy was assessed using independently tracked markers that were not involved in image updating. Overall dorsal accuracy was influenced by fiducial localization error, iSV error, and registration error. Tissue and ligamentous structures remaining on the dorsal surface of the vertebrae also contributed to iSV-pCT registration error, since iSV surfaces included these structures while corresponding pCT surfaces were based only on bony structures. Ventral accuracy was assessed using iCT, and overall accuracy included effects from identification error, iCT-uCT co-registration error, and was modulated by errors from iSV-pCT registration. In some cases, iCT-uCT rigid registration compensated for part of the iSV-pCT registration error; therefore, ventral error was lower than dorsal error (cases 2-5). Overall uCT accuracy exceeded 2 mm largely due to iSV-pCT registration errors. Subsequently, we improved our algorithms and preliminary data suggest registration errors can be reduced to 2 mm. We will conduct more comprehensive and complete analyses of registration error improvements resulting from these algorithmic advances in the future.

Limitations

Limitations do exist. First, stereo correspondence can be occluded. For example, in angled snapshots, features on the opposite side were often blocked by spinous processes. In this study, we only assessed fiducial marker points fully visible in both left and right images. Images centered on spinous processes captured both sides and were reconstructed accurately. Second, our current HHS unit was limited to a single focal distance of 150 mm with a FOV of 2 to 3 segments. Thus, surgery with larger openings (>3 levels) required more than one iSV capture to recover the full surgical field. Since iSV surfaces were tracked, they can be concatenated with little additional computational cost. Overall interruption to surgical flow was minimal (<3 s) since 2 to 3 image pairs covered the entire lumbar section. For anatomy that requires longer (or shorter) acquisition distances beyond the optimal range of the current device, HHS can be precalibrated at multiple focal distances. Then, focal distance can be adjusted to accommodate various anatomies and surgical conditions. In this study, specimens varied in size and a focal distance of 150 mm yielded clinically acceptable accuracy in all cases. Third, since tracking data was not synchronized with image acquisition, hand motion caused errors. Here, hand motion was estimated during acquisition. Experimental data indicate that when HHS was held steady while taking a snapshot (<1 mm in ∼0.1 s), its effect on iSV error was minimal. Additional experimental data were acquired with HHS operated in a “swept” mode to introduce larger hand motion (4-12 mm), and resulting iSV errors increased to 3 to 11 mm, which are not clinically acceptable. HHS was, therefore, held still during acquisition of each snapshot to minimize errors from asynchronization. All images were acquired with hand motion < 0.9 mm for all operators, and the median was 0.3 mm (Figure 6B). Although HHS accuracy does not depend on exposure length, deformity or posture of the spine, type of tissue, or landmarks, image updating requires common features to appear in both iSV and pCT for registration and is only applicable in open spine surgery. In this study, the large and wide exposure was created to evaluate accuracy in various regions, but does not represent the exposure, landmarks (eg, spinous process or lamina), or vertebral alignment required for the image updating technique. Additional preliminary data suggest that vertebral levels can be difficult to acquire when the surgical exposure is narrow due to iSV line of sight restrictions, but the net effect on HHS and subsequent uCT accuracy remains to be assessed. In addition, simulations based on existing data indicate that image updating performance is similar for various lengths of exposure (6, 4, and 3 levels¹³). In principle, image updating can be repeated after pedicle screw placement, although its accuracy may be affected by the limited bony surface available after instrumentation. Studies designed to evaluate pedicle screw placement based on uCT and comparisons of image updating accuracy associated with exposure size are currently underway.

CONCLUSION

We developed an HHS system to acquire intraoperative spine surfaces in open spinal surgery, and assessed its accuracy in 6 cadaver porcine spines. Results show that overall iSV accuracy was 1.2 ± 0.6 mm, and overall computational time was <1 s with batch processing. The uCT results show an overall accuracy of 3.1 ± 0.7 and 2.0 ± 0.5 mm on dorsal and ventral sides, respectively, and validate HHS as a viable image-based registration approach, although further studies are needed to demonstrate uCT accuracy can be maintained at variable HHS focal distances and spine exposure sizes. Overall performance of HHS reported here falls within clinically acceptable ranges, and offers potential for navigation during open spine surgery.

Disclosures

This study was funded by the National Institutes of Health (grant R01EB025747-01). Medtronic Navigation (Medtronic, Louisville, Colorado; Dublin, Ireland) provided the StealthStation. The authors are inventors on patents and patents-pending related to stereovision assigned to the Trustees of Dartmouth College. Drs Fan, Mirza and Paulsen are involved with early-stage commercialization of some of the technologies described in the paper through start-up companies, InSight Surgical Technologies and PEER Technologies.