Augmented reality and artificial intelligence-assisted surgical navigation: Technique and cadaveric feasibility study

Abstract

Purpose:

Augmented reality-based image overlay of virtual bony spine anatomy can be projected onto real spinal anatomy using computer tomography-generated DICOM images acquired intraoperatively. The aim of the study was to develop a technique and assess the accuracy and feasibility of lumbar vertebrae pedicle instrumentation using augmented reality-assisted surgical navigation.

Subjects and Methods:

An augmented reality and artificial intelligence (ARAI)-assisted surgical navigation system was developed. The system consists of a display system which hovers over the surgical field and projects three-dimensional (3D) medical images corresponding with the patient's anatomy. The system was registered to the cadaveric spine using an optical tracker and arrays with reflective markers. The virtual image overlay from the ARAI system was compared to 3D generated images from intraoperative scans and used to percutaneously navigate a probe to the cortex at the corresponding pedicle starting point. Intraoperative scan was used to confirm the probe position. Virtual probe placement was compared to the actual probe position in the bone to determine the accuracy of the navigation system.

Results:

Four cadaveric thoracolumbar spines were used. The navigated probes were correctly placed in all attempted levels (n = 24 levels), defined as Zdichavsky type 1a, Ravi type I, and Gertzbein type 0. The virtual overlay image corresponded to the 3D generated image in all the tested levels.

Conclusions:

The ARAI surgical navigation system correctly and accurately identified the starting points at all the attempted levels. The virtual anatomy image overlay precisely corresponded to the actual anatomy in all the tested scenarios. This technology may lead more uniform outcomes between surgeons and decrease minimally invasive spine surgery learning curves.

INTRODUCTION

Minimally invasive approaches to spine surgery provide many potential advantages over open procedures. These include decreased paraspinal dissection and other stabilizing tissue injuries, blood loss, and pain.[1] Computer-assisted three-dimensional (3D) navigation was introduced in spine surgery to improve pedicle screw placement accuracy.[2] Spine surgery navigation utilizes preoperative 3D imaging of spinal anatomy obtained by computed tomography (CT), magnetic resonance imaging, or X-rays.[3] There are several limitations which are common to all of the currently available navigation systems. All force surgeons to look away and verify the navigated instrument position on a flat screen monitor located away from the surgical field. This requires surgeons to develop complex and unnatural hand–eye coordination skills. These skills have a steep learning curve and rely on the surgeon's visuospatial skills and ability to translate 2D images into 3D anatomical structures. They also do not afford surgeons a means to visualize internal anatomical structures without soft-tissue exposure. Finally, they are not intuitive or ergonomic for surgeons, which limits their functionality and prolongs surgical time. Augmented reality has the potential to address the limitations of the current surgical navigation systems by improving anatomic visualization, ergonomics, and intraoperative workflow. The complexity of vertebral anatomy leads to a steep learning curve for pedicle screw placement in trainees.[4] An augmented reality navigation system may decrease the learning time required to safely perform minimally invasive surgical (MIS) procedures. The aim of this study was to assess the accuracy and feasibility of lumbar pedicle screw placement using a newly developed augmented reality and artificial intelligence (ARAI) surgical navigation system.

SUBJECTS AND METHODS

Four intact human cadaver lower torso and spine (T12-pelvis) with no history of the spine surgery were used. The study was conducted in compliance with the ethical guidelines for human cadaver studies.

The cadaver was placed prone on the surgical table. A surgical registration grid (RG) with reflective markers was applied topically to the lumbar area [Figure 1]. A spine array with reflective markers was inserted in the posterior superior iliac spine. Next, a preoperative CT scan was performed with O-arm Surgical Imaging System (Medtronic Inc., Minneapolis, MN, USA). The RG was registered to the spine array using an artificial intelligence sphere identification algorithm, while the reflective spheres were identified and their position was recorded by the V120 infrared optical tracker (NaturalPoint, Inc. Corvallis, OR, USA). The ARAI (Holo Surgical Inc., Chicago, IL, USA) surgical navigation system was brought into the operative field and registered to the spine array.

Figure 1.

Registration grid placed over the patient (black) and anatomy array clamped to the spinous process. Both the registration grid and anatomy array were fully visible by the optical tracker

Using the systems' augmented reality 3D display, a metal probe (Medtronic Inc., Minneapolis, MN, USA) was hovered over each lumbar spine vertebrae. The probe's virtual projection, as seen on the 3D display, was used to localize the level and mark the location of the skin incision. After a stab incision was made, the probe was percutaneously advanced through the soft tissues and docked on the pedicle starting point using the ARAI system guidance. The probe was then advanced into the pedicle with a mallet, and its position was secured [Figure 2]. Figure 3a demonstrates a surgeon using the system and advancing the navigated probe into the pedicle. Next, an intraoperative CT scan was obtained to assess the adequacy of probe placement [Figure 3b]. A probe was used for analysis instead of pedicle screws in order to facilitate image analysis and decrease the amount of imaging artifacts. Starting point placement adequacy was judged by two orthopedic spine surgeons using intraoperative imaging [Figure 4]. The adequacy of the pedicle probe placement was assessed with the Gertzbein grading system,[5] Ravi score,[6] and Zdichavsky score.[7] In the Gertzbein scale, cortical breaches are described by the extent of extra-cortical screw violation[5] and the relative position of the screw to the pedicle is graded as follows: (i) completely within the pedicle; (ii) pedicle wall breach <2 mm; (iii) pedicle wall breach = 2–4 mm; and (iv) pedicle wall breach >4 mm⁵.[8] In the Ravi scale, the trajectory was classified as acceptable, straight, lateral, or medial.[6] The Zdichavsky scale classification is depicted in Figure 5.[7]

Figure 2.

(a) Virtual three-dimensional projection of the lumbar spine; virtual Jamshidi needle along with the orthogonal planes demonstrating placement of the navigated needle. Cadaver in prone position, head to the left. (b) Same cadaver specimen in prone position demonstrating the actual Jamshidi needle placement. Head to the left

Figure 3.

(a) User placing Jamshidi needle in the lumbar spine pedicle under ARAI system guidance. (b) During the procedure, validation was performed by identifying the relationship of the virtual Jamshidi needle (pictured on computer monitor) to the actual Jamshidi needle as seen on intraoperative scans

Figure 4.

(a) Axial image from intraoperative computed tomography scan demonstrating the starting point with Jamshidi needle in place on the L5 pedicle. (b) Sagittal image from intraoperative computed tomography scan demonstrating starting point with Jamshidi needle in place in the L4 pedicle

Figure 5.

Zdichavsky scale

In such fashion, the lumbar pedicles L3, L4, and L5 were instrumented with the navigated probe bilaterally. After surgery, the pedicle's morphology was measured on the presurgery CT: sagittal and transverse width and sagittal and transverse angle with the use of Osirix [Figure 6 and Table 1]. The measurements were performed according to the descriptions presented in the literature.[9,10,11] The transverse pedicle angle was measured as an angle between the cord and midvertebral line [Figure 6b]. A total of four board-certified orthopedic surgeons, one per cadaver, took part in the experiment. The time it took each surgeon to cannulate the pedicle was recorded.

Figure 6.

(a) Sagittal pedicle angle. (b) Transverse pedicle angle

Table 1.

The average measurements of instrumented pedicles

Pedicle measurements	L3	L4	L5
Transverse width [mm]	8.9	7.9	15.9
Sagittal width [mm]	18.1	8.9	13.5
Transverse pedicle angle [°]	29	31	39
Sagittal pedicle angle [°]	10	7	9

RESULTS

A total of 24 pedicles were instrumented with a navigated probe in four cadavers, six levels per cadaver. All the navigated probes were placed in an excellent position via the percutaneous technique as described above. Excellent position was defined as Zdichavsky type 1a, Ravi type I, and Gertzbein type 0 [Table 2] placement. The highest level of instrumentation was L3 and the lowest was L5. The average time to cannulate a pedicle using the system was 38.2 s (range 29–47 s) per surgeon [Table 3].

Table 2.

The accuracy of screw placement

	L3	L4	L5
Zdichavsky	1a	1a	1a
Ravi	0	0	0
Gertzbein	I	I	I

Table 3.

Average time to cannulate pedicle per surgeon

Surgeon	Average time to cannulate pedicle with Jamshidi (6 pedicles per surgeon)
1	29 seconds
2	47 seconds
3	45 seconds
4	32 seconds
Average	38.25

DISCUSSION

The current trends toward MIS techniques represent an important shift in the practice of contemporary spinal surgery.[7] Accurate placement of pedicle screws provides optimal mechanical strength while improving patient safety,[7] and decreasing the morbidity of open spine procedures.[1]

The development of augmented reality computer-assisted spine surgery requires that 3D spine images be derived from CT and two-dimensional (2D) X-rays using 3D/2D registration.[12] Proper pedicle screw placement is dependent on the accuracy of this 3D/2D registration. In this study, extrinsic and intrinsic methods for 3D/2D registration were developed. Extrinsic methods rely on the implantation of instruments or markers into the bone.[12,13,14]

The markers visualized on both 2D and 3D images and their dimensional position are adjusted as needed.[15] The adjusted markers are then utilized to align the spine. Intrinsic methods utilize anatomical structures to perform 3D/2D registration. This can be anatomic feature based,[12] imaging intensity based[16,17] or gradient based.[18] Feature-based registrations are concerned with ending the transformation that minimizes the distances between 3D features, extracted from the preinterventional image and the corresponding 2D features.[12] The features utilize geometrical entities such as isolated points, contours, or surfaces.[12] In point-based methodology, the tip of the spinous or transverse processes is identified in the preoperative CT images and localized during the surgery in the physical space of the patient by touching with a probe.[17] Surface-based methodology uses a probe which is moved along the surface of the vertebra. The recorded surface points are matched to that of a vertebral bone surface model extracted from the CT image.[17] Contour-based methods require segmentation of the object surface in both preoperative CT images and intraoperative X-ray images. Registration is in turn performed by minimizing the distance between the 3D surface model and lines presenting the contour of anatomical structures derived from the X-rays.[17,19] A significant advantage of utilizing an intrinsic versus an extrinsic method is the noninvasive character of the registration. Intrinsic methods do not need the implementation of markers on or into the patient's body. This study demonstrated that our new intrinsic, feature-based 3D/2D registration method for CT and X-rays images was able to accurately identify pedicle screw starting points, with 3D image projection that surgeons could visualize and use to alter pedicle screw trajectory in real time.

Efficient pedicle instrumentation placement shortens surgical time, total anesthesia time, and in percutaneous spine surgery, radiation exposure for both patient and operating room staff.[20] He et al. reported a mean time of intraoperative X-ray imaging for inserting one pedicle screw of 7.99 ± 2.45 min (range 5.25–14 min).[21] In addition, when comparing fluoroscopic X-ray assistance with robot-assisted pedicle screw placement, a decrease in overall screw placement time was found, although there were no significant differences in single screw placement times.[22] The time required to place a single pedicle screw remained between 5 and 6 min using both techniques.[22] In this cadaver study, we demonstrated that the average time it took a board-certified orthopedic spine surgeon to cannulate the pedicle with a navigated pedicle probe was 38.2 s. Although this did not include the time it took the time from cannulation to screw insertion, this technique could significantly shorten the pedicle instrumentation portion of the surgical procedure while drastically decreasing radiation exposure to the operating room staff.

The ARAI surgical navigation system correctly and accurately identified the starting points at all attempted levels. The virtual anatomy image overlay precisely corresponded to the actual anatomy in all the tested scenarios. The virtual anatomy image projected was responsive, and surgeon's perspective was updated in real time. These encouraging preliminary results warrant further investigation, as ARAI surgical navigation systems could provide a powerful tool to increase the speed and accuracy of minimally invasive spine surgeons, leading to improved patient outcomes.

CONCLUSION

The ARAI surgical navigation system correctly and accurately identified the starting points at all attempted levels. The virtual anatomy image overlay precisely corresponded to the actual anatomy in all the tested scenarios. The virtual anatomy image projected was responsive, and surgeon's perspective was updated in real time. These encouraging preliminary results warrant further investigation, as ARAI surgical navigation systems could provide a powerful tool to increase the speed and accuracy of minimally invasive spine surgeons, leading to improved patient outcomes.

Financial support and sponsorship

This study was financially supported by HoloSurgical Inc.

Conflicts of interest

The following authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Dr. Siemionow – Holosurgical Inc.; Dr. Luciano – Holosurgical Inc., Dr. Lewicki – Holosurgical Inc.