Augmented Reality Device for Preoperative Marking of Spine Surgery Can Improve the Accuracy of Level Identification

Abstract

Introduction

Wrong-site spine surgery is an incident that could result in possible severe complications. In this present spinal surgery, the accurate spinal level is confirmed via preoperative or intraoperative radiographic marking. However, the location of radiographic marking has been determined from the manual palpation on the landmarks of the body surface. As a result, severe spine deformity can make it hard to identify the spinal level by manual palpation, thus leading to misidentification of the spinal level.

Recently, the use of mixed reality in spine surgery is gradually increasing. In this study, we will demonstrate a head-mounted display (HMD) device that can project a hologram (3D image) of the patient's bone onto the actual patient's body to improve the accuracy of level identification for spine surgery.

Technical Note

3D CT images are created preoperatively, and the bone's STL data (3D data) are generated with the workstation. The created STL data are downloaded to the augmented reality software Holoeyes, installed on the HMD. Through this device, surgeons can view the hologram (3D image) of a patient's bone overlaying on an actual patient's body.

We temporally estimated the spinous process level only by manual palpation without an HMD. Then, we estimated the spinous process level again after matching this hologram to a real bone with an HMD. The accuracy of the level identification with an HMD and without an HMD was examined by radiographic marking in order to evaluate the misidentification rate of the level. Without an HMD, the misidentification rate of the level was at 26.5%, while with it, the rate was reduced to 14.3%.

Conclusions

On preoperative marking, an HMD-projecting bone image onto a patient's body could allow us to estimate the spinal level more accurately. Identification of the spinal level using mixed reality is effective in preventing wrong-site spine surgery.

Introduction

Wrong-site surgery is an unpleasant complication that is sometimes inevitable. In fact, it has been reported that one in two spine surgeons experiences it during their careers¹⁾. Unnecessary exposure of soft tissues or bone resection due to wrong-site surgery because of level misidentification should be avoided for the patient. For that, it is essential to confirm the correct surgical level via radiological examination^1,2). Level confirmation with marking is widely performed preoperatively and intraoperatively by single X-ray or C-arm fluoroscope^1,2). In the thoracolumbar spine, the level is estimated using Jacoby's lines in preoperative marking, often done in lumbar anesthesia. In the cervical spine, the level is estimated from the position of the scapula. In many cases, the level is temporally estimated by palpation of the tip of the spinous process because we can imagine the level by the shape of the spinous process, especially with the difference in size between the cephalad and caudal ones. Then, a needle is inserted into the spinous process as a marker referring to the temporally estimated level, and the position is then confirmed by X-ray or C-arm^1,2). Inserted at the edge of the spinous process or incorrectly inserted between the spinous processes, the needle would be unstable and could go out of the bone during intraoperative exposure. These incidences could then lead to level misidentification, forcing us to reconfirm the level in surgery, which would be a waste of time. Therefore, it is mandatory to stably insert the marker into the spinous processes in the intended position before surgery²⁾.

Recently, studies using various mixed reality (MR) devices have been conducted in the medical field, and reports of their usefulness have increased^3-5). However, there are no reports on the use of MR devices for preoperative marking. This study reports the usefulness of head-mounted display (HMD): HoloLens, with relatively inexpensive augmented reality (AR) software: Holoeyes⁵⁾, for preoperative marking to improve marking accuracy.

Technical Note

CT images were taken using Aquilion one vision edition (Canon Medical Systems Corporation, Tochigi, Japan) or Brilliance 64 CT scanner (Philips Japan, Tokyo, Japan). 3D CT images of the bone were created using a workstation (Zio station 2 plus; Ziosoft Corporation, Japan) (Fig. 1). The STL data of bony elements were then output by the workstation (Fig. 2).

Figure 1.

3D constructed CT image.

The left side image showed a 3D image of the cervical spine with the skull and scapula as landmark. The right side image showed a 3D image of the lumbar spine with the pelvis as a landmark.

Figure 2.

Creating an image with Holoeyes MD (augmented reality software).

The STL data (3D data) are uploaded to the Holoeyes MD server using a web browser. The cervical spine data on the left and the lumbar spine data on the right were uploaded. After uploading the data, it takes a few minutes to complete the data for Holoeyes MD in a head-mounted display. The 3D hologram can be observed by downloading the completed data to the head-mounted display.

The bone image data are displayed on the HMD, AR devices (HoloLens 2; Microsoft Corporation, Redmond, Washington, and Magic Leap 1; Magic Leap, Fort Lauderdale, Florida) using a dedicated application (Holoeyes MD; Holoeyes Corporation, Japan). In Holoeyes MD, the bone can be observed in three dimensions⁵⁾. This 3D image can be overlaid on the patient's body through HMD. With MR technology, this 3D image is displayed as 3D as it actually is in the real body. In other words, if the observer moves and looks at the patient from any direction, the 3D image will be displayed as if viewed from that direction (Fig. 3). Therefore, by displaying the 3D data in true 3D using an HMD, the 3D data can be intuitively grasped in the brain of users (Fig. 3-5).

Figure 3.

The view of the cervical spine in mixed reality.

In mixed reality, the patient’s cervical spine appears to actually exist in the space. By moving and changing the viewing location, the patient’s cervical spine can be observed from all directions in 360 degrees. In the upper left picture, the patient was observed from the right lateral side. In the lower left picture, the patient was observed from the cephalad dorsal side. In the upper right picture, the patient was observed from the caudal right dorsal side. In the lower right picture, the patient was observed from a slightly ventral side of the caudal side.

Figure 4.

Adjusting the hologram to the actual bone.

The hologram is adjusted to the real body on the left picture using Magic Leap 1 (Augmented reality device). While touching the C2 spinous process as a landmark with his left thumb, he uses the controller in his right hand to fine-tune the hologram’s position. The right side picture showed the actual hologram seen by Magic Leap 1 and the posterior neck of the patient. He touched the unevenness of the spinous process with a finger and compared the unevenness on the dorsal aspect of the spinous process in the hologram at the same time. He then confirmed a good match between the tip of the spinous process in the hologram and that in the real body.

Figure 5.

Hologram view in the lumbar spine.

The left side picture is the view of the hologram superimposed on the actual bone position of the patient. The right side picture is the view of the hologram shifted to the patient’s right side so that both the hologram and the actual body can be seen simultaneously. As shown in the right picture, he touched the unevenness of the spinous process with a finger and compared the unevenness on the dorsal aspect of the spinous process in the hologram at the same time. He confirmed a good match between the tip of the spinous process in the hologram and that in the real body.

Marking technique

First, evaluating preoperative X-rays and CT, we realize the relationship between the dorsal unevenness and the spinous processes from the difference in size. With former information in the image of mind, the spinous process level was estimated by manual palpation on the cervical spine referring to the position of the scapular spine and on the thoracolumbar spine in relation to the Jacoby's line or the posterior superior iliac spine. Then, using a skin pen, we drew a circle on the skin at the position of the spinous process and noted its estimated level on the skin (Fig. 6). After that, we use an HMD to estimate the level again. The hologram of the bone is superimposed on the actual bone in the HMD (Fig. 4, 5). The spinous process is then manually palpated to confirm the agreement with the hologram's position. In addition, the position of the scapular spine or iliac crest in the hologram was checked for consistency (Fig. 4, 5). Confirming a high match between the hologram and actual bone, we again noted its estimated level on the skin.

Figure 6.

Method for confirming the level of spinous processes.

As shown in the picture, we draw a circle or line and the estimated level on the dorsal side of the spinous process.

An 18-gage needle was inserted into the spinous process near the surgical site, in the cervical spine cases intraoperatively and in the thoracolumbar spine cases preoperatively. The level is confirmed by this needle marker on the X-ray (Fig. 7). We then examined the agreement between the confirmed level and the estimated level with and without the HMD.

Figure 7.

Confirmation of the level using radiographical markers.

The left intraoperative radiograph showed the insertion of an 18-gage needle into the spinous process of the cervical spine to confirm the level. The right preoperative radiograph showed the insertion of 18-gage needles inserted into the spinous processes of the lumbar spine to confirm the level.

Marking results (Table 1)

Table 1.

The Number of the Level Misidentification Cases Divided by the Surgical Site.

	Cervical	Thoracic	Lumbar	Total
Number of cases	20	1	28	49
Male/Female	13/7	1/0	16/12	30/19
Mean age	68.8	64	67.7	68.1
CSM/OPLL/LCS/LDH/VF/ST	11/8/0/0/0/1	0/0/0/0/0/1	0/0/21/3/2/2
Misidentification cases without an HMD	1	1	11	13
Misidentification cases with an HMD	0	1	6	7

CSM cervical spondylotic myelopathy, OPLL ossification of the posterior longitudinal ligament, LCS lumbar canal stenosis, LDH lumbar disc herniation, VF vertebral fracture, ST spinal cord tumor, HMD head-mounted display

In total, 49 patients who could be marked using an HMD between December 2020 and August 2021 were included in this study. Cervical spine surgery was performed in 20 cases, thoracic spine surgery in 1 case, and lumbar spine surgery in 28 cases. In 13 (26.5%) of the 49 cases, the estimated level by palpation without an HMD was different from the actual level. The estimated level by palpation with an HMD was different from the actual level in 7 (14.3%) of the 49 cases. On 13 misidentification cases without an HMD, 6 of them (46.2%) could be reconfirmed accurately with the addition of an HMD assessment. The misidentification rates by palpation without an HMD were 5% (1/20) in cervical, 100% (1/1) in thoracic and 39.3% (11/28) in lumbar. The misidentification rates by palpation with an HMD were 0% (0/20) in cervical, 100% (1/1) in thoracic and 21.4% (6/28) in lumbar. The misidentification rate tended to be higher in the thoracolumbar spine, and the misidentification rate was noted to decrease with the use of an HMD. When the HMD was not used, level misidentification was within two vertebral bodies. However, when the HMD was used, level misidentification was within one vertebral body. The range of misidentification was reduced by using an HMD.

Discussion

The incidence of wrong-site spine surgery has been reported to be 0.03% (1/3110) to 0.05% (4.5/10000)^1,2). Although infrequent, it remains to be an unpleasant complication that can happen to any surgeon. The gold standard is for the markings to confirm the level before and during surgery to prevent wrong-site surgery^1,2). Although various methods using markings have been tested, it has been reported that the important factor in preventing misidentification is to ensure that the markings are placed on a stable intended site¹⁾. Identifying the accurate spinal level from the anatomy of the body surface, it is possible to mark a stable intended site and draw an appropriate skin incision line preoperatively. This study showed that the misidentification rate of the level was 26.5% by manual palpation of landmarks on the body surface and by information from the preoperative images. Meanwhile, with the use of an HMD, the level of misidentification rate was reduced to 14.3%. We could confirm the surgical level more accurately by using an HMD superimposing a 3D image on the patient's body. In all 49 cases, the surgical level was confirmed almost appropriately, and no wrong-site surgery occurred.

In the past reports of level confirmation using body surface landmarks, the agreement rate of the level was 30-60%⁶⁾. In this study, the agreement rate of the level could be increased up to 85.7% with the assistance of an HMD.

The first possible mechanism by which misidentification may occur even when using HMDs is the difference in terms of posture. There is a difference between the posture in the preoperative image and the intraoperative posture. In the cervical spine, the relative position of the spinous processes of the vertebrae to the occipital bone and scapula changes significantly with posture. Even in the lumbar spine, the relative position of the lumbar spinous processes to the pelvis can change with changes in posture. It has been assumed that the change in the position of the bone that appears with the difference in posture makes it difficult to confirm the position of the spinous processes. The second possible mechanism for misidentification is the inability to directly touch the spinous processes. The thickness of the soft tissue intervening between the spinous processes and the skin varies from person to person. Thick soft tissue makes it difficult to grasp the detailed shape of the spinous processes and other bony prominences by palpation. The difficulty in grasping the shape of the spinous processes due to soft tissues can be a cause of misidentification of the level of the spinous processes. It may be possible in the future to improve the accuracy of level identification by taking preoperative images in the intraoperative posture as much as possible. Adding markers at predetermined locations on the skin during preoperative imaging and displaying these markers in the HMD may increase the accuracy of level identification. Currently, the accuracy with HMDs is not perfect, so marking using a needle is deemed necessary. However, this HDM technique could help to mark the stable intended site and prevent the unexpected site from being marked. The use of HMDs can increase the reliability of marking with needles, which are by no means perfect. If the accuracy could be 100% by using HMDs in the future with the advancement of technology, the marking with needles would be no longer necessary and could reduce the radiation exposure to the patient.

In our cases, the misidentification rate without an HMD was 5% (1/20) for the cervical spine and 41.4% (12/29) for the thoracolumbar spine. This result indicated that the error was more likely to occur in the thoracolumbar spine than in the cervical spine. With an HMD, the misidentification rate was 0% (0/20) for the cervical spine and 24.1% (7/29) for the thoracolumbar spine.

In the cervical, the misidentification rate was lower regardless of the use or non-use of HMDs. This may be because body surface landmarks are easier to recognize in the cervical spine.

The most easily recognized landmark on the body surface is the tip of the spinous process. In the cervical spine, the size of the spinous process often changes significantly at C6 or C7, and this step of the spinous process is considered a good body surface guidance. In the thoracolumbar spine, it is rare to have a stepped spinous process unless there is a slipped vertebra, and the overall gentle nature of the tip of the spinous process makes it difficult to determine by manual palpation alone.

A problem with using an HMD is the time required to create the 3D image data. It took about half an hour to create the data for each case in this study. Another problem is that marking with an HMD takes about 2 minutes longer than the normal marking procedure. Additionally, it costs money to purchase an HMD and use the Holoeyes MD software. Its costs are 4,000 USD and 30,000 USD a year. But, in Japan, most spine surgeries performed using this device system can be billed to the national health insurance as creating intraoperative support images.

It has been reported that the use of AR technology as an intraoperative navigation system can reduce radiation exposure during screw insertion and thus contribute to the safety of surgery by providing intraoperative and preoperative information on the location of vital organs such as blood vessels³⁾. This HMD system also seems to have a promising future, not only for use in marking but also for use in preoperative surgical planning and intraoperative navigation.

Conflicts of Interest: The authors declare that there are no relevant conflicts of interest.

Sources of Funding: None

Author Contributions: Ryoma Aoyama wrote and prepared the manuscript, and all of the authors participated in the study design. All authors have read, reviewed, and approved the article.

Ethical Approval: This study's protocol was not reviewed and approved by the Institutional Review Board, because this is a technical note. However, we conducted this study in compliance with the principles of the Declaration of Helsinki. We made informed consent for participating in this study and got written consent from all patients.