The Clinical Application of Augmented Reality in Orthopaedics: Where Do We Stand?

Abstract

Purpose of Review

The surgical community is constantly working to improve accuracy and reproducibility in patient care, with the goal to improve patient outcomes and efficiency. One area of growing interest with potential to meet these goals is in the use of augmented reality (AR) in surgery. There is still a paucity of published research on the clinical benefits of AR over traditional techniques, but this article aims to present an update on the current state of AR within orthopaedics over the past 5 years.

Recent Findings

AR systems are being developed and studied for use in all areas of orthopaedics. Most recently published research has focused on the areas of fracture care, adult reconstruction, orthopaedic oncology, spine, and resident education. These studies have shown some promising results, particularly in surgical accuracy, decreased surgical time, and less radiation exposure. However, the majority of recently published research is still in the pre-clinical setting, with very few studies using living patients.

Summary

AR supplementation in orthopaedic surgery has shown promising results in pre-clinical settings, with improvements in surgical accuracy and reproducibility, decreased operating times, and less radiation exposure. Most AR systems, however, are still not approved for clinical use. Further research is needed to validate the benefits of AR use in orthopaedic surgery before it is widely adopted into practice.

Introduction

Augmented reality (AR) is gaining popularity within the medical community, and its potential use within orthopaedics continues to grow. The proposed benefits of computer-assisted surgery, and in particular AR, include improved accuracy, reproducibility, and decreased radiation exposure, to name a few. AR is distinct from virtual reality (VR) in that it superimposes digital images in real time onto real space [1], while VR uses an exclusively digital environment.

In a paper published in 2000 by Nikou et al., they described that an AR system was essentially comprised of three parts: a position tracking system, a display system, and a system control software. The position tracking system, as the name implies, allows for monitoring of both the location and orientation of an object of interest. The display system shows the real image combined with a virtual image on either a standard screen or a head-mounted, semi-transparent display. Finally, the control software has the crucial task of merging the data from the tracking system with the virtual images for display on the screen. This software must process and update information with great speed in order to provide the most accurate and real time images for the surgeon [2]. There are a variety of AR systems currently on the market, with advances in the technology rapidly being developed, but they generally still require these three essential elements. Most systems also require supplementation with traditional imaging modalities such as fluoroscopic images or CT scans which are obtained preoperatively. The system control software will use these images, as well as markers that are attached to the patient and/or instruments, to determine the relative position of objects of interest within the operative field [3]. This article aims to present some of the areas of AR research within orthopaedics over the past 5 years.

Current State of Augmented Reality in Orthopaedics

Within the realm of orthopaedics, there is a wide array of AR technology being studied, with new technologies being developed rapidly. As surgical procedures grow in complexity without changes in the stringency of duty hour restrictions, AR offers an innovative way to insure surgical trainees develop their surgical skills with efficiency and in an environment that is safe. As mentioned above, the hope is that AR will be able to improve surgical accuracy and reproducibility, decrease operative time, and reduce radiation exposure to patients and staff.

Fracture Care and Osteotomies

AR is slowly beginning to gain interest in fracture care. Percutaneous sacroiliac (SI) screw insertion, for example, is an orthopaedic procedure with a narrow margin for error, requiring a high level of skill and precision [4]. Not only are the intraoperative images difficult to correlate with the operative site, but screws can perforate adjacent vessels or nerves, leading to potentially severe complications. A deep understanding of 3-dimensional anatomy is critical for executing this case. A recent cadaveric study by Wang et al. looked at the use of AR in percutaneous SI screw insertion. They used a see-through, head mounted display (HMD), an optical tracking system, LCD monitor, and infrared markers to percutaneously insert 3mm guidewires and then cannulated 6.5mm partially threaded screws across the SI joint. They were able to insert these screws with less than 5mm in variation in trajectory accuracy and without any screw perforation.

Augmented reality has also been utilized in the insertion of distal locking screws in intramedullary nails of long bones. Many variations of the traditional “perfect circle” technique for placement of these screws have been developed, but all require the use of a significant number of x-ray images. One study lists as many as 48 x-ray images to successfully place the distal locking screws [5]. Ma et al. developed a novel AR system that superimposes 3D images of the intramedullary nail as well as the planned path of the drill and screw onto the patient’s extremity. This system required no x-ray guidance for placement of the screw and also shortened mean time to complete screw placement [6]. Other studies have demonstrated a reduction in total time for intramedullary nailing by nearly 36% with the use of a Microsoft HoloLens 2 AR system [7].

Osteotomies are another realm of orthopaedics where AR has the potential to make a positive impact. One example is the Ganz’ periacetabular osteotomy (PAO), which consists of four technically demanding osteotomies. Kiarostami et al. published a proof of concept study in which they used head-mounted AR guidance to perform PAO’s on 15 sawbone models. They found an increased level of accuracy in performing these osteotomies using the AR guidance in comparison to 15 osteotomies performed in the traditional freehand technique [8].

Adult Reconstruction

Computer navigation has been used in total joint replacement for many years with its popularity among surgeons and patients growing rapidly. However, its initial cost can be prohibitive, and most systems still require the surgeon to take their eyes off of the operative field to view a monitor [9].

In total hip arthroplasty (THA), the anteversion and inclination angles of the acetabular component are critical for hip biomechanics. Logishetty et al. performed a randomized trial in which they looked at the accuracy of acetabular implant orientation performed between two groups of fourth-year medical students. Both groups received 4 weeks of training in acetabular cup placement. One group was trained by an arthroplasty surgeon, while the other group used an AR system with live holographic feedback using the Microsoft HoloLens. During their training sessions, the AR group had smaller mean error in implant orientation compared with the non-AR group [10].

Ogawa et al. developed an AR system for placement of the acetabular component in an attempt to improve the accuracy and reproducibility of these angles. Their system, the AR-HIP system, is a free smartphone application that allows the surgeon to view a superimposed image of the acetabular cup on the surgical field through a display on a smartphone. They found that the AR-HIP System had similar accuracy in measuring the anteversion and inclication angles of the acetabulum in comparison to the use of a goniometer [11].

Tsukada et al. published an article on an AR system for total knee arthroplasty (TKA) that used a similar smartphone application, the AR-KNEE system. This application displays superimposed images of the target angles of varus/valgus and posterior slope of the tibia to allow for real-time adjustments by the surgeon [12]. The hope is that technologies such as these will improve precision in placement of implants.

Oncology

Oncology is another area of orthopaedics with continual innovations in AR. Cho et al. developed an AR system for the resection of bone tumours in long bones. They were able to create a system using a low-cost tablet PC which was able to act as a work station and a position tracker at the same time. This system demonstrated greater accuracy in tumour resection in comparison with conventional resection techniques [13].

Pelvic tumours pose a particular challenge in treatment. With the complexity of pelvic anatomy and proximity to critical internal organs and vasculature, AR has the potential to change outcomes in these surgeries. In 2018 a similar tablet PC based AR system was developed to assist in the resection of these tumours. Using this system, the team was able to display safety margins of tumour resection in real time, which allowed for a significantly smaller resection error and zero tumour violations in comparison to conventional resection techniques [13, 14].

Spine

One area of particular interest within spine surgery is pedicle screw placement using AR. Accurate placement of pedicle screws is critical in order to avoid severe and potentially life-threatening complications. Molina et al. published research using an AR head-mounted display which was able to show a retinal projection of a 3D reconstruction of the spine and planned trajectories for pedicle screw placement in cadaveric specimens. Postprocedural CT scans demonstrated high levels of accuracy and reproducibility in their study [15]. Similarly, Siemionow et al. published a cadaveric study which used a virtual image overlay projected onto the spine. They found that in all of their cadavers, they were able to percutaneously find the starting point for pedicle screw insertion with a high level of accuracy, as assessed by CT, and an average time of 38.2 s per pedicle [16].

In one of the few clinical studies recently published on this subject, Elmi-Terander et al. used an AR surgical navigation system encompassing a surgical table, motorized C-arm with 2D and 3D capabilities, optical cameras, and noninvasive patient motion tracking markers to place pedicle screws. They demonstrated a 94.1% accuracy in placement of 253 lumbosacral and thoracic pedicle screws in 20 live patients [17].

Other areas of AR research within spine surgery include AR-assisted rod bending, facet joint injection, MIS spine surgery, and tumour biopsies, to name a few [18].

Conclusion

In the last few decades, there have been great advances and growing interest in the use of AR within orthopaedic surgery. As the use of robotics and navigation become the norm in certain specialties, AR is looked to as the next horizon. Proposed benefits of AR-assisted orthopaedic procedures are continually being discovered and include improved surgical accuracy and reproducibility, less morbidity as a result, decreased operative time, and reduced radiation exposure to both the surgical staff and patient. There is also great potential for the use of AR in resident education. A recent article by Ha et al. reviewed some of the educational benefits of using AR in resident training. They noted that AR has the potential to be a great educational supplement in the current state of duty hour restrictions and less resident autonomy in the operating room [19].

Although technological developments are rapidly growing, there are still significant roadblocks that make adoption of this technology into actual practice difficult. Many of the current technologies are still in their infancy with the majority of published research in pre-clinical settings using cadaveric or sawbones models. Not only that, systems can be cumbersome and often have high learning curves. Cost-to-benefit ratio is also an area of significant concern for many hospital systems. Therefore, well-established safety improvements and overall benefits to AR usage in comparison to traditional techniques will likely need to be validated before these systems are widely adopted into clinical practice.

It should be noted that this review is certainly not comprehensive. It is an attempt to give a broad update on the current progress of AR in orthopaedics using literature over the past 5 years. Future studies will be needed to confirm the favourability of AR usage in orthopaedics.

Declarations

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Conflict of Interest

J. Hunter Matthews owns stock in Johnson & Johnson.

John S. Shields declares no conflict of interest.