Abstract
Background
Computer assisted surgical navigation systems are designed to improve outcomes by providing clinicians with procedural guidance information. The use of new technologies, such as mixed reality, offers the potential for more intuitive, efficient, and accurate procedural guidance. The goal of this study is to assess the positional accuracy and consistency of a clinical mixed reality system that utilizes commercially available wireless head-mounted displays (HMDs), custom software, and localization instruments.
Methods
Independent teams using the second-generation Microsoft HoloLens© hardware, Medivis SurgicalAR© software, and localization instruments, tested the accuracy of the combined system at different institutions, times, and locations. The ASTM F2554-18 consensus standard for computer-assisted surgical systems, as recognized by the U.S. FDA, was utilized to measure the performance. 288 tests were performed.
Results
The system demonstrated consistent results, with an average accuracy performance that was better than one millimeter (.75 ± SD .37 mm).
Conclusion
Independently acquired positional tracking accuracies exceed conventional in-market surgical navigation tracking systems and FDA standards. Importantly, the performance was achieved at two different institutions, using an international testing standard, and with a system that included a commercially available off-the-shelf wireless head mounted display and software.
Keywords: Augmented reality, mixed reality, computer-assisted surgery, surgical navigation, HoloLens
Background
Computer assisted surgical navigation systems are designed to provide clinicians with information to guide procedures more effectively.1-3 The core capabilities of these systems, specifically image-patient registration and real-time instrument tracking, depend on accurate positional tracking data to precisely localize anatomical targets, guide the placement of medical devices, avoid critical structures, and identify tumor margins.4,5
Conventional navigation systems utilize two-dimensional monitors, located away from of the surgical field. However, this orientation requires the operator to constantly shift their attention from the patient to the video monitor, which is inefficient, ergonomically challenging, and results in cognitive fatigue. 6 Furthermore, conventional navigation tracking systems typically require cumbersome and high-cost hardware installation, thereby limiting the availability of these tools to those who can afford them, as well as restricting useability and applicability beyond the operating room. 7
Head-mounted mixed reality guidance systems, that leverage commercial off-the-shelf hardware, have the potential to address these conventional limitations. However, literature review of such systems reveals a wide range of mean registration errors ranging from .76 mm to 8.22 mm.8-25 Moreover, these different accuracy and performance assessments have utilized a variety of non-standardized custom measurement protocols and reporting practices, which prevents direct intersystem comparisons and generalizability.
The Veterans Health Administration (VHA) is the largest integrated health care system in the United States and serves a unique population who are at increased risk of poor outcomes due to older age and multiple co-morbidities.26-29 As part of ongoing efforts to enhance the quality, safety, and efficiency of health care, VHA is working on ways to advance care, such as with image guided surgical systems. The focus of this study is to assess and quantify the positional accuracy of a co-developed mixed reality (MR) system to enhance presurgical planning and procedural guidance.
Methods
The main technical components of the system utilized included: 1) second-generation HoloLens® (HL2) by Microsoft Corporation (Redmond, Washington, USA); 2) SurgicalAR® clinical MR software from Medivis Inc (New York, New York); 3) localizer Instrument from Medivis; 4) wireless local area network (WLAN) from Netgear Inc (San Jose, California); 5) FDA recognized American Society of Testing and Materials (ASTM) F2554-18 consensus standard precision phantom and guidelines.
Head Mounted Display
The second-generation Microsoft HoloLens (HL2) is a commercial off-the-shelf general-purpose mixed reality HMD that runs on the Microsoft Windows operating system. The HL2 incorporates a custom holographic processing unit (HPU), central processing unit (CPU), graphical processing unity (GPU), and sensors; including a time-of-flight (ToF) infrared depth sensor, gyroscope, magnetometer, accelerometer, dual eye tracking cameras, microphones, four grayscale environment tracking cameras and a 1080p color video camera, which also supports coupling with real-time hand tracking, eye tracking, and voice commands. The HL2 natively supports Wi-Fi/Bluetooth connectivity and 5G cellular network transmission via USB-C dongle. The spatial self-awareness of HL2 is driven by its ‘simultaneous localization and mapping’ (SLAM) algorithm which constructs detailed 3D maps of the surrounding environment’s geometry and calculates its own position within that real world space. This capability, along with other complimentary algorithms, such as Late Stage Reprojection (LSR), allow projected holograms to maintain their position in the real-world coordinate space even as the user moves in their environment. The HL2 utilizes a microelectromechanical (MEMS) laser-based full-color stereoscopic waveguide display, providing true depth perception of virtual objects within the user’s visual field.30,31
Wireless Local Area Network
A wireless local area network (WLAN) was utilized to wirelessly connect the HL2 using a Netgear AX1800 access point.
Software
The SurgicalAR® software from Medivis is 510(k) FDA-cleared under product code LLZ for medical visualization and surgical planning. The software converts standard two-dimensional computed tomography (CT) and magnetic resonance imaging (MRI) images into interactive three-dimensional holograms using HMDs such as the HL2. The localization features of this software were utilized for navigation and localization testing on the ASTM F2554-18 consensus standard precision phantom.
Localizer Instrument
The Medivis designed Point Localization Instrument (PLI) has a 115 mm stem, with a sharp tip for precise point acquisition on one end, and two-toned localization marker on the other end for high-fidelity optical tracking (Figure 1).
Figure 1.
Point Localization Instrument (PLI) designed and manufactured by Medivis, Inc.
Measurement Phantom
A phantom is a physical object that is used to assess accurate measurements, and verify systems meet national or international technical and scientific standards. 32 ASTM International, founded in 1898 and formerly known as American Society of Testing and Materials, is an international standards organization that has developed and published >12,000 consensus technical standards for a wide range of materials, products, systems, and services in >140 participating countries. 33 This study utilized the phantom and protocol from ASTM F2554-18 – “Standard Practice for Measurement of Positional Accuracy of Computer-Assisted Surgical Systems.” 34 This standard was originally issued in 2010 and most recently updated in December 2018. ASTM F2554-18 was selected as the most appropriate testing standard for our study, as it is recognized as a U.S. Food and Drug Administration (FDA) consensus standard for stereotaxic instruments, (FR Recognition Number 11-350). 35 This ASTM standard covers tracking system accuracy and repeatability for locating individual points in real world space. The standard also includes measuring techniques and reporting guidelines to determine the location of a point relative to a coordinate system, relative linear point-to-point accuracy, and repeatability of coordinates of a single point over a wide range of instrument angles and orientations. In this study, the coordinate system of reference is the HoloLens SLAM Cartesian coordinate system. Per the ASTM protocol, a standard half circle protractor marked in degrees was utilized for determining positional angles of the PLI. 34
Per the consensus standard guidelines, the ASTM phantom was machined from a solid block of stainless steel with the Cartesian X, Y, Z coordinates of forty-seven divots spread across five machined faces (Figure 2). The location of each divot was certified to .001 mm precision using a calibrated Coordinate Measuring Machine (CMM), making the phantom 10X more accurate than the accuracy required to be reported by the tracking system being assessed. 34 Certification of the CMM was performed by Hexagon Manufacturing Intelligence (North Kingstown, Rhode Island).
Figure 2.
Illustration (left) and photograph (right) of the ASTM F2554-18 phantom. *Illustration reprinted, with permission, from ASTM F2554-18, Standard Practice for Measurement of Positional Accuracy of Computer-Assisted Surgical Systems, copyright ASTM International. A copy of the complete standard may be obtained from www.astm.org. Photograph from the author team.
Data Collection
Two co-authors independently acted as testers, and during the testing, two additional co-authors recorded the data for the ASTM F2554-18 protocol at their respective facilities. Co-author (LQ) performed the ASTM testing, and during the testing, co-author (CTM) recorded the data of that testing on August 16, 2021, at Medivis Headquarters in New York, New York with stable overhead fluorescent lighting, 75o F ambient temperature and 58% ambient humidity. Co-author (TFO) also performed the ASTM testing, and during the testing, co-author (DMA) recorded the data of that testing on October 13, 2021, at the VA Palo Alto Health care System in Palo Alto, California with stable overhead fluorescent lighting, 72o F ambient temperature and 77% ambient humidity. A Dell Precision personal computer and Dell 1080p monitor were utilized to display, capture, and record the data.
Point Accuracy Measurements
The ASTM consensus standard contains five different point accuracy measurement assessment sections: 1) Single Point Measurement, 2) Angle of Rotation, 3) Angular Position – Perpendicular, 4) Angular Position – Parallel, and 5) Distance Measurement Between Points.
In keeping with the ASTM consensus standard, the phantom was systematically rotated on the table, in relationship to the operator, at 90o intervals, during each of the accuracy measurement assessments. This allowed each of the measurement assessments to be repeated at four different phantom orientations (Figure 3):
1. Perpendicular to the Operator = 0o
2. Maximum Trackable Rotation = 180o
3. Orthogonal Direction #1 = 90o
4. Orthogonal Direction #2 = 270o
Figure 3.
Photographs demonstrating the four ASTM F2554-18 phantom orientation subsections that were utilized for the experiments: 1) perpendicular to operator plane = 0o, 2) maximum trackable rotation = 180o, 3) orthogonal direction #1 = 90o, 4) orthogonal direction #2 = 270.o
Single Point Measurement
The ASTM F2554-18 Section 8.3.3 protocol was followed to assess single point measurement accuracy. The PLI tip was touched to point #20 on the phantom for 1 second. Six tests were performed separately by each operator, for each of the four phantom orientations.
Angle of Rotation
The ASTM F2554-18 Section 8.3.4 protocol was followed to assess whether accuracy of the coordinates measured at the tip of the tool was affected by the angle of rotation of the tool about its axis. The PLI’s tip position, at point #20 on the phantom, was recorded as the tool was rotated about its axis at 15° intervals from 0°–360°. The rotational angles were recorded when data was lost and regained because the attached PLI marker was rotated in and out of the field of view.
Angular Position – Perpendicular
The ASTM F2554-18 Section 8.3.5 protocol was followed to assess if the accuracy of the coordinates measured at the tip of the PLI was affected by the right-left angular position of the PLI in the plane perpendicular to the HMD. A half-circle protractor scale was used on the phantom at point #20 where the PLI tip was placed. The PLI was leaned back and forth along the plane perpendicular to the 90°–270° axis. The PLI tip position accuracy was recorded at 10° intervals within the measurement range of 70°–120°.
Angular Position – Parallel
The ASTM F2554-18 Section 8.3.6 protocol was followed to assess if the accuracy of the coordinates measured at the tip of the PLI was affected by the angular position in the plane parallel to the camera. A half-circle protractor scale was used on the phantom at point #20. The PLI was leaned forward and backward along the plane perpendicular to the 0°–180° axis. The PLI tip position accuracy was recorded at 20° intervals within the measurement range of 50°–150°.
Distance Measurement Between Points
The ASTM F2554-18 Section 8.3.6.1 was followed to assess the accuracy of measuring distances between different point pairs on the phantom. The distance error was computed as the difference measured by the mixed reality system, and the known distance between the points on the certified phantom. Six different point-pairs were measured on the phantom (Table 1).
Table 1.
Ground Truth Distances Between Six Selected Divot Point-pairs on the ASTM F2554-18 CMM-Verified Phantom.
| Point pairs | Ground Truth Distance (mm) |
|---|---|
| 1 – 18 | 129.99 |
| 1 – 19 | 129.95 |
| 1 – 20 | 45.96 |
| 20 – 24 | 45.96 |
| 24 – 36 | 59.26 |
| 36 – 45 | 36.94 |
This study received Determination of Non-Human Subjects Research from Stanford IRB (Stanford University, Stanford, CA, USA). Protocol # 65205.
Stastistics
The position error was calculated for both testers who performed all of the described point accuracy measurement tests. The Wilcoxon signed-rank test was used to measure statistical differences for intra- and inter-variability between investigators measurements. The results are compiled into a boxplot for each section. All statistical analyses were performed using R version 4.0.4. Mean values ±standard deviation was calculated and reported for each investigator, section, and subsection yielding a total system error.
Results
Results for the five different point accuracy measurement sections were analyzed separately for both testers. The calculated accuracy of the system was obtained by subtracting the system measured distance, from the known phantom positional measurement distance (measured in millimeters).
Single Point Measurement (ASTM F2554-18 Section 8.3.3)
For the Single Point Measurement, the combined mean accuracy for all subsections measurements were less than one millimeter. However, subsection measurements were observed as being statistically different between the two different testers at phantom position orientations of 0o and 270o (Table 2 and Figure 4).
Table 2.
Single Point Measurement Error results are Presented for Each Phantom Orientation Subsection Measured in Millimeters; With Mean, SD, Min and Max Reported Across 6 Tests per Tester.
| Phantom Orientation | Tester | Mean | SD | Min | Max |
|---|---|---|---|---|---|
| 0o | 1 | .45 | ±.19 | .22 | .74 |
| 2 | 1.05 | ±.04 | .98 | 1.11 | |
| 180o | 1 | .65 | ±.44 | .05 | 1.51 |
| 2 | .38 | ±.27 | .09 | .87 | |
| 90o | 1 | .37 | ±.19 | .20 | .70 |
| 2 | .49 | ±.44 | .21 | 1.48 | |
| 270o | 1 | .62 | ±.11 | .50 | .77 |
| 2 | .14 | ±.05 | .04 | .21 | |
| All subsections | 1 | .52 | ±.23 | ||
| 2 | .51 | ±.20 |
Figure 4.
Single Point Measurement error results are presented for each phantom orientation subsection, measured in millimeters; with Mean, SD, Min and Max reported across 6 tests per tester.
Angle of Rotation (ASTM F2554-18 Section 8.3.4)
For the Angle of Rotation measurement, the tracking of the PLI rotation was lost at 90o and regained at 285o for each phantom position orientation. In other words, the PLI was trackable at 12 PLI rotational positions along its axis (0o, 15o, 30o, 45o, 60o, 75o, 285o, 300o, 315o, 330o, 345o and 360o). The data from the 12 trackable rotational PLI rotational positions were assessed for each of the four phantom positions. The overall mean for tester 1 was slightly over one millimeter, and approximately half a millimeter for tester 2. There was no statistical difference observed between the two different testers for each of the subsections assessed (Table 3 and Figure 5).
Table 3.
Angle of Rotation Error results for Each Phantom Orientation Subsection in Millimeters With Mean, SD, Min and Max Reported Across 12 Individual Tests per Tester Within the Trackable Range.
| Phantom Orientation | Tester | Mean | SD | Min | Max |
|---|---|---|---|---|---|
| 0o | 1 | 1.40 | ±1.08 | .34 | 4.80 |
| 2 | .39 | ±.23 | .00 | .89 | |
| 180o | 1 | .80 | ±.36 | .33 | 1.47 |
| 2 | .45 | ±.13 | .28 | .68 | |
| 90o | 1 | 1.40 | ±1.29 | .44 | 5.57 |
| 2 | .58 | ±.30 | .29 | 1.16 | |
| 270o | 1 | .83 | ±.52 | .31 | 2.20 |
| 2 | .70 | ±.34 | .19 | 1.29 | |
| All subsections | 1 | 1.11 | ±.81 | ||
| 2 | .53 | ±.25 |
Figure 5.
Angle of Rotation error results for each phantom orientation subsection, measured in millimeters; with Mean, SD, Min and Max reported across 12 individual tests per tester within the trackable range.
Angular Position – Perpendicular (ASTM F2554-18 Section 8.3.5)
For the Angular Position – Perpendicular, assessment the overall average error across all subsections was approximately 1 millimeter. There were no statistical differences observed between each tester for each of the subsections assessed (Table 4 and Figure 6).
Table 4.
Angular Position – Perpendicular Error results are Presented for Each Phantom Orientation Subsection in Millimeters With Mean, SD, Min and Max Reported Across 6 Tests per Tester.
| Phantom Orientation | Tester | Mean | SD | Min | Max |
|---|---|---|---|---|---|
| 0o | 1 | .98 | ±.34 | .76 | 1.49 |
| 2 | .74 | ±.50 | .24 | 1.77 | |
| 180o | 1 | 1.08 | ±.50 | .40 | 1.99 |
| 2 | .81 | ±.40 | .42 | 1.45 | |
| 90o | 1 | 1.13 | ±.48 | .42 | 1.89 |
| 2 | 1.01 | ±.42 | .59 | 1.84 | |
| 270o | 1 | 1.12 | ±.96 | .31 | 3.08 |
| 2 | .77 | ±.35 | .16 | 1.27 | |
| All subsections | 1 | 1.08 | ±.57 | ||
| 2 | .83 | ±.41 |
Figure 6.
Angular Position – Perpendicular error results are presented for each phantom orientation subsection, measured in millimeters; with Mean, SD, Min and Max reported across 6 tests per tester.
Angular Position – Parallel (ASTM F2554-18 Section 8.3.6)
For the Angular Position-Parallel assessment, the overall accuracy for both testers was less than one millimeter. There were statistically different results between testers at phantom orientations 90o and 180o (Table 5 and Figure 7).
Table 5.
Angular Position – Parallel Error results are Presented for Each Phantom Orientation Subsection, Measured in Millimeters; With Mean, SD, Min and Max Reported Across 6 Tests per Tester.
| Phantom Orientation | Tester | Mean | SD | Min | Max |
|---|---|---|---|---|---|
| 0o | 1 | .72 | ±.21 | .35 | 1.03 |
| 2 | .63 | ±.41 | .30 | 1.41 | |
| 180o | 1 | .43 | ±.21 | .21 | .85 |
| 2 | 1.49 | ±.40 | 1.01 | 2.11 | |
| 90o | 1 | .35 | ±.18 | .17 | .64 |
| 2 | 1.10 | ±.29 | .76 | 1.57 | |
| 270o | 1 | .54 | ±.18 | .28 | .81 |
| 2 | .43 | ±.25 | .12 | .76 | |
| All subsections | 1 | .51 | ±.19 | ||
| 2 | .91 | ±.32 |
Figure 7.
Angular Position – Parallel error results are presented for each phantom orientation subsection measured in millimeters; with Mean, SD, Min and Max reported across 6 tests per tester.
Distance Measurement Between Points (ASTM F2554-18 Section 8.3.6.1)
For the Distance between points measurement, the overall accuracy measurement of tester 1 was slightly less than one millimeter, and the overall accuracy of tester two was slightly greater than one millimeter. There were no statistical differences observed between the two testers for each of the subsections. (Table 6 and Figure 8).
Table 6.
Distance Measurement Between Points Error results are Presented for Each Phantom Orientation Subsection Measured in Millimeters; With Mean, SD, Min and Max Reported Across 6 Tests per Tester.
| Phantom orientation | Tester | Mean | SD | Min | Max |
|---|---|---|---|---|---|
| 0o | 1 | 1.00 | ±.98 | .20 | 3.19 |
| 2 | 1.07 | ±.96 | .01 | 2.89 | |
| 180o | 1 | 1.18 | ±.78 | .22 | 2.54 |
| 2 | 1.17 | ±.65 | .37 | 2.57 | |
| 90o | 1 | .78 | ±.43 | .21 | 1.70 |
| 2 | 1.38 | ±.74 | .15 | 2.25 | |
| 270o | 1 | .91 | ±.74 | .08 | 2.36 |
| 2 | 1.09 | ±.70 | .42 | 2.65 | |
| All subsections | 1 | .97 | ±.73 | ||
| 2 | 1.18 | ±.76 |
Figure 8.
Distance Measurement Between Points error results are presented for each phantom orientation subsection, measured in millimeters; with Mean, SD, Min and Max reported across 6 tests per tester.
Discussion
The promise and potential of HMDs to advance computer-assisted surgical navigation systems has been celebrated since first demonstrated in the 1960s. 36 However, usability and technical limitations such as precise registration and tracking have curtailed utilization. Our collaborative efforts in this study were focused on improving and rigorously testing head-mounted MR technology to enhance the quality, safety, and efficiency of clinical procedures in the near future.
There are several important findings in this study. Notably, the average positional accuracy results achieved with our system not only exceeded FDA regulatory performance standards (<2 mm Euclidian error) using a consensus standard phantom and protocol, but the results are also among the best published for any surgical tracking system including conventional ‘gold-standard’ systems in clinical use. 8 Therefore, these findings demonstrate that the combination of commercially available HMD hardware, and custom software, can provide exceptional positional accuracy, while also offering a more intuitive interface for clinicians. There was also significant reproducibility; the accuracy measurements were very similar for the different tests and between the different independent testers, differing by less than one millimeter.
An important contribution of this work is the rigorous use of comprehensive and standardized testing procedures and measurement equipment, recognized by the FDA. This is particularly important as prior publications have described the use of a diversity of different non-standard custom phantoms and protocols, which were not verified or validated by a recognized consensus body standard, to determine the accuracy and precision of HMD-based surgical systems.8-25 This is the first known study that systematically assesses the accuracy of a HMD optical tracking system using the FDA recognized ASTM-F2554-18 consensus standard. This internationally and FDA recognized standard, was specifically developed and designed to determine the accuracy and repeatability of tracking systems to locate individual points in real world space. Further adoption of this standard will enable more reliable and objective quality and performance assessments as well as more direct comparisons within and between different navigation systems.
A limitation of this study was that it was performed in a controlled lab environment on a metal phantom. Future work is required to determine positional tracking performance and clinical applicability within real-world surgical environments.
Conclusion
The integration of commercially available mixed reality hardware, custom software and instruments, as well as wireless technology resulted in a total system mean positional tracking error of .75 ± SD .37 mm which exceeds FDA standards.
Supplemental Material
Mixed Reality Surgical Navigation System; Positional Accuracy Based on Food and Drug Administration Standard by Christopher T. Morley, David M. Arreola, Long Qian, Amy L. Lynn, Zachary P. Veigulis, and Thomas F. Osborne in Surgical Innovation
Acknowledgements
We would like to thank Osamah Choudhry MD for sourcing the construction of the ASTM F2554-18 consensus standard precision phantom, and Wenbo Lan for the software user interface design.
Authorship Contribution: Authors CTM and TFO conceived the project, TFO wrote the initial draft, TFO and LQ performed the testing, DMA collected the data at VA, CM and LQ collected the data at Medivis, ZPV performed the statistical analysis of the data. All authors made a substantial contribution to the concept and design and interpretation of data as well as contributed to, revised, and approved the manuscript.
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Authors with affiliation at the US Department of Veteran Affairs Palo Alto Health care System have no conflicts of interest to declare. Authors with affiliation at Medivis, Inc. Are employees and/or owners and declare stock ownership in the company.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
Disclaimer: The contents do not represent the views of the US Department of Veterans Affairs or the US government
Supplemental Material: Supplemental material for this article is available online.
ORCID iDs
David M. Arreola https://orcid.org/0000-0002-2865-3797
Thomas F. Osborne https://orcid.org/0000-0002-8896-2487
References
- 1.Vries JK, Sclabassi RJ. An image processing system for use in neurosurgery. Neurosurgery. 1984;31:75-89. [DOI] [PubMed] [Google Scholar]
- 2.Roberts DW, Strohbehn JW, Hatch JF, Murray W, Kettenberger H. A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope. J Neurosurg. 1986;65(4):545-549. [DOI] [PubMed] [Google Scholar]
- 3.Watanabe E, Watanabe T, Manaka S, Mayanagi Y, Takakura K. 3-Dimensional digitizer (Neuronavigator)-New equipment for computed tomography-guided stereotaxic surgery. Surg Neurol. 1987;27(6):543-547. [DOI] [PubMed] [Google Scholar]
- 4.Mezger U, Jendrewski C, Bartels M. Navigation in surgery. Langenbeck's Arch of Surg. 2013;398:501-514. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Maintz JA, Viergever MA, An overview of medical image registration methods. Symposium of the Belgian hospital physicists association (SBPH/BVZF) 1996;12:1-22. [Google Scholar]
- 6.É Léger, Drouin S, Collins DL, Popa T, Kersten‐Oertel M. Quantifying attention shifts in augmented reality image‐guided neurosurgery. Healthc Technol lett. 2017;4(5):188-192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Watkins IV RG, Gupta A, Watkins RG. Cost-effectiveness of image-guided spine surgery. Open J Orthop. 2010;4:228. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Mongen MA, Willems PW. Current accuracy of surface matching compared to adhesive markers in patient-to-image registration. Acta Neurochirurgica. 2019;161:865-867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.van Doormaal TP, van Doormaal JA, Mensink T. Clinical accuracy of holographic navigation using point-based registration on augmented-reality glasses. Oper Neurosurg. 2019;17(6):588-593. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Sun Q, Mai Y, Yang R, Ji T, Jiang X, Chen X. Fast and accurate online calibration of optical see-through head-mounted display for AR-based surgical navigation using Microsoft HoloLens. Int J Comput Assist Radiol Surg. 2020;15:1907-1909. [DOI] [PubMed] [Google Scholar]
- 11.Meulstee JW, Nijsink J, Schreurs R, Verhamme LM, Xi T, Delye HH, Borstlap WA, Maal TJ. Toward holographic-guided surgery. Surg Innov. 2019;26(1):86-94. [DOI] [PubMed] [Google Scholar]
- 12.Maruyama K, Watanabe E, Kin T, Saito K, Kumakiri A, Noguchi A, Nagane M, Shiokawa Y. Smart glasses for neurosurgical navigation by augmented reality. Oper Neuros. 2018;15(5):551-556. [DOI] [PubMed] [Google Scholar]
- 13.Teatini A, Kumar RP, Elle OJ, Wiig O. Mixed reality as a novel tool for diagnostic and surgical navigation in orthopaedics. Int J Comput Assist Radiol Surg. 2021;16:407-414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kuhlemann I, Kleemann M, Jauer P, Schweikard A, Ernst F. Towards X‐ray free endovascular interventions–using HoloLens for on‐line holographic visualisation. Healthc Technol Lett. 2017;4(5):184-187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Scherl C, Stratemeier J, Rotter N, Hesser J, Schönberg SO, Servais JJ, Männle D, Lammert A. Augmented reality with HoloLens® in parotid tumor surgery: a prospective feasibility study. ORL. 2021;83(6):439-481. [DOI] [PubMed] [Google Scholar]
- 16.Tu P, Gao Y, Lungu AJ, Li D, Wang H, Chen X. Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2. Comput Biol Med. 2021;133:104402. [DOI] [PubMed] [Google Scholar]
- 17.Incekara F, Smits M, Dirven C, Vincent A. Clinical feasibility of a wearable mixed-reality device in neurosurgery. World Neurosurg. 2018;118:e422. [DOI] [PubMed] [Google Scholar]
- 18.McJunkin JL, Jiramongkolchai P, Chung W, Southworth M, Durakovic N, Buchman CA, Silva JR. Development of a mixed reality platform for lateral skull base anatomy. Otology and Neurotology: Official Publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology; 2018;39:e1137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Frantz T, Jansen B, Duerinck J, Vandemeulebroucke J. Augmenting Microsoft’s HoloLens with vuforia tracking for neuronavigation. Healthc Technol Lett;5:221-225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Rieder JS, Van Tol DH, Aschenbrenner D. Effective close-range accuracy comparison of Microsoft HoloLens Generation one and two using Vuforia ImageTargets. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops. Lisbon, Portugal. 27 March 2021 - 01 April 2021:552-553. [Google Scholar]
- 21.Qi Z, Li Y, Xu X, Zhang J, Li F, Gan Z, Xiong R, Wang Q, Zhang S, Chen X. Holographic mixed-reality neuronavigation with a head-mounted device: technical feasibility and clinical application. Neurosurg Focus. 2021;51(2):E22. [DOI] [PubMed] [Google Scholar]
- 22.Liebmann F, Roner S, von Atzigen M, Scaramuzza D, Sutter R, Snedeker J, Farshad M, Fürnstahl P. Pedicle screw navigation using surface digitization on the Microsoft HoloLens. Int J Comput Assist Radiol Surg. 2019;14:1157-1165. [DOI] [PubMed] [Google Scholar]
- 23.Kunz C, Maurer P, Kees F, Henrich P, Marzi C, Hlaváč M, Schneider M, Mathis-Ullrich F. Infrared marker tracking with the HoloLens for neurosurgical interventions. Curr Dir Biomed. 2020;6(1). [Google Scholar]
- 24.Gu W, Shah K, Knopf J, Josewski C, Unberath M. A calibration-free workflow for image-based mixed reality navigation of total shoulder arthroplasty. Comput Methods Biomech Biomed Eng Imaging Vis. 2022;10(3):243-251. [Google Scholar]
- 25.Condino S, Carbone M, Piazza R, Ferrari M, Ferrari V. Perceptual limits of optical see-through visors for augmented reality guidance of manual tasks. IEEE Transact on Biomed Eng. 2019;67(2):411-419. [DOI] [PubMed] [Google Scholar]
- 26.Kizer KW, Demakis JG, Feussner JR. Reinventing VA health care: systematizing quality improvement and quality innovation. Medi care. 2000;38:I7-16. [PubMed] [Google Scholar]
- 27.About VHA . Washington, DC: Veterans Health Administration; 2019. https://www.va.gov/health/aboutvha.asp. Accessed May 1, 2020. [Google Scholar]
- 28.Osborne TF, Suarez P, Edwards D, Hernandez-Boussard T, Curtin C. Patient electronic health records score for preoperative risk assessment before total knee arthroplasty. JBJS Open Access. 2020;5(2). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Wang ZJ, Cidade M, Larsen M, Pearman G, Schimpf M, Dhanireddy P, Main results report, prepared for Strategic Analysis Service (SAS) Office of Strategic Planning and Analysis (OSPA) Veterans Health Administration. 2018 survey of veteran enrollees’ health and use of health care. In: Department of Veterans Affairs; Rockville, MD. 2019. Jan 9.The link to the survey is: https://www.va.gov/VHASTRATEGY/SOE2021/2021_Enrollee_Data_Findings_Report-508_Compliant.pdf [Google Scholar]
- 30.Ungureanu D, Bogo F, Galliani S, Sama P, Duan X, Meekhof C, Stühmer J, Cashman TJ, Tekin B, Schönberger JL, Olszta P, Hololens 2 research mode as a tool for computer vision research . arXiv preprint arXiv:2008.11239. 2020 Aug 25.
- 31.Microsoft HoloLens 2 Hardware Specifications. https://docs.microsoft.com/en-us/hololens/hololens2-hardware
- 32.https://www.nist.gov/physics/what-are-imaging-phantoms
- 33.https://www.astm.org/about/overview.html
- 34.ASTM F2554-18 . Standard Practice for Measurement of Positional Accuracy of Computer Assisted Surgical Systems. West Conshohocken, PA: ASTM International; 2018. [Google Scholar]
- 35.https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfStandards/detail.cfm?standard__identification_no=39079
- 36.Sutherland IE. A head-mounted three dimensional display. Proceedings of the December. 1968;9:757-764. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Mixed Reality Surgical Navigation System; Positional Accuracy Based on Food and Drug Administration Standard by Christopher T. Morley, David M. Arreola, Long Qian, Amy L. Lynn, Zachary P. Veigulis, and Thomas F. Osborne in Surgical Innovation