Can an Augmented Reality Headset Improve Accuracy of Acetabular Cup Orientation in Simulated THA? A Randomized Trial

Abstract

Background

Accurate implant orientation reduces wear and increases stability in arthroplasty but is a technically demanding skill. Augmented reality (AR) headsets overlay digital information on top of the real world. We have developed an enhanced AR headset capable of tracking bony anatomy in relation to an implant, but it has not yet been assessed for its suitability as a training tool for implant orientation.

Questions/purposes

(1) In the setting of simulated THA performed by novices, does an AR headset improve the accuracy of acetabular component positioning compared with hands-on training by an expert surgeon? (2) What are trainees’ perceptions of the AR headset in terms of realism of the task, acceptability of the technology, and its potential role for surgical training?

Methods

Twenty-four study participants (medical students in their final year of school, who were applying to surgery residency programs, and who had no prior arthroplasty experience) participated in a randomized simulation trial using an AR headset and a simulated THA. Participants were randomized to two groups completing four once-weekly sessions of baseline assessment, training, and reassessment. One group trained using AR (with live holographic orientation feedback) and the other received one-on-one training from a hip arthroplasty surgeon. Demographics and baseline performance in orienting an acetabular implant to six patient-specific values on the phantom pelvis were collected before training and were comparable. The orientation error in degrees between the planned and achieved orientations was measured and was not different between groups with the numbers available (surgeon group mean error ± SD 16° ± 7° versus AR 14° ± 7°; p = 0.22). Participants trained by AR also completed a validated posttraining questionnaire evaluating their experiences.

Results

During the four training sessions, participants using AR-guidance had smaller mean (± SD) errors in orientation than those receiving guidance from the surgeon: 1° ± 1° versus AR 6° ± 4°, p < 0.001. In the fourth session’s assessment, participants in both groups had improved (surgeon group mean improvement 6°, 95% CI, 4–8°; p < 0.001 versus AR group 9°, 95% CI 7–10°; p < 0.001). There was no difference between participants in the surgeon-trained and AR-trained group: mean difference 1.2°, 95% CI, -1.8 to 4.2°; p = 0.281. In posttraining evaluation, 11 of 12 participants would use the AR platform as a training tool for developing visuospatial skills and 10 of 12 for procedure-specific rehearsals. Most participants (11 of 12) stated that a combination of an expert trainer for learning and AR for unsupervised training would be preferred.

Conclusions

A novel head-mounted AR platform tracked an implant in relation to bony anatomy to a clinically relevant level of accuracy during simulated THA. Learners were equally accurate, whether trained by AR or a surgeon. The platform enabled the use of real instruments and gave live feedback; AR was thus considered a feasible and valuable training tool as an adjunct to expert guidance in the operating room. Although there were no differences in accuracy between the groups trained using AR and those trained by an expert surgeon, we believe the tool may be useful in education because it demonstrates that some motor skills for arthroplasty may be learned in an unsupervised setting. Future studies will evaluate AR-training for arthroplasty skills other than cup orientation and its transfer validity to real surgery.

Level of Evidence

Level I, therapeutic study.

Introduction

Accurate positioning of orthopaedic devices reduces wear and increases stability. Incorrect implant orientation has long been considered an independent risk factor for dislocation and premature revision [7, 12, 24-26, 38]. In the hip, patient-, pathology-, or implant-specific targets may reduce impingement, edge-loading, and dislocation [11, 27, 34]. There is a well-established learning curve associated with arthroplasty [9, 19]. Low-volume or junior surgeons [33, 41] are more likely to malposition implants outside of safe zones [23, 29], although narrow targets are unlikely to be achievable for most using freehand techniques [11, 43]. Restricted working hours and an increased emphasis on patient safety have reduced training opportunities in the traditional apprenticeship-based model and prompted the growth of simulation-based proficiency-progression training [5]. In Dale’s “Cone of Experience,” experiential learning—with learners undertaking purposeful, hands-on experience of tasks that represent reality—results in high knowledge retention and skill development [10]. This has been borne out with virtual reality arthroscopic simulators [3, 20, 35, 47] and their integration into curricula [4].

For arthroplasty, simulation training has conventionally been delivered on dry bone or cadavers. The former is of low fidelity, and the latter is costly and requires infrastructure. Neither can provide an objective measure of technical or three-dimensional orientation skill [44]. Unlike arthroscopy, there are no validated simulators for tracking instruments in relation to anatomy, reproducing haptics, or training for psychomotor skills in arthroplasty.

Augmented reality (AR) headsets use transparent screens and reflective lenses to enable digital information to be overlaid on the real world in the wearer’s field of vision. They “augment” the user’s vision while maintaining the haptics of the real world. The HoloLens AR headset (Microsoft, Redmond, WA, USA) is one such system and can track hand movements in relation to its built-in cameras. Using the HoloLens, we have developed an enhanced AR headset capable of tracking an implant’s position and orientation in relation to the pelvis. However, this headset has not been assessed for its suitability as a training tool.

We, therefore, asked: (1) In the setting of simulated THA performed by novices, does an AR headset improve the accuracy of acetabular component positioning compared with hands-on training by an expert surgeon? (2) What are trainees’ perceptions of the AR headset in terms of realism of the task, acceptability of the technology, and its potential role for surgical training?

Materials and Methods

Development of an Enhanced AR Headset

A MicronTracker (ClaroNav, Toronto, Canada) camera was integrated with the HoloLens AR headset with bespoke software. The source code is not protected and will be available on reasonable request. It was developed in C++, and relates specifically to the MicronTracker and HoloLens hardware, and the spatial organisation between the implant and the three-dimensional acetabular morphology. This platform was validated against a gold-standard–a motion-capture laboratory with 18 Prime 13W cameras (OptiTrack, Corvallis, OR, USA). The AR headset measured and tracked an acetabular component in relation to a phantom hip model to within 1 mm of translation, 0.2° of inclination and 0.9° of anteversion (Appendix; Supplemental Digital Content 1, http://links.lww.com/CORR/A106). The difference between the target and achieved orientation, in operative anteversion and inclination as defined by Murray et al. [37], was measured as orientation error° by applying a Pythagorean calculation:

Clinical Validation of the Enhanced AR Headset

Setting and Participants

This randomized simulation trial using an AR headset and a simulated THA was undertaken in an orthopaedic-skills room within a university teaching hospital with institutional ethical approval (MEEC 1617-10) and the trial was registered on ResearchRegistry.com (researchregistry3908). Twenty-four study participants (medical students in their final year of school, who were applying to surgery residency programs and with no prior arthroplasty experience) were recruited to participate (mean age 24 ± SD 2, 19 males and five females, 22 right-hand dominant, and all with no experience of augmented reality headsets or performing arthroplasty) (Table 1). They provided voluntary informed consent to complete four once-weekly training and assessment sessions to learn cup orientation for THA. Participants were excluded if they had previously assisted in any THA or had any prior experience with an AR headset. They were randomly assigned to an AR simulator-trained group or a group trained by an expert surgeon (KL), a fellowship-trained experienced arthroplasty surgeon who consistently achieved orientation errors of < 5° during validation (Appendix; Supplemental Digital Content 1, http://links.lww.com/CORR/A106). A block randomization scheme (two blocks of 12) was created using SPSS 24 (IBM Statistics, Chicago, USA), to ensure balanced allocation of participants to the two groups. Results were recorded in individual, closed envelopes so that group assignment was revealed at time of each subject’s enrollment. Randomization was independently performed 2 weeks before the first training session by a research manager (HJ) not involved in testing. Enrollment was performed by KL, and data analysis (by KL and EA) was performed with the assessors blinded to study group allocation. There was no crossover and no dropouts from either group (Fig. 1).

Table 1.

Participant demographics

Fig. 1.

This CONSORT Flow Diagram denotes the study recruitment, allocation and analysis.

Before training, participants in both groups were equally inaccurate in achieving the target orientation: surgeon group mean error ± SD 16° ± 7° compared with AR error 14° ± 7°, mean difference 1.4° (95% CI -0.2 to 3.1°), p = 0.223.

The groups had comparable demographics and exposure to surgery. All 24 novice surgeons completed four training and assessment sessions in 4 consecutive weeks. The orientation data from three participants during the first training section of session were corrupt and could not be included.

Training and Assessment Schedule

All participants individually attended a standardized educational program, which included technical instruction and four clinical vignettes pertaining to component orientation in THA. Subsequently, each weekly session was split into three components: a pretraining assessment, a training period, and a posttraining assessment (Fig. 2). The pretraining assessment consisted of each participant performing one attempt at positioning a hemispheric acetabular cup in six clinically relevant cup orientations [14, 29, 32, 34] in a randomized sequence on a phantom hip model prepared in the lateral position (Sawbones®, Vashon Island, WA, USA) (Table 2).

Fig. 2.

This flowchart shows how study participants were randomized and then completed a four-week training and assessment schedule.

Table 2.

Participants placed the cup in these orientation targets, which reflect clinically relevant acetabular component inclination and anteversion target angles

The training period followed immediately after. Participants were guided to position the cup to a randomly selected three of the six previously described orientations. Each orientation was trained for twice. Participants allocated to the AR group wore the enhanced HoloLens AR headset with the cup orientation projected as a live hologram onto a target board within their field of vision (Fig. 3). This provided a visual representation of anteversion and inclination of the implant in relation to the anterior pelvic plane as a red dot moving on a crosshair. This dot turned green once centered on the crosshair to indicate a difference of < 1° between the target and achieved orientations. There was no additional guidance or feedback. Participants allocated to the surgeon group were guided by an expert hip surgeon standing on the assistant’s side of the phantom hip who provided tactile and verbal feedback, simulating intraoperative training. The precision of the training delivered by the AR headset and the ES was measured using a head-mounted MicronTracker camera.

Fig. 3.

The heads-up display view within the AR headset showed a crosshair projected over the platform (shown here without soft tissue covering) for cup orientation.

The posttraining assessment immediately followed the training period. Each participant was assessed performing each of the six cup orientations in a randomized sequence with the orientation error° measured using a head-mounted MicronTracker camera. The subsequent session took place after a 5- to 7-day interval.

In the final session, all participants undertook one more, surprise assessment. The phantom hip was tilted 15° in the sagittal plane away from the operative side using a concealed wedge. Each participant was assessed performing three randomly selected target orientations and the orientation error° was measured. There was no set time limit for the participant to achieve each orientation during training or assessment sections in any of the sessions during the study and no additional guidance or feedback outside of the training periods.

Questionnaire

On completion of the final assessment, participants in the AR group were invited to complete a web-based survey without observation (Appendix; Supplemental Digital Content 2, http://links.lww.com/CORR/A107). It included statements related to the face validity and acceptability of the hip platform and AR headset, and the application of AR technology for surgical training in general, alongside 10-point Likert scales.

Statistical Analysis

The final orientation of the acetabular cup was recorded in inclination and anteversion in relation to the phantom pelvis’ anterior pelvic plane. The orientation error° was calculated as the difference between the target and achieved orientations with Equation 1. Data were assessed for normality using the Shapiro-Wilk test. The Wilcoxon signed-rank test compared error between the AR- and expert surgeon-trained participants after the last session and two-tailed paired t-tests were performed to compare demographic and orientation error results within the same training group. A p value of < 0.05 was considered statistically significant and accuracy in degrees reported in whole numbers. G*Power 3.1.9 (Heinrich Heine, Dusseldorf, Germany) was used for power calculation and SPSS 24 was used for data analysis.

Our primary outcome measure was the orientation error° between the target and achieved orientation, calculated from the difference between intended and achieved operative inclination and anteversion values, as an average of the errors from the six orientation tasks performed during each session. Using previous data on the differences in orientation accuracy between expert and training surgeons [15], we required a minimum of nine participants in each arm to detect a 5° difference, with 80% power and α 0.05. We chose to recruit 12 participants in each arm to account for dropout over the four sessions or technical difficulties related to this novel device. Feedback from the poststudy survey was weighted 1 to 10 with scores of ≥ 7 deemed positive and ≤ 4 deemed negative, and median scores were calculated for individual statements.

Results

Accuracy

When learning to orient the cup across all four sessions, participants guided by the expert surgeon achieved the target orientation with error of 6° ± 4°. Those trained using AR were more accurate as they confirmed the final orientation when the headset light turned from red to green, achieving an error of 1° ± 1° (mean difference 4.7° (95% CI, 3.9–5.5°); p < 0.001) (Fig. 4).

Fig. 4.

Graph showing average orientation error achieved by the study participants during training when guided by expert surgeon or AR headsets.

We compared the accuracy of orienting the cup before training in the first session to the final session’s assessment. Both groups reduced their error: The surgeon group final error 8° ± 5°, mean difference from pretraining was -7.8° (CI 95%, -5.5 to -10.2°; p < 0.001) (Fig. 5) and the AR group final orientation error was 9° ± 6°; the mean difference from pretraining was -8.4° (CI 95%, -7.0 to -9.8°; p < 0.001) (Fig. 6). At the final assessment, there was no difference between participants in the expert-trained and AR-trained group: mean difference 1.2° (CI 95%, -1.8 to 4.2°; p = 0.281).

Fig. 5.

Graph showing average orientation error achieved by the study participants during pretraining and assessment in consecutive sessions trained by an expert surgeon.

Fig. 6.

Graph showing average orientation error achieved by the study participants during pretraining and assessment in consecutive sessions trained with AR headsets.

Three participants of 12 in the surgeon group and 2 of 12 in the AR group achieved “expert” levels of orientation error (< 6°). There were no differences in accuracy between the expert surgeon-trained group and the AR group on the test using concealed pelvic tilt (surgeon group error 6° ± 4° versus AR group 7° ± 5°, mean difference 0.8° (95% C.I -2.5 to 4.2°); p = 0.301).

Questionnaire About Trainees’ Perceptions of AR

The feedback from the online survey relating to how the training aided the participants’ understanding of the relevance of orientation, anatomic landmarks, and implant position in THA was predominantly positive (Table 3). The dry-bone platform only moderately represented real life (mean 6 out of 10), but the equipment used, and the orientation task were rated highly (mean 8). Participants felt that preoperative rehearsal on this platform would be valuable (mean 7). Participants reported that they developed their visuospatial and motor skills (mean 8) and found live feedback a useful tool to achieve implant orientation (mean 8). AR technology for surgical training was perceived as potentially useful for simulation (mean 8) to develop intraoperative decision-making (mean 8) and visualizing three-dimensional concepts (mean 8) and as an assistive tool during surgery (mean 8). No participants saw AR as the ideal learning and training method by itself, two participants preferred to be taught by an expert alone, and 10 of 12 participants stated that a combination of surgical tutor for learning and AR technology for training would be their preferred method of developing their surgical skills (Table 4).

Table 3.

Questionnaire results from AR group participants on their training experience , scored from 0 (completely disagree) to 10 (completely agree)

Table 4.

Questionnaire results from AR group participants on their perspectives on AR as a technology to enhance learning, training, and surgical outcomes

Discussion

Accurate implant orientation in THA is a subtle technical skill that can be improved by clinical experience [6] or assistance from intraoperative navigation [17]. It would be valuable for a training surgeon to develop this skill outside of the operating room. We developed an augmented reality platform that can accurately track an acetabular cup implant in relation to a pelvis and display real-time feedback of orientation in a learner’s field of view. To our knowledge, there has not been a formal evaluation of such a tool. We found no difference in error between participants trained to orient the cup implant by AR or by an expert surgeon after a structured training and assessment program. Our questionnaire found that learners found the AR headset to be usable and feasible. They saw AR as a tool for unsupervised training, to supplement learning with an expert surgeon inside the operating room.

This study’s findings should be viewed in context of several limitations. First, with just 12 participants in each group, it is possible that the study is underpowered and there are differences as large as 5° of error between the groups. This level of accuracy may be sufficient, as in clinical practice it is suggested that surgeons should achieve orientations within 5° of a patient-specific target to reduce complications [16, 43]. Future studies of this tool with more participants are needed to address this possibility of type 2 error.

Second, we used a phantom platform of low fidelity compared with cadavers or real surgery. This included soft tissue structures around the hip, including skin and musculature divided through the posterior surgical approach. This exceeded the fidelity of platforms used on dry-bone arthroplasty workshops and several validated arthroscopic orthopaedic simulators [21, 22, 36, 42]. Participants reported enough realism to train for and perform this orientation task, as it is largely related to bony anatomy and not the soft tissue dissection. We continue to develop this platform beyond just implant orientation, and future studies will use cadavers for testing other skills.

In addition, this study did not assess skill retention at another time point after the final session and did not assess if participants transferred their skills to real surgery. Furthermore, like several other orthopaedic simulator studies [2, 28, 35, 42], we recruited participants in this study who had no surgical experience with hip arthroplasty. While this ensured that our results ere not confounded by prior surgical experience or patient variability, this limits the external generalizability of the findings to orthopaedic residency, and we cannot comment on the construct or transfer validity of the simulation training. Alvand et al. [1, 2] showed in arthroscopic-simulated studies that some individuals may be more innately capable of specific technical skills than others. In our study, three participants in the expert-trained group and two in the AR group were proficient by the final session, whereas one of these had achieved proficiency after just three sessions. One participant (in the AR group) achieved errors of > 10° throughout and > 20° at final assessment, although we cannot determine whether further training may have improved their performance. Future studies will recruit surgeons of varying experience and will assess skill retention and transfer validity.

Current orthopaedic simulators cost approximately USD 100,000 [30], and most have yet to be validated to demonstrate acquisition of skills by learners. The phantom hip and AR platform developed here cost approximately USD 10,000. Although this figure does not include institutional costs for software development, the platform is less expensive than currently available arthroscopic and open surgical platforms [39, 46]. Future studies will need to compare its effectiveness to these and conventional training approaches.

In general, we found no differences in the accuracy of implant placement between participants trained to insert an acetabular component during a simulated THA using AR and those given hands-on instruction from an expert surgeon. Virtual and augmented reality simulators have been validated to develop technical expertise for open surgical procedures, including accurately orientating knee arthroplasty implants [48], drilling bone [45], and applying a femoral plate for fracture fixation [8]. As in our study, these have demonstrated that similar unsupervised training improves performance in novices. It remains to be seen if these approaches are at least equivalent to supervised training for multiple steps of a given procedure, and whether they translate to improvements in the operating room, as has been demonstrated in arthroscopic simulators [20, 47].

The feedback from the online questionnaire suggested that although the hip platform itself was of moderate fidelity, the use of real instruments, live feedback, and clinical vignettes resulted in an enjoyable and valuable experience for both groups of participants. Participants envision the use of this platform as an adjunct to receiving teaching from an expert surgeon and not a replacement. This preference for blended implementation of AR training is common with other studies of virtual reality simulators [13, 18, 31] and curricula which integrate simulators into orthopaedic residency [40].

In this small, randomized, controlled simulation trial comparing training from augmented reality platform with an expert surgeon, novices learned to orient an acetabular cup to the same level of accuracy. We discerned that despite the moderate fidelity of AR, participants valued the use of real surgical instruments combined with real-time digital feedback afforded by this technology, which is a combination of qualities not available from fully-immersive virtual reality headsets. Future directions will involve improving this tool to include training for acetabular reaming and femoral broaching, and for bone cuts and implant orientation during knee arthroplasty. Future studies with larger groups should also measure if these skills can be transferred to the operating room. Our goal is to incorporate such motor-skills training into residency curricula, so that learners can improve surgical skills outside the operating room.