Cochlear Implant Surgery: Virtual Reality Simulation Training and Transfer of Skills to Cadaver Dissection—A Randomized, Controlled Trial

Abstract

Background

: Cochlear implantation requires excellent surgical skills; virtual reality simulation training is an effective method for acquiring basic competency in temporal bone surgery before progression to cadaver dissection. However, cochlear implantation virtual reality simulation training remains largely unexplored and only one simulator currently supports the training of the cochlear implantation electrode insertion. Here, we aim to evaluate the effect of cochlear implantation virtual reality simulation training on subsequent cadaver dissection performance and self-directedness.

Methods:

This was a randomized, controlled trial. Eighteen otolaryngology residents were randomized to either mastoidectomy including cochlear implantation virtual reality simulation training (intervention) or mastoidectomy virtual reality simulation training alone (controls) before cadaver cochlear implantation surgery. Surgical performance was evaluated by two blinded expert raters using a validated, structured assessment tool. The need for supervision (reflecting self-directedness) was assessed via post-dissection questionnaires.

Results:

The intervention group achieved a mean score of 22.9 points of a maximum of 44 points, which was 5.4% higher than the control group’s 21.8 points (P = .51). On average, the intervention group required assistance 1.3 times during cadaver drilling; this was 41% more frequent in the control group who received assistance 1.9 times (P = .21).

CONCLUSION:

Cochlear implantation virtual reality simulation training is feasible in the context of a cadaver dissection course. The addition of cochlear implantation virtual reality training to basic mastoidectomy virtual reality simulation training did not lead to a significant improvement of performance or self-directedness in this study. Our findings suggest that learning an advanced temporal bone procedure such as cochlear implantation surgery requires much more training than learning mastoidectomy.Keywords: Cochlear implant, simulation, medical patient simulation, clinical competence, humans, assessment

Introduction

Cochlear implant (CI) surgery requires fine-tuned motor skills and excellent three-dimensional anatomical understanding.¹ Repeated and deliberate practice is needed to acquire these skills and to ensure competent sparing of delicate structures, such as the facial nerve and intracochlear tissue,² while achieving sufficient exposure of the round window and careful, nontraumatic electrode insertion. Cadaver dissection remains the gold standard in temporal bone training but limited availability of cadaver specimens and high operating costs of dissection labs have necessitated new training methods.³ Virtual reality (VR) is one such method and has been demonstrated to be an effective educational tool for learning mastoidectomy.⁴ In mastoidectomy, skills acquired during VR simulation training persist during subsequent cadaver dissection—so-called transfer of skills.^5,6 Assessing transfer is essential in simulation training because learning skills in a simulated environment that do not increase proficiency in the patient setting would deem the simulation training irrelevant. Nevertheless, the transfer of skills remains largely unexplored for more advanced temporal bone procedures including cochlear implant surgery; here, a single study assessed drilling of the posterior tympanotomy preceding the insertion but did not feature a CI electrode.⁷ Correspondingly, no studies have evaluated the transfer of skills from VR simulation of CI surgery to performance in cadaveric dissection training—mainly because, until recently, no VR simulation platform allowed full simulation of cochlear implant surgery. The question remains whether the favorable findings from mastoidectomy VR simulation training can be directly extrapolated to more advanced temporal bone procedures such as CI surgery. Answering this question would be highly relevant because knowing the effect of interventions is essential when planning evidence-based surgical training curricula that can advance “see one, do one, teach one” to “see one, learn the procedure with simulation training, do one.”⁸

In this study, we aim to evaluate the effect of cochlear implantation virtual reality simulation training on transfer to subsequent cadaver surgery. We hypothesize that VR simulation training in CI surgery improves novices’ early skills acquisition and cadaver dissection performance.

Methods

Study Design

This was a randomized, controlled trial of an educational intervention (simulation-based training).

Setting and Participants

The setting was a 4-day cadaver dissection course at the University of Copenhagen held January 21–24, 2019. The course consisted of lectures, VR mastoidectomy simulation training, and cadaver surgery in a dissection lab. Data were collected at Copenhagen Academy for Medical Education and Simulation (CAMES) (VR simulation training) and the dissection lab at the University of Copenhagen (cadaver dissection). Participants were 18 otolaryngology residents from 8 ORL training departments in Denmark; they were naïve to independent temporal bone surgery including CI surgery, as the temporal bone course is considered a prerequisite for supervised surgery. The only exclusion criterion was previous VR simulation training in CI surgery.

All participants attended lectures on temporal bone surgery including CI and 3 hours of VR mastoidectomy simulation training. The standard VR mastoidectomy simulation training comprised drilling of the basic mastoidectomy until the point of final thinning of the posterior wall of the ear canal. Specifically, this VR training of mastoidectomy did not include the posterior tympanotomy nor did it entail attainment of proficiency.

Participants were then randomized to either an intervention or control group. We used a random sequence generator for randomized allocation⁹; group allocation was revealed by drawing envelopes. The first author (MF) generated the random allocation sequence, enrolled participants, and handled the envelopes. Participants were randomized in a 1 : 1 ratio to either the intervention group who received an additional 2 hours of VR simulation training including CI, or the control group who received no CI VR training. Blinding of participants was not possible. A trial flowchart is presented in Figure 1.

Figure 1.

Flowchart describing the enrolment, randomization, training, and testing of the 18 otorhinolaryngology residents included in the study. (A) All 18 participants were assessed for eligibility, randomized, underwent assigned training, tested, and included in the final analysis.

Intervention

Participants randomized to the intervention were first introduced to the CI-procedure and watched a video demonstrating a good performance in the simulator. They were then given 5 minutes hands-on warm up to familiarize themselves with the simulator and controls pertaining to the CI procedure. The training intervention consisted of 2 hours of CI VR simulation training based on the concept of directed, self-regulated learning where the trainee practices independently without instructor intervention.^10,11 The CI procedure started from the point of drilling down the cells of the posterior canal wall, followed by a posterior tympanotomy, removal of the round window bony overhang, and finally CI insertion through the round window. The simulation platform was the Visible Ear Simulator (VES) version 3.0, a free software package for VR temporal bone surgery, which includes a CI module¹² (Figure 2).

Figure 2. a-c.

Simulator and cadaver dissection. (a) Simulator setup, (b) virtual reality electrode insertion, and (c) cadaver electrode insertion.

Transfer Test in Cadaveric Surgery

After VR simulation training, both groups performed the CI procedure on human cadavers. As an introduction, participants received a standardized walk-through of the anatomy and the surgical steps pertaining to CI surgery using anatomical drawings, which were available throughout the dissection as reference. The cadavers were randomly assigned to a participant from the intervention group who drilled one side and a participant from the control group drilled the other side (to control for varying anatomical difficulty in the cadavers). Participants performed the procedure on the same side as they performed simulation training. During the drilling, participants of both groups could request guidance from the same experienced CI surgeons. After completing the procedure, participants filled out a questionnaire specifying the amount of help or guidance in relation to specific steps (e.g., “Widening of the posterior tympanotomy”). For the actual CI electrode insertion, participants could not request assistance.

Outcomes

The primary outcome was a surgical performance during cadaver surgery using the Cochlear Implant Surgery Assessment Tool (CISAT).¹³ The CISAT is specifically designed for compatibility with both VR simulation and cadaver CI surgery and is supported by validity evidence. It comprises 11 items that are rated on a 1–5 point Likert scale with descriptive anchors: items 1–6 concern drilling technique and result; items 7–11 vector, speed, and result of the electrode insertion (supplementary digital material). To demonstrate the true obtained performance score, we deducted the baseline score of one point for each item of the CISAT in the analysis and presentation of results for a range of 0–44 points.

The secondary outcome was self-directedness during the procedure, that is, ability to perform the surgery without instructor guidance/feedback.

Finally, we evaluated the intervention group’s performance during the CI VR simulation training.

Performances were assessed by two blinded expert raters (authors MS and SA) based on videos of the CI cadaver surgery. For the CI VR simulation procedures completed by the intervention group, a simulator file showing the final drilling result was used in tandem with videos. Raters were blinded to all participant data, including group allocation and—for the VR procedures—procedure number.

Sample Size and Statistical Methods

Enrollment was based on a convenience sample of eligible participants at a cadaver dissection course. Due to repeated measures (multiple raters and multiple performances), a linear mixed models (LMM) analysis of the performance data was done with rater and group as fixed factors.¹⁴ Independent samples t- or Mann–Whitney U-test (depending on whether data were normally distributed), and Fisher’s exact test were used to compare groups’ variables when there were no repeated measurements. Statistical reporting was based on the Statistical Analyses and Methods in the Published Literature (SAMPL) reporting guidelines;¹⁵ the overall reporting was based on the health-care simulation extension to the Consolidated Standards of Reporting Trials (CONSORT) guidelines.¹⁶

Ethics

This trial was conducted in accordance with the Helsinki Declaration. After thorough oral and written information on the study, written, informed consent was obtained from all participants. Participation was voluntary and no compensation was given for trial participation. This educational study’s protocol was submitted to the Capital Region of Denmark Ethics Committee and deemed exempt from needing ethics committee approval.

Results

All 18 participants were assessed for eligibility, randomized, underwent the assigned training, and were included in the data analysis (Figure 1). There were no significant baseline differences between the intervention and control group participants (Table 1).

Table 1.

Participant Background Data (n = 18)

	Intervention (n = 9)	Controls (n = 9)	P
Age, (years, mean (SD))	33.2 (3.2)	34.6 (2.7)	.36
Sex, n (%)
Female	33% (3)	44% (4)	.63
Male	66% (6)	56% (5)
ORLa experience (years; median)	3.5	3.5	.73
(min–max)	(2–7)	(2.3–5)
Other surgical experience (years; median)	.5	1	.39
(min-max)	(0–1)	(0–5.5)
aORL, otolaryngology.

Primary Outcome (Dissection Performance)

The mean cadaver dissection performance score in the intervention group was 22.9 CISAT points out of a maximum of 44 points [95% confidence interval (20.5–25.4)] vs. 21.8 points in the control group [95% confidence interval (19.3–24.2)]. The intervention group thereby outperformed the control group by 5.4%; however, this difference was not statistically significant (P = .51; LMM).

Evaluating the drilling components separately, the intervention group scored a mean of 12.2 points [95% confidence interval (10.1–14.3)] while the control group scored 11.5 points [95% confidence interval (9.4–13.6)]; P = .63; LMM) of the maximum drilling-related 24 points. Scores for insertion where 10.7 points in the intervention group [95% confidence interval (9.4–12.0)] vs. 10.3 points in the control group [95% confidence interval (9.0–11.6); P =.63; LMM)] of a maximum insertion-related score of 20 points.

Secondary Outcomes (Self-Directedness and VR Performance)

The intervention group requested guidance during the cadaver drilling 1.3 times on average, whereas the control group requested guidance 1.9 times (P =.22; Mann–Whitney U-test), that is, 42% more often. Nevertheless, all but 1 (11%) in the intervention group and 2 (22%) participants in the control group got help at least once during the drilling.

The intervention group completed a median of 6.5 (range 2–8) simulated CI VR procedures and attained a mean performance score during CI VR simulation training of 30.3 points out of a maximum of 44 points [95% confidence interval (29.1–31.4)]. Performance gradually improved during the VR CI simulation training: mean scores improved by 22.5% from 25.3 points [95% confidence intervals (23.1–27.6)] at procedure one to 30.4 [95% confidence interval (27.7–33.2)] points at procedure three (P = .065, LMM).

Discussion

In this randomized, controlled study on the effect of VR simulation training on CI surgery, we found that the addition of CI VR simulation training led to a slight performance increase during subsequent cadaver CI surgery. Second, we found that CI VR simulation training led to a higher degree of self-directedness, that is, less need for instruction or help during cadaver surgery. None of these results, however, reached statistical significance.

The minor performance difference between the intervention and control groups during cadaver surgery contrasts with studies on mastoidectomy, where VR simulation training was highly beneficial for subsequent cadaver surgery.¹⁷ The analysis of the individual parts of the procedure (drilling and insertion) echoed the overall finding of equal performance in the two groups. We expected an added benefit of simulating the handling and insertion of a CI electrode compared to CI-related VR training with no electrode option.⁷ However, no such benefit could be documented even when electrode-related items were isolated and analyzed separately.

Potential reasons include the limited amount of training (2 hours), which was based on feasibility in relation to a cadaver dissection course, rather than achievement of competency in simulated CI surgery. Training revolving on attaining a set level of proficiency—that is, mastery learning—likely increases the rate of surgical skills acquisition. In a recent validity study on the CISAT used,¹³ we compared novices and CI surgeons to establish a suggested pass-fail level for progression from VR simulation training, for example, to cadaver dissection. A pass-fail score of 34.5 CISAT points was determined. In contrast, participants in the present study only reached a score of 30.4 points. Accordingly, they did not reach the recommended level of competency, suggesting that a mastery learning approach would have been useful. In addition, an evaluation of the learning curves (i.e., the progression of skills acquisition) during CI VR simulation training showed that while initial training yields a substantial improvement per procedure, further training leads to progressively less learning per procedure.¹⁸ This is consistent with the general understanding of surgical skills acquisition.¹⁹ Correspondingly, acquiring the remaining 4 points to reach the 34.5 point pass-fail score in this study requires more training than the 2 hours given. Altogether, this suggests that the training volume should be much higher to produce substantial results during cadaver dissection. This finding is of key relevance to the training of temporal bone surgery: results from studies on mastoidectomy training cannot be directly extrapolated to more advanced temporal bone procedures (such as CI surgery) that require more sophisticated and diverse technical skills. When designing advanced temporal bone surgery curricula, the training and its effects on skills should be carefully evaluated for the individual procedures trained because knowledge of mastoidectomy training might not apply to these advanced procedures. Furthermore, advanced otologic learning interventions such as training programs, courses, and demonstrations should take into account the complex nature and likely longer learning curve of these procedures by providing ample time for training to ensure that trainees master the procedures.

Next, our study features a limited sample size. Here, normal variations in learners’ surgical skills acquisition can have a relatively large impact: for example, if a few “fast learners”²⁰ are randomly allocated to the control group, this might have an effect comparable to the intervention, resulting in a type II error. Surgical skills acquisition is highly individual and some trainees require numerous procedures to achieve the level that others attain very quickly.

The concept of “apprenticeship learning,” entailing ongoing supervision, has been the backbone of learning the surgical craft. However, with reduced work hours,²¹ increasing number of trainees,²² and growing surgical complexity, this traditional paradigm is challenged. The advance in simulation training and research supports the notion that simulation combined with a more independent and individualized approach to training can alleviate some of these challenges and provide high-quality training in the modern era. Directed, self-regulated learning, in which the trainee practices independently with the aid of learning supports to provide instructions, feedback, and guidance, has been demonstrated to be beneficial over instructor-led training.¹⁰ Our VR simulation was designed to support directed, self-regulated learning and we indeed found a trend toward participants requiring less guidance and instruction during the CI cadaver surgery. This could imply that they were better acquainted with the procedural steps or more confident performing the drilling independently.

This study is limited by the fact that we enrolled residents who were largely novices in CI surgery. In the clinical setting, trainees undertaking CI surgery are normally proficient at mastoidectomy, and this should be kept in mind when considering the external validity of our findings.

A strength of our study is the use of rigorous performance assessment in accordance with contemporary educational research methods: the assessment tool used is supported by substantial validity evidence according to Messick’s framework²³ and 2 blinded raters evaluated all performances. Further, the randomized, controlled design allowed us to account for many potential sources of confounding. Finally, we evaluated performance as a transfer of skills from VR simulation to cadaver surgery, with the latter being considered the gold standard for temporal bone training before supervised, real-life patient surgery.³ As such, performance during—and findings pertaining to—cadaver surgery are likely the simulation type most representative of performance during live patient surgery.²⁴

VR simulation provides an evidence-based approach to alleviating the paucity of training opportunity facing surgical trainees,³ as it allows for an unlimited number of training procedures, inside or outside the hospital setting, before progressing to cadavers or real-life surgery.^17,25 Although the mastoidectomy procedure has largely been the only option for VR simulation training of temporal bone surgery, other procedures are likely also worthwhile training: CI, cholesteatoma, vestibular schwannoma, and so on. However, developing cutting-edge simulation-based training is an iterative process, and the present study suggests there is room for refinement of the simulator used because the possible training effect was small. Consequently, the VES’ CI module functions and haptics are being improved to allow for an easier and more intuitive electrode and forceps.

Although some argue that evaluating whether simulation training works is obsolete because it usually does,²⁶ we believe that examining the effect of new training types on skills acquisition is imperative. Evaluating “face validity” or other outdated outcome measures such as confidence or satisfaction is no longer considered sufficient for exploring the effect and utility of training.²⁷ Therefore, we hope that future studies on temporal bone simulation training can leverage modern skills assessment and research methods to improve the training of temporal bone surgery. In addition to VR simulation training of other temporal bone procedures, this could be in the form of 3D-printed temporal bone training,²⁸ studies on implementation,²⁹ cost-effectiveness, and stakeholder engagement for simulation-based training.

Conclusion

VR simulation of CI surgery is a feasible training option and can be implemented before dissection training. Nevertheless, in this study, the effect of 2 hours of VR simulation of CI surgery only led to a modest and non-significant increase in performance during dissection, suggesting that learning CI surgery differs substantially from learning mastoidectomy. The simulation training further seemed to reduce the need for guidance during the subsequent dissection. Overall, this study only provides preliminary support for VR simulation training in relation to novices’ early acquisition of CI surgical skills. Technical refinements of the simulator and a larger number of participants in future studies will better clarify the value of CI VR simulation training.

Registration

This educational study was not submitted to a trial registry.