Evaluating the effectiveness of the Voxel-Man TempoSurg virtual reality simulator in facilitating learning mastoid surgery

Abstract

INTRODUCTION

The Chief Medical Officer's 2008 annual report highlighted the importance of simulation in medical training.¹ Simulator development has focused on increasing authenticity and fidelity. Development has not necessarily been guided by evidence for educational improvement. On reviewing 34 years of literature, Issenberg et al identified ten features of high-fidelity medical simulators that facilitate learning.² This study compares cadaveric temporal bone (CTB) simulation with the Voxel-Man TempoSurg (VT) virtual reality simulator in addressing these features.

SUBJECTS AND METHODS

A questionnaire was designed comparing the VT with CTB. Fourteen trainees and six consultants completed the questionnaire after using the simulator.

RESULTS

The VT is better at allowing repetitive practice, ease of control of difficulty, and capturing clinical and pathological variation. The VT is as good as CTB in curriculum integration, allowing multiple learning strategies, providing a controlled environment, individualising learning and defining benchmarks. It appears worse with regards to face validity and feedback.

CONCLUSIONS

Virtual reality simulation and CTB have features that allow effective learning. Some of these are common to both, in some CTB is better and in others virtual reality is better. Virtual reality could be a significant mode of learning supplementary to CTB and experience in the operating theatre.

The increasing importance of the role of simulation in medical training was highlighted in the Chief Medical Officer's 2008 annual report.¹ The driving arguments are the reduced experience that trainee doctors have due to compliance with the European Working Time Regulations, the increasing availability of good simulation, the evidence demonstrating good skills acquisition through simulation and the increased focus on patient safety in medicine.

Technology companies have driven the development of simulators and have focused on increasing authenticity and fidelity. Development has not necessarily taken into account evidence of educational impact. In order to redress this, Issenberg et al reviewed 34 years of literature and identified 10 features and uses of high-fidelity medical simulation that leads to effective learning² (Fig. 1).

Figure 1.

Ten features and uses of high-fidelity medical simulation that leads to effective learning

Simulation with cadaveric temporal bone (CTB) has been used in mastoid surgery for over 50 years. The Voxel-Man TempoSurg (VT) is a high-fidelity virtual reality temporal bone simulator, the core aim of which is to simulate temporal bone drilling. The key elements of the simulator are the visual display and virtual drill. By viewing the display with the glasses provided, a three-dimensional image is perceived. The trainee can adjust the angle of view and magnification. The virtual drill is a stylus that is attached via a hinged lever arm to the simulator. This generates a feedback of mechanical force that correlates to the drilling perceived through the visual display. This is known as haptic feedback. There are options to change between a diamond and metal burr as well as a full range of burr sizes. The visual display offers a variety of optional aids to learning. These include CT images showing the position of the burr within the bone and a chart on the screen displaying the distance of the drill from key landmarks such as the dura and facial nerve.

This study aims to make an assessment of this simulator in addressing the ten features highlighted by Issenberg et al.² To allow a tangible assessment, the VT is compared against training by dissection of human CTB.

Subjects and Methods

A questionnaire (Fig 2) was designed comparing the VT with CTB in the ten areas identified by Issenberg et al.² Fourteen trainees, all of whom had experience of drilling CTB, attended a two-day mastoid surgery course, during which a number of procedures were performed on the Voxel-Man TempoSurg (Voxel-Man Group, Hamburg, Germany) virtual reality simulator. The course started with instructions on using the simulator and procedures being demonstrated on the simulator by the senior author with the image projected on to a large screen. Six simulators were available and all trainees performed cortical mastoidectomy, atticotomy and posterior tympanotomy. Additionally, specific, defined tasks (such as exposing the sinodural angle and exposing the facial nerve) were performed. The procedures were performed only once but participants were shown how to rewind procedures should they wish to repeat sections. Six consultant trainers provided guidance and also performed tasks on the simulator. on completing the course, all participants were asked to fill in the questionnaire. The results were tabulated and analysed in Microsoft® Excel®.

Figure 2.

Questionnaire

Results

Twenty participants completed the questionnaire. These comprised two senior house officers, twelve registrars and six consultants. Figure 3 illustrates the range of scores, interquartile range and median scores.

Figure 3.

Scoring for questions

Discussion

Attaining the level of excellence expected by medical professionals in technically complex tasks requires extensive training. Traditionally, training involved years of practice under the supervision of a specialist. Many drivers have resulted in this being less feasible in modern healthcare systems. Reduced training hours due to the European Working Time Regulations and an expectation that specialists rather than trainees deliver treatment have resulted in trainees receiving less operative experience.³

The current UK requirements to attain a Certificate of Completion of Training are ten mastoid operations as the only scrubbed surgeon. This alone is unlikely to be sufficient training. Trainees must therefore increasingly utilise additional resources to acquire the skills needed to achieve expertise.

CTB dissection is the established mode of obtaining the skills needed for mastoid surgery. The legal requirements for acquiring CTB and the health and safety requirements for CTB dissection rooms have made CTB less available. Trainees continue to attend CTB courses but the cost, often borne by the trainee, means that the amount of experience they receive from this source is limited. There is therefore a need to find further practical methods for trainee surgeons to acquire the skills needed for mastoid surgery.

The systematic review by Issenberg et al aimed to address the question ‘What are the features and uses of high-fidelity medical simulations that lead to the most effective learning?'.² Ten features and uses that contribute to effective learning were identified. This study targeted these ten areas and, to make a tangible comparison, the VT virtual reality simulator was compared with the established mode of simulation using CTB.

The study set out to give an indication of the strengths and weaknesses of the VT rather than to achieve statistically significant data. The most meaningful result for comparison was thought to be the interquartile range. With this in mind, the VT appears to be more effective in allowing repetitive practice, it allows the trainee to control difficulty level more effectively and is more able to capture clinical and pathological variation.

In comparison with CTB, the VT is easy to set up and allows procedures to be performed in many stages by saving at each stage. This makes it easier to practise around clinical commitments. The VT comes with several pre-programmed temporal bones with varying anatomy and pathology. The trainee can also control the difficulty level of each bone by altering the amount of intrinsic and extrinsic feedback. additionally, the simulator is able to model a temporal bone from real patient CT data. Preoperative rehearsal and postoperative repeat of the exact procedure, or alternative approaches, on a particular patient provide the potential for individually focused simulation and learning from mistakes, in a way that is likely to be hugely heneficial. Thus, not only could simulation contribute to trainee education, it could enhance reflective practice in all ear surgeons. The senior author has used the facility to practise unfamiliar procedures immediately prior to surgery with good effect.

The VT appears as good as CTB in curriculum integration, allowing multiple learning strategies, providing a controlled environment, individualising learning and defining benchmarks. other than ease of curriculum integration, these findings are unsurprising. The ability to record and review dissection sessions on a personal computer, with trainers and the possibility of practising alone intuitively suggest that the VT would allow easier curriculum integration.

The VT appears worse with regard to feedback and face validity. Concerning feedback, it is the authors’ impression that the question ‘In which simulation is the feedback more useful in aiding learning?’ was unclear. It could be interpreted as intrinsic feedback from the feel of the drill against the bone and visual clues of proximity to anatomical structures. The alternative meaning is extrinsic feedback. The simulator provides this by way of a display of the drill in three planes of radiological anatomy, metrics showing the distance between the drill and important anatomical structures, and the ability to record performance for asynchronous feedback with a trainer. The evidence from Issenberg et al's paper was for extrinsic feedback.² This should have been made more clear in the questionnaire.

With regard to face validity, it is unsurprising that virtual simulation has notyetreached the realism of CTB. However, importantly, it should be noted that face validity is the least important of the ten features. Though not formally assessed, the novelty of the simulator seems to influence the scoring of face validity. More than half the assessors had little or no prior exposure to the simulator. The authors’ experience is that extensive use results in a loss of this novelty effect and, increasingly, the user can become as immersed in the simulated experience as in real mastoid surgery.

This process of ‘willing suspension of disbelief’ is a key component of all forms of simulation. again, the authors’ experience is that the virtual drill has a very similar haptic feedback to sharp single-use burrs found on many modern high speed drill systems. With less experienced trainees the impression of validity was possibly based more on expectations of how the drilling should feel, perhaps using old burrs and drills in a temporal bone lab, rather than the experience of actual surgery. It would be interesting to investigate how real the simulation appeared to a group of surgeons who had extensive experience, both of mastoid surgery and on the simulator.

This sort of study often raises as many issues as it answers. As well as the varied experience of assessors on real bones and simulators, and potential differences in the interpretation of questions, the study showed many layers of expectation among participants. One commonly expressed belief was that ‘real bone must be better’ and that facilities for practising on ‘real bone’ were being sacrificed in order to embrace new technology. Some felt that their training was being compromised in this way, for reasons of cost or utility. These factors and others would all influence each individual's scoring of the questionnaire and would explain the spread in responses.

Other commonly expressed opinions were that the simulator was an excellent aid to learning the three-dimensional ‘operative’ anatomy that is clearly integral to competent mastoid surgery. one annoyance of the system is the great difficulty to skeletonise structures without damaging them. This has been communicated to the manufacturer, who aims to make adaptations.

Conclusions

These perceptions and beliefs of trainees and trainers about virtual reality simulation are clearly fundamental to its degree of acceptance and educational utility. There is a need for rigorous studies with qualitative methodology to explore this area in greater depth.

The discussion that we would hope to promote is that both cadaveric bone and virtual reality simulation have features and uses that allow effective learning. Some of these are common to both, in some cadaveric bone is better and in others virtual reality is better. Additionally, in the ‘real world’ where instant access to a set up CTB is unlikely, virtual reality will be a significant supplementary mode of learning in addition to CTB and experience in operating theatre.