Otologic Skills Training

Synopsis

This review presents a summary of the current activity of simulation training for otologic skills. Simulation training has been demonstrated in a large spectrum of skills from simple otoscopy to advanced temporal bone surgical procedures and these are individually addressed. There is a wide variety of educational approaches, assessment tools and simulators in use including simple low cost task trainers to complex computer based virtual reality systems. A systematic approach to otologic skills training using adult learning theory concepts such as repeated and distributed practice, self-directed learning, and mastery learning is necessary for these educational interventions to be effective. Future directions include development of valid, universally accepted measures of performance to assess efficacy of simulation training interventions and for complex procedures, improvement in fidelity based on the educational goals for the particular skill.

Introduction

Otologic Skills Training encompasses a range of procedures from those that need to be mastered by all medical doctors such as otoscopy, basic procedures needed by the general otologist such as myringotomy and more advanced procedures such as mastoidectomy and lateral skull base procedures. Currently, in the United States, training in otologic skills in the specialty of otolaryngology is accomplished throughout the 5-year course of clinical study. This has traditionally consisted of a gradual increase in exposure and practice beginning with the most basic procedures such as cerumen removal to the more complex lateral skull base procedures. This time honored training approach however has come under pressure for change as a result of a number of factors including less time available for individual teaching and emphasis placed on patient safety whereby attending physicians are more sensitized to trainee involvement in patient care. Given these challenges that face all medical/surgical training programs, educators have increasingly looked toward simulation as a potential tool to mitigate these issues. In Otolaryngology, several recent reviews of simulation activity noted that the field of otology is by far and away one of the most developed with respect to simulation applications.^1,2 This review presents a summary of all otological procedures where simulation-based training is currently available and described in the literature ranging from diagnostic procedures to mastoidectomy and more advanced procedures. This includes both physical and computer based “virtual reality” (VR) models. The review will be divided based upon procedure type, and where applicable, subsections will address the need for training, target trainees, available training systems, evidence for efficacy in training and means for assessment of technical skill. Key features of effective simulation based training in otology and future directions will be presented.

Otoscopy

Otoscopy is the visual examination of the ear canal and the tympanic membrane and is used to diagnose a wide range of common ear canal and middle ear diseases such as external otitis, acute and serous otitis media, and tympanic membrane perforation, in addition to identifying infrequent but important pathology such as cholesteatoma that needs referral for surgery. Otoscopy is a common procedure and a key skill for all clinicians including general practitioners and pediatricians. The otologist will most often prefer otomicroscopy to allow magnification and simultaneous procedures such as removal of cerumen but much of the following discussion applies to otomicroscopy training as well.

Otoscopy skills can be taught on peers or patients because it causes little discomfort. However, otoscopy relies on the coordination of the instrument and the examiner’s visual field, making it difficult to supervise in the training situation and to ensure that a systematic approach is learned unless a video otoscope is routinely used for training with feedback. In addition, adequate exposure to the full range of pathology can be difficult to achieve: often, training consists of practical training on the patients supplemented by textbook/atlas images of pathology. Optimally, training in otoscopy consists of repeated hands-on practice with feedback of otoscope and patient handling while also directing a systematic approach to the examination. There is a need for improvement in otoscopy skills training as general practitioners and medical students have demonstrated comparable but mediocre otoscopy skills³.

Currently a range of simulation-based training models for otoscopy have been reported: mannequin models for otoscopy and pneumatic otoscopy (Spectrum Nasco, Newmarket, Ontario, Canada, and Limbs and Things, Bristol, UK)⁴, a web-based platform with 3D models of the ear displayed on a computer screen^5,6, and more advanced models with a variety of cases that can also track the otoscope for annotation⁷ and provide automated feedback⁸. The OtoSim (OtoSim Inc., Toronto, Ontario, Canada) has been widely marketed and consists of the physical interface of an adult auricle and external canal with a small LED screen which displays scaled images of normal and pathologic tympanic membranes. The instructor can control which image is displayed and point out pathology from a laptop computer connected to the interface. Recently, the system has been upgraded to include pneumatic otoscopy capabilities. There is some evidence that simulation-based training can increase medical students’ confidence in otoscopy ⁹ and the diagnostic accuracy of residents ¹⁰.

For the assessment of otoscopy skills in a pediatric setting, a standardized checklist for otoscopy performance evaluation (SCOPE) has been developed¹¹. There are currently no widely used or accepted instruments for the assessment of otoscopy skills in adults or in simulation-based training of otoscopy.

Myringotomy and Tympanoplasty

Myringotomy is the incision of the tympanic membrane to equilibrate pressure or drain fluid from the middle ear and is often accompanied by the insertion of a tympanostomy tube or grommet. Myringotomy is the most common surgical procedure in otology and one of the first skills learned by the ORL trainee. Tympanoplasty is the reconstruction of the tympanic membrane with and without ossicular chain reconstruction and is typically performed by a surgical otologist.

Physical models of the tympanic membrane are commonly used for initial training of myringotomy with tympanostomy tube insertion. Numerous models have been described ranging from DIY models based on readily available materials¹² to more complex physical models^13,14. In addition, a VR simulation model for myringotomy and tube insertion with supporting haptic feedback has been developed ^15–18. Although global rating scales and task-based checklists have been developed and validated^12,19 to assess for myringotomy with tube insertion competency, evidence to support the efficacy of myringotomy simulation training is limited and high quality studies examining the transfer of skills to improved patient outcomes are needed (see SimTube project below).

Tympanoplasty with/without ossicular chain reconstruction is frequently taught on cadavers with the exception of the plastic Pettigrew temporal bone model that can incorporate relevant disease processes¹⁴, nevertheless, reports on tympanoplasty training are scant and evidence for efficacy is lacking.

Mastoidectomy

Mastoidectomy involves drilling of the temporal bone mastoid air cells with the purpose of treating infection or pathology such as cholesteatoma or to gain access to the middle ear, the cochlea or the sinodural angle for lateral skull base procedures. Basic understanding and competency in mastoidectomy is expected of all otologists and is an important part of ORL residency training.

Temporal bone surgery requires precise motor skills to handle the otosurgical drill and suction irrigation under magnification of the operating microscope. In addition, the temporal bone anatomy is complex with vital anatomic structures such as the facial nerve, the chorda tympani, the sigmoid sinus, the dura mater, the vestibular organ and the ossicles. These complex skills are typically taught through cadaveric dissection during temporal bone courses or in training facilities with a temporal bone lab, all followed by supervised surgery. Cadaveric dissection is the gold-standard training modality for mastoidectomy even as evidence of its role in training has recently begun to emerge^20,21. However, the availability of human temporal bones in addition to the cost of maintaining facilities among other issues considerably limits the availability of cadaveric dissection training of mastoidectomy. Plastic and plaster models,^14,22,23 and recently 3D printed temporal bones^24,25, have been introduced to alleviate these issues, however the physical properties and fidelity of vital structures in these models, along with costs, limit the adaptation of these simulators into mainstream temporal bone surgical training.

A number of VR simulators have been developed and validated for temporal bone surgical training with the potential of immediate assessment of performance and monitoring of individual trainees’ progress while providing automated feedback and tutoring. Most VR temporal bone simulators use a volumetric model that supports haptic interaction for drilling with force feedback and uses different technologies to accomplish 3D stereo graphics. VR simulators are based on either CT-derived data (Voxel-Man²⁶, the Ohio State University²⁷, the Stanford BioRobotics lab²⁸, and the University of Melbourne²⁹), or on cryo sections of a fresh frozen human temporal bone (Visible Ear Simulator^{30, 31}).

Strong evidence exists to support the use of VR simulation training in temporal bone surgical skills training. First, novice vs. expert performance can be discriminated in VR simulation of temporal bone surgery^32–34 and many simulator-based metrics correlate with experience^1,35. Second, VR simulation performance has been demonstrated to be similar to the dissection performance in trainees³⁶ and VR simulation training seems to be superior to training methods such as video demonstration³⁷. Also, VR simulation training allows for self-directed training of mastoidectomy with an acceptable level of performance, minimizing the need for human instructional resources ³⁸. Finally, repeated VR simulation practice results in improved learning curves, ^39,40 acceptable retention of the procedure skills⁴¹, and significantly increased cadaveric dissection performance⁴².

Several assessment tools for mastoidectomy performance and competency have been described for use in the operating room, cadaveric dissection lab or VR simulation setting. These tools incorporate global rating scales, task-based checklists and/or final-product analysis. A recent review provides an excellent overview of the different mastoidectomy assessment tools and the current validity evidence for each tool⁴³.

Advanced Otologic Procedures

Surgical implantation of cochlear implants and other implantable hearing devices, as well as middle ear reconstruction using prostheses and stapes surgery, are considered advanced procedures of interest for sub-specialist training in otology. Currently, this advanced training is acheived on cadavers and supervised surgery as few training models are available beyond research prototypes. Physical models for the placement of stapes prosthesis include simulators using readily available materials,^44,45 the Pettigrew Temporal Bone Model with stapedotomy and prosthesis placement¹⁴, and models for electrode placement in cochlear implantation. In 2011, a prototype VR simulation model for cochlear implantation with haptic feedback was described ⁴⁶. To date, there limited research on the training and assessment of advanced otologic skills with the exception of a single report on the development and validation of an assessment tool for competency in cochlear implant surgery⁴⁷.

Key features of effective simulation-based otological skills training

Repeated practice and mastery learning

Studies have shown that 10–15 operative procedures are needed for technical and basic competency in mastoidectomy^48,49. However, proficiency in surgical procedures requires substantially more practice—up to 100 procedures—but often at the cost of patient discomfort, longer procedural times and increased risk of complications⁵⁰. The learning curve must be considered and simulation-based training allows for repetitive practice tailored to the individual trainee’s needs. Such mastery learning is crucial in competency-based education and marks a paradigm shift in surgical education, where length of training or number of procedures had been measured outcome. Mastery learning redefines the goals of training as consistent performance with evidence-based level of proficiency⁵¹. To achieve mastery learning, deliberate practice with feedback is key^52,53,54. The controlled environment of a simulation setting offers an ideal platform for setting standards and defining mastery levels before training on cadavers or supervised surgery. For otological procedures, these standards have yet to be defined using validated assessment tools..

Practice organization

Supported by established theory of motor skills learning and current evidence in other surgical procedures, a study of learning curves in VR simulation-based training of mastoidectomy found that practice should be distributed. In other words, practice sessions should be short (3–4 procedures per session) and spaced (by at least 3 days) for optimal learning³⁹. In reality, dissection training is often organized as short and intense courses; massed practice consistently leads to suboptimal skills acquisition, retention and transfer^41,55–57. The positive effect of distributed practice can be attributed to time dependent consolidation of memory⁵⁷ and even for a simple procedure such as myringotomy, spacing practice by a single day is insufficient in improving novice performance⁵⁸. Therefore, the increasingly popular “ORL surgical boot camps”^59,60 with simulation-based, massed practice of a range of surgical skills may have limited long term effectiveness for learning otologic skills.

Instructional design, feedback and self-directed training

The single most important feature of effective simulation-based training is feedback⁶¹. One of the benefits of VR simulation is the potential for self-directed learning using simulator-integrated tutoring, guidance and feedback eliminating the need for human instructors, who are often limited by clinical duties and other time constraints., A directed, and self-regulated approach promotes independent learning in a structured setting with a strong instructional design⁶². In mastoidectomy, two hours of self-directed VR simulation training with automated guidance has been found to be superior to small-group tutorials using operative videos and temporal bone models ⁶³ and, in a different study, was found to increase subsequent dissection performance by 52% ⁴².

In self-directed training, repeated practice steeply increased the performance of novices, plateauing after nine repetitions³⁹ with intact retention after 3 months of non-practice⁴¹. Simulator-integrated tutoring in VR simulation training of mastoidectomy increases the slope of the initial part of the learning curve further supporting directed, self-regulated learning³⁹. Nevertheless, many novices have difficulty knowing when to stop drilling or may injure vital structures in the anatomical boundaries of the mastoidectomy which may lead to a suboptimal performance⁶⁴. These concerns for novices suggests that self-directed training in mastoidectomy could be further improved through instructional design with specific and explicit process goals ⁶⁵.

Evidence-based training

Although research in otological skills training has focused on mastoidectomy, evidence from other technical skills and procedural training supports distributed and deliberate practice, mastery learning, and directed, self-regulated techniques in simulation-based training. A major challenge is the implementation of these evidence-based principles into high-quality training programs in otology. More importantly, simulation-based training should be “part of a coherent strategy based on clear educational aims and must mirror actual practice”.⁶⁶ Furthermore, training should employ scaffolding where subsequent learning experiences such as dissection training and supervised surgery build further on the skills acquired in simulation. Ultimately, this will lead to improved patient outcomes.

Future directions

Although there have been many technological advances leading to sophisticated computer based simulators, the above examples demonstrate several novel non-computer based simulators. Having said this, it needs to be recognized by those developing otologic simulators that the simulators are simply tools that need to be integrated into well thought out educational approaches employing adult learning theory such as accurate needs assessments/task analysis, curriculum development, and valid outcome measures in addition to the key features for effective learning noted above. For those involved in the growing simulation movement, the challenge to not focus on the simulators themselves but on the educational goals cannot be over emphasized. A well-designed simulator is one that meets an educational goal and not just a physical model or computer program that tries to replicate the real-life experience. The challenges that remain include development of universally agreed upon and valid measures of assessment, development of national and/or international frameworks for performing large scale randomized controlled trials to provide robust statistical evidence for efficacy emphasizing patient outcomes as the ultimate measure of successful interventions. Learning effectiveness studies must move from more subjective types of evidence such as self-assessments (Kirkpatrick level 1) to demonstration of results (Kirkpatrick level 4)⁶⁷. For these goals to be achieved a firm commitment must be championed not only from grass roots training programs but also from the healthcare education governance bodies.

One current attempt at instituting a nationwide otologic simulation skills training program that meets these criteria is underway. The “SimTube” project, sponsored by the American Academy of Otolaryngology Head and Neck Surgery Foundation, Inc. was developed by the Simulation Task Force to attempt to institute a large multicenter based trial of a simulation based training program for myringotomy and tube (M&T) insertion. The three hypotheses for this project include: 1. A specialty-wide multi-institutional simulation study is feasible, 2. Novices trained using the M&T simulator, compared with standard training, will achieve higher scores on both simulator and initial intra-operative OSATS, and 3. Novices trained on the simulator will reach competency sooner than those not trained on the simulator. The simulator used in this study is that which was previously described by Malekzadeh et al. ¹². The protocol includes a randomized controlled trial of novice otolaryngology trainees randomized either to traditional training at their home institution or traditional training plus initially supervised and then independent training using the simulator (Figure 1). Study subjects are assessed using a validated assessment tool modified from the Malekzadeh study which includes both a task based checklist and global rating scale. Assessments are made on all subjects initially using the simulator after each has watched an online video, one hour of training on the simulator for those randomized to simulator training and then subsequently while the subject is performing the procedure on real patients. Assessments are continued until the rater judges the subject as “competent”. Study subjects keep record of how many procedures they perform until deemed competent as well as time using the simulator. Currently, there are 65 out of the 106 otolaryngology training programs across the United States enrolled in the project with the goal to recruit a total of 314 study subjects (157 per arm). This ongoing study is one of the first of its kind to be implemented in otolaryngology. Even if the study does not show a statistically significant difference between the 2 study arms, it will provide a framework and infrastructure to perform more definitive studies of simulation based training programs.

Figure 1.

The “SimTube” study protocol demonstrating the nationwide randomized controlled simulation based training project for myringotomy and tube placement. See text for information regarding the three hypotheses of the study.

Courtesy of the American Academy of Otolaryngology—Head and Neck Surgery Foundation, Alexandria, VA; with permission.

Advancement in computer based, “VR” systems deserves special mention when future directions are addressed. Despite the fact that these systems have been present for over 10 years they have not reached mainstream training. There appears to be several limitations with these systems include suboptimal realism, lack of scientifically rigorous validation studies, cost and lack of valid and reliable assessment tools to assess performance after training⁶⁸ . The lack of realism has been pointed out in particular in temporal bone VR simulators. Although simulation training systems do not need to replicate reality, they do need to provide adequate fidelity to present key features of the real experience. In temporal bone surgery, one of the cornerstone skills that need to be developed is the identification of bone embedded landmark structures while drilling by thinning bone to a thickness that allows transillumination of the structure beneath thus allowing identification of the structure before it is violated. To date no system has achieved the ability to render bone in a virtual environment sufficiently to support practice of this skill. Currently, visual rendering has progressed as a result of unique computer algorithms taking advantage of advancements in graphics processing units (GPU) (Figure 2 and 3).⁷⁸ Haptic rendering is another field of active development and development in this area continues⁶⁹.

Figure 2.

Image of temporal bone rendered using a technique called “global illumination.” Global illumination is a graphics rendering technique that takes into account the distribution of light throughout a scene.

Figure 3.

Three examples from the Visible Ear Simulator 1-3 (VES 1-3) of a motif where realistic transparency is crucial for navigation and safe drilling. In VES 3 “natural transparency” means that only the first few voxels below a bony surface are actually transparent. Internal luminescence and external vascular texture enhance the realism of the facial nerve as observed through a surgical microscope in its canal behind a thin layer of bone.

Furthermore, the lack of scientifically rigorous validation/outcome studies continues to limit integration of VR and other simulators into regular training programs^1,68. Otolaryngology is not alone in this area as most studies of technology enhanced simulation training and assessment in health professions fall short of modern psychometric research⁷⁰. Most studies demonstrate weak designs incorporating small numbers of study subjects and outmoded concepts of validity evidence⁷¹. The challenge is to develop and execute scientifically rigorous experiments comparing the effectiveness of these systems to standard training and organizing training programs in such a way as to leverage a large number of trainees at multiple institutions. This will require a national effort orchestrated by our larger organizations such as the American Academy of Otolaryngology, the Otolaryngology Residency Review Committee and The Society of University Otolaryngologists and others.

Lastly, valid and reliable assessment of otological skills is essential in a competency-based surgical curriculum but providing useful feedback, opportunity to practice on a wide range of cases, and valid formative and summative assessment is resource intensive. VR simulation uniquely allows real time feedback and automated assessment based on simulator registered metrics. In VR simulation training of mastoidectomy, for example stroke and drilling technique can be used for effective and accurate feedback⁷², and simulator metrics can form the basis for automated assessment of the mastoidectomy performance^73,74. With further refinement, and in combination with methodologies such as CUSUM (cumulative sum)^75,76 automated feedback and assessment provided by the simulator will in the future provide individualized, directed, self-regulated and mastery learning in high quality surgical training. There are currently several assessment tools which could potentially be integrated into VR systems for temporal bone surgery. A complete review of these is provided by Sethia et al.⁴³

Pre-Operative Planning

Although not directly thought of in relation to initial skills training, pre-operative planning/practice is a type of “just in time” training and a review of otologic simulation platforms for pre-surgical planning was recently published⁷⁷. The authors found that there are several computer-based simulation platforms that have shown at least some form of feasibility to use in pre-surgical planning and practice. The systems reviewed use patient specific image data which is then presented in three-dimensional format within the simulator allowing users to view and drill on the virtual temporal bone prior to the actual surgery. The key points were that the current systems were relatively nascent for this application, that they have demonstrated feasibility in that they can import patient specific imaging studies in a timely fashion but lack the fidelity to provide significant benefit for experienced surgeons. Perhaps with the integration of the fidelity enhancements noted above, these systems will show efficacy in improving patient outcomes.

Summary and Key Points

Simulation in otologic skills training is available across the spectrum, from otoscopy to advanced lateral skull base approaches involving the temporal bone. There exists a wide variation in educational approaches, validity evidence and in simulators. For educational programs to be effective, educators must integrate concepts of adult learning theory such as distributed, self-regulated practice and mastery learning. Studies demonstrate the effectiveness of simulation training in otologic training through improved skills and performance. The field of otologic simulation continues to grow and future work will focus on improvements in simulator fidelity for advanced techniques and the development of universally accepted assessment tools for performance.

Key Points.

• Otology skills training runs the breadth of simple procedures such as otoscopy to complex lateral skull base surgery and simulation-based training of most otological procedures is possible.

• Keys to effective learning of otologic skills in a simulation based curriculum include: distributed practice, deliberate practice, mastery learning, and directed, self-regulated learning with feedback.

• Future directions are likely to include further improvement of simulator fidelity and realism. However, the development of simulation-based curricula centered on adult learning theory, national efficacy studies, and validation of assessment strategies are required for learners to fully benefit from simulation technologies.