Simulation in surgical education

In recent years, simulation has come to the foreground as a method of teaching technical skills to surgical trainees. Rapid changes in health care systems underlie the shift to simulation-based education in many countries. Pressures include a decrease in work hours for residents, a shortening of training programs, a decrease in available operating-room time, and ethical imperatives to protect patients from harm.^1–3 The expectation that trainees will acquire all necessary skills in a purely clinical environment is no longer realistic. Simulation offers the benefits of a safe environment for practice and error, opportunities for feedback and assessment, and standardized experience for trainees.^1,3–5 It also realizes the concept of the “pretrained” novice, whereby a resident learns basic technical skills in a simulated environment so that when faced with an analogous clinical situation, he or she has already completed the early phase of the learning curve. The result is improved patient safety and learning efficiency.^6,7

Although acquisition of technical skills is undoubtedly essential for surgical trainees, non-technical factors such as communication, leadership and teamwork have also been shown to play a substantial role in operative success. A breakdown in these nontechnical factors has been shown to increase the rate of errors in the operating room.^8–11 In addition to their role in acquisition of technical skills, simulators also have potential for use in teaching and assessment of non-technical skills.

We review the simulators currently available to surgical educators, discuss the potential uses for simulation technology in surgery and review the current evidence for the role of simulation in both the teaching and assessment of technical and non-technical skills.

Methods

We searched MEDLINE and the Cochrane Database of Systematic Reviews to identify studies of potential relevance. The Cochrane search yielded one relevant article. In the MED-LINE search, we used the Medical Subject Headings (MeSH) “computer simulation” and “education, medical, graduate” (retrieved articles 143, relevant articles 22) and the headings “laparoscopy” and “computer simulation” (retrieved 229, relevant 23). The following keywords were used in the MED-LINE search: “technical skills” and “assessment” (retrieved 283, relevant 64) and “non-technical skills” (retrieved 41, relevant 15). We focused the search on articles published from 1996 onwards, and we limited it to articles published in English. We did not include case reports and data from abstracts in our data synthesis. All of the abstracts of identified articles were examined for relevance. Retrieved studies were screened for duplication, and additional studies were identified using a manual search of the reference lists of the relevant included articles. Given that our review is focused on the role of simulation in surgery, our search strategy was limited to identifying articles focusing on surgical education.

Available simulation models

A wide variety of models is currently available for teaching technical skills. These models range from high-fidelity animal or cadaveric models to virtual-reality simulators.

Animals and human cadavers

Some of the original models used to teach procedural skills are human cadavers and animals (e.g., the porcine model for bronchoscopy, endoscopic retrograde cholangiopancreatography, laparoscopic cholecystectomy, or the canine model for coronary bypass).¹² The advantages of animal models include realism and opportunities to mimic complications. However, they have been criticized for their expense, the fact that their anatomy can differ from that of humans, and ethical reasons.^12–15 Cadavers are infrequently used in surgical education because of cost, limited availability and inability to simulate complications such as bleeding.^12,15

Synthetic benchtop models and tower trainers

Synthetic models include those designed to teach open surgical procedures (commonly referred to as benchtop models) and those used for teaching minimally invasive procedures (referred to as tower trainers or video-box trainers). Numerous benchtop models are available for the simulation of a variety of procedures, including knot-tying, fascial closure, and suturing¹⁴ (Figure 1). The most obvious limitation of such models is that they teach only one surgical technique rather than an entire operation.⁵ Their use requires the presence of an expert to demonstrate the procedure and provide feedback on performance.^1,5

Figure 1.

A synthetic benchtop model for teaching abdominal wall closure.

Tower trainers include a box with slits on the superior surface for trocar insertion. Real laparoscopic instruments are inserted through the trocars into the box, where the procedure is simulated. A camera inside the box provides video output to a monitor on which trainees can watch their own movements. These models can simulate a variety of techniques, including laparoscopic suturing, knot-tying, clip-applying and coordination drills.¹⁴ One of the most well-described tower trainers is the physical laparoscopy trainer known as the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills. This system consists of five tasks performed on an endotrainer box.¹⁶ The tasks include peg transfer, cutting, placing a ligating loop and intracorporeal suturing.

Virtual-reality simulation

Training by virtual-reality simulation encompasses systems designed to teach laparoscopic, endoscopic and percutaneous interventions.¹³ The fidelity of virtual-reality simulators ranges widely, from the Minimally Invasive Surgical Trainer – Virtual Reality (MIST–VR) system, which is a low-fidelity system designed to teach general laparoscopic proficiency, to systems that teach component procedural skills and other systems that replicate entire operations¹⁷ (Figure 2 and Videos 1 and 2, available at www.cmaj.ca/cgi/content/full/cmaj.091743/DC1). Unlike bench-model learning, the presence of an expert is not necessary when virtual-reality simulation is used. Such models allow for practice at varying levels of difficulty, can simulate complications and automatically provide objective measures of assessment, allowing for both formative and summative trainee assessment.¹⁸ Virtual-reality models have been criticized, however, for high initial cost of system acquisition, limited force-feedback with use of surgical instruments and lack of realism of graphics.^14,19

Figure 2.

A virtual reality simulator. (Image reproduced with permission from Surgical Science, Inc.)

The virtual operating room

Benchtop models, virtual-reality models, and animal and cadaveric simulators are all focused on teaching the technical skill set necessary to perform a wide range of surgical procedures. To both teach and assess non-technical and technical aspects of performance, virtual operating rooms have been developed.^4,20–22 These rooms consist of an operating table, lights, suction equipment and an anesthesia machine. They contain a simulator (either synthetic or virtual reality) and a control room, which is separated from the operating room by a one-way mirror. Virtual operating rooms are designed to take into account the technical skills required to perform an operation, along with the intangible factors, such as interpersonal dynamics or teamwork, that are also responsible for operative success.

The role of simulators in surgical education

A critical role of simulators is to teach trainees a specific skill set. When assessing simulators for utility in a particular curriculum, an examination of the learning curve on a particular simulator and evidence for transfer validity of the taught skills are important. Learning curves refer to improvement of performance as measured on a particular simulator. Transfer validity, or the degree to which skills learned on the simulator result in an improvement in skills in an analogous clinical situation, is essential.⁷

Parallel to the development of simulators for technical skills training has been that of assessment tools for measuring these acquired skills. Assessment allows for monitoring of skills acquisition of surgical trainees and provides a base for constructive feedback along the learning curve for new procedures.²³ Objective measures of performance are crucial for determining advancement to the next stage of training by surgical trainees or re-entry after a career break by staff surgeons.²³ Simulators must demonstrate acceptable reliability and validity before they are integrated into high-stakes assessment.

Assessment tools can be classified into three categories: observational tools, virtual reality and motion analysis (Table 1). Observational tools rely on an expert observer who assesses technical ability according to defined criteria on either a global rating scale or a procedure-specific checklist. Virtual-reality simulators provide immediate and automatic assessment by recording metrics such as time taken and error score. Analyses of dexterity equate technical ability with the number and speed of a surgeon’s hand movements.

Table 1.

Tools for assessing technical skill, by level of validity

Assessment tool	Examples of specific tools	Evidence for validity	Advantages	Disadvantages
High validity
Global rating scales	Objective structured assessment of technical skills (OSATS) Global operative assessment of laparoscopic skills (GOALS)	Observational studies23–28	Applicable in a wide range of situations (open surgery, bench-station exams laparoscopic surgery, video-based assessment)	Requires time of an expert assessor
Virtual-reality simulators	Laparoscopy Bronchoscopy Endovascular	Observational studies36–38	Automatic, non-biased assessment	Feedback not useful for the trainee
Moderate validity
Checklist scales	Laparoscopic cholecystectomy Carotid endartorectomy	Observational studies32–35	Novices can see specifically where they need to improve their skills	Requires time of an expert assessor. Binary assessment.
Analysis of dexterity	Imperial College surgical assessment device (ICSAD) Advanced Dundee endoscopic psychomotor tester (ADEPT)	Observational studies39–40	Automatic, nonbiased assessment	Feedback not useful for the trainee. Feasibility-related concerns.
Procedure-specific checklists	Laparoscopic colorectal surgery Laparoscopic cholecystectomy Gastric bypass	Observational studies29–31	Novices can see specifically where they need to improve their skills	Requires time of an expert assessor

Several assessment tools for non-technical scales have also been designed for surgeons. Yule and colleagues developed a taxonomy and behaviour-rating system for surgeons termed Non-Technical Skills for Surgeons. This rating scale consists of five categories: situation awareness, decision-making, task management, communication and teamwork, and leadership.^41,42 The inter-rater reliability and intra-class correlation coefficients of this tool have been demonstrated in initial validation studies.⁴¹ A similar tool, known as the Non-Technical Skills Scale, has been adapted from the aviation industry and shown acceptable reliability and validity for use in surgery.^43,44 Using these assessment tools effectively, however, requires some degree of training.

Effectiveness in teaching technical skills

Ultimately, the goal of learning technical skills on simulators is to improve technical performance in the operating room. Technical skills learned both on low-fidelity benchtop simulators and on high-fidelity virtual-reality systems have been shown to transfer to the operating room. Assessing which simulator is more useful in which type of training situation, however, is more difficult to determine.

Animal and cadaveric models

Few trials have examined the role that animal or cadaveric models play in a surgical skills curriculum. Several studies have compared these high-fidelity models to their low-fidelity counterparts in teaching technical skill. In a randomized single-blinded trial, Grober and colleagues demonstrated that micro-surgical training on the vas deferens of a live rat compared with use of a low-fidelity silicone tubing model produced an equivalent improvement in technical skills on both models using a previously validated checklist and global rating scale.⁴⁵

These findings are similar to those of an earlier randomized single-blinded study by Anastakis and colleagues, in which training on a cadaver was found to be equivalent to low-fidelity benchtop training.¹⁵ Among trainees with increased technical experience, however, learning on higher-fidelity models (i.e., animal or cadaveric) appear to result in greater improvement in technical proficiency compared with training on low-fidelity models.⁴⁶ These studies illustrate that inexpensive, reliable and valid models exist that are equivalent to animal or cadaveric models for training novice surgeons.

Synthetic models

The evidence for effective learning using synthetic training tools is well established. In a blinded, randomized controlled trial, Traxer and colleagues found that practice on a video-trainer resulted in a 51% reduction in time (as measured on the simulator, p = 0.003) and an improvement in technical ability (measured by a validated global assessment tool in a porcine model p = 0.0008) as compared with a no-training control group.⁴⁷ Similarly, Fried and colleagues established the educational value of the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills by showing evidence of its transfer validity to an animal model.^16,48,49 Transfer validity to the operating room has also been shown in two randomized controlled trials.^50,51 Finally, a 2006 systematic review of 14 studies involving a total of 261 participants found that video-based tower training is superior to either no training or conventional training.¹⁹

Not all training on synthetic models is equivalent, and surgical educators must therefore determine the training schedule for a particular curriculum thoughtfully. The quality and amount of expert feedback provided during the learning process plays an essential role in the acquisition of technical skills. Summary feedback (i.e., given at the completion of a task) is reported to be most useful for learning as compared to concurrent feedback (given as the task is occurring).^52–55 Moreover, distributed (as opposed to massed) practice significantly improves learning.⁵⁶ Finally, as with the equivalence shown between high-fidelity animal models and low-fidelity benchtop models, high-fidelity benchtop models are equivalent to low-fidelity benchtop models in acquisition of technical skills by novices.⁵⁷

Virtual reality

Novices attain expert levels of proficiency after at least six trials on a simulator, whereas experts and intermediates tend to plateau much faster, according to observational studies.^36,58–65 Individual variations exist in the rate of acquisition of technical skills, and therefore training to expert proficiency (as opposed to training for a certain duration) may be the most efficient means by which to enhance learning of technical skills on virtual-reality simulators.^66,67

Two randomized unblinded trials showed that, compared with participants with no training, those trained using virtual-reality tools showed significantly improved learning, as measured by both previously validated global assessment measures and automatic simulator-generated parameters.^68,69 The technical skills acquired during virtual-reality training have been shown to be transferable both to an animal model and to patients in the operating room.^70–73 In a randomized double-blinded study, Seymour and colleagues showed that particpants who received low-fidelity virtual-reality training showed improved time to complete laparoscopic cholecystectomy (29% faster than residents not trained using virtual reality), were five times more likely to make progress and six times less likely to make errors compared with residents not trained using virtual reality.⁷⁴ These results were replicated in a later randomized blinded controlled trial by Grantcharov and colleagues.²⁴ Moreover, the effect of virtual-reality training was shown to persist for the first 10 laparoscopic cholecystectomies performed by a novice resident.⁷⁵ Finally, Larsen and colleagues reported improved performance after virtual-reality training compared with a control group in operative salpingectomy with respect to time (p < 0.0001) and on a global rating scale (p < 0.0001), in a randomized blinded study using previously validated assessment measures.⁷⁶

Both video-trainers and virtual-reality models have been shown to be effective for learning technical skills, and these skills have been shown to transfer to real clinical situations. In several randomized controlled trials, as well as a recent systematic review looking only at novice surgical trainees, virtual-reality training was shown to be at least as effective as benchtop training in the acquisition of technical skills.^65,77–80 Notably, many of the identified studies had methodological limitations, including small sample sizes, inadequate blinding, disparate interventions and use of surrogate outcomes.^19,81,82

A small number of studies, however, suggest that virtual reality training may be superior to benchtop-model training. In a randomized trial, Youngblood and colleagues reported that surgically naïve medical students who trained on a virtual-reality simulator, as opposed to a tower trainer, showed improved time, accuracy and global rating scores, as measured in a porcine model.⁸³ The assessment measures used in this study, however, were not validated. Hamilton and colleagues found that surgical residents who received virtual-reality (as opposed to tower) training showed improved operative performance during laparoscopic cholecystectomy, as measured by a global rating scale completed by blinded faculty observers.⁸⁴ Although these findings suggest that virtual-reality training may provide more benefit than benchtop training, more well-designed randomized trials need to be performed before firm conclusions can be drawn.

Each virtual-reality system has various levels of realism and capabilities for haptics. Haptics allow for trainees practicing on virtual-reality simulators to feel force on their instruments, thus providing a higher degree of realism to the device. These features add substantial cost to the device. In a crossover study, Panait and colleagues investigated the role of haptics in technical skills acquisition. Students learned tasks on a virtual-reality model with haptics either present or absent.⁸⁵ The results showed a trend toward improved skill acquisition for complex tasks only when haptics were enabled, suggesting that the cost of haptics-enhanced simulation may be more worthwhile for complex tasks, such as learning operations in their entirety.

The evidence suggests that trainees should learn technical skills in a gradated manner on simulators. Early learning could occur on low-fidelity synthetic or virtual-reality models. Trainees could then progress to training on high-fidelity synthetic or animal models, or to virtual-reality systems that simulate complex procedures with the added effect of force-feedback.

Effectiveness in teaching non-technical skills

The development and validation of a simulated operating room where nurses, anesthesiologists and surgeons can work through a scenario designed to assess and teach both technical and non-technical skills has been described by Moorthy and colleagues and by Undre and colleagues.^4,22 Both of these studies showed the feasibility of interdisciplinary training and also confirmed participant satisfaction in the virtual operating-room experience. Construct validity (i.e., ability to distinguish between experts and novices) was similarly demonstrated for a laparoscopic-crisis scenario in a simulated operating room.²¹

The results are ambiguous, however, with respect to using these simulated environments to specifically teach complex non-technical skills. While simulated operating rooms are generally thought to be useful for training members of the operating-room team, they are complex, and expensive to maintain. By contrast, the Integrated Procedural Performance Instrument developed by Kneebone and colleagues at the Imperial College in London^86,87 is a simulation model that combines use of physical benchtop simulators to test technical skill with use of standardized patients to assess non-technical skills. Using this model, LeBlanc and colleagues found in an observational study that communication skills are not correlated with technical skill, thus emphasizing the need for specific training in non-technical skills in surgical training programs.⁸⁸

Gaps in knowledge

The goal of implementing simulation-based training in surgery is to provide a complementary experience that accelerates the learning curve and enhances patient encounters.¹⁴ Although the literature supports both synthetic and virtual-reality training in technical skills acquisition, there are some weaknesses in this body of literature. First, although a large variety of virtual-reality simulators is available on the market, many of the existing studies investigated the earlier virtual-reality models. Second, many of the new procedural models lack evidence for validity, which raises concern that the field will be dominated by technology rather than educational principles. Finally, the studies describe disparate interventions, sometimes on the same simulator, making comparisons difficult. Further research in the field should therefore be directed toward the newer procedural simulators, because they are substantially more costly than the lower-fidelity models, and should target surgical trainees rather than medical students.

Simulators are only part of the solution to training staff and residents outside of the operating room. As Gallagher stated, “their power can only be truly realized if they are integrated into a validated comprehensive curriculum.”⁷ Although there are studies looking at specific tasks within a proficiency-based curriculum,^24,89–91 a more holistic curriculum that looks at teaching and assessing both technical and non-technical skills needs development and validation. To overcome the focus in the literature on validating specific simulators for specific surgical tasks, Schout and colleagues recommend designing and evaluating a comprehensive training program rather than validating only one aspect of the curriculum that can be performed on a simulator.¹⁸

Key points

• Both technical and non-technical skills are essential for surgeons in training.

• Technical skills learned on low-fidelity benchtop models, video-based trainers and virtual-reality models are transferable to the operating room.

• Current evidence does not show a clear advantage of either virtual reality or synthetic models in teaching technical skills.

• Evidence is lacking on how best to integrate each type of simulator into a holistic, proficiency-based curriculum.