Application of Surgical Skill Simulation Training and Assessment in Orthopaedic Trauma

Introduction

Importance of simulation

The surgical restoration of a severely fractured extremity is a complex procedure requiring skills that a surgeon trainee must acquire during their education. Traditional surgical education has consisted of a mixture of didactic lessons with periodic clinical and surgical apprenticeship-based experience. These experiences, although beneficial, are not uniform and do little to assure technical competence. The apprenticeship model is challenged not only by a lack of opportunities to expose trainees to the necessary variety of procedures,¹ and the expense of such training,^2,3 but also by the need to ensure patient safety while exposing trainees to new experiences.⁴

There is strong evidence that the current training approach may not be optimally safe. In a review of surgical errors, 63.5% of cases involved technical error and 29% included an error in judgment.⁵ Both of these types of errors can be ascribed to a lack of experience. These and similar findings have led to the call for greater transparency in the training and assessment of surgical residents. The American Board of Surgery has mandated that, rather than just documenting the surgeries a resident participated in, proficiency in basic laparoscopic skills must be documented prior to allowing graduates to be tested for certification.⁶

Surgical simulation can help address shortcomings in the traditional apprenticeship training model by providing residents with opportunities to (1) practice important procedures that they may not otherwise encounter and (2) practice procedures efficiently until competency is achieved, (3) without exposing live patients to undue risk. Simulation can provide immediate and detailed feed-back that can improve learning efficiency. With newly imposed restrictions on resident work hours, financial pressures, and increased public scrutiny, simulationbased technical training and assessment tools are receiving renewed attention. Before a surgical simulator construct can be used to assess competency, however, the scientific validity of the simulator and of the performance assessment must be established.⁷ A well-designed and rigorously validated simulator can provide quantitative, repeatable assessment of specific surgical skills and can predict performance in the operating room.

The benefits of training using surgical simulation are already being realized in general surgery, particularly for laparoscopic surgery. Medical students and residents trained on simulators demonstrate improved performance in actual surgeries. For example, in one study, training on a laparoscopic simulator was shown to lead to reliably fewer errors during actual surgery on an anesthetized animal.⁸ In another study, residents were trained to a certified level in the laparoscopic simulator and then performed their first actual surgery with fewer errors and caused fewer injuries than did a control group of non-simulation-trained residents.⁹

Deficiencies in fracture simulation

Orthopaedic surgery in general, and orthopaedic trauma surgery specifically, has lagged behind other surgical disciplines in developing and incorporating simulation of surgical skills into education and assessment paradigms. What little simulation there has been has mostly consisted of learning anatomy and surgical approaches on cadavers and placing products often supplied by medical device companies on surrogate bone specimens. At the present time there are no validated methods to assess surgical skill in orthopaedic trauma surgery. Currently, only self-reported web based case logs (as mandated by the ACGME) are utilized to document surgical experience, which does little to assess actual involvement, skill, or competency. This is particularly unfortunate, since orthopaedic trauma is a subspecialty where patients do not have the luxury of choosing their surgeon. They must depend upon on call surgeons to have the skills required to safely and successfully care for their injuries. This is troubling, because currently the ability to competently perform orthopaedic trauma surgery is learned by trainees in residency programs and practiced everywhere around the country without public assurance of even minimal procedural competence.

Rationale for articular fracture as a good target for simulation

The surgical restoration of a joint surface following a comminuted articular fracture is a complex skill that poses technical challenges for the surgeon and potential limb-threatening risks for the patient. Yet, the first time a surgeon faces these distinct challenges will most likely be in the operating room. Precise anatomical restoration of the joint is critical to avoid debilitating post-traumatic arthritis. Treatment risks and complications, including wound breakdown and infection, osteomyelitis, non-union, and hardware failure, can lead to poor patient outcomes, even amputation. Limited percutaneous surgical approaches utilizing fluoroscopy have decreased the complication risk for the patient, but present even greater challenges to the surgeon than do traditional open approaches, due to visualization and soft tissue constraints.

Introduction to these advanced psychomotor and visual-spatial skills in a controlled, simulated environment would be advantageous to both the surgeon and patient. It allows for deconstruction of a complex procedure such as articular fracture repair into discrete tasks such as fracture reduction, temporary stabilization, hardware placement, and fluoroscopic assessment. Critical analysis of simulator performance also provides valuable information about which specific tasks young surgeons are struggling with, and a surgical skills training program could be tailored to address these deficiencies.

We have developed a new fracture simulation training program targeted for junior residents that is designed to improve cognitive and technical skills required to reduce an articular fracture. The purpose of this study is to describe the development and content of the fracture simulation training program and to report resident experience.

Articular Fracture Reduction Simulation Training Program

The Simulation Model

The main objective of the study was to develop a comprehensive training and assessment program focused on improving articular reduction skills in a tibial plafond fracture model (Figure 1).¹⁰ The simulation environment utilizes various multi-segment, radio-dense polyurethane foam (bone surrogate) fracture constellations inside a synthetic soft tissue housing. The simulation task is to temporarily stabilize a three-segment fracture with Kirschner wires (Figure 2). Using fluoroscopic guidance, trainees reduce the fragments of the simulated fracture through a limited anterior window in the housing. The hand motions of the participant are tracked with submm accuracy during the simulation using a four-camera optoelectronic Qualisys motion capture system (Qualisys AB, Gothenburg, Sweden).

Figure 1. The image to the left shows a percutaneous articular fracture reduction surgery. The middle image shows the current simulator, in which a cast polyurethane surrogate of a fractured bone is contained inside an anatomic rubber housing. the fiducials attached to the back of the surgeon's hand enable motion capture. the illustration on the right shows the fiducials as the Qualisys camera detects them (blue for the left hand, green for the right), from which hand motion is measured.

Figure 2. These images were taken during a simulation session and demonstrate, on the left, fracture reduction and stabilization with Kwires; the subject is wearing a head-mounted camera, center, use of fluoroscopy. And on the right, temporary stabilization of three-segment tibial plafond fracture model with K-wires (removed from surrogate soft-tissue envelope).

Subject performance is also captured on multiple video channels, including video from a head-mounted camera that allows determination of when and where attention is being focused (Figure 3). The video streams are consolidated into a single composite split-screen video, coupled with audio capture, for later one-on-one feedback by a traumatologist.

Figure 3. Video Capture. Clockwise direction starting from upper left: head mounted camera, top view mounted camera, C-arm fluoroscopy screen view, wide view mounted camera.

Trainees are assessed on time-to-completion and objective quality of the obtained fracture reduction (from post hoc 3D laser scans that quantify the re-apposition inaccuracy of each fragment, again with sub-mm resolution). Economy of subject hand movement, including the number of deliberate actions, and cumulative hand motion, are extracted from the motion capture data. [Analysis of the hand motion data is the subject of ongoing work and those data are not presented in this paper.] Radiation dose and fuoroscopy time are also recorded. An orthopaedic traumatologist scores each performance using a modifed objective structured assessment of technical skill (OSATS) checklist, which was initially described and validated by Faulkner et al.¹¹ In pilot studies with this simulator,¹⁰ we found that senior orthopaedic residents had more deliberate hand motions (less cumulative hand distance, a surrogate for less iatrogenic wound bed trauma) and more accurate fragment reductions than did junior residents.

The broader educational program consisted of two modules, cognitive and motor. The cognitive module was implemented through an online course that included a pretest, general knowledge about plafond fractures and fuoroscopy, and online video performance reviews. The second half of the program focused on acquiring motor skills by direct instruction and dedicated practice on the simulated model with real-time feedback from an orthopaedic traumatologist.

Cognitive Knowledge-Online Instruction

Online instruction was provided by participant enrollment in an online course on the surgical management of articular fractures (Figure 4). The course was implemented in ICON (Iowa Courses Online), the University of Iowa's online course management system, supported by the University's information technology services. ICON incorporates capabilities that support video content and interaction between trainee and instructor, and it makes the content available through a number of portable devices (computer, iPad, iPod, iPhone etc.).

Figure 4. ICON (Iowa Courses Online) Surgical Management of Articular Fractures Homepage.

A requisite pre-test was performed for baseline knowledge assessment regarding plafond fractures and articular fracture management. Participants were encouraged to review/study available online content. Course content (Figure 5) included learning objectives, background of plafond fractures, relevant fracture and surgical anatomy, principles of surgical technique and principles of fluoroscopic evaluation. In addition, other available resources including video links to both expert and novice surgical simulations, and baseline video self-performances were available for independent review.

Figure 5. Surgical Management of Articular Fractures Course Content.

Training (Direct Instruction, Video review, Dedicated Practice)

For the skills training module, two separate two-hour evening sessions were held during which participants received a brief didactic introduction into the objectives of the training program and surgical simulation exercise. Surgical instruments (Figure 6), reduction strategies, and fxation techniques were discussed in a small group setting.

Figure 6. Surgical instruments available for simulated articular fracture reduction exercise.

Video review with each participant was conducted during a one-on-one coaching session, led by an orthopaedic traumatologist who had previously reviewed and documented particular strengths/weaknesses of the performance. These coaching sessions were held in private, and in a non-adversarial manner.

Subjects were allowed to practice with constraints similar to the real simulation exercise, with the one exception being that a different articular fracture surrogate model pattern was used. Fracture reduction, fxation, and fuoroscopic assessment by the subject was directly observed and guided by an orthopaedic trauma surgeon. Questions from the subjects were addressed on a real-time basis allowing participants to receive immediate feedback.

Subjects

Six PGY 1 and six PGY 2 orthopaedic residents at the University of Iowa enrolled in this study. Three participants in each of the PGY1 and PGY2 years were randomized for participation in the comprehensive training program while the other half were not, for a total of six in the intervention group and six in the control group.

Assessment

All 12 PGY 1 and PGY 2 residents performed baseline and fnal surgical simulation exercises separated by a 4-week interval. The participants flled out questionnaires assessing the usefulness of the simulation training program and their overall experience. General comments were elicited as well. These responses were then compared between the two groups.

Resident Experience

The feedback received from the residents about the simulation exercise and the training program was overall very positive (Table 1). All six residents in the intervention group rated each aspect of the training program as “extremely helpful” (Table 2).

Table 1.

Resident feedback

• “This is a fantastic thing for resident education. Let's do more.”

• “One-on-one training and encouragement made this project extremely useful.”

• “I definitely feel these simulation labs are beneficial for the future in the OR.”

Table 2.

Questionnaire responses – Simulation training program

	Median score (range)
How helpful was the intervention overall?	1 (1-2)
How useful was the ICON content?	1.5 (1-2)
How useful was the video analysis?	1.5 (1-3)
How useful was the hands-on skill session?	1 (1-2)

Scale – 1(extremely helpful) to 4(not helpful)

When asked to compare their overall experience between the two sessions on a five-point scale, the control group responded that the second session was the same in diffculty (median score 3/5) compared to the frst session, while the intervention group thought that it was easier (median score 1.5/5) (Table 3). Subjectively, the control group thought that they performed the same or slightly worse (3.5/5) during the second session, while the intervention group believed that they were much better (1/5). The overall median scores combined were 2/5 for both questions.

Table 3.

Questionnaire responses – Overall experience

	Median score (range)
	Control	Intervention	Overall
Compared to the first simulation session, how much easier was the second session?	3 (2-4)	1.5 (1-2)	2 (1-4)
How much better do you think you did the second time compared to the first time?	3.5 (2-4)	1 (1-2)	2 (1-4)

Scale – 1(much easier/much better) to 5(much harder/much worse)

Discussion

To our knowledge, there are no current validated methods to assess surgical skill competency in orthopaedic trauma. Mabrey et al.¹² in a 2010 review of virtual reality simulators in orthopaedic surgery identifed 16 described simulators. Further review revealed that nine papers involved knee arthroscopy simulators (1995–2006), four involved shoulder simulators (1999–2008), and only three involved the management of fractures (2007–2008.) This stands in contrast to the 246 citations identifed for laparoscopic virtual reality simulation. In the area of fracture surgery, Blyth and coworkers have developed an entirely PC based virtual reality training system for basic hip fracture fxation, including a surgical simulator and an assessment component.^13,14 Results from the recent report by Froelich et al.¹⁵ suggest that a computer-based simulator with haptic feedback could identify measureable differences in surgical profciency between junior and senior orthopaedic surgery residents and may play an expanding role in resident education.

In this paper, we describe in detail the content of a comprehensive articular fracture reduction simulationtraining program in PGY 1 and PGY 2 orthopaedic surgery residents. The simulation training program was felt to be extremely useful to the participants. We received great enthusiasm and positive feedback from participants. When the participants were asked to refect on their performance between the two sessions, the intervention group believed that they performed much better during the second exercise while the control group thought they did the same or slightly worse. Our work strongly encourages the need to develop and implement surgical simulation programs in orthopaedics.

This simulation-training program was comprised of both a cognitive and skills module. The cognitive module was a web-based course that provided general background knowledge about plafond fractures and use of fuoroscopy, and technical tips on how to reduce an articular fracture. These core principles were then reinforced in a brief didactic session led by a fellowship-trained traumatologist, and then executed during the guided skills session. One-on-one video analysis was another instructional tool that allowed for direct expert feedback. Each element of the intervention program was rated as extremely helpful.

The training program reported in this paper consisted of only one session of dedicated practice; a longer follow-up study with multiple practice sessions will be needed to detect meaningful objective improvement. A randomized-controlled trial that examined the effect of deliberate simulator practice on the performance of a vascular anastomosis in an in vivo model suggests that residents exposed to an expert-guided tutorial with dedicated practice (10 anastomosis) performed much better than those who received only a tutorial.¹⁶

The number of practice sessions needed, and the timing of these sessions in order to maximize learning potential is difficult to determine. A recent study by Alvand et al.¹⁷ compared a group of medical students exposed to arthroscopy training versus no training to evaluate whether both groups were able to achieve a level of competency as characterized by stabilization of their learning curve after twenty episodes. Their results indicated that there were a group of medical students who could not be trained despite repetition. However, twenty episodes may not have been enough to capture all skill levels. Along these lines, we have not yet established a “pass” or “fail” score for our simulation model, but it would be interesting to investigate whether there is a cohort of residents who are unable to demonstrate a certain level of proficiency in the articular fracture simulation model, and whether this correlates to in vivo surgical skills.

A final remark is that in addition to expert-guided dedicated practice, we incorporated video feedback as a teaching tool. There are mixed reviews regarding video feedback as an effective teaching tool in orthopaedic surgery,¹⁸ although it has shown to be effective in other disciplines such as sports.¹⁹ It was rated highly by the residents in our study, but further work needs to be done to better define its role in skills training. Future planned studies include assessment of hand motion as an objective metric of technical skill, optimizing the training program, working towards a fluoroscopy free simulation and assessing the effect of other training aids to optimize performance.

Conclusion

In summary, the articular fracture reduction surgical simulation training program consisting of a cognitive and skills module represents a step forward in the development of a comprehensive orthopaedic surgical skills educational curriculum. This program presents a strong model for future surgical skill training programs, and more studies are needed to establish its reproducibility on a nationwide-level.