Abstract
Purpose
The American Society for Surgery of the Hand developed the Surgery Training and Educational Platform (STEP) in order to assess essential skills in hand surgery. The American Society for Surgery of the Hand designed modules spanning both osseous and soft tissue skills aimed to be cost effective for the purpose of orthopedic surgical education. The STEP curriculum was adapted and implemented at a single military orthopedic residency program.
Methods
The following six modules were implemented: (1) depth of plunge, (2) scaphoid pinning, (3) phalangeal fracture pinning, (4) microsurgery, (5) full-thickness skin graft harvest, and (6) wrist arthroscopy. Both first- (PGY1) and second-year (PGY2) residents participated. Scores were calculated according to the original STEP curriculum criteria and were compared with historic data from the previous year. All residents responded to an evaluation questionnaire following the performance of the tasks.
Results
The PGY2 cohort outperformed PGY1 cohorts across all modules except for the depth of plunge and scaphoid fixation modules. In the phalangeal pinning module, PGY2s did significantly better when compared with pooled PGY1 performance and their own PGY1 performance (P < .05). In the microsurgery module, PGY2s scored better than pooled PGY1s. In the full-thickness skin grafting module, PGY2s outperformed PGY1s (P < .05). On the post-task evaluation, residents unanimously responded that this was a valuable exercise, but the time required to complete all the modules was significant, similar to that of the previous year survey.
Conclusions
The STEP simulation is a cost effective and reliable program to engage residents in hand surgery–related skills. However, adaptations should be encouraged according to institutional resources to provide the most inclusive training platform possible per institutional constraints. The STEP simulation is interpreted by residents as a valuable exercise but requires a significant time commitment that could be a barrier to implementation and regular use.
Type of study/level of evidence
Therapeutic IV.
Key words: Hand surgery, Resident education, Simulation
Orthopedic surgery resident education largely follows an apprenticeship model in which a trainee assists and learns the technical aspects of orthopedics from more practiced surgeons. Although this system has been historically successful, concerns remain pertaining to the time, expense, and patient risk required for the development of trainees.1 Critical to surgical training is the concept of graduated autonomy, in which a trainee is given progressively more responsibility after they have demonstrated competency with certain milestones.2 Graduated autonomy is balanced by supervision in the operating room, and ideal surgical training requires both in order to develop competent orthopedic surgeons. However, cited barriers toward resident autonomy in the operating room include faculty supervisor requirements, pressure for increased faculty productivity, and patient safety concerns.1 Moreover, the COVID-19 pandemic further limited operating room availability and surgical resident case volumes across multiple surgical specialties including orthopedics.3, 4, 5 Additionally, success in residency is often measured by the performance of in-training examinations and board pass rates; however, these examinations only evaluate clinical knowledge in just one of the categories of core competencies that the Accreditation Council for Graduate Medical Education requires for performance evaluation of surgical trainees. To date, there is not a validated, objective platform with which to measure the technical performance of surgical residents.
In order to combat this, interest in simulation models to augment orthopedic surgery residency education has been renewed. Simulation models have been developed for various orthopedic subspecialties, with most training focused on arthroscopy.6 Simulation models have been shown to not only be a useful tool to assess surgeon skills7 but also may lead to improved technical performance in the operating room.8 Unfortunately, many models have not been widely adopted because of cost, lack of validity, and limited data on the knowledge transfer to direct patient care. As a result, the American Society for Surgery of the Hand developed a simulation curriculum taskforce, which developed a cost effective psychomotor training and assessment tool for key hand surgery skills that residents should master before graduation.9 The Surgical Training and Education Platform (STEP) includes eight different modules spanning both general and hand specific surgical skills. A subsequent study further demonstrated its construct validity in determining resident skill thresholds at various educational levels.10
Military surgical residencies face several challenges in comparison to their civilian counterparts, particularly related to case volumes and variety,11 making validated simulation curricula a promising tool to augment surgical training. The purpose of this study was to adapt and implement the STEP curriculum into junior resident education at our military residency program based in a large joint-service (Army and Air Force) tertiary center and evaluate intersubject performance over the course of their junior residency training. First- and second-year residents were evaluated, and the second-year residents’ results were compared with their own historic results from the previous year. We hypothesized that the second-year residency class would improve upon their previously demonstrated performance due to repetition and familiarity with the module in combination with practical application of the module skills during their first year of residency training.
Methods
Post-graduate year (PGY) residents in the first (class of 2027) and second year (class of 2026) of residency from a single military orthopedic surgery residency program participated in the STEP simulator testing. Six of the eight initially described modules were utilized. Modules were scored using a metric-based system based on the Founds of Laparoscopic Surgery with points deducted based on time required and penalties assessed, according to the original STEP curriculum.9 PGY1s were tested at the beginning of the academic year and had 0 to 1 month of hand rotation experience. PGY2s were tested at the end of the academic year, and all had 2 months of hand experience. The PGY2 cohort had previously completed the modules as interns (2026 PGY1s), and their new results were compared with their results as PGY1s and with the current intern class. The total cost of materials was also collected.
The STEP curriculum describes eight modules: flexor tendon repair, lag screw fixation, phalangeal fracture pinning, scaphoid fixation, depth of plunge during bicortical drilling, microsurgery, full-thickness skin graft harvest, and wrist arthroscopy.9
Flexor tendon repair
We elected to not implement this module because of the time of preparation, administration, and execution of this module by the single hand attending involved in the study.
Lag screw fixation of an oblique fracture
We elected to not implement this module as it would require significant use of hardware that was difficult to obtain as a loaner set.
Phalangeal fracture pinning
This task simulated cross-pinning of proximal-third proximal phalanx fractures and distal-third middle phalanx fractures in a sawbones hand model. This was conducted while viewing through a phone camera to simulate fluoroscopy. Points were deducted for repeat cortical perforations and inappropriate distance from the simulated fracture line.
Scaphoid fixation
This module simulated a central-axis scaphoid pin. The original description utilized a sawbones hand model with a mobile carpal row. This was adapted to utilize a model without a mobile carpal row in order to utilize the same sawbones model for phalangeal fracture, scaphoid, and depth of plunge modules. A target exit point was labeled at the distal pole of the scaphoid, and trainees attempted to place a K-wire from a dorsal and proximal start point, aiming for the target exit point. As with the phalangeal pinning module, this task was performed through the simulated “fluoroscopy” while viewing through a phone camera. Points were deducted for volar penetration and increased distance from the targeted point.
Depth of plunge
This module tested residents’ ability to drill for a bicortical screw in a long bone. A saw bone of a distal radius was utilized with a 3.5-mm drill bit, and residents were instructed to make three bicortical drill holes over a foam block. The total time for completion was recorded. The foam block was then cut in order to expose the drill hole, and a depth gauge was used to measure the amount of perforation into the foam block which served as a measure of the amount of plunge with each drill hole (Fig. 1A).
Figure 1.
Implementation of the depth of plunge A, microsurgery B, skin grafting C, and wrist arthroscopy modules D.
Microsurgery
This module simulated microsurgical suturing of a lacerated artery. A microscope was set up with a microsurgical set. An approximately 1-cm laceration was made in the digit of the glove to simulate the lacerated vessel. Residents then repaired the laceration with a 9–0 suture and were graded on time to completion, distance of each pass from the wound edge, and how secure each knot was tied (Fig. 1B).
Skin grafting
The goal of this module was to simulate a full-thickness skin graft harvest. We utilized an unripe mango and drew a diamond template for harvest. Residents were graded on the overall precision and safety of instrument handling, time to completion, and the percentage of remaining fat on the harvested tissue (Fig. 1C). A fellowship-trained hand surgeon then graded the amount of mango pulp remaining on the skin and allotted a “percentage” of remaining fat on the specimen to be used in the grading system.
Wrist arthroscopy
In this module, we constructed a plastic box that housed metal prongs as well as a slot for a card for pattern tracing as well as portal localization. Residents were first instructed to move a metal washer from one prong to the other, followed by two attempts at simulating an accessory portal by pushing a spinal needle through the back of the box as close to a bullseye as possible. Next, they were instructed to trace a triangle pattern with a marker and finally transfer the metal washer back to the original prong. Scoring was based on time of completion, proximity of the spinal needle to the card bullseye, and distance of marker tracing from the triangle (Fig. 1D).
Statistical analysis
The mean score for each module was calculated for each resident cohort (current PGY1 and PGY2s, as well as the intern scores for the current PGY2s). The intern scores for the 2026 and 2027 classes were pooled and compared with class of 2026 PGY1 and PGY2 scores. Minimum and maximum values, as well as standard deviations, were calculated utilizing Excel. Comparisons between groups were calculated using Student t tests, and statistical significance was set at P < .05.
Results
A total of 12 residents participated, six in the current PGY1 cohort and six in the PGY2 cohort. PGY1 level residents on average had 0 to 1 month of hand experience prior to participation. PGY2 residents had roughly 2 months of experience in hand service before participation. The current PGY2 cohort had also participated in the modules the previous year. The total start-up cost was approximately $475 and about $70 per resident, with the majority of the start-up cost being used to procure the surgical instruments for the microsurgery module as well as the sawbones for the scaphoid pinning, phalangeal pinning, and depth of plunge modules (Fig. 2).
Figure 2.
Sample list of start-up materials, materials available at the hospital, and recurring materials needed for implementation of the curriculum.
The results of the simulation modules are summarized in Table 1. For the depth of plunge module, the mean scores were 34.3 ± 25.3 (0–64) and 25.7 ± 27.2 (0–55) out of a total score of 120 for the intern and PGY2 cohorts. For the scaphoid pinning module, the mean scores were 116.5 ± 22.7 (70–147) and 102.3 ± 23 (65–120) for the PGY1 and PGY2 cohorts out of a possible score of 180 points. For the phalangeal pinning module, the PGY1 and PGY2 mean scores were 326.4 ± 238 (0–662) and 658.2 ± 14.7 (632–677) out of a total score of 750. The mean score was 118.1 ± 208.6 (0–688) and 339.5 ± 327.9 (0–727) of 1,800 in the microsurgery module for the first- and second-year cohorts, respectively. For the skin grafting module, the mean score was 13.1 ± 27.7 (0–89) and 31.3 ± 34.4 (0–88) of 230 points. The 2026 PGY1 cohort received a score of 0 as they were unable to complete the module in the given time allotment. In the wrist arthroscopy module, the mean score was 587.2 ± 165.4 (253–759) and 617 ± 67 (533–688) points of an available 1,000 for the first- and second-year cohorts, respectively.
Table 1.
STEP Curriculum Performance Results
| STEP Module | Pooled 2026/2027 PGY1 Mean ± SD (Range) |
2026 PGY1 Mean ± SD (Range) |
2026 PGY2 Mean ± SD (Range) |
|---|---|---|---|
| Depth of plunge (total = 120) | 34.3 ± 25.3 (0–64) | 51.4 ± 16.9 (22–64) | 25.7 ± 27.2 (0–55) |
| Scaphoid fixation (total = 180) | 116.5 ± 22.7 (70–147) | 102.2 ± 21.9 (70–129) | 102.3 ± 23 (65–120) |
| Phalanx fracture pinning (total = 750) | 326.4 ± 238 (0–662) | 327.6 ± 203.8 (0–453) | 658.2 ± 14.7 (632–677) |
| Microsurgery (total = 1800) | 118.1 ± 208.6 (0–688) | 217.2 ± 287.1 (0–688) | 339.5 ± 327.9 (0–727) |
| Skin grafting (total = 230) | 13.1 ± 27.7 (0–89) | 0 | 31.3 ± 34.4 (0–88) |
| Wrist scope (total = 1000) | 587.2 ± 165.4 (253–759) | 613.2 ± 214.5 (253–759) | 617 ± 67 (533–688) |
PGY, post-graduate year; SD, standard deviations.
The results of the comparison across the various cohorts are summarized in Figure 3. The PGY2 cohort outperformed all PGY1 cohorts across all modules except for the depth of plunge and scaphoid fixation modules. In the phalangeal pinning module, PGY2s did significantly better when compared with pooled PGY1 performance (P < .05) and their own PGY1 performance (P < .05). In the microsurgery module, PGY2s scored better than pooled PGY1s, which approached significance (P = .054). In the full-thickness skin grafting module, PGY2s outperformed themselves as PGY1s (P < .05), although there was no statistical significance in pooled PGY1 performance.
Figure 3.
Results of comparison of performance results across study cohorts.
On the post-task evaluation, residents unanimously responded that this was a valuable exercise and perceived that these modules would be useful augments to our current hand curriculum. However, they noted that the time required to complete all the modules was significant and may be a barrier to implementation in our program.
Discussion
The purpose of this study was to share a military orthopedic surgery training program’s experience utilizing the STEP simulation curriculum for hand surgery resident education. Overall, we found that the STEP simulation is a cost effective and reliable program to engage residents in hand surgery–related skills. However, some skills in the STEP curriculum require difficult-to-obtain materials, such as tendon tissue, which may be a hindrance to completing the full curriculum. The STEP simulation is interpreted by residents as a valuable exercise but requires a significant time commitment that could be a barrier to implementation and regular use. Additionally, we did note improvement in scores, particularly in the phalangeal fracture pinning and microsurgical and skin grafting modules, comparing the second-year cohort with their scores as first-year residents, suggesting that these modules may yield improvement in surgical skills and comfort with specialized hand surgery instruments. These results are consistent with previous work that also demonstrated increases in scores as resident experience increased.10
There have been several proposed simulation models for use in orthopedic surgery training; however, they are heterogeneous in terms of their type, outcome measures, and training protocols.6 Despite this, simulation models have been shown to be effective tools for assessing surgeon competence and improving real-world surgical skills. Cannon et al8 performed a comparative study to test the utility of a virtual reality arthroscopy simulator in improved resident arthroscopy skills. They randomized PGY3 residents into either an 11-hour virtual reality training program or no simulation training and found that the residents who completed the virtual reality simulator had significantly better probing skills than their counterparts who did not undergo the simulation training. An et al12 performed a study of 16 participants of a simulation-based arthroscopy course and tested their ability to identify anatomic landmarks before and after the course. They similarly found improved gaze strategies and decreased time to complete the task after completion of the simulation course. Orthopedic simulation models have also been developed for other orthopedic subspecialties including arthroplasty, complex fracture surgery, and spine surgery.13, 14, 15
Developing simulation models for use in hand surgery training is challenging because of the breadth and variety of skills required as well as the associated start-up costs. Farrell et al developed a 3-D printed hand model that could simulate hand fracture surgery, but the cost to produce a single simulator was between $200 and $300, although the authors did note that mass production of these simulators may lower this cost.16 Other models have been developed for ganglion cyst removal,17 and a 3-D printed flexor tendon repair model.18 These proposed simulation models do appear to be efficacious in surgical training, but their costs to implement may be difficult for residencies to consistently and reliably implement. Moreover, these models each focus on one aspect of hand surgery and may not adequately train the breadth of surgical skills needed to be successful in hand surgery. In contrast, the STEP curriculum offers modules that encompass and assess competency in multiple basic orthopedic and hand-surgical skills, such as fracture management, soft tissue procedures, microsurgery, and arthroscopy. Additionally, we found that the start-up cost was roughly $450, but once up and running, the cost per resident was approximately $25, possibly making these modules more realistic to implement in smaller residency programs with fewer financial resources.
Competency with hand-surgical techniques likely also varies by residency program. Hinds et al19 performed a review of the Accreditation Council for Graduate Medical Education case log reports from 2007 to 2013 and found that the number of microsurgical cases performed by residents increased significantly during that time frame. However, in their subanalysis, they found this only true for residents in the upper percentiles of case numbers, and the case volumes did not increase for residents in lower percentiles of case volumes, suggesting that there is substantial variability in resident exposure to microsurgery cases. The STEP modules, which were originally designed as an assessment tool, may offer a standardized training model that programs may adapt to help augment hand surgery education in programs with lower volumes of certain hand cases. However, it is important to recognize that the STEP modules may be more beneficial to junior-level residents, as more senior-level residents who have more surgical skill and experience as scores tend to naturally increase with more years in residency.10
There were several limitations and difficulties with the models that we experienced while implementing the modules. First, the scoring system was largely time-based with deductions made for poor surgical skills or outcomes, which may not be an accurate way of assessing competency in certain surgical procedures. For example, we noted that for the skin grafting module, the final score was largely based on how quickly the resident “completed” the task that was largely subjective. In fact, we noticed that residents who finished quicker, but had a less precise or adequate harvest, tended to score higher than those residents who may have been more meticulous with their harvest but finished the task more slowly. Another limitation of this study relates to modifications from our modules compared with the original STEP study design. For example, for the scaphoid fixation module, we were unable to obtain a sawbones model with a mobile carpal row that may have altered the scoring and performance of the scaphoid fixation module. However, we felt this modification to the module did not significantly alter the intended assessment goal of the module for our residents, and the modified module still allowed for the assessment of resident tactical and proprioceptive skills. Additionally, the time to complete the entire curriculum is significant for both participants and graders, which may limit the frequency these modules may be completed or the feasibility of incorporating them into an already busy rotation schedule. Another limitation is the curriculum does not offer specific guidelines or a progression-based model for improvement in scores. Progression-based competency and performance feedback have been shown to significantly improve the success of simulation models.20 Given these limitations when adapting the STEP curriculum into an orthopedic residency training program, we recommend modifying and including modules specific to the needs of the surgical residency. Moreover, we recommend developing a method of providing feedback and specific guidance for improvement in order to make these modules more efficacious for learners.
Conflicts of Interest
No benefits in any form have been received or will be received related directly to this article.
References
- 1.Dougherty P.J., Cannada L.K., Murray P., Osborn P.M. Progressive autonomy in the era of increased supervision: AOA critical issues. J Bone Joint Surg Am. 2018;100(18) doi: 10.2106/JBJS.17.01515. [DOI] [PubMed] [Google Scholar]
- 2.Franzone J.M., Kennedy B.C., Merritt H., Casey J.T., Austin M.C., Daskivich T.J. Progressive independence in clinical training: perspectives of a national, multispecialty panel of residents and fellows. J Grad Med Educ. 2015;7(4):700–704. doi: 10.4300/JGME-07-04-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ilonzo N., Koleilat I., Prakash V., et al. The effect of COVID-19 on training and case volume of vascular surgery trainees. Vasc Endovascular Surg. 2021;55(5):429–433. doi: 10.1177/1538574420985775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Higginbotham D.O., Zalikha A.K., Stoker S.K., Little B.E. The impact of COVID-19 on the orthopaedic surgery residency experience. Spartan Med Res J. 2021;6(2) doi: 10.51894/001c.25963. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Smood B., Spratt J.R., Mehaffey J.H., et al. COVID-19 and cardiothoracic surgery: effects on training and workforce utilization in a global pandemic. J Card Surg. 2021;36(9):3296–3305. doi: 10.1111/jocs.15773. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Polce E.M., Kunze K.N., Williams B.T., et al. Efficacy and validity of orthopaedic simulators in surgical training: a systematic review and meta-analysis of randomized controlled trials. J Am Acad Orthop Surg. 2020;28(24):1027–1040. doi: 10.5435/JAAOS-D-19-00839. [DOI] [PubMed] [Google Scholar]
- 7.Akhtar K., Sugand K., Sperrin M., Cobb J., Standfield N., Gupte C. Training safer orthopedic surgeons. Construct validation of a virtual-reality simulator for hip fracture surgery. Acta Orthop. 2015;86(5):616–621. doi: 10.3109/17453674.2015.1041083. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Cannon W.D., Garrett W.E., Jr., Hunter R.E., et al. Improving residency training in arthroscopic knee surgery with use of a virtual-reality simulator. A randomized blinded study. J Bone Joint Surg Am. 2014;96(21):1798–1806. doi: 10.2106/JBJS.N.00058. [DOI] [PubMed] [Google Scholar]
- 9.ASSH Surgical Simulation Taskforce. Wright D.J., Uong J. Establishing validity of a comprehensive hand surgical training and educational platform (STEP) J Hand Surg Am. 2020;45(12):1105–1114. doi: 10.1016/j.jhsa.2020.07.019. [DOI] [PubMed] [Google Scholar]
- 10.Olson J.J., Zhang B., Zhu D., et al. Do resident surgical volumes and level of training correlate with improved performance on psychomotor skills tasks: construct validity testing of an ASSH training platform (STEP)? JB JS Open Access. 2021;6(1) doi: 10.2106/JBJS.OA.20.00123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dougherty P.J. CORR® curriculum-orthopaedic education: whither the military residency? Clin Orthop Relat Res. 2023;481(6):1075–1077. doi: 10.1097/CORR.0000000000002676. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.An V.V.G., Mirza Y., Mazomenos E., Vasconcelos F., Stoyanov D., Oussedik S. Arthroscopic simulation using a knee model can be used to train speed and gaze strategies in knee arthroscopy. Knee. 2018;25(6):1214–1221. doi: 10.1016/j.knee.2018.05.019. [DOI] [PubMed] [Google Scholar]
- 13.Sepehri A., von Roth P., Stoffel K., et al. Surgical skills training using simulation for basic and complex hip and knee arthroplasty. Orthop Clin North Am. 2021;52(1):1–13. doi: 10.1016/j.ocl.2020.08.001. [DOI] [PubMed] [Google Scholar]
- 14.Yehyawi T.M., Thomas T.P., Ohrt G.T., et al. A simulation trainer for complex articular fracture surgery. J Bone Joint Surg Am. 2013;95(13) doi: 10.2106/JBJS.L.00554. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wang Z., Shen J. Simulation training in spine surgery. J Am Acad Orthop Surg. 2022;30(9):400–408. doi: 10.5435/JAAOS-D-21-00756. [DOI] [PubMed] [Google Scholar]
- 16.Farrell D.A., Miller T.J., Chambers J.R., Joseph V.A., McClellan W.T. Three-dimensionally-printed hand surgical simulator for resident training. Plast Reconstr Surg. 2020;146(5):1100–1102. doi: 10.1097/PRS.0000000000007025. [DOI] [PubMed] [Google Scholar]
- 17.Rajaratnam V., Gan G., Ahmad A.A., et al. Design, development, and validation of a high-fidelity “ganglion cyst” model for cadaveric hand surgery training. J Hand Microsurg. 2022;14(1):58–63. doi: 10.1055/s-0040-1710169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Papavasiliou T., Nicholas R., Cooper L., et al. Utilisation of a 3D printed ex vivo flexor tendon model to improve surgical training. J Plast Reconstr Aesthet Surg. 2022;75(3):1255–1260. doi: 10.1016/j.bjps.2021.11.027. [DOI] [PubMed] [Google Scholar]
- 19.Hinds R.M., Klifto C.S., Guss M.S., Capo J.T. Microsurgery case volume during orthopedic surgery residency: a 7-year assessment. Hand. 2017;12(6):610–613. doi: 10.1177/1558944716675128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Angelo R.L., Ryu R.K.N., Pedowitz R.A., et al. A proficiency-based progression training curriculum coupled with a model simulator results in the acquisition of a superior arthroscopic bankart skill set. Arthroscopy. 2015;31(10):1854–1871. doi: 10.1016/j.arthro.2015.07.001. [DOI] [PubMed] [Google Scholar]