A Dedicated Simulator Training Curriculum Improves Resident Performance in Surgical Management of Pediatric Supracondylar Humerus Fractures

Abstract

Background:

The primary goal of including simulation in residency training is to improve technical skills while working outside of the operating room. Such simulation-related skill improvements have seldom been measured in the operating room. This is largely because uncontrolled variables, such as injury severity, patient comorbidity, and anatomical variation, can bias evaluation of an operating surgeon's skill. In this study, performance during the wire navigation phase of pediatric supracondylar humerus fracture fixation was quantitatively compared between 2 groups of orthopaedic residents: a standard training group consisting of residents who participated in a single simulator session of wire navigation training and an expanded training group consisting of residents who participated in a dedicated multifaceted wire navigation simulation training curriculum.

Methods:

To evaluate performance in the operating room, the full sequence of fluoroscopic images collected during wire navigation was quantitatively analyzed. Objective performance metrics included number of fluoroscopic images acquired, duration from placement of the first wire to that of the final wire, and wire spread at the level of the fracture. These metrics were measured from 97 pediatric supracondylar humerus fracture pinning surgeries performed by 28 different orthopaedic residents.

Results:

No differences were observed between the groups for wire spread in the final fluoroscopic images (t(94) = 0.75, p = 0.45), an important clinical objective of the surgery. Residents who received the expanded simulator training used significantly fewer fluoroscopic images (mean of 46 vs. 61 images, t(85) = 2.25, p < 0.03) and required less time from first to final wire placement (mean of 11.2 vs. 14.9 minutes, t(83) = 2.53, p = 0.013) than the standard training group. A post hoc review of Accreditation Council for Graduate Medical Education case logs for 24 cases from the standard training group and for 21 cases from the expanded training group indicated that, at the time of surgeries, residents who received expanded training had completed fewer comparable cases than residents in the standard training group (mean of 13 vs. 21, t(42) = 2.40 p = 0.02). Further regression analysis indicated that the expanded simulator training produced an effect comparable with that associated with completing 10.5 similar surgical case experiences.

Conclusions:

This study demonstrates that training on a wire navigation simulator can lead to improved performance in the operating room on a critical skill for all orthopaedic residents. By taking fewer images and less time while maintaining sufficient pin spread, simulator-trained residents were objectively measured to have improved performance in comparison with residents who had not participated in the pediatric elbow simulator curriculum.

Clinical Relevance:

As programs aim to provide safe and effective training for critical orthopaedic skills such as pinning a pediatric elbow, this study demonstrates a simulator curriculum that has demonstrated the transfer of skill from a learning environment to the operating room.

Introduction

“See one, do one, teach one” may no longer be an acceptable approach for surgical training. The traditional apprenticeship model of surgical education exposes patients to inexperienced healthcare practitioners and thereby risks less than optimal care^1,2. Revision rates for some hip surgeries are greater for surgical residents than for more experienced surgeons³. An established “July phenomenon” marking the start of the residency academic year coincides with a higher rate of preventable and potentially preventable complications (p < 0.013)⁴. The involvement of junior residents leads to greater fluoroscopy exposure in supracondylar humerus fixation (p = 0.01)⁵.

Like other skilled tasks, surgical skills generally improve with experience along a learning curve⁶. New skills are developed early and improve with experience. Surgical novices learning new skills generally take longer, move less efficiently, use more fluoroscopy, and may unnecessarily expose patients to increased harm. Supervising physicians aim to ensure patient safety and a technically successful outcome, but increased risk is commonly accepted as an inevitable cost of surgical training.⁶

Orthopaedic residents face many competing demands for their time and attention during training, a period during which they work to become skillful⁷. These demands for time have been compressed in the era of work hour restrictions⁸. At the same time, the expectations for technical competence from the public have increased and the demands for operating time efficiency have limited resident surgical autonomy^7,9,10. As a response to these competing pressures, focused skills training and simulation outside of the operating room is becoming an integral part of how residents gain skills^11-13. The transfer of these skills to the operating room has been demonstrated in general surgery, but in orthopaedics, the demonstration of such transfer validity has primarily been limited to arthroscopy^14-16 and arthroplasty^17,18 training¹⁹. Several simulators in the domain of trauma have shown promise, but there remains a critical gap in demonstrating that simulator training improves operating room performance^6,20-22.

Two increasingly common evaluation tools of resident performance, the Objective Structured Assessment of Technical Skill (OSATS)²³ and Overall Performance (OP) score²⁴, provide feedback on performance in the operating room. These assessments, which feature a series of Likert scale handles in different categories, are most often completed by a supervising surgeon. These evaluative tools emphasize less on the behavior and technique of the resident surgeon than the subjective impressions of how the resident performed while completing the surgical task²⁵. What is lacking in trauma are quantitative analyses of objective measurables. Such measurables in the surgical fixation of fractures include procedure duration, movement of tools and implants, geometric reconstruction of reduced fractures, and final placement of implants. These objective metrics capture different aspects than the OSATS and OP score and are less prone to evaluator bias.

Surgical treatment of pediatric supracondylar humerus fractures by closed reduction and percutaneous pin placement is a desirable target for developing such metrics for the task of wire navigation. We previously developed a wire navigation simulator as part of our Core Requisites for Orthopedic Wire Navigation Skills (CROWNS) curriculum, originally for training freehand wire navigation as would be done in treating an intertrochanteric hip fracture. A new simulator module provides a vehicle for teaching and allowing residents to practice placing a pin for the treatment of pediatric supracondylar humerus fractures²⁶ (Fig. 1). The wire navigation simulator replicates the look and feel of navigating a wire through bone using fluoroscopic guidance. Two differentiating features of the platform are that (1) camera-based tracking of a wire replaces fluoroscopic radiation exposure and (2) a foam bone surrogate replicates the feel of drilling a Kirschner wire through actual bone, which is contained in a synthetic soft-tissue envelope. The simulator does not yet incorporate a reduction step, but it does accurately represent the anatomy and challenges encountered in percutaneously placing a wire²⁷.

Fig. 1.

Fig. 1-A The pediatric supracondylar humerus fracture surgical simulator. Fig. 1-B A resident searches for a start position on the capitellum. Fig. 1-C The resident drives the wire into the simulated elbow. Fig. 1-D Simulated fluoroscopy images are displayed that show progress in pin placement.

The task is challenging for several reasons: the end of the pediatric humerus is small and has a narrow safe starting point and corridor for the wire; adjacent nonosseous structures are critical to extremity function (nerve, blood vessel); and the final position of each of the wires matters for stability of the final fixation construct. The resident surgeon trainee identifies a wire starting point near the capitellum and confirms the angle of the wire in anteroposterior (AP) and lateral fluoroscopic images, adjusting the wire position as needed. Then during the simulation training exercise, participants are asked to place 3 wires from lateral to medial, with the most distal and proximal wire spanning at least one-third of the distance across the fracture line²⁸. Figure 2 represents a fluoroscopic image obtained during placement of a second wire. The medial wire has already been placed. The lateral wire rests on the capitellum as the surgeon considers the trajectory in the AP plane.

Fig. 2.

An anteroposterior fluoroscopic image illustrating the navigation of the second wire to fixate a pediatric supracondylar humerus fracture.

In this study, we sought to determine whether participation in a dedicated multifaceted wire navigation simulation training curriculum would have an impact on objectively measurable indicators of resident surgical performance in the operating room. We reviewed all intraoperative fluoroscopic images obtained from pediatric supracondylar humerus fracture cases with resident participation for (1) one group of residents who participated in a single simulator session of wire navigation training (standard training group) and (2) a second group of residents who participated in a dedicated wire navigation simulation training curriculum (expanded training group).

Methods

This study was completed entirely at a single institution. The study was approved by the Institutional Review Board of the University of Iowa (IRB# 201409755). Inclusion criteria for cases were as follows: fractures were Gartland type 2 or 3; 2 or 3 lateral to medial-based pins were contained; and complete fluoroscopic image sequences were available. Complete fluoroscopic sequences from 97 supracondylar fracture closed reduction and percutaneous pin placement cases were reviewed (Fig. 3). The wire navigation portion of the surgery had been performed by 28 different orthopaedic surgery residents in their second through fifth years of residency over the period from July 2018 to December 2021. The standard training group consisted of 13 residents who participated in 47 cases. The expanded training group consisted of 15 residents who participated in 50 cases. For 45 of the cases, the identity of the resident performing the navigation portion of the surgery was confirmed with the resident and the Accreditation Council for Graduate Medical Education (ACGME) case logs for the resident were available. Within this subset of cases, the standard training group consisted of 11 residents who participated in 24 cases and the expanded training group consisted of 8 residents who participated in 21 cases.

Fig. 3.

Pediatric supracondylar humerus fracture (SCH Fx) cases included or excluded based on specific inclusion/exclusion criteria.

Resident Training Protocols

The standard training protocol was integrated in our resident training program before the development of the dedicated pediatric elbow training module. During their intern year, each resident participated in a boot camp skills month that covered a variety of basic skills. During that month, the residents were all exposed to training on a hip wire navigation simulator that focused on placing a center-center guide wire as if treating an intertrochanteric hip fracture. While this training was intended to provide generic wire navigation skills, it did not focus on specifics of other procedures like placing pins for a supracondylar humerus fracture. Residents also attended regular anatomy lessons and conference meetings discussing relevant cases and techniques.

The expanded simulator training protocol captured residents during their second year of residency before taking their first weekend call. The training began with an 8-minute video describing the epidemiology, classification, surgical indications, and technical goals in surgical management of supracondylar humerus fractures. Participants next completed a series of immersive online exercises in which they identified elbow fracture lines (Fig. 4), placed virtual pins on fluoroscopic images, and selected optimal final pin configurations.

Fig. 4.

After clicking on the medial and lateral aspects of the humerus, feedback informs the resident whether they have correctly identified the fracture line. Properly identifying the fracture line is critical to ensuring sequential pins have appropriate spread at the fracture.

After this training, residents completed a series of exercises on the simulator. The exercises began with residents placing 3 lateral pins without feedback or coaching to establish baseline performance. Residents then repeated the simulation working purposefully with goals to maximize pin spread while balancing procedure duration and limiting the number of fluoroscopic images required. Throughout, residents could request either an AP or lateral image at their discretion. Automated formative feedback was provided at the end of each exercise, including AP pin spread, mid-wire ratio, image count, and total time.

The residents then completed 2 training modules designed to reinforce skills learned on the simulator that focused specifically on wire navigation skills and pin placement (Fig. 5). The first training module consisted of targeting a series of circles overlaid on AP and lateral images to provide a construct with good wire spread. The second training module projected a line from the active pin in a given fluoroscopic image. This projected line indicated future wire position and was found to be helpful for junior residents planning the 3-wire pin construct. Finally, the residents completed a summative assessment without feedback, repeating the earlier pretest exercise in which they placed 3 lateral pins.

Fig. 5.

The left 2 images illustrate the circle (bubble) target exercise. These targets provide guided positions for trainees to place their pins when learning how to build their pin construct. In the images on the right, a projected line shows the path the wire will take if advanced further.

Intraoperative Performance Evaluation

Intraoperative fluoroscopic image sequences obtained during pediatric supracondylar humerus fracture closed reduction and percutaneous pin placement were analyzed. These images included the first through the last acquired during the wire navigation portion of the surgery. The first wire navigation image was the first fluoroscopic image in the sequence after the fracture was reduced that included a surgical wire. The final wire navigation image was the last image in which a wire position had moved relative to the bone.

Objective performance metrics included number of fluoroscopic images, duration of wire navigation, and quality of the final wire placement. Wire navigation duration captures the time between the first and last fluoroscopic images during wire navigation. This information was time stamped in recorded image metadata. The last AP image was used to measure the final wire positions. Custom software was used to locate anatomical and geometric features near the K-wires in each successive fluoroscopic image to enable the measurement of relative spatial relationships key to successful wire placement²⁹. The software automated the process of measuring the wire spread, which was defined as the separation of the outermost 2 wires at the fracture line and expressed as a fraction of the width of the fracture line (to correct for variation in patient age and size).

Prior Surgical Experience

A resident surgeon's prior experience was captured based on ACGME case log data. The following Current Procedural Terminology codes: 24538, 24545, 24546, 24575, 24579, and 27236, were used to identify case experiences logged. An online survey was sent to each resident after a case so the resident could confirm that they were the surgeon who performed the pinning portion of the case. Separate from confirmation of the actual resident involved, we were able to definitively categorize cases into the proper group (standard or expanded simulator training). This was because the pediatric elbow training was implemented in December 2019, so any cases performed before that date were in the standard simulator training group.

Statistical Analysis

The statistical analysis was performed using Minitab software (version 21; Minitab, LLC). Differences between resident groups were assessed with 2-sided, 2-sample t tests with a confidence level of 0.95 and no assumption of equal variance. Equal variance was assessed with Bonnet's test with a confidence level of 0.95 and hypothesized ratio of 1. The model was fit with a general linear model, beginning with simulator training as a categorical variable and wire spread, natural log of images, and natural log of duration variables and their first-order interactions as continuous variables. The duration and number of images variables were transformed with the natural log to follow the tendency of training curves and improve the normalcy of measurement residuals. Then, the least significant model terms were iteratively removed to yield the final model.

Source of Funding

This study was funded in part by the Agency for Healthcare Research and Quality (R18 HS025353) and the American Board of Orthopaedic Surgery.

Results

Table I displays the summary features of each case, including the number of images in the wire navigation portion of the surgery, wire spread, duration of the wire navigation portion of the surgery in minutes, and the number of similar cases in the resident's ACGME surgical log completed before the time of the surgery. Cases not in the log subset do not have values for the logged cases column. These data are also plotted in a graphical form in Figure 6.

Fig. 6.

Duration, number of images, and wire spread for each simulator training cohort.

TABLE I.

Summary Case Features^*

Group 1				Group 2
No. of Images	Wire Spread (%)	Duration (min)	No. of Previous Cases	No. of Images	Wire Spread (%)	Duration (min)	No. of Previous Cases
58	0.38	16	1	41	0.41	10	18
115	0.34	29	5	55	0.62	9	32
34	0.47	8	24	31	0.40	7	17
61	0.38	10	12	23	0.44	7	21
25	0.30	7	25	44	0.36	12	20
144	0.36	26	19	33	0.51	7	22
62	0.43	9	17	73	0.28	17	24
49	0.41	11	13	25	0.43	23	15
57	0.37	22	33	115	0.38	24	1
42	0.47	13	6	50	0.34	8	8
34	0.67	9	27	54	0.36	10	6
35	0.53	12	34	42	0.51	11	23
40	0.51	10	49	29	0.47	11	27
25	0.44	8	46	71	0.58	18	2
34	0.53	18	48	51	0.35	14	4
97	0.50	16	23	55	0.34	18	2
84	0.57	16	0	46	0.49	9	7
34	0.41	9	11	36	0.41	10	17
27	0.64	22	16	26	0.61	9	1
115	0.37	16	18	30	0.41	7	1
127	0.32	33	19	49	0.43	9	2
37	0.49	18	21	68	0.62	13	N/A
91	0.34	34	22	61	0.61	15	N/A
190	0.51	45	22	43	0.31	11	N/A
87	0.49	23	N/A	65	0.54	11	N/A
68	0.38	13	N/A	71	0.5	18	N/A
61	0.4	16	N/A	59	0.49	12	N/A
16	0.31	9	N/A	36	0.34	9	N/A
10	0.29	2	N/A	98	0.32	23	N/A
32	0.22	10	N/A	122	0.66	26	N/A
22	0.43	7	N/A	40	0.48	10	N/A
85	0.6	15	N/A	59	0.68	14	N/A
73	0.71	13	N/A	61	0.57	16	N/A
65	0.3	20	N/A	116	0.58	26	N/A
45	0.49	14	N/A	25	0.43	10	N/A
52	0.35	10	N/A	11	0.38	3	N/A
51	0.5	14	N/A	24	0.41	6	N/A
44	0.57	12	N/A	33	0.32	9	N/A
31	0.53	9	N/A	23	0.35	6	N/A
32	0.53	11	N/A	30	0.2	9	N/A
27	0.46	10	N/A	14	0.39	5	N/A
77	0.55	15	N/A	94	0.26	16	N/A
45	0.48	7	N/A	16	0.53	6	N/A
59	0.42	15	N/A	16	0.33	5	N/A
76	0.44	10	N/A	10	0.33	4	N/A
120	0.45	15	N/A	16	0.33	6	N/A
54	0.44	14	N/A	12	0.32	6	N/A
				24	0.38	4	N/A
				33	0.35	8	N/A
				27	0.46	6	N/A

Mean 61	0.45	15	21	46	0.43	11	13
Std Dev 37	0.10	8	13	28	0.11	6	10

^*With mean and SD below, grouped by trained and untrained cases.

No statistically significant differences were observed between the groups for the final wire spread (t(94) = 0.75, p = 0.45). Overall, residents in the expanded training group used fewer fluoroscopic images (45.7 vs. 60.6 images, t(85) = 2.25, p < 0.03) and required less time (11.24 vs. 14.86 minutes, t(83) = 2.53, p < 0.02) to achieve the result. The variance was not significantly different between the 2 groups for the wire spread (B = 0.923, p = 0.559), the number of images (B = 1.33, p = 0.21), nor the duration (B = 1.38, p = 0.21).

Within the group of residents for whom ACGME log data were available, residents in the expanded training group averaged fewer similar prior comparable cases at the time of the surgeries than did residents in the traditional training group (13 vs. 21, t(42) = 2.40 p = 0.02). A regression model of the natural log of the number of fluoroscopic images was fit to estimate the surgeon's experience on similar cases, with simulator training as a categorical variable (Fig. 7). Table II presents the analysis of variance of this model. Both the number of images and simulator training terms are significant at the p < 0.05 level, and no significant evidence for a lack of fit was found. The adjusted R² of this model is 17.6%. Similar results were obtained with number of images rather than their natural log, but the residuals exhibit larger variance for larger image numbers, which is avoided with the use of natural logs.

Fig. 7.

Scatterplot of the natural log of the number of images in the wire navigation portion of the surgery vs. the number of similar cases logged by the resident performing the wire navigation. The blue circles represent cases in which the resident had participated in the standard simulator training. The red squares represent cases in which the resident had the expanded simulator training. The blue line represents the regression for the standard training cases and the red line the regression of the expanded training cases.

TABLE II.

Analysis of Variance of the Regression Model

Source	df	Adj SS	Adj MS	F Value	P Value
Regression	2	1,485.1	742.5	5.70	0.006
Natural log(# images)	1	688.3	688.3	5.28	0.027
Simulator-trained?	1	1,157.7	1,157.7	8.89	0.005
Error	42	5,471.3	130.3
Lack of fit	36	4,011.3	111.4	0.46	0.933
Pure error	6	1,460.0	243.3
Total	44	6,956.3

The resulting model is as follows:

Residents in the standard training group

Previous surgical experience = 52.6 – 7.76 ln(# images)

Residents in the expanded training group

Previous surgical experience = 42.1 – 7.76 ln(# images)

Discussion

Consistency of performance is essential to ensure that each patient receives the best possible care. This study demonstrated that residents with dedicated simulator training (expanded training group) were able to navigate first to final wires in a shorter amount of time, and they took significantly fewer total images to achieve this goal. This strongly suggests that resident training and practice to proficiency on a pediatric elbow fracture simulator improves objective measures of technical skill as compared with generic wire navigation simulator training alone (standard training group).

Previous work would suggest that residents with more operative experience take significantly fewer fluoroscopic images and spend less time in this portion of the procedure than residents with less experience. This is consistent with the findings from Akhtar et al., who found that novice surgeons used a greater amount of fluoroscopy time than intermediate surgeons (p = 0.001) and experts (p = 0.004) during a simulated dynamic hip screw case³⁰. A study analyzing the supracondylar humerus fracture interventions in July, when residents begin their residency or advance to a new level of responsibility in their residency, found that cases performed with junior residents demonstrated longer fluoroscopy time (p = 0.01)⁵. Another study separating orthopaedic residents by year of training found that first and second-year residents used more simulated fluoroscopic time than third through fifth-year residents (p = 0.03)³¹.

The lack of significant differences in the wire spread that we observed is consistent with a similar study of dynamic hip screw cases in which training did not significantly affect the clinically relevant metric of tip-apex distance¹⁹. We posit that the attending surgeon will demand a clinically acceptable pin construct. Therefore, one should not measure skill from the achievement of the pin spread alone.

The regression model suggests that as residents gain more experience, they use fewer images. They likely also use less time because these 2 components are typically correlated. In this study, the adjusted R² of the correlation between the duration and number of images was 0.66. To summarize present findings, a resident's using a larger number of fluoroscopic images during the navigation phase of this surgery correlated with lower levels of experience. Roughly every 7.5 cases of experience correspond to a reduction of approximately 63% in the number of images used. For this correlation, the expanded simulator training was found to be equivalent to a resident's experience gained while completing 10.5 additional cases.

This study had several limitations. This trial was conducted at a single institution and was not formally blinded. The attention to the procedure may have introduced a Hawthorne effect creating a bias for some of the earlier cases from the standard simulator training group. A more complete design would include formal blinding and multiple institutions, a study which would be justified by the results presented here.

Conclusion

This study demonstrates that dedicated simulator training can shorten the duration of wire navigation and reduce the number of fluoroscopic images required for the wire navigation portion of pediatric supracondylar fracture closed reduction and percutaneous pin placement. No differences were observed in the final wire spread at the fracture line after dedicated simulator training. Furthermore, these findings suggest that the number of images required to perform the wire navigation portion of the surgery decreases with resident experience, including the use of dedicated simulation. Regression analysis would indicate that through training provided in the CROWNS module and dedicated wire navigation simulator, a resident can improve their performance the equivalent of participating in roughly 10.5 comparable surgical cases.