A Novel Operative Coaching Program for General Surgery Chief Residents Improves Operative Efficiency

Abstract

Introduction:

We evaluated the effect of an operative coaching (OC) model on general surgery chief residents’ operative efficiency (OE) measured by operative times. We hypothesized that higher levels of entrustment surgeons intend to offer resident in future similar cases are associated with improved OE.

Materials and Methods:

From July 2018 to June 2019, we used a validated instrument to score prospective resident entrustment in 228 evaluations of six chief residents during 12 OC sessions each (3 lap colectomy, 3 lap cholecystectomy, 3 ventral hernia, 3 inguinal hernia). Operative times of matched case CPT codes performed by coached chiefs (N=500) were matched via CPT code to the cases of uncoached chiefs in the academic year 2016–17 (N=478). Statistical analysis was performed using Pearson correlation and one-way ANOVA.

Results:

Prospective entrustment scores from coached chief residents were associated with significantly shorter operative times in matched complex cases (CC) (r=−0.58, p=0.0047). A similar trend was observed in non-complex cases (NCC) (r=−0.29, p=0.18). Compared to the historical cohort, coached chief residents showed a decrease in mean operative time during complex cases (p=0.0008, d=0.44), but an increase in mean operative times for non-complex cases (p<0.0001, d=0.33).

Conclusions:

An OC model improves chief residents’ prospective entrustment leading to increased OE in cases with greater levels of operative complexity, showing a decrease in mean operative time compared to uncoached residents in certain procedures. This is the first report showing formal coaching may be a method to enhance chief resident OE.

Introduction

The operating room (OR) is a critical training setting for surgery residents to learn clinical knowledge and procedure-specific skills from attending surgeons. Although the OR provides the resident and attending surgeon with uninterrupted time together, training in the OR is less purposeful¹ and may be impeded by a lack of analytic reflection on performance² that is critical to connect learning and practice.³ A resident learns to be an independent surgeon in the OR, and prospective resident entrustment (i.e. entrustment surgeons intend to offer the resident in future similar cases) is a determinant of their future practice autonomy in the OR.^4–5 However, disparities between attending surgeons’ and residents’ perceptions of entrustment and autonomy^6–7 might undermine resident OR learning efficiency, which in turn would hinder residents’ readiness for independent practice upon graduation.⁸ To address these challenges in surgical training, many programs provide various types of coaching for residents and/or attending surgeons, such as video-based coaching⁹ and peer coaching.^10–11 In 2018, we developed a new operative coaching (OC) program focused on general surgery chief residents. We assimilated guidelines of coaching in medical education¹² and a theoretical framework of spaced learning.^13–14 The goal of our OC program was to enhance chief residents’ readiness for independent practice during their final year of residency training.

Numerous studies have been conducted to examine the impact of coaching on resident performance improvement.^15–16 However, most of them used surgical skill measures and/or patient safety metrics; few studies have used surgical process performance metrics, such as operative time. In this study, we aimed to evaluate the effect of our new OC model on general surgery chief residents (GSCR) operative efficiency measured by operative times. We hypothesized that several OC sessions occurring throughout the academic year would have a positive impact on GSCR overall operative performance and practice readiness as assessed by prospective resident entrustment scores. We expected higher levels of prospective resident entrustment would contribute to improved operative efficiency, which could be measured by comparing operative times of the coached chief residents with the operative times in similar cases performed by a historical chief resident cohort.

Materials and Methods

Setting and Participants

The Ohio State University College of Medicine General Surgery residency is a university-based, Accreditation Council for Graduate Medical Education (ACGME) accredited training program. In Academic Year 2018 – 2019, our program had 6 categorical chief residents and 56 teaching faculty practicing in four teaching hospitals. The Ohio State University Institutional Review Board (IRB) approved this study.

Operative Coaching (OC) Model

Our OC model was designed to provide systematic resident-centered coaching for residents based on a resident’s performance demonstrated throughout an observed case. The coach provided objective feedback to help residents identify opportunities for improvement of specific technical and OR management skills, with a suggested feedback structure (Table 1) to strengthen resident self-regulated learning.^17–18 During this study, an experienced academic surgeon who was retired from clinical practice served as the operative coach for residents, providing 12–14 coaching sessions in the OR per resident per year. The coach observed a resident’s surgical performance without providing any teaching interventions or instructions in the OR during the coaching session. After the OC session, the coach, the attending surgeon, and the resident filled out a validated procedure-specific evaluation form¹⁹ (see example in Supplement) within 3 days of the coaching case. Each evaluation form captured resident performance in learning attributes (e.g. learning goal, achieved learning goal), procedure-specific attributes, and transferrable/general attributes (e.g. general skills, team management, and operative plan), and included an opportunity to provide open-ended feedback. The coach provided a debrief session either in-person or via phone for the observed resident with synthesized information of the three evaluations completed by the coach, attending surgeon, and resident. The debrief session was scheduled at the convenience of both coach and resident and on average lasted 15 minutes.

Table 1.

A Recommended Structure of Coach Written Feedback

The OR coaching written feedback includes three parts (with examples):

Part I - Room setup, Time-out, and Learning goal: What was doing well and/or what was missing?

Example: “Positioning of patient, monitors, and lights as well as timeout were excellent. You achieved your learning goal set for this case.”

Part II – Specific Operative Skills: What was good/bad and the reason why? If time allows, providing (good behavior) reinforcement, self-directed learning prompting questions, or actionable tips to improve resident behavior. You may focus on resident’s learning goal if it were about certain procedure step.

Example: “During the gallbladder dissection you did well. You had some difficulty identifying the plane and attending gave guidance. Extraction was very good. During the operation, there was some bleeding. This seemed to bother you, as it should, but slowed your pace. However, with more experience you will learn that some bleeding will occur and sometime direct intervention is not needed. I suggest visualizing a lap chole with bleeding that needs control. What would you do? Pressure and grasp bleeding point are both acceptable while blind application of electrocautery is not acceptable.”

Part III – General Skills: Instrument handling, respect tissue, operative flow/case progress, and team management.

Example: “You conducted yourself professionally as always and managed the team well. Your stance was excellent and you demonstrated excellent dexterity with both hands. You do better when you do not rush. Speed in the OR is all related to efficient motion defined as not repeating motions.”

Data Collection and Analysis

We provided formal OC sessions for chief residents from July 2018 to June 2019. Six GSCR received at least 12 OC sessions each (3 laparoscopic colectomy, 3 laparoscopic cholecystectomy, 3 ventral hernia, 3 inguinal hernia) during the final year of training. We also collected prospective resident entrustment scores estimated by the coach, attending surgeons, and residents in 228 evaluations using a 5-point Likert scale format ranging from 1 (low) to 5 (high).²⁰

Diagnosis-related group (DRG) information and operative times of matched case CPT codes with those procedures (Appendix 1) performed by OC GSCR during 2018–2019 (N=500) were extracted from hospital billing as well as those of uncoached GSCR in 2016–2017 (N=478). We excluded cases involving multiple procedures, prisoners and robotic devices, as well as cases performed by surgeons who had been in our department less than one year. DRG information was used to classify cases as complex (DRGs with complication or comorbidity, or major complication or comorbidity) or non-complex (DRGs without complication or comorbidity).

We performed Pearson correlation to examine the relationships between prospective resident entrustment and operative time and one-way ANOVA to investigate differences in operative time between coached and uncoached GSCR cohorts. All statistical analyses were performed using JMP Pro (version 13; SAS Institute Inc, Cary, NC). A p-value less than 0.05 was statistically significant.

Results

In total, 76 OC sessions were provided for six GSCR (range of 12–14 sessions per GSCR). On average, each OC session lasted 161.5 (SD±72.2) minutes. Fifty percent of GSCR (3/6) received 12 OC sessions during 07/2018 – 06/2019, 33.3% GSCR (2/6) had 13 OC sessions, and 16.7% (1/6) had 14 OC sessions.

As seen in Figure 1, OC GSCR increased prospective entrustment scores were associated with significantly shorter operative time in complex cases (CC) (r=−0.58, p=0.0047). A similar trend was observed in non-complex cases (NCC) (r=−0.29, p=0.18).

Figure 1.

Correlations between Prospective Entrustment and Operative Time of Coached Cases. CC cases: r=−0.58, p=0.0047; NCC cases: r=−0.29, p=0.18.

Table 2 shows the comparison of operative time with matched CPT codes between the OC GSCR cohort (N=500) and the historical uncoached GSCR cohort (N=478). Compared to historical uncoached GSCR cohort, coached GSCR’s mean operative time of CC decreased from 204.39 minutes to 174.38 (p=0.0008) with a medium effect size (d=0.44). However, coached GSCRs’ mean operative time of NCC cases showed an increase from 126.23 minutes to 145.96 (p<0.0001, d=0.33).

Table 2 –

Operative Time Comparison Between Coached and Uncoached cohorts

Cohort*	2016–2017 Uncoached	2018–2019 Coached	P value	Effect size (Cohen’s d)
Total Case Number (N)	478	500
Overall Average Time Length of CC Minute (95% CI)†	204.39 (191.48–217.31)	174.38 (162.61–186.16)	p=0.0008	d=0.44
Overall Average Time Length of NCC Minute (95% CI)	126.23 (120.19–132.27)	145.96 (139.93–152.00)	p<0.0001	d=0.33

Note:

^*2017–2018 cohort was excluded due to a preliminary pilot test of coaching between April – June 2018;

^†CI – 95% Confidence Interval. CC = Complex cases; NCC = Non-complex cases.

Discussion

We developed a novel operative coaching model and implemented it for chief residents in a general surgery residency program. The results of this study suggest that our OC program has a positive impact on chief residents’ training performance in the OR, leading to improved operative efficiency (Figure 2).

Figure 2.

Conceptual Relationship between Operative Coaching and Operative Efficiency

Entrustment is associated with residents’ operative skill development and practice readiness in the workplace.²¹ Our study results suggest this association is also manifested specifically for OR training during the chief year. Increased prospective resident entrustment was correlated with higher efficiency as measured by operative time. One possible reason might be the operative trust an attending surgeon intends to offer a resident in future similar cases directs the surgeon’s OR teaching and guiding plan²² and drives their intra-operative behaviors.²³ In this study, operative coaching likely accelerated chief residents’ growth of operative skills and their practice readiness, which in turn lead to more prospective entrustment.

Coaching is a classic approach used to improve resident and/or surgical faculty performance development as well as to identify strategies for success and foster insight and lifelong learning skills.^24–25 Our coaching sessions were spaced in time throughout the academic year to enhance recall and maximize opportunities for application. Furthermore, residents are expected to demonstrate progress in interpreting feedback.²⁶ To minimize possible interruption of OR case progress and to improve residents’ time utilization, we engaged the three coaching participants (coach, attending surgeon, and resident) in the feedback process^17–18 using a structured performance evaluation. Based on the literature, feedback from workplace-based assessments (e.g. resident operative performance assessment), particularly multisource feedback, results in a positive effect on postgraduate medical trainees’ learning.²⁷

We hypothesized that our OC model would increase training productivity in the OR and result in shorter OR times in similar cases. Our results demonstrate that when comparing OC GSCR to the control group on matched cases the mean operative time in complex cases (CC) decreased on average 30 minutes. Interestingly, when comparing the OC GSCR cohort to the historical uncoached GSCR cohort, the mean operative time of non-complex (NCC) cases performed by coached GSCR cohort was almost 20 minutes longer than that of the uncoached cohort. One possible explanation for this phenomenon might be that the attending surgeon is allowing the chief resident more autonomy in these cases.²⁸ For example, attending surgeons might give a chief resident more time to safely struggle or teach the junior resident or medical student in the OR^29–30 given that the attending surgeon tended to trust the chief resident and (the surgeon) himself/herself was able to control any situations in that non-complex case to ensure patient safety. To improve operative efficiency of NCC cases, faculty development is recommended to enhance attending surgeons’ OR teaching skills and strategies (e.g. setting a limit of slow-down frequency and duration).²⁹

Our study has some limitations. First, it was a single institutional study and only focused on general surgery chief residents. Second, it did not have a large sample size. Third, the observable presence of a coach means that study participants may suffer from components of the Hawthorne Effect. Fourth, no pre-coaching entrustment benchmark existed for comparison. However, our study still provides helpful insights for residency programs regarding use of an operative coaching intervention to improve resident operative efficiency.

Conclusions

A prospective operative coaching model increases general surgery chief residents’ prospective entrustment, leading to improved operative efficiency in cases with greater levels of operative complexity, showing a decrease in mean operative time compared to uncoached residents in laparoscopic colectomy, laparoscopic cholecystectomy, ventral hernia, and inguinal hernia procedures. This is the first report showing formal operative coaching may be a method to enhance general surgery chief residents’ training and operative efficiency.

Highlights.

• Increased prospective resident entrustment is associated with decreased operative time.

• A new operative coaching model enhanced general surgery chief residents’ prospective entrustment, leading to improved operative efficiency in cases with greater levels of operative complexity.

• Compared to uncoached resident cohort, mean operative time in laparoscopic colectomy, laparoscopic cholecystectomy, ventral hernia, and inguinal hernia procedures decreased.

Appendix 1. Coached Procedure CPT Codes

Coached Procedures	CPT Codes
Laparoscopic Colectomy	44204, 44207
Laparoscopic Cholecystectomy	47562, 47563
Ventral Hernia	49560, 49652
Inguinal Hernia	49505, 49650