Post Game Analysis: Using Video-Based Coaching for Continuous Professional Development

Abstract

Background

The surgical learning curve persists for years after training, yet existing CME efforts targeting this are limited. We describe a pilot study of a scalable video-based intervention, providing individualized feedback on intra-operative performance.

Study Design

Four complex operations performed by surgeons of varying experience – a chief resident accompanied by the operating senior surgeon, a surgeon with <10 years in practice, another with 20–30 years, and a surgeon with >30 years of experience – were video-recorded. Video playback formed the basis of 1-hour coaching sessions with a peer-judged surgical expert. These sessions were audio-recorded, transcribed, and thematically coded.

Results

The sessions focused on operative technique, both technical aspects and decision-making. With increasing seniority, more discussion was devoted to the optimization of teaching and facilitation of the resident’s technical performance. Coaching sessions with senior surgeons were peer-to-peer interactions, with each discussing his preferred approach. The coach alternated between directing the session (asking probing questions) and responding to specific questions brought by the surgeons, depending on learning style. At all experience levels, video review proved valuable in identifying episodes of failure-to-progress and troubleshooting alternative approaches. All agreed this tool is a powerful one. Inclusion of trainees seems most appropriate when coaching senior surgeons; it may restrict the dialogue of more junior attendings.

Conclusions

Video-based coaching is an educational modality that targets intra-operative judgment, technique, and teaching. Surgeons of all levels found it highly instructive. This may provide a practical, much needed approach for continuous professional development.

Introduction

Although central to a surgeon’s profession, operative skill remains a poorly defined construct. Despite the fact that the majority of adverse surgical events are attributable to technical error (1–2), no evidence-based method exists to help surgeons evaluate – much less improve – their intra-operative performance. Indeed, traditionally, operative privileges have been granted based upon proxies for skill such as completion of residency, board certification, personal recommendations, and arbitrarily defined case volumes rather than any concrete measure of technical competence. The inadequacy of these metrics in capturing technical skill, whether formatively or summatively, has been described both within and outside the discipline of surgery (3–7).

Research has demonstrated an inverse relationship between surgeon volume and surgical mortality (8), as well as operative experience and surgical complication rates (9–11); however, no one has definitively ascertained the minimum number of cases or years of experience needed to safely perform any particular operation. Furthermore, the highly individualized nature of learning curve (12–13) renders the possibility of finding a universally applicable threshold – whether expressed in caseload or time commitment – remote at best. In their study of the effect of practice experience on operative times and complication rates in reduction mammaplasty, Carty et al (11) demonstrated considerable variance in performance, particularly among junior surgeons. Additionally, their data suggest that the surgical learning curve for even this “bread-and-butter” case persists for more than a decade beyond formal training. The length of time required to achieve expertise is an issue of increasing relevance in our current training environment – an era in which the time that residents spend in the operating room (OR) is reduced (14), secondary to work hour regulations, compounded by constant procedural and technological innovation (4, 6).

Thus, the need for a means of targeting individual technical skill and decision making is critical. Despite repeated calls by the American College of Surgeons (7) and the American Surgical Association (3) for technical evaluation and retooling, no effective system exists for surgeons to accomplish this task. Indeed, while the adult learning literature condemns traditional continuing medical education (15), most educational interventions for surgeons incorporate its ineffective principles – teacher rather than learner-driven, didactic rather than interactive, and amassed rather than distributed. The new paradigm, continuous professional development, aims to correct these inadequacies (16), but is largely undeveloped in the operative domain.

Physician self-assessments are known to be unreliable (17). The use of a third party to provide perspective and immediate, targeted feedback is an intuitive approach that is well described in accelerated skill acquisition in other disciplines (18). Athletes are a particularly apt analogy; even the elite among them retain coaches. The world of sports provides a unique methodology as well: performances are routinely recorded and reviewed post-hoc with coaches who guide the learner throughout a longitudinal, self-directed improvement process. Thus, we sought to develop a methodology for continuous professional development in operative skill using video-based coaching. We report the results of a qualitative study designed to assess feasibility, gather feedback, and identify recurring themes of discussion that may be further refined into a scalable intervention.

Methods

Over the past two years, we piloted and refined the use of audiovisual recording technology in the operating rooms at our institution. Our configuration allows us to record a view of the operative field in detail, a view of the entire operating room (OR), and all conversations in synchrony. This study was approved by the Partners Human Research Committee (IRB).

We identified general surgery and surgical oncology operations with expected complication rates of >20% by review of the pre-admission testing center and OR schedules. We appointed a highly respected, recently retired surgical oncologist with extensive experience in all cases undergoing review as our surgical coach. He is widely recognized within our institution as an expert in operative management and is the surgeon to whom others most frequently turn for informal consultation both in and outside of the OR.

Five operative surgeons were offered an opportunity to engage in a single video review with our coach, with whom each is familiar; four invitations were accepted. These cases represented a wide spectrum of surgical experience (Table 1): a chief resident accompanied by the operating senior surgeon, a surgeon with <10 years in practice (junior surgeon), another with 20–30 years (senior surgeon), and a surgeon with >30 years of experience (very senior surgeon).

Table 1a.

Participating Surgeons

Surgeon	Experience
Resident/Senior Surgeon	PGY-7/20–30 years in practice
Junior Surgeon	<10 years in practice
Senior Surgeon	20–30 years in practice
Very Senior Surgeon	>30 years in practice, departmental and national leader

Video review sessions were scheduled in 1 hour blocks and were moderated by the principal investigator (CCG) and an educational psychologist (SEP). The moderators introduced each session by stating the overarching goal of the project: to review their operative performance using the coach as a sounding board. It was explicitly stated that the goals of these sessions were to improve performance (formative evaluation) and not to develop an approach to formal evaluation for privileging or certification (summative assessment). Each participant was told to “talk through what happened” in his case, prioritizing topics of conversation as he saw fit. They were instructed to request fast forwarding as needed. The moderators also indicated that they would inquire about fast forwarding if the conversation seemed to stall. Surgeons were shown a video of the operative field as recorded by the in-light camera. As feedback accumulated throughout the roll-out of the project, we made preoperative imaging and the view from the room camera available during the review sessions; audio was not utilized for the purposes of this study. The review sessions were audio-recorded, transcribed, and coded, using grounded theory analysis, for recurrent themes pertaining to coaching techniques, the content of the conversations, the educational value of the session, and ideas for improvement.

The study was closed when thematic saturation was reached; when the discussion topics of the sessions started to recapitulate one another and new ones ceased to emerge, we stopped offering reviews with our coach. Surgeons were still afforded the ability to review videos on their own; none, however, took advantage of this opportunity. The final number of cases, 4, represents 21 hours of operative time and is an expected and reasonable sample size for qualitative research.

Results

Coaching Techniques

The surgeon-coach discussions proceeded naturally and required little prompting on the part of the moderators. In every session, the conversation was driven alternately by the surgeon and the coach; while all of our surgeons were active learners, able to direct the progression of their own coaching sessions, the coach also readily identified technical and/or decision-making points that were novel to each surgeon. The frequency with which each coaching technique appeared is shown in Table 2. Nearly three times as many instances were initiated by the coach as the operative surgeon.

Table 2.

Frequency of Use of Different Coaching Techniques

Technique Employed	Resident/Senior Surgeon	Junior Surgeon	Senior Surgeon	Very Senior Surgeon	Total
SURGEON-DRIVEN					23
Asking pointed questions	-	5	-	-	5
Narrating video	7	2	1	8	18
COACH-DRIVEN					63
Asking questions to prompt reflection	8	4	1	8	21
Suggesting alternative approaches	9 (6/3)	13	4	5	31
Framing in terms of resident performance	2	2	5	2	11

Note: Numbers represent instances, e.g. 5 instances of “asking pointed questions” was noted in the junior surgeon’s coaching session.

Surgeon-Driven

Surgeon-driven learning occurred in two ways. The first was best exemplified in the session of the junior surgeon, who arrived with specific questions in mind. Indeed, this surgeon had the option of choosing either of two recorded cases, and he selected the more technically challenging one to review. Explaining that there was “a preventable intraop event,” his goal for the session was to “go over anything we could have done differently.” Throughout his coaching session, he explicitly queried the coach for advice about a range of topics, from positioning of the patient and assistants to dissection techniques. For example, when discussing placement of the incision:

Surgeon: What was your approach for (these cases), in general?

Coach: Depended on whether they were….on one side or the other, then I tended to roll up the patient up and do it through some transverse kind of thing.…I think it largely depends on whether you think that…you might need to go off to the side up into the chest and down the midline…I don’t think I ever worried much about which incision was better; it was just which one could you see better through…and what would be the biggest problem.

All surgeons used an explanatory technique to engage the coach at some point during their session; they fast-forwarded to points of interest and narrated the events being replayed. This technique was most frequently used by the chief resident, the chief resident’s attending surgeon, and the very senior surgeon, with 7–8 instances appearing in each case. However, its intent seemed to differ with seniority. The attending surgeons generally provided a rationale for their on-screen actions and/or for choosing a particular moment to review:

Surgeon: What I do is dissect or transect distally then do the left gastric, then transect proximally…meticulously doing the dissection. It’s interesting to me that when, and I do the same with an esophagogastrectomy for a GE junction lesion, it’s interesting to me that when the thoracic guys do this, they essentially just take a white load across the left gastric. Now does that make a difference? I don’t know…I do spend a little bit of time doing that. The other reason I do that, it also gives me something to do while the pathologists are telling me about the margins.

In contrast, the chief resident’s accounts of various clips were less specific, perhaps intentionally so, to allow the coach to interject as opportunities for teaching points arose:

Surgeon: So this is where we’re coming…underneath the (organ).

In either scenario, the coach was able to take advantage of the narrative cue and move the discussion forward. In response to one surgeon’s explanation of his movements, the coach provided cautionary words:

Surgeon: I always keep my left hand on the mesocolon so that you’re separating the…mesocolon off of the omentum or the adhesions in the lesser sac.

Coach: I buggered a middle colic one time…doing exactly that maneuver of trying to get the stomach up off of the transverse mesocolon.

The chief resident’s introduction of his dissection of the porta, while less self-analytic, was nevertheless met with a teaching point:

Surgeon: Here’s where I start things out. I was trying to find the tip (of the clamp) and I think I was catching the tissue in the tip.

Coach: Writers (from a well-known institution) make a big point out of dividing the common bile duct and I think…it’s a help…I think the artery is, as well.

Coach-Driven

Depending on the individual surgeon’s level of responsiveness to each, the coach switched between several different techniques of prompting discussion. In the first, reflection was triggered with a direct question. As illustrated in the following example, the coach’s question activated an introspection that gradually led the surgeon to realize that he might benefit from changes to his routine practice:

Coach: Why do you stand on the left there?

Surgeon: Pretty much I’m always on the left because when I was a resident, the operative surgeon was on the right, and when you graduated to being an operative surgeon, you got to stand on the right, so I always felt like, for the resident, if they are on the left, they felt like they are the assistant…Well, actually you’re right. They should be on the left when you are going into the pelvis. And I’ve got to admit, I’ve stood on the left so much that I was just more comfortable on the left, and then, when I first started working as an attending, dissecting the pelvis, I had more control if I was on the left.

At times, the same technique required more clarification of the teaching point; the surgeon had to be guided towards a particular thought process:

Coach: So you’re above the duodenum taking down the porta? So you’re doing that before dealing with the gallbladder?

Surgeon: We have not dealt with the gallbladder at this point.

Coach: Why not?

Surgeon: We skeletonized the portal structures, identified our vasculature, the GDA – we didn’t have imaging, so I think we looked for, but did not find a replaced right. After we kind of skeletonized that all out, then we took the gallbladder down and transected the common duct.

Coach: You can usually tell a replaced right hepatic by the location with respect to the common bile duct.

This prodding did ultimately result in surgeon self-contemplation. Afterwards, the operative attending admitted, “I usually make the tunnel before dividing anything. Probably in this case, I didn’t need to do that. We could have taken down the gallbladder.”

The coach’s second technique utilized explicit suggestions, and was employed most frequently with the chief resident and the junior attending. Noting particular moments on the video, the coach offered alternative approaches. For example, regarding the chief resident’s incision and retractor placement, he explained:

Coach: The trouble with using…the Richardson-type retractor on the liver is that the angle is such that your hand bumps into it. If you use a malleable, you can fold it back under, and it allows the person working from the patient’s left side to get his hand in without hitting your knuckles against the retractor quite so much.

When the chief resident replied that he remembered having more trouble with “the inferior portion,” the coach agreed and offered additional input:

Coach: Well, it looks like you’re struggling with the inferior portion, too, and that ought to be basically no retraction at all – if you’ve got a big enough incision…It looks to me like you’re struggling there with the incision…You need more to the right because you’re trying to get posterior.

Similarly, he proposed a different incision to the junior attending:

Surgeon: Because the tumor seemed to be coming up behind and around it…we intended to take the kidney.

Coach: Which does raise…the question of whether T-ing off the incision to the right would’ve been helpful to you. That is, starting low, then coming over…If you think that it’s all the way around the kidney, then some exposure toward the back there can be helpful.

The surgeon responded that he had been debating this technique, but had “tended to stay midline.” Using video replay, the coach highlighted the surgeon’s struggle to dissect the kidney to illustrate his reasons for suggesting another incision:

Surgeon: You just don’t have that much girth to deal with, so you can kind of reach down there and easily feel where the kidney is. And so that dissection, sometimes I’ll do it bluntly if I know that, because once you slide that, then you could actually mobilize that out of the retroperitoneum.

Coach: It looks like where you are there, that you are having trouble at the lower end, and…sensing you’re having difficulty would make you move to taking the kidney out…It’s just having the incision back posterior just facilitates that getting the kidney up into the air and that end of it.

When we followed-up with this surgeon, he reported that he had adopted the coach’s approach in a subsequent case and had found it advantageous.

Feedback was occasionally framed in terms of resident performance; references to the residents who were present in the case, but not present for the review session, eased the discomfort of self-evaluation and elicited discussion. As the coach explained it, invoking the resident “makes our interaction less confrontational.” He attempted this technique in every session, whether he was making a specific, corrective teaching point:

Coach: When somebody’s picking their way through it like this, I always insisted they take up a knife to do it. And the reason is then they would have what looks to them like a lot of bleeding. It’s always lots more than they thought they ought to have, but it’s all perfectly minor league stuff…Just as a lesson to what you can get away with.

or generally fostering a sense that he and the surgeon were colleagues with a common learning goal:

Coach: See, I’m not sure what he’s accomplishing with this maneuver. Looks like he’s still got adhesions to the patient’s presumably left side there and can’t see what he’s doing and just feeling, and it’s not going to move things along. He’s going to have to take down these adhesions in order to get at the pelvis.

Reactions to it were variable. Some surgeons were particularly responsive; once the assessments were directed at their resident’s performance, they became noticeably less inhibited, participating more fully in the session and accepting feedback more readily. On occasion, the surgeon expressed frustration with the resident, and the coach, in order to make an effective teaching point, merely had to suggest a solution. In this way, the following quote emphasizes the utility of the session in helping surgeons optimize use of their assistants:

Surgeon: There were at least 4–5 times that I couldn’t see the field because (Resident)’s paws were in the way.

Coach: Well, the solution to that is to change retractors, change from self-retaining to assistant-retaining.

The technique, however, was less successful in other instances. One surgeon felt compelled to defend his resident, and was therefore less likely to benefit from the teaching point or the sense of alliance that the coach was trying to create by evoking him:

Coach: Holding the knife like a pencil. Afraid they might cut something.

Surgeon: Yeah, it’s one of our technically best residents, I think…And this particular resident actually, he can do a lot on his own…For a fourth year, he is quite stellar.

Following this response, the coach switched to validation to preface his teaching points for the remainder of the session. His success is seen in the surgeon’s affirmative response, and reflects both the variability in surgeons’ responses and the coach’s ability to astutely respond to these cues:

Coach: Both you and the resident were very skillful at not wasting time and wasting motions…with pawing over things and hemming and hawing over stuff. What you see a lot is people who don’t have their mind made up about where they’re going to go and what they’re going to do, and they paw at the tissue…You guys clearly had a game plan and went to it. I think this is a wonderful example of that.

Surgeon: What we had talked about was, even before we had gotten started, was we went over the films and said, “Okay, this is what we are going to expose first, and then we are going to work on this part, and this is going to be the hard part.” And then, intra-operatively, we just say, “I think we’ve sort of worn out how much we can do from there right now. Let’s just go to a different area, and we just kept essentially going back and forth to the superior to the inferior extent.”

Coach: It is helpful to remember to move your traction-retraction.

Surgeon: Yeah, that’s another thing – that’s a good point.

Coach: Everybody loses sight of the retractor…as they’re burrowing forth, away.

Conversational Content

Operative Technique

Both technical and decision-making aspects of the cases were addressed in the coaching sessions, and were, to some extent, inseparable. The frequency with which each discussion topic appeared is displayed in Table 3. Considerations ranged from the positioning of the patient:

Table 3.

Frequency of Appearance of Discussion Topics

Discussion Topic	Resident/Senior Surgeon	Junior Surgeon	Senior Surgeon	Very Senior Surgeon	Total
OPERATIVE TECHNIQUE					76
Positioning of patient	-	1	1	-	2
Positioning of assistants	-	1	2	-	3
Positioning of retractors	3	6	2	3	14
Incision	4	4	1	2	11
Exposure	7	15	-	10	32
Failure to progress	3	4	6	1	14
TEACHING RESIDENTS	1	-	6	8	15

Note: Numbers represent instances, e.g. 10 separate discussions occurred about exposure in the very senior surgeon’s session.

Coach: The other thing that I would have done before this maneuver would have been to put him in Trendelenberg…because it just starts getting stuff out of your field.

to the choice and placement of retractors:

Coach: I think is helpful is to constantly be changing the Bookwalter…Using it as if…it were a medical student. You’ve got stuff down to the right that you don’t need, and you need more pull to the left.

Surgeon: So just to relax on the blades?

Coach: Yeah…People don’t seem to think of that. They fight it; they think the Bookwalter’s going to keep them in one place, exposing everything forever…It just doesn’t do that.

There was a heavy focus on the choice of incision:

Coach: The concept of having the patient in position or using an incision that might allow you to use the left chest to get at the esophageal-gastric junction is just something (the residents) simply don’t think of.

Surgeon: If I needed to do that, I’d probably close and flip, as opposed to crossing the costal margin, and so in this case, really he was consented for total or subtotal. And even when I do a total in someone of his size – and actually, he lost enough weight that…I thought the exposure would be okay – even if I’m doing a total, most of the time, I can do that through the abdomen.

and the quality of exposure obtained in the approach:

Coach: You were intraperitoneal, rather than retroperitoneal, on the right…You didn’t think you needed to take down the left colon to…be in the retroperitoneum on the left.

Surgeon: No, because her tumor…was coming out of the right retroperitoneum. So what I wanted to do was…If we just took down the right mesocolon completely, and…chased the right colic vessels close to their origin off of the SMA, then I could essentially get at the tumor from that side without having to come all the way into the ret(roperitoneum) and to the left side. Wait, so you would have come in through the (left)?

Coach: I don’t know…The question is: Is it going to be tethered that way by the inferior mesenteric?

Surgeon: I see.

Coach: That is: Is your exposure going to be compromised by the fact that you can’t lift it off the aorta because of the involvement of the IMA?

Remarkably, a failure to progress was noted in all cases; every surgeon recognized at least one episode during his case in which forward movement had stalled. This failure to progress was often pointed out by the operative surgeons themselves. As one surgeon described, “Watching it…I…felt like there was a lot of mucking around.” The failure to progress was obvious to surgeons even when they remembered the case as having proceeded smoothly. This observation accentuates a singular asset of video: it allows one to view oneself in the third person and provides incontrovertible evidence to counteract the inaccuracies of one’s memory.

In many instances, after a failure to progress was identified, the coach detailed potential maneuvers to resolve the situation. As the chief resident recounted the actions taken by his attending to recover from his own failure to progress, the coach clarified the rationale behind them, thus encouraging their uptake:

Surgeon A: I was doing this probably with less aggression than I might have…(Surgeon B) just swept his fingers up, and…the tissue plane is clearly visible…All he had to do was come along with his finger, and I was…going centimeter by centimeter – or less – with the right angle.

Coach: Well, you want to do the maneuver he’s doing there early on…because you want to be sure that the head of the pancreas comes off the cava. And so, as you get that mobility from the cava, you’re putting tension on the peritoneal attachments…along the inferior border of the duodenum…It’s an avascular plane, so you can move on.

Teaching

Senior surgeons also tended to use the coach to discuss techniques for teaching residents intra-operatively. The tension between surgeons’ often competing priorities in the OR was addressed; surgeons exchanged tips for inducing desired behavior changes in their residents without compromising care. In the following quote, we see a surgeon explain his method of teaching residents to use both hands when operating. He starts with explicit instructions and, throughout the case, physically reminds his residents of them.

Surgeon: I tell these guys, “I don’t really care what you are doing with your left hand if there’s a forceps in your hand, but I want to see your left hand doing things.” So often the hand’s here, and they’re doing this…Then I end up pulling my way and using a right angle…to try to push it…So it might be forceps, it might be your hand, but it’s got to be something working so that tension/counter-tension.

The Value of Post Game Analysis

Participants universally endorsed the sessions as educationally invaluable. Our coach delineated the unique advantages of using video in allowing surgeons to view themselves in the third person:

Coach: I think there’s always a benefit in watching yourself do something because you don’t see it as you’re doing it. You don’t recognize the fumbling and the stuff that you’re doing. And there’s some value to that.

a sentiment with which both the senior and junior attendings agreed:

Surgeon: I remember that case, and I remember I got to this point, and I wasn’t happy. And then we’re watching it, you know, “See what you think…see what you say.” Looking at it, it puts it into perspective.

Surgeon: I think it always helps…I don’t think there’s any operation that you can’t do better…It’s like…when I went to tennis camp. The first thing they do is video you serving. You realize, I thought I looked like Nadal, and I don’t look like Nadal at all.

Several surgeons underscored the utility in receiving targeted feedback from the coach, particularly for residents and recent graduates. Such junior surgeons are likely to need help interpreting their videos, as well as advice to direct their self-improvement – especially when encountering a failure to progress:

Surgeon: With junior faculty, if they are out of their comfort zone, there’s a lot of futzing… Really, no forward progress…You can sort of help people along.

For senior surgeons, the value of the video review session lay in the peer-peer interaction – in the way it allowed surgeons to see from a new vantage point. The very senior surgeon likened it to operating with another experienced surgeon. He saw the session as an opportunity for two senior surgeons to learn new tricks from one another, despite (and perhaps because of) their individual success in the OR:

Surgeon: Every time I’ve ever operated with another senior surgeon, I’ve learned something, so there’s value in (Coach) and I being in on the same case. Every time I’ve operated with someone, I’ve learned something – a different technique from me or the way they’ve approached it. I mean, every single time. We don’t do that very often.

The coach agreed with this assessment, “I just thought of it as a conversation. I was always interested to learn how people do things and what they do.”

In addition to informing operative surgeons, participants felt this dialogue could serve to advance resident education, whether by exposing trainees to the range of techniques and/or philosophies available to them:

Coach: I think there’s some advantage to hearing the fact that the way (Surgeon) holds his scissors and the way other people hold their scissors is debatable…There’s pros and cons about it. And it’s worth hearing that debate.

or by providing a protected time and space to reconcile missed intra-operative teaching opportunities:

Coach: I think all of us just miss those kinds of opportunities…(Resident’s Attending) is at a point of being unteachable…but he can learn where it would’ve been useful to show (Resident) something.

or by triggering joint reflection between the operative attending and the resident:

Surgeon: I think working with (Coach) and…working with residents, saying, “Ok, what’s your reaction? Here’s my reaction,” and, “How could I be a better teacher? How could you be a better student? How could we do this better?” You know, I think that’s healthy.

A caveat was made for junior faculty members; participants felt that resident presence might hinder the candor of a discussion between the coach and a less experienced attending:

Surgeon: I just don’t know whether the junior faculty would be very defensive with residents in there.

Coach: Discussing each little maneuver – as to why you do it, what are the advantages, what are the disadvantages, how would you go about it, what can be done – would be a little more pointed and directed more towards them than it would be with a resident overhearing our discussion.

Finally, participants expressed the opinion that the session was beneficial to the coach as well as the operative surgeon. In addition to its value as a peer-to-peer interaction from which both coach and surgeon may learn, the program provides an avenue for surgeons to maintain a role in surgical education at the end of their operative careers:

Surgeon: It also keeps them pretty involved…I think there’s a real value for that…There is probably a market for people who are willing to be the coach.

Discussion

Continuous professional development (CPD) is the new paradigm in surgical education. In contrast to traditional continuing medical education, which, by design, is episodic and aimed at heterogeneous audiences, CPD “emphasizes ongoing professional development of individuals across the continuum of their careers…It is learner-centered and self-directed.” (19) It is the endorsed teaching modality of the Division of Education at the American College of Surgeons (16, 19), yet few interventions that address operative skill in this manner have emerged.

Reznick advocates for deliberate practice in the acquisition of surgical technique (20). Only with deliberate practice may expertise be attained. In order to improve performance, one must reflect on his/her own thought processes and actions, then, in the future, monitor and make specific, intentional adjustments to them. Feedback, Ericsson describes, mediates this self-directed development; it aids in the clarification of thinking and the setting of goals (18).

Video is among the most effective means of illustrating surgical technique. It is used extensively for technical demonstrations at national conferences, and skills assessments of surgical trainees using video have proven reliable (21–23). As a vehicle for allowing one to reflect on his/her own performance, video is unparalleled. It has already been deployed successfully in the delivery of customized feedback to physicians in other settings, improving trauma resuscitation time (24–25), epidural placements by anesthesiology residents (26), laparoscopic bench task performance by practicing urologists (27), and residents’ adequacy of exposure and motion efficiency during inguinal hernia repairs (28). Head-to-head comparisons with verbal feedback alone demonstrate the superiority of video-based interventions in sustaining and continually effecting behavior change over time (25–26). By providing participants with a third-person view of themselves (29) and/or a means of benchmarking themselves against others (30), video seems to mitigate the inaccuracy problems inherent to physician self-assessment; it thwarts denial. One study demonstrated a positive effect of video even without an accompanying review: after gastroenterologists merely became aware that they were being recorded, the quality of their colonoscopies improved (31).

It should be noted, however, that such a Hawthorne effect will only be observed if the subject is already aware of the modifications needed to augment performance; without someone to help interpret it, the video loses significant meaning. In a study of medical students, verbal commentary from an expert was more effective in improving motion efficiency than self-accessed, computer-generated feedback (32). Our coach picked up on nearly three times as many educational cues as the surgeons, a fact that underscores the tremendous value added by his input to the video alone. Truly constructive feedback can only be furnished by an experienced surgeon, for reasons of credibility, as well as knowledge. Indeed, the coaching relationship has similarities to mentorship, a hallmark of the classic Halstedian model of teaching (33), for which expertise is a fundamental prerequisite. Video-based coaching co-opts the elements of mentorship that are responsible for its ability to effect real practice change: the identification of an individual’s gaps in knowledge and/or skill and the sharing of strategies to overcome these deficiencies (34–35).

Like mentoring, video-based coaching must be psychologically acceptable to the individual – suitable to his/her concept of self as a professional and a student of surgery. As we have described, the feedback given by our coach was tailored to each surgeon based upon learning needs and learning style, and the session was rated as a highly effective educational tool by surgeons spanning a wide range of personality types and professional experience. When we asked our coach about his deliberateness in applying various coaching techniques, he replied that he refined this practice through extensive experience with intra-operative consults. He attributed his success to “not making people feel that they’re being judged” and the “fairly light hand, socially” that he has cultivated. Clearly, the success of the program is predicated upon the adaptability of the coach, a quality that may be acquirable through training.

This flexibility extends to refitting of the coaching paradigm to the individual surgeon. Our initial prototype was one of Instructional Coaching (also known as Expert Coaching), in which a veteran consultant helps a relative novice incorporate changes into his/her practice by encouraging self-reflection and modeling behavior (36). However, the sessions shaded towards Reciprocal Coaching, particularly when the surgeon under review was more senior. The relationship of coach and senior surgeon tended towards bidirectionality, with each learning from the other. The field of education considers both Expert and Reciprocal styles variants of Peer Coaching; both coach and trainee are licensed professionals in either situation. As in education, coaching in surgery must be offered in a non-threatening manner. Peer Coaching, based upon observation and constructive feedback, is grounded in partnership; it is non-evaluative and non-judgmental by nature (37). Its use in the professional development of physicians as teachers (38) and academics (39) has been reported. As our study illustrates, it forms a spectrum from Instructional to Reciprocal, depending on the experience of the operative surgeon relative to the coach.

Video-based review is highly practical, eliminating many of the inconveniences and risks associated with live intra-operative mentoring. Fast-forwarding confers a time savings of 50–80% upon would-be mentors without compromising their ability to assess (21–22, 29). Additional efficiency (and anonymity) may be gained by orchestrating coaching sessions via video conference, as the electronic transmissibility of video lends itself nicely to remote viewing. Concerns about the ethical and medico-legal responsibility of the coach are circumvented, and any sense of urgency or distraction that concurrent patient care may provoke in the operative surgeon is removed, allowing him/her to fully concentrate on his/her performance. Video demonstrates similar advantages over simulation: it requires little upfront investment in time or expense from the operative surgeon or his/her department, and the fidelity of the exercise to actual practice is irrefutable.

By the nature of its format, Post Game Analysis is easily scalable; videos no longer require sophisticated equipment to make, are physically and electronically portable, and are reproducible, i.e. may be reviewed repeatedly, by multiple users. The program may fill a void in national surgical educational initiatives directed at operative skill, and has the potential to be of particular relevance to surgeons practicing in geographically remote areas and/or within solo practices – those who experience barriers in accessing traditional learning opportunities.

We suggest a number of considerations as surgical coaching programs are developed:

1. Context should be provided to aid the coach in putting him/herself “in the position of the surgeon…where I would put the incision, how I would proceed…based on my preoperative ideas about the anatomy.” Our coach asked specifically for radiologic imaging, as well as an informal, oral synopsis of the patient’s initial presentation.

2. Coaching, in the truest sense of the word, implying a gradient in expertise between mentor and mentee, should perhaps be targeted at junior surgeons. For senior surgeons, Post Game Analysis may function better as a peer-peer interaction – dedicated time and space in which participants may learn from one another.

3. Trainees may benefit from the interplay between a senior surgeon and the coach, as it illustrates the rationale behind alternative approaches to an operation. Senior surgeons seem to be secure enough with their performance so as to not feel threatened by trainee presence.

4. We recommend a video display including both a close-up overhead view of the operative field and a complete view of the operating table in order to give the coach a sense of the surgeons’ fine and gross motor movements, as well as the overall set-up, including the placement of retractors and the positioning of assistants. Audio may be considered as a means of identifying missed teaching opportunities, but may ultimately be deemed distracting to the discussion.

5. Although our surgeons underwent only one session each, every one of them felt that it was an educationally valuable experience. We anticipate that coaching is best utilized longitudinally. With repeated sessions, the coach may refit his/her comments to the surgeon’s changing performance, and the intervention may realize its true potential as a vehicle for continuous practice development.

Coaching may be a highly effective method for improving surgical outcomes; however, there are many unanswered questions about this new educational modality. Further investigation is needed to optimize the approach.

1. Resident involvement was suggested by 2 of the 3 surgeons who did not undergo review with a trainee. We conducted one such session, which was well-received by all participants. More data is needed determine the optimal approach to integrating trainees into the post-game analysis sessions.

2. The optimal format for Post Game Analysis must be determined for each cohort of potential beneficiaries. Examples of those most likely to benefit include recent graduates, remote/isolated surgeons, and surgeons re-entering practice following extended sabbaticals. Each population will likely require sessions customized to its own needs.

3. Further study is indicated to determine the optimal approach to coaching, including comparisons of the various potential modalities (traditional live intra-operative mentoring v. in-person video-review v. intra-operative video-conferencing v. post-hoc video-conferencing) and/or participant configurations (individual v. team reviews).

4. In order for the intervention to be scaled effectively, more evidence is needed about the specific attributes of a successful coach, including those which are baseline prerequisites and those which may be achieved through training. We have described the characteristics of our coach that helped him communicate effectively with a diverse group of surgeons. Further study of coaching provided by other surgeons may enlighten us to a still wider range of coaching techniques and/or surgical topics.

5. We allowed the Post Game Analysis sessions to flow organically; we did not direct the content of the discussions or instruct the coach in his approach. Further information about the appropriateness and effectiveness of various coaching topics would aid in streamlining the intervention. Ultimately, our goal is to refine these techniques and topics into guidelines and/or training for would-be coaches to improve reproducibility.

6. All participants commented spontaneously on the educational value of the session, and some later reported making concrete practice changes based upon it. Further investigation is required into the impact that surgical coaching may have on future performance, preferably using objective metrics related to patient outcomes.

Sachdeva writes, “Frequent low-stakes assessments that are coupled with specific and meaningful feedback should be the hallmark of activities aimed at CPD” (19). Video-based Post Game Analysis is a novel program that accomplishes just that. It provides a means to transform our daily experience into deliberate practice.

Table 1b.

Operations under Review

Operation

Pancreaticoduodenectomy

Radical resection of large retroperitoneal sarcoma including adjacent organs; involvement of major blood vessels

Reoperation for resection of retroperitoneal mass including adjacent organs

Subtotal gastrectomy with celiac node dissection