Is Teaching Simple Surgical Skills Using an Operant Learning Program More Effective Than Teaching by Demonstration?

Abstract

Background

A surgical procedure is a complex behavior that can be constructed from foundation or component behaviors. Both the component and the composite behaviors built from them are much more likely to recur if it they are reinforced (operant learning). Behaviors in humans have been successfully reinforced using the acoustic stimulus from a mechanical clicker, where the clicker serves as a conditioned reinforcer that communicates in a way that is language- and judgment-free; however, to our knowledge, the use of operant-learning principles has not been formally evaluated for acquisition of surgical skills.

Questions/purposes

Two surgical tasks were taught and compared using two teaching strategies: (1) an operant learning methodology using a conditioned, acoustic reinforcer (a clicker) for positive reinforcement; and (2) a more classical approach using demonstration alone. Our goal was to determine whether a group that is taught a surgical skill using an operant learning procedure would more precisely perform that skill than a group that is taught by demonstration alone.

Methods

Two specific behaviors, “tying the locking, sliding knot” and “making a low-angle drill hole,” were taught to the 2014 Postgraduate Year (PGY)-1 class and first- and second-year medical students, using an operant learning procedure incorporating precise scripts along with acoustic feedback. The control groups, composed of PGY-1 and -2 nonorthopaedic surgical residents and first- and second-year medical students, were taught using demonstration alone. The precision and speed of each behavior was recorded for each individual by a single experienced surgeon, skilled in operant learning. The groups were then compared.

Results

The operant learning group achieved better precision tying the locking, sliding knot than did the control group. Twelve of the 12 test group learners tied the knot and precisely performed all six component steps, whereas only four of the 12 control group learners tied the knot and correctly performed all six component steps (the test group median was 10 [range, 10–10], the control group median was 0 [range, 0–10], p = 0.004). However, the median “time to tie the first knot” for the test group was longer than for the control group (test group median 271 seconds [range, 184–626 seconds], control group median 163 seconds [range 93–900 seconds], p = 0.017), whereas the “time to tie 10 of the locking, sliding knots” was the same for both groups (test group mean 95 seconds ± SD = 15 [range, 67–120 seconds], control group mean 95 seconds ± SD = 28 [range, 62–139 seconds], p = 0.996).

For the low-angle drill hole test, the test group more consistently achieved the ideal six-step behavior for precisely drilling the low-angle hole compared with the control group (p = 0.006 for the median number of technique success comparison with an odds ratio [at the 95% confidence interval] of 82.3 [29.1–232.8]). The mean time to drill 10 low-angle holes was not different between the test group (mean 193 seconds ± SD = 26 [range, 153–222 seconds]) and the control group (mean 146 seconds ± SD = 63 [range, 114–294 seconds]) (p = 0.084).

Conclusions

Operant learning occurs as the behavior is constructed and is highly reinforced with the result measured, not in the time saved, but in the ultimate outcome of an accurately built complex behavior.

Level of Evidence

Level II, therapeutic study.

Introduction

Time and practice are necessary for an orthopaedic resident to learn and perform musculoskeletal surgery. Time constraints from work-hour regulations, time pressures on surgeon teachers, and the limited experience with using tools that many residents bring to orthopaedic programs make learning these skills more challenging [16, 18]. Accomplishing this, with both time and financial sensitivity, is a challenge that requires creative solutions, including efficient teaching models, creative and inexpensive skills modules, and motivating performance criteria [10]. The classical learning models that are presently used, especially those that teach procedural skills, are similar to traditional apprenticeship-style approaches and involve emulation, self-shaping (learning) of complex activities, and extend for the duration of the residency [1, 6]. Such models can be tedious and demotivating, resulting in slow progress. This is in part the result of teaching procedures that generally focus on what students do incorrectly.

An alternative to the classic teaching models is a teaching procedure that is based on operant learning. The foundational tenet of operant learning is that a behavior is more likely to occur if it is reinforced. Behaviors in humans have been successfully reinforced using the acoustic stimulus from a clicker, where the clicker serves as a conditioned reinforcer that marks the moment at which the criterion has been achieved in a way that is language- and judgment-free [13, 14]. An example of this method is the teaching procedure TAGteach (teaching with acoustic guidance) (TAGteach International, Indian Trail, NC, USA) that uses operant learning methodologies (including a variety of conditioned reinforcers) along with a process known as chaining [13, 17]. Chaining is the act of breaking complex activities into simpler behavior links, reinforcing those behavior links, and then stringing those learned behavior links back together into the desired complex behavior chain [13, 17]. When the learned complex behavior (the skill) becomes automatic through reinforcement, deliberate practice, and repetition, the learner’s full attention can then be focused on performing the complex task, in any environment, with an expectation of success [5, 13, 20]. Operant learning with deliberate practice is not a new concept and has been used in a variety of human athletic and performance activities successfully enhancing performance in football lineman, golfers, pole-vaulters, and dancers [7, 14, 19], but to our knowledge, a program that takes advantage of these concepts and uses them for orthopaedic skill training has not been tested before.

For this project, we compared two teaching strategies: (1) an operant learning methodology (adapted from TAGteach); and (2) a more classical approach using demonstration alone. Our hypothesis was that a group that is taught a surgical skill using an operant learning procedure will be able to more precisely perform that skill than a group that is taught by demonstration alone. To accomplish this, we identified two tasks from our surgical skills program, “tying the locking, sliding knot” and “making a low-angle drill hole,” and taught those skills to the test learners using an operant learning procedure. We compared the learners’ fluency (accuracy and time of performance) achieved with the operant learning approach with that achieved using a typical demonstration approach often used in surgical teaching.

Materials and Methods

The Albert Einstein College of Medicine institutional review board approved testing and found the project exempt because of its use of normal educational practices. Sixteen tool-specific modules (with 72 behavior-specific, submodules) were designed for and presented to the 2014 Postgraduate Year (PGY)-1 orthopaedic resident class Two of these submodules, “tying the locking, sliding knot” and “making a low-angle drill hole,” were selected for evaluation in this study. Both of the selected submodules were composed of steps with performance criteria that could be consistently measured as successful or not successful.

Tying the Locking, Sliding Knot

Both the test group and the control group were asked to learn to tie a locking, sliding knot (a Tennessee Slider variant) using a two-foot piece of one-fourth-inch braided nylon rope.

The test group was composed of six PGY-1 residents (three men and three women) and six volunteer, randomly selected, first-year and second-year medical students (five men, one woman). The control group was composed of five PGY-1 and PGY-2, nonorthopaedic, surgical residents (three men, two women) and six volunteer, first- and second-year medical students (five men, one woman).

Each test learner was taught, one-on-one, by a single senior surgeon who was trained in operant learning methods (IML), who used operant methodology and followed a precise script (Table 1); the goal, “tying the locking, sliding knot,” was identified, brief background information about the purpose of the knot was given followed by a demonstration of tying the knot. The first component step (behavior) of the complex knot-tying behavior was identified. Instructions were then given and that component behavior was demonstrated. The tag point, that is, the aspect of the behavior to be marked with acoustic feedback (a click), was named using five words or less. The learner was then asked to reproduce that behavior. A correct performance of the behavior (achieving criteria) was marked with a clicker and repeated five times. Each component behavior was learned in succession and then linked together into a behavior chain until the behavior of “tying the locking, sliding knot” was completed. The time to achieve the first correct knot was recorded. The time started with the first instruction and ended with completion of the first correctly performed knot. The behavior, “tying the locking, sliding knot,” was repeated and marked five times. The learners were then asked to practice tying the knot for 15 minutes. Once the behavior was acquired, the 15-minute practice time allotted allowed for practice of at least 100 knots. Practice was done in isolation with no verbal instructions, diagrams, or video available.

Table 1.

The script for the submodule ‘tying the locking, sliding knot

Background: Goal	The goal is to learn how to tie a locking, sliding knot.
Background: Importance	The locking, sliding knot is used to fix soft tissue to bone and it can be delivered through a cannula.
Demonstrate
Behavior 1
Describe the behavior	The goal is to pinch the 1/3 of the rope with your left hand and place it directly over the 2/3 of the rope pinched with the right.
Instruct	1/3 left directly over 2/3 right.
Demonstrate	See Figure 1A.
Repeat demonstration with instruction	1/3 left directly over 2/3 right.
Identify the tag point	The tag point is “over.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 2
Describe the behavior	The goal is to wrap the long end around your left palm, then over the rope pinched with the left hand and then between the two limbs of the rope.
Instruct	Wrap, over and through.
Demonstrate	See Figure 1B.
Repeat demonstration with instruction	1/3 left directly over 2/3 right, then wrap, over and through.
Identify the tag point	The tag point is “through.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 3
Describe the behavior	The goal is to wrap the long end of the rope around your right palm and then hold it with a “fakey” pinch.
Instruct	Wrap to a “fakey” pinch.
Demonstrate	See Figure 1C.
Repeat demonstration with instruction	1/3 left directly over 2/3 right, wrap, over and through, then wrap to a “fakey” pinch.
Identify the tag point	The tag point is “fakey pinch.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 4
Describe the behavior	The goal is to maintain the loop.
Instruct	Maintain the loop.
Demonstrate	See Figure 1C.
Repeat demonstration with instruction	1/3 left directly over 2/3 right, wrap, over and through, wrap to a “fakey pinch,” then maintain the loop.
Identify the tag point	The tag point is “maintain the loop.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 5
Describe the behavior	The goal is to pass the long end behind and then over the left pinched rope and then pull it through the loop with a “backside grab.”
Instruct	Pass the long end behind, over and through to a “backside grab.”
Demonstrate	See Figure 1D.
Repeat demonstration with instruction	1/3 left directly over 2/3 right, wrap, over and through, wrap to a “fakey pinch,” maintain the loop, pass the long end behind, over and through to a “backside grab.”
Identify the tag point	The tag point is “backside grab.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 6
Describe the behavior	The goal is to dress, tighten, and deliver the knot.
Instruct	Dress, tighten, and deliver.
Demonstrate	See Figure 1E.
Repeat demonstration with instruction	1/3 left directly over 2/3 right, wrap, over and through, wrap to a “fakey pinch,” maintain the loop, pass behind, over and through to a “backside grab,” then dress, tighten and deliver.
Identify the tag point	The tag point is “deliver.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Performance cue	Tie a locking, sliding knot.

Each control learner received two demonstrations with verbal instructions of the locking, sliding knot, by a single instructor (IML). The control learners were then given an exploded diagram of the knot and asked to construct it. The time to the first correct knot was recorded. The time started with the beginning of the demonstration and ended on completion of the first correctly made knot. They were given 15 minutes (900 seconds) to complete the task. Technique was not commented on or corrected at this point. After successfully completing the knot, the learners were then asked to practice tying the knot for 15 minutes. Practice was done in isolation with no verbal instructions or video available.

Both the test and control learners were then asked to perform 10 knots. Precision of the knot-tying technique along with speed of performance (fluency) was observed and recorded by a single observer (IML), one learner with the observer at a time. To precisely perform the knot-tying technique, the learner had to successfully complete each of the critical steps. For the locking, sliding knot, there were six steps (Fig. 1A–E): (1) place one-third of the rope directly over two-thirds of the rope (Fig. 1A); (2) wrap, then go over and through (Fig. 1B); (3) wrap to a fakey pinch (the rope coming from palm side to the thumb-index pinch [Fig. 1C]); (4) maintain the loop (Fig. 1C); (5) go behind over and through (Fig. 1D); and (6) dress and slide (Fig. 1E). We determined the performance of a component step to have been completed successfully or not completed. Performance of the complex behavior was successful only if all of the component steps were performed successfully. We then timed the surgeon-instructor tying 10 knots correctly, repeating the behavior five times, to establish a best time target to use as the speed component for fluency determination.

Fig. 1A–E.

(A) The figure shows component behavior 1. The goal is to pinch one-third of the rope with your left hand, placing it directly over the two-thirds of the rope pinched with the right (“pinch” with index and thumb is a foundation behavior that has been previously taught). The TAGpoint (position to be marked with the clicker) is “over.” (B) The figure shows component behavior 2. The goal is to wrap the long end around the left palm, then over the rope pinched with the left hand, and finally between the two limbs of the rope. The TAGpoint is “through.” (C) The figure shows component behaviors 3 and 4. The goal of behavior 3 is to wrap the long end of the rope around your right palm and then hold it with a fakey pinch. (“Fakey pinch” is a foundation behavior that has been previously taught.) The TAGpoint is “fakey pinch.” The goal of behavior 4 is to maintain the loop built in behavior 3. The TAGpoint is “maintain loop.” (D) The figure shows component behavior 5: the goal is to maintain the loop, pass the long end behind and then over the left pinched rope, pulling the rope through the loop with a “backside grab.” (“Backside grab” is a foundation behavior that has been previously taught.) The TAGpoint is “backside grab.” (E) The figure shows component behavior 6. Dress, tighten, and deliver. The TAGpoint is “deliver.”

The success of technique for tying the knot was compared for the two groups using the Wilcoxon rank-sum test for median comparison. The median time (in seconds) to the first knot between test and control groups was compared using Wilcoxon rank-sum test. The time (in seconds) needed to perform 10 knots after 15 minutes of practice was compared for the two groups using the two-sample t-test for mean comparison.

Making a Low-angle Drill Hole

For this submodule, both the test group and the control group were asked to learn to make a low-angle drill hole (for this case, a drill hole that was 45° to a tangent of a circular polyvinyl chloride [PVC] pipe) using a one-eighth-inch drill bit and a 12-volt cordless drill (DeWALT Industrial Tool Co, Baltimore, MD, USA) drilled into a two-foot piece of ¾-inch PVC pipe marked with lines along the longitudinal axis of the pipe that were spaced 120° apart.

The low-angle drill hole test group was composed of eight first- and second-year medical students (five men and three women) who were randomly selected from a group of 16 who volunteered. For the low-angle drill hole control group, eight first- and second-year medical students (seven men and one woman) were randomly selected from the same group.

Each test learner was taught, one-on-one, by a single senior surgeon who was trained in operant learning methods (IML), used an operant methodology, and followed a precise script (Table 2); the goal, “making a low-angle drill hole,” was identified, brief background information about the purpose of the drill hole was given followed by a demonstration of making the low-angle drill hole. The first component step (behavior) of “making a low-angle drill hole” was identified. Instructions were then given and that component behavior was demonstrated. The aspect of the behavior to be marked (the tag point) was named with five words or less. The learner was then asked to reproduce that behavior. A correct performance of the behavior (the criteria were achieved) was marked (with a clicker) and repeated five times. Each of the six component behaviors were taught, learned in succession, and linked together into a behavior chain until the behavior of “making a low-angle drill hole” was completed. The behavior “making a low-angle drill hole” was repeated and marked five times. The learners were then asked to practice the drill hole for 10 minutes on 12 inches of marked ¾-inch PVC. Learners averaged 20 practice holes and were limited by the available space on the practice pipe. Practice was done in isolation with no verbal instructions, diagrams, or video available.

Table 2.

The script for the submodule ‘making a low-angle drill hole’

Background: goal	The goal is to learn how to properly make a low-angle drill hole.
Background: importance	A variety of bone-implant constructs require a low-angle drill hole.
Demonstrate
Behavior 1
Describe the behavior	Grip means to comfortably hold the drill in your nondominant hand.
Instruct	Grip the drill with your nondominant hand in a position to use it.
Demonstrate
Repeat demonstration with instruction	Grip the drill with your nondominant hand in a position to use it.
Identify the tag point	The tag point is “grip.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 2
Describe the behavior	Locate means to place the drill tip on the target.
Instruct	Locate the drill tip on the target.
Demonstrate
Repeat demonstration with instruction	Grip and then locate the drill tip on the target.
Identify the tag point	The tag point is “locate.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 3
Describe the behavior	The goal is to stabilize the drill tip, perpendicular to the surface at the desired location, with the nondominant hand.
Instruct	Stabilize the drill, perpendicular to the surface at the desired location, with the nondominant hand.
Demonstrate
Repeat demonstration with instruction	Grip, locate, and then stabilize the drill tip, perpendicular to the surface at the desired location, with the nondominant hand.
Identify the tag point	The tag point is “stabilize.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 4
Describe the behavior	The goal is to make a dimple in the material and remove the drill bit while it is spinning at low speed.
Instruct	Make a dimple with the drill running at very low speed and remove it while it is still spinning.
Demonstrate
Repeat demonstration with instruction	Grip, locate, stabilize, and then make a dimple with the drill running at very low speed and remove it while it is still spinning.
Identify the tag point	The tag point is “make a dimple.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 5
Describe the behavior	The goal is with the drill tip in the dimple, angle the drill bit at the target.
Instruct	In the dimple, angle the drill bit at the target.
Demonstrate
Repeat demonstration with instruction	Grip, locate, stabilize, make a dimple, and then in the dimple angle the drill at the target
Identify the tag point	The tag point is “angle.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Behavior 6
Describe the behavior	The goal is to drill the low-angle hole and remove the drill bit from the material.
Instruct	Drill the low-angle hole and then remove the drill while spinning at full speed.
Demonstrate
Repeat demonstration with instruction	Grip, locate, stabilize, make a dimple, angle and drill the low-angle hole and then remove the drill bit while spinning at full speed.
Identify the tag point	The tag point is “remove at full speed.”
Practice	Practice to fluency with a peer marking the behavior 5x with a clicker.
Performance cue	Make a low-angle drill hole.

Each control learner received two demonstrations with verbal instructions for making a low-angle drill hole by a single instructor (IML). The control learners were then tested.

Both the test and control groups were asked to make 10 low-angle drill holes in PVC pipe. Starting points and targets were 120° apart and were marked so the drill holes were identical for all tests. Precision of the drilling technique along with speed of performance were observed and recorded by a single observer (IML). To precisely perform the drilling technique, the learner had to successfully complete each of the critical steps. There were six steps for “making a low-angle drill hole”: (1) grip the drill; (2) locate the drill point; (3) stabilize the drill tip perpendicular to the surface; (4) make a dimple while running the drill at low speed; (5) in the dimple, angle the drill; and (6) drill the hole and back the drill out at full speed. Drill point slips occurring while stabilizing the drill, making a dimple, or angling the drill were recorded. The component step performance was determined to be “completed successfully” or “not completed successfully.” Accuracy was evaluated and determined to be successful if the drill exited within a target zone. The boundaries of the target zone were the two drawn longitudinal lines, 120° apart. A drill exiting within the confines of the lines or piercing the lines was considered within the target zone. Any drill piercing at greater than 120° was considered a failure. Performance of the complex behavior was considered successful only if all of the component steps were performed successfully with no slips and with the drill emerging in the target zone. The surgeon-instructor was then asked to drill 10 holes precisely and repeat the behavior five separate times to establish a best time target to use as the speed component for fluency determination.

The median number of technique successes for drilling a low-angle hole was compared for the two groups using the Wilcoxon rank-sum test for median comparison. The mean time (in seconds) to create 10 drill holes was compared for the test and control groups using a two-sample t-test for mean comparison.

Results

Tying the Locking, Sliding Knot

The operant learning (test) group achieved better precision tying the locking sliding knot when compared with the control group but took longer to acquire the first knot. Twelve of the 12 test group learners tied the knot precisely 10 times compared with only four of the 11 in the control group (Table 3). The median number of test group learners who tied correct knots was 10 (range, 10–10) and the median number of control group learners who tied correct knots was 0 (range, 0–10). Using the Wilcoxon rank-sum test for median comparison, the test group was determined to be more precise (p = 0.004) (Table 4). The odds ratio could not be estimated accurately as a result of the small sample size and the 100% success rate in the test group.

Table 3.

‘Tying the locking, sliding knot’--time to first knot, time for 10 knots after practice, and number of knots with the behavior criteria achieved for the test and control groups

PGY-1 + MS test group	Time (seconds) to first knot	After 15 minutes 10 repetitions (sec)	Number of knots behavior criteria achieved
1A	247	106	10
1B	626	92	10
1C	244	78	10
1D	595	97	10
1E	470	111	10
1F	274	67	10
1G	280	101	10
1H	234	120	10
1J	236	80	10
1K	184	92	10
1L	449	89	10
1M	267	110	10
Mean	342	95/12

NOS + MS control group	Time (seconds) to first knot	After 15 minutes 10 repetitions (sec)	Number of knots behavior criteria achieved
2A	143	85	10
2B	218	76	10
2C	110	129	0
2D	93	68	0
2E	DNF 900		0
2F	436	62	10
2G	507	139	0
2H	209	95	10
2J	158	68	0
2K	129	113	0
2L	168	118	0
Mean	279	95/10

PGY = postgraduate year orthopaedic resident; MS = medical student; NOS = PGY1 and 2 nonorthopaedic surgeon.

Table 4.

‘Tying the locking, sliding knot’: time comparisons between test and control groups

Comparison	Test (N = 12)	Control (N = 10)	p value
Time (seconds) to first knot, median (range)	271 (184–626)	163 (93–900)	0.017*
Time (seconds) to 10 repetitions, mean (SD)	95 (15)	95 (28)	0.996†
Number that achieved behavior criteria, median (range)	10 (10–10)	0 (0–10)	0.004*

* p value from Wilcoxon rank-sum test for median comparison; ^† p value from two-sample t-test for mean comparison.

The median “time to the first knot” for the test group (median 271 seconds [range, 184–626 seconds]) and control groups (median 163 seconds [range, 93–900 seconds]) was compared. Using the Wilcoxon rank-sum test, the test group was determined to take longer than the control group to achieve the first knot (p = 0.017) (Tables 3, 4).

After the test, the control group learner who was unable to construct the knot after 900 seconds was taught to construct the knot using the operant learning methodology in 201 seconds.

The 12 test group members averaged 95 seconds ± SD = 15 (range, 67–120 seconds) to complete the 10-knot task and the 10 control group members averaged 95 seconds ± SD = 28 (range, 62–139 seconds) to complete the 10-knot task (Tables 3, 4). Using the two-sample t-test for mean comparison, it was determined that there was no difference between the mean time for tying 10 knots for the test group and control groups after 15 minutes of practice (p = 0.996) (Table 4).

No learner reached the mean fluency ceiling (precision and speed of performance) for successfully performing 10 knots established by the surgeon-instructor (mean 61 seconds ± SD = 4.9 (range, 56–69 seconds).

Making the Low-angle Drill Hole

The operant learning group achieved better precision but averaged longer time to perform the low-angle drill task than did the control group. The technique of the test group more consistently achieved the ideal drilling behavior compared with the control group (p = 0.006 for the median number of technique success comparison with an odds ratio of 82.3 [29.1–232.8] [for a 95% confidence interval {CI}]). It is important to note that three members of the control group damaged their drill bits during the 10 low-angle drill hole tests (Table 5).

Table 5.

Drill time and technique success comparison between test and control groups

Comparison	Test (N = 8)	Control (N = 8)	p value
Time (seconds), mean (SD)	193 (26)	146 (63)	0.084*
Number of technique successes, median (range)	10 (6–10)	0 (0–9)	0.006†

* p value from two-sample t-test for mean comparison; ^† p value from Wilcoxon rank-sum test for median comparison.

The mean time to drill 10 low-angle drill holes for the test group was 193 seconds (range, 153–222 seconds) and the mean time to drill 10 low-angle drill holes for the control group was 146 seconds (range, 91–294 seconds) (Table 5). With the numbers available, there was no difference between the test group and control group in terms of the time it took to complete 10 low-angle drill holes (test group mean was 193 seconds ± SD = 26 and the control group mean was 146 seconds ± SD = 63, p = 0.084) (Tables 5, 6). No learner reached the mean fluency ceiling, to perform 10 drill holes successfully, established by the surgeon-instructor (mean 94 seconds [range, 84–104 seconds]).

Table 6.

‘Making a low-angle drill hole’–time for 10 holes, number of slips, and number of precise drilling behaviors

MS test group	Time for 10 holes (seconds)	Slips	Precise low-angle drilling behavior
3A	170	1	9
3B	165	0	10
3C	201	4	6
3D	211	0	10
3E	153	1	9
3F	211	1	9
3G	208	0	10
3H	222	0	10
Mean	193	0.9	9

MS control group	Time for 10 holes (seconds)	Slips	Precise low-angle drilling behavior
4A	150	5	Bent drill bit 0
4B	121	0	0
4C	150	3	Bent drill bit 0
4D	91	10	0
4E	114	1	0
4F	118	1	Bent drill bit 0
4G	294	8	0
4H	132	2	9
Mean	146	3.8	1.1

MS = medical student.

Discussion

Substantial effort has been made by the surgical and orthopaedic communities to create learning platforms and use effective teaching methodologies to train surgical procedural skills [1]. To acquire fluency with these complex (composite) skills, a learner needs to be both accurate and fast (fluent) with the component (foundation) behaviors of those skills [2, 3]. Operant learning methodologies effectively shape new behaviors, incrementally, by using a high rate of reinforcement, event markers to isolate and identify the desired physical movement, and by connecting or “chaining” foundation behaviors into the desired complex behaviors. Instruction, demonstration, expert feedback, video recording, and computer analysis have all been used to improve technical skill performance and have met with varied success. However, these procedures often use criticism, reprimand, public correction, and other punitive methods, which can interrupt and delay learning. In addition, overestimating simulation realism (fidelity) may result in missing the desired outcome as a result of cognitive and sensory overload [4]. By using an operant learning procedure, these punitive approaches can be avoided. Operant learning procedures have been used successfully with a variety of human athletes and performers (football linemen, golfers, pole-vaulters, and dancers), but to our knowledge, a program that takes advantage of these concepts and uses them for orthopaedic skill training has not been tested before [7, 14, 19]. We used an operant learning model for two specific complex behaviors, “tying a locking, sliding knot” and “making a low-angle drill hole.” Behaviors were established using precise scripts to generate those behaviors and clickers to mark achievement of the behavior in a nonjudgmental manner. For the learning model presented here, the operant test group more consistently achieved precision of performance than when only demonstration was used. However, the time to first success and the time of performance were faster in the demonstration/control group. Reinforcement and the resulting behavior can take longer to build and longer to perform but the precision of that behavior is undoubtedly worth the time.

An important limitation of this study is the control comparison. We have attempted to emulate approaches that we commonly see, in which the skill is demonstrated (but not necessarily broken into its component parts) or where the learner is allowed to self-shape critical aspects of the skill. However, to more completely validate our operant approach, we need to compare other learning methods that more rigorously teach the component elements but do not use positive reinforcement. Also, although our expectation is that operant learning will be effective over the entire skill learning landscape, the present study cannot make such claims. In this study, we have only evaluated two submodules of 72 (Table 7). (Future studies will need to evaluate these submodules to determine to what degree an operant teaching method is effective in disseminating other relevant skills and psychomotor tasks.

Table 7.

Fourteen tools were taught in 16 modules*

Tool module	Submodule (simple behaviors)
Saw 1 (cast saw)	Rules For Tools
	Plunge cut
	Run cut
Saw 2 (oscillating)	Rules For Tools
-Fully incorporated TAG methodology	Run Cut
	Plunge Cut
	Flush Cut
Saw 3 (reciprocating)	Rules For Tools
	Push Cut
	Pivot Cut
Scissor	Rules For Tools
	Cutting
	Spreading
Suture	Rules For Tools
-Fully incorporated TAG methodology	Square Knot (two hand)
	Square Knot (non-dominant)
	Square Knot (one hand)
	Square Knot (non-dominant)
	Slider Knot
	Bowline
	Suture Passing
Drill 1 (fundamentals)	Rules For Tools
-Fully incorporated TAG methodology	Dimple
	Drill Hole
	Angular Drilling
	Plunge Avoidance
	Paired Targeting
Drill 2 (screwdriver)	Rules For Tools
	Screw Placement
	Over Drilling
	Compression screw
	Countersink
Drill 3 (drilling options)	Rules For Tools
	Single hand drilling
	Angular drill to target
	Perfect circles
Drill 4 (high speed)	Rules For Tools
	Straight lines
	Circles
	Keyholes
	Cutting metal
Drill 5 (reamer)	Rules For Tools for Acetabular Reamer
	-Plunge reaming
	-Side reaming
	Rules For Tools for Cannulated Reamer
	-Tunnel cutting
Arthroscope + diagnostic	Rules For Tools
	Insertion
	Quadrant visualization
	Eyepiece rotation
	Forward and back
	Probe triangulation
	Grasper
Arthroscope + manual tools	Rules For Tools
	Straight Biters
	Angled Biters
	Suture Passers
	Chisels and Rasps
Arthroscope + power tools	Rules For Tools
	Arthroplasty system
	Drills and Reamers
Osteotome	Rules For Tools RFT
	Straight cut
	Circular cut
	Shaving and gouging
	Channel cuts
Rongeur/curette	Rules For Tools
	Front bite
	Side bite
	Rip
Compartment pressure monitor	Rules For Tools
	Monitor pressure

* Seventy-two submodules enabled us to train foundation, simple, and complex behaviors using a variety of inexpensive materials.

Another area of potential concern is the composition of the test and control groups. Both were made up of combinations of residents and medical students and for one control group, nonorthopaedic surgical residents. None of the residents or medical students had any prior experience with the technical challenges they were presented with and none had any experience with operant learning. The medical students were part of the medical school’s orthopaedic interest group and were motivated to be successful because they were performing in front of the orthopaedic residency program director. The nonorthopaedic surgical group was used for the knot-tying challenge and demonstrated no differences in knot-tying skill acquisition. Finally, the fact that in both tests, the surgeon-instructor was able to outperform the test and control groups suggests that the learners may not have achieved their performance ceiling in 15 or 10 minutes of practice. In future studies, extending the practice period for longer durations may be necessary to evaluate if fluency has been achieved.

Orthopaedic residents are adult learners and as such a teaching methodology needs to be centered on the learner, the first steps of which should be modeling, coaching, and scaffolding [15]. Surgical skill instructors have attempted to do this. Expert instruction enhances performance of simple surgical procedures, specifically joint injection and knot-tying [8, 9, 12]. Evidence also suggests that the use of computer-based video instruction can be effective in teaching learners surgical skills such as knot-tying [11, 21]. However, whether it is laparoscopic simulation, knot-tying and suturing, using power tools, or doing carpal tunnel surgery, the present teaching models are generally taught as a demonstration/correction or a self-directed/correction couple. Correction is demotivating. In addition, unsupervised repetition of incompletely formed behaviors can build in mistakes or habits that will increase effort and reduce efficiency of performance. Only when students can perform at half the proficiency skill are they likely to participate in and benefit from independent practice [3].

In contrast, operant learning occurs as the behavior is constructed and is highly reinforced with the result measured, not in the time saved, but in the ultimate outcome. In this study, the behavior is built step by step and precisely formed with the aid of an instructor who uses a precise script and acoustic markers to indicate the behavior criteria have been achieved. The students taught using operant methods were precise, whereas our control students had a much higher rate of error; they tended to omit crucial elements of the skill resulting in failure of performance or even damage to the tool (bent drill bits). Our findings suggest that the use of operant learning to teach the foundation behaviors necessary to use the tools of the operating room is effective and can be expected to create learners that use those tools with precision. What is the disadvantage? No doubt, to accomplish this will take effort on the part of teachers to learn the language and methods of behavioral science and operant methodology; as the new tool skills need to be learned, so will the new strategies for teaching them. In the end, our goal is to define a methodology for teaching our orthopaedic residents the fundamental tool skills in a way that is positive, time-efficient, reproducible and motivating, and allows the learner to pass on the acquired knowledge in the same precise way he or she learned it.