Mentor-Guided Self-Directed Learning Impacts Resident Practice

Abstract

Objective

Self-directed learning (SDL) can be as effective as instructor-led training. It employs less instructional resources and is potentially a more efficient educational approach. Although SDL is encouraged among residents in our surgical training program via 24-hour access to surgical task trainers and online modules, residents report that they seldom practice. We hypothesized that a mentor-guided SDL approach would improve practice habits among our residents.

Design

From 2011–2013, 12 PGY-2 general surgery residents participated in a six-week minimally invasive surgery (MIS) rotation. At the start of the rotation, residents were asked to practice laparoscopic skills until they reached peak performance in at least three consecutive attempts at a task (individual proficiency).

Setting

Trainees met with the staff surgeon at weeks three and six to evaluate progress and review a graph of their individual learning curve. All trainees subsequently completed a survey addressing their practice habits and suggestions for improvement of the curriculum.

Results

By the end of the rotation, 100% of participants improved in all practiced tasks (p < 0.05), and each reported that they practiced more in this rotation than during rotations without mentor-guided SDL. Six (50%) reported that their skill level had improved relative to their peers. Some residents (n=3) felt that the curriculum could be improved by including task-specific goals and additional practice sessions with the staff surgeon.

Conclusions

Mentor-guided SDL stimulated surgical residents to practice with greater frequency. This repeated deliberate practice led to significantly improved MIS skills without significantly increasing the need for faculty-led instruction. Some residents preferred more discrete goal setting and increased mentor guidance.

INTRODUCTION

Trainees who practice skills in a simulated environment show improvement.^1,2 The addition of simulation training into a surgical training curriculum results in decreases in operative time and intraoperative errors while improving patient outcomes and shortening length of stay.^3–5 However, even at programs which offer a state-of-the-art simulation lab that is accessible to trainees on a 24-hour basis, residents report that they seldom practice and our group has recently demonstrated that this can lead to non-proficiency in surgical tasks.⁶ With the continued expansion of medical and procedural knowledge, healthcare professionals and regulatory organizations are in agreement that emphasis in medical education should be placed on life-long learning.^7,8 Resident education and procedural training must evolve to facilitate these rapid changes in health care alongside the pressure of duty hour restrictions, limited funding and decreased instructor availability. Ongoing restructuring of the graduate medical education model, from one of purely instructor-guided training, to that of self-directed learning (SDL), has been reported to create learning environments that facilitate the required flexibility of changing informational and procedural needs.⁹ This study aimed to investigate how mentor-guided SDL impacts resident propensity toward SDL in the simulation-based setting.

METHODS

Aims

The general aim of this study was to determine whether a mentor-guided SDL approach would increase a resident’s aptitude toward SDL, stimulate practice, and improve surgical skill acquisition in the simulation lab.

Participants

Approval from the Intuitional Review Board (IRB) was obtained. Twelve (n=12) PGY-2 categorical general surgery residents participated in the study from 2011–2013. All residents participated in a six-week minimally invasive surgery (MIS) rotation. The residents all had been exposed to the simulation laboratory during their PGY-1 year of training in mandatory simulation-based training sessions. The average laparoscopic operative experience of the participants in the study was 10 basic and 6 complex laparoscopic cases as surgeon (jr) during their PGY 1 year and an additional 30 basic and 15 complex laparoscopic cases as assistant. Each resident was informed that there would be no external incentives or compensation for practicing, or disincentives for not, and time spent in the simulation lab would not be counted toward duty hours. The staff surgeon was MIS trained and had experience with FLS. Further the staff surgeon received advanced training in surgical education methods such as the American College of Surgeons ‘Surgeons as Educators’ course and had more than 10 years’ experience in teaching and curriculum development.

Materials and Tasks

Three trainers were utilized for SDL in this study: SimSurgery Simulator (Simsurgery Oslo, Norway), Lapsim 2014 (Surgical Science, Göteborg, Sweden) and a standard box trainer for intracorporeal suturing. Modules available for trainee practice included basic tissue handling, precision movement tasks, ligation techniques, basic and advanced suturing and intracorporeal knot tying. Residents had 24-hour 7 days a week access to the training facility, located on campus connected through other clinic building from the surgical clinical facility. In addition, each resident had a foldable box trainer available for practice at home.

Procedure

During the course of the study, the residents met with the same MIS trained staff surgeon (JB) at the beginning, middle and end of their rotation. At the initial meeting, the PGY-2 residents met with the staff surgeon to discuss learning goals and to identify resources that could facilitate the residents’ ability to self-assess learning deficits. The staff surgeon did not provide a mandatory practice schedule or require task specific learning objectives be met or agreed upon. Instead, the staff surgeon asked the residents to practice laparoscopic skills on a computer-based trainer module or box trainer until they reached peak performance at least three consecutive times. The residents were charged with the task of identifying individual proficiency through the utilization of a graphically depicted learning curve. The residents defined proficiency through their peak performance over three consecutive attempts.

When the residents met with the staff surgeon again at the midpoint of their rotation in the training facility, the staff surgeon aimed to provide ancillary support for the residents’ learning needs, technical equipment and advice negotiating logistical hurdles. Suggestions and technical advice were provided to help ameliorate and adjust for continued SDL progression and task acquisition Feedback was based on task selected by the resident for practice. For example if the resident selected an FLS module the staff surgeon provided feedback on improvement of the FLS based task as well as requirements of the FLS curriculum.

At the completion of their MIS rotation, the resident met for a final time with the staff surgeon. The individual learning curves were reviewed. Each feedback session was standardized in its format (e.g. technical support in the training center, learning curve assessment in the office at the end), however the content was directed by the residents self-identified learning needs. The residents were subsequently asked to complete a final survey that aimed to address their practice habits and provide feedback regarding suggestions for improvement of the aforementioned curriculum. The post-practice survey consisted of six questions regarding the time they spend practicing laparoscopic skills previously and during the current rotation, their skills at the start and end of the rotation as well as the residents’ perceived improvement in laparoscopic skills compared to their peers. (Table 1).

Table 1.

Post rotation questionnaire for self-directed learning aspect of minimally invasive surgery rotation

Relative to other PGY-2’s at the time, how would you describe your laparoscopic skills prior to your rotation?

Unacceptable

Below Average

Average

Above Average

Outstanding

Relative to other PGY-2’s at the time, how would you describe your laparoscopic skills after completing your rotation?

Unacceptable

Below Average

Average

Above Average

Outstanding

During your rotation how much time per week on average did you spend practicing your laparoscopic skills?

None

1–15 Minutes

16–30 Minutes

31–60 Minutes

> 1 Hour

How does this compare to the amount of time you typically spend practicing per week?

More than usual

Same as usual

Less than usual

Did you practice differently, or use any different practice techniques as compared to usual?

Yes

No

In what ways did you practice differently? Please describe in detail.

RESULTS

The self-assessed learning curves, for all PGY-2 residents (n=12), displayed improvement in all tasks by the end of the rotation (p < 0.05). Figure 1 and Figure 2 demonstrate individual resident performance logs. On the post-test survey, residents reported that they practiced more on this MIS rotation than during rotations without mentor-guided SDL (Figure 3). Analysis of self-reported skill improvement before and after the six-week rotation showed that 50% (n=6) of the residents felt their skill level had improved relative to their peers (Figure 4). The post-rotation survey indicated that some residents (n=3) felt the curriculum could be improved by including task-specific goals and additional practice sessions with the staff surgeon.

Figure 1.

Resident # 3 individual performance log tracking laparoscopic simulation knot tying over a six-week period demonstrates overall improvement and increasing consistency.

Figure 2.

Resident # 6 individual performance log tracking laparoscopic simulation knot tying over a six-week period. Self-assessed improvement in total time, path length, and angular path during simulation training shows overall improvement.

Figure 3.

Overall time spent practicing laparoscopic skills training for twelve PGY-2 general surgery residents during a six-week minimally invasive surgery rotation.

Figure 4.

Resident self-reported skill improvement over the six-week minimally invasive surgery rotation.

DISCUSSION

This study demonstrated that mentored SDL led to surgical trainees practicing more and improving more than they would have without a framework of SDL; all 12 PGY-2 residents reached their own levels of mastery. These results demonstrate an improvement over a previously practiced approach of un-mentored SDL for surgical tasks (Rowse et al again). During the un-mentored SDL period the training center, on-line videos, and portable trainer boxes were available and not used and the skills did not improve to mastery level. Further, the faculty time needed for this curriculum was limited compared to a weekly hourly skills training curriculum with faculty present.

The study also demonstrated that learners desired task-specific goals as part of their SDL curriculum. While such a curriculum is not without limitations, the involvement of limited expert guidance can serve to optimize the benefit of self-directed learning sessions.

The prevailing model of SDL proposed by Malcomb Knowles¹⁰ describes how even though the social and institutional requirements for learning exist; the process of effective lifelong learning is dependent on an intrinsic motivation to learn. Self-directed learning requires learners to 1) diagnose their learning needs, 2) formulate learning goals, 3) identify resources for learning, 4) select and implement learning strategies, and 5) evaluate learning outcomes (Figure 5). Such a model inherently requires a paradigm shift of learners from a passive to active learning role.¹⁰ In the residency setting, such skill acquisition is elemental to the development of life-long learning.^11,12 Studies have shown that individuals who establish their own goals and objectives independent of external influences are more successful at meeting their objectives.¹³ In addition, when these objectives are set in concordance with individualized resident and faculty written contracts of commitment, there is not only an increase in overall knowledge gained, but individuals demonstrate increased proclivity and accomplishment of SDL tasks.¹⁴

Figure 5.

Method and timeline for planning, including goal setting and identification of resources and subsequent evaluation of learner outcome of self-directed learning.

Although the establishment of both learning contracts and objectives allows residents to establish a learning focus and increased utilization of SDL practices, the SDL framework relies upon an inherent ability of individuals to competently self-assess their skills accurately. A majority of studies have concluded that a learner’s ability to self-assess accurately is quite poor.¹⁵ Eva and Regehr¹⁶ defuse this SDL paradox by recognizing that such fundamental skills required of SDL are not an inherently stable characteristic. Instead, one’s ability to self-assess is contextually bound and requires ongoing monitoring and incorporation of expert external evaluation to promote intrinsic SDL skills.¹⁷ In this study we model one approach to monitor and promote the SDL skill by providing mentor guidance in three ways: Defining the need for self-assessment at certain time points within the rotation, providing access to possible metrics and a time table.

The possibility of decreased operative exposure secondary to duty hour limitations has significant impact on surgical training paradigms. Though it has been shown that outcomes and operative times are improved when residents practice on simulation training, actually encouraging adult learners to go and practice is a difficult challenge. Using our mentor-guided approach appeared to be successful in this regard. There may be various aspects which contributed to this change in resident practice patterns. During the study time there was no change in laparoscopic training modules or simulators; therefore, increased interest relating to a “novel” instrument for learning in the simulation center did likely not play a role. Whether the increased practice comes from faculty engagement and suggestion that practice should occur or knowing that they were being evaluated based on their skills is not determined. Anecdotally it was helpful to some residents that “somebody cared” if they practiced. Additionally, during this MIS rotation time for task practice was not overshadowed by extensive service obligations as the inpatient census is usually low.on this service. The experience compares favorably to our and other training programs having poor success with resident attendance and training progress when faculty were not present.^6,18

In response to some trainees’ request for increased faculty guidance, the trainees are now provided with a video recording of their initial participation in a surgical case. They are invited to review the case at their leisure and determine their learning goals. Faculty feedback on goal selection is available as desired by the trainee.

While the development of SDL characteristics are understood to be initially dependent on both intrinsic and extrinsic factors, how these external influences are best integrated into a SDL curriculum in a beneficial way remains challenging. This concept is further complicated in surgical residency training, where the learning environment and apprentice model previously relied upon for basic surgical training has drastically changed. With the advent of simulation and computer-based video training models, trainees are faced with embodying the SDL framework with limited facilitator guidance. While simulation training alone has been found to result in equal cognitive performance and prolonged knowledge retention when compared to didactic lecture with expert feedback, the same does not hold true for technical performance.^19,20 In addition, trainees using simulation training alone tend to develop a concurrent plateau in cognitive effort that is subsequently devoted to skill acquisition, leading to suboptimal skill mastery.²¹ Yet, when simulation is combined with concurrent minimal expert guidance, not only do trainees demonstrate a higher level of skill mastery when compared to didactic-led instruction, trainees evade the cognitive plateau by means of developing improved self-assessment characteristics.^22–24

Improvement in patient outcomes represents the holy grail of educational interventions in medicine. A study examining simulation-based mastery learning has recently demonstrated improved patient outcomes in selected tasks such as laparoscopic inguinal hernia repair, as well as decreased operative times in the clinical environment.^3–5 Such literature not only speaks to the benefits of simulation, but the benefits of goal-directed practice, as the participants in the study were asked to perform a task as many times as necessary in order to achieve a pre-defined performance standard.

Limitations of our study are small sample size, heterogeneity of practice methods, as well as the reporting of data, including limitations of self-reporting. This SDL curriculum continues and we will continue to analyze resident performance. There is also the possible effect of resident practice being secondary to residents being aware that they were being measured, leading to increased practice.

CONCLUSIONS

Mentor-guided SDL stimulated surgical residents to practice with greater frequency. This repeated deliberate practice led to significantly improved MIS skills in laparoscopic skills training modules. While the need for faculty-led instruction was low, some residents preferred more discrete goal setting and increased mentor guidance.

ABBREVIATIONS

SDL

self-directed learning

IRB

Institutional Review Board

MIS

minimally invasive surgery