Stakeholder perspectives and status of surgical simulation and skills training in Canadian urology residency program

Abstract

Introduction

With the shift to competency-based training, surgical skills lab training (SSLT) may become a mandatory part of Canadian urology residency programs (CURPs). This study aims to identify: 1) the status of SSLT in CURP; 2) stakeholder perspectives on the utility of SSLT; 3) barriers to developing and implementing SSLT; and 4) how to address these barriers.

Methods

Surveys were developed and issued to three groups of stakeholders: 1) SSLT directors at all 13 CURPs (response rate 100%); 2) teaching faculty (response rate 33%); and 3) urology residents (response rate 24%). Surveys 2 and 3 were sent to 10 English CURPs. Results were collected through email and SurveyMonkey^®.

Results

Nine of 13 CURPs have a dedicated SSLT; 46% of CURP have 1–3 sessions per year, 8% have 5–7, and 30% >7. Most residents have independent lab access, but 80% do so less than once monthly. Over 90% of stakeholders find SSLT useful, of which high-fidelity models are most preferred (faculty rated 3.66/4, residents 3.18/4). Program directors (PDs) identified lack of protected faculty time, funding, and infrastructure as the top three barriers to SSLT implementation. Residents found lack of faculty time, protected academic time, and infrastructure as barriers. PDs viewed protecting faculty time and more funding as potential solutions, while residents suggested protected faculty and academic time, and after-hours lab access.

Conclusions

Residents, faculty, and PDs in CURPs view SSLT as useful. Most CURPs have defined SSLT; programs without this have labs for resident use but are underused. To continue to develop and progress SSLT, more time, participation, and funding must be made available.

Introduction

Postgraduate medical education in Canada is undergoing a major shift in emphasis to competency-based medical education (CBME) known as Competence by Design (CBD) by the Royal College of Physicians and Surgeons of Canada. Frank et al highlight the need for new educational technologies in order to make the transition to CBME.¹ The rise of surgical skills laboratory training (SSLT), including surgical simulation, is one example of post-\graduate medical educational programs addressing the need identified by Frank et al.

Surgical skills centers and simulation labs have been developed at many postgraduate training programs in order to meet the need for novel teaching strategies. In the U.S., the national Residency Review Committee mandated that a surgical skills laboratory be available for all general surgery residency programs.² SSLT is currently a recommended training requirement for Canadian urology residency programs (CURPs), but is not yet required by the Royal College of Physicians and Surgeons of Canada. In 2012, 59% of CURPs had a dedicated urology SSLT.³

Urology surgical skills training and simulation broadly covers four domains: open surgery, laparoscopy and/or robotic surgery, endoscopic surgery, and non-technical skills (such as teamwork and crisis management simulations). There is abundant research reporting the development and effectiveness of individual training exercises or programs covering the breadth of the specialty from endoscopy,^4,5 cadaver labs and open surgical skills training,⁶ and laparoscopy/robotics.^7–11 The research regarding satisfaction with individual simulation/skills training exercises is typically quite positive. Despite generally high approval of individual SSLT sessions, there is relatively little research regarding stakeholder satisfaction with SSLT overall in urology and none specifically regarding SSLT in Canada. Most reports of satisfaction focus on learner satisfaction and not that of faculty. An important aspect of ensuring faculty remain engaged and do not burn out is job satisfaction. The current status and perception of simulation and SSLT across urology residency programs across Canada is unknown.

The purpose of this study is four-fold: 1) to identify the current status of SSLT in CURPs; 2) to identify stakeholder perspectives on SSLT; 3) to identify barriers to implementing SSLT; and 4) to determine stakeholder perspectives on how to address these barriers. Stakeholders surveyed included program directors (PDs) of CURPs or the designated director of the urology SSLT program, teaching faculty, and residents.

Methods

Three electronic surveys were developed. The surveys were distributed via email to all stakeholders. Email addresses were obtained from the PD or program administrators. Survey 1 was sent to the PD or faulty lead for SSLT at all 13 CURPs. Survey 2 was sent to teaching faculty and survey 3 was sent to residents at the 10 English Canadian urology residency programs via SurveyMonkey^®. Surveys were validated prior to distribution. Results of this survey were summarized as frequencies with percentages, as well as weighted averages.

Results

Survey 1 had a 100% response rate. Surveys 2 and 3 had response rates of 33% and 24%, respectively. Table 1 shows the breakdown of responses from each survey based on stakeholder type and their respective institution. There were responses from faculty at 10 English-speaking CURPs and from residents at nine CURPs.

Table 1.

Responses from stakeholders categorized by institution and stakeholder type

Institution	Program director	Faculty	Resident
University of British Columbia	1	6	2
University of Alberta	1	4	3
University of Manitoba	1	2	0
Queen’s University	1	7	5
McMaster University	1	1	2
University of Toronto	1	8	2
University of Western Ontario	1	5	1
University of Ottawa	1	4	8
McGill University	1	2	5
Dalhousie University	1	8	6
University of Montreal	1	N/A	N/A
University of Sherbrooke	1	N/A	N/A
Laval University	1	N/A	N/A
Total	13	47	34

Status of SSLT in Canadian urology residency programs

Nine CURPs have SSLT programs (69%) and three have access to a lab but do not run their own sessions. Fig. 1 summarizes the length of time that SSLT programs have been in place. Eight of 13 CURPs had a designated faculty member responsible for the development and implementation of a surgical skills program in their curriculum. Only one program reported providing support, such as protected time or salary support, for the faculty member responsible for their surgical skills program.

Fig. 1.

Number of years skills lab training (SLT) sessions have been offered by Canadian urology residency programs (CURP). The majority of CURP have been offering SLT for 10 years (23.1%). This question was skipped by four respondents.

Six of 13 CURPs have 1–3 sessions per year, while four have >7 sessions per year (Fig. 2). Most programs (10/13) do not use their skills labs for resident assessment. Twelve of 13 CURPs allow residents to access the SSLT independently, but for most (83%), independent practice is not mandatory. Fig. 3 summarizes the types of models or simulators that are used by various programs. Funding for SSLT programs mainly comes from departmental/divisional funds (92%) and educational grants (61%). Estimated cost for SSLT for each program ranged from <$1000 to over $10 000 (Fig. 4).

Fig. 2.

Number of surgical skills lab training (SSLT) sessions offered by Canadian urology residency programs (CURP). The majority of CURP offer between one and three sessions per year (46.2%), followed by more than seven session per year (30.8%); 15.4% of CURP offer zero session per year.

Fig. 3.

Types of simulation modalities offered by Canadian urology residency programs (CURPs). Most CURPs offer high- and low-fidelity models (92.3% and 76.9%, respectively), whereas cadaveric models are used less frequently (38.5%).

Fig. 4.

The approximate cost of running a skills lab training (SLT) program over a one-year period at the Canadian urology residency program (CURP). One program director did not respond to this question. Most SLT cost between $5000 and $10 000 over a one-year period (46.2%).

Stakeholder perspectives

PDs/SSLT leads

Eight PDs/SSLT (62%) leads felt that SSLT was “very useful” for resident training, four (31%) felt it was “useful,” and one (8%) felt it was “not very useful.” Based on a weighted average for usefulness (with 4 being “very useful”), high-fidelity models scored the highest of the various types of training at 3.69 (100% useful/very useful); low-fidelity models scored 3.23 (92% useful/very useful) and virtual reality simulators and cadaveric models both scored 3.15 (85% and 77% useful/very useful, respectively) (Table 2). High-fidelity models were defined as life-like, such as pig and tissue labs. Cadaveric models were categorized separately to identify the number of centers specifically using this unique simulation model.

Table 2.

Comparison of the stakeholders’ perspective on the relative usefulness of various simulation modalities by weighted average

Simulation modalities	Program directors	Faculty	Residents
Low-fidelity	2.08	1.79	1.82
High-fidelity	3.31	3.39	3.18
Virtual reality	2.08	2.49	2.58
Cadavers	2.54	2.42	2.35

Most useful by weighted average is bolded.

Strengths of SSLT according to PDs/SSLT leads were, from highest to lowest usefulness (with 5 being the most useful): helps residents learn fundamental surgical skills (4.54, 70% most important), allow residents to practice outside of the operating room (OR) (3.15, 15% most important), increase resident confidence in the OR (3.15, 8% most important), help residents to learn uncommon procedures or practice critical skills (2.38, 8% most important), and increase foundational knowledge (1.77, 0% most important).

PDs/SSLT leads identified the following as reasons why residents were not using the skills labs for independent practice (in this order): practice is not mandatory (average weighted score 2.58, 67% top reason), residents are too busy (on call, service demands) (2.33, 33% top reason), and that the residents do not see benefit of independent practice (1.25, 12.5% top reason).

Barriers to the implementation of surgical simulation were as follows (weighted scores from most to least important): lack of faculty time to develop and run labs (3.92, 46% most important), lack of funding (3.31, 23% most important), lack of infrastructure (2.92, 15% most important), not enough time in academic schedule (2.77, 0% most important), and detracts from resident clinical time (2.08, 15% most important) (Table 3). Additionally, PD/SSLT leads commented on “lack of resident interest for out of OR practice” and “lack of urology-specific modules” as other barriers to SSLT.

Table 3.

Comparison of the stakeholders’ perspective on the barriers to implementation of surgical simulation by weighted average

Barriers	Program directors	Faculty	Residents
Lack of infrastructure	2.92	3.03	2.94
Lack of funding	3.31	4.00	2.88
Lack of faculty time to develop and run labs	3.92	3.60	3.21
Not enough time in academic schedule	2.77	2.63	3.19
Detracts from resident clinical time (operating room, clinics, etc.)	2.08	2.14	2.67

Biggest barrier by weighted average is bolded.

From the PD/SSLT lead point of view, barriers can be addressed by (in order of weighted rank): protected time for faculty to develop and implement skill labs (4.17, 42% most effective), increased funding of labs (3.50, 25% most effective), commitment by university and/or hospital to develop infrastructure (3.08, 25% most effective), ensuring time is built into academic schedule for labs (2.67, 8% most effective), and ensuring residents can access labs after hours for independent practice (1.58, 0% most effective).

Teaching faculty

Forty-seven teaching faculty members responded to the survey. Twenty-eight (61%) were involved in the development of these training sessions and 40 (85%) were involved in supervising these SSLT sessions. Seventeen (41%) supervise one SSLT session per year, 16 (39%) supervise two sessions, four (10%) supervised three sessions, and four (10%) are supervise four sessions per year.

Thirty-four percent of teaching faculty surveyed felt SSLTs were “very useful,” 62% felt they were “useful,” and 4% felt they were “not very useful.” High-fidelity models were ranked as the most useful, with a weighted average of 3.66 (98% useful/very useful), followed by cadaveric models (2.87, 74% useful/very useful), virtual reality simulators (2.76, 72% useful/very useful), and lastly low-fidelity models (2.66, 66% useful/very useful) (Table 2).

Ninety-six percent of teaching faculty believe that SSLT was most useful for early learning curve practice (weighted average 4.28), followed by practicing rare skills or life-threatening scenarios (87%, 4.09), allowing residents to have independent practice outside of the OR (85%, 4.00), as an assessment tool (68%, 3.70), and allowing residents to practice advanced skills (60%, 3.55).

The biggest benefit of SSLT according to faculty was helping residents to learn foundational surgical skills (with a weighted average of 4.21, 47% biggest benefit). This was followed by: allowing residents to practice skills outside of the OR (4.05, 34% biggest benefit), increasing resident confidence in the OR (2.68, 10% biggest benefit), helping residents learn uncommon procedures (2.27, 5% ranked biggest benefit), and finally increasing foundational knowledge (1.95, 2% biggest benefit).

Barriers to SSLT implementation according to faculty were: lack of funding (weighted average 4.00, 46% biggest barrier), lack of faculty time to develop and run labs (3.60, 23% biggest barrier), lack of infrastructure (3.03, 18% biggest barrier), not enough time in academic schedule (2.63, 12% biggest barrier), and detracting from resident clinical time. (2.14, 10% biggest barrier) (Table 3). Other barriers mentioned in the comments section included: “resident initiative,” “lack of vision from medical school leadership,” “lack of evidence in the literature supporting simulation,” and the question of whether “the small amount of learning is worth the expenses and time.”

In order to overcome these barriers, the following solutions were ranked based on priority: increasing funding of labs (3.95, 38% first priority), commitment by university and/or hospital to the development of infrastructure (3.34, 29% first priority), protected time for faculty for the development and implementation of labs (3.00, 18% first priority), ensuring residents can access the lab after hours for independent practice (2.78, 20% first priority), and ensuring there is time built into the academic schedule for labs (2.43, 5% first priority). Other possible solutions brought up by the teaching faculty included: getting the Royal College to “mandate” these sessions, and having “journal club on surgical simulation to review the literature and update all staff and residents” on the utility of SSLT.

Residents

Thirty-four residents responded to the survey. Five (15%) were in postgraduate year (PGY) 1, nine (26%) in PGY2, seven (21%) in PGY3, five (15%) in PGY4, and eight (24%) in PGY5.

Most residents felt there was a benefit SSLT in resident education in urology. Thirteen of 34 (38%) felt SSLT was “very useful,” while eighteen (53%) felt it was “useful,” two (6%) felt it was “not very useful,” and only one (3%) felt it was “useless.” When asked to rank the various simulation models, high-fidelity models scored the highest (3.18, 55% most useful), followed by virtual reality simulators (2.58, 12% most useful), cadaveric models (2.35, 24% most useful), and finally low-fidelity models (1.82, 9% most useful) (Table 2).

While over 90% of residents reported they can access the skills lab independently, only five (15%) independently access the lab frequently (1–4 times per month). Twenty-one (62%) access the lab infrequently (less than once per month), six (18%) never access the lab, and two (6%) do not have access to the lab.

Ninety-seven percent (97%) of residents feel SSLT is useful for early learning curve training as a beginner. Based on weighted averages, following early learning curve training as a beginner (4.32, 97% agree/strongly agree), other useful benefits of SSLT included practicing rare skills or potentially life-threatening scenarios (3.68, 62% agree/strongly agree), independent practice outside of the OR (3.65, 65% agree/strongly agree), assessment of surgical skills (3.36, 55% agree/strongly agree), and practicing advanced skills (3.29, 50% agree/strongly agree).

Residents ranked the benefits of SSLT as: helping residents learn foundational surgical skills (3.69, 34% biggest benefit), followed by allowing to practice skills outside of the OR (3.42, 32% biggest benefit), increasing confidence in the OR (2.66, 3% biggest benefit), increasing foundational knowledge (2.63, 13% biggest benefit), and lastly helping residents to learn uncommon procedures and practice critical skills (2.52, 15% biggest benefit).

According to the residents, barriers to the implementation of SSLT in order of rank (most important to least) are as follows: lack of faculty time to develop and run labs (3.21, 18% biggest barrier), not enough time in the academic schedule (3.19, 25% biggest barrier), lack of infrastructure (2.94, 25% biggest barrier), lack of funding of labs (2.88, 12% biggest barrier), and lastly that it detracts from resident clinical time in the ORs and clinics (2.67, 18% biggest barrier) (Table 3). Other perceived barriers include: “no supplies” and “no perceived value among staff/residents,” “no local champions of simulation,” and “lack of education and experience amongst staff/residents.”

Residents believed barriers could be addressed by (ranked from most to least effective): protected time for faculty (3.50, 22% most effective), adding SSLT into the academic schedule (3.42, 36% most effective), ensuring residents can access the lab after hours (2.72, 12.5% most effective), increasing funding (2.70, 18% most effective), and commitment by university and/or hospital to the development of infrastructure (2.63, 9% most effective). In the comments section, the need to build SSLT into the curriculum and providing time for the residents to access simulation was reiterated by multiple respondents.

Discussion

Since 2012, the percentage of CURPs that have urology-specific SSLT has increased only marginally (59%³ vs. 69%). However, the use of various teaching modalities appears to be increasing more substantially. For example, the reported use of virtual reality simulators has increased from 29% in 2012 to 69% in 2017, as reported from our survey, while cadaver use has increased from 8% to 39% in that same time frame.³ Existing programs appear to be expanding. Almost all (92%) CURPs have access to a surgical skills lab even if they do not run their own sessions.

In our survey study, SSLT in CURPs is viewed as useful or very useful by nearly all stakeholders (90%). High-fidelity models tend to be preferred; however, other models are useful depending on the skills being practiced. In the general surgery literature, Hagen at al reported that junior residents rate skills lab training as more beneficial compared to senior residents, which likely reflects their relative lack of operative exposure.¹² Our study also showed junior residents rated the benefits of SSLT higher than more senior residents. Hagen et al argue that resources should be targeted to those residents who would benefit most from skills lab training (i.e., junior residents),¹² which may be something for CURPs to consider when developing and implementing SSLT.

When it comes to SSLT, the notion of “if you build it, they will come” does not apply. The results of this survey suggest that having the skills lab and simulators alone is not useful unless they are used within a structured teaching program, as most residents (80%) infrequently or never access the lab for independent practice. Ehdaie et al found that laparoscopic simulators were rarely used by residents on their own initiative and advocated for their use in a structured curriculum.¹⁰ As identified by the PDs/SSLT leads, this underutilization may be due to the fact that independent practice is not a mandatory component of training and that residents are busy with clinical duties. As reported in this survey, residents feel that SSLT should be incorporated into their academic schedule, as their free time outside of academic activities in residency is limited and is often used for research and for non-academic activities.

PDs and teaching faculty rated having SSLT sessions built into the academic schedule as less important to addressing barriers to SSLT, with the emphasis on increasing funding, increasing faculty time for SSLT development, and bolstering infrastructure. This emphasis likely comes from the fact that faculty are the ones who need to develop and implement the training sessions, while residents are feeling lack of time to use the lab. Evidence suggests that the most useful implementation of SSLT, aside from basic skills like knot-tying, involves dedicated faculty involvement with supervision and feedback in a structured curriculum that includes didactic teaching and setup that allows for independent practice by trainees.^13–16

Implementation of such a program requires dedicated faculty time and support. Acton et al found there were substantial increases in faculty hours for implementing SSLT in general surgery training, which is something programs need to consider when developing SSLT.² While SSLT is viewed as highly beneficial by many stakeholders, most urology programs across Canada only have three or fewer sessions per year (Fig. 2). This supports the notion that there are barriers to implementing SSLT sessions and implies that SSLT may be underused. Over 90% of PDs and faculty report a lack of support to develop and implement these sessions. This includes lack of protected academic time and/or salary support. If SSLT sessions are viewed as beneficial to all stakeholders and are anticipated to be used for not only training but also evaluation of surgical trainees in CBD, more resources should be allocated into implementing a number of well-developed, mandatory SSLT sessions. These training sessions should be incorporated into the formal academic curriculum and trainees should be encouraged to access the lab during protected academic time and during their free time for independent practice when possible. Programs can also investigate if there are pre-developed simulation sessions available or share SSLT curricula in order to help overcome the barriers of time and funding. Addressing these barriers may become increasingly important if SSLT becomes a mandatory training requirement in CBD in future years. Such an initiative, however, will require buy-in from all stakeholders and substantial time and financial investments.

Limitations

There are some limitations to this study. The first and likely most important was the limited response rate for surveys 2 and 3. Given the limited response rate from faculty and residents, certain programs may have been over or underrepresented, which may introduce response bias in the results. Also, as there are currently no standardized SSLT urology sessions being used across Canada, different programs may be using different SSLT modalities to teach the same skills to their residents. Depending on the efficacy of these models, stakeholders may develop different perspectives on the utility of SSLT across Canada. A more detailed look at what simulation modalities are being used to teach which urological skills might give better insight as to how SSLT and simulation can be used to enhance urology residents’ surgical training. Additionally, as surveys 2 and 3 were not distributed to French-speaking teaching faculty and residents, the opinions and feedback from our French-speaking faculty and residents are unknown. However, the PDs from these programs were included.

Conclusions

SSLT is viewed as beneficial by most stakeholders; however, SSLT may still be underused at many institutions. Barriers to the development and implementation of SSLT include lack of protected time for faculty to develop and implement these sessions, lack of funding, and lack of protected academic time for residents to participate in these sessions. In order to overcome these barriers, stakeholders are looking for more funding, more support from their university/hospital to develop these programs, and protected academic time for SSLT development and implementation. Collaboration between CURPs so as to mitigate some of these barriers may be useful in the future in order to expand high-quality SSLT across Canadian urology residency programs.