Survey-Based Analysis of Fundamental Tasks for Effective Use of Electrosurgical Instruments

Abstract

Background

Despite widespread use of electrosurgical instruments, there are no widely accepted tasks for training and evaluation of technical skills. The purpose of this study is to propose a set of tasks and report experts' evaluations of the proposed tasks for validity, technical skills vs. knowledge requirements, and utility for future privileging curriculum.

Methods

A set of seven hands-on tasks involving electrical energy were identified based on the Fundamental Use of Surgical Energy (FUSE) curriculum of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES). A web-based survey was developed based on the seven identified tasks. Seven questions were asked for each task. These questions rated candidate tasks on their suitability for validation studies, the role of manual skills, and the appropriateness of including the task in privileging curricula. Members of the FUSE committee were sent the web-based survey and responses were recorded.

Results

Sixteen (16) of the twenty-two (27) members of the FUSE committee responded to the survey. 775 Likert-style responses were recorded and quantified to a 1-5 range. Overall, responses within task-question pairs had a mean standard deviation of 0.83, suggesting general agreement. Tasks requiring bi-manual dexterity scored higher than single-handed tasks on a combined, four-question Likert-scale index for task validation (P<0.0001), and a two-question index for manual skills (P<0.0001).

Conclusions

Survey responses indicated general agreement that the identified tasks represent important technical skills and are consistent with actions in the OR. Bimanual tasks were favored for validation purposes over single-handed tasks. The Traction and Monopolar Dissection and Monopolar Coaptation tasks had the highest agreement with validation-oriented questions (97% and 87%, respectively).

Introduction

Radio-frequency electrosurgery is used in the majority of cases across surgical specialties. Despite broad use, there is no formal training on the theory and safe use of surgical energy [1]. Although total injury rates can be difficult to quantify, reviews have estimated between one and two out of every thousand cases that use monopolar instruments result in energy-associated injury to the patient [2] and that approximately half of the inadvertent injuries during laparoscopic surgery were due to electrosurgery [3]. To address the gap in knowledge, the Fundamental Use of Surgical Energy (FUSE) committee was formed within the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in 2010. The outcomes of this FUSE committee are a comprehensive manual [4] and a high stakes examination that is scheduled to be rolled out in the near future.

While the acquisition and testing of domain knowledge has been addressed by the FUSE committee, no hands-on training component currently exists for FUSE. Based on the success of the Fundamentals of Laparoscopic Skills (FLS) program [5] in assessing both cognitive and psychomotor skills relevant to laparoscopic surgery, it may be inferred that developing a skills simulator is the next logical step. This conclusion may also be drawn from the recently launched Fundamentals of Endoscopy (FES) program [6] of SAGES that has adopted the virtual reality-based simulator LapMentor II from Simbionix (Cleveland, OH) for training. In contrast to the skills targeted by FLS and FES, those required for safe use of surgical energy have not been identified by and established within the surgical community [1].

The FLS skills tasks were developed iteratively, beginning primarily with the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) program [7]. Some of the proposed MISTELS tasks, such as “mesh placement,” were subsequently removed due to poor validation results [8]. Despite these changes, the majority of MISTELS tasks are remarkably well conserved in the current incarnation of FLS.

In comparison, few such hands-on skills tasks have been proposed for the use of surgical energy. This may be due to the combination of technical skill, cognitive knowledge and decision making required for safe energy use in the OR. Energy devices are inherently modal, and depending on the current settings (e.g., power, waveform, etc.), the tissue effects will differ and the specifics of use change. This suggests that hand motions appropriate in one mode may cause undesirable damage, or insufficient effect, for instruments in another mode. Such modal settings can complicate learning due to increased cognitive load [9].

In this study, we sought expert guidance to identify the fundamental tasks that could be used in a hands-on simulator focused on the skills required for electrosurgery. Our approach attempts to follow the success of FLS, where, early in the development of the MISTELS, the SAGES FLS committee was surveyed to gauge coverage and omissions of the proposed tasks [10].

To date, there have been few proposed simulation tasks for electrosurgery. One early work focused on cholecystectomy and proposed an electrosurgical task requiring the peeling of a tangerine fruit using forceps and electrosurgical tool [11]. The LAP Mentor™ virtual reality simulator includes an “electrocautery” task using hook and forceps to dissect simulated tissue [12].

Building on these previously proposed tasks, we have identified seven candidate tasks for training and evaluating electrosurgery skills. To gather expert feedback on these tasks, we developed a survey and distributed it to the FUSE committee.

Materials and Methods

We developed a computerized survey concerning surgical tasks that make use of energy. This survey was sent electronically to the FUSE committee of the SAGES using a web-based service provided by SurveyMonkey Inc. (Palo Alto, CA). The FUSE committee is composed of twentyseven expert surgeons, technologists and medical educators responsible for the development of a curriculum for teaching and evaluating knowledge required for the use of surgical energy-based devices in the operating room.

Seven tasks were proposed as candidates for hands-on skill training and evaluation. Following the precedent of the FUSE manual, the proposed tasks are rooted in the clinical effects resulting from applied electrical energy. The skills required to achieve particular clinical effects are specific to the class of instrument used. Thus electrosurgery requires a more specific and context-based approach to skills training compared to the more abstract manual-skills focus of FLS.

For each task, the survey asked the seven questions (Q1-7) shown in Table 1. The questions were designed to determine the ability of each task to serve as a measurement tool. The Likert-style questions targeted the utility of each candidate task for possible methods of validation. Questions 2, 3, 5 and 6 correspond to the common forms of validity assessment used in the behavioral sciences: face, content, construct, concurrent and predictive [13,14]. Questions 1 and 4 provide a measure of the extent that motor skills are required by the task. Question 7 asks the suitability of the task for privileging or certifying curriculum. Each question was in the form of a statement, which respondents indicated their degree of agreement with “Strongly Disagree” (1), “Disagree” (2), “Neutral” (3), “Agree” (4) and “Strongly Agree” (5). Taken together, these survey questions provide a method for established experts in surgical energy to assist simulation designers in choosing a set of tasks that, once fully developed, will hopefully serve as a valid measurement tool for electrosurgical skills.

Table 1.

Questions accompanying each of the candidate tasks. The second column indicates the question's category of focus (M: Motor skills; Ct: Content validity; Pe: Predictive validity; F: Face validity; Cs: Construct validity; Pi: Privileging).

Q1.	M	This task requires significant and repeated manual practice to achieve mastery.
Q2.	Ct	This task represents, or acts as a surrogate for, tasks performed in the OR.
Q3.	Pe	Skilled performance on this task is a necessary component of safe and effective use of electrosurgery in the OR.
Q4.	M	Knowledge alone is insufficient to master this task.
Q5.	F	This task is indicative of important technical skills for using electrosurgery in the OR.
Q6.	Cs	Expert and novice surgeons are likely to perform similarly on this task.
Q7.	Pi	This task should be included in a privileging or certifying curriculum for manuals skills of electrosurgery.

The responses (73% response rate, N=16) provided expert opinions on the seven hands-on skills tasks. Out of the total 784 possible responses (seven questions covering seven tasks with 16 respondents), respondents omitted nine responses. While the illustrations, shown in Figure 1, accompanied each task description, the respondents were instructed that “the provided images are illustrative only; answer each question according to the textual task description.” For aggregate and numerical analysis, responses to Q6 were subtracted from six, so that a high score indicated a task likely to be able to be validated, aligning Q6's numeric scores with the other questions.

Figure 1.

The proposed set of seven tasks considered as candidates in the survey.

Responses were solicited and collected electronically, and participants remained anonymous. The institutional review board (IRB) of Rensselaer Polytechnic Institute (RPI) exempted the survey from human study guidelines prior to distribution.

Results

Figure 2 through Figure 8 show the response distributions and their numeric means. Respondents rated all of the tasks as indicative of important technical skills (Q5) and the most common response for all tasks was “agree.” Traction and Monopolar Dissection (T5) was rated as slightly more indicative (mean=4.4) and with all respondents either agreeing or strongly agreeing. The inspection task (T7) was least technically indicative, with only 86% of respondents indicating agreement or strong agreement (two respondents left the question unanswered).

Figure 2.

Responses to each task for question Q1: “This task requires significant and repeated manual practice to achieve mastery.”

Figure 8.

Responses to each task for question Q7: “This task should be included in a privileging or certifying curriculum for manuals skills of electrosurgery.”

Questions Q1 and Q4 distinguish the roles of abstract knowledge and manual skills across the tasks by questioning the need for repeated practice and skill beyond “knowledge alone.” In both questions, agreement implies a need for manual skill training in addition to didactic instruction. Combining the results of these two questions suggests that some manual skill is required for all of the proposed tasks, with only 11% (31 of 287, one omission) of responses across all tasks indicating disagreement or strong disagreement.

The tasks with highest levels of agreement for requiring manual skills (Q1 & Q4) were Monopolar Coaptation (T6) and Traction and Monopolar Dissection (T5), with arithmetic means of 4.0 and 4.5, respectively, and 91% and 100% of respondents indicating agreement or strong agreement. In addition, 88% of respondents agreed or strongly agreed with the requiring of manual skill for the final bimanual task (T4). Respondents agreed that the bimanual tasks (T4,T5,T6) required manual skill significantly more often than tasks (T1,T2,T3) that could be accomplished with a single hand (P<0.001, Z=-4.28, Mann-Whitney test, one-tailed).

Showing construct validity is a key initial step for any assessment tool. Responses to question 6, “Expert and novice surgeons are likely to perform similarly on this task,” tabulated in Figure 7 suggest that any of the surveyed tasks would provide a basis for proving the construct validity of a simulator, with 80% disagreeing or strongly disagreeing. Tasks T4 and T5 were slightly more strongly indicated, with means of 4.3 and 4.2, respectively.

Figure 7.

Responses to each task for question Q6: “Expert and novice surgeons are likely to perform similarly on this task.”

Question Q7 asked the respondents' agreement that the task should be included in a privileging or certifying curriculum. Overall, 61% of responses across all tasks indicated agreement, with T5 and T3 most favored with means of 3.9 and 3.7, respectively. However, strong disagreement to including tasks T4 and T7 was recorded, representing 12% and 6% of the responses for those tasks respectively.

Combining the scores for the four validation-oriented questions (V = (Q2+Q3+Q5+Q6)/4) suggests that task T5, Traction and Monopolar Dissection, was expected to be the most readily validated (V=4.4) followed by T4 and T6, with V=3.99 and V=3.96 respectively. The task suggested least easily validated is T2, with V=3.76 and 13% of respondents indicating strong disagreement across all four validation questions.

Comparing scores of the combined validation-oriented questions for one-handed tasks (T1,T2,T3) and bimanual tasks (T4,T5,T6), shows bimanual tasks were expected to be significantly more likely to be validated (P < 0.001, Z=-4.31, Mann-Whitney test, one-tail).

Discussion

Whether simulation can provide useful training and assessment for the complex knowledge and manual skills involved in surgical uses of energy has yet to be shown. This survey represents a first step in identifying such a fundamental set of tasks. The identification of candidate tasks and the expert judgment of these tasks on a range of assessment-related criteria provides initial guidance to those developing new approaches to teaching and assessing the use of energy in surgery. The challenges lie in developing the assessment metrics and well as valid pass/fail scores, analogous to the process developed for FLS.

In this study we have focused on hands-on tasks. Learning electrosurgical skills is a complex process that also requires knowledge of tool settings and the injuries that can result from incorrect use of such settings [15]. These cognitive skills can be incorporated within the tasks proposed here. The next step is the development of specific, hands-on cognitive components and establishing their validity within a simulation (VR or physical) environment.

Figure 3.

Responses to each task for question Q2: “This task represents, or acts as a surrogate for, tasks performed in the OR.”

Figure 4.

Responses to each task for question Q3: “Skilled performance on this task is a necessary component of safe and effective use of electrosurgery in the OR.”

Figure 5.

Responses to each task for question Q4: “Knowledge alone is insufficient to master this task.”

Figure 6.

Responses to each task for question Q5: “This task is indicative of important technical skills for using electrosurgery in the OR.”