Abstract
Background:
ESD is an endoscopic technique for en bloc resection of gastrointestinal lesions. ESD is a widely-used in Japan and throughout Asia, but not as prevalent in Europe or the US. The procedure is technically challenging and has higher adverse events (bleeding, perforation) compared to endoscopic mucosal resection. Inadequate training platforms and lack of established training curricula have restricted its wide acceptance in the US. Thus, we aim to develop a Virtual Endoluminal Surgery Simulator (VESS) for objective ESD training and assessment. In this work, we performed task and performance analysis of ESD surgeries.
Methods:
We performed a detailed colorectal ESD task analysis and identified the critical ESD steps for lesion identification, marking, injection, circumferential cutting, dissection, intraprocedural complication management and post-procedure examination. We constructed a hierarchical task tree that elaborates the order of tasks in these steps. Furthermore, we developed quantitative ESD performance metrics. We measured task times and scores of 16 ESD surgeries performed by four different endoscopic surgeons.
Results:
The average time of the marking, injection, and circumferential cutting phases are 203.4 (σ:205.46), 83.5 (σ: 49.92), 908.4 sec. (σ: 584.53) respectively. Cutting the submucosal layer takes most of the time of overall ESD procedure time with an average of 1394.7 sec. (σ: 908.43). We also performed correlation analysis (Pearson’s test) among the performance scores of the tasks. There is a moderate positive correlation (R=0.528, p=0.0355) between marking scores and total scores, a strong positive correlation (R=0.7879, p=0.0003) between circumferential cutting and submucosal dissection and total scores. Similarly, we noted a strong positive correlation (R=0.7095, p=0.0021) between circumferential cutting and submucosal dissection and marking scores.
Conclusions:
We elaborated ESD tasks and developed quantitative performance metrics used in analysis of actual surgery performance. These ESD metrics will be used in future validation studies of our VESS simulator.
Keywords: Endoscopic training, Endoscopic Submucosal Dissection, Colorectal Cancer, ESD
1. Introduction
The risk of developing colorectal cancer during the lifetime is 1 in 21 for men and 1 in 23 for women [1]. According to the World Center Research Fund International [2], with an average of 1.4 million new cases per year, colorectal cancer is the third most commonly encountered cancer type in the world. In the U.S., it is the second leading cause of death, with 14.8 deaths per 100,000 men and women according to the reports published by National Cancer Institute [3]. The estimated new colon cancer diagnoses in 2017 is 135,430, which is equivalent to 8% of all new cancer diagnoses. The estimated colon cancer-related death in 2017 is 50,260 people [3]. Unless timely steps in terms of screening, diagnosis and treatment are taken, colorectal cancer will continue to be a major health challenge to the U.S. and the world.
There are several techniques for the management of pre-malignant colorectal polyps and superficial colorectal cancer (i.e., not penetrating the submucosa). For mucosal-based lesions, the most widely used technique in the United States currently is endoscopic mucosal resection (EMR). En bloc resection can often be accomplished for lesions less than 20 mm [4] with EMR. For lesions larger than 20mm or for non-standard shaped lesions (i.e., over a fold), complete removal often requires piecemeal resection. Piecemeal resection can obscure resection borders on the pathologic specimen and can make a pathological diagnosis of an R0 resection difficult. Therefore, ESD was developed in order to dissect larger lesions (>20mm) with an en bloc resection, allowing for more complete pathological review [5–9].
In general, ESD has a longer procedure time compared to EMR [4, 10]. Compared with EMR, ESD has higher rates of procedural complications (bleeding and perforation) [11]. While it causes higher risks of perforation, ESD has a lower rate of delayed bleeding [6], higher en bloc resection rate, and lower local recurrence rate [10, 12]. Furthermore, due to the en bloc resection of the lesion, it is easier for a complete pathological analysis of the tumor by the pathologist [13].
ESD for colorectal tumors is a complex procedure due to the relatively thin colonic wall and difficulties keeping the position of the endoscope stable due to the shape, folds and contractions of the colon [14]. Given the relative difficulty of ESD compared to EMR, an expert endoscopist is required to have extensive training in the procedure. This study is motivated by the established need for a standardized method of training and evaluating colorectal ESD.
1.1. ESD Training in the United States
Training in ESD has been difficult to coordinate for practitioners in the U.S. In Asia, most endoscopists start ESD by observing and performing ESD of superficial gastric tumors. Given the dramatic differences in incidence of gastric cancer and resultant differences in gastric cancer screening policies, endoscopists in the U.S. generally do not encounter many superficial gastric cancers in clinical practice [15]. In addition, guidelines from US-based GI and endoscopy societies provide little guidance [16].
Conventional training methods for ESD are mostly based on animal models [17–20] and patient-based observed and proctored training. M. Fujishiro et al. [21] derived a labor-intensive proposal of training for ESD with multiples steps. Ex-vivo simulators pose some problems with translatability (difficulty with electrosurgical settings in desiccated specimens, inability to simulate intraprocedural bleeding) and set-up. While in-vivo simulations with live animal models [22] are realistic, they are not widely used due to ethical concerns and associated cost. With the use of animal models, there is a learning curve of 15–30 cases for gastric ESD [18, 22]. Given the difficulties in training noted above, there is tremendous opportunity for another modality of ESD training in the United States.
Alternative training methods such as virtual reality (VR)-based simulators may be an effective adjunct to a training regimen for ESD. A high-fidelity VR simulator would allow for skills training in both the advanced psychomotor and cognitive skills aspects of ESD. The simulator can be used to train not only endoscopists but also other operating room (OR) personnel (endoscopy nurses and technicians) in all of procedural aspects [23]. The advantages of endoscopy-based VR simulators can be summarized as: (1) no ethical dilemma with initial training through an apprentice model on human subjects, with no risk of harm to the patient or malpractice, since no-human subject is involved [24]; (2) repeatable on different tumor-types and sizes; (3) affordable compared to recurrent costs of other animate and inanimate training models [25, 26]; (4) potentially reduced training time with a steeper learning curve once human ESD work commences [27]; (5) and quantifiable measurement for both assessment and training [28].
VR simulators have become widespread in the last decade to improve psychomotor learning outcomes in laparoscopic surgery and natural orifice transluminal endoscopic surgery (NOTES) [28, 29]. However, there is no virtual reality-based simulation for colorectal ESD. The VR-development for ESD is challenging given the constricted and detailed endoscopic operating space, large deformation of tissue, complicated modeling (e.g. various tumor models), and complex physics-based interactions (e.g. incision, cutting, dissection, etc.) between different kinds of endoscopic accessories and colon tissue [28].
Our ultimate goal is to design and develop a high-fidelity VR-based colorectal ESD simulator using haptics technology. We aim to create a teaching and assessment platform for structured ESD training with objective feedback. We hypothesize that this platform could significantly improve ESD training and equip trainees with a high level of proficiency in the ESD steps, which would prepare them for initial proctored training in humans. In order to achieve this, the aim of this current study is to perform a hierarchical task analysis (HTA) of the ESD procedure, derive metrics for quantitative performance measurement of actual endoscopic steps and to perform a time analysis of the actual ESD procedures in an attempt to develop authentic measures.
2. Materials and Methods
2.1. Task Analysis Methods
HTA, in the context of an overall procedural analysis, is a method that details the steps of a procedural process from beginning to end. The specific aim of this hierarchical expression is to identify crucial details in each step of the procedure and allow the creation of metrics for each step. These details will then be translated and programmed into the VR simulation platform. We performed a thorough literature review of the colorectal ESD procedure and had discussions with expert endoscopists from the US, Japan and Korea during the analysis. As a result, we detailed the ESD phases and tasks performed in each phase. The task tree, the systematic representation of the tasks and their order, and objective grading metrics were created as part of our task analysis.
2.2. Grading Metrics
Based on the tasks that were discovered in the HTA, we developed grading metrics for each task and subtask of each ESD phase with the input of expert endoscopists. We utilized a Likert-like scale for task scoring. The highest score point of 3–5 is given for the best/optimal action. A suboptimal action is given a point value of 1–3, depending on the importance and consequence of the action. A trainee receiving a 0 score in a task signifies that no proper action is taken. In some metrics (such as in bleeding management criteria), a 0 score denotes an overall failing of the training exercise. This is termed a “kill switch” for the training procedure. The grading system was derived by consensus from expert endoscopists.
2.3. Time and Performance Analysis
Based on the HTA, we created timing guidelines with specified actions for “start” and “end” for each specified task and subtask. This was created by expert consensus. Sixteen videos of ESD procedures performed by four different endoscopists were analyzed. In the time analysis of the ESD videos, three raters reviewed each video individually and independently using VLC and Windows media players. All raters were also debriefed with the evaluation guidelines including possible rare cases. It was noted that multiple subtasks (such as hemostasis or injection) could be recurrent events in multiple phases of the procedure. The raters were trained to time these tasks within duration of phases. Several segments or clips of the ESD videos were timed and scored with raters prior to the study to increase the rater training. Once raters confirmed that they have a clear understanding about HTA and all ESD steps in detail and timing and scoring guidelines, they timed all phases’ and subtasks’ start and end time and rated scores (based upon scoring criteria devised in Results section 3.2 “Grading Metrics”) for all the videos. We performed inter-rater reliability (IRR) for all phases using Fleiss’ kappa [30, 31] to evaluate the agreement among raters.
Prior to our study, we hypothesized that there may be relation between the task duration and scores from the devised grading metrics. This stems from the common observation of relation noted in the literature that skills and experience level of the surgeon affects the total surgery time, time spent in a task and performance score [32, 33]. In order to quantify the relation, we performed Pearson’s correlation tests between times and scores. We used in RStudio version 1.0.153 with R version 3.4.1 for the statistical analysis.
3. Results
3.1. Task Analysis
Six major phases for the ESD procedure were identified: (1) procedural preparation, (2) coagulative marking circumferentially outside of the borders of the lesion, in order to improve visualization of the boundaries of the tumor during the procedure, (3) injection of a solution into the submucosal space to lift the lesion and create a protective cushion [34] for cutting, (4) circumferential cutting around the lesion using endoscopic electrocautery knives which have been described elsewhere (Matsui et al. [35]), (5) submucosal dissection by using an electrosurgical knife, and (6) evaluation of the colon for bleeding and perforations throughout the procedure and at the conclusion of the procedure. Furthermore, we described all the tasks performed for each of six ESD phases. We determined the necessary tasks and optional/selective tasks for each phase. The hierarchy of the phases and execution order of tasks form the hierarchal task tree. The tasks trees from preparation to evaluation phase are found in Figure 1–6. In addition to the phases, we created task trees for perforation (Figure 7) and bleeding management (Figure 8) which are commonly performed sets of tasks but not phases per se. In these tasks trees, normal processes are illustrated in rectangle and decisions are shown in diamonds. Arrows show the progression of the tasks in the order presented and they should be performed in order until the completion of the tasks in the tree. Some surgical tasks are recurrent where the tasks should be repeated until the task’s goal is met. These repetitive tasks are illustrated with an arrow that directs the execution order back to the former steps.
Figure 1:
Steps in preparation phase
Figure 6:
Steps in evaluation phase
Figure 7:
Steps in perforation management
Figure 8:
Steps in bleeding management
Preparation Phase
The preparation step (Figure 1) is the first step of ESD. Before the procedure, tools and injection solutions must be prepared. The type of the endoscope is selected according to the location of the lesion; an upper esophagogastroduodenoscope may be used if the lesion is located in the left colon or rectum, whereas a pediatric or adult colonoscope is most likely used if it is located in the right colon [12]. In the United States, all ESD procedures are performed at a minimum with procedural moderate sedation and often with general anesthesia depending on the availability of the anesthesiologist, practice and procedure specific considerations (e.g., estimated duration of procedure, estimated difficulty of procedure, co-morbidities of the patient). In ESD, a carbon dioxide (CO2) insufflation system is used to reduce the patient’s pain from colonic gas retention and risk of peritoneal distention should a perforation occur during the procedure. Furthermore, a clear endoscopic cap is used at the distal end of the scope. This helps to facilitate traction while dissecting the submucosal space. Finally, ideally the patient will be positioned so that the lesion will be in an anti-gravity position for increasing the lifting and elevation during the dissection and to aid in dissection [5, 36–39].
Marking Phase
The marking phase in the task tree is illustrated in Figure 2. In the beginning of the marking step, the selected scope is inserted through the anus and navigated through the colon until the lesion is located. There are four potential methods for detecting and confirming the aspects and pathological components of the lesion: (1) high-definition white light, (2) narrow band imaging, (3) near focus/magnification, and (4) chromoendoscopy. The marking step could be completely skipped if the lesion is clear and visible after chromoendoscopy. Depending on the case, a physician may choose to utilize multiple methods. After successful detection and evaluation, the next step is to clean the lesion and surrounding mucosa with a water jet. In some cases, small debris may be present and must be removed with the suction function of the endoscope. Thereafter, the knife, depending on the endoscopist’s preference, is inserted into the working channel of the scope. Then, the electrosurgical unit (ESU) should be confirmed to be set for the desired tissue effect and at the discretion of the expert. The recommended settings for each ESD phase specific to ESU can be found in [5]. After setting the ESU, the cutting surface/blade of the knife is exposed in the colonoscopy view.
Figure 2:
Steps in marking phase
Although there are various techniques for marking the mucosal layer, two techniques are very common; the first technique for marking is performed by placing the knife onto the surface of the mucosa and creating coagulation marks approximately 2–3 mm apart and 5mm peripheral to the lesion. The second technique is to use an argon plasma coagulation (APC) probe in close proximity to mucosa and use the PULSED APC mode on the ESU.
Injection Phase
Based on the primary ESD knife being used, the injection phase will vary (Figure 3). If the endoscopist is not using a multi-purpose knife such as the HybridKnife, then the current marking tool (knife or APC probe) is retracted and removed from the working channel of the scope and an injection needle is inserted into the working channel. After the injection needle catheter is present into the colonic lumen, the needle tip is exposed, and the needle tip is inserted into the submucosal layer. Injection is performed until sufficient elevation is achieved. After sufficient elevation is achieved, the needle tip is retracted. Note that it is not necessary to perform last two steps if the instrument is a dual-purpose knife. By using a multi-purpose knife, injection of the lifting solution can be performed at any time without switching instruments.
Figure 3:
Steps in injection phase
Circumferential Cutting Phase
The first step in this phase is inserting the selected knife into the working channel of the scope (see Figure 4). Then, the ESU should be set to the correct specification. Once the cutting mode and power settings in the ESU are adjusted, the cutting surface/blade of the knife is exposed. The next step is to perform the marginal incision around the lesion until the submucosal fibers are exposed. This is accomplished by first penetrating through the mucosa (in a lesion already injected with submucosal injection) while cutting; this is performed carefully so as not to damage the underlying muscle layer – this is the initial cut. After this first incision is made through the mucosa, a cut around the circumference of the lesion is made. Oftentimes, the circumferential cut will be performed in stages (i.e., not entirely circumferential initially) so as to preserve the submucosal cushion for a longer period of time. Generally, this cutting is performed just outside the perimeter of the previous markings, with care not to cut into the lesion itself. Throughout the circumferential cut, the depth of cutting should be monitored, to ensure cutting is not too deep (into muscle) or too shallow (not entirely through mucosa). A high quality circumferential cut will help the efficiency of the later steps of the ESD. In the case of inadequate lesion elevation, the injection step should be repeated. When bleeding occurs, the site is examined, washed and, if necessary, hemostasis is performed.
Figure 4:
Steps in circumferential cutting phase
Submucosal Dissection Phase
Submucosal dissection (Figure 5) is the most critical step of the ESD procedure. Inserting the knife, setting the ESU and exposing the knife’s tip are performed the same as in the previous steps. Then, the submucosal layer is dissected by manipulating the knife below the lesion. Again, similar to the previous phase, the injection step should be repeated until elevation is sufficient and also cleaning or hemostasis is repeated when bleeding occurs. Otherwise, dissection is advanced until the lesion separates from the underlying colon and is completely resected. The approach to the lesion will need to be altered throughout the dissection phase. It is important that all cutting is performed within the submucosal plane, preferably closer to the muscle side than the mucosa side, with avoidance of cutting the muscle. Small and precise movements are of paramount importance. During the dissection, small veins and arteries will be encountered penetrating the submucosal space. These should be prophylactically managed with either the coagulation setting of the knife or with the coagrasper device. The clear endoscopic cap is used to facilitate traction of the submucosal space while cutting. After complete resection, the lesion is removed with appropriate graspers, or by suction into the endoscopic cap. Although it is not common, a snare can be used as a salvage technique to remove the lesion if dissection becomes tedious.
Figure 5:
Steps in submucosal dissection phase
Evaluation Phase
The last phase of the ESD is a colonoscopic review (see Figure 6) performed in order to detect possible perforations, residual lesion or visible vessels after the removal of lesion. When perforation (see Figure 7) occurs, CO2 should be decreased and consideration for peritoneal decompression should ensue. Often with large perforations, the intraluminal colon will no longer insufflate even with regular flow. Bleeding (see Figure 8) is treated with endo-clips or hemostasis tools (e.g., coagrasper, APC, bipolar probe). At the end of the procedure, pinning the specimen on a mounting board allows for numerous objectives to be met in addition to analyzing the dimensions of the resection, including allowing for proper pathological fixation without distortion and also to examine the surface of the lesion for en bloc status.
3.2. Grading Metrics
Metrics for each task and subtask in the HTA and grading schema were determined by expert consensus (Table 1). There are some common and repeated metrics in miscellaneous scores such as bleeding intervention times, address of knife angle to the dissection plane, knife handling, cleaning blood from the field, etc. These tasks need to be performed during most of the phases rather than a specific phase or task. For some of these metrics, the criteria might occur multiple times such as clearing the visibility or hemostasis times. In these cases, the endoscopist will be assessed only once and the minimum score will be recorded. For example, if the endoscopist requires more than 60 seconds for hemostasis while using coagraspers or doing coagulation hemostasis, or more than 120 seconds for hemostasis by hemoclip in one hemostasis scenario, s/he will receive one point. In addition, there are no acceptable metrics in timing performance in interventional flexible endoscopy available in the literature. The subject matter experts determined timing metrics based on expert opinion.
Table 1.
Grading metrics for all main phases and subtasks
| PREPARATION | Position of Patient | |
| Lesion is in an anti-gravity position | 3 | |
| Lesion is not in an anti-gravity position | 0 | |
| Selection of the Endoscope | ||
| Correct endoscope | 3 | |
| Incorrect endoscope | 0 | |
| Status of the distal cap attachment | ||
| Clear distal cap | 3 | |
| No distal cap | 0 | |
| MARKING | Insertion of HD Endoscope | |
| Navigate to the lesion | 2 | |
| Advance beyond the lesion | 1 | |
| Stopping proximal to the lesion | 0 | |
| Identification of the lesion within 60 seconds | ||
| Identified | 3 | |
| Not identified | 0 | |
| Cleaning the lesion and surrounding mucosa by water jet | ||
| Washed satisfactorily | 3 | |
| There is still debris | 1 | |
| If there is small debris around the colon | ||
| Suctioned completely | 3 | |
| Not suctioned completely | 1 | |
| Marking - distance between each mark | ||
| 2–3mm (ideal margin) | 5 | |
| 3–5mm (acceptable) | 3 | |
| 0–1mm or > 5mm (poor margin) | 1 | |
| No marking | 0 | |
| Peripheral zone of lesion marking | ||
| 5–7mm | 5 | |
| More than 7 mm | 3 | |
| Less than 5mm | 1 | |
| Knife position into mucosa on marking step | ||
| Close to mucosa (in PULSED APC mode) or 0.5mm inside to mucosa (in soft coagulation mode) | 5 | |
| Less than 0.5mm (soft coagulation mode) | 3 | |
| Greater than 0.5mm (soft coagulation mode) | 1 | |
| Chromoendoscopy is used | ||
| Spraying enough dye | 5 | |
| Spraying insufficient dye | 1 | |
| INJECTION | Injection of solution | |
| Enough mL solution | 5 | |
| Too much solution | 3 | |
| Not enough solution | 1 | |
| Not injected any solution | 0(fail) | |
| Lesion elevation | ||
| Sufficient Elevation (enough fluid cushion) | 5 | |
| Not enough elevation (not enough fluid cushion) | 3 | |
| CIRCUMFERENTIAL CUTTING AND SUBMUCOSAL DISSECTION | Choosing Knife type | |
| Correct Knife | 5 | |
| Incorrect tool* | 0(fail) | |
| Initial Incision | ||
| Start circumferential cutting 1 −2 mm outer distal proximity of marking points | 5 | |
| Start circumferential cutting greater than 2 mm outer distal proximity of marking points | 3 | |
| Start circumferential cutting on marking points | 1 | |
| Start circumferential cutting 1–2 mm inner distal proximity of marking points | 0(fail) | |
| Dissection | ||
| Cutting middle zone or muscle zone of the submucosal layer | 5 | |
| Cutting while not having adequate cushion | 3 | |
| Cutting the muscle (muscularis propia) layer | 1 | |
| Cutting the mucosa layer inadvertently | 0 | |
| Perforation of the lesion during the dissection | ||
| No perforation | 5 | |
| Perforation | 0 | |
| Resecting the lesion | ||
| En bloc resection | 5 | |
| Piecemeal complete resection | 3 | |
| Incomplete resection | 0 | |
| Removing the lesion | ||
| Lesion successfully removed | 3 | |
| Failure to remove the lesion | 0 | |
| EVALUATION | Colonoscopic review | |
| Detecting perforations | 5 | |
| Not detecting perforations | 0 | |
| MISCELLANEOUS | Bleeding intervention time+ | |
| Less than 60 secs (coagrasper, doing coagulation hemostasis) or 120 secs (hemoclip) | 3 | |
| More than 60 secs (coagrasper, doing coagulation hemostasis) or 120 secs (hemoclip) | 1 | |
| No intervention | 0(fail) | |
| Perforation: Avoidance | ||
| Injecting more solution | 3 | |
| Not injecting more solution | 0 | |
| Visible bleeding location | ||
| Hemostasis | 5 | |
| No Hemostasis | 0(fail) | |
| Non-visible bleeding location | ||
| Injecting solution | 5 | |
| Not injection | 0 | |
| Cleaning the blood | ||
| Spray the water | 3 | |
| Not spray the water | 0 | |
| Knife handling | ||
| Smoothness and Gentleness in tool handling | 5 | |
| Discrete motions in tool handling | 3 | |
| Aggressive tool handling | 0 | |
| Knife angle to the dissection plane | ||
| 15–35 degree with respect to the dissection surface (Except IT knife) | 3 | |
| Degree [5–15] or [35–45] with respect to the dissection surface | 1 | |
| Other angles | 0(fail) | |
| Knife exposing | ||
| Expose the knife on accurate step | 3 | |
| Expose the knife on another step | 0 | |
| Knife Retracting | ||
| Retract the knife on accurate step | 3 | |
| Retract the knife on another step | 0 | |
| Position of HD Endoscope | ||
| Near the lesion | 2 | |
| Far from the lesion | 1 | |
| Tasks execution order | ||
| Completion of tasks executed in order | 2 | |
| Completion of tasks executed not in order | 0 |
This metric is attributed to instrument choices made that is not relevant for a specific phase.
The subject matter experts determined timing metrics based on expert opinion.
3.3. Time and Performance Analysis
The specified timing guidelines were created for all phases and subtasks (see Table 2) by expert consensus. These specify criteria when raters need to start and end time for each phase and tasks. This is needed to avoid ambiguity and eliminate inconsistency among the videos and reviewers.
Table 2.
Start and end times
| Phase | Tasks | |
|---|---|---|
| Marking | Insertion of the HD Endoscope | |
| Start Time Event | Insertion of the HD endoscope into anus | |
| End Time Event | Seeing the lesion | |
| Detection of the lesion | ||
| Start Time Event | Seeing the lesion | |
| End Time Event | Exposing the knife | |
| Washing the lesion and surrounding mucosa | ||
| Start Time Event | Starting flushing water | |
| End Time Event | Stopping flushing water | |
| Suction the debris | ||
| Start Time Event | Starting suction of debris | |
| End Time Event | Stopping suction of the debris | |
| Spraying dye | ||
| Start Time Event | Insertion of the spraying tool | |
| End Time Event | Finishing spraying dye | |
| Marking the margin of the lesion | ||
| Start Time Event | Insertion of the knife into mucosa | |
| End Time Event | Completion of the circumferential marking around the lesion | |
| Cleaning the bleeding if occurs | ||
| Start Time Event | Start spraying water | |
| End Time Event | Stop Spraying water | |
| Hemostasis if bleeding occurs | ||
| Start Time Event | Insertion of the hemostasis tool into the submucosa | |
| End Time Event | Removing the tool from the submucosa | |
| Injection | Injection | |
| Start Time Event | Insertion the injection needle into the endoscope | |
| End Time Event | Retracting the injection needle | |
| Circumferential Cutting | Cutting the mucosal layer | |
| Start Time Event | Insertion of the knife into the mucosal layer surrounding the lesion | |
| End Time Event | Removing the knife from the mucosal layer when circumferential incision is completed | |
| Cleaning the bleeding if occurs | ||
| Start Time Event | Spraying water | |
| End Time Event | Stop flushing | |
| Hemostasis if bleeding occurs | ||
| Start Time Event | Insertion of the hemostasis tool into the submucosa | |
| End Time Event | Removing the tool from the submucosa | |
| Injection | ||
| Start Time Event | Same as previous Injection Step | |
| End Time Event | Same as previous Injection Step | |
| Submucosal Dissection | Cutting the submucosal layer | |
| Start Time Event | Insertion of the knife into the submucosal layer surrounding the lesion | |
| End Time Event | Resection of the lesion | |
| Cleaning the bleeding if occurs | ||
| Start Time Event | Spraying water | |
| End Time Event | Stop flushing | |
| Hemostasis if bleeding occurs | ||
| Start Time Event | Insertion of the hemostasis tool into the submucosa | |
| End Time Event | Removing the tool from the submucosa | |
| Injection | ||
| Start Time Event | Same as previous Injection Step | |
| End Time Event | Same as previous Injection Step | |
| Removing the lesion | ||
| Start Time Event | Insertion of the grasping tool | |
| End Time Event | Removing from the endoscope | |
| Evaluation | Colonoscopic review | |
| Start Time Event | Identification of the colon wall around the resected area | |
| End Time Event | Removing the endoscope from colon | |
| Cleaning the bleeding if occurs | ||
| Start Time Event | Spraying water | |
| End Time Event | Stop flushing | |
| Hemostasis if bleeding occurs | ||
| Start Time Event | Insertion of the hemostasis tool into the submucosa | |
| End Time Event | Removing the tool from the submucosa | |
Sixteen ESD videos were reviewed. These videos are internal colonoscopy recordings of live ESD procedures except one that was performed on a porcine colon. In only five videos the “Removing the lesion” subtask was recorded. Similarly, the “Evaluation” phase was only recorded in five videos. In one of the videos, the marking phase was not recorded. In these cases, the timings and scores for these tasks were not considered in the statistical analysis. All videos started at the marking phase; therefore, time analysis and scores are not available for the preparation phase.
According to time analysis performed, the submucosal dissection phase is the longest phase with maximum time variation (see Table 3 and Figure 9). The average time of the marking phase (see Figure 10) includes detection of the lesion, marking the lesion, washing the lesion and spraying the dye times (if performed). Circumferential cutting has three subtasks (see Figure 11): cleaning the bleeding, hemostasis, injection tasks. We determined that the resolution of the bleeding task time takes longer time in the circumferential cutting phase compared to the submucosal dissection time (See Figure 9 and Figure 10).
Table 3.
Time analysis of phases
| Phases | Avg. (sec) | Min (sec) | Max (sec) | σ (sec) | |
|---|---|---|---|---|---|
| Marking | 203.4 | 23 | 885 | 205.46 | |
| Injection | 83.5 | 24 | 212 | 49.92 | |
| Circumferential Cutting | 908.4 | 301 | 2390 | 584.53 | |
| Subtasks | Cleaning the bleeding | 24.9 | 1 | 97 | 29.76 |
| Hemostasis | 28.8 | 5 | 75 | 22.98 | |
| Injection | 104.7 | 8 | 245 | 75.04 | |
| Submucosal Dissection | 1394.7 | 75 | 3196 | 908.43 | |
| Subtasks | Cleaning the bleeding | 15.1 | 1 | 41 | 14.03 |
| Hemostasis | 28.4 | 1 | 90 | 26.70 | |
| Injection | 117.6 | 15 | 454 | 128.73 | |
| Total time of phases | 2327.6 | 856 | 6297 | 1399.28 | |
| Total time of videos | 3049.8 | 1020 | 9553 | 2021.20 | |
Figure 9:
Time analysis of submucosal dissection phase
Figure 10:
Time analysis of marking phase
Figure 11:
Time analysis of circumferential cutting phase
We also computed scores with respect to our ESD metrics presented in Table 3. These scores were computed by the same independent raters that performed the task timing of the videos. Based on our results, the average score of the videos is 79.3 (max: 93.8 points, min: 49.0 points) (see Figure 12). The majority of the attainable scores belong to the category of the circumferential cutting and submucosal dissection scores and miscellaneous scores. The miscellaneous scores are mostly referred to the tasks that can be performed at any phase (e.g. cleaning the blood, tool handling, identifying bleeding location) so they do not fall under one specific phase of the surgery.
Figure 12:
Average scores of all phases
We computed the IRR for 3 raters of the 16 videos (4 phases evaluated in videos) for agreement in scores. The agreement rate of the preparation phase is k=1.00. There is a substantial agreement (k=0.69) for the marking phase. The discrepancy in scores are primarily due to the difficulty to assess the exact marking distance (in mm) from the videos. There is excellent agreement (k=1.00) for the injection phase. For the circumferential cutting and submucosal dissection phases, there is a substantial agreement (k=0.76). For the miscellaneous scores, the agreement rate is an excellent agreement (k=0.84). In general, we conclude that the raters have a high level of inter-rater reliability on the average scores for the ESD tasks.
We categorized Pearson’s correlation test results in time-score correlation (see Table 4) and score-score correlation of phases (see Table 5). Our results demonstrate that the majority of the task times and task scores are negatively related.
Table 4.
Time - score correlations
| Time | Score | R | p | Correlation |
|---|---|---|---|---|
| Cleaning the bleeding in circumferential cutting | Marking phase | -0.6159 | 0.0252 | Negative Moderate |
| Cleaning the bleeding in submucosal dissection | Marking phase | -0.6824 | 0.0298 | Negative Moderate |
| Detection of the lesion on marking phase | Circumferential cutting and submucosal dissection | -0.6921 | 0.0061 | Moderate Negative |
| Cutting the mucosal layer in circumferential cutting phase | Circumferential cutting and submucosal dissection | -0.6323 | 0.0086 | Moderate Negative |
| Detection of the lesion in | Total | -0.6439 | 0.0131 | Moderate |
| marking phase | Negative | |||
| Marking phase | Total | -0.5139 | 0.0505 | Negative Moderate |
| Cutting the submucosal layer in submucosal dissection phase | Marking phase | 0.0095 | 0 | No Correlation |
Table 5.
Score - score correlations
| Score | Score | R | p | Correlation |
|---|---|---|---|---|
| Marking | Total | 0.528 | 0.0355 | Moderate Positive |
| Circumferential cutting and submucosal dissection | Total | 0.7879 | 0.0003 | Strong Positive |
| Miscellaneous | Total | 0.8738 | <0.00001 | Strong Positive |
| Circumferential cutting and submucosal dissection | Marking | 0.7095 | 0.0021 | Strong Positive |
4. Discussion
In this study, we performed a hierarchical task analysis and task trees to determine the individual integral steps to the colonic ESD procedure. Furthermore, we produced a scoring metric to evaluate colonic ESD procedures. Finally, we independently evaluated colonic ESD procedure videos to determine the timing of each component of the HTA and to administer the scoring metric for these procedures.
In our scoring metrics, higher points in Likert scale are given for the optimal or desired actions that are designated to reflect better surgery performance (see Section 2.2 and 3.2). Negative correlations in our results (Table 4 & 5) convey that the endoscopists who have higher scores in our metrics generally complete the procedure in a shorter period of time than the ones with lower scores. This may be attributed to a.) the skill level of surgeon or b.) the difficulty of the specific lesion, that can result in more rapid task completion times overall.
We identified that there is a strong positive correlation between miscellaneous scores and total scores, circumferential cutting and submucosal dissection scores and total scores. Endoscopists that have a high score from miscellaneous tasks tend to have a high score from circumferential cutting and submucosal dissection tasks as well.
One limitation of the study is that we analyzed a total of 16 videos from four endoscopists. It is possible that validating the scoring metric with a more extensive study that involves more endoscopists and more procedures would provide more robust procedural data. Furthermore, colorectal ESD is a dynamic field, with frequent modifications to the tools available and techniques employed. There can be many variations including various assistive devices for traction or colonic stability, or dissection strategies similar to the pocket technique.
In conclusion, we performed a HTA and developed a rubric for performance metrics for ESD. Based on the HTA and metrics, we carried out time and performance analysis of actual ESD videos. We presented correlations between task times and scores. We will integrate the steps and sub-steps of the ESD procedure and relative timing of each step (discovered in the procedural video analysis) as the foundation for the software and hardware development for our VR ESD simulator. Furthermore, the performance metrics will be integrated into the simulation environment to provide the trainee with real-time feedback via quantitative performance measurement. The identified correlations will be compared with our simulator as a future work.
Acknowledgment
This project was made possible by the Arkansas INBRE program, supported by a grant from the National Institute of General Medical Sciences, (NIGMS), P20 GM103429 from the National Institutes of Health. This project was also supported by National Institutes of Health (NIH) Grant NIH/NHLBI 1R01HL119248‐ 01A1, NIH/NIBIB 2R01EB005807, 5R01EB010037, 1R01EB009362, and 1R01EB014305.
Footnotes
Disclosures
Berk Cetinsaya has nothing to disclose.
Mark A. Gromski has nothing to disclose.
Sangrock Lee has nothing to disclose.
Zhaohui Xia has nothing to disclose.
Doga Demirel has nothing to disclose.
Tansel Halic reports grants from National Institutes of Health, during the conduct of the study.
Coskun Bayrak has nothing to disclose.
Cullen Jackson reports grants from NIH via subcontract from RPI, during the conduct of the study.
Suvranu De grants from National Institutes of Health, during the conduct of the study.
Sudeep Hegde reports grants from National Institutes of Health, during the conduct of the study; other from State University of New York at Buffalo, outside the submitted work.
Jonah Cohen has nothing to disclose.
Mandeep Sawhney has nothing to disclose.
Stavros N. Stavropoulos has nothing to disclose.
Daniel B. Jones reports personal fees from Allurion, outside the submitted work.
References
- 1.Key Statistics for Colorectal Cancer. https://www.cancer.org/cancer/colon-rectal-cancer/about/key-statistics.html. Accessed 14 Aug 2017
- 2.Worldwide data | World Cancer Research Fund International. https://www.wcrf.org/int/cancer-facts-figures/worldwide-data. Accessed: 28 May 2018
- 3.Cancer of the Colon and Rectum - Cancer Stat Facts. https://seer.cancer.gov/statfacts/html/colorect.html. Accessed 13 Apr 2017
- 4.Kantsevoy SV, Adler DG, Conway JD, Diehl DL, Farraye FA, Kwon R, Mamula P, Rodriguez S, Shah RJ, Song LMWK, Tierney WM (2008) Endoscopic mucosal resection and endoscopic submucosal dissection. Gastrointest Endosc 68:11–18 . doi: 10.1016/j.gie.2008.01.037 [DOI] [PubMed] [Google Scholar]
- 5.Yamamoto K, Michida T, Nishida T, Hayashi T, Naito M, Ito T (2015) Colorectal endoscopic submucosal dissection: Recent technical advances for safe and successful procedures. World J Gastrointest Endosc 7:1114–1128 . doi: 10.4253/wjge.v7.i14.1114 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Fujihara S, Mori H, Kobara H, Nishiyama N, Matsunaga T, Ayaki M, Yachida T, Morishita A, Izuishi K, Masaki T (2014) Current Innovations in Endoscopic Therapy for the Management of Colorectal Cancer: From Endoscopic Submucosal Dissection to Endoscopic Full-Thickness Resection. BioMed Res Int 2014: . doi: 10.1155/2014/925058 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Saito Y, Yamada M, So E, Abe S, Sakamoto T, Nakajima T, Otake Y, Ono A, Matsuda T (2014) Colorectal endoscopic submucosal dissection: Technical advantages compared to endoscopic mucosal resection and minimally invasive surgery. Dig Endosc Off J Jpn Gastroenterol Endosc Soc 26 Suppl 1:52–61 . doi: 10.1111/den.12196 [DOI] [PubMed] [Google Scholar]
- 8.Shono T, Ishikawa K, Ochiai Y, Nakao M, Togawa O, Nishimura M, Arai S, Nonaka K, Sasaki Y, Kita H (2011) Feasibility of Endoscopic Submucosal Dissection: A New Technique for En Bloc Resection of a Large Superficial Tumor in the Colon and Rectum. Int J Surg Oncol 2011:e948293 . doi: 10.1155/2011/948293 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wang J, Zhang X-H, Ge J, Yang C-M, Liu J-Y, Zhao S-L (2014) Endoscopic submucosal dissection vs endoscopic mucosal resection for colorectal tumors: A meta-analysis. World J Gastroenterol WJG 20:8282–8287 . doi: 10.3748/wjg.v20.i25.8282 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hotta K, Oyama T, Shinohara T, Miyata Y, Takahashi A, Kitamura Y, Tomori A (2010) Learning Curve for Endoscopic Submucosal Dissection of Large Colorectal Tumors. Dig Endosc 22:302–306 . doi: 10.1111/j.1443-1661.2010.01005.x [DOI] [PubMed] [Google Scholar]
- 11.Oka S, Tanaka S, Kaneko I, Mouri R, Hirata M, Kawamura T, Yoshihara M, Chayama K (2006) Advantage of endoscopic submucosal dissection compared with EMR for early gastric cancer. Gastrointest Endosc 64:877–883 . doi: 10.1016/j.gie.2006.03.932 [DOI] [PubMed] [Google Scholar]
- 12.Niimi K, Fujishiro M, Goto O, Kodashima S, Koike K (2012) Safety and Efficacy of Colorectal Endoscopic Submucosal Dissection by the Trainee Endoscopists. Dig Endosc 24:154–158 . doi: 10.1111/j.1443-1661.2012.01251.x [DOI] [PubMed] [Google Scholar]
- 13.Sakamoto T, Takamaru H, Mori G, Yamada M, Kinjo Y, So E, Abe S, Otake Y, Nakajima T, Matsuda T, Saito Y (2014) Endoscopic submucosal dissection for colorectal neoplasms. Ann Transl Med 2: . doi: 10.21037/3536 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kato H, Sakamoto T, Otsuka H, Yamada R, Watanabe K (2012) Colorectal Cancer - From Prevention to Patient Care. InTech
- 15.Fukami N (2014) Endoscopic Submucosal Dissection, An Issue of Gastrointestinal Endoscopy Clinics, E-Book. Elsevier Health Sciences [DOI] [PubMed]
- 16.ASGE Standards of Practice Committee, Faulx AL, Lightdale JR, Acosta RD, Agrawal D, Bruining DH, Chandrasekhara V, Eloubeidi MA, Gurudu SR, Kelsey L, Khashab MA, Kothari S, Muthusamy VR, Qumseya BJ, Shaukat A, Wang A, Wani SB, Yang J, DeWitt JM (2017) Guidelines for privileging, credentialing, and proctoring to perform GI endoscopy. Gastrointest Endosc 85:273–281 . doi: 10.1016/j.gie.2016.10.036 [DOI] [PubMed] [Google Scholar]
- 17.Gromski MA, Cohen J, Saito K, Gonzalez J-M, Sawhney M, Kang C, Moore A, Matthes K (2017) Learning colorectal endoscopic submucosal dissection: a prospective learning curve study using a novel ex vivo simulator. Surg Endosc. doi: 10.1007/s00464-017-5484-2 [DOI] [PubMed]
- 18.Kato M, Gromski M, Jung Y, Chuttani R, Matthes K (2013) The learning curve for endoscopic submucosal dissection in an established experimental setting. Surg Endosc 27:154–161 . doi: 10.1007/s00464-012-2402-5 [DOI] [PubMed] [Google Scholar]
- 19.Parra-Blanco A, Arnau MR, Nicolás-Pérez D, Gimeno-García AZ, González N, Díaz-Acosta JA, Jiménez A, Quintero E (2010) Endoscopic submucosal dissection training with pig models in a Western country. World J Gastroenterol WJG 16:2895–2900 . doi: 10.3748/wjg.v16.i23.2895 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Berr F, Ponchon T, Neureiter D, Kiesslich T, Haringsma J, Kaehler GF, Schmoll F, Messmann H, Yahagi N, Oyama T (2011) Experimental endoscopic submucosal dissection training in a porcine model: learning experience of skilled Western endoscopists. Dig Endosc Off J Jpn Gastroenterol Endosc Soc 23:281–289 . doi: 10.1111/j.1443-1661.2011.01129.x [DOI] [PubMed] [Google Scholar]
- 21.Fujishiro M (2009) Endoscopic submucosal dissection for colorectal neoplasms. World J Gastrointest Endosc 1:32 . doi: 10.4253/wjge.v1.i1.32 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Tsuji Y, Fujishiro M, Kodashima S, Niimi K, Ono S, Yamamichi N, Koike K (2014) Desirable training of endoscopic submucosal dissection: further spread worldwide. Ann Transl Med 2: . doi: 10.3978/j.issn.2305-5839.2014.02.05 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Sankaranarayanan G, Li B, Miller A, Wakily H, Jones SB, Schwaitzberg S, Jones DB, De S, Olasky J (2016) Face validation of the Virtual Electrosurgery Skill Trainer (VEST©). Surg Endosc 30:730–738 . doi: 10.1007/s00464-015-4267-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.DEMIREL D, YU A, HALIC T, SANKARANARAYANAN G, RYASON A, SPINDLER D, BUTLER KL, CAO C, PETRUSA E, MOLINA M, JONES, DE S, DEMOYA M, JONES S (2016) Virtual Airway Skills Trainer (VAST) Simulator. Stud Health Technol Inform 220:91–97 [PMC free article] [PubMed] [Google Scholar]
- 25.Qiao W, Bai Y, Lv R, Zhang W, Chen Y, Lei S, Zhi F (2014) The Effect of Virtual Endoscopy Simulator Training on Novices: A Systematic Review. PLOS ONE 9:e89224 . doi: 10.1371/journal.pone.0089224 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Triantafyllou K, Lazaridis LD, Dimitriadis GD (2014) Virtual reality simulators for gastrointestinal endoscopy training. World J Gastrointest Endosc 6:6–12 . doi: 10.4253/wjge.v6.i1.6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Aggarwal R, Black SA, Hance JR, Darzi A, Cheshire NJW (2006) Virtual reality simulation training can improve inexperienced surgeons’ endovascular skills. Eur J Vasc Endovasc Surg Off J Eur Soc Vasc Surg 31:588–593 . doi: 10.1016/j.ejvs.2005.11.009 [DOI] [PubMed] [Google Scholar]
- 28.Pan JJ, Chang J, Yang X, Liang H, Zhang JJ, Qureshi T, Howell R, Hickish T (2015) Virtual reality training and assessment in laparoscopic rectum surgery. Int J Med Robot Comput Assist Surg MRCAS 11:194–209 . doi: 10.1002/rcs.1582 [DOI] [PubMed] [Google Scholar]
- 29.Beyer-Berjot L, Berdah S, Hashimoto DA, Darzi A, Aggarwal R (2016) A Virtual Reality Training Curriculum for Laparoscopic Colorectal Surgery. J Surg Educ 73:932–941 . doi: 10.1016/j.jsurg.2016.05.012 [DOI] [PubMed] [Google Scholar]
- 30.Rücker G, Schimek-Jasch T, Nestle U (2012) Measuring Inter-observer Agreement in Contour Delineation of Medical Imaging in a Dummy Run Using Fleiss’ Kappa. Methods Inf Med 51:489–494 . doi: 10.3414/ME12-01-0005 [DOI] [PubMed] [Google Scholar]
- 31.Shrout PE, Fleiss JL (1979) Intraclass correlations: Uses in assessing rater reliability. Psychol Bull 86:420–428 . doi: 10.1037/0033-2909.86.2.420 [DOI] [PubMed] [Google Scholar]
- 32.Demirel D, Butler KL, Halic T, Sankaranarayanan G, Spindler D, Cao C, Petrusa E, Molina M, Jones DB, De S, deMoya MA (2016) A hierarchical task analysis of cricothyroidotomy procedure for a virtual airway skills trainer simulator. Am J Surg 212:475–484 . doi: 10.1016/j.amjsurg.2015.08.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Demirel D, Yu A, Cooper-Baer S, Dendukuri A, Halic T, Kockara S, Kockara N, Ahmadi S (2016) A hierarchical task analysis of shoulder arthroscopy for a virtual arthroscopic tear diagnosis and evaluation platform (VATDEP). Int J Med Robot n/a-n/a . doi: 10.1002/rcs.1799 [DOI] [PMC free article] [PubMed]
- 34.Hsu W-H, Sun M-S, Lo H-W, Tsai C-Y, Tsai Y-J (2013) Clinical Practice of Endoscopic Submucosal Dissection for Early Colorectal Neoplasms by a Colonoscopist with Limited Gastric Experience. Gastroenterol Res Pract 2013:e262171 . doi: 10.1155/2013/262171 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Matsui N, Akahoshi K, Nakamura K, Ihara E, Kita H (2012) Endoscopic submucosal dissection for removal of superficial gastrointestinal neoplasms: A technical review. World J Gastrointest Endosc 4:123–136 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Maple JT, Dayyeh BKA, Chauhan SS, Hwang JH, Komanduri S, Manfredi M, Konda V, Murad FM, Siddiqui UD, Banerjee S (2015) Endoscopic submucosal dissection. Gastrointest Endosc 81:1311–1325 . doi: 10.1016/j.gie.2014.12.010 [DOI] [PubMed] [Google Scholar]
- 37.Ribeiro MS, Wallace MB (2015) Endoscopic Treatment of Early Cancer of the Colon. Gastroenterol Hepatol 11:445–452 [PMC free article] [PubMed] [Google Scholar]
- 38.Shida T (2011) Endoscopic Submucosal Dissection for Colorectal Lesions. doi: 10.5772/20825 [DOI]
- 39.Yamamoto H (2010) Endoscopic Submucosal Dissection for Colorectal Tumors. 27:287–295 . doi: 10.1159/000258370 [DOI] [Google Scholar]
- 40.Saito Y, Otake Y, Sakamoto T, Nakajima T, Yamada M, Haruyama S, So E, Abe S, Matsuda T (2013) Indications for and Technical Aspects of Colorectal Endoscopic Submucosal Dissection. Gut Liver 7:263–269 . doi: 10.5009/gnl.2013.7.3.263 [DOI] [PMC free article] [PubMed] [Google Scholar]