Development of a Surgical Competency Assessment Tool for Sentinel Lymph Node Dissection by Minimally Invasive Surgery for Endometrial Cancer

Abstract

Introduction:

Sentinel lymph node dissection is widely utilized in staging of endometrial cancer. Variation in surgical technique potentially impacts diagnostic accuracy and oncologic outcomes and poses barriers to comparison of outcomes across institutions or clinical trial sites. Standardization of surgical technique and surgical quality assessment tools are critical to the conduct of clinical trials. By identifying mandatory and prohibited steps of sentinel lymph node dissection in endometrial cancer, the purpose of this study was to develop and validate a competency assessment tool for use in surgical quality assurance.

Methods:

A Delphi methodology was applied, included thirty-five expert gynecological oncology surgeons from sixteen countries. Interviews identified key steps and tasks which were rated mandatory, optional or prohibited using questionnaires. Using the surgical steps for which consensus was achieved, a competency assessment tool was developed and subjected to assessments of validity and reliability.

Results:

Seventy percent (70%) consensus agreement standardized the specific mandatory, optional and prohibited steps of sentinel lymph node dissection for endometrial cancer and informed development of a competency assessment tool. Consensus agreement identified twenty-one mandatory and three prohibited steps to complete a sentinel lymph node dissection. The competency assessment tool was used to rate surgical quality in three preselected videos, demonstrating clear separation in the rating of the skill level displayed with mean skills summary scores differing significantly between the three videos (F score = 89.4; p<0.001). Internal consistency of the items was high (Cronbach α = 0.88).

Conclusion:

Specific mandatory and prohibited steps of sentinel lymph node dissection in endometrial cancer have been identified and validated based on consensus amongst a large number of international experts. A competency assessment tool is now available and can be used for surgeon selection in clinical trials and for ongoing, prospective quality assurance in routine clinical care.

INTRODUCTION

Surgical trials pose methodological challenges(1) because surgeon training, experience, and skills influence delivery of surgical interventions, leading to variability in health practices and outcomes(2). Surgical quality assurance can aid adherence to pre-defined standards and outcome measures and enable reliable comparison across multiple clinical trial sites(3–7).

Management guidelines for apparent uterine-confined disease prescribe total hysterectomy, bilateral salpingo-oophorectomy for removal of the primary tumour and assessment of locoregional lymph nodes to establish the stage of disease (‘staging’)(8). This information is prognostic and may guide postoperative treatment decisions(8, 9). Historically, surgical staging entailed full or limited pelvic/paraaortic lymph node dissection. This practice was informed by results of observational, clinicopathologic studies(10, 11) then adopted by FIGO in 1988(12). Subsequent prospective studies failed to demonstrate differences in survival outcomes(13, 14). Contemporary surgical staging involves sentinel lymph node dissection (15). According to the sentinel lymph node (SLN) concept, tumour cells metastasise to one or two lymph nodes first, before involving further lymph nodes(16). Presumed benefits of sentinel lymph node dissection include increased surgical staging precision, whilst sparing removal of other regional lymph nodes(17). Sentinel nodes are examined histopathologically using immunohistochemical ultrastaging(18). Sentinel lymph node dissection obtains accurate information about lymph node status(18) such that many clinicians now elect it in place of a full lymphadenectomy(19).

With rapid and global adoption of sentinel lymph node dissection (19) comes variability of surgical technique. Local institutional guidelines have been developed to minimize variation in outcomes(20). However, these algorithms are insufficient to facilitate harmonization of the detailed surgical technique across a group of surgeons. There remains a need to define the precise surgical steps required to accomplish satisfactory bilateral sentinel lymph node dissection; assess a surgeon’s proficiency before enrolment of patients into clinical trials; and assist with ongoing surgical quality assurance(21).

The purpose of this study was to establish a consensus on the specific mandatory and prohibited steps of sentinel lymph node dissection in endometrial cancer, as well as develop a competency assessment tool. This facilitates assessment of surgical quality in clinical trials aiding in both selection of surgeons and prospective Quality Assurance.

METHODS

Study participants

The study was approved by the institutions Human Research Ethics Committee and informed written and/or eConsent was obtained from all participants. Participants were expert gynecological oncology surgeons from five continents currently performing sentinel lymph node dissection, henceforth referred to as “the group”. Experts were recruited using snowball sampling, i.e. first contacting surgeons known to perform sentinel lymph node dissection per scientific reports or presentations in peer reviewed forums, and then asking these surgeons to nominate other experts. Participant characteristics were summarized using descriptive statistics.

Standardization of Sentinel Lymph Node Dissection

A four-round Delphi methodology was applied in order to achieve standardization of sentinel lymph node dissection steps and tasks. Several rounds of questionnaires were sent out to experts with the responses then aggregated, de-identified and shared with the group after each round. Experts adjusted their answers in subsequent rounds, based on their interpretation of the group response provided to them. Over multiple rounds of questionnaires, the Delphi method seeks to reach best response through consensus(22). Study data was collected and managed on a secure, web-based REDCap electronic database hosted at The University of Queensland(23, 24).

Delphi Consensus Process and Hierarchical Task Analysis

Round one (semi-structured interviews):

After providing written informed consent, interviewees described their opinion about the mandatory, optional and unwarranted steps taken in performing sentinel lymph node dissection for endometrial cancer. The interviews were conducted individually and were audio recorded. Recordings were transcribed and thematically analysed by two reviewers (KM, AO). Each reviewer independently identified important and recurring codes (e.g. uterine manipulation, identifying anatomy, trouble shooting). Codes were then compared to confirm important themes. The reviewers jointly examined codes and themes and interpreted the data. Where discordance in coding was identified, themes were refined through discussions between the two reviewers. Interviews were conducted until saturation in variations of technique. Key steps and tasks of sentinel lymph node dissection were identified by a process of hierarchical task analysis.

Round two-to-four (consensus process):

An initial questionnaire was devised including all of the variations identified in the interviews. Members of the group were invited to indicate their agreement or disagreement with variations. In accordance with other published work (25), consensus agreement level was set at 70% (4, 26). Variations where consensus was reached were iteratively moved into an operation guide; those with <70% agreement remained for a subsequent survey round. In accordance with the journal’s guidelines, we will provide our data for the reproducibility of this study in other centers if such is requested.

Operation Guide

A sentinel lymph node dissection operation guide was created including the mandatory, optional and prohibited/unwarranted steps that reached 70% agreement level.

Competency Assessment Tool

Development

The final competency assessment tool was limited to mandatory and prohibited steps in the intraoperative phase of sentinel lymph node dissection. A score of one to four was allocated to each step – ‘skillful’, ‘adequate’, ‘inconsistent’, ‘lacking/deficient’; for troubleshooting steps ‘not applicable’ was also offered.

Content validity

Three surgical videos were selected having been agreed by KM and AO to represent poor, inconsistent or optimal technique of sentinel lymph node dissection according to the ratings conferred by application of the competency assessment tool. The videos featured the 11 surgical steps of sentinel lymph node dissection assessed by the competency assessment tool, but did not include tracer preparation and injection, surgical trouble shooting or pathological assessment of tissues. Content validity was assessed by KM and AO who discussed each step of the competency assessment tool in detail, before watching those individual steps performed with various skill levels across the three surgical videos, confirming that competency assessment tool items adequately reflected the skill required.

Contrast validity and internal reliability

Contrast validity(27) was assessed via invitation of the group members to utilize the competency assessment tool in rating the three pre-selected videos, each representing distinct performance levels. Due to the occurrence of some cells with a cell size <5, Fisher’s exact tests were performed to assess if the proportion of experts who rated each of the three videos as ‘skillful’, ‘adequate’, ‘inconsistent’, ‘lacking/deficient’ differed according to the quality of the video. An average competency assessment tool score (possible range 11–44) was computed for each video. One-way ANOVA modelling determined if the overall competency assessment tool score assigned by the sentinel lymph node dissection experts to each video differed significantly. The summary score was used to assess the internal consistency (Cronbach alpha) of the competency assessment tool.

RESULTS

Thirty-five international gynecological oncology surgeons and experts in sentinel lymph node dissection from 16 countries participated. Some demographic data was not available for five participants, but 28 surgeons were above 40 years of age (80%) and 27 were male (77%) (Table 1).

Table 1:

Participating surgeons demographic characteristics

Variables		n= 351 (%)
Age (years)	31–41	4 (11)
	41–50	15 (43)
	51–60	14 (40)
	>61	1 (3)
Gender	Female	8 (23)
	Male	27 (77)
Continent	Europe	10 (29)
	North America	9 (26)
	Australia	10 (29)
	Asia	4 (11)
	South America	2 (6)
Does your Institution have its own sentinel node mapping protocol in endometrial cancer?	No	14 (40)
	Yes	19 (54)
How did you learn to perform sentinel node biopsies?	Self-taught	16 (46)
	Learned from research papers	15 (43)
	Standard operating procedures/protocols	11 (31)
	Taught by senior colleague/s	11 (31)
	Videos	10 (29)
	Trained by surgeon/s overseas	7 (20)
	Formal course/s	4 (11)
	Other	7 (20)
How many years gynonc? (years)	Less than 10	11 (31)
	10–19	14 (40)
	20–29	9 (26)
	30 or more	1 (3)
How many years performed SLND 2 ? (years)	Less than 5	13 (37)
	5–9	8 (23)
	10 or more	11 (31)
Number of SLND 2	Less than 50	14 (40)
	50–99	10 (29)
	100 or more	6 (17)

¹Not all 35 participants provided data on demographics

²Sentinel Lymph Node Dissection

Twenty-four surgeons had practiced gynecological oncology for more than 10 years (69%), and 21 had performed sentinel lymph node dissection for more than five years (60%). Nineteen surgeons (54%) reported that their institution had an endometrial cancer sentinel lymph node dissection standard protocol. Twenty-one surgeons (60%) performed more than 50 sentinel lymph node dissections annually, excluding those performed for cancer of the vulva. Participating surgeons reported using between one and eight methods to learn sentinel lymph node dissection; e.g. being self-taught (46%), learning from research papers (43%) or being trained by a senior colleague (31%).

Standardization of Sentinel Node Dissection

Delphi Round One (Hierarchical Task Analysis)

Saturation in variation of sentinel lymph node dissection technique was reached after 25 interviews. Analysis of transcripts allocated themes into four phases: preoperative (dye selection and preparation, injection); intraoperative (pelvic dissection, identification of key anatomical structures, definition and dissection of sentinel node, extraction of tissue); trouble shooting; and a postoperative (pathology) phase. Task variations were defined as management of specific surgical steps in different ways. In total 107 task variations were identified across the interviews (Table 2).

Table 2:

Hierarchical task analysis including task variations

Phase	Theme	Sub-themes	No. of sub tasks/task variations
Peri-operative	Tracer Injection	Choice of tracing agent Site of injection Tracer Concentration Total volume of injected Injection technique	34
	Uterine Manipulation	Use of manipulator at all Timing of manipulator insertion Type of uterine manipulator	9
	Sequence of Initial Steps	Timing of entry Timing of adhesiolysis Timing of staging inspection	7
Operative	Preparation/opening spaces	Transperitoneal identification of channels Pelvic Side wall spaces	10
	Identifying anatomy, lymphatic channels and sentinel nodes	Anatomical structure Methods of locating nodes	24
	Excision and confirmation of mapped nodes	Defining the SLN Technique of nodal excision Mode of ex vivo SLN confirmation	7
	Specimen retrieval	Mode of containment	5
Troubleshooting	Action plan for no nodes mapped		5
Post-Operative	Specimen labelling	Anatomical site Laterality	4
	Pathology processing		2

Delphi Rounds Two – Three (Consensus Process)

The first survey (Delphi round two) featured 107 task variations and was completed by all 35 participants (Supplementary Table 1). The second survey (Delphi round three) was informed by the results of the first survey and 33 of 35 participants responded (Supplementary Table 2). Over rounds two and three, >70% consensus was achieved in 33 of the 107 (30.8%) task variations(4, 26) on mandatory, optional and prohibited steps of sentinel lymph node dissection. Of the variations that reached consensus, 21 were classified as mandatory, nine optional and three prohibited. For example, in round two, 79% of participants agreed that “a transperitoneal approach of injecting dye into the uterus” should be prohibited; while 75% of participants agreed that “the internal iliac artery must be identified for sentinel node mapping” was mandatory. An operation guide consisting of the final list of steps for which consensus was obtained is provided in Table 3.

Table 3: Operation Guide.

Final consensus on mandatory and prohibited steps of SLND by minimally invasive surgery in Endometrial Cancer.

Surgical Step	Descriptor	Consensus recommendation
Tracer	ICG	Mandatory
	Blue Dye	Optional
	Radio-labelled technetium	Optional
Injection location	Ectocervix in two or four positions	Mandatory
Injection technique	Superficial injection into the ectocervix	Mandatory
	Transperitoneal injection into the uterus	Prohibited
	Hysteroscopic injection into the uterus	Prohibited
	Surgeon appreciation of resistance at tracer injection	Mandatory
Injection needle	Gauge between 20G and 25G	Mandatory
	Length sufficient to ensure easy and accurate access to the cervix	Mandatory
Uterine manipulator	If being used, insert uterine manipulator after tracer injection	Mandatory
White light inspection	Prior to SLN mapping, conduct an inspection of the pelvic areas	Mandatory
Round ligament & Infundibulopelvic ligament	Preserve	Optional
	Divide	Optional
External vessels	Identify the external iliac vessels	Mandatory
Internal iliac artery	Identify the internal iliac artery	Mandatory
Ureter	Identify the ureter	Mandatory
Obliterated umbilical ligament	Identify the obliterated umbilical ligament	Mandatory
Uterine artery	Identify the uterine artery (medial to the ureter)	Optional
Paravesical space	Open the paravesical space	Mandatory
Direction of dissection	Start sentinel lymph node mapping at the level of the uterine artery and continue dissection LATERALLY away from the uterus	Mandatory
	Start sentinel lymph node mapping at the level of the uterine artery and and continue MEDIALLY towards the uterus	Optional
	Start sentinel lymph node mapping at the level of the uterine artery and and continue towards the pre-sacral area	Optional
	Start sentinel lymph node mapping at the most highlighted node and dissect proximally (TOWARDS cervix)	Optional
	Start sentinel lymph node mapping at the most highlighted node and dissect cephalad (AWAY from cervix)	Optional
Dissection technique	Use blunt or electrosurgical technique	Mandatory
	Avoid disrupting lymphatic channels during dissection	Mandatory
	Ensure isolation of node from local anatomy	Mandatory
Definition of the Sentinel Node	A sentinel node is defined as …
	• The most proximal node1, irrespective of the nodal station in which the node is found	Mandatory
	• A single mapped node or a single node plus its next station echelon node(s).	Mandatory
SLN dissection	SLN dissection should be completed in one hemipelvis before proceeding to the contralateral side	Mandatory
Troubleshooting	Troubleshooting when no nodes are mapping includes any one, or combination of, the following OPTIONS:	Mandatory
	• Wait; undertake dissection on the contralateral side before returning to original side
	• Extend retroperitoneal dissection to encompass common, pre-sacral and/or paraaortic areas
	• Re-inject ICG
	• Undertake a side-specific lymphadenectomy
Specimen extraction	Removal of nodes without using a containment device	Prohibited
Prove of sentinel node	Use ex-vivo green fluorescence to prove the sentinel node	Mandatory
Specimen labelling	Label specimens according to laterality (right/left) AND nodal station (obturator/external iliac/internal iliac/presacral/common iliac/aortic/caval)	Mandatory
Ultrastaging	Use enhanced pathology techniques, such as immunohistochemistry, for ultrastaging of sentinel nodes	Mandatory

¹The most proximal node is defined as the node closest to the uterus, regardless of location.

There was consensus that the tracer of choice must be indocyanine green (ICG) but adding other tracers is optional. There was consensus that ICG should be administered by superficial injection (1–2 mm) into the cervix. Superficial injection was defined by the group as sub-mucosal injection into the epithelium of the ectocervix (akin to intra-dermal injection techniques used in sentinel lymph node mapping for vulvar cancer). Deep injection (1 cm) was considered optional only when combined with mandatory superficial injection. Deep injection alone was considered prohibited by consensus. There was no consensus about the dilution of ICG (between 0.5 mg/l and 1.5 mg/ml), the total volume injected or timing of injection (before or after establishing a pneumoperitoneum). The use of a uterine manipulator was considered optional, but if used, it should be inserted after tracer injection. There was consensus that dividing the round ligament and the infundibulopelvic ligament can be performed either before or after sentinel lymph node dissection. The pelvic structures and spaces that should be demonstrated for sentinel lymph node dissection include external and internal iliac vessels, ureter, obliterated umbilical ligament and the paravesical space. The direction of the sentinel lymph node dissection was considered optional (starting close to the cervix or dissecting towards the cervix).

The group agreed that the sentinel node should be defined as the most proximal node irrespective of the nodal station in which the node is found. Eighteen participants felt that mapping of presacral nodes should be optional (56.3%). There was lack of consensus on a side-specific lymphadenectomy if no nodes are mapped on one side. Participants agreed that the sentinel node should be a single mapped node with or without its next station (second echelon node(s)). A majority of participants (59.4%) but less than required for consensus, agreed that not all second, third and fourth echelon nodes should be removed. Greater than seventy percent of participants agreed that specimen extraction should be within a containment device. There was consensus that ex-vivo fluorescence should be used to prove the sentinel node; that labelling of the sentinel node should be according to laterality and nodal station; and enhanced pathology techniques for ultrastaging of sentinel nodes should be used.

Contrast validity and internal reliability:

Twenty-seven (77.1%) Delphi participants were involved in rating the quality of surgery of the three preselected videos using the competency assessment tool (Figure 1). For each of the 10 initial surgical steps, there was clear separation in the rating of the skill level displayed between the three videos (Table 4).

Figure 1:

SLND Competency Assessment Tool

Table 4:

Assessment of contrast validity

	Poor Video	Inconsistent Video	Optimal Video	Fishers exact test
White light inspection	1 (4)	6 (22)	22 (81)	47.1; p<0.00
External vessels	0 (0)	5 (19)	21 (78)	56.0; p<0.001
Internal iliac artery	0 (0)	2 (7)	22 (82)	75.3; p<0.001
Ureter	0 (0)	6 (22)	20 (74)	70.6; p<0.001
Paravesical space	0 (0)	4 (15)	19 (70)	58.9; p<0.001
Obliterated umbilical ligament	0 (0)	2 (7.4)	19 (70)	60.3; p<0.001
Dissection technique	1 (4)	2 (7.4)	16 (59)	36.9; p<0.001
Proof of sentinel node	6 (22)	2 (7.4)	20 (74)	33.6; p<0.001
Specimen extraction	0 (0)	9 (33)	21 (78)	84.2; p<0.001
SLN mapping	1 (3.7)	5 (19)	10 (37)	15.9; p=0.03

n (%) of reviewers who rated the performance as skilful

For example, while 78% of experts rated the “optimal technique video” as skillfully performing the dissection of the iliac vessels, only 19% and 0% of experts rated the “inconsistent technique video” and “poor technique video” as skillful (Fishers exact test = 56.0; p<0.001). For the last step (“completion of SLND in one hemipelvis before proceeding to the contralateral side”), 25 of the 27 group members rated this step as not applicable. Overall, the mean skills summary score differed significantly between the three videos from 35.6 (SD =4.7) for the “optimal technique video”, to 25.3 (SD=5.9) for the “inconsistent technique video” and 17.7 (SD=4.1) for the “poor technique video” (One-way ANOVA F score = 89.4; p<0.001). Internal consistency of the items was high (Cronbach α = 0.88).

DISCUSSION

Summary of Main Results

We report the creation of a competency assessment tool, derived by consensus amongst a large number of international experts, outlining the mandatory, optional and prohibited steps of a sentinel lymph node dissection procedure for endometrial cancer. The competency assessment tool is validated by gynecological oncology surgeons and can be used by trial governance committees as a decision aide for surgeon selection and for ongoing Quality Assurance in surgical clinical trials.

Results in the Context of Published Literature

While local health service protocols(20) suggest specific steps for a sentinel lymph node dissection, the present publication summarizes an operating consensus based on the opinion of a considerable number of international experts in sentinel lymph node dissection. Consensus was achieved about definition of the sentinel node (the node closest to the uterus) regardless of whether it is located at the lateral pelvic wall, the aortic/caval or the presacral area. There was also agreement that the number of sentinel nodes removed should be kept to a minimum. There was no consensus on mandatory need for completion lymphadenectomy on the ipsilateral side of a pelvis that fails to map. This most likely reflects the possibility for patient and uterine factors to indicate against full lymph node dissection. Greater than seventy percent consensus was reached on the need to extract nodes through a containment system, the need for ex-vivo green fluorescence to prove the sentinel node; on specimen labelling and on pathologic ultrastaging.

The competence assessment tool development undertaken in this study follows similar efforts in other surgical specialties. In general surgery, a recent systematic review reporting on Quality Assurance in randomised controlled trials of laparoscopic colorectal surgery identified three distinct categories of surgical Qulaity Assurance measures: (i) trial entry criteria for surgeons and centres; (ii) standardization of surgical techniques; and (iii) continuous monitoring of surgeons and/or units(25). A competence assessment tool was developed, validated and implemented to assess technical surgical performance in the context of a summative assessment process for the National Training Programme in Laparoscopic Colorectal Surgery (Lapco)(28). Subsequently COLOR III(4) investigators have described standardization of surgical interventions followed by development and assessment of objective surgical Quality Assurance tools for use in colorectal trials.

Strengths and Weaknesses

The strengths of our study include the large number of international experts who identified the mandatory, optional and prohibited steps of sentinel lymph node dissection based on consensus. In addition, the competence assessment tool was able to demonstrate contrast validity and internal reliability. Our expert participants reported a range of experience in sentinel lymph node dissection despite recruitment using snowball sampling. We did not use more objective measurements of expertise such as a requirement for study participants to submit their own outcomes data, or videos of their individual technique. Another weakness of our study relates to the impact of our findings on meaningful clinical outcomes such as sensitivity for detection of metastases, or false negative rates. These parameters were not the focus of our study, but we aknowledge the absolute necessity for individual surgeons to monitor their performance including prscopective audit of sensitivity, false negatives, recurrence patterns and rates. This predictive clinical validity of sentinel lymph node dissection technique can only be determined with accumulation of clinical outcomes after using the competence assessment tool in sentinel lymph node dissection clinical trials and educational programs, as has been demonstrated for a colorectal competence assessment tool in the Lapco program (29).

Implications for Practice and Future Research

Despite the benefits of sentinel lymph node dissection, including shorter operating times compared to a lymphadenectomy, it remains unknown in what ways sentinel lymph node dissection impacts clinically relevant outcomes, such as the need for postoperative radiation treatment or chemotherapy, recovery from surgery and quality of life, the incidence of adverse events, disease recurrence and survival(21). Additionally, whilst new surgical procedures may appear promising, there remains a need to evaluate novel surgical procedures for safety and effectiveness(30). Such surgical trials rely on the standardized delivery of the intervention (with minimal variation) to allow a meaningful and reliable comparison between intervention and control groups across multiple surgeons or trial sites. In the context of sentinel lymph node dissection, variability in technique and failure to identify sentinel nodes could translate into the need for frozen section assessment of the uterus, acceptance of unknown nodal status, or may increase the risk of an “empty package”(31), all of which may confound the results of sentinel lymph node dissection efficacy trials. Depending on local protocols, some patients may even require re-staging, a full ipsilateral lymphadenectomy (20), or might warrant adjuvant chemotherapy or radiation treatment based on uterine risk factors. These scenarios may have significant impact on short and long-term patient and trial outcomes. Using this competency assessment tool, institutions or clinical trialists can define quality standards of sentinel lymph node dissection and measure individual surgical performance.

Significant efforts are made by chief investigators and trial management committees to minimize variability in surgical technique and outcomes, including limiting the trial to sites with a high surgical volume. Recently, principal investigators completed a site visit and all surgeons were observed in-person(18, 32) or unedited videos were reviewed to confirm standardization of the technique(33). In other trials, participating surgeons were required to have completed a minimum number of procedures, before the initiation of enrollment(34). While these measures were valuable within institutions, volume, minimal number or observation of one surgery may be inaccurate without application of a standardized assessment tool.

Conclusion

The output from this work includes a list of mandatory and prohibited steps of a sentinel lymph node dissection that independent assessors can use to check for both surgical proficiency as well as whether sentinel lymph node dissection has been performed in accordance to an agreed standard. The work provides specific steps of sentinel lymph node dissection, and the Quality Assurancecriteria developed as part of this process will help with selection of prospective surgeons into surgical trials evaluating sentinel lymph node dissection. The goal is to shorten the learning curve(34) but also to control for heterogeneity in surgical performance that could override the true efficacy(4).