Accuracy and agreement of a large panel of endosonographers for endomicroscopy-guided virtual biopsy of pancreatic cystic lesions

Abstract

Background:

Although emerging data evidences that EUS-guided needle-based confocal laser endomicroscopy (nCLE) accurately diagnoses pancreatic cystic lesions (PCLs), there are a lack of interobserver agreement (IOA) studies utilizing reference histopathological diagnosis and for specific PCL subtypes. Hence, we sought to assess the IOA, intra-observer reliability (IOR), and diagnostic performance of EUS-nCLE using a large cohort of patients with histopathological diagnosis amongst a broad panel of international observers.

Methods:

EUS-nCLE videos (n = 76) of subjects with PCLs [intraductal papillary mucinous neoplasm (IPMN), mucinous cystic neoplasm (MCN), serous cystadenoma (SCA), pseudocyst, and cysticneuroendocrine tumors/solid pseudopapillary neoplasm (cystic-NET/SPN)], simulating clinical prevalence rates were obtained from 3 prospective studies. An international panel of 13 endosonographers with nCLE experience, blinded to all PCL data, evaluated the video library twice with a two-week washout for PCL differentiation (mucinous vs. non-mucinous) and subtype diagnosis.

Results:

The IOA (κ = 0.82, 95% CI 0.77e0.87) and IOR (κ = 0.82, 95% CI 0.78–0.85) were “almost perfect” to differentiate mucinous vs. non-mucinous PCLs. For PCL subtype, IOA was highest for SCA (almost perfect; κ = 0.85), followed by IPMN (substantial, κ = 0.72), and cystic-NET/SPN (substantial, κ = 0.73). The IOA was moderate for MCN (κ = 0.47), and pseudocyst (κ = 0.57). Compared to histopathology, observers differentiated mucinous vs. non-mucinous PCLs with high accuracy (94.8%, 95% CI 93.3e96.1). For detecting specific PCLs subtypes, EUS-nCLE was highly accurate in diagnosing non-mucinous cysts (SCA: 98%; cystic-NET/SPN: 96%; pseudocyst: 96%) and slightly less accurate for mucinous lesions (IPMN: 86%; MCN: 84%).

Conclusion:

Diagnosis of PCLs by EUS-nCLE guided virtual biopsy is very accurate and reliable for the most prevalent pancreatic cysts in clinical practice.

1. Introduction

Pancreatic cystic lesions (PCLs) are common, with prevalence estimates that range from 3% with computed tomography (CT) to 49% with high-quality magnetic resonance imaging (MRI) [1]. Mucinous PCLs (intraductal papillary mucinous neoplasms [IPMNs] and mucinous cystic neoplasms [MCNs]) are precursors of pancreatic adenocarcinoma that require preemptive surgical resections or surveillance for management [2,3]. Non-mucinous PCLs encompass a heterogeneous group of lesions, where pseudocysts and serous cystadenomas (SCA) do not need surveillance, whereas cystic neuroendocrine tumors (cystic-NET) and solid pseudopapillary neoplasms (SPN) require surveillance or surgery [4]. Current diagnostic approaches (CT, MRI, endoscopic ultrasound [EUS] with fine-needle aspiration [FNA]) are suboptimal in differentiating PCLs [3], with nearly 20% of patients undergoing pancreatic resection for a presumptive premalignant or malignant cyst ultimately having a benign lesion [5,6]. Hence, there is a continued need for diagnostic tests with high accuracy and reliability for optimal management of PCLs.

EUS-guided needle-based confocal laser endomicroscopy (EUS-nCLE) provides a virtual biopsy of PCLs by real-time in-vivo histopathologic imaging of the inner epithelium [7–9]. Several studies have demonstrated that EUS-nCLE is safe, feasible, and more accurate than conventional methods on differentiating PCLs [10–12]. Over the last decade, EUS-nCLE diagnostic criteria have been validated for the most common PCLs (Fig. 1), providing high sensitivity, specificity, and accuracy [8,9,12–20]. Few studies have evaluated the inter-observer agreement (IOA) and intra-observer reliability (IOR) of these EUS-nCLE parameters, with overall favorable results [9,14,16,21,22]. However, these studies were limited by small patient populations (15–33 subjects), few observers (n = 4 to 6), and a sparse representation of PCLs other than IPMNs. Moreover, most of these studies used retrospective cohorts, with many cysts lacking definitive pathologic diagnosis, and in some instances included observers without large EUS-nCLE experience [9,14,16,21,22]. In addition, the variables that impact the accuracy and reliability of EUS-nCLE, such as case volume and diagnostic confidence, have not been previously reported.

Fig. 1.

EUS-nCLE diagnostic characteristics of PCL subtypes

Panel A: Intraductal papillary mucinous neoplasm with papillary fronds; Panel B: Mucinous cystic neoplasm with layered epithelial bands; Panel C: Cystic neuroendocrine tumor with dense nests and cords of cells (trabecular pattern); Panel D: Serous cystadenoma with a fern-pattern (superficial vascular network) of vascularity; Panel E: Pseudocyst with poorly organized clumps of inflammatory cells; Panel F: Solid pseudopapillary neoplasm with trabecular pattern.

To address prior limitations, we used data from three large prospective studies that have collectively enrolled over 350 patients for EUS-nCLE of PCLs. Specifically, only subjects with definitive histopathology were enrolled in the present study. We sought to assess the diagnostic performance, IOA, and IOR of EUS-nCLE in a large international panel of nCLE users to differentiate mucinous from non-mucinous PCLs and diagnose specific PCL subtypes. As a secondary endpoint, we aimed to evaluate the impact of case volume and observer’s confidence on EUS-nCLE diagnostic parameters and reliability.

2. Methods

2.1. Study design

This is a reliability study conducted among an international panel of endosonographers with experience in EUS-nCLE between September 2020 and November 2020. We used CLE videos obtained from subjects enrolled in 3 prospective observational studies evaluating EUS-nCLE in the diagnosis of PCLs. The studies included: (a) CONTACT, a French multi-center study (n = 206; 2012–2016; Clinicaltrials.gov NCT01563133); (b) INDEX, a US single-center study (n = 144; 2015–2018; NCT02516488); and (c) CLIMB, a US multi-center study (2018-present, NCT03492151). The Institutional Review Board of each participating center from these 3 cohorts approved the study protocol. All the study participants, or their legal guardian, provided informed consent. All listed authors had access to the analyzed data and approved the final manuscript.

2.2. Patient population

For the present study, we selected subjects who had representative EUS-nCLE videos and a definitive histopathologic diagnosis of the most prevalent PCLs (IPMN, MCN, SCA, pseudocyst, SPN, cysticNET). A total of 76 subjects fulfilled eligibility criteria and were included.

2.3. EUS-nCLE procedure and video editing

All study subjects underwent EUS-nCLE by experienced endosonographers using local standardized protocols. The entire video sequence was recorded in the nCLE processor. The mean (± standard deviation (SD)) duration of nCLE video per patient was 6.0 ± 2.6 min. Because the nCLE videos captured low-yield intracystic non-epithelial images, each patient’s video was shortened to a representative clip of ~1 min that best represented the PCL epithelium using a previously described methodology [9,15]. All edited deidentified videos were uploaded to a private channel on YouTube (Google Inc, Mountain View, California).

2.4. Selection of observers

The sampling frame was international endosonographers with expertise in managing patients with PCLs and experience in EUS-nCLE. We utilized a non-random purposeful sampling strategy to generate a list of observers. The recruitment of observers was based on a comprehensive literature search of human studies in MEDLINE using the following MeSH and keyword terms: “confocal microscopy”, “confocal endomicroscopy”, “needle-based confocal endomicroscopy”, “pancreatic cyst”, and “pancreatic cystic neoplasms”. Additional observers who were known to perform EUS-nCLE in clinical practice were also eligible. Those who had performed <10 EUS-nCLE procedures in their careers were ineligible. The final sample included a total of 13 EUS-nCLE users from different institutions in the United States (4), China (3), Thailand (1), Singapore (1), France (2), Italy (1), and United Kingdom (1). Observers were blinded to all clinical data and final diagnoses. Baseline observer characteristics including years of EUS-nCLE experience and procedural volume were collected. Each observer completed an online 8-min instructional video (https://youtu.be/ZiGTjE_SFPQ) of EUS-nCLE patterns of PCLs before starting assessments. The nCLE images used in the instructional video were different from the study cases.

2.5. Assessment of EUS-nCLE videos

Observers evaluated the nCLE videos in two phases separated by a 2-week washout period (Fig. 2). The observers were provided two weeks in each phase to complete assessments. In the second phase, the video sequences were randomly rearranged to avoid recollection bias. The YouTube playlist permitted pausing, changing playback speed, and replaying sequences, simulating the functionality of the CLE platform. However, the video library lacked the ability to select and measure regions of interest (e.g., pixels, size). Compared to the CLE platform (Cellvizio processor), there is some loss of resolution and downscaling of videos during conversion from “.mkt” (proprietary format) to “.mp4” video format to facilitate YouTube upload.

Fig. 2.

Study flow

EUS-nCLE: Endoscopic ultrasound guided needle based confocal laser endomicroscopy; nCLE: needle based confocal laser endomicroscopy; IPMN: intraductal papillary mucinous neoplasm; MCN: mucinous cystic neoplasm; SCA: serous cystadenoma; NET/SPN: neuroendocrine tumor/solid pseudopapillary neoplasm.

All evaluations and observer data entry were managed via Qualtrics (Qualtrics XM, Utah, USA). A dedicated online survey was designed to assess the following aspects in each of the nCLE subject videos: A) PCL classification as mucinous or non-mucinous); B) diagnosis of specific PCL subtype; C) diagnostic confidence; and D) video quality. If observers classified the cyst as mucinous, they were subsequently asked whether the cyst was an IPMN or MCN. When a cyst was classified as non-mucinous, observers had to diagnose the cyst as a SCA, pseudocyst, or cystic-NET/SPN. Since cystic-NETs and SPNs have similar nCLE features, both cysts were grouped into one category. The level of confidence was scored as “high”, “moderate,” or “low.” The quality of each video was rated as “good” (very clearly visible epithelial/vascular patterns), “mild” (visible epithelial/vascular patterns) and “poor” (hardly visible epithelial/vascular patterns). After completing the assessments, data entries were locked and could not be altered.

2.6. Statistical analysis

Descriptive statistics are reported as frequencies (percentage), mean ± standard deviation (SD), and median (interquartile range [IQR]), as appropriate. The primary analysis was calculating the reliability and diagnostic performance of EUS-nCLE for differentiating mucinous from non-mucinous PCLs and for diagnosing specific cyst types separately for phase 1 and 2. For reliability, we estimated the IOA using Fleiss’s kappa and the IOR using Cohen’s kappa. We interpreted k-values according to Landis and Koch scale: <0, no agreement; 0 to 0.20, slight; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, substantial; and 0.81 to 1, almost perfect [23]. We calculated sensitivity, specificity, and accuracy, using confirmatory histopathology as the gold standard. We compared results of phase 1 and 2 using Chi-square tests. In a secondary analysis, we evaluated the effect of EUS-nCLE experience and level of confidence on diagnostic parameters. We used Pearson correlation to assess the effect of experience comparing individual diagnostic accuracy and EUS-nCLE case volumes. We also performed a stratified analysis comparing medium-volume (≤50 overall EUS-nCLEs) vs. high-volume users (>50 EUS-nCLEs). We did a subgroup analysis using only high-confident assessments to assess the effect of diagnostic confidence. All estimates are reported with 95% confidence intervals (CI). Statistical significance was defined as P < 0.05. All statistical analyses were performed using SAS 9.4 (SAS Institute, Cary NC).

3. Results

3.1. Baseline characteristics

The study included 76 (64.5% female, mean age 60.2 ± 14.3 years) patients with PCLs and the majority (67.1%) were mucinous lesions (Table 1). IPMNs (48.7%, 37/76) were most frequent followed by an even distribution of MCNs (18.4%), cystic-NETs/SPNs (17.1%), and SCAs (11.8%). Pseudocysts were less represented in the cohort (3.9%). All the 13 observers were experienced endosonographers, with median EUS experience of 11 years (IQR, 9–14) and median nCLE experience of 5 years (IQR, 3–6). Observers had performed a median of 50 EUS-nCLE procedures in their careers (range 10–270; IQR, 20–80).

Table 1.

Baseline patient and observers characteristics.

Patient characteristics (n = 76)
Study source, n (%)
INDEX (US)	40 (52.6%)
CONTACT (France)	21 (27.6%)
CLIMB (US)	15 (l9.7%)
Mean age, years (SD)	60.2 (14.3)
Gender, n (%)
Female	49 (64.5%)
Male	27 (35.5%)
Cyst size, mm (SD)	37.7 (18.7)
Cyst location, n (%)
Head/uncinate	21 (27.6%)
Neck/body	31 (40.8%)
Tail	24 (31.6%)
Cyst type, n (%)
Mucinous	51 (67.1%)
Non-mucinous	25 (32.9%)
Histopathologic diagnosis, n (%)
IPMN	37 (48.7%)
MCN	14 (18.4%)
SCA	9(11.8%)
Cystic NET	10(13.2%)
SPN	3 (3.9%)
Pseudocyst	3 (3.9%)
Method of tissue acquisition for final diagnosis
Surgical resection	74 (97.4%)
Through-the-needle biopsy	2 (2.6%)
Observer characteristics (n = 13)
Practice location, n (%)
Asia	5 (38.4%)
Europe	4 (30.8%)
North America	4 (30.8%)
Type of practice, n (%)
Academic	11 (84.6%)
Private	1 (7.7%)
Community	1 (7.7%)
Years of EUS experience, median (IQR)	11 (9,14)
Years of nCLE experience, median (IQR)	5 (3, 6)
Number of EUS-nCLEs performed, median (IQR)	50 (20, 80)
EUS-nCLE experience, n (%)
10–50	7 (53.8%)
>50	6 (46.2%)

IPMN: intraductal papillary mucinous neoplasm; MCN: mucinous cystic neoplasm; SCA: serous cystadenoma; cystic-NET/SPN: cystic neuroendocrine tumors/solid pseudopapillary neoplasm; EUS: endoscopic ultrasound; nCLE: needle-based confocal laser endomicroscopy; SD: standard deviation; IQR: inter-quartile range.

3.2. Image quality and diagnostic confidence

A total of 988 video assessments were conducted at each of the 2 study phases (Supplementary Table 1). Most videos had discernible epithelial and vascular nCLE patterns (phase-1: 91.2%; phase-2: 93%) and were rated as good quality (phase-1: 60%; phase-2: 65%). Differentiation of mucinous from non-mucinous PCLs was made with high confidence in 88% of the video assessments. Also, a high confidence was documented for differentiating among non-mucinous cyst types (phase-1: 75.7%; phase-2: 81.5%) and differentiating IPMNs from MCNs (phase-1: 70.3%; phase-2: 74%).

3.3. IOA and IOR with EUS-nCLE

When classifying PCLs as mucinous and non-mucinous, nCLE interpretations had almost perfect IOA (k = 0.82) and IOR (k = 0.82, Table 2). For nCLE-guided diagnosis of PCL subtypes, reliability was highest for SCAs with almost perfect IOA (phase-2: 0.85) and substantial IOR (0.80). There was substantial IOA and IOR for nCLE characterization of IPMNs (IOA phase-2: 0.72; IOR: 0.70) and cystic-NETs/SPNs (IOA: 0.73; IOR: 0.78). Pseudocysts were diagnosed with substantial IOR (0.62), but with only moderate IOA (phase-2: 0.57). The nCLE characterization of MCNs had the lowest reliability, with moderate IOA (phase-2: 0.47) and IOR (0.45). There were no statistical differences for any of the IOA estimates between phase 1 and 2. In a subgroup analysis of PCLs identified correctly as mucinous or non-mucinous, there were no differences in the IOAs for specific cyst types (Supplementary Table 2).

Table 2.

IOA and IOR of EUS-nCLE pattern interpretation of PCLs among international observers (n = 13).

	IOA Phase 1		IOA Phase 2		IOR between phases 1–2
	κa	95% CI	κa	95% CI	κa	95% CI
Mucinous vs. Non-mucinous	0.82	(0.76, 0.88)	0.82	(0.77, 0.87)	0.82	(0.78, 0.85)
Specific cyst type
IPMN	0.69	(0.61, 0.77)	0.72	(0.64, 0.79)	0.70	(0.65, 0.74)
MCN	0.48	(0.37, 0.59)	0.47	(0.35, 0.59)	0.45	(0.38, 0.53)
SCA	0.67	(0.53, 0.80)	0.85	(0.76, 0.95)	0.80	(0.74, 0.86)
Cystic-NET/SPN	0.76	(0.67, 0.85)	0.73	(0.64, 0.83)	0.78	(0.72, 0.83)
Pseudocyst	0.67	(0.47, 0.86)	0.57	(0.36, 0.79)	0.62	(0.52, 0.72)

IOA: interobserver agreement; IOR: interobserver reliability; EUS-nCLE: Endoscopic ultrasound guided needle-based confocal laser endomicroscopy; PCLs: pancreatic cystic lesions; IPMN: intraductal papillary mucinous neoplasm; MCN: mucinous cystic neoplasm; SCA: serous cystadenoma; NET/SPN: neuroendocrine tumor/solid pseudopapillary neoplasm; CI: confidence interval.

^aLandis and Koch interpretation for k values: <0, no agreement; 0—0.20, slight; 0.21—0.40, fair; 0.41—0.60, moderate; 0.61—0.80, substantial; 0.81—1, almost perfect agreement.

3.4. Diagnostic performance of EUS-nCLE

EUS-nCLE was highly accurate in differentiating mucinous from non-mucinous cysts with 95.2% sensitivity, 94.2% specificity, and 94.8% accuracy, during phase-2 (Table 3). For diagnosis of specific PCL type, EUS-nCLE was highly accurate for diagnosing SCAs (sensitivity: 94.9%; specificity: 98.6%; accuracy: 98.2%), cystic-NETs/SPNs (sensitivity: 80.5%; specificity: 98.9%; accuracy: 95.8%), and pseudocysts (sensitivity: 87.2%; specificity: 96.2%; accuracy: 95.9%) during phase-2. In contrast, the diagnostic accuracy of EUS-nCLE was slightly lower for diagnosing IPMNs (sensitivity: 84.4%; specificity: 88.0%; accuracy: 86.2%) and MCNs (sensitivity: 57.1%; specificity: 90.2%; accuracy: 84.1%) during phase-2. When comparing phases 1 and 2, there was a significant improvement in sensitivity (82.9% vs. 94.9%, p = 0.03) and accuracy (95.8 vs. 98.2%, p = 0.002) for nCLE diagnosis of SCAs during the latter phase. Otherwise, there were no statistical differences in other diagnostic parameters between the phases.

Table 3.

Diagnostic performance of EUS-nCLE interpretation of PCLs compared with definitive histopathology.

	Sensitivity, % (95% CI)		p	Specificity, % (95% CI)		p	Accuracy, % (95% CI)		p
	Phase 1	Phase 2		Phase 1	Phase 2		Phase 1	Phase 2
Mucinous vs. non-mucinous	94.7 (92.7, 96.2)	95.2 (93.3, 96.6)	0.71	94.5 (91.4, 96.5)	94.2 (91.1, 96.2)	0.87	94.6 (93.1, 95.9)	94.8 (93.3, 96.1)	0.84
IPMN	84.6 (81.1, 87.6)	84.4 (80.9, 87.4)	0.93	90.3 (87.5, 92.6)	88.0 (84.9, 90.5)	0.23	87.6 (85.4, 89.5)	86.2 (84.0, 88.2)	0.39
MCN	64.3 (57.1, 70.9)	57.1 (49.9, 64.1)	0.16	90.9 (88.8, 92.7)	90.2 (88.0, 92.1)	0.61	86.0 (83.7, 88.1)	84.1 (81.7, 86.3)	0.23
SCA	82.9 (75.1, 88.7)	94.9 (89.3, 97.6)	0.03	97.5 (96.2, 98.3)	98.6 (97.6, 99.2)	0.08	95.8 (94.3, 96.8)	98.2 (97.1, 98.8)	0.002
Cystic-NET/SPN	86.4 (80.4, 90.8)	80.5 (73.9, 85.7)	0.14	98.3 (97.2, 99.0)	98.9 (97.9, 99.4)	0.29	96.3 (94.9, 97.3)	95.8 (94.3, 96.8)	0.57
Pseudocyst	92.3 (79.7, 97.4)	87.2 (73.3, 94.4)	0.46	97.2 (95.9, 98.0)	96.2 (94.8, 97.3)	0.25	97.0 (95.7, 97.9)	95.9 (94.4, 96.9)	0.18

EUS-nCLE: Endoscopic ultrasound guided needle-based confocal laser endomicroscopy; PCLs: pancreatic cystic lesions; IPMN: intraductal papillary mucinous neoplasm; MCN: mucinous cystic neoplasm; SCA: serous cystadenoma; NET/SPN: neuroendocrine tumor/solid pseudopapillary neoplasm; CI: confidence interval.

3.5. Effect of EUS-nCLE experience

The diagnostic accuracy of EUS-nCLE interpretation to differentiate mucinous from non-mucinous PCLs was not associated with individual EUS-nCLE procedural volume (correlation coefficient R = 0.152, p = 0.42) (Fig. 3A). There was no significant correlation between individual EUS-nCLE procedural volume and accuracy in diagnosing any of the specific PCL types (Fig. 3B). When comparing medium-volume (10–50 EUS-nCLEs, n = 7) and high-volume users (>50 EUS-nCLEs, n = 6), there was higher specificity in differentiating mucinous from non-mucinous lesions with greater EUS-nCLE experience (98.7% vs. 90.3%, p < 0.05). Otherwise, there was no statistical improvement in any of the other diagnostic parameters or IOA for specific PCLs (Table 4).

Fig. 3.

Pearson correlation comparing the overall number of EUS-nCLE cases performed by each observer and individual accuracy for diagnosis of: A) mucinous vs. non-mucinous PCLs; and B) specific PCL type *

EUS-nCLE: Endoscopic ultrasound guided needle-based confocal laser endomicroscopy; PCLs: pancreatic cystic lesions; IPMN: intraductal papillary mucinous neoplasm; MCN: mucinous cystic neoplasm; SCA: serous cystadenoma; NET/SPN: neuroendocrine tumor/solid pseudopapillary neoplasm

* The Y-axis represents the accuracy to diagnose PCLs among each of the 13 observers. The X-axis represents the number of EUS-nCLE procedures performed by each observer by the time of video assessments.

Table 4.

IOA, sensitivity, specificity and accuracy of EUS-nCLE to diagnose PCLs based on nCLE procedural volume (≤50 [n = 7] vs. ≥ 50 [n = 6]).

EUS-nCLE experience	IOA, K (95% CI)		Sensitivity, % (95% CI)		Specificity, % (95% CI)		Accuracy, % (95% CI)
	≤50	≥50	≤50	≥50	≤50	≥50	≤50	≥50
Mucinous vs. non-mucinous	0.79 (0.72, 0.86)	0.85 (0.78, 0.92)	96.6 (94.2, 98.1)	93.5 (90.1, 95.7)	90.3 * (85.0, 93.9)	98.7 * (95.3, 99.6)	94.6 (92.3, 96.2)	95.2 (92.8, 96.8)
IPMN	0.81 (0.73, 0.88)	0.67 (0.57, 0.76)	91.1 * (87.0,94.0)	76.6 * (70.6, 81.7)	85.0 (80.3, 88.7)	91.5 (87.2, 94.4)	88.0 (84.9, 90.5)	84.2 (80.6, 87.3)
MCN	0.52 (0.38, 0.66)	0.47 (0.35, 0.59)	53.1 (43.3, 62.6)	61.9 (51.2,71.6)	92.4 (89.5, 94.5)	87.6 (83.9, 90.6)	85.2 (81.9, 87.9)	82.9 (79.2, 86.1)
SCA	0.82 (0.69, 0.94)	0.92 (0.84, 1.00)	92.1 (82.7, 96.6)	98.2 (90.2, 99.7)	98.1 (96.4, 99.0)	99.3 (97.8, 99.8)	97.4 (95.6, 98.4)	99.1 (97.8, 99.7)
Cystic-NET or SPN	0.69 (0.58, 0.80)	0.77 (0.66, 0.88)	75.8 (66.1, 83.5)	85.9 (76.5, 91.9)	99.1 (97.7, 99.7)	98.7 (96.9, 99.4)	95.1 (92.9, 96.6)	96.5 (94.4, 97.8)
Pseudocyst	0.65 (0.44, 0.85)	0.51 (0.25, 0.77)	85.7 (65.4, 95.0)	88.9 (67.2, 96.9)	97.7 (95.9, 98.7)	94.5 (92.0, 96.3)	97.2 (95.4, 98.3)	94.3 (91.8, 96.1)

IOA: interobserver agreement; EUS-nCLE: Endoscopic ultrasound guided needle based confocal laser endomicroscopy; PCLs: pancreatic cystic lesions; IPMN: intraductal papillarymucinous neoplasm; MCN: mucinous cysticneoplasm; SCA: serous cystadenoma; NET/SPN: neuroendocrine tumors/solid pseudopapillaryneoplasm; CI: confidence interval.

^*p < 0.05.

3.6. Effect of diagnostic confidence

Compared to all video assessments, the accuracy of nCLE interpretation was significantly better in high-confident evaluations of IPMNs (91.9% vs. 86.2%, p < 0.05) and MCNs (90.2% vs. 84.1%, p < 0.05). This was driven by better sensitivity of nCLE to diagnose IPMNs (91.7% vs. 84.4%, p < 0.05) and improved specificity for MCNs (94.6% vs. 90.2%, p < 0.05) (Supplementary Table 3).

4. Discussion

In this large interobserver study using prospective multicenter databases of PCLs with confirmatory histopathology, we have demonstrated that nCLE-guided virtual biopsy of PCL is very accurate and reliable in differentiating PCLs and in diagnosing specific types of cysts. Further, independent observers rated a majority of the images as high quality and were able to achieve diagnostic conclusions with a high-level of confidence.

For the differentiation of mucinous from non-mucinous PCLs, nCLE achieved a high level of diagnostic accuracy (95%) among independent blinded observers. This finding aligns with previously published literature where nCLE imaging of papillary frond-like structures and epithelial bands diagnosed mucinous PCLs with high sensitivity (>90%), specificity (>97%), and accuracy (>90%) [8,11,12,16–18]. Amongst the observers, the IOA and IOR for nCLE were almost perfect for discriminating mucinous from non-mucinous PCLs. This study, utilizing gold-standard histopathology, is a substantial update to prior literature with fewer (n = 29 subjects) PCLs where the reference standard was nCLE-expert interpretation [9].

Subsequent to differentiation of PCLs, the observers’ accuracies for nCLE-guided diagnosis of specific mucinous cysts were 86% for IPMNs and 84% for MCNs, with IOA and IOR that were substantial for IPMNs and moderate for MCNs. When the confidence to diagnose these cysts was high, the accuracy for IPMNs and MCNs was above 90%, but the sensitivity remained ~60% for MCNs. Similar findings were reported in a retrospective study of 31 patients with PCLs, in which 4 nCLE-naïve observers diagnosed IPMNs and MCNs with high accuracy (90%), but with low sensitivity that ranged from 67 to 80% and with fair to moderate IOA [14]. The nCLE imaging of MCNs and IPMNs with low-grade dysplasia are sometimes similar and indistinguishable where both reveal a flat and thin layering epithelium (Fig. 1) [15]. Nearly half of the MCNs in clinical practice reveal additional findings of inflammatory cells and other uniform epithelioid cells (likely representing the luteal stroma). Although these additional characteristics are helpful to differentiate MCNs from IPMNs [15], these may not be sufficient to diagnose MCNs with high sensitivity and reliability even among experts. In clinical practice, other demographic and morphologic characteristics are complementary and could facilitate this diagnostic differentiation.

Amongst the PCLs determined to be non-mucinous by nCLE, the observers demonstrated high accuracy for diagnosing PCL subtypes (SCA, pseudocysts, and NET/SPNs: >95%). The diagnosis of SCAs was achieved with high specificity (98.6%) and sensitivity (94.9%), and almost perfect IOA (k = 0.85). These results are consistent with several other studies [8,9,12,18–20]. It is worth mentioning that the reported sensitivity of EUS-nCLE for SCAs was lower in early studies (59%–76%) [7,13], and is substantially better in recent studies (95%–100%) [12,14,18]. This may be due to the refinement of the nCLE criteria diagnostic for SCAs [18,19]. Pseudocysts were diagnosed with high specificity (96.2%) and sensitivity (87.2%), substantial IOR (k = 0.62), and moderate IOA (k = 0.57) in our study. Four studies previously reported similar high specificity with EUS-nCLE for pseudocysts, but with lower sensitivity (43%–67%), and better IOA (k, 0.79–1) and IOR (k, 0.78–0.91) [9,14,16,18]. Differences across study results may be because nCLE features of pseudocysts (autofluorescing inflammatory cells against a dark background, Fig. 1) are sometimes mimicked by MCNs when inflammatory cells are present and epithelial bands are unidentifiable due to epithelial denudation/atrophy [24,25]. Overall, EUS-nCLE features of SCAs and pseudocysts are highly accurate and specific, which may reduce unnecessary surgeries and surveillance in patients with these benign PCLs.

EUS-nCLE imaging facilitated the diagnosis of cystic NETs/SPNs with high accuracy (96%), and substantial IOA and IOR. The nCLE finding of well-demarcated dark clusters of cells separated by interstitial spaces/tumor stroma has previously been shown to be highly accurate (>90%), sensitive (>98%), and specific (>95%) for diagnosing cystic-NETs [8,9,18]. Only one retrospective study with limited number of NETs (n = 2) had previously reported on observers’ agreement of the nCLE trabecular pattern, demonstrating almost perfect IOA and substantial IOR [9]. Our study grouped NETs and SPNs (NET = 10, SPN = 3) into a single category, since we and others have demonstrated that nCLE imaging in both these cysts reveals identical patterns. Further differentiation of NETs and SPNs needs fine needle aspiration/biopsy and immunostaining [8,18].

A novel aspect of our study is the relationship between EUS-nCLE experience and diagnostic performance of image interpretation. Among the 13 observers, the individual procedural volume of EUS-nCLE did not influence the accuracy or agreement of nCLE-guided diagnosis of PCLs. These endosonographers had prior training in nCLE image analysis and experience of at least ten EUS-nCLE procedures. Although this study does not provide comprehensive data to develop learning curves, the results suggest that at least 10 EUS-nCLE procedures are necessary for accurate and reliable nCLE image interpretation.

The limitations of this analysis need deliberation. First, there is a potential for selection bias. We sought to compare nCLE findings to gold-standard histopathological diagnoses; hence, the study cohort primarily consisted of IPMNs and was under-represented by pseudocysts, similar to published large surgical series [6,26]. In clinical practice, however, a majority of PCLs referred for EUS-FNA are IPMNs and there is a minority of pseudocysts. Second, all the assessments were conducted by observers with experience in EUS-nCLE thus limiting the generalizability of these results. Although this study includes a large international observer base (n = 13 raters) that hopefully accounts for needed heterogeneity in training and practice, we cannot extrapolate our findings to those who are not performing EUS-nCLE. As discussed in a Delphi consensus report, a minimum experience of 10 cases of EUS-nCLE is required for independent image acquisition and interpretation [27]. While future studies need to assess training strategies and learning curves to attain high accuracy and reliability among endosonographers interested in performing nCLE, we have recently demonstrated that a ‘training-and-test’ strategy accomplished adequate diagnostic accuracy and IOA among nCLE-naïve observers [28]. Third, the study used multiple sources of prospective data to achieve a large sample size of PCLs with histopathology. This expanded pool of patients provided inclusion of non-IPMN PCLs that were under-represented in prior studies.

To conclude, we demonstrate that endosonographers’ review of representative EUS-nCLE videos can differentiate mucinous from non-mucinous PCLs with high accuracy and excellent inter-and- intraobserver agreement. Besides, nCLE imaging provides in-vivo virtual histologic diagnosis of the most prevalent PCLs subtypes with high accuracy and reliability. Considering the lack of optimization using standard of care or novel technologies in diagnosing PCLs, the addition of EUS-nCLE can facilitate appropriate management of these patients.

Abbreviations:

PCL

Pancreatic Cystic Lesion

nCLE

Needle Based Confocal Laser Endomicroscopy

IPMN

Intraductal Papillary Mucinous Neoplasm

IOA

Interobserver agreement

MCN

Mucinous Cystic Neoplasm

SCA

Serous Cystadenoma

Cystic-NET

Cystic Neuroendocrine Tumor

SPN

Solid Pseudopapillary Neoplasm

SD

Standard Deviation