Abstract
Purpose
This pilot study aimed to assess the feasibility of intraoperative assessment of safe margins with confocal laser endomicroscopy (CLE) during oropharyngeal squamous cell carcinoma (OPSCC) surgery.
Methods
We included five consecutive patients confirmed OPSCC and planned tumor resection in September and October 2020. Healthy appearing mucosa in the marginal zone, and the tumor margin, were examined with CLE and biopsy during tumor resection. A total of 12,809 CLE frames were correlated with the gold standard of hematoxylin and eosin staining. Three head and neck surgeons and one pathologist were asked to identify carcinoma in a sample of 169 representative images, blinded to the histological results.
Results
Healthy mucosa showed epithelium with uniform size and shape with distinct cytoplasmic membranes and regular vessel architecture. CLE optical biopsy of OPSCC demonstrated a disorganized arrangement of variable cellular morphology. We calculated an accuracy, sensitivity, specificity, PPV, and NPV of 86%, 90%, 79%, 88%, and 82%, respectively, with inter-rater reliability and κ-value of 0.60.
Conclusion
CLE can be easily integrated into the intraoperative setting, generate real-time, in-vivo microscopic images of the oropharynx for evaluation and demarcation of cancer. It can eventually contribute to a less radical approach by enabling a more precise evaluation of the cancer margin.
Keywords: Confocal laser endomicroscopy, Oropharyngeal squamous cell carcinoma, Safe surgical margins, Head and neck malignancies
Introduction
The incidence of oropharyngeal squamous cell carcinoma (OPSCC) is rising in the last decades due to an increased number of infections with oncogenic human papillomavirus [1] and is currently one of the most frequent malignancies in the head and neck region [2]. In early-stage OPSCC, therapeutic options recommend surgical tumor removal or radiotherapy as a single treatment modality [3]. Treatment recommendations for advanced stage OPSCC suggest either primary chemoradiation therapy or primary surgery followed by risk-adapted adjuvant treatment. Both primary radiation and primary surgery proved to be therapeutic options [4].
According to the World Health Organization, squamous cell carcinoma (SCC) is classified into highly differentiated (grade 1), moderately differentiated (grade 2), and poorly differentiated (grade 3) cancer [5]. Grade 1 SCC exhibits a growth pattern that resembles a normal epithelium, whereas grade 3 SCC shows more considerable cell pleomorphism, minimal keratinization, or none at all, high nuclear to cytoplasmic ratio, larger immature cells, and a markedly stromal reaction [5]. Several optical imaging methods, such as narrow-band imaging, fluorescence endoscopy, or optical coherence tomography, have been suggested as having the potential to improve the analysis of mucosal lesion with white light [6–9]. A further imaging modality for visual identification of suspicious mucosal lesions is CLE, magnifying power up to 1000 times using fluorescein to outline the intercellular spaces [9]. While a few studies in the last years analyzed CLE accuracy in detecting malignant mucosal alterations focusing on the diagnosis of a primary lesion, there is no data regarding the assessment of safe margins during oncological surgery [10–13]. CLE provides “real-time” optical biopsies [14], which defines the appropriate resection margin during surgical tumor removal due to its property making cellular structures visible.
This study aimed to assess the diagnostic value of probe-based CLE in identifying the oropharyngeal region's malignant lesions compared to the accepted gold standard of histopathological examination. Additionally, we aimed to assess the feasibility of intraoperative assessment of safe margins with CLE during planned oropharyngeal tumor resection.
Materials and methods
Study design
We conducted this prospective pilot study at a tertiary hospital and academic cancer center. The study was approved by the local institutional ethics committee and carried out following the Declaration of Helsinki. We obtained written, informed consent from all study participants.
Eligibility criteria
A total of five patients with confirmed OPSCC and planed tumor resection were included in this study in September and October 2020. We confirmed malignancy by biopsy out of the center of the tumor in a preceding panendoscopy. Exclusion criteria were a prior head and neck cancer, presence of distant metastasis, pregnancy, thyroid dysfunction, severe kidney failure, and allergy to Fluorescein.
Technical details
We performed intraoperative imaging using a GastroFlex probe and a 488 nm Cellvizio laser scanning system (Mauna Technologies, Paris, France). The probe has a diameter of 2.6 mm, a penetration depth of 55—65 μm, a field of view of 240 μm, and a resolution of 1 μm. We used 5 ml Fluorescein Alcon 10% (Alcon Pharma, Freiburg, Germany) as an optical imaging dye. Fluorescein distributes to the intercellular spaces and cytoplasmatic components via microcirculation, without staining the cell nuclei. Thus, enabling outline visualization and structural analysis of cellular tissue and the loss of nuclear polarity and abnormal microvasculature patterns which are usually present in tumor tissue [15, 16]. With this imaging method, laser light is emitted and applied to tissue at a selected depth (e.g., 60 µm). The reflected fluorescence of light is refocused for detection. A pinhole excludes scattered or reflected light from other depth planes, thus enabling an increased spatial resolution [17].
Surgical procedure and intraoperative imaging
We used an Mc Ivor tongue retractor or a Jennings mouth gag and exposed the tumor. Subsequently, 2.5 ml Fluorescein Alcon 10% was injected intravenously, and after 2 min, the laser scanning unit was initiated (Fig. 1). After around 5 min of examination, we administrated additional 2.5 ml because the quality of imaging began to degrade this time interval [18]. We correlated the CLE imaging with histopathology's gold standard. The regions recorded were marked with suture, or a separate biopsy was performed. The histopathological assessment followed a standard protocol with hematoxylin and eosin (H&E) staining. After completing the CLE examination, we performed the tumor resection with a macroscopic safety margin of 1 cm. Our and international treatment standards were not altered or influenced in any way by the application of CLE.
Data analysis
The investigator edited CLE data postoperatively using Cellvizio Viewer software 1.6.2. A total of 12.809 CLE frames were analyzed, and 169 representative images from different mucosal areas of the tumor margin, the marginal zone, and the resection margin were selected. All images were of high quality and free of artifacts. Besides, each image contains a corresponding H&E pattern as a reference standard. The 169 representative images were extracted and presented to four medical professionals (blinded examiners) for assessment. The professionals consisted of three head and neck surgeons with confocal microscopy experience and one pathologist without previous experience in this technique. The blinded examiners had to allocate the CLE image into the categories of “healthy” and “carcinoma in situ, or invasive carcinoma.” As has already been mentioned, the maximum penetration depth is approximately 60 µm. Therefore, it is impossible to distinguish carcinoma in situ from invasive carcinoma since stromal invasion cannot be used as a criterion to differentiate between the two [12, 19].
Statistical analysis
We performed statistical analysis using SPSS version 22.0 (IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY, USA). The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy were calculated for each examiner. Assessment of the results was blinded to histological findings. The histological findings were regarded as the reference standard. The inter-rater reliability/agreement was tested statistically using Cohen’s kappa (Cohen’s kappa coefficient) and Fleiss kappa. The κ-values were interpreted according to the widely accepted Landis and Koch classification [20]. If the agreement obtained between the two raters is mathematically equivalent to chance, κ is zero (no agreement). Values of κ between 0 and 0.20 are defined as low, between 0.21 and 0.40 as adequate, between 0.41 and 0.60 as moderate, between 0.61 and 0.80 as substantial, and between 0.81 and 1.0 almost perfect.
Results
Patient characteristics
In 08—10/2020, we enrolled five patients (all male; mean age 58.2 years, range 51–61 years) to undergo pharyngeal in vivo CLE during planned transoral tumor resection. Patient characteristics, including stage, are shown in Table 1. All patients underwent transoral tumor resection surgery. One case was performed without the need for reconstruction. The other four cases underwent free flap reconstruction (two with anterior lateral thigh flap and two with radial forearm flap). SCC and healthy mucosa (safe surgical margins) were found in CLE and confirmed by histopathology in all patients.
Table 1.
Case | Age (years) | Tumor stage | Location | Grade | p16 | CLE frames | Time | |
---|---|---|---|---|---|---|---|---|
1 | 57 | pT2 | Tonsil | > 0,5 cm | G2 | Neg | 2217 | 4 m 37 s |
2 | 61 | pT2 | Soft palate | > 0,5 cm | N/A | Pos | 1960 | 4 m 5 s |
3 | 51 | pT3 | Soft palate | > 0,5 cm | G3 | Neg | 2418 | 5 m 2 s |
4 | 61 | pT2 | Soft palate | > 0,5 cm | G3 | Neg | 3498 | 7 m 17 s |
5 | 61 | pT3 | Tonsil | > 0,5 cm | G3 | Neg | 2716 | 5 m 40 s |
N/A not applicable
CLE image acquisition
The average image acquisition time for each case was 5 min, 20 s (range: 4 min 5 s–7 min 17 s). In amount, we acquired 12,809 CLE frames, which were assessed and classified according to SCC or healthy epithelium.
In CLE, SCC is characterized by unorganized tissue architecture whereby the intercellular gaps are not definable in some places. Cells are irregular in shape, the intensity of contrast, and size, with intensified fluorescein leakage from the dilated capillaries (Fig. 2).
Benign squamous epithelium presents with homogeneous cells of uniform size, shape with distinct cytoplasmic membranes, and regular vessel architecture (Fig. 3). A regular cell border and normal capillaries are the most distinct features in comparison to SCC.
Diagnostic metrics
The performance of our binary classification is illustrated in Table 2. The examiners achieved an accuracy, sensitivity, specificity, PPV, and NPV were in the range of 79–92%, 84–95%, 52–95%, 77–97%, and 76–86%. In summary, we could show an accuracy, sensitivity, specificity, PPV and NPV of 85.8%, 89.9%, 78.6%, 87.9% and 81.9%, respectively. The four observers' overall agreement was calculated with a Fleiss’ κ of 0.60 for the 169 images and can, therefore, be considered sufficient. In computing the inter-rater reliability in the selective analysis between two examiners, Cohens’ κ ranged from 0.32 to 0.78 with a percentage agreement level ranging from 69.2 to 89.3% (Table 3).
Table 2.
Diagnostic Metrics | P | S1 | S2 | S3 | All |
---|---|---|---|---|---|
Value (95%CI) | Value (95%CI) | Value (95%CI) | Value (95%CI) | Value (95%CI) | |
Sensitivity | 89.72% (82.35–94.76) | 95.33% (89.43–98.47) | 90.65% (83.48–95.43) | 84.11% (75.79–90.46) | 89.95% (86.71–92.63) |
Specificity | 80.65% (68.63–89.58) | 51.61% (38.56–64.50) | 95.16% (86.50–98.99) | 87.10% (76.15–94.26) | 78.63% (73.00–83.56) |
PPV | 88.89% (82.74–93.03) | 77.27% (72.38–81.52) | 97.00% (91.46–98.99) | 91.84% (85.43–95.57) | 87.90% (85.10–90.24) |
NPV | 81.97% (71.93–88.96) | 86.49% (72.46–93.97) | 85.51% (76.53–91.43) | 76.06% (67.03–83.23) | 81.93% (77.22–85.85) |
Accuracy | 86.39% (80.28–91.17) | 79.29% (72.39–85.13) | 92.31% (87.21–95.84) | 85.21% (78.94–90.19) | 85.80% (82.94–88.34) |
S surgeon, P pathologist, PPV positive predictive value, NPV negative predictive value; (95% Confidence Interval)
Table 3.
Observer pair | Cohens Kappa/Fleiss-Kappa | Agreement (%) |
---|---|---|
S1 vs S2 | 0.45 | 75.2 |
S1 vs S3 | 0.32 | 69.2 |
S1 vs P | 0.44 | 76.3 |
S2 vs S3 | 0.73 | 86.9 |
S2 vs P | 0.78 | 89.3 |
S3 vs P | 0.60 | 81.1 |
S1 vs S2 vs S3 vs P | 0.60 |
Interobserver agreement for observer pairs (Cohens Kappa) and for multiple Raters (Fleiss Kappa) as well as percentual agreement
Discussion
In this pilot study, we report on the feasibility of in-vivo real-time intraoperative assessment of safe margins during tumor resection of OPSCC via confocal laser imaging. The blinded examination of representative images of oropharyngeal cancer and healthy epithelium cells showed a sensitivity and specificity of, respectively, 89.9% and 78.6%. The tumor's exposure was performed in a standardized fashion, and the examination added around 10 min to standard operation time without any side effects. The 2.6 mm CLE probe provided sufficient spatial resolution and tissue contrast to distinguish cellular architecture, borders, and size to characterize healthy and malignant tissue in vivo specimens. Imaging of healthy areas of the pharynx (confirmed by histopathological assessment) after tumor resection allowed for the benign squamous epithelium classification as having cells of uniform size and shape with distinct cytoplasmic membranes and regular vessel architecture. We could also detect a transition between healthy appearing tissue and suspicious lesions observed with CLE, showing that CLE can precisely evaluate the cancer margin. CLE optical biopsy of SCC demonstrated a disorganized and variable cellular morphology, lack of cytoplasmic membrane, which could correlate with H&E staining. The blinded examiners (S1-3 and P) showed a moderate interrater agreement. Available reports on in vivo confocal laser endomicroscopy images on the head and neck are mostly related to the larynx and oral mucosal lesions. Only very few data on oropharyngeal cancer and the tonsillar region is available [10, 12, 13, 21, 22]. Linxweiler et al. examined, among other areas, the tonsillar region of 12 patients. However, this was performed on formalin-fixed samples with topical use of acriflavine or using the cells’ autofluorescence, which does not adequately portray the in vivo intraoperative situation, limiting its transferability [11].
We used intravenous fluorescein as a dye to characterize the morphological features of normal epithelium and SCC with CLE. A sufficient resolution of in vivo cellular architecture within a few minutes of fluorescein administration was feasible. CLE image interpretation has a moderate learning curve, and regarding the interrater reliability, there is inconsistency in the literature [13, 21, 23]. Oetter et al. developed and validated a classification and scoring system to diagnose oral SCC through CLE [10]. However, this system has not yet been validated for OPSCC. Indeed, CLE’s application by one of the raters, a maxillofacial surgeon with extensive experience in the surgical treatment and intraoperative use of CLE in oral cancer, shows a relatively low specificity and high sensitivity (S1: 51.6%, 95.3%; Table 2). We postulate that this is due to the atypical aspect of oropharyngeal and tonsillar epithelium compared to the oral mucosa. Large mucosal cells are typical in the oropharyngeal epithelium and were not considered in the DOI Classification by Oetter et al. (Fig. 4).
An accurate histopathological diagnosis of OPSCC is vital in guiding clinical management. CLE images have a similar resolution to traditional H&E and thus could serve as valuable adjunct technology by potentially minimizing the need for intraoperative frozen section analysis. Pathological assessment of frozen sections and biopsy specimens does not typically involve the surgeon; however, a trained head and neck surgeon can perform CLE image interpretation for a real-time distinction of normal from abnormal tissue. The ability to sample the entirety of a tumor and surrounding tissue at the time of surgery may help guide the procedure's extension. This is of paramount relevance since an extensive resection may require free flap reconstructive surgery with considerable morbidity and prolonged operating time. Current evidence suggests that a safe margin of 1 mm may be sufficient for OPSCC [24]. However, intraoperative frozen sections are limited by the under-sampling, processing time, and the use of the tissue of interest [25], thus underscoring the need for improved intraoperative diagnostics. CLE enables a rapid scanning of the tumor margin and could potentially contribute to a more accurate resection, thus reducing the resection extension. Although this technique’s potential has been recognized over the years by a few groups, it has not yet achieved a broad clinical application.
A further possibility to objectivize CLE findings is utilizing automatic classification [26]. Aubreville et al. showed that an approach based on transfer learning from intermediate endpoints within a pre-trained inception v3 network with preprocessing could reach an overall 94.8% accuracy, significantly improving overall performance over the traditional state of the art feature-based machine learning approaches [26]. Automatic classification methods for confocal laser endomicroscopy in the head and neck were developed for vocal cords and oral mucosa, but until this point, there is no available data for the oropharyngeal region, especially for the palatine tonsil. Prerequisite in developing such an automatic classification method based on deep learning-based approaches in acquiring large amounts of data and the correct labeling of such images in the healthy epithelium of a specific anatomic region and cancerous cells. This study demonstrated image acquisition's feasibility in the oropharyngeal region and CLE application in an intraoperative setting. We showed that a blinded examination of these images could be performed with 85.8% accuracy-based solely on known characteristics of healthy epithelium and cancerous tumor cells in general. Interestingly, compared to a previous study on the transferability of automatic classification based on algorithms trained with images of vocal cords applied on the oral cavity and vice versa, an accuracy of 68.5% and 89.5% was found. This suggests that the epithelium of these areas and the carcinomas that arise are similar on CLE; however, there seem to be significant differences that limit its detection rate. The same difficulties can also be pointed to the blinded examiners, mostly in the oral cavity (S1) and vocal cords/Larynx (S2, S3).
The present work increases the information and knowledge until now less examined anatomic region. The absence of classification criteria and small data available limit CLE's clinical application in head and neck until this day. CLE is also associated with considerable costs as a scanning unit costs around $200 000, and a single application costs $275 ($250 for one probe use and $25 for the contrast agent Fluorescein 10% 5 ml) with a significant decrease in image quality after 8 min of recording [17]. Since CLE was not yet demonstrated sufficient clinical added value, the costs are difficult to justify in the face of increased economic pressure outside of clinical trials. CLE has a considerable potential to aid intraoperative resection, dependent on further validation in clinical trials.
Conclusion
CLE is a promising imaging technology that may aid in intraoperative decision-making during oropharyngeal tumor removal CLE can be used to generate real-time, in-vivo microscopic examination of the oropharynx for evaluation and demarcation of cancer. This evaluation can be easily integrated into the intraoperative setting to help the surgeon guide its resection and eventually contribute to a less radical approach.
Acknowledgements
This project was supported by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) Grant Number GO 3182/2-1, MA 4898/17-1, OE 743/1-1, STE 1877/7-1; Project Number 439264659.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Compliance with ethical standards
Conflict of interest
None of the authors have a personal conflict of interest to declare.
Ethical approval
All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Ethical approval (approval number 60_14 B) was given by the Non-Intervention Clinical Research Ethics Committee of the Medical Faculty.
Informed consent
A formal informed consent procedure was obtained from all study participants.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Change history
4/9/2021
Funding note was missing and included in this version.
References
- 1.Chaturvedi AK, Engels EA, Pfeiffer RM, Hernandez BY, Xiao W, Kim E, Jiang B, Goodman MT, Sibug-Saber M, Cozen W, Liu L, Lynch CF, Wentzensen N, Jordan RC, Altekruse S, Anderson WF, Rosenberg PS, Gillison ML. Human papillomavirus and rising oropharyngeal cancer incidence in the United States. J Clin Oncol. 2011;29:4294–4301. doi: 10.1200/JCO.2011.36.4596. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Hussein AA, Helder MN, de Visscher JG, Leemans CR, Braakhuis BJ, de Vet HCW, Forouzanfar T. Global incidence of oral and oropharynx cancer in patients younger than 45 years versus older patients: A systematic review. Eur J Cancer. 2017;82:115–127. doi: 10.1016/j.ejca.2017.05.026. [DOI] [PubMed] [Google Scholar]
- 3.Turner MT, Byrd JK, Ferris RL. Current role of surgery in the management of oropharyngeal cancer. J Oncol Pract. 2016;12:1176–1183. doi: 10.1200/JOP.2016.015263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Boscolo-Rizzo P, Gava A, Baggio V, Marchiori C, Stellin M, Fuson R, Lamon S, Da Mosto MC. Matched survival analysis in patients with locoregionally advanced resectable oropharyngeal carcinoma: platinum-based induction and concurrent chemoradiotherapy versus primary surgical resection. Int J Radiat Oncol Biol Phys. 2011;80:154–160. doi: 10.1016/j.ijrobp.2010.01.032. [DOI] [PubMed] [Google Scholar]
- 5.Agaimy A, Weichert W. Grading von Tumoren der Kopf-Hals-Region [Grading of head and neck neoplasms] Pathologe. 2016;37:285–292. doi: 10.1007/s00292-016-0173-9. [DOI] [PubMed] [Google Scholar]
- 6.Tirelli G, Marcuzzo AV, Boscolo Nata F. Narrow-band imaging pattern classification in oral cavity. Oral Dis. 2018;24:1458–1467. doi: 10.1111/odi.12940. [DOI] [PubMed] [Google Scholar]
- 7.Betz CS. Optical diagnostic methods for improving early tumor diagnosis in the upper aerodigestive tract. HNO. 2016;64:41–48. doi: 10.1007/s00106-015-0104-8. [DOI] [PubMed] [Google Scholar]
- 8.Piazza C, Dessouky O, Peretti G, Cocco D, De Benedetto L, Nicolai P. Narrow-band imaging: a new tool for evaluation of head and neck squamous cell carcinomas. Review of the literature. Acta Otorhinolaryngol Ital. 2008;28:49–54. [PMC free article] [PubMed] [Google Scholar]
- 9.Lingen MW, Kalmar JR, Karrison T, Speight PM. Critical evaluation of diagnostic aids for the detection of oral cancer. Oral Oncol. 2008;44:10–22. doi: 10.1016/j.oraloncology.2007.06.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Oetter N, Knipfer C, Rohde M, von Wilmowsky C, Maier A, Brunner K, Adler W, Neukam FW, Neumann H, Stelzle F. Development and validation of a classification and scoring system for the diagnosis of oral squamous cell carcinomas through confocal laser endomicroscopy. J Transl Med. 2016;14:159. doi: 10.1186/s12967-016-0919-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Linxweiler M, Kadah BA, Bozzato A, Bozzato V, Hasenfus A, Kim YJ, Wagner M, Igressa A, Schick B, Charalampaki P. Noninvasive histological imaging of head and neck squamous cell carcinomas using confocal laser endomicroscopy. Eur Arch Otorhinolaryngol. 2016;273:4473–4483. doi: 10.1007/s00405-016-4145-8. [DOI] [PubMed] [Google Scholar]
- 12.Goncalves M, Iro H, Dittberner A, Agaimy A, Bohr C. Value of confocal laser endomicroscopy in the diagnosis of vocal cord lesions. Eur Rev Med Pharmacol Sci. 2017;21:3990–3997. [PubMed] [Google Scholar]
- 13.Goncalves M, Aubreville M, Mueller SK, Sievert M, Maier A, Iro H, Bohr C. Probe-based confocal laser endomicroscopy in detecting malignant lesions of vocal folds. Acta Otorhinolaryngol Ital. 2019;39:389–395. doi: 10.14639/0392-100X-2121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Parikh ND, Gibson J, Nagar A, Ahmed AA, Aslanian HR. Confocal laser endomicroscopy features of sessile serrated adenomas/polyps. United Eur Gastroenterol J. 2016;4:599–603. doi: 10.1177/2050640615621819. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.De Palma GD. Confocal laser endomicroscopy in the “in vivo” histological diagnosis of the gastrointestinal tract. World J Gastroenterol. 2009;15:5770–5775. doi: 10.3748/wjg.15.5770. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Nguyen NQ, Leong RW. Current application of confocal endomicroscopy in gastrointestinal disorders. J Gastroenterol Hepatol. 2008;23:1483–1491. doi: 10.1111/j.1440-1746.2008.05469.x. [DOI] [PubMed] [Google Scholar]
- 17.Chauhan SS, Dayyeh BKA, Bhat YM, et al. Confocal laser endomicroscopy. Gastrointest Endosc. 2004;80:928–938. doi: 10.1016/j.gie.2014.06.021. [DOI] [PubMed] [Google Scholar]
- 18.Becker V, von Delius S, Bajbouj M, Karagianni A, Schmid RM, Meining A. Intravenous application of fluorescein for confocal laser scanning microscopy: evaluation of contrast dynamics and image quality with increasing injection-to-imaging time. Gastrointest Endosc. 2008;68:319–323. doi: 10.1016/j.gie.2008.01.033. [DOI] [PubMed] [Google Scholar]
- 19.Volgger V, Girschick S, Ihrler S, Englhard AS, Stepp H, Betz CS. Evaluation of confocal laser endomicroscopy as an aid to differentiate primary flat lesions of the larynx: a prospective clinical study. Head Neck. 2016;38:1695–1704. doi: 10.1002/hed.24303. [DOI] [PubMed] [Google Scholar]
- 20.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. doi: 10.2307/2529310. [DOI] [PubMed] [Google Scholar]
- 21.Moore C, Mehta V, Ma X, Chaudhery S, Shi R, Moore-Medlin T, Lian T, Nathan CA. Interobserver agreement of confocal laser endomicroscopy for detection of head and neck neoplasia. Laryngoscope. 2016;126:632–637. doi: 10.1002/lary.25646. [DOI] [PubMed] [Google Scholar]
- 22.Nathan CA, Kaskas NM, Ma X, Chaudhery S, Lian T, Moore-Medlin T, Shi R, Mehta V. Confocal Laser endomicroscopy in the detection of head and neck precancerous lesions. Otolaryngol Head Neck Surg. 2014;151:73–80. doi: 10.1177/0194599814528660. [DOI] [PubMed] [Google Scholar]
- 23.Chang TC, Liu JJ, Hsiao ST, Pan Y, Mach KE, Leppert JT, McKenney JK, Rouse RV, Liao JC. Interobserver agreement of confocal laser endomicroscopy for bladder cancer. J Endourol. 2013;27:598–603. doi: 10.1089/end.2012.0549. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Just T, Stave J, Bombor I, Kreutzer HJ, Guthoff R, Pau HW. In vivo diagnosis of epithelial changes of the oropharynx using confocal microscopy. Laryngorhinootologie. 2007;87:174–180. doi: 10.1055/s-2007-966971. [DOI] [PubMed] [Google Scholar]
- 25.Aubreville M, Knipfer C, Oetter N, Jaremenko C, Rodner E, Denzler J, Bohr C, Neumann H, Stelzle F, Maier A. Automatic classification of cancerous tissue in laserendomicroscopy images of the oral cavity using deep learning. Sci Rep. 2017;7:11979. doi: 10.1038/s41598-017-12320-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Aubreville M, Stoeve M, Oetter N, Goncalves M, Knipfer C, Neumann H, Bohr C, Stelzle F, Maier A. Deep learning-based detection of motion artifacts in probe-based confocal laser endomicroscopy images. Int J Comput Assist Radiol Surg. 2019;14:31–42. doi: 10.1007/s11548-018-1836-1. [DOI] [PubMed] [Google Scholar]