A tablet-interfaced high-resolution microendoscope with automated image interpretation for real-time evaluation of esophageal squamous cell neoplasia

Abstract

Background and Aims

In recent years, high-resolution microendoscopy (HRME) has shown potential to improve screening for esophageal squamous cell neoplasia (ESCN). Furthering its utility in a clinical setting, especially in lower-resource settings, could be accomplished by reducing the size and cost of the system as well as incorporating the ability of real-time, objective feedback. This article describes a tablet-interfaced HRME with fully automated, real-time image analysis.

Methods

The performance of the tablet-interfaced HRME was assessed by acquiring images from the oral mucosa in a normal volunteer. An automated, real-time analysis algorithm was developed and evaluated using training, test, and validation images from a previous in vivo study of 177 patients referred for screening or surveillance endoscopy in China. The algorithm was then implemented in a tablet HRME that was used to obtain and analyze images from esophageal tissue in 3 patients. Images were displayed alongside the probability that the imaged region was neoplastic.

Results

The tablet-interfaced HRME demonstrated comparable imaging performance at lower cost compared with first-generation laptop-interfaced HRME systems. In a post-hoc quantitative analysis, the algorithm identified neoplasia with a sensitivity and specificity of 95% and 91% in the validation set, compared with 84% and 95% achieved in the original study.

Conclusion

The tablet-based HRME is a low-cost tool that provides quantitative diagnostic information to the endoscopist in real-time. This could especially be beneficial in lower-resource settings for operators with less experience interpreting HRME images.

Introduction

Esophageal cancer (EC) is the sixth leading cause of cancer death worldwide. Most EC deaths occur in developing countries, where nearly all cases are esophageal squamous cell carcinoma (ESCC)^1,2,3. Only 20% of patients diagnosed with ESCC survive longer than 3 years, primarily due to late-stage diagnosis. Early diagnosis is associated with significantly improved outcomes^4,5. Therefore, there is need for a low-cost and accurate means of diagnosing early stage ESCC and its precursor lesion, high-grade squamous dysplasia, which together are called early esophageal squamous cell neoplasia (ESCN), in resource-limited settings.

Lugol’s chromoendoscopy (LCE) is the current criterion standard for ESCN screening. Although the sensitivity of LCE for ESCN is over 95%, LCE has poor specificity (< 65%) due to false positive findings from inflammatory lesions, which are visually indistinguishable from high-grade dysplasia or cancer^6,7. The low specificity of LCE leads to many unnecessary biopsies, which increases the cost of screening⁸. Other imaging modalities such as confocal laser endomicroscopy (CLE) and narrow-band imaging (NBI) can discriminate between non-neoplastic and neoplastic tissue with higher specificity^6,9,10. However, the size and cost of CLE and the limited specificity of NBI are drawbacks in lower-resource settings^9,10. Developing low-cost, rugged tools to evaluate suspicious regions with high specificity in real-time could lessen the time needed for a definitive diagnosis and treatment decision which would be invaluable in lower-resource settings.

High-resolution microendoscopy (HRME) uses a low-cost, fiber-optic fluorescence microscope to image cellular morphology of the surface epithelium^11,12. The cost of goods to build an HRME system is less than $5000^13,14. The technique has been used in the oral cavity, cervix, and esophagus^13,14,15,16. In an endoscopy setting, a fiber-optic probe is inserted through the endoscope biopsy channel, adding approximately 4 to 6 minutes to the procedure¹³. Post-hoc quantitative analysis of HRME images has reported the ability to discriminate between normal and neoplastic tissue with high sensitivity and specificity^14,16. In conjunction with LCE, HRME was able to achieve a sensitivity of 91% and specificity of 88% in a per biopsy analysis¹³. Although promising, widespread use of the HRME is limited by the need for a bulky laptop to control the system and the need for training to interpret acquired images. Reducing the cost and size of the system as well as minimizing the learning curve for image interpretation would make HRME more accessible. This article describes an improved HRME system that is smaller and more economical than its predecessor while adding the functionality of real-time image interpretation.

Materials and Methods

Optical Design

Figure 1A shows a schematic of the tablet-based HRME. The system is a more-portable, less-costly adaptation of a previous design¹¹; key changes include a smaller, less-expensive camera sensor (Point Grey, CMLN-13S2M-CS) and a single achromatic lens in place of a microscope objective to couple light into the fiber optic bundle. The HRME system is connected to a tablet interface (Microsoft Surface Pro) through a USB connection (Figure 1B). The tablet displays images collected by the HRME at video rate while the user saves images from regions of interest using a foot pedal.

Figure 1.

Table- based high-resolution microendoscopy (HRME). A, Optical schematic of HRME system. B, Photograph of system plugged into tablet interface

Algorithm Design

We developed a new, fully automated algorithm to identify regions of interest free from artifact, to segment nuclei within these regions, and to calculate morphometric image features (Figure 2). The algorithm initially segments and filters the raw image to calculate a region of interest (ROI) free of debris in the imaging field of view (FOV). Next, the algorithm adjusts the contrast of the image and converts the image into a binary image in order to separate the nuclei and cytoplasm. Each distinct object in the binary image is then sorted and separated into 2 images, a non-cluster image and a cluster image¹⁷. In this case, non-clusters refer to well-segmented nuclei. Clusters refer to overlapping nuclei which require additional watershed segmentation. Both images are then filtered to remove very small objects and combined again to form a single binary image. The algorithm then computes morphological parameters derived from nuclear size and shape.

Figure 2.

Flow chart of image analysis algorithm steps

Algorithm Development and Validation

Three existing HRME image datasets of in vivo squamous esophageal mucosa obtained from 177 patients with the original HRME system were used to develop, validate, and evaluate algorithm performance¹⁶. In the original study, patients undergoing standard upper endoscopy for screening or surveillance of esophageal squamous neoplasia were imaged in vivo using high-resolution microendoscopy. Imaged sites were biopsied and a consensus diagnosis provided by 2 expert gastrointestinal pathologists. A training set (104 biopsied sites from 54 patients) and a test set (104 biopsied sites from 45 patients) were used to develop and validate candidate algorithms. Details of the selection criteria and composition of the 3 image sets are described in Shin et al¹⁶. The algorithm was developed using the images from the training set only; data from the test and validation sets were used to independently evaluate performance of the algorithm. Candidate algorithms were ranked on the basis of their performance with the test set. Finally, the best-performing algorithm was evaluated using an independent validation set comprised of images from 167 biopsied sites from 78 patients.

For each image, we calculated the nuclear-to-cytoplasmic area ratio and the mean and standard deviation of the following features: nuclear area, nuclear eccentricity, and solidity. A 2-class linear discriminant algorithm was developed to discriminate between neoplastic (high-grade dysplasia or ESCC) and non-neoplastic (normal, esophagitis, or low-grade dysplasia) images for each single feature in the training and test sets. Diagnostic performance for each metric was determined by generating a receiver operating characteristic (ROC) curve and then calculating the corresponding area under the curve (AUC) for the training and test sets using a custom MATLAB script (Mathworks, Natick, Mass).

We also evaluated algorithms based on the fraction of nuclei within a FOV that exceeded certain morphometric criteria. In this approach, individual nuclei were classified as normal or abnormal based upon a threshold for nuclear area and eccentricity. Nuclear area and eccentricity were selected as the criteria due to their high performance in this analysis; previous studies have also shown that these parameters are useful to classify neoplasia^16,17. The threshold was defined and evaluated in 4 separate ways such that a nucleus would be classified as abnormal if (1) the area was greater than the cutoff, (2) the eccentricity was greater than the cutoff, (3) the area OR the eccentricity was greater than the respective cutoffs or, (4) the area AND eccentricity were greater than the respective cutoffs. The cutoff values for area and eccentricity were determined from the 2-class linear discriminant algorithm for the mean nuclear area and nuclear eccentricity described above; the threshold was set at the value of the parameter at the point of the ROC curve where the sensitivity and specificity were maximized corresponding to the test set. Again, a 2-class linear discriminant algorithm was developed to discriminate between neoplastic and non-neoplastic images for each of the 4 features described and applied to the training and test sets. The best performing algorithm was then applied to the validation set and the corresponding ROC curve and AUC were calculated.

System Characterization and Clinical Performance

The performance of the tablet-based HRME was characterized by using optical standards and by imaging oral mucosa in a healthy volunteer. The normal volunteer was consented in accordance with a protocol approved by the Rice University Institutional Review Board. Image features were compared with those obtained with the original HRME. To evaluate the performance of the tablet-based HRME and image analysis algorithm in a clinical setting, the system was used to measure images from endoscopically normal and abnormal esophageal tissue in 3 patients who underwent endoscopy for suspicion of esophageal cancer. These patients were consented in accordance with a protocol approved by the Institutional Review Boards at The First Hospital of Jilin University (Changchun, China), The Cancer Institute at the Chinese Academy of Medical Sciences (Beijing, China), and Ben Taub Hospital (Houston, Tex). Results of the real-time algorithm were compared with visual interpretation of the HRME image and histologic diagnosis.

Results

Algorithm Evaluation

Table 1 shows the AUC for data in the training and test sets for algorithms based on each of the features described above. Based on performance in the test set, algorithms based on the fraction of nuclei exceeding the eccentricity AND area thresholds demonstrated the highest performance with an AUC of 0.973 for the test set. The cutoff values for nuclear area and eccentricity used were 171 μm² and 0.705 respectively. Algorithms based on other combinations of 2 or 3 features did not show appreciable performance improvement (data not shown).

Table 1.

Parameters evaluated for the training and test sets and corresponding areas under the curve (AUC)

	AUC (Training)	AUC (Test)
Nuclear to cytoplasmic area ratio	0.934	0.811
Mean area	0.771	0.874
St. Dev. Area	0.852	0.909
Mean eccentricity	0.924	0.967
St. Dev eccentricity	0.795	0.776
Mean solidity	0.842	0.779
St. Dev solidity	0.821	0.798
% nuclei in FOV above area threshold	0.744	0.887
% nuclei in FOV above eccentricity threshold	0.916	0.968
% nuclei in FOV above area OR eccentricity threshold	0.903	0.964
% nuclei in FOV above area AND eccentricity threshold	0.873	0.973

As the best performing metric, the fraction of nuclei exceeding the eccentricity AND area thresholds was evaluated with an independent validation set and resulted in an AUC of 0.937 with a sensitivity of 95% and a specificity of 91% using a cutoff of 27.5% (Figure 3). The average computation time for the algorithm was approximately 5 seconds.

Figure 3.

Receiver operating characteristic curves for linear discriminant analysis algorithm based on the percentage of nuclei above the mean area and eccentricity threshold for the (A) training set, (B) test set, and (C) validation set. The Q points correspond to a sensitivity and specificity of (A) 93% and 82%, (B) 93% and 94%, and (C) 95% and 91%.

System Characterization and Clinical Performance

Figure 4A–B shows images obtained with both the original and tablet-based HRME systems of a USAF resolution target; both systems resolve lines 4.4 μm in width. Figure 4C–D shows images of in vivo human oral mucosa of a normal volunteer; the ratio of the nuclear-to-cytoplasmic signal is similar for both systems (1.3 vs 1.4). Table 2 compares the technical specifications between the original and tablet-based HRME systems. Despite being a fraction of the size, weight, and cost, the tablet-based HRME performs comparably with the original HRME system.

Figure 4.

Comparison of system performance. Image of standard US Air Force resolution target using (A) standard HRME system and (B) tablet HRME system. Image of squamous epithelium in normal volunteer bottom lip using (C) standard HRME system and (D) tablet HRME system. Scale bar corresponds to 100 μm.

Table 2.

Comparison of technical specifications of original HRME and tablet – based HRME

	Standard HRME	Tablet HRME
Dimensions (L × W × H) cm	20 × 25 × 6.4	23 × 13 × 6.4
Weight (kg)	2.3	0.91
Frame Rate (Hz)	10	15
Output Power (mW)	1.0	1.0
Cost ($)	5000	1500

As a proof-of-concept, HRME images were then obtained from 3 patients using the tablet-based system; all imaging sites were taken from Lugol’s voiding regions identified from LCE. Real-time algorithm results were compared with visual interpretation of the HRME image and histology (Figure 5). Nuclei below the threshold for size and eccentricity are outlined in yellow; those above the threshold are outlined in red in the processed image. The HRME image from site 1 was classified visually as non-neoplastic; this was consistent with both the result of the real-time algorithm (7% of nuclei exceeded threshold) and the histologic diagnosis of esophagitis. The HRME image from site 2 was classified visually as neoplastic; however, the histologic diagnosis was also esophagitis. In this case, the real-time algorithm classified the site correctly as non-neoplastic (18.5% of nuclei exceeded threshold). The HRME image from site 3 was classified visually as neoplastic; this was consistent with both the result of the real-time algorithm (34.5% of nuclei exceeded threshold) and the histologic diagnosis of high-grade dysplasia.

Figure 5.

Screenshots of tablet user interface with integrated image processing for 3 esophageal site (A, B, and C). Corresponding visual interpretation and pathology for each site is listed on the right. Nuclei outlined in yellow are classified as normal whereas nuclei outlined in red are classified as abnormal based on its area and eccentricity. The numerical value and slide bar at the bottom right indicate the percentage of nuclei classified as abnormal. The slide bar changes color based on the algorithm’s evaluation of the image (green-normal, red-neoplastic).

Discussion

With ESCC primarily impacting developing countries, there is a need for a low-cost technology that is operable by less experienced providers and can provide accurate results. Although LCE can identify ESCC with high sensitivity, its low specificity leads to unnecessary biopsies which are especially burdensome in lower-resource settings. Previous work has shown that HRME used in conjunction with LCE can improve specificity¹³. Here, we present a smaller, more economical, tablet-based HRME system that incorporates automated image interpretation at the point-of-care with the goal of making the HRME more accessible in lower-resource settings. The cost-of-goods for the tablet HRME itself is <$1500 compared with $5000 for the first generation HRME. The tablet-HRME has similar imaging performance to the original system; its smaller size, lower cost, and automated analysis help address many of the barriers to use in low-resource settings and by non-expert users. In addition, the shift to a tablet-based interface allows the HRME system to be easily integrated into the clinical workflow. The real-time analysis algorithm yielded a sensitivity of 95% and a specificity of 91% in an independent validation set; this compares favorably with a prior post-hoc quantitative analysis of HRME images using the same training, test, and validation sets which yielded a sensitivity of 84% and a specificity of 95% in the validation set¹⁶.

In this preliminary evaluation, the high accuracy of the image analysis algorithm described suggests that it can (1) accurately distinguish between normal and neoplastic tissue, (2) provide objective feedback for the endoscopist during the imaging procedure, (3) potentially aid in reducing the number of biopsies taken during a procedure, and (4) reduce the learning curve needed for a clinician to read and interpret HRME images. Although the next-generation HRME system was originally tested for use in conjunction with endoscopy, it is not limited to this application. The system and image processing both can be translated for use in other tissue types. A training set of images will need to be compiled first to optimize the algorithm for a specific tissue type. Prospective in vivo evaluation of the real time algorithm is still needed. Currently, the software and system are being evaluated in a prospective esophageal imaging study in China to compare the accuracy of visual assessment to that of automated image analysis. Initial results discussed in this article show that the tablet HRME with integrated image analysis could improve ESCN screening while still also being accessible for low-resource settings.

Acronyms

HRME

High-resolution microendoscopy

ESCN

Esophageal squamous cell neoplasia

EC

Esophageal cancer

ESCC

Esophageal squamous cell carcinoma

LCE

Lugol’s chromoendoscopy

CLE

Confocal laser endomicroscopy

NBI

Narrow-band imaging

ROI

Region of interest

FOV

Field of view

ROC

Receiver operating characteristic

AUC

Area under the curve

USAF

United States Air Force