‘Artificial intelligence in Barrett’s Esophagus’

Abstract

Despite advances in endoscopic imaging modalities, there are still significant miss rates of dysplasia and cancer in Barrett’s esophagus. Artificial intelligence (AI) is a promising tool that may potentially be a useful adjunct to the endoscopist in detecting subtle dysplasia and cancer. Studies have shown AI systems have a sensitivity of more than 90% and specificity of more than 80% in detecting Barrett’s related dysplasia and cancer. Beyond visual detection and diagnosis, AI may also prove to be useful in quality control, streamlining clinical work, documentation, and lessening the administrative load on physicians. Research in this area is advancing at a rapid rate, and as the field expands, regulations and guidelines will need to be put into place to better regulate the growth and use of AI. This review provides an overview of the present and future role of AI in Barrett’s esophagus.

Introduction

Barrett’s esophagus (BE) is the only known premalignant precursor for esophageal adenocarcinoma (EAC) and is thought to follow a linear progression from nondysplastic BE to low-grade dysplasia to high-grade dysplasia (HGD) and finally to cancer. ¹ Early detection of dysplastic lesions and cancer confined to the mucosa allows for minimally invasive curative endoscopic treatment, which provides a less invasive method of treatment than surgical resection and/or neo adjuvant therapy for advanced lesions. ²

However, neoplasia within BE may be subtle and difficult to recognize, with a recent meta-analysis showing high miss rates of around 25% for HGD and cancer within 1 year of a negative index examination. ³ The reasons for this are likely multifactorial, including the lack of recognition of subtle lesions, lack of detailed inspection of the esophageal mucosa, nonoptimum cleaning techniques, and less experienced endoscopists. ⁴ In addition, the currently recommended Seattle biopsy protocol for tissue sampling has been shown to sample 5% or less of the entire BE mucosa thereby potentially missing focal areas of neoplasia. ⁵ Faced with these challenges, the need for better detection of neoplastic areas is, thus, paramount. One possible way to improve detection of these lesions is through enhanced imaging technologies such as volumetric laser endomicroscopy (VLE), confocal laser endomicroscopy (CLE), virtual chromoendoscopy, and dye-based chromoendoscopy. The American Society of Gastroenterology (ASGE) has released criteria for the preservation and incorporation of valuable endoscopic innovations (PIVI) to evaluate the use of advanced imaging techniques. It requires that an imaging modality have a per patient sensitivity of ⩾90%, specificity of ⩾80%, and NPV of ⩾98% in detecting HGD and EAC in order to replace four quadrant biopsies. ⁶ Of the currently available technologies, dye-based and virtual chromoendoscopy are the only two technologies that meet these criteria. ⁶ However, the use of these image enhancing technologies is operator dependent and subjective, especially by nonexperts.^7,8

Artificial intelligence (AI) has emerged in recent years as a promising tool in improving clinical performance in gastrointestinal (GI) endoscopy. The hope is that computer-aided diagnosis will play an adjunct role to endoscopists in the early detection and characterization of neoplastic lesions in BE patients.

Definitions and terminologies

Machine learning relies on the use of mathematical model to capture structure and patterns in data. The algorithms can automatically “learn” experientially and do not need explicit programming. ⁹ A subtype of machine learning is deep learning, in which a neural network model has the ability to automatically learn features and can be useful in endoscopy with learning images and videos. ¹⁰ A further subtype of deep learning is convolutional neural network (CNN) which receives input (e.g. endoscopic images), learns specific features (e.g. pit pattern), and processes this information through multilayered neural networks to produce an output (e.g. presence or absence of neoplasia). ¹⁰

Developing a machine learning model typically requires three sets of data: a training set, validation set, and testing set. The training set is used to build the model using labels. The validation set is used to provide a nonbiased evaluation of the model and to ensure there is no overfitting to the training set data. The test set evaluates the final predictive model. ⁹

Overview of current literature

Use of AI with white light and virtual chromoendoscopy

Lesion Detection—One of the first studies that used a detection algorithm for BE lesions and compared it to expert annotations was done using a computer algorithm of machine learning trained with 100 images from 44 BE patients (see Table 1). ¹¹ Using color and texture filters, the algorithm was able to diagnose neoplastic lesions with a sensitivity of 86% and specificity of 87% at a per patient level. ¹¹ Ebigbo et al. ¹² were also able to validate a CNN system to detect EAC in real time with the endoscopic examination of 14 patients using 62 images, and showed a sensitivity of 83.7% and specificity of 100%. Hashimoto et al. ¹³ developed a CNN trained by 916 images of BE and validated with 458 images with a reported accuracy of 95.4% for detection of early neoplasia.

Table 1.

Bridging the gaps in Barrett’s esophagus care with artificial intelligence.

Domains in Barrett’s esophagus care	Current gaps	Artificial intelligence (AI) application
Neoplasia detection	High rates of missed neoplasia 3	Sensitivity of more than 90% and specificity of more than 80% in detecting Barrett’s related neoplasia14,15
Neoplasia characterization	Not reliably done by nonexperts	AI systems show sensitivity of 90% and specificity of 88% in characterization of dysplasia 11
Histopathology diagnosis	Low interobserver agreement among histopathologists especially with low-grade dysplasia 16	AI algorithm can classify histology images into nondysplastic, dysplastic, and cancer with an accuracy of 0.85, 0.89, and 0.88, respectively 17
Quality control	No current reliable way of measuring quality of an upper endoscopic examination	AI system prompts identification of blind spots during upper endoscopy, provides a grading score of percentage of mucosa that is adequately visualized, and measures inspection time 18

Going beyond still images, de Groof et al. ¹⁹ performed one of the initial studies of CAD on live endoscopic procedures on 20 patients; 10 with BE dysplasia and 10 without dysplasia. The sensitivity of the system per level analysis was 91% and specificity was 89% for detection of BE neoplasia.

Lesion Characterization—A more recent study used a computer aided diagnostic (CAD) system (Hybrid ResNet-UNet) that classified images as nondysplastic or dysplastic with sensitivity of 90% and specificity of 88% and achieved higher accuracy that nonexpert endoscopists. ¹¹

Determining depth of invasion—In a study by Ebigbo et al., a deep learning system was trained and tested to differentiate between T1a and T1b Barrett’s cancer using 230 white light endoscopic images (108 T1a and 122 T1b) from three tertiary care centers and compared to experts’ classification. The sensitivity, specificity, and accuracy of the AI system 0.77, 0.64, and 0.71, respectively, in differentiating T1a from T1b, with no significant difference to that of experts, indicating that accurate prediction of submucosal invasion remains challenging for both experts and AI. ²⁰ Use of AI in VLE:

Interpretation of VLE images from BE patients can be quite difficult and requires a steep learning curve. An AI software called intelligent real-time image segmentation has been developed to identify VLE features by different color schemes. A pink color scheme indicates a hyper-reflective surface which implies increased cellular crowding, increased maturation, and a greater nuclear to cytoplasmic ratio. A blue color scheme indicates a hypo-reflective surface which implies abnormal BE epithelial gland morphology. An orange color scheme indicates lack of layered architecture which differentiates squamous epithelium from BE. ²¹ Another study created an algorithm to identify early BE neoplasia on ex vivo VLE images showing a sensitivity of 90% and specificity of 93% in detection with better performance than the clinical VLE prediction score. ²² A CAD system reported by Struveynberg et al. analyzed multiple neighboring VLE frames and showed improved neoplasia detection in BE with an area under the curve of 0.91. ²³

Published systematic reviews and meta-analyses

The main drawback of majority of the published studies is the lack of external validity, generalizability, as well as the limited sample sizes to adequately power for diagnostic accuracy as there is a need to annotate large test datasets. A recent systematic review and meta-analysis by Bang et al. ¹⁴ included 19 studies (10 image-based and 9 patient-based) on the use of AI in detecting esophageal cancer. The majority of the included studies used white light examination (except 7 with narrow-band imaging) and a convolutional neural network CAD algorithm (except 5 studies that used support vector machine). For the image-based studies, the sensitivity was 94% (95% CI, 89–96%) and specificity was 88% (95% CI, 76–94%) for detection of neoplasia. For the patient-based studies, the sensitivity was 93% (95% CI, 86–96%) and the specificity was 85% (95% CI, 78–89%) ¹⁴ for detection of neoplasia.

Another systematic review and meta-analysis by Lui et al. included 561 endoscopic images of patients with Barrett’s esophagus (6 studies) showing the pooled sensitivity of detection of neoplastic lesions to be 88% (95% CI, 82–92.1%), specificity to be 90.4% (95% CI, 85.6–94.5%), and area under the curve of 0.96 (95% CI, 0.93–0.99). From the included studies, 3 used CNN and 3 studies non-CNN models with no significant difference in the performance between the two models. ¹⁵

Histopathology interpretation and AI

Histological diagnosis of BE associated neoplasia is challenging and is another area where AI may prove to be useful. ¹⁶ This may be especially true with low-grade dysplasia which has been shown to have a very low interobserver agreement even among expert histopathologists. ¹⁶ A study by Tomita et al. ¹⁷ using a CNN-based algorithm to classify histology images into nondysplastic BE, dysplastic BE, and EAC showed a classification accuracy of 0.85, 0.89, and 0.88, respectively.

Quality control and AI

An AI system named ENDOANGEL has been studied by Chen et al. ¹⁸ to prompt identification of blind spots during upper endoscopy, provide a grading score of percentage of mucosa that is adequately visualized, and measure inspection time to precisely determine the quality of the examination. Such use of AI will be beneficial moving forward to monitor and record the quality of exams as physician reimbursement rates likely will be increasingly based on outcome measures performance with the goal of providing value-based care. ²⁴ In addition, as these AI quality systems become more sophisticated, it is possible that their use will expand to other applications, such as providing real-time feedback and objective metrics that can be used to train young endoscopists.

Regulatory, ethical, medico-legal, and financial considerations

In September 2019, the first multidisciplinary global gastroenterology AI meeting was held in Washington, D.C., to discuss practice, policy, ethics, data security, and patient care issues related to identification and implementation of appropriate use of AI in gastroenterology. ²⁵ The American Society of Gastrointestinal Endoscopy has also recently formed a task force on AI with the primary goal of setting clinical and research priorities for AI applications. ²⁶ The involvement of regulatory agencies such as the Food and Drug administration will also be paramount to the introduction of AI into clinical practice in order to provide regulatory pathways and ensure safety of use. ²⁷ It will be important to clarify medical legal issues surrounding use of AI especially in cases where misdiagnosis may occur.

Ethical considerations will also be important in the development and application of machine learning in advancing health. Care should be taken not to de-skill future physicians who are training with AI technology. The goal should be to enhance physicians’ abilities through human–computer collaboration rather than replacing human cognition. This will be important in cases where AI may make diagnostic errors. Given the black box nature of these systems, these errors will be unpredictable and unexplainable. Ultimately, the responsibility of interpreting the data will fall on the shoulders of the physician, and clinical advice provided by AI software will likely be required to be reviewed and verified by an expert healthcare professional.^28,29

As these systems become commercial, it will also be important to track costs and efficiency. It is likely these systems will have high costs upfront but will ultimately may be cost-efficient if they help maintain high-quality examination performance. ²⁵ For instance, AI systems may help identification of focal neoplastic areas that can be sampled with targeted biopsies and as such render random biopsies unnecessary. This can decrease the number of pathology samples and ultimately cost.

Proposed clinical use of AI in upper endoscopy

Once AI becomes a clinical reality available to physicians, the proposed use during upper endoscopy will be with live video images that will be sent to the AI application and analyzed in real time. ²⁶ The application will be able to detect areas suspicious for neoplasia and measure the size and morphology of lesions (see Images 1–3). It will alert the endoscopist to suspicious areas either with a screen alert or location box. The endoscopist can then decide if the area needs to be sampled based on the characterization provided by the machine or managed endoscopically (example: use of endoscopic mucosal resection). It is possible that the AI system will also generate the endoscopy report at the end, including automated Prague C&M measurements, measurements of hiatal hernia, and so on to be reviewed by the endoscopist for verification ²⁶ (see Flowchart 1).

Image 1.

High definition white-light image of cancer in a Barrett’s esophagus segment.

Image 2.

Heat map identifying cancer in a Barrett’s esophagus segment.

Image 3.

Heat map superimposed on endoscopic image of Barrett’s esophagus.

Flowchart 1.

Proposed use of AI during endoscopy.

Future directions and applications

As AI systems develop, it will be important that they are tested and validated in real-world settings, in diverse patient populations, with physicians of varying expertise, with different endoscope types and in different practice settings. There has been a proposal to develop a large open-source image library as a resource to validate AI systems while limiting data variability and the ASGE AI task force is actively pursuing this vision.

In addition, although the major focus of AI technologies thus far has been in the field of visual detection and diagnosis, the future will likely include additional applications such as streamlining endoscopic and clinical workflows by automatically documenting and interpreting clinical encounters to decrease the administrative workload on physicians. AI may also help with practice management by facilitating scheduling, surveillance procedures, billing, and payment.

Conclusion

Barrett’s esophagus related dysplasia and early adenocarcinoma can present as very subtle lesions that are difficult to detect and characterize endoscopically. AI systems show promising results to detect these lesions; however, they still need further study and validation, especially in the real-world setting. Commercially developed AI will need to demonstrate cost-effective care that will provide meaningful value and impact on patient care and outcomes. The future appears extremely bright as the field continues to expand with accelerating momentum. Once clinically available, AI promises to significantly impact the field of BE detection, diagnosis, and endoscopic treatment.