Artificial Intelligence and the Future of Gastroenterology and Hepatology

Abstract

The integration of artificial intelligence (AI) into gastroenterology and hepatology (GI) will inevitably transform the practice of GI in the coming decade. While the application of AI in health care is not new, advancements are occurring rapidly, and the future landscape of AI is beginning to come into focus. From endoscopic assistance via computer vision technology to the predictive capabilities of the vast information contained in the electronic health records, AI promises to optimize and expedite clinical and procedural practice and research in GI. The extensive body of literature already available on AI applications in gastroenterology may seem daunting at first; however, this review aims to provide a breakdown of the key studies conducted thus far and demonstrate the many potential ways this technology may impact the field. This review will also take a look into the future and imagine how GI can be transformed over the coming years, as well as potential limitations and pitfalls that must be overcome to realize this future.

Introduction

The term artificial intelligence (AI) was first introduced in the 1950s and refers to the application of computers to perform complex tasks, such as solving problems and making nuanced decisions, that have traditionally been associated with human intelligence.¹ Interest in AI-driven health care and medicine has been exploding in recent decades, owing largely to the vast increase in data available since the establishment of the electronic health record (EHR). In fact, from 2008 to 2018, over 23,000 papers concerning the application of AI in health care have been published.² The current emphasis of AI in medicine is a direct response to the challenges facing health-care systems including increasingly complex patients, higher billing and coding requirements, and a growing population that leaves less time for interactions between a health-care provider and their patients than ever before. The hope is for a future where AI-driven health care can lead to improvements through all aspects of patient care by leveraging the vast quantities of data currently stored in the EHR.^3,4

Machine learning (ML) lies at the crux of the most modern AI and encompasses algorithms on a wide spectrum from “supervised” to “unsupervised” learning. These frameworks, as described in the previous issue by Rattan et al., allow researchers to accomplish tasks with accuracy, speed, and precision not previously possible.^5,6 The ability to organize and analyze large amounts of clinical information such as documentation, laboratory, and imaging data makes ML particularly powerful. As the majority of the information included in the EHR is stored as unstructured data, it has previously only been accessible to research via laborious review of individual charts. By accessing these data, ML algorithms can accomplish a wide variety of tasks such as disease identification and outcome prediction with accuracy that outperforms classical statistical methodology.⁵

The application of AI in gastroenterology has shown significant promise, primarily due to the strong body of ML research involving image recognition. These algorithms have already been applied to endoscopy with encouraging results and have subsequently led to improved disease detection and classification with greater accuracy than even the most experienced endoscopists.⁷ However, many challenges such as identifying potential biases in algorithms, expanding generalizability, and increasing interpretability still exist.⁸ Additionally, for these new advances to truly transform health care, widespread adoption is necessary. A substantial degree of skepticism exists in the medical community regarding the utility of AI in clinical practice, which will need to be addressed in the coming years.⁹

In this article, we highlight important studies utilizing AI in gastroenterology and hepatology to demonstrate the possibilities through which AI can transform clinical practice. We discuss how AI has been applied in various domains of gastroenterology and hepatology (GI) including upper endoscopy, colonoscopy, capsule endoscopy, pathology, radiology, and EHR data while also looking at potential breakthroughs on the horizon that promise to significantly improve patient care while alleviating the administrative burden on health-care providers.

Computer Vision in Endoscopy

Over the past 10 years, research into the possible applications of novel ML algorithms in upper endoscopy, colonoscopy, and wireless capsule endoscopy (WCE) has been increasing exponentially. Already, major advancements have been made in identifying early stages of diseases, determining potential treatment options, and reducing the overall burden of endoscopists. The extensive work that has been done in understating the ways in which ML can transform endoscopic practice has been outlined in a number of high-quality reviews that focus on specific endoscopic modalities.^7,10,11 We will highlight notable studies in ML-driven endoscopy and the potential of this technology to revolutionize clinical practice over the coming years (Figure 1).

Figure 1.

Potential applications of artificial intelligence in GI endoscopy. SCC, squamous cell cancer; WCE, wireless capsule endoscopy.

Upper Endoscopy

In upper endoscopy, much of the ML research thus far has focused on Barrett’s esophagus (BE), esophageal squamous cell cancer (SCC), Helicobacter pylori (H. pylori) infection, and gastric cancer.

BE is the primary risk factor in the development of esophageal adenocarcinoma, which is associated especially with poor survival.¹² Screening for BE and early esophageal adenocarcinoma is challenging, as evidenced by the fact that only 1 in 10 cases of esophageal adenocarcinoma are diagnosed within a screening program.¹³ Thus, an AI-enabled aid that can assist in the detection of neoplastic changes in BE has been a focus of AI research and has the potential to improve clinical outcomes.¹⁴ In 2016, a Support Vector Machine (SVM) model was created based on endoscopy images to identify early neoplastic changes in patients with BE and was able to do so with a sensitivity and specificity of 0.86 and 0.87, respectively.¹⁵ An algorithm combining multiple ML models has also been developed to detect neoplasia from volumetric laser endomicroscopy images and was able to detect neoplasia with a sensitivity and specificity of 90% and 93%, respectively.¹⁶ In 2020, de Groof et al¹⁷ developed a deep learning model based on 1704 endoscopic images of nondysplastic BE and early stage neoplastic BE. This model was able to classify images as either neoplasia or nondysplastic BE with 90% sensitivity and 88% specificity. Additionally, when compared to general endoscopists, the system identified dysplasia with greater accuracy, 88% vs 73%.¹⁷ This model also proved useful in identifying the optimal site for biopsy in over 92% of cases.¹⁸ Another model was created using a convolutional neural network (CNN) trained on 916 endoscopic images of patients with histology-proven neoplastic BE and 919 control images from patients with nondysplastic BE. This model was able to detect early neoplasia with a sensitivity and specificity of 96.4% and 94.2%, respectively.¹⁹ The first true real-time application of a deep learning model to identify early neoplastic BE was published by Ebigbo et al²⁰ in 2020 and deployed their previously developed CNN during live endoscopy to accurately identify cases of neoplastic BE with an accuracy of 89.9%, similar to that of expert endoscopists.

Esophageal SCC is another disease in gastroenterology where the possibilities of ML are beginning to be explored. In 2019, a deep neural network (DNN) was trained on 2428 endoscopy images and was able to identify esophageal SCC with a sensitivity and specificity of 97.8% and 85.4%, respectively. Accuracy in identification was 91.4%, compared with 88.8% in a group of senior endoscopists and 77.2% in junior endoscopists. However, when the endoscopists were given access to the DNN tool to assist with decision-making, the diagnostic ability of all the endoscopists improved considerably.²¹ A model was also created to specifically identify dysplasia and early esophageal SCC. This CNN-driven model was based on 6473 narrow-band images and was able to correctly identify dysplasia and early esophageal SCC with 98% sensitivity and 95% specificity.²² These studies demonstrated the feasibility of deploying a ML-driven model during live endoscopy to assist with the diagnosis of esophageal SCC.

Upper endoscopy can be beneficial in the diagnosis of H. pylori gastritis; however, accuracy of diagnosis during endoscopy is quite difficult with a sensitivity of 62% and specificity of 89%.²³ An early ML model from 2004 involved a CNN trained on endoscopy images from 30 patients and was able to predict the presence of gastric atrophy, intestinal metaplasia, and H. pylori-related gastric inflammation with a sensitivity of 85% and a specificity of 90%.²⁴ Further progress was made in 2017 by Shichijo et al who developed a CNN based on over 30,000 endoscopic images that could be employed during live endoscopy to predict the presence of H. pylori infection. This model that was able to predict the presence of H. pylori infection with a sensitivity of 89% and a specificity of 87% was significantly quicker and more accurate than experienced endoscopists.²⁵

Gastric cancer also presents an opportunity for improvements in care with ML, as diagnosing early gastric cancer is challenging, with reports of miss rates as high as 11% during upper endoscopy.²⁶ In 2012, Kubota et al²⁷ developed a model able to differentiate T1 through T4 gastric cancer staging with a modest accuracy of 64%. An improved model utilized a CNN trained on 790 endoscopic images that was able to determine depth of tumor invasion with an area under the curve (AUC) of 0.94. This model demonstrated a higher accuracy and specificity in determining tumor invasion than human endoscopists.²⁸ Another model was developed to identify early gastric cancer, in which a CNN was trained on over 24,000 images and was able to differentiate early gastric cancer with an accuracy of 92.5%, sensitivity of 94%, and specificity of 91%.²⁹

Colonoscopy

The majority of ML research in colonoscopy has focused on colorectal cancer, from improving the detection of colorectal cancer during colonoscopy to ensuring adequate bowel preparation and ultimately in improving adenoma-detection rates. While most attention in colonoscopy-related ML has been focused on improving adenoma-detection rates, ensuring adequate bowel preparation is also essential as poor prep can result in greater than 30% adenoma miss rate.³⁰ A reliable and reproducible method to evaluate the effectiveness of bowel preparation has previously been developed in the Boston Bowel Preparation Scale; however, interobserver reliability in application of the scale has been shown to be only 0.74.³¹ To address this important problem, a CNN was trained by Zhou et al³² in 2020 that applied Boston Bowel Preparation Scale to live endoscopy with over 93% accuracy, significantly better than human endoscopists. This is a system that, if applied in the clinical setting, could potentially lead to decreased rate of missed adenomas.³²

Overall, adenoma miss rate during colonoscopy has been shown to be as high as 25%, with a colorectal cancer miss rate of approximately 5%.^33,34 While there have been over 50 published studies detailing ML-driven algorithms built to detect malignant and premalignant colorectal lesions, essential studies in ML polyp detection demonstrate the greatest potential for advancements in this field, especially those that implement real-time ML-driven polyp-detection systems.⁷ These models have shown tremendous promise, for example, a CNN trained on over 8000 colonoscopy images was able identify polyps with an accuracy of 96.4%. This model was able to improve human endoscopists in detecting polyps, with expert reviewers able to identify over twice as many missed polyps with model assistance.³⁵

Wang et al³⁶ were the first to deploy an ML-driven polyp-detection model to assist with live endoscopy. Patients were randomized to either standard colonoscopy or colonoscopy with a built-in CNN-based polyp-detection system that would alert the endoscopist when a polyp was detected. This system resulted in a 9% increase in adenoma-detection rate and a greater number of adenomas detected per patient.³⁶ Recently, multiple additional live ML assistant models have been developed which significantly increase adenoma-detection rate, with one model increasing detection rate by over 14% and the number of adenomas detected per procedure increasing from 0.71 to 1.07.37, 38, 39

Another area of ongoing ML research in colonoscopy is in the determination of ulcerative colitis severity. Severity of inflammation is a key factor in the risk of colorectal cancer development, so accurate detection of more severe disease is crucial.⁴⁰ Unfortunately, the determination of disease severity remains somewhat subjective, and consistency varies widely among endoscopists.⁴¹ To address this problem, a CNN model was trained on 16,514 endoscopic images to distinguish moderate-to-severe ulcerative colitis from remission based on endoscopic images. This model was able to distinguish endoscopic remission from moderate-to-severe disease with an area under the receiver operating characteristic curve (AuROC) of 0.966, which is similar to human reviewers.⁴² This model demonstrates the potential for computer assistance in detecting subtle changes in inflammation, potentially allowing for future standardization of endoscopic description of inflammation. Another CNN-based model trained on endoscopic images from 29 consecutive patients with ulcerative colitis (UC) was able to identify patients with UC flares requiring treatment escalation.⁴³ Alternatively, a DNN was trained on over 40,000 endoscopic images and was able to identify patients with endoscopic remission with accuracy of 90.1%.⁴⁴

Wireless Capsule Endoscopy

The use of WCE in gastroenterology has been increasing since its clinical introduction in 2000 and has transformed visualization of the small bowel.⁴⁵ WCE is the recommended first-line investigation in patients with obscure GI bleeding but is also used to identify small bowel tumors, inflammatory bowel disease (IBD), celiac disease, and for surveillance of heritable polyposis syndromes.⁴⁶ During a single WCE, an average of 12,000 images are generated, which take at least 30 minutes on average for an experienced gastroenterologist to read and interpret. Thus, there has been significant research into the automation of this process through AI.

Despite the generation of a large number of images, WCE is currently only able to locate the source of bleeding in approximately 60% of cases.⁴⁷ An SVM model was developed in 2014 and trained on WCE images from 50 subjects with obscure GI bleeding and 200 subjects with normal mucosa. This model had a sensitivity of 93% and a specificity of 95% for identifying the source of bleeding.⁴⁸ Another study carried out a few years later improved on this model and was able to detect small bowel bleeding based on WCE images with sensitivity and specificity >99%.⁴⁹ Additional models were also created using CNNs that were able to identify mucosal erosions, ulcers, and small bowel angioectasia with high accuracy.^50,51 The success of these models led to a study evaluating the effectiveness of a CNN trained to identify small-bowel mucosal breaks in clinical practice. This study reported that the time to read WCE reports was reduced by as much as 75% with no significant change in detection rate of mucosal breaks.⁵²

In celiac disease, WCE has proven to be a useful alternative to upper endoscopy, with WCE demonstrating good sensitivity and excellent specificity for the detection of villous atrophy.⁵³ Since the adoption of WCE as an acceptable means of diagnosing celiac disease, a number of studies have evaluated the utility of incorporating ML to streamline and improve the detection of disease. The first notable study was published by Ciaccio et al⁵⁴ in 2010 and was able to detect the presence of celiac disease with 80% sensitivity and 96% specificity using still images from WCE. An SVM model trained on WCE images from 13 control and 13 celiac patients was able to identify celiac disease with a sensitivity of 88% and a specificity of 87%.⁵⁵ Another SVM model improved upon these results and was able to detect disease with sensitivity and specificity of 97% and 96%, respectively.⁵⁶

WCE also plays an important role in IBD, in particular, Crohn’s disease as it allows for assessment of the entire small bowel. WCE is currently used to identify disease, assist in determining disease severity, and help identify response to therapy. In 2020, a CNN was trained on over 17,000 WCE images from 49 patients and was found to be able to differentiate normal vs diseased mucosa in subjects with known Crohn’s disease with an AUC of 0.99.⁵⁷ Another study attempted to decrease the time to read WCE reports to assess disease severity in IBD and trained an ML model to identify 100 images from the WCE video that are most likely to contain abnormalities. They had 2 sets of endoscopists review WCE reports and assess inflammatory activity; however, one group only had access to the ML-identified 100 images. There was strong agreement in the degree of inflammation between the 2 reading groups; however, those only reviewing the 100-image set identified by the model were able to come to a conclusion up to 30 minutes quicker.⁵⁸ Another CNN-based algorithm was developed which demonstrated an accuracy of 91% in classifying ulcers in Crohn’s disease.⁵⁹

The Future

In the coming years, ML-driven innovation in endoscopy is likely to impact practice in numerous ways. Endoscopists' diagnostic capabilities during procedures are likely to be enhanced through the use of this technology, with the computer able to assist in challenging diagnosis, such as differentiating cancer vs adenoma in the duodenum.⁶⁰ In the near future, computer vision will serve as an augmentation tool for endoscopists, improving accuracy and decreasing procedural time. These new systems will also likely serve as training tools for learners.¹⁰ This future is rapidly approaching, with the FDA recently approving Breakthrough Device Designation for a real-time endoscopic AI system⁶¹ (Table).

Table.

Future Directions for ML in Computer Vision Endoscopy

Problem	ML solution
Classification of esophageal varices and risk stratification	Agreement among endoscopists varies widely regarding size of esophageal varices. Automated stratification and risk classification could significantly impact practice and provide endoscopists with a tool to more accurately define varices.62
Differentiation of ulcerative colitis vs Crohn’s disease	Distinction between ulcerative colitis and Crohn’s disease can be challenging endoscopically as well as histologically.63,64 Numerous scoring tools have been created to assist endoscopists in diagnosis; however, this problem lends itself well to ML assistance in the future.
Assessment of biliary strictures during ERCP	The differentiation between benign and malignant biliary strictures during ERCP is challenging even for the most advanced endoscopists, and it is possible that ML could help identify features associated with benign and malignant disease that could assist the endoscopist.65
Predicting bile duct cannulation difficulty	One of the most challenging aspects of ERCP is determining the likelihood of success in achieving bile duct cannulation, and thus presents another opportunity for ML assistance to transform practice.65
Quality assessment of mucosal inspection	ML could not only assist with training of endoscopists but also provide real-time feedback regarding the percent of mucosa examined during EGD or colonoscopy to ensure adequate examination is taking place.66
Standardized training in endoscopy	ML-developed systems can likely help standardize the training and assessment of fellows in the development of endoscopy skills.10

EGD, esophagogastroduodenoscopy; ERCP, endoscopic retrogradecholangiopancreatography.

Significant changes will likely be seen in clinical practice over the next decade as a result of these innovations, and endoscopy is likely to be impacted more in the near term than any other area of gastroenterology. For example, endoscopists can expect computer vision systems to be implemented in the near future to assist in the diagnosis of early esophageal adenocarcinoma and polyp detection and classification. The reading of WCE images is also likely to be heavily impacted over the coming years, with ML algorithms able to significantly reduce reading time and improve accuracy. This will likely greatly expand the use of capsule endoscopy.

However, there is a vital need for clinical trials that can ascertain long-term benefits, risks, and the cost-effectiveness of increased early disease detection.⁶⁷ We must determine whether these models are generalizable across different populations and clinical settings. After high-quality clinical trials take place, it is not hard to imagine a future in which ML-driven endoscopy improves diagnostic capability of all endoscopists, reduces the burden on overworked endoscopists, and also serves a vital role in the learning process for young endoscopists.

AI and ML in GI Radiology/“Radiomics”

At the beginning of the ML revolution in medicine, the majority of the early research focused on the automation of radiology interpretation, including ultrasound, computed tomography (CT), and magnetic resonance imaging (MRI). The potential applications of ML in radiology are broad, from decreasing radiologist burden to improving diagnostic accuracy. In gastroenterology, ML-assisted radiology has shown significant promise in hepatology and pancreatology (Figure 2).

Figure 2.

Potential applications of artificial intelligence in GI radiology.

Hepatology

ML has been applied extensively to radiology in the field of hepatology to identify and assess various conditions. Ahn et al⁶⁸ published a recent review on the use of ML in hepatology, specifically highlighting recent advancements in ML-assisted radiology.

Ultrasound is often the first test those with suspected liver disease undergo for diagnostic purposes; thus, much attention has been focused on automating the interpretation of hepatic ultrasound and improving diagnostic accuracy via ML. In 2016, Gatos et al⁶⁹ developed an SVM model that was able to distinguish subjects with liver disease from controls using ultrasound shear wave elastography with a sensitivity of 83.3%, specificity of 89.1%, and an accuracy of 87%. Another model was able to predict fibrosis stage in subjects with hepatitis B using real-time elastography with an accuracy of 83%.⁷⁰ A recent multicenter study was published by Wang et al⁷¹ in 2019 in which an ML model to predict liver fibrosis using shear wave elastography 2D images was developed. This model was able to achieve an AUC of 0.97 for cirrhosis, 0.98 for advanced fibrosis, and 0.85 for identifying significant fibrosis.⁷¹ The identification of liver lesions via ML-guided ultrasound has also been demonstrated, with a DL model developed to detect and characterize benign and focal liver lesions based on ultrasound images. This model had an AUC of 0.935 for identifying liver lesions and 0.916 for classifying these lesions as either benign or malignant.⁷²

CT and MRI are also heavily used modalities in diagnosing and monitoring various liver diseases. Nayak et al⁷³ developed an radial basis function kernel algorithm based on CT images to identify cirrhosis and hepatocellular carcinoma (HCC). This model was able to accurately identify cirrhosis and HCC with an accuracy of 80%.⁷³ A model was also built to identify hepatic lesions, including HCC, identified on multiphasic MRI and achieved an accuracy of 92%, a sensitivity of 92%, and specificity of 98%. Notably, computation time was 5.6 ms per lesion, highlighting the significant time that can be potentially saved by these models. This group further identified exact features that its model was utilizing to make decisions, improving interpretability of these predictions and potentially allowing for implementation into the radiologist workflow.^74,75 Multiple recent models have demonstrated that adipose tissue, muscle mass, and bone density can be determined by applying deep learning methods to CT images, ultimately allowing for clinicians to identify trends in patients' nutritional status that could signify future clinical decline. These models have been trialed in patients with liver disease and have been shown to accurately predict future mortality.^76,77

Pancreas

Diagnosis of diseases of the pancreas also relies heavily on imaging. Limited research in ML-based radiology in diseases of the pancreas has taken place; however, these few studies have shown promise for the future of this field. In 2019, Kuwahara et al⁷⁸ developed a DNN based on endoscopic ultrasound images to predict intraductal papillary mucinous neoplasms, a precursor lesion of pancreatic adenocarcinoma. This model had an accuracy, a sensitivity, and specificity of 92.6%, 94.0%, and 95.7%, respectively, for identifying intraductal papillary mucinous neoplasms, which was much better than the accuracy of 56% for radiologists.⁷⁸ Liu et al⁷⁹ trained a CNN based on CT images to identify pancreatic cancer. They utilized 370 CT scans from subjects with pancreatic cancer and 320 controls and found that the developed algorithm had a sensitivity of 98%, which was significantly higher than the 93% among experienced radiologists.⁷⁹ In 2021, Janssens et al⁸⁰ trained a CNN on 469 CT scans to identify intrapancreatic fat and pancreatic ductal adenocarcinoma and were able to identify markers that could potentially be used to identify pancreatic adenocarcinoma in the prediagnostic phase.

The Future

In the near future, the application of ML algorithms in radiology will likely improve radiology workflow, improve diagnostic accuracy, and decrease time to diagnosis. For example, ML models that can not only analyze the current radiology image but also interpret this image in the context of all the previous imaging a certain patient has had and the interval changes that have occurred can be developed. Additionally, one could also envision a future in which ML-driven mobile-based ultrasound is available for patient use at home for tasks such as volume status evaluation, determination of degree of ascites, and bladder scans. ML-driven radiology is a crucial component of a future in which individualized medicine is at the forefront of gastroenterology.

AI and ML in GI Pathology

While there has been significant focus on the application of ML in endoscopy, ML-driven histopathology has the potential to significantly impact clinical practice. Histopathology has a unique advantage over imaging modalities in that models can quantify precise cellular features such as shape, size, and color from hematoxylin- or immunohistochemistry-stained slides that can be used to create ML models.⁸¹ Recent technological advances have made it possible to digitally archive traditional glass slides at full resolution, leading to the possibility of standardizing histopathological analysis through the use of ML.^82,83 These models have the potential to reduce the clinical burden on pathologists and improve diagnostic accuracy. Kobayashi et al⁸¹ recently published a comprehensive review on the state of ML in GI pathology. We will review high-impact studies in this field that demonstrate the ways in which ML can transform clinical practice (Figure 3).

Figure 3.

Potential applications of artificial intelligence in GI pathology. NAFLD, nonalcoholic fatty liver disease.

Luminal GI

In 2019, Wei et al⁸⁴ developed a CNN with the goal of creating a deep learning system that can assist pathologists with the diagnosis of celiac disease. The CNN was trained on hematoxylin and eosin-stained duodenal tissue and was able to identify and differentiate between normal tissue and celiac disease with an accuracy of 91% and 95%, respectively.⁸⁴ A similar approach was taken by Martin et al⁸⁵ who trained a CNN on 300 hematoxylin-stained slides (100 normal, 100 H. pylori, 100 reactive gastropathy) and found the model could differentiate between the 3 with sensitivity and specificity greater than 70%. Neural networks have also shown promise in the classification of colorectal polyps. Wei et al⁸⁶ build a DNN that could classify colorectal polyps as either tubular adenoma, villous adenoma, hyperplastic polyp, and sessile serrated adenoma with an accuracy of 87%. The authors found this rate of correct classification to be equivalent to that of pathologist review and suggested that this model could be deployed to assist pathologists in improving efficiency and diagnostic accuracy in classifying colorectal polyps.⁸⁶ A recent publication by Gehrung et al⁸⁷ found that pathologic diagnosis of BE via a nonendoscopic Cytosponge-TFF3 test could be feasible with an ML-driven process and that review of these slides by pathologists could be decreased by 57% with a similar diagnostic accuracy.

GI Malignancy

In colorectal cancer, ML-driven histopathology has shown the potential to impact prognosis and determine appropriate treatment. For example, in 2019, a DNN was found to be able to predict survival in colorectal cancer based on only cancer histopathological images with an hazard ratio of 1.63 (95% CI 1.14–2.33).⁸⁸ Another study found that a DL model could successfully identify microsatellite instability and mismatch-repair deficiency in hematoxylin and eosin slides obtained from colorectal tumors with an AuROC of 0.92.⁸⁹ In the evaluation and screening of esophageal cancer, Tomita et al⁹⁰ created a CNN-based model that was able to differentiate esophageal tissue based on histopathological slides as either normal, nondysplastic Barrett’s, dysplastic Barrett’s, and adenocarcinoma with an accuracy greater than 85%. Similar techniques have also been used for the detection of gastric cancer, with a CNN trained on whole-slide images from gastric biopsies able to identify gastric adenocarcinoma with an AUC up to 0.97.⁹¹ A DL model has also been developed to identify and distinguish between the 2 primary types of liver cancer, hepatocellular carcinoma and cholangiocarcinoma. This model developed by Kiani et al⁹² in 2020 was trained on hematoxylin and eosin-stained whole-slide images and was able to identify and distinguish between these 2 types of hepatic malignancies with an accuracy of 86%. Predicting prognosis in HCC has also been shown to be possible via pathology-based ML, with a DL model shown to have a c-statistic of greater than 0.75 in predicting prognosis after a histopathological diagnosis of HCC, performing better than previous prediction models based on alpha-fetoprotein level, vascular invasion, or disease stage.⁹³

Hepatology

The current standard for diagnosis of nonalcoholic fatty liver disease (NAFLD) and non-alcoholic steatohepatitis (NASH) is manual histologic assessment by an expert pathologist, a time-consuming process that results in significant interpreter variability. Vanderbeck et al⁹⁴ created a model to assess histological features of NAFLD, with a goal of creating a scoring system that could provide continuous measurement of disease features. They created this model using an SVM that was able to identify the presence of NAFLD with 89% accuracy and was able to identify specific histopathological features such as macrosteatosis, bile ducts, portal veins, and sinusoids with high accuracy.⁹⁴ Models have also been developed to evaluate degree of fibrosis in NAFLD, such as an SVM-based model that had an ROC of greater than 90% for detecting normal fibrosis, bridging fibrosis, and presence of nodules or cirrhosis.⁹⁵ Another CNN model was built to characterize disease severity and treatment response in NASH and found a Cohen's kappa coefficient of 0.8 for staging NASH.⁹⁶

The Future

The future of ML in transforming the practice of GI pathology is incredibly promising, and we expect a number of changes to clinical practice in the coming years. For example, these models will lead to accurate and standardized grading of disease states such as esophageal dysplasia and NAFLD/NASH that significantly decrease intraobserver variability. We also expect more precise interpretations of histopathology, where previously specific components would be estimated in broad categories. With ML, we will be able to quantify these components with greater granularity, for example, in defining degree of fibrosis, percent dysplastic tissue in GI malignancy, degree of inflammation in eosinophilic esophagitis, and many more. Similar to endoscopy, these models will serve as a second set of eyes for pathologists in reducing cognitive burden and improving standardization, while also serving as an invaluable teaching tool.

AI and ML in GI—EHR Data

Clinicians consider a vast variety of information when making a diagnosis, including clinical history, laboratory values, and vital signs, all of which are reviewed in the EHR. When many clinicians think of AI in medicine, they imagine a future in which the vast amounts of information included in the EHR can be utilized to identify particular diseases and predict outcomes. A significant amount of research has already taken place in making this future a reality, and we will highlight the advances made in HER-based ML algorithms in the identification of disease and the prediction of outcomes (Figure 4).

Figure 4.

Potential applications of artificial intelligence via utilization of the electronic health record.

Disease Identification

ML has shown significant promise in identifying disease utilizing data present in the EHR. For example, a clinical decision-support system was developed in 2011 to assist in the diagnosis of celiac disease using only patient-reported symptoms and physical exam findings extracted from the EHR, with a Bayesian classifier model having an AUC of 0.84 in the identification of celiac disease.⁹⁷ A random forest (RF) model was created in 2016 that was able to diagnose colorectal cancer prior to clinical diagnosis with an AUC of 0.891 using only raw data included in the EHR.⁹⁸ Another disease-identification model was developed by Wu et al in 2019 that attempted to identify NAFLD utilizing only clinical data. The authors created an RF model and included variables such as age, gender, vital signs, and labs and were able to identify NAFLD with an AUC of 0.925.⁹⁹ Another RF-based model was created to identify patients with esophageal varices and those with esophageal varices requiring treatment. Screening for esophageal varices requires endoscopy, and the authors hypothesized that a noninvasive strategy could be appropriate in screening for varices. The model was developed using clinical data available in the EHR including age, gender, vital signs, labs, and reported complications of cirrhosis and was found to have an AUC of 0.82 for identifying esophageal varices and 0.74 for identifying varices requiring treatment.¹⁰⁰

Outcome Prediction

The potential of ML-driven outcome prediction in gastroenterology was first demonstrated in 2003 when Das et al¹⁰¹ published in the Lancet a model for predicting outcomes following lower GI bleeding. They built an artificial neural network (ANN) using clinical variables available in the EHR at initial presentation of lower GI bleed and were able to predict mortality, recurrent bleeding, and need for endoscopic intervention with accuracy over 90%.¹⁰¹ Another ML model utilized 6 parameters extracted from the EHR (age, baseline hemoglobin, presence of gastric ulcer, gastrointestinal diseases, malignancies, and infections) and was able to predict recurrent peptic ulcer bleeding in 1 year with an AuROC of 0.775% and with 84% accuracy.¹⁰² Sato et al¹⁰³ developed an ANN-based model to predict survival in patients diagnosed with esophageal cancer who underwent endoscopic resection with curative intent and were able to predict 5-year survival with an AUC of 0.884. Another ANN model was able to predict progression to severe pancreatitis, organ failure, and mortality in those hospitalized with pancreatitis with an AUC greater than 0.8.¹⁰⁴ A novel risk prediction model was created by Eaton et al¹⁰⁵ in 2020 to predict outcomes in those with primary sclerosing cholangitis. This model utilized 9 clinical variables (bilirubin, albumin, alkaline phosphatase, platelets, aspartate aminotransferase, hemoglobin, sodium, patient age, and number of years since primary sclerosing cholangitis was diagnosed) to build a gradient boosted model that was able to predict hepatic decompensation with c-statistic of 0.85.¹⁰⁵

Waljee et al¹⁰⁶ have published multiple ML studies predicting outcomes in IBD. In 2017, this group developed an RF-based model that was able to predict future IBD-related hospitalization with an AuROC of 0.87. The main independent risk factors identified in this model included age, albumin concentration, immunosuppressive medication use, and platelet count.¹⁰⁶ They next used an RF algorithm to predict response to vedolizumab among those with UC. AuROC for this algorithm was 0.73 when including data through the first 6 weeks of therapy, potentially giving clinicians further insight into the individualized utility of this expensive medication.¹⁰⁷ Finally, this group again employed an RF model to predict response to ustekinumab in patients with Crohn’s disease and found their model had an AuROC of 0.76.¹⁰⁸

Significant work has also been carried out in predicting outcomes in a variety of liver diseases. For example, one study following up patients with early-stage cirrhosis until liver transplant or death developed an RF model to predict development of HCC. This model was created utilizing EHR data and was able to predict HCC with 80.7% sensitivity and 46.8% specificity.¹⁰⁹ A Classification and Regression Tree model was developed to improve prognostication in acetaminophen overdose that incorporated demographics, labs, medications administered, and other clinical variables extracted from the EHR and was able to predict mortality with an AUC of 0.79.¹¹⁰ Predicting survival after liver transplant was also shown to be possible, with a recurrent neural network developed that was able to predict 5-year survival after liver transplantation with an AUC of 0.864.¹¹¹

In 2017, Konerman et al¹¹² developed an RF model that utilized clinical and laboratory variables extracted from the EHR to predict transplant-free survival in hepatitis C-related cirrhosis with an AuROC of 0.85. Multiple studies have developed models to predict hospital readmission among those with cirrhosis, a source of significant morbidity and mortality. However, the most promising model was built using a traditional LR model that predicted readmission with an AUC of 0.67.¹¹³ Another model utilized multiple ML models to predict readmission and mortality in those hospitalized with cirrhosis; the best model was an SVM that achieved an AUC of 0.62.¹¹⁴ The limited predicted capabilities of these models are likely due to the difficulties with considering multiple data points in relation to each other over time when building a model. However, if just attempting to predict mortality in those with cirrhosis, multiple models have been developed using deep learning techniques and have been able to predict 1-year mortality with greater accuracy than the model for end-stage liver disease-sodium score.^115,116

The Future

The overall goal when initially creating the EHR was to improve health-care quality. As described, ML is showing significant promise in utilizing these vast data to individualize medicine. However, it is still very challenging to determine if guideline-driven quality health care is being provided, especially considering situations when there are reasonable explanations for deviations in standard of care. Utilizing only structured data in the EHR, it is likely impossible to achieve these goals. However, it is evident that these data are present as it can be determined on individual chart review. But how do we evaluate this data on large scales? The answer may be in the use of natural language processing (NLP) to create models based on unstructured data.¹¹⁷

AI and ML in NLP

The current frontier in ML research is NLP. The majority of the previously discussed ML advances in gastroenterology involve structured data, or data that can easily be stored and organized into tables. However, significant amounts of clinical data for individual patients exist in an unstructured format, in places such as clinical notes and pathology and radiology reports. NLP aims to automate the extraction of this unstructured clinical data so that it can be used to develop clinical models.

Studies involving NLP in gastroenterology have primarily been published in the last 5 years, beginning with a study in 2016 that utilized colonoscopy reports to determine quality of colonoscopy and adherence to proper surveillance intervals. This NLP-based algorithm was able to report adenoma-detection rate with 100% accuracy and determine proper colonoscopy surveillance intervals across 13 medical centers.^118,119 Another NLP-based study attempted to improve upon the identification and risk stratification of patients with cirrhosis. Previous International Classification of Diseases (ICD) code-based studies to identify cirrhosis performed well, with a positive predictive value of 90% and an negative predictive value of 87%.¹²⁰ However, an NLP model based on radiology reports was able to identify cirrhosis with a positive predictive value of 92% and an negative predictive value of 97%.¹²¹ Similar models have been successfully built using NLP in Crohn’s disease, where an NLP-based model was shown to improve retrospective disease identification from 83% to 95%.^122,123 NLP has also been used to identify cases of hepatorenal syndrome, again performing better than ICD-based identification models with an AUC of 0.93.¹²⁴ The ability of NLP in retrospective identification of disease will greatly increase the ease and feasibility of conducting large population-based cohort studies.

Another type of unstructured data that is becoming increasingly prevalent in medicine is patient messages through the EHR. The possibilities for this type of data have already started to be explored with NLP. For example, an NLP-based algorithm has shown promise in detecting early hepatic encephalopathy based on sentence and word length in patient messages.¹²⁵

NLP also has the potential to greatly reduce physician administrative burden through the automatic generation of ICD codes based on clinical notes, automated prior authorization requests, and the ability to automatically generate sufficient documentation for billing based on unstructured notes.¹²⁶ This has the potential to allow clinicians to return to a system where clinical notes are simple, readable, and oriented toward providing important information to patient care as opposed to a document designed to maximize billing potential. NLP also has the potential to serve as a valuable clinician assistance tool, automatically extracting clinical information from the chart and presenting applicable scoring tools and clinical evidence for treatment. This again highlights the potential for NLP to ease physician burden while increasing evidence driven care.¹²⁶ Preliminary studies have even shown promise in developing automated chat-bots that can collect patient symptoms via messages in the EHR and provide simple medical advice or recognize when higher level of care is needed.¹²⁷

Conclusion—Where Do We Go From Here?

The application of AI and ML in gastroenterology is not new; however, advancements are occurring rapidly, and the future landscape of AI-driven gastroenterology is beginning to come into focus. Before this future is fully realized, a few key developments that are on the horizon need to take place. First, the medical community needs to fully recognize that AI-driven algorithms can and should be treated as a medical device, and thus regulated by the FDA.¹²⁸ Naturally, there remains some hesitancy in integrating ML-based solutions into clinical practice, and increased awareness and understanding of these methods by the gastroenterology community is a crucial next step. Next, the medical community must determine who is responsible for the accuracy of AI-based algorithms and who holds responsible when errors inevitably result as a consequence of these models.⁵ There is also an urgent need for randomized trials and extensive external validation of developed algorithms in diverse populations to mitigate inherent biases. Although many of the above-described trials exhibit outstanding performance, it is expected that the real-world performance of these models will decrease considerably. Finally, the pathway to developing AI solutions needs to be simplified so that physicians without computer engineering skills can develop models and deploy them into clinical practice.¹²⁹ The practice of gastroenterology and hepatology will significantly change over the coming years as advances in AI and ML continue to accelerate (Figure 5). Thus, it is important that the current state of this technology and the future possibilities are understood so that they can be incorporated into future practice.

Figure 5.

Future impact of artificial intelligence on the practice of gastroenterology and hepatology.