Potential applications of artificial intelligence in colorectal polyps and cancer: Recent advances and prospects

Abstract

Since the advent of artificial intelligence (AI) technology, it has been constantly studied and has achieved rapid development. The AI assistant system is expected to improve the quality of automatic polyp detection and classification. It could also help prevent endoscopists from missing polyps and make an accurate optical diagnosis. These functions provided by AI could result in a higher adenoma detection rate and decrease the cost of polypectomy for hyperplastic polyps. In addition, AI has good performance in the staging, diagnosis, and segmentation of colorectal cancer. This article provides an overview of recent research focusing on the application of AI in colorectal polyps and cancer and highlights the advances achieved.

Core Tip: In recent years, the application of artificial intelligence in the diagnosis and treatment of colorectal polyps and cancer has increased. These applications include automated polyp detection and classification as well as qualitative and staging diagnosis of colorectal cancer. This article provides an overview of recent research focusing on the application of artificial intelligence in colorectal polyps and cancer and highlights the advances achieved.

INTRODUCTION

Artificial intelligence (AI) is a wide-ranging branch of computer science concerned with building smart machines capable of performing tasks that typically require human intelligence. AI technology has made great progress, mainly owing to the development of analytical methods such as support vector machines and deep learning. Through continuous learning from data and experience accumulation, the task processing ability of the machine is greatly enhanced. AI has been improved through algorithm learning and knowledge management. It has gradually been applied in imaging and pathological diagnosis, disease management, drug research and development[1], and promoting the development of genetics and molecular medicine. The research results of Theofilatos et al[2] confirm this view. In their research, AI was used to find new treatment methods through a protein interaction algorithm, which may become a new direction in the development of molecular medicine. AI systems that use deep learning have been utilized in images of lesions such as esophageal cancer, glaucoma, and skin cancer with good performance[3-5].

In recent years, the application of AI in the diagnosis and treatment of colorectal polyps and cancer has also increased[6-8]. In the field of gastroenterology, there has been considerable interest in utilizing AI as an adjunctive detection technique in endoscopy. AI provides the promise of increasing polyp detection and even optical polyp diagnosis, all requiring minimal training of the endoscopist. For instance, a fast detection algorithm named ResYOLO was pretrained with a large database of nonmedical images and then refined with images extracted from colonoscopic videos. Evaluated on 17574 frames from 18 endoscopic videos, the proposed method could find frames with polyps with an accuracy of 88.6%, recall of 71.6%, and a processing speed of 0.15 s per frame[6]. With the advent of deep learning algorithms and significant advances in computer capabilities, more and more AI assistance, some of which may be used in real time during colonoscopy, is now being implemented.

We searched for relevant literature in the MEDLINE and PubMed databases (2015–2020) using the following keywords: “deep learning,” “computer-assisted diagnosis,” “artificial intelligence,” “colorectal polyps,” and “colorectal cancer.” We only reviewed full journal articles published in English. The inclusion criteria were as follows: (1) Studies that associated AI with the detection and classification of colorectal polyps; and (2) Studies that associated AI with the diagnosis of colorectal cancer (CRC). This review highlights recent advances in the application of AI in colorectal polyps and cancer in the past 5 years (Figure 1).

Figure 1.

Applications of artificial intelligence in colorectal polyps and cancer. AI: Artificial intelligence; CRC: Colorectal cancer; LNM: Lymph node metastasis.

AI IN COLORECTAL POLYPS

At present, colonoscopy is still the most important diagnostic method for colorectal polyps. It is estimated that the prevalence of precancerous polyps in the 50+ years old screening population will be more than 50%[9]. Adenoma is the most common precancerous polyp. The adenoma detection rate (ADR) is an indicator of the colonoscopist’s ability to detect adenomas. However, the ADR by colonoscopists varies from 7% to 53%[10]. Many studies have shown that endoscopists with a higher ADR in screening colonoscopy can more effectively protect patients from the subsequent risk of colon cancer[10,11]. Corley et al[10] evaluated 314872 colonoscopies performed by 136 colonoscopists. The results showed that for every 1.0% increase in ADR, the risk of CRC was reduced by 3.0%. However, the rate of missed adenoma during colonoscopy is still high and estimated to be between 6% and 27%[12]. Thus, new techniques are required to increase the ADR during colonoscopy. In recent years, more and more scholars have investigated the application of AI in the diagnosis of colonic polyps[13-26]. All these studies/applications with detailed data are summarized in Tables 1 and 2.

Table 1.

Characteristics of studies on artificial intelligence in the detection and classification of colorectal polyps

Ref.	Study type	Algorithm	Imaging modality	Image type	Training set	Testing set	Processing time
Mori et al[13]	Pilot study	-	EC	Real-time	-	-	0.3 s/image
Misawa et al[14]	Ex vivo	Machine learning: SVM	EC, NBI	Still	979 images (381 non-neoplasms, 598 neoplasms)	100 images (50 non-neoplasms, 50 neoplasms)	0.3 s/image
Kominami et al[15]	-	Machine learning: SVM	Colonoscopy, NBI	Real-time	2247 cutout training images from 1262 colorectal lesions	118 images	20 frame/s
Mori et al[16]	International web-based trial	Machine learning: SVM	EC	Still	6051 endocytoscopic images	205 small polyps (147 neoplastic and 58 non-neoplastic)	0.2 s/image
Misawa et al[17]	Pilot study	Machine learning: SVM	EC, NBI	Still	1661 EC-NBI images (1213 neoplasm images, 448 non-neoplastic images)	124 (19 neoplastic and 105 non-neoplastic)	-
Chen et al[18]	Pilot study	Deep neural network	Colonoscopy, magnifying NBI	Still	2157 (1476 neoplastic polyps vs 681 hyperplastic polyps)	284 (96 hyperplastic and 188 neoplastic polyps)	0.45 s/image
Misawa et al[19]	Ex vivo	Machine learning	Colonoscopy, WL	Video	411 (105 positive and 306 negative)	135 (50 positive and 85 negative)	-
Shin et al[20]	Pilot study	Machine learning	Colonoscopy, WL	Video	1525 (561 polyp patches and 964 normal patches)	366 (196 polyp patches and 170 normal patches)	95 ms/frame
Wang et al[21]	Ex vivo	Deep learning	Colonoscopy, WL	Still	5545 (3634 images contained polyps and 1911 images did not contain polyps)	27 113 (5541 images contained polyps and 21572 images did not contain polyps)	-
Kudo et al[22]	Pilot study	Texture analysis	EC stained or NBI image	Still	69 142 EC images (43197 stained images and 25945 NBI images)	100 polyps	0.4 s/image
Min et al[23]	Pilot study	Gaussian mixture model	Colonoscopy, linked color imaging	Still	139 images of adenomatous polyps and 69 images of non-adenomatous polyps	115 images of adenomatous polyps and 66 images of non-adenomatous polyps	-
Sánchez-Montes et al[24]	Pilot study	SVM	Colonoscopy, WL	Still	-	-	-
Horiuchi et al[25]	Pilot study	-	Colonoscopy, autofluorescence imaging	Real-time	-	-	-
Byrne et al[26]	Ex vivo	Convolutional neural network	EC, NBI	Video	223 polyp videos	125 polyp videos	50 ms/frame

EC: Endocytoscopic images; SVM: Support vector machine; NBI: Narrow-band imaging; WL: White light.

Table 2.

Performance of artificial intelligence in the detection and classification of colorectal polyps

Ref.	Patients, n	Samples, n	Sensitivity, %	Specificity, %	Accuracy, %	NPV, %	PPV, %
Mori et al[13]	152	176	92.0	79.5	89.2	-
Misawa et al[14]	-	100	84.5	97.6	90.0	82.0	98.0
Kominami et al[15]	41	118	95.9	93.3	94.9	93.3	95.9
Mori et al[16]	123	205	89.0	88.0	89.0	76.0	95.0
Misawa et al[17]	58	64	94.3	71.4	87.8	83.3	89.2
Chen et al[18]	193	284	96.3	78.1	90.1	91.5	89.6
Misawa et al[19]	73	155	90.0	63.3	76.5	-	-
Shin et al[20]	-	366	95.9	95.9	95.9	-	96.4
Wang et al[21]	1138	27113	94.4	95.9	-	-	-
Kudo et al[22]	89	100	96.9 (stained)	100.0	98.0	94.6	100.0
			96.9 (NBI)	94.3	96.0	94.3	96.9
Min et al[23]	91	181	83.3	70.1	78.4	71.2	82.6
Sánchez-Montes et al[24]	-	225	92.3	89.2	91.1	87.1	93.6
Horiuchi et al[25]	77	258	80.0	95.3	91.5	93.4	85.2
Byrne et al[26]	-	106	98.0	83.0	94.0	97.0	90.0

NBI: Narrow-band imaging; NPV: Negative predictive value; PPV: Positive predictive value.

Application of AI in colorectal polyp detection

AI is increasingly applied in gastrointestinal endoscopy, especially in the detection of colorectal polyps[27,28]. The ideal automatic detection tools for polyps should have a high sensitivity for polyp detection, a low rate of false positives, and a low latency so that polyps can be tracked and identified during real-time colonoscopy. Bowel preparation quality is an important factor affecting the accuracy of routine colonoscopy. Becq et al[29] evaluated the performance of a deep learning method for polyp detection during routine colonoscopy with variable bowel preparation quality and found that the deep learning method could effectively identify polyps by colonoscopy, even in the setting of variable bowel preparation quality.

In recent years, many studies have found that an AI system can remind the endoscopist in real time to avoid the omission of nonpolypoid lesions and other abnormalities during colonoscopy, which increases the ADR[30,31] (Figure 2). However, this requires validation in large multicenter trials. In addition to conventional computer-assisted diagnosis (CAD), a convolutional neural network (CNN) system using AI has rapidly developed over the past 5 years[32]. A novel online and offline three-dimensional deep learning integration framework based on a three-dimensional fully convolutional network was proposed by Yu et al[33]. This framework can learn more representative spatiotemporal features from colonoscopy videos and has stronger recognition ability compared with previous methods such as two-dimensional CNN or hand-crafted features[33]. Recently, a novel AI system (GI-Genius, Medtronic) was reported to have a sensitivity of 99.7% in the detection of colorectal polyps. The proportion of false positive frames found from colonoscopy was less than 1% of the total frames. Furthermore, the reaction time was shorter using this novel AI system compared with visual inspection by endoscopists in 82% of the cases[34]. A meta-analysis including six studies of AI on polyp detection showed a pooled area under the receiver operating characteristic curve (AUC) of 0.90. The pooled sensitivity and specificity of AI for polyp detection were 95.0% and 88.0%, respectively[35].

Figure 2.

Workflow of the artificial intelligence system in endoscopy. The location and diagnostic probability of polyps can be marked on the screen in real time with an alarm. AI: Artificial intelligence.

Wireless capsule endoscopy (WCE) is a noninvasive alternative to conventional endoscopes and is an essential tool for diagnostic inspection of the gastrointestinal tract. Due to large amounts of data captured by WCE, it takes a few hours for the doctor to make a diagnostic decision as the images need to be checked frame by frame. Therefore, an automatic CAD system is essential to assist physicians in analyzing and separating polyp images from whole data. For this purpose, Yuan et al[36] proposed a novel deep feature learning algorithm, named stacked sparse autoencoder with image manifold constraint, to identify polyps in the WCE images. The average accuracy of this algorithm for WCE images was 98.0%. Although this accuracy is high, it is far from perfect. The proposed algorithm did not perform well if inhomogeneous illuminations existed in the WCE images. Thus, there is still a lot of work to do to improve this method.

Application of AI in colorectal polyp classification

In the preservation and incorporation of valuable endoscopic innovations of the American Society for Gastrointestinal Endoscopy recommendation, endoscopists are required to receive intensive training on image-enhanced endoscopy to achieve a negative predictive value of > 90% in predicting the absence of adenomatous histology[37]. At present, many AI systems have reached the above standards. A total of 7680 colonic polyp images from 18 studies were included in a meta-analysis of polyp histology prediction utilizing an AI system. The pooled sensitivity in polyp histology prediction was 92.3%, and pooled specificity was 89.8%. The AUC of the AI in polyp histology prediction was 0.96[35]. When compared with visual inspection by endoscopists, the results of one study show that AI had similar precision (87.3% vs 86.4%) but a higher recall rate (87.6% vs 77.0%) and higher accuracy (85.9% vs 74.3%)[38]. Sánchez-Montes et al[24] developed a CAD system that can help the identification of dysplastic lesions. This system includes three stages: (1) Image preprocessing; (2) Extraction of textons. They used three texton feature images (branching, tubularity, and contrast) generated from textons extracted from the input image; and (3) Characterization. The sensitivity, specificity, accuracy, negative predictive value, and positive predictive value were 92.3%, 89.2%, 91.1%, 87.1%, and 93.6%, respectively[24].

More and more studies have revealed that deep learning using CNNs is a good option for colonic polyp classification[39] (Figure 3). Song et al[40] reported that the overall diagnostic accuracy of CAD using a deep learning model was 81.3%-82.4%, which was significantly higher than that of the trainees (63.8%-71.8%, P < 0.01) and comparable with that of experts (82.4%-87.3%)[40]. This result suggests that CAD using deep learning is helpful for trainees in diagnosing colorectal polyps. Similar results were also seen in the evaluation of diminutive (< 5 mm) colorectal polyps by a CNN model, which also significantly reduced the time of diagnosis by endoscopists (from 3.92 s to 3.37 s/polyp, P = 0.042)[41]. An optical diagnosis model based on CNN was specifically designed to identify hyperplastic/serrated and adenomatous polyps, and the performance of this model exceeded the threshold of preservation and incorporation of valuable endoscopic innovations for both “diagnose and leave” and “resect and discard” strategies independent of narrow-band imaging utilization[42]. With the development of AI systems, endoscopists may accurately predict the pathology of polyps less than 3 mm in diameter[43].

Figure 3.

Deep learning using deep neural network for colonic polyp classification.

The above studies were all retrospective. In order to further verify the effectiveness of AI in recent years, more and more scholars have begun to carry out prospective research to examine the application of AI in the diagnosis of colorectal polyps[44-48]. Wang et al[46] conducted a nonblinded, prospective randomized controlled study from September 2017 to February 2018, which included the largest sample size to date. This prospective study enrolled 1058 patients including 522 randomized to colonoscopy with CAD and 536 randomized to standard colonoscopy. The results showed that the AI system significantly increased the ADR (29.1% vs 20.3%, P < 0.001) and the mean number of adenomas per patient (0.53 vs 0.31, P < 0.001)[46]. In order to eliminate the operational bias in their nonblinded study and evaluate the effectiveness of the CAD system more rigorously, the authors performed a randomized, double-blind trial from September 2018 in a single center. The ADR was significantly higher in the CAD group than in the sham group (34% vs 28%, P = 0.03)[47].

Application of AI in colorectal polyp histopathological recognition

Histopathological characterization of colorectal polyps is still the gold standard for diagnosis of polyps. It is critical for determining future endoscopic resection or regular follow-up in patients. However, this characterization is a challenging task and suffers significant intra- and interobserver variability. Thus, an automatic image analysis AI system that can help pathologists to identify different types of colorectal polyps accurately is necessary. In recent years, many scholars have begun to probe into this area[49-51]. Korbar et al[50] proposed an AI system based on a deep neural network model to identify the types of colorectal polyps on whole slide, hematoxylin and eosin-stained images. The results of this system showed a precision of 89.7%, F1 score of 88.8%, recall of 88.3%, and accuracy of 93.0%[50]. In another study, a deep learning model was proposed to recognize four different stages of cancerous tissue development, including normal mucosa, early preneoplastic lesion, adenoma, and cancer. An overall accuracy of > 95% was achieved[51].

AI IN CRC

CRC usually begins with a benign tumor, initially in the form of polyps, which will develop into cancer over time. It is the third most common cancer and second most common cause of cancer-related mortality worldwide[52]. The number of patients with new onset CRC is approximately 12 million a year with 600000 deaths[53]. The high mortality and poor prognosis of CRC make this disease a huge threat to the social economy and people’s health. Early diagnosis and treatment patients with CRC have always been the focus of clinical work. The systematic research in application of AI in the diagnosis of CRC is still lacking. However, with the continuous development of AI and more applications in the field of medicine, it has now emerged in the diagnosis of CRC.

Application of AI in the qualitative diagnosis of CRC

The diagnosis of colorectal tumors can be divided into qualitative diagnosis and staging diagnosis. Qualitative diagnosis refers to colonoscopy and pathological biopsy to determine the presence of colorectal tumors. Colonoscopy has been an effective tool in the early detection of neoplastic lesions. Although magnifying endoscopy[54], narrow-band imaging[55], endocytoscopy[56], and confocal laser endomicroscopy[57] have a higher accuracy, the results are operator dependent. It is difficult to train all endoscopists to perform all methods well. Thus, a CAD system for endocytoscopy was developed to solve this problem. Takeda et al[58] carried out a study to evaluate the diagnostic ability of a CAD system for endocytoscopy for invasive CRC. In this study, a CAD system for endocytoscopy analyzed endocytoscopy images that are based on the information from texture analysis and nuclei. All 296 features (288 from texture analysis, 8 from nuclei) are used as the data for evaluating endocytoscopy images. A support vector machine analyzed these features and classified the images into three histological groups: Invasive cancer, adenoma, and non-neoplasm. The sensitivity, specificity, accuracy, negative predictive value, and positive predictive value were 89.4%, 98.9%, 94.1%, 90.1%, and 98.8%, respectively[58]. This technology is expected to bridge the gap in diagnosis quality for endoscopists at different levels. Histopathological diagnosis can be made by pathologists based on images of tissues obtained from a colonoscopic biopsy. Recently, many scholars have begun to explore the application of AI in identifying histopathological images of CRC[59,60]. Yoon et al[60] evaluated the performance of the CNN model in histologic diagnosis. The results for sensitivity, specificity, and accuracy were 95.10%, 92.76%, and 93.48%, respectively. The CNN model correctly classified 294 of 309 normal images and 667 of 719 tumor images[60].

Application of AI in the staging diagnosis of CRC

AI is also used in the staging diagnosis of CRC. Computed tomography, magnetic resonance imaging (MRI), and other imaging techniques are commonly used to stage CRC. Ito et al[60] used CNN to assist in the diagnosis of cT1b CRC. With CNN learning, the sensitivity, specificity, and accuracy were 67.5%, 89.0%, and 81.2%, respectively, and the AUC was 0.871[61]. Whether additional surgery is required after endoscopic resection of T₁ CRC is currently based on international guidelines. A recent study reported that an AI model predicted positivity or negativity for lymph node metastasis by analyzing 45 clinicopathological factors of T₁ CRC. The sensitivity, specificity, and accuracy were 100%, 66%, and 69%, respectively, which were higher compared to the current guidelines[62]. These results suggested that AI may help to reduce unnecessary additional surgery after endoscopic resection of T₁ CRC. MRI is the best method for confirming the diagnosis of pelvic lymph node metastasis before surgery. Radiologists make diagnostic decisions usually based on their subjective experience. Thus, this diagnosis lacks accuracy and objectivity. To address this problem, a faster region-based CNN was trained to read pelvic MRI images and to make diagnoses with an AUC of 0.912. The diagnosis time in one case by faster region-based CNN was 20 s, which was much shorter than that (600 s) for the radiologists’ diagnoses[63,64]. These results suggest that the faster region-based CNN enables an accurate and rapid diagnosis of CRC lymph node metastasis.

Liver is another common metastatic site of CRC. Therefore, screening of CRC patients with a high risk of liver metastasis is very important for individualized surveillance. One hundred and fifty-two tumor features extracted from computed tomography imaging and six clinical factors were used to develop a new noninvasive AI model for this task. The hybrid model, which combined relevant imaging features and clinical variables, improved accuracy of both training (90.63%) and validation (85.50%) sets with an AUC of (0.96; 0.87)[65].

Colorectal tumor segmentation is an important step in the analysis and diagnosis of CRC. However, the manual delineation of tumors is a time-consuming procedure and requires a high level of expertise. Thus, many deep learning models for automatic localization and segmentation of rectal cancers on MRI images have been proposed with high accuracy in recent years[66,67]. These AI systems can save radiologists a lot of time, but more randomized controlled studies are required to verify the stability of the results.

FUTURE PROSPECTS

Although the results from previous studies appear to be promising, supporting evidence of AI systems applied in colonoscopy is still lacking as most studies were designed retrospectively. Due to the retrospective nature of most studies and the potential selection bias involved, further prospective double-blinded clinical trials are required to confirm the role of AI-assisted colonoscopy in clinical practice. We suggest the following for further research: (1) A prospective evaluation with real-time use of AI is required; (2) In order for the proposed method to have practical value in clinical trials, further testing with a large number of pathologically proved data sets is very important to verify the effectiveness and stability of the proposed classification method; (3) A study in an international, multicenter setting should be conducted to ensure the reproducibility and stability of the results; and (4) The efficacy of AI should be evaluated in all types of colorectal lesions. Other important types of lesions such as sessile serrated lesions, ulcerative colitis, or colitis-associated cancer should be also investigated as targets of an AI system. In addition, the establishment of a clinical AI system requires the use of a large amount of clinical data from patients. Compared with research in other directions, the application of medical data also involves protection of patient privacy and ethical issues. Once the information is leaked, it may cause unpredictable consequences. Therefore, the safe management of medical data should also be a key issue. When these problems are appropriately addressed, AI can be used clinically for colorectal diseases.

CONCLUSION

AI is an exciting new field in colorectal diseases. AI technologies such as deep learning can speed up the processing of large amounts of imaging or clinical data, allowing machines to assist physicians in many important tasks such as colorectal polyp detection and classification as well as qualitative and staging diagnosis of CRC. In order to utilize AI wisely, clinicians should strive to understand the feasibility of AI and mitigate its drawbacks.