Table 2.
Detailed characteristics of endoscopic imaging datasets
| Name | Findings | Clinical detailsa |
|---|---|---|
| Endoscopy datasets of polyps (n = 16) | ||
| CVC-ColonDB | Polyps | No |
| CVC-ClinicDB (CVC-612) | Polyps | Yes |
| ETIS-Larib Polyp DB | Polyps and normal mucosa | No |
| ASU-Mayo | Polyps and normal mucosa | No |
| GI lesions in Regular Colonoscopy Dataset | Serrated adenomas, hyperplastic polyps and adenomas | Yes |
| EndoScene | Polyps | No |
| Kvasir SEG | Polyps | No |
| NBIPolyp-Ucdb | Adenomas and hyperplastic polyps | No |
| WLPolyp-UCdb | Polyps and normal mucosa | No |
| KUMC | Adenomas and hyperplastic polyps | No |
| SUN | Polyps | Yes |
| PICCOLO | Polyps | Yes |
| CP-CHILD | Polyps and normal mucosa | No |
| LD Polyp Video | Polyps | No |
| SUN-SEG | Polyps | Yes |
| PolypGen | Polyps and normal mucosa | Yes |
| Endoscopy datasets of small bowel lesions (n = 6) | ||
| KID | SB lesions (vascular lesions, inflammatory lesions, lymphangiectasias, polypoid lesions) and normal mucosa | No |
| CAD-CAP | SB lesions (vascular lesions, fresh blood, ulcer-inflammatory lesions, and other lesions) and normal mucosa | No |
| Kvasir-Capsule | Anatomy (pylorus, ampulla of Vater, ileocecal valve), SB lesions (mucosal atrophy, lymphangiectasia, erythema, angiectasia, blood-fresh, blood-hematin, erosion, ulcer, polyp, foreign body) and normal mucosa | No |
| Endoscopy Crohn’s Disease dataset | Clearness degree classification in Crohn’s tract (clearness, blur and invisible) | Yes |
| CrohnIPI | Crohn’s lesions (erythema, edema, aphthoid ulceration, 3–10 mm ulceration, > 10 mm ulceration, stenosis) and normal mucosa | No |
| The AICE Project | SB lesions (angiodysplasia, erosion, stenosis, lymphangiectasia, lymph follicle, polyp-like lesions, submucosal tumor, bleeding, diverticulum, erythema, foreign body, vein lesions) and normal mucosa | No |
| Endoscopy datasets of gastro-esophageal lesions (n = 2) | ||
| IPCL | Four classes of abnormal esophageal IPCL (A, B1, B2, B3) and normal mucosa | No |
| IM and GA Benchmark | GA, IM and normal images in five gastric lesion location (antrum, angle, cardia, fundus and body) by WLI and LCI | No |
| Endoscopy datasets of comprehensive GI detection (n = 5) | ||
| Kvasir | 8 classes in GI tract (Z-line, pylorus, cecum, esophagitis, polyps, ulcerative colitis, “dyed and lifted polyp” and “dyed resection margins”) | No |
| Hyper Kvasir | 23 classes of GI findings including anatomical landmarks, quality of mucosal views, pathological findings, therapeutic interventions and so onc | No |
| Rhode Island | Findings in four anatomical organs (esophagus, stomach, small bowel, colon) | No |
| WCE Curated Colon Disease | 3 classes of GI lesions (ulcer, polyps, esophagitis) and normal mucosa | No |
| ERS | 34 classes of colonoscopy findings, 70 classes of upper endoscopy findings and 7 classes of healthy tissues, 2 classes of blood and 10 classes of imaging qualityc | Yes |
| Atlases of GI endoscopy (n = 4) | ||
| Gastrolab | Findings in GI tract (14 anatomical structures, 7 classes of GI lesions and 4 others)c | No |
| WEO Clinical Endoscopy Atlas | Findings in 6 classes of GI tract (lumen, contents, mucosa, flat lesions, protruding lesions, excavated lesions) | No |
| Atlas of Gastrointestinal Endoscopy | Findings in GI tract (esophagus, stomach, capsule endoscopy, duodenum and ampulla, inflammatory bowel disease, colon and ileum, miscellaneous) with several rare diseases | No |
| El Salvador atlas | Findings in GI tract | No |
| Others (n = 7) | ||
| GIANA 2017 | Polyps and angiodysplasia | No |
| Nerthus | 4 degrees of bowel preparation (BBPS 0–3) | No |
| GIANA 2018 | Polyps and lesions in WCE | No |
| EAD 2019 | 7 classes of artifacts during GI endoscopy process (imaging artefacts, pixel contrast, specular reflections, motion blur, bubbles, pixel saturation and instrument) | No |
| Cho et al. 2019 | Cecal landmarks in insertion, withdrawal and stopping point | No |
| EDD 2020 | 5 classes of GI lesions (Barrett’s esophagus, suspicious area, high-grade dysplasia; adenocarcinoma and polyps) | No |
| Kvasir-Instrument | Diagnostic and therapeutic tools in endoscopic images (such as biopsy forceps, metallic clip, probe, snares) | No |
| Name | Source of ground truth annotations | Endoscopy system brand | ||
|---|---|---|---|---|
| Ground truthb | Content | Performed by | ||
| Endoscopy datasets of polyps (n = 16) | ||||
| CVC-ColonDB | Binary mask | Polyps | Expert endoscopists | NR |
| CVC-ClinicDB (CVC-612) | Binary mask | Polyps and specular highlights | Expert endoscopists | NR |
| ETIS-Larib Polyp DB | Bounding box | Polyps | Expert endoscopists | NR |
| ASU-Mayo | Binary mask and bounding box | Polyps | Expert endoscopists | NR |
| GI lesions in Regular Colonoscopy Dataset | Annotated file and bounding box in videos | Polys | Histopathology, and the human operators’ opinion (4 expert endoscopists and 3 beginners) | Olympus |
| EndoScene | Binary mask | Polyps, specular highlights, and lumen | Expert endoscopists | NR |
| Kvasir SEG | Binary mask and bounding box | Polyps | Expert endoscopists | Olympus |
| NBIPolyp-Ucdb | Binary mask | Polyps | Endoscopists (experience not specified) | Olympus |
| WLPolyp-UCdb | Annotated file | Polyps | NR | Olympus |
| KUMC | Bounding box | Polyps in dataset 3 and 4 | Endoscopists (experience not specified) | NR |
| SUN | Bounding box | Polyps | Expert endoscopists and research assistants | Olympus |
| PICCOLO | Binary mask | Polyps | Expert endoscopists | Olympus |
| CP-CHILD | Annotated file | Polyps | Expert endoscopists | Olympus for A; Fujifilm for B |
| LD Polyp Video | Bounding box | Polyps | Annotation tool | NR |
| SUN-SEG | Binary mask, bounding box, scribble, and polygon | Polyps | 10 expert endoscopists and 2 researchers | Olympus |
| PolypGen | Binary mask and bounding box | Polyps | 6 expert endoscopists, 2 post-doctoral researchers and 1 student | Multi-brands |
| Endoscopy datasets of small bowel lesions (n = 6) | ||||
| KID | Binary mask and graphical annotation | Small bowel lesions | Expert endoscopists | NR |
| CAD-CAP | Binary mask | Small bowel lesions | Expert endoscopists | Medtronic |
| Kvasir-Capsule | Bounding box | Small bowel lesions | Expert endoscopists, master students and junior endoscopists | Olympus |
| Endoscopy Crohn’s Disease dataset | Annotated file | Clearness degrees in Crohn’s tract | Expert endoscopists and junior endoscopists | Jinshan Groupd |
| CrohnIPI | Annotated file | 6 Crohn’s lesions | Expert endoscopists and one reader | Medtronic |
| The AICE Project | Annotated file | Small bowel lesions | NR | Medtronic |
| Endoscopy datasets of gastro-esophageal lesions (n = 2) | ||||
| IPCL | Annotated file | IPCL in esophageal | Histopathology and expert endoscopists | Olympus |
| IM and GA Benchmark | Annotated file | Intestinal metaplasia and gastritis atrophy | Biopsy and endoscopists (experience not specified) | Fujifilm |
| Endoscopy datasets of comprehensive GI detection (n = 5) | ||||
| Kvasir | Annotated file | 8 classes in GI tract | Expert endoscopists | Olympus |
| Hyper Kvasir | Annotated file, binary mask, and bounding box | Annotated file for 23 classes; bounding box and binary mask only for polyps | Expert endoscopists and junior endoscopists | Olympus and Pentax |
| Rhode Island | Annotated file | Findings in four anatomical organs | Endoscopists (experience not specified) | Medtronic |
| WCE Curated Colon Disease | Annotated file | 3 classes of GI lesions | Expert endoscopists | NR |
| ERS | Binary mask and bounding box | 3600 precise and 22,600 approximate segmentation masks | Expert endoscopists | NR |
| Atlases of GI endoscopy (n = 4) | ||||
| Gastrolab | Annotated file | Findings in the GI tract | NR | NR |
| WEO Clinical Endoscopy Atlas | Annotated file | Findings in the GI tract | Expert endoscopists | NR |
| Atlas of Gastrointestinal Endoscopy | Annotated file | Findings in the GI tract | Expert endoscopists | NR |
| El Salvador atlas | Annotated file | Findings in the GI tract | NR | NR |
| Others (n = 7) | ||||
| GIANA 2017 | Binary mask and bounding box | Binary mask for polyps and bounding box for angiodysplasia | NR | NR |
| Nerthus | Annotated videos | Bowel preparation quality | Expert endoscopists | Olympus |
| GIANA 2018 | Binary mask and bounding box | Polys | NR | NR |
| EAD 2019 | Bounding box | Endoscopy artifact | Expert endoscopists and postdoctoral fellows | Multi-brands |
| Cho et al. 2019 | Annotated file | Cecal landmarks | Expert endoscopists | Olympus |
| EDD 2020 | Binary mask and bounding box | GI lesions | Expert endoscopists and post-doctoral researchers | NR |
| Kvasir-Instrument | Binary mask, bounding box, and image annotation | Diagnostic and therapeutic tools in gastrointestinal endoscopy | Research assistants and expert endoscopists | Olympus and Pentax |
| Name | No. of patients | No. of images | ||
|---|---|---|---|---|
| Total | Annotated/Not annotated | Classification of annotated images | ||
| Endoscopy datasets of polyps (n = 16) | ||||
| CVC-ColonDB | 13 | 0 | 0 | 0 |
| CVC-ClinicDB (CVC-612) | 23 | 0 | 0 | 0 |
| ETIS-Larib Polyp DB | NR | 1500 | 300/1200 | 300 of polyp |
| ASU-Mayo | 20 | 19,400 | 19,400/0 | 5200 of polyp; 14,200 of normal mucosa |
| GI lesions in Regular Colonoscopy Dataset | NR | 0 | 0 | 0 |
| EndoScene | 36 | 0 | 0 | 0 |
| Kvasir SEG | NR | 1000 | 1000/0 | 1000 of polyp |
| NBIPolyp-Ucdb | 10 | 0 | 0 | 0 |
| WLPolyp-UCdb | 42 | 3040 | 3040/0 | 1680 of polyp; 1360 of normal colon mucosa |
| KUMC | NR | 0 | 0 | 0 |
| SUN | 99 | 0 | 0 | 0 |
| PICCOLO | 40 | 3433 | 3433/0 | 3433 of polyp (2131 by WLI, 1302 by NBI) |
| CP-CHILD | NR | 9500 | 9500/0 | 1400 of polyp (1000 by Olympus, 400 by Fujifilm); 8100 of non-polyp (7000 by Olympus, 1100 by Fujifilm) |
| LD Polyp Video | NR | 0 | 0 | 0 |
| SUN-SEG | 99 | 0 | 0 | 0 |
| PolypGen | > 300 | 6282 | 6282/0 | 3762 of polyp; 2520 of normal colon mucosa |
| Endoscopy datasets of small bowel lesions (n = 6) | ||||
| KID | NR | 2500 | 2500/0 | 2500 of SB lesions |
| CAD-CAP | NR | 0 | 0 | 0 |
| Kvasir-Capsule | NR | 0 | 0 | 0 |
| Endoscopy Crohn’s disease dataset | 15 | 466 | 466/0 | 323 of clearness; 101 of blur; 42 of invisible |
| CrohnIPI | ≥ 200 | 3498 | 3498/0 | 2124 of normal SB; 1360 of lesions; 14 of inconclusive findings |
| The AICE Project | NR | 18,481 | 18,481/0 | 12,320 images with 19,459 annotations (931 of angiodysplasia, 5988 of erosion, 477 of stenosis, 612 of lymphangiectasia, 6792 of lymph follicle, 547 of SMT, 3236 of polyp, 875 of bleeding); 6161 of normal SB |
| Endoscopy datasets of gastro-esophageal lesions (n = 2) | ||||
| IPCL | 114 | 67,740 | 67,740/0 | 39,662 of lesion; 28,078 of normal mucosa |
| IM and GA Benchmark | 630 | 21,420 | 21,240/0 | 2438 of WLI-IM; 3381 of WLI-GA; 5854 of WLI-Normal; 2549 of LCI-IM; 3270 of LCI-GA; 3928 of LCI-Normal |
| Endoscopy datasets of comprehensive GI detection (n = 5) | ||||
| Kvasir | NR | 8000 | 8000/0 | 1000 of Z-line; 1000 of pylorus; 1000 of cecum; 1000 of esophagitis; 1000 of polyps; 1000 of ulcerative colitis; 1000 of “dyed and lifted polyp”; 1000 of “dyed resection margins” |
| Hyper Kvasir | NR | 110,079 | 10,662/99,417 | 3452 of 7 classes from upper GI tract; 7210 of 16 classes from lower GI tractc |
| Rhode Island | 424 | 0 | 0 | 0 |
| WCE Curated Colon Disease | NR | 6000 | 6000/0 | 1500 of polyp; 1500 of ulcerative colitis; 1500 of esophagitis; 1500 of normal mucosa |
| ERS | 1135 | 0 | 0 | 0 |
| Atlases of GI endoscopy (n = 4) | ||||
| Gastrolab | NR | ≥ 1498 | ≥ 1498/0 | ≥ 1498 of GI findingsc |
| WEO Clinical Endoscopy Atlas | NR | 148 | 148/0 | 31 of lumen; 5 of content; 25 of mucosa; 14 of flat lesions; 49 of protruding lesions; 24 of excavated lesions |
| Atlas of Gastrointestinal Endoscopy | NR | 1259 | 1259/0 | 1259 of GI findings |
| El Salvador atlas | NR | 0 | 0 | 0 |
| Others (n = 7) | ||||
| GIANA 2017 | NR | ≥ 1500 | ≥ 1500/0 | 600 of angiodysplasia; ≥ 900 of poly |
| Nerthus | 21 | 5525 | 5525/0 | 5525 of four classes of bowel cleanliness |
| GIANA 2018 | NR | 8262 | 8262/0 | 8262 of polyp and small bowel lesion |
| EAD 2019 | NR | ≥ 2500 | ≥ 2500/0 | ≥ 2500 of endoscopy artifacts |
| Cho et al. 2019 | 112 | 0 | 0 | 0 |
| EDD 2020 | NR | 385 | 385/0 | 385 images with 502 ground truth annotations (160 of non-dysplastic Barrett’s; 88 of precancerous lesion; 74 of high-grade dysplasia; 53 of cancer; 127 of cancer) |
| Kvasir-Instrument | NR | 590 | 590/0 | 590 of diagnostic and therapeutic tools |
| Name | No. of videos (frames) | Image/ frame resolution (pixels) | Imaging modality | ||
|---|---|---|---|---|---|
| Total | Annotated/Not annotated | Classification of annotated videos(frames) | |||
| Endoscopy datasets of polyps (n = 16) | |||||
| CVC-ColonDB | 13 (300) | 13 (300)/0 | 13 (300) of polyp | 500 × 574 | WLI |
| CVC-ClinicDB (CVC-612) | 31 (612) | 31 (612)/0 | 31 (612) of polyp | 576 × 768 | WLI |
| ETIS-Larib Polyp DB | 0 | 0 | 0 | 1225 × 966 | NR |
| ASU-Mayo | 0 | 0 | 0 | 512 × 512 | NR |
| GI lesions in Regular Colonoscopy Dataset | 76 (NR) | 76 (NR)/0 | 15 of serrated adenoma; 21 of hyperplastic polyp; 40 of adenoma | 30 frames of 768 × 576 pixels every second | NBI and WLI |
| EndoScene | 44 (912) | 44 (912)/0 | 44 (912) of polyp (300 from CVC-ColonDB and 612 from CVC-ClinicDB) | 224 × 224 | WLI |
| Kvasir SEG | 0 | 0 | 0 | 320 × 320 | WLI |
| NBIPolyp-Ucdb | 11 (86) | 11 (86)/0 | 10 videos of adenoma and 1 video of hyperplastic polyp | 726 × 576 | NBI |
| WLPolyp-UCdb | 0 | 0 | 0 | 726 × 576 | WLI |
| KUMC | 157 (35,981) | 76 (4955)/81 (31,026) | 76 (4955) of polyp (38 videos of adenoma and 38 videos of hyperplastic polyp) | 224 × 224 | WLI |
| SUN | 113(158,690) | 113 (158,690)/0 | 100 (49,136) of polyp; 13 (109,554) of non-polyp | 416 × 416 | WLI |
| PICCOLO | 0 | 0 | 0 | 854 × 480 or 1920 × 1080 | NBI and WLI |
| CP-CHILD | 0 | 0 | 0 | 256 × 256 | NR |
| LD Polyp Video | 263 (901,666) | 160 (40,266)/103 (861,400) | 33,884 frames of polyp; 6382 frames of non-polyp | 560 × 480 | WLI |
| SUN-SEG | 1106 (158,690) | 1106 (158,690)/0 | 378 (49,136) of polyp; 728 (109,554) of non-polyp | 416 × 416 | WLI |
| PolypGen | 0 | 0 | 0 | 384 × 288 to 1920 × 1080 | NR |
| Endoscopy datasets of small bowel lesions (n = 6) | |||||
| KID | 47 | 47/ 0 | 47 videos of SBesions | NR | NR |
| CAD-CAP | 1686 (25,124) | 1686 (25,124)/0 | 1480 (5124) of abnormal SB findings (3103 of vascular lesion, 651 of fresh blood, 1370 of ulcer-inflammatory lesion); 206 (20,000) of normal SB | NR | WLI |
| Kvasir-Capsule | 117 (4,741,504) | 43 (47,238)/74 (4,694,266) | 1529 frames of pylorus; 10 frames of ampulla of Vater; 4189 of ileocecal valve; 2906 of mucosal atrophy; 592 of lymphangiectasia; 159 of erythema; 866 of angiectasia; 466 of fresh blood; 12 of blood-hematin; 506 of erosion; 854 of ulcer; 55 of polyp; 776 of foreign body; 4189 of normal mucosa | 256 × 256 to 512 × 512 | WLI |
| Endoscopy Crohn’s disease dataset | 0 | 0 | 0 | 240 × 240 | WLI |
| CrohnIPI | 0 | 0 | 0 | NR | WLI |
| The AICE Project | 0 | 0 | 0 | NR | WLI |
| Endoscopy datasets of gastro-esophageal lesions (n = 2) | |||||
| IPCL | 0 | 0 | 0 | 256 × 256 | ME-NBI |
| IM and GA Benchmark | 0 | 0 | 0 | 1280 × 1024 | WLI and LCI |
| Endoscopy datasets of comprehensive GI detection (n = 5) | |||||
| Kvasir | 0 | 0 | 0 | 720 × 574 to 1920 × 1072 | WLI |
| Hyper Kvasir | 374 | 374/ 0 | 60 videos of 14 classes from upper GI tract; 314 videos of 16 classes from lower GI tractc | 332 × 352 to 1921 × 1073 | WLI |
| Rhode Island | 424 (5,247,588) | 424 (5,247,588)/0 | 13,715 frames of esophagus; 557,049 frames of stomach; 4,111,865 frames of small bowel; 564,959 frames of colon | 320 × 320 | WLI |
| WCE Curated Colon Disease | 0 | 0 | 0 | 224 × 224 | NR |
| ERS | 1520 (≥ 1,230,000) | 121,000/1230000 (frames) | 6000 frames of precisely labeled; 115,000 frames of approximately labeledc | NR | NR |
| Atlases of GI endoscopy (n = 4) | |||||
| Gastrolab | 0 | 0 | 0 | NR | NR |
| WEO Clinical Endoscopy Atlas | 0 | 0 | 0 | NR | NR |
| Atlas of Gastrointestinal Endoscopy | 0 | 0 | 0 | NR | NR |
| El Salvador atlas | 5138 (NR) | 5138 (NR)/0 | 5138 videos of GI findings | NR | NR |
| Others (n = 7) | |||||
| GIANA 2017 | ≥ 38 | ≥ 38/0 | ≥ 38 videos of polyp | NR | NR |
| Nerthus | 0 | 0 | 0 | 720 × 576 | WL |
| GIANA 2018 | 38 | 38/0 | 38 videos of polyp and SB lesions | NR | NR |
| EAD 2019 | 0 | 0 | 0 | NR | Multi-modal (WLI, BNI and fluorescence) |
| Cho et al. 2019 | 112 (328,927) | 2 (100) available | 2 (100) of cecal landmarks | 850 × 750 | WLI |
| EDD 2020 | 0 | 0 | 0 | NR | NR |
| Kvasir-Instrument | 0 | 0 | 0 | 720 × 576 to 1280 × 1024 | NR |
NR not reported, SB small bowel, GA gastric atrophy, IM intestinal metaplasia, WLI white light imaging, LCI linked color imaging, BBPS Boston bowel preparation scale, WCE wireless capsule endoscopy, NBI narrow band imaging
aClinical details includes clinical metadata (patients’ information, location of polyp, type of polyp and so on) or annotation details (annotators’ number, annotation process and so on)
bAnnotated file for classification task, binary mask for segmentation task, and bounding box for detection task
cComplete information of classification was shown in Supplementary Table 1
dChongqing Jinshan Science & Technology (Jinshan Group), a national high-tech enterprise that integrates research and development, manufacturing, marketing, and service of digital medical devices