Table 1.
Overview of the published work regarding convolutional neural network (CNN) development for capsule endoscopy.
| Publication, Author, Year |
Study Aim |
Capsule Types |
Centers n |
Exams n |
Frames n | Types of CNN |
Dataset Methods |
Analysis Methods |
Classification Categories | Performance Metrics | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Total | Lesion | SEN | SPE | AUC | ||||||||||
| Specific CNN for Small Bowel Lesions | Aoki, 2020 [37] |
Detection of blood | SB2 SB3 |
1 | 66 | 38,055 | 6711 | ResNet | Frame labeling of all datasets (normal vs. blood content) |
Train–test split (73–26%) |
Blood | 97 | 100 | 100 |
| Afonso, 2021 [38] |
Detection of blood | SB3 | 2 | 1483 | 23,190 | 13,515 | Xception | Frame labeling of all datasets (normal vs. blood or hematic residues) |
Staged incremental frame with train–test split (80–20%) | Blood/hematic residues | 98 | 98 | 100 | |
| Leenhardt, 2019 [39] |
Detection of angiectasia | SB3 | French national of still frames database (from 13 centers) | NA | 1200 | 600 | YOLO | Previous manual annotation of all angiectasias for the French national database | Deep feature extraction of dataset already manually annotated (300 lesions and 300 normal frames) Validation with classification of new dataset (300 lesions and 300 normal frames) |
Angiectasia | 100 | 96 | NK | |
| Tsuboi, 2020 [40] |
Detection of angiectasia | SB2 SB3 |
2 | 169 | 12,725 | 2725 | SSD | Manual annotation of all angiectasias | CNN is trained exclusively in positive frames (2237 with angiectasias) Validation on mixed data with positive and negative frames (488 angiectasias and 10,000 normal frames) |
Angiectasia | 99 | 98 | 100 | |
| Houdeville, 2021 [41] |
Detection of angiectasia | SB3 Mirocam |
NA | NA | 12,255 | 613 | YOLO | Previous trained on SB3 devices | Validation with 626 new SB3 still frames and 621 new Mirocam still frames | Angiectasia (SB3) | 97 | 99 | NK | |
| Angiectasia (Mirocam) | 96 | 98 | NK | |||||||||||
| Ribeiro, 2021 [42] |
Detection of vascular lesions + categorization of bleeding potential | SB3 | 2 | 1483 | 11,588 | 2063 | Xception | Frame labeling of all datasets (normal (N) vs. red spots (P1V) vs. angiectasia or varices (P2V)) |
Train–test split (80–20%) with 3 × 3 confusion matrix |
N vs. all | 90 | 97 | 98 | |
| P1V vs. all | 92 | 95 | 97 | |||||||||||
| P2V vs. all | 94 | 95 | 98 | |||||||||||
| Aoki, 2019 [43] |
Detection of ulcerative lesions | SB2 SB3 |
1 | 180 | 15,800 | 5800 | SSD | Manual annotation of all ulcers or erosions | CNN is trained exclusively in positive frames (5630 with ulcers) Validation on mixed data with positive and negative frames (440 lesions and 10,000 normal frames) |
Ulcers or erosions | 88 | 91 | 96 | |
| Klang, 2020 [44] |
Detection of ulcers + differentiation from normal mucosa |
SB3 | 1 | 49 | 17,640 | 7391 | Xception | Frame labeling of all datasets (normal vs. ulcer) |
5-fold cross-validation with train–test split (80 vs. 20%) |
Ulcers (mean of cross-validation) |
95 | 97 | 99 | |
| Barash, 2021 [45] |
Categorization of severity grade of ulcers | SB3 | 1 | NK | Random selection of 1546 ulcer frames from Klang dataset | ResNet | Frame labeling of all datasets (mild ulceration (1) vs. moderate ulceration (2) vs. severe ulceration (3)) |
Train–test split (80–20%) with 3 × 3 confusion matrix | 1 vs. 2 | 34 | 71 | 57 | ||
| 2 vs. 3 | 73 | 91 | 93 | |||||||||||
| 1 vs. 3 | 91 | 91 | 96 | |||||||||||
| Afonso, 2021 [46] |
Detection of ulcerative lesions + categorization of bleeding potential | SB3 | 2 | 2565 | 23,720 | 5675 | Xception | Frame labeling of all datasets (normal (N) vs. erosions (P1E) vs. ulcers with uncertain/intermediate bleeding potential (P1U) vs. ulcers with high bleeding potential (P2U)) |
Train–test split (80–20%) with 4 × 4 confusion matrix |
N vs. all | 94 | 91 | 98 | |
| P1E vs. all | 73 | 96 | 95 | |||||||||||
| P1U vs. all | 72 | 96 | 96 | |||||||||||
| P2U vs. all | 91 | 99 | 100 | |||||||||||
| Saito, 2020 [47] |
Detection of protruding lesions | SB2 SB3 |
3 | 385 | 48,091 | 38091 | SSD | Manual annotation of all protruding lesions (polyps, nodules, epithelial tumors, submucosal tumors, venous structures) | CNN is trained exclusively in positive frames (30,584 with protruding lesions) Validation on mixed data with positive and negative frames (7507 lesions and 10,000 normal frames) |
Protruding lesions | 91 | 80 | 91 | |
| Saraiva, 2021 [48] |
Detection of protruding lesions + categorization of bleeding potential | SB3 | 1 | 1483 | 18,625 | 2830 | Xception | Frame labeling of all data (normal (N) vs. protruding lesions with uncertain/intermediate bleeding potential (P1PR) vs. protruding lesions with high bleeding potential (P2PR)) |
Train–test split (80–20%) with 3 × 3 confusion matrix |
N vs. all | 92 | 99 | 99 | |
| P1PR vs. all | 96 | 94 | 99 | |||||||||||
| P2PR vs. all | 97 | 98 | 100 | |||||||||||
| Specific CNN for Colonic Lesions | Yamada, 2021 [49] |
Detection of colorectal neoplasias | COLON2 | 1 | 184 | 20,717 | 17,783 | SSD | Manual annotation of all colorectal neoplasias (polyps and cancers) | CNN is trained exclusively in positive frames (15,933 with colorectal neoplasias) Validation on mixed data with positive and negative frames (1805 lesions and 2934 normal frames) |
Colorectal neoplasias | 79 | 87 | 90 |
| Saraiva, 2021 [50] |
Detection of protruding lesions | COLON2 | 1 | 24 | 3640 | 860 | Xception | Frame labeling of all datasets (normal vs. protruding lesions: polyps, epithelial tumors, subepithelial lesions) |
Train–test split (80–20%) |
Protruding lesions | 91 | 93 | 97 | |
| Ribeiro, 2022 [51] |
Detection of ulcerative lesions | COLON2 | 2 | 124 | 37,319 | 3570 | Xception | Frame labeling of all datasets (normal vs. ulcer or erosions) |
train–validation (for hyperparameter tuning)–test split (70–20–10%) | Ulcers or erosions | 97 | 100 | 100 | |
| Majtner, 2021 [52] |
Panenteric (small bowel and colon) detection of ulcerative lesions | CROHN | 1 | 38 | 77,744 | 2748 | ResNet | Frame labeling of all datasets (normal vs. ulcer or erosions) |
Train–validation–test (70–20–10%) with patient split | Ulcers or erosions | 96 | 100 | NK | |
| Ferreira, 2022 [53] |
Panenteric (small bowel and colon) detection of ulcerative lesions | CROHN | 2 | 59 | 24,675 | 5300 | Xception | Frame labeling of all datasets (normal vs. ulcer or erosions) |
Train–test split (80–20%) |
Ulcers or erosions | 98 | 99 | 100 | |
| Saraiva, 2021 [54] |
Detection of blood | COLON2 | 1 | 24 | 5825 | 2975 | Xception | Frame labeling of all datasets (normal vs. blood or hematic residues) |
Train–test split (80–0%) |
Blood or hematic residues | 100 | 93 | 100 | |
| Complex CNN for Enteric and Colonic Lesions | Ding, 2019 [55] |
Detection of abnormal findings in the small bowel without discrimination capacity | NaviCam | 77 | 1970 | 158,235 + validation set | NK | ResNet | Frame labeling of training set (inflammation, ulcer, polyps, lymphangiectasia, bleeding, vascular disease, protruding lesion, lymphatic follicular hyperplasia, diverticulum, parasite, normal) | Testing with 5000 independent CE videos | Abnormal findings | 100 | 100 | NK |
| Aoki, 2021 [56] |
Detection of multiple types of lesions in the small bowel | SB3 | 3 | NK | 66,028 + validation set | 44,684 | Combined 3 SSD + 1 ResNet | Manual annotation of all mucosa breaks, angiectasias, protruding lesions and blood contents | CNN is trained on mixed data with positive and negative frames (44,684 lesions and 21,344 normal frames) Validation on 379 full videos |
Mucosal brakes vs. other lesions | 96 | 99 | NK | |
| Angiectasias vs. other lesions | 79 | 99 | NK | |||||||||||
| Protruding lesions vs. other lesions | 100 | 95 | NK | |||||||||||
| Blood content vs. other lesions | 100 | 100 | NK | |||||||||||
| Saraiva, 2021 [57] |
Detection of multiple types of lesions in the small bowel + categorization of bleeding potential | SB3 OMON |
2 | 5793 | 53,555 | 35,545 | Xception | Frame labeling of all data (normal (N) vs. lymphangiectasias (P0L) vs. xanthomas (P0X) vs. erosions (P1E) vs. ulcers with uncertain/intermediate bleeding potential (P1U) vs. ulcers with high bleeding potential (P2U) vs. red spots (P1RS) vs. vascular lesions (angiectasias or varices) (P2V) vs. protruding lesions with uncertain/intermediate bleeding potential (P1P) vs. protruding lesions with high bleeding potential (P2P) vs. blood or hematic residues) |
Train–test split (80–20%) with 11 × 11 confusion matrix |
N vs. all | 92 | 96 | 99 | |
| P0L vs. all | 88 | 99 | 99 | |||||||||||
| P0X vs. all | 85 | 98 | 99 | |||||||||||
| P1E vs. all | 73 | 99 | 97 | |||||||||||
| P1U vs. all | 81 | 99 | 99 | |||||||||||
| P2U vs. all | 94 | 98 | 100 | |||||||||||
| P1RS vs. all | 80 | 99 | 98 | |||||||||||
| P2V vs. all | 91 | 99 | 100 | |||||||||||
| P1P vs. all | 93 | 99 | 99 | |||||||||||
| P2P vs. all | 94 | 100 | 99 | |||||||||||
| Blood vs. all | 99 | 100 | 100 | |||||||||||
| Saraiva, 2022 [58] |
Detection of pleomorphic lesions or blood in the colon | COLON2 | 2 | 124 | 9005 | pl5,930 | Xception | Frame labeling of all datasets (normal (N) vs. blood or hematic residues (B) vs. mucosal lesions (ML), including ulcers, erosions, vascular lesions (red spots, angiectasia and varices) and protruding lesions (polyps, epithelial tumors, submucosal tumors and nodes)) |
Train–test split (80–20%) with 3 × 3 confusion matrix |
N vs. all | 97 | 96 | 100 | |
| Blood vs. all | 100 | 100 | 100 | |||||||||||
| ML vs. all | 92 | 99 | 90 | |||||||||||
| Xie, 2022 [59] |
Detection of multiple types of lesions in the small bowel + differentiation from normal mucosa | OMON | 51 | 5825 | 757,770 | NK | EfficientNet + Yolo | Frame labeling of all datasets Protruding lesions (venous structure, nodule, mass/tumor, polyp(s)), flat lesions (angiectasia, plaque (red), plaque (white), red spot, abnormal villi), mucosa (lymphangiectasia, erythematous, edematous), excavated lesion (erosion, ulcer, aphtha) and content (blood, parasite) |
CNN is trained on mixed data with positive and negative frames Validation on 2898 full videos |
Venous structure vs. all | 98 | 100 | NK | |
| Nodule vs. all | 97 | 100 | NK | |||||||||||
| Mass or tumor vs. all | 95 | 100 | NK | |||||||||||
| Polyp vs. all | 95 | 100 | NK | |||||||||||
| Angiectasia vs. all | 96 | 100 | NK | |||||||||||
| Plaque (red) vs. all | 94 | 100 | NK | |||||||||||
| Plaque (white) vs. all | 95 | 100 | NK | |||||||||||
| Red spot vs. all | 96 | 100 | NK | |||||||||||
| Abnormal villi vs. all | 95 | 100 | NK | |||||||||||
| Lymphangiectasia vs. all | 98 | 100 | NK | |||||||||||
| Erythematous mucosa vs. all | 95 | 100 | NK | |||||||||||
| Edematous mucosa vs. all | NK | NK | NK | |||||||||||
| Erosion vs. all | NK | NK | NK | |||||||||||
| Ulcer vs. all | NK | NK | NK | |||||||||||
| Aphtha vs. all | NK | NK | NK | |||||||||||
| Blood vs. all | NK | NK | NK | |||||||||||
| Parasite vs. all | NK | NK | NK | |||||||||||
| Complex CNN for Gastric Lesions | Xia, 2021 [60] |
Detection of multiple types of lesions + differentiation from normal mucosa | NaviCam MCE | 1 | 797 | 1,023,955 | NK | ResNet | Frame labeling of training set (erosions, polyps, ulcers, submucosal tumors, xanthomas, normal) | testing with 100 independent CE videos | Pleomorphic lesions | 96 | 76 | 84 |
| Pan, 2022 [61] |
Detect in real time of both gastric anatomic landmarks and different types of lesions | NaviCam MCE |
1 | 906 | 34,062 + validation set | NK | ResNet | Frame labeling of all datasets (ulcerative (ulcer and erosions), protruding lesions (polyps and submucosal tumors), xanthomas, normal mucosa) | Prospective validation on 50 CE exams | Gastric lesions | 99 | NK | NK | |
| Anatomic landmarks | 94 | NK | NK | |||||||||||
| Saraiva, 2023 [62] |
Detection of pleomorphic gastric lesions | SB3 CROHN OMON |
2 | 107 | 12,918 | 6074 | Xception | Frame labeling of all datasets (normal vs. pleomorphic lesion (vascular, ulcerative or protruding lesion or blood/hematic residues)) |
Train–test split (80–20%) with patient split design and 3-fold cross-validation during training set |
Pleomorphic lesions (mean of cross-validation) |
88 | 92 | 96 | |
| Pleomorphic lesions (test set) |
97 | 96 | 100 | |||||||||||