Table 1.
Comparison of studies regarding detection of MSI/dMMR from histology slides using deep learning.
| Author/Year | Organ (% by Training Cohort) |
Neural Network | Training Cohort | Type of Internal Validation | External Validation | Test Cohort(s) with AUC (95% CI) or Accuracy | Methodology for MSI Analysis |
|---|---|---|---|---|---|---|---|
| Zhang et al., * 2018 [39] |
Colorectum (51.3%) UCEC (48.7%) Stomach † |
Inception-V3 without adversarial training | TCGA CRC (n = 585 WSIs) | Random split | Yes | TCGA CRC Accuracy: 98.3% TCGA UCEC Accuracy: 53.7% |
- |
| TCGA CRC (n = 585 WSIs) TCGA UCEC (n = 556 WSIs) |
TCGA CRC Accuracy: 72.3% TCGA UCEC Accuracy: 84.2% TCGA STAD (n = 209 WSIs) Accuracy: 34.9% |
||||||
| Inception-V3 with adversarial training |
TCGA CRC Accuracy: 85.0% TCGA UCEC Accuracy: 94.6% TCGA STAD (n = 209 WSIs) Accuracy: 57.4% |
||||||
| Klaiman et al., * 2019 [40] | Colorectum | N/A | Roche internal CRC80 dataset (n = 94 pts) | Random split | No | Roche internal CRC80 dataset: 0.9 | - |
| Kater et al., 2019 [41] |
Stomach (19.2%) |
ResNet-18 | TCGA STAD FFPE (n = 216 pts) |
Random split | Yes | TCGA STAD FFPE (n = 99 pts): 0.81 (0.69–0.90) DACHS FFPE (n = 378 pts): 0.60 (0.48–0.69) KCCH FFPE (n = 185 pts): 0.69 (0.52–0.82) |
TCGA: PCR DACHS: PCR 1 KCCH: IHC |
| Colorectum (23.1%) |
TCGA CRC FFPE (n = 260 pts) |
TCGA CRC FFPE (n = 100 pts): 0.77 (0.62–0.87) DACHS FFPE (n = 378 pts): 0.84 (0.720–0.92) |
|||||
| Colorectum (23.8%) |
TCGA CRC Frozen (n = 269 pts) |
TCGA CRC Frozen (n = 109 pts): 0.84 (0.73–0.91) DACHS FFPE (n = 378pts): 0.61 (0.50–0.73) |
|||||
| UCEC (33.9%) |
TCGA UCEC FFPE (n = 382 pts) |
No | TCGA UCEC FFPE (n = 110 pts): 0.75 (0.63–0.83) | ||||
| Pressman et al., * 2020 [42] | Colorectum | ResNet18 | TCGA (n = 360 WSIs) | - | Yes | TCGA: 0.79 Gangnam sev (n = 170 WSIs): 0.76 |
- |
| Schmauch et al., 2020 [43] | Colorectum (62.8%) |
HE2RNA with ResNet50 | TCGA CRC FFPE (n = 465 pts) |
Three-fold cross validation | No | TCGA CRC FFPE: 0.82 | PCR |
| Stomach (37.2%) |
TCGA STAD FFPE (n = 276 pts) |
TCGA STAD FFPE: 0.76 | |||||
| Kather et al., 2020 [44] | Colorectum | ShuffleNet | TCGA CRC FFPE (n = 426 pts) |
Three-fold cross-validation | Yes | DACHS FFPE (n = 379 pts): 0.89 (0.88–0.92) | TCGA: PCR DACHS: PCR 1 |
| Cao et al., 2020 [45] |
Colon | ResNet-18 | TCGA-COAD Frozen (Total number including test cohort: 429 WSIs) |
Random split | Yes | TCGA-COAD: 0.8848 (0.8185–0.9512) Asian-CRC FFPE (n = 785 WSIs): 0.6497 (0.6061–0.6933) |
TCGA-COAD: NGS 2
Asian-CRC; PCR |
| TCGA-COAD Frozen (90%) + Asian-CRC FFPE (10%) |
- | No | Asian-CRC FFPE (n = 785 WSIs): 0.8504 (0.7591–0.9323) | ||||
| TCGA-COAD Frozen (30%) + Asian-CRC FFPE (70%) |
- | No | Asian-CRC FFPE (n = 785 WSIs): 0.9264 (0.8806–0.9722) | ||||
| Echle et al., 2020 [46] |
Colorectum | ShuffleNet | MSIDETECT CRC (n = 6406 pts) |
Random split | Yes | MSIDETECT CRC: 0.92 (0.90–0.93) | DACHS: PCR TCGA: PCR QUASAR and NLCS: IHC 3 YCR-BCIP: IHC |
| Three-fold cross validation | MSIDETECT CRC: 0.92 (0.91–0.93) YCR-BCIP-RESECT (n = 771 pts): 0.96 (0.93–0.98) YCR-BCIP-BIOPSY (n = 1531 pts): 0.78 (0.75–0.81) |
||||||
| YCR-BCIP-BIOPSY (n = 1531 pts) |
Three-fold cross validation | No | YCR-BCIP-BIOPSY: 0.89 (0.88–0.91) | ||||
| Valieris et al., 2020 [47] | Stomach | Resnet-34 | TCGA-STAD FFPE (Total number including test cohort: 369 pts) |
Random split | No | TCGA-STAD FFPE: 0.81 (0.689–0.928) | NGS 4 |
| Yamashita et al., 2021 [48] | Colorectum | MSInet | Stanford dataset (n = 85 pts) | Random split | No | Stanford dataset (n = 15 pts): 0.931 (0.771–1.000) | Stanford dataset: IHC/PCR TCGA: PCR |
| Four-fold cross-validation |
Yes | Stanford dataset (n = 15 pts): 0.937 TCGA (n = 479 pts): 0.779 (0.720–0.838) |
|||||
| Krause et al., 2021 [49] | Colorectum | ShuffleNet | TCGA FFPE (n = 256 pts) | Random split | No | TCGA FFPE (n = 142 pts): 0.742 (0.681–0.854) | PCR |
| Lee et al., 2021 [50] |
Colorectum | Inception-V3 | TCGA FFPE (n = 470,825 patches) SMH FFPE (n = 274 WSIs) |
10-fold cross validation | No | TCGA FFPE: 0.892 (0.855–0.929) SMH FFPE: 0.972 (0.956–0.987) |
TCGA: PCR SMH: PCR/IHC |
| TCGA FFPE (n = 470,825 patches) |
Yes | TCGA FFPE: 0.861 (0.819–0.903) SMH FFPE: 0.787 (0.743–0.830) |
|||||
| TCGA Frozen (n = 562,837 patches) |
No | TCGA Frozen: 0.942 (0.925–0.959) | |||||
| Hong et al., 2021 [51] |
UCEC | InceptionResNetV1 | TCGA and CPTAC (Total number including test cohort: 456 pts) |
Random split | Yes | TCGA and CPTAC: 0.827 (0.705–0.948) NYU set: 0.667 |
TCGA: PCR CPTAC: NGS 5 |
AUC, Area Under the Curve; UCEC, Uterine Corpus Endometrial Carcinoma; TCGA, The Cancer Genome Atlas study; CRC, ColoRectal Cancer; WSI, Whole Slide Images; STAD, STomach ADenocarcinoma; pts, patients; FFPE, Formalin-Fixed Paraffin-Embedded; DACHS, Darmkrebs: Chancen der Verhütung durch Screening (CRC prevention through screening study abbreviation in German); KCCH, Kangawa Cancer Center Hospital (Japan); Stanford dataset, Stanford University Medical Center (USA) Gangnam sev, Gangnam Severance Hospital (South Korea); COAD, COlonic ADenocarcinoma; MSIDETECT: A consortium composed of TCGA, DACHS, the United Kingdom-based Quick and Simple and Reliable trial (QUASAR), and the Netherlands Cohort Study (NLCS); YCR-BCIP: Yorkshire Cancer Research Bowel Center Improvement Programme; SMH, Saint Mary’s Hospital (South Korea); CPTAC, Clinical Proteomic Tumor Analysis Consortium; NYU, New York University hospital. * Conference paper or abstract not officially published. † The stomach cancer cohort was used only in test cohort. 1 3-plex PCR (BAT25, BAT26, CAT25) 2 MSI sensor algorithm 3 2-plex IHC 4 Mutation signature 5 Mutation load, MMR gene mutation status, MSI sensor score, MSMuTect score, and MLH1methylation.