Table 2.
Summarizes the representative multimodal fusion works that combine pathomics and genomics for better clinical prediction tasks
| Tumor type | Models | References | Data modality | Performance |
|---|---|---|---|---|
| Pan-cancer | Resnet-50; direct fusion on patch-level | Schmauch et al. [39] | RNA-seq and WSI | MSI status prediction, AUC: 0.81 (RNA + WSI) |
| PRAD | Resnet-50; autoencoder fusion in combined late space | Tan et al. [15] | RNA-seq and WSI | Malignant/benign tissue recognition, AUC: 0.74 (RNA + WSI) |
| BRCA | DenseNet-121; direct fusion on patch-level | He et al. [7] | RNA-seq and WSI | Subtype prediction, AUC: 0.83 ± 0.05 (RNA + WSI) |
| KIRC | Customized architecture; multivariate feature selection | Ning et al. [82] | RNA-seq and WSI | Survival prediction, C-index:0.79 (0.73–0.86) (RNA + WSI) |
| KIRC, GBM | VGG19; multimodal tensor fusion | Chen et al. [5] | RNA-seq, CNV, gene mutation and WSI | Survival prediction, KIRC C-index: 0.72 ± 0.031 (RNA + WSI); GBM C-index: 0.82 ± 0.010 (RNA + WSI) |
| Pan-cancer | SqueezeNet; unsupervised compression in a single feature vector | Cheerla et al. [63] | Clinical data, WSI, miRNA and mRNA | Survival prediction, C-index: 0.78 (miRNA + mRNA + WSI) |
| ER-BC | ResNet-101; canonical correction analysis | Xu et al. [83] | RNA-seq and WSI | Cancer-specific prediction, P-value 7.23e−06 (gene + WSI) |
| GBM | Customized architecture; two-stage feature aggregation | Hao et al. [84] | RNA-seq and WSI | Survival prediction, C-index: 0.70 ± 0.029 (RNA + WSI) |
| LIHC | Two-stage feature aggregation | Zhan et al. [85] | WSI and RNA-seq | Survival prediction, C-index: 0.75 (RNA + WSI) |
| KIRC, LIHC, LUAD | PCA; shared representation learning | Ning et al. [86] | WSI and RNA-seq | Survival prediction, C-index: 0.658–0.685 (WSI + RNA) |
DL deep learning, PRAD prostate cancer, BRCA breast cancer, GBM glioblastoma, ER-BC estrogen receptor-positive breast cancer, RNA-seq RNA-sequencing, CNVs copy number variations, WSI whole slide image, MSI microsatellite, AUC the area under curves, mRNA messenger-RNA, miRNA micro-RNA, ATAC-seq assay for transposase-accessible chromatin using sequencing, TCGA The Cancer Genome Atlas, TCIA The Cancer Imaging Archive, GDC Genomic Data Commons portal, C-index concordance index, PCA principal components analysis, KIRC kidney renal clear cell carcinoma, LIHC liver hepatocellular carcinoma, LUAD lung adenocarcinoma