MADSurv: An Uncertainty-Aware Framework for Multimodal Cancer Survival Analysis

Abstract

Multimodal learning significantly improves cancer survival prediction using multiple data types for each patient, including clinical data, pathological images, and genomic data. However, existing works often assume that each data type contributes equally, neglecting potential conflicting or unreliable information between different modalities by simply concatenating features to form a fused representation. In addition, current frameworks are trained only to rank relative risk among patients and cannot quantify the actual likelihood of survival at specific time points, limiting their utility in real-world clinical settings. In this work, we propose the Modality-Aware Discrete-Time Survival (MADSurv) framework to address these gaps. First, it implements an uncertainty-aware attention mechanism in which modality-specific expert encoders learn both predictive features and their own confidence for intelligent data fusion. This approach leads to more robust and personalized predictions by dynamically focusing on the modalities that are most reliable for each individual patient. Second, in addition to a single overall risk score, MADSurv produces a sequence of survival probabilities for discrete yearly intervals. We evaluated our proposed method on five different cancer datasets. In addition to ranking patients’ overall risk using the concordance index, we also assessed the accuracy of the model’s survival probability estimates on each yearly milestone using the Brier score. Our experimental results demonstrate that MADSurv achieves superior and competitive performance compared to state-of-the-art methods.

1. Introduction

Cancer survival analysis focuses on estimating the risk of mortality related to cancer, providing valuable information for patient prognosis and treatment planning [7, 20]. With the increasing availability of biomedical data, recent research has adopted multimodal approaches that integrate various sources, such as clinical records, histopathological images, and molecular profiles, to improve predictive accuracy. These different modalities offer complementary perspectives that can improve patient risk stratification and support more personalized treatment strategies.

Current approaches to multimodal cancer survival analysis have two main limitations. First, they often use simple fusion strategies to combine data from different modalities. In early fusion, features from all modalities are concatenated at the input level and fed into a shared predictive model. This approach assumes that all modalities are equally important and complement each other. In contrast, late fusion processes each modality separately to produce intermediate predictions, which are then combined through averaging or another aggregation function to generate the final output. Although these strategies can improve predictive performance compared to unimodal models, they do not consider the unique contributions or reliability of each modality. This limitation becomes particularly apparent when trying to identify which modality contributes the most to the predictions or how to resolve conflicting information from different modalities.

Second, many existing methods focus primarily on optimizing risk ranking, which involves training models to assess whether patient A has a higher risk of death compared to patient B. The risk scores generated by these models are inherently relative; they indicate whether a patient is at a higher or lower risk than others, but they do not provide an actual quantification of survival likelihood. This emphasis on ranking neglects an essential aspect of real-world clinical decision-making. Clinicians are interested in not only relative risk, they also want to understand how a patient’s survival probability changes over time. For instance, accurately estimating 1-year or 5-year survival probabilities can significantly influence follow-up intensity, treatment aggressiveness, and end-of-life planning, with the 5-year survival probability serving as a particularly important clinical benchmark [24].

In this work, we propose a novel framework for cancer survival prediction called Modality-Aware Discrete-Time Survival (MADSurv). MADSurv uses various data sources, including clinical features, genomic pathways, and pathological images; first learning survival predictions specific to each modality and their associated uncertainties. These modality-specific outputs are then dynamically combined using an attention mechanism guided by uncertainty, allowing the model to prioritize more reliable modalities for each patient. Furthermore, rather than just modeling relative risk rankings, MADSurv offers hazard probabilities for discrete time intervals, each lasting 12 months. This method enables the estimation of survival probabilities at each yearly milestone, providing more actionable insights for prognosis and informed decision-making. The code and all preprocessed data have been released.¹

2. Related Work

2.1. Unimodal Survival Analysis

Recent advances in biomedical data acquisition technologies have led to the development of survival analysis models using unimodal input, including whole slide images (WSI), genomic profiles, and clinical records. WSIs are high-resolution digitized pathology slides derived from patient biopsies. These large images are typically divided into smaller patches, which are then analyzed using deep learning models. A widely used strategy is multiple instance learning (MIL), where each WSI is treated as a “bag” of instances (i.e., patches), and the final prediction is obtained by aggregating instancelevel predictions [1, 9]. Recent approaches extend this idea by incorporating spatial and contextual relationships between patches using graph-based models [14]. Genomic data captures high-dimensional molecular information about patients, such as gene expression levels and somatic mutations. Predictive models based on this modality often employ feedforward neural architectures, including multilayer perceptrons (MLPs) [16] and self-normalizing networks (SNNs) [26], to learn effective representations. Clinical indicators–including demographic information, diagnostic test results, and tumor characteristics–have also been used as the sole modality in survival prediction tasks. Prior studies have applied clinical-only models to predict survival outcomes for diseases such as breast cancer [21], non-small cell lung cancer [27], chronic liver failure [39], and chronic kidney disease [15].

2.2. Multimodal Survival Analysis

In clinical practice, patients are commonly evaluated using multimodal data, including clinical records, genomic profiles from biological samples, pathology slides, radiological images, and more. These data sources collectively support diagnosis and prognosis efforts [34]. Multimodal approaches for survival analysis can be broadly categorized into tensor-based and attention-based fusion methods. Tensor-based methods typically construct joint representations by applying operations such as concatenation, weighted summation, Kronecker products, or bilinear pooling [3, 4, 17, 30, 35, 36, 41]. More recently, attention-based fusion methods have gained popularity for their ability to model cross-modal interactions. For example, MCAT learns a dense co-attention map between whole slide images (WSIs) and genomic features in a shared embedding space [5]. MOTCat introduces a gene-guided co-attention mechanism to align WSI patches with gene embeddings, effectively selecting informative regions from gigapixel WSIs [38]. Similarly, a modality-attention network was proposed in [31] to integrate pathological images and genomic data. Other work explores cross-modal alignment via early fusion [42]. SurvPath models dense interactions between pathways and histologic patches using cross-attention mechanisms [19].

Most attention-based fusion mechanisms aim to find a consensus among different modalities, but do not explicitly account for the reliability or uncertainty of each data source. This can be problematic when one modality is noisy or provides conflicting signals for a specific patient, as the model may struggle to dynamically reduce its influence. Furthermore, most existing frameworks focus on risk discrimination, producing relative risk scores instead of calibrated survival probabilities. This limits the ability of clinicians to evaluate the prognosis of a patient at specific time points, such as the 5-year survival probability.

3. Methodology

3.1. Problem Formulation

In each dataset, a patient $i$ is represented as a tuple $({\mathbf{x}}_i,t_i,c_i)$. Here, ${\mathbf{x}}_i$ represents the multimodal data of the patient, $t_i$ (in months) is the disease-specific survival time, and $c_i$ is the censorship status. For patients who died from cancer, $t_i\in {\mathbb{R}}^{+}$ is the time from diagnosis to the event of death. For censored patients, $t_i$ is the time from diagnosis to the last follow-up or death from unrelated causes. The censorship indicator $c_i\in \left\{\mathrm{0,1}\right\}$ specifies whether the event of interest, cancer-specific death, was observed $(c_i=1)$ or whether the patient was censored $(c_i=0)$. Our objective is to train a survival prediction model that takes a patient’s multimodal data as input and produces a sequence of hazard probabilities over $K$ discrete time intervals. These hazard predictions can then be used to construct a survival curve that estimates the patient’s personalized probability of survival at each yearly milestone. The general framework is illustrated in Figure 1.

Figure 1:

Architecture of the proposed Modality-Aware Discrete-Time Survival (MADSurv) framework

3.2. Data Preprocessing and Encoding

In this work, we use three types of patient-level data: clinical, genomic, and pathological.

Clinical Data.

We selected clinical features that had at least 90% of the values not missing throughout the cohort. The final feature set includes age, weight, tumor grade, treatment (the first planned after initial diagnosis), aneuploidy score, microsatellite instability score (MANTIS), tumor mutation burden (non-synonymous), tumor necrosis percentage, and tumor nuclei percentage. Continuous features were standardized to have a mean of zero and a variance of one. Categorical variables were encoded as follows: tumor grade was treated as an ordinal variable, while race was one-hot encoded to reflect its non-ordinal nature. Treatment information was represented using binary indicators for the first proposed treatment type. The resulting clinical feature vector ${\mathbf{x}}_i^{\mathbf{c}}\in {\mathbb{R}}^{13}$ with a dimensionality of 13.

Genomic Data.

Following the approach outlined by [19], we use genomic data at the biological pathway level. A pathway consists of a group of genes or proteins that interact to perform specific biological functions or processes. This strategy allows us to avoid using raw gene-level expression data, which typically includes hundreds of thousands of genes, and instead focus on semantically meaningful gene groups. These pathway-level features not only provide a stronger biological connection with pathological features, but they also significantly reduce dimensionality and improve interpretability.

We first obtained RNA-seq gene expression data for each patient from the Xena database [13]). We selected biological pathways from two sources: Reactome [12], which contains 1,281 human pathways, and the Hallmark collection from the Molecular Signatures Data-base (MSigDB) [28], which includes 50 curated pathways. After filtering for completeness, we retained only those pathways for which at least 90% of the associated genes had non-missing expression values across the cohort. This process resulted in a final set of 331 pathways comprising 4,999 unique genes. For each patient, we calculated the average expression value of genes within each pathway, which results in a 331-dimensional pathway-level genomic feature vector ${\mathbf{\text{x}}}_i^{\text{g}}\in {\mathbb{R}}^{331}$.

Pathological Image Features.

Instead of directly using WSI image patches, we use quantitative features extracted through a CNN-based computational pathology pipeline developed by [32]. This method generates eight features for each WSI, which are grouped into three meaningful categories. The first category, Tissue and Cellular Composition, includes the proportion of cancerous cells, non-cancerous cells, necrotic tissue area, and supportive tissue area. The second category, Immune Infiltrate, measures the abundance of lymphocytes, monocytes, and neutrophils. Finally, the Quantitative Morphometry category is represented by the average area of tumor cell nuclei. These biologically meaningful features provide an efficient and interpretable summary of the pathology slide, allowing for integration with other modalities in survival prediction tasks. Before being inputted into the model, all eight values are normalized to the range [0, 1], resulting in a pathological feature vector ${\mathbf{\text{x}}}_i^{\text{p}}\in {\mathbb{R}}^8$.

After preprocessing, each patient’s multimodal input is represented as ${\mathbf{\text{x}}}_i=\left\{{\mathbf{\text{x}}}_i^{\text{c}},{\mathbf{\text{x}}}_i^{\text{g}},{\mathbf{\text{x}}}_i^{\text{p}}\right\}$, where ${\mathbf{\text{x}}}_i^{\text{c}}\in {\mathbb{R}}^{13}$ denotes the clinical features, ${\mathbf{\text{x}}}_i^{\text{g}}\in {\mathbb{R}}^{331}$ the genomic pathway features, and ${\mathbf{\text{x}}}_i^{\text{p}}\in {\mathbb{R}}^8$ the pathology-derived visual features. Each modality is independently encoded to preserve its specific structure and to enable modality-specific survival prediction.

The genomic data ${\mathbf{x}}_i^g$ is treated as a sequence of 331 biological pathways. Each scalar pathway feature is embedded into a 256-dimensional token vector via a learnable linear projection, resulting in a sequence ${\mathbf{\text{Z}}}_i^g\in {\mathbb{R}}^{331\times 256}$. These tokens are processed using a lightweight Transformer encoder to model pathway-level interactions. The encoded token sequence is aggregated into a fixed-size vector ${\mathbf{z}}_i^g\in {\mathbb{R}}^{256}$ using attentive pooling.

For the clinical and pathological modalities, the raw features ${\mathbf{x}}_i^c$ and ${\mathbf{x}}_i^{\text{p}}$ are passed through separate multilayer perceptrons (MLPs) with hidden layers and non-linear activations to obtain fixed-length embeddings ${\mathbf{z}}_i^{\mathbf{c}}\in {\mathbb{R}}^{256}$ and ${\mathbf{z}}_i^{\text{p}}\in {\mathbb{R}}^{256}$, respectively.

3.3. Uncertainty-Aware Fusion Mechanism

The foundation of our MADSurv framework is a fusion mechanism that dynamically adjusts the importance of each modality based on its learned features and estimated uncertainty. This process consists of three steps: estimating uncertainty, using an attention gate to generate weights, and performing a final weighted fusion of the feature vectors.

Uncertainty Estimation.

For each modality, we estimate the model’s confidence in its feature representation using Monte Carlo (MC) Dropout [10]. This technique requires that dropout layers, included within each modality-specific encoder, remain active during inference. We then perform $T$ stochastic forward passes for each patient. The variance across the resulting $T$ feature vectors for a modality $m$ serves as its uncertainty score, $u_i^m$. A low variance signifies high confidence.

Uncertainty-Aware Attention Gate.

An attention gating network is used to generate a set of weights, ${\alpha }_i=\left[{\alpha }_i^c,{\alpha }_i^g,{\alpha }_i^p\right]$, for each patient. To do this, we first form a combined evidence vector, ${\mathbf{e}}_i$, by concatenating all feature vectors and their corresponding uncertainty scores:

$${\mathbf{e}}_i=\left[{\mathbf{z}}_i^{\text{c}};{\mathbf{z}}_i^{\text{g}};{\mathbf{z}}_i^{\text{p}};u_i^c;u_i^g;u_i^p\right]$$ (1)

This evidence vector is then passed through a gating network, $f_{\text{gate}}$ (a shallow MLP), to produce three unnormalized scores (logits). A softmax function is applied to these logits to yield the final attention weights, which sum to one:

$${\alpha }_i=\text{softmax}\left(f_{\text{gate}}\left({\mathbf{e}}_i\right)\right)$$ (2)

This mechanism learns to assign higher weights to modalities that are relevant (based on features) and reliable (based on low uncertainty).

Weighted Feature Fusion.

The final fused feature vector for patient $i,{\mathbf{z}}_i^{\text{fused}}$, is computed as a weighted average of the modality-specific feature vectors, using the attention weights generated by the gate:

$${\mathbf{z}}_i^{\text{fused}}=\sum _{m\in \{c,g,p\}}{\alpha }_i^m\cdot {\mathbf{z}}_i^m$$ (3)

This fused vector provides a robust and context-aware representation of the patient.

3.4. Survival Prediction

The fused feature vector ${\mathbf{z}}_i^{\text{fused}}$ is passed to a final Prediction Head—a two-layer MLP with a sigmoid output layer of size $K$—to generate the definitive hazard probability vector ${\stackrel{\mathbf{ˆ}}{\mathbf{h}}}_i=\left[{\hat{h}}_{i,1},\dots ,{\hat{h}}_{i,K}\right]$. Given this fused hazard vector, we construct the discrete-time survival probability for patient $i$ through interval $j$ as:

$$S_{i,j}=\prod _{k=1}^j\left(1-{\hat{h}}_{i,k}\right)$$ (4)

The resulting sequence ${\left\{S_{i,j}\right\}}_{j=1}^K$ defines a personalized, stepwise survival curve.

3.5. Training Objective

The model is trained end-to-end by minimizing the negative loglikelihood loss (NLL) [40] of the observed survival data. Let $j_i$ be the index of the interval containing the observed time $t_i$ for patient $i$, and $c_i\in \left\{\mathrm{0,1}\right\}$ be the event indicator. The individual likelihood ${ℒ}_i$ is given by:

$${ℒ}_i=\left\{\begin{array}{ll}{S}_{i,j_i-1}\cdot {\hat{h}}_{i,j_i}& \text{if}{c}_i=1\\ S_{i,j_i}& \text{if}{c}_i=0\end{array}\right.$$ (5)

The overall loss is the NLL summed across all $N$ patients. By minimizing this loss, all components of the network, including the encoders and the attention gate, are optimized simultaneously.

$$ℒ=-\sum _{i=1}^N\text{log}{ℒ}_i$$ (6)

4. Experimentation

4.1. Dataset and Implementation Details

We conducted experiments on five publicly available datasets from The Cancer Genome Atlas (TCGA) project [37], accessed via cBioPortal² [2, 11]. The datasets include Bladder Urothelial Carcinoma (BLCA, $n=411)$, Breast Invasive Carcinoma (BRCA, $n=1084$), Head and Neck Squamous Cell Carcinoma (HNSC, $n=523$), Lung Adenocarcinoma (LUAD, $n=566$), and Uterine Corpus Endometrial Carcinoma (UCEC, $n=529$), where $n$ denotes the number of patients in each cohort.

Clinical variables and quantitative features obtained from pathological WSIs were downloaded from cBioPortal. These data were then preprocessed and cleaned to eliminate incomplete records and standardize feature formats. Genomic pathway features for BLCA, BRCA, HNSC, LUAD, and UCEC were acquired using the method described in Section 3.2, which was originally proposed by [19].

While prior studies have predominantly used overall survival (OS) [4, 27, 31, 43]—which includes death from any cause—this can obscure cancer-specific risk, as non-cancer-related deaths may be included. To mitigate this issue, we focus on disease-specific survival (DSS), which considers only deaths directly attributed to the primary cancer, providing a more clinically meaningful prognosis assessment.

Each dataset was evaluated using 5-fold cross-validation, and the reported results reflect the average performance across the five folds. Our framework was implemented in PyTorch and trained on Google Colab Pro using an NVIDIA A100 GPU (driver version 550.90.12, CUDA 12.4), with 83.5 GB RAM and 40 GB GPU memory. We trained each model for 50 epochs using a batch size of 8. The AdamW optimizer [25] was used with an initial learning rate of 1 × 10⁻³ and a weight decay of 1 × 10⁻⁴.

4.2. Evaluation Metrics

We use two metrics to evaluate MADSurv, Concordance Index (C-index) and Brier score. The C-index measures the model’s ability to correctly rank patients by their predicted risk scores. It is defined as the proportion of comparable patient pairs whose predicted survival outcomes are ordered consistently with their observed event times. A pair of patients $(i,\ell )$ is considered comparable if one experienced the event (death) earlier before the other’s event or censoring time.

For each patient $i$, we compute their risk score as the predicted probability of surviving through the end of their observed interval $j_i$, given by the discrete-time survival function defined in Equation 4. In this work, C-index is reported on the dataset level, calculated as:

$$\text{C-index}=\frac{1}{\left|𝒫\right|}\sum _{(i,\ell )\in 𝒫}\mathbb{I}\left[S_{i,j_i}<S_{\ell ,j_{\ell }}\right]$$ (7)

Here, $𝒫$ denotes the set of all comparable patient pairs in the dataset, and $\mathbb{I}[\cdot ]$ is the indicator function, which returns 1 if the condition inside is true and 0 otherwise. A higher C-index indicates stronger discriminatory power in identifying higher-risk patients.

The Brier score (BS) evaluates the accuracy of the predicted survival probabilities. To provide an overall measure of performance, we use the Integrated Brier Score (IBS), which averages the time-dependent Brier score over the entire follow-up period. The time-dependent Brier score at time $t$ is defined as the mean squared error between the predicted survival probability and the actual patient status, with weighting to account for censoring:

$$\text{BS}(t)=\frac{1}{N}\sum _{i=1}^N{w}_i(t){\left(\hat{S}\left(t\mid {\mathbf{x}}_i\right)-I\left(t_i>t\right)\right)}^2$$ (8)

In this equation, $\hat{S}\left(t\mid {\mathbf{x}}_i\right)$ is the model’s predicted survival probability for patient $i$ at time $t,I\left(t_i>t\right)$ is the indicator of whether the patient actually survived past time $t$, and $w_i\left(t\right)$ is a weight that adjusts for censoring to provide an unbiased estimate. A lower Brier score indicates more accurate and better-calibrated predictions.

Additionally, we use Kaplan-Meier [22] (KM) survival curves and the Log-rank test [29] to visualize and statistically assess the model’s ability to stratify patients into distinct low-risk and high-risk groups.

4.3. Experimental Results

We compare our approach against two groups of existing methods: (1) unimodal baselines that use only pathological image.genomic data, or clinical data and (2) state-of-the-art multimodal approaches that integrate multiple data modalities. For unimodal approaches, we use MLP [16], SNN [26], and SNNTrans [26, 33] for genomic baselines; CoxPH [8], RSF [18], DeepSurv [23] for pathological baselines; and MLP, CoxPH, and RSF for clinical baselines. For mutlimodal methods, we compare our model with MCAT [5]. SURPATH [19], Porpoise [6], CMTA [42], and LD-CVAE [43].

From the results in Table 1, a clear trend emerges where multimodal methods consistently outperform unimodal baselines across all five datasets. This highlights the importance of integrating diverse data sources for a comprehensive prognostic signal. For instance, in the BRCA cohort, the best-performing unimodal model (SNNTrans) achieves a C-index of 0.604, while nearly all multimodal approaches, including our own, surpass this benchmark.

Table 1:

Performance comparison of different models using C-index (the higher the better) and Brier Score (the lower the better) across five cancer datasets. The best result in each column is in bold, and the second-best is underlined. g., p., and c. refer to genomic pathways, pathological images, and clinical data, respectively.

		BLCA (N=411)		BRCA (N=1084)		HNSC (N=523)		LUAD (N=566)		UCEC (N=529)
Model	Modality	C-index	Brier	C-index	Brier	C-index	Brier	C-index	Brier	C-index	Brier
MLP	g.	0.605	0.284	0.597	0.271	0.565	0.298	0.582	0.290	0.599	0.265
SNN	g.	0.582	0.292	0.531	0.285	0.550	0.301	0.571	0.293	0.588	0.272
SNNTrans	g.	0.579	0.289	0.604	0.297	0.591	0.295	0.575	0.289	0.605	0.262
CoxPH	p.	0.558	0.302	0.563	0.291	0.542	0.314	0.550	0.300	0.576	0.281
RSF	p.	0.572	0.295	0.577	0.285	0.555	0.305	0.565	0.294	0.583	0.274
DeepSurv	p.	0.590	0.288	0.584	0.282	0.570	0.299	0.578	0.289	0.595	0.270
MLP	c.	0.598	0.283	0.602	0.270	0.562	0.300	0.580	0.288	0.607	0.264
CoxPH	c.	0.576	0.290	0.581	0.279	0.545	0.310	0.565	0.293	0.584	0.273
RSF	c.	0.589	0.286	0.593	0.275	0.556	0.303	0.573	0.290	0.592	0.268
MCAT	g.+p.	0.619	0.280	0.611	0.269	0.584	0.292	0.625	0.282	0.620	0.260
SURVPATH	g.+p.	0.630	0.277	0.648	0.263	0.609	0.287	0.618	0.281	0.699	0.258
Porpoise	g.+p.	0.619	0.281	0.599	0.272	0.562	0.302	0.590	0.291	0.628	0.259
CMTA	g.+p.	0.672	0.270	0.639	0.260	0.618	0.286	0.674	0.277	0.695	0.261
LD-CVAE	g.+p.	0.662	0.273	0.670	0.258	0.601	0.288	0.659	0.279	0.684	0.253
MADSurv (ours)	g.+p.+c.	0.683	0.268	0.652	0.257	0.615	0.289	0.666	0.271	0.710	0.253

When compared to other state-of-the-art multimodal approaches, MADSurv demonstrates highly competitive or superior performance. Notably, our model achieves the highest C-index in three of the five datasets (BLCA, LUAD, and UCEC) and the best (lowest) Brier score in two datasets (BRCA and UCEC, tied with LD-CVAE). This strong performance is particularly evident as MADSurv is the only model in this comparison that integrates all three data modalities (genomics, pathology, and clinical). This underscores the effectiveness of our uncertainty-guided fusion mechanism in leveraging all available information to create a more accurate and robust prognostic model.

In Figure 2, we evaluate our model using statistical analysis and visualize the Kaplan-Meier curve to display the predicted survival probabilities for different risk groups. Each patient’s predicted risk is calculated based on the most recent interval, K. The high and low-risk groups are classified according to the median risk value of all patients within each cancer dataset. The separation between the two groups demonstrates the strong discriminative power of our proposed method.

Figure 2:

Kaplan-Meier curves for predicted high-risk (red) and low-risk (green) groups. These groups are determined based on the median predicted risks during the last time interval for the patients. A P-value of less than 0.05 indicates statistical significance, and the shaded regions represent the confidence intervals.

4.4. Ablation Study

We conducted a two-part ablation study. First, we assessed the contribution of each data modality by training model variants using all possible pairwise combinations. Second, we evaluated our uncertainty-aware attention mechanism in comparison to several conventional fusion strategies. The results are summarized in Table 2.

Table 2:

Ablation study of the MADSurv framework, assessing different modality combinations and different fusion techiniques. C-index and Brier Score are reported across five cancer datasets.

		BLCA		BRCA		HNSC		LUAD		UCEC
Study	Variant	C-index	Brier	C-index	Brier	C-index	Brier	C-index	Brier	C-index	Brier
Modality Ablation	g.+p.	0.661	0.274	0.640	0.261	0.605	0.293	0.651	0.276	0.690	0.268
	g.+c.	0.675	0.272	0.645	0.259	0.608	0.291	0.659	0.274	0.698	0.266
	p.+c.	0.655	0.279	0.621	0.267	0.599	0.298	0.630	0.282	0.672	0.273
Fusion Strategy	Early - Concat	0.608	0.331	0.595	0.319	0.561	0.314	0.602	0.305	0.619	0.304
	Early - Bilinear Pooling	0.640	0.283	0.614	0.282	0.577	0.312	0.625	0.295	0.622	0.297
	Late - Mean Avg (w/o uncertainty-aware and attention)	0.629	0.298	0.601	0.288	0.569	0.306	0.610	0.311	0.637	0.292
	Late - Regular Attention (w/o uncertainty-score)	0.635	0.295	0.611	0.284	0.574	0.313	0.618	0.304	0.625	0.319
	Late - Uncertainty-Weighted Average (w/o attention)	0.673	0.272	0.647	0.261	0.610	0.291	0.659	0.274	0.679	0.275

In the modality analysis, we discovered that no combination of two modalities could match the performance of the full trimodal model, underscoring the complementary value of genomic, pathological, and clinical data. Among the combinations we tested, the pairing of genomic and clinical features (g. + c.) consistently showed the best performance. Conversely, the combination of pathological and clinical features (p. + c.) had the lowest performance. This suggests there is a particularly strong synergy between genomic and clinical information. The relatively poor performance of the p. + c. combination may be partly attributed to the limited number of features extracted from pathology images compared to the other modalities.

For the fusion ablation study, we replaced our uncertainty-guided fusion approach with alternative strategies to assess their contributions. In the Early - Concat setting, we concatenated the embeddings from the three modalities and input them into a single prediction head. The Early - Bilinear Pooling variant combined modality embeddings by modeling their multiplicative interactions before making predictions. In the Late - Mean Average setup, the model averaged the final hazard predictions from the three modality experts without applying attention or uncertainty modeling. The Late - Regular Attention variant used a standard attention mechanism to compute weights for the modality-specific predictions, although it did not incorporate uncertainty scores. Lastly, the Late - Uncertainty-Weighted Average variant directly utilized the uncertainty scores generated by each expert to weight their predictions, completely eliminating the need for a learnable attention gate.

We can see that early fusion using simple feature concatenation after encoding performs the worst. Bilinear pooling yields better results than concatenation. In the analysis of late fusion, the best results are achieved using the uncertainty score without attention. The relatively lower performance observed when using the average hazard prediction from the three modalities highlights the significance of both uncertainty and gated attention.

4.5. Case Study: 5-Year Survival Prediction

To demonstrate the clinical utility of our framework, we conducted a case study evaluating the accuracy of MADSurv’s predictions at a key clinical milestone: the 5-year survival mark. To achieve this, we set $K$ to 5 in Equation 4 to compute the survival probabilities at the 5-year mark. We assessed the performance of these predictions using two time-dependent metrics: the Brier Score at 5 years, which measures the accuracy of the probability estimates, and the Area Under the Curve (AUC) at 5 years, which evaluates the model’s ability to discriminate between patients who survive beyond five years and those who do not. The reported results for each dataset are the averaged values from 5 independent testing sets.

The results, presented in Table 3, show that MADSurv achieves good performance for 5-year survival predictions. The best results were obtained from the uterine cancer dataset (UCEC), with an AUC of 0.79 and a Brier Score of 0.265.

Table 3:

Performance of MADSurv for 5-Year Disease-Specific Survival Prediction. Brier Score measures the accuracy of the probability predictions (lower is better), while AUC measures discrimination (higher is better).

Dataset	Brier Score at 5-year	AUC at 5-year
BLCA	0.278	0.76
BRCA	0.275	0.74
HNSC	0.301	0.72
LUAD	0.294	0.75
UCEC	0.265	0.79

4.6. Limitations

While MADSurv shows strong and consistent performance across five large TCGA cohorts, its evaluation has mainly relied on retrospective data from North American populations. Although these datasets are well-established benchmarks, it is crucial to validate the model on external cohorts—particularly those from different institutions, countries, or prospective clinical settings. This will help confirm the model’s robustness and general applicability in broader clinical contexts.

Our framework uses quantitative features extracted from pathology images instead of learning directly from raw gigapixel WSIs. This design decision enhances computational efficiency and allows for the incorporation of clinically validated biomarkers. However, it may restrict the discovery of new visual patterns beyond the predefined feature set. Expanding MADSurv to include other data types, such as raw WSIs and pathological reports, presents a promising opportunity for more comprehensive representation learning.

Lastly, while the uncertainty-aware attention mechanism helps identify which modalities the model relies on for each patient, the modality-specific expert encoders themselves remain somewhat opaque. Currently, the framework does not provide detailed explanations of the specific features—such as genomic pathways or clinical variables—that contribute most significantly to a high-risk prediction. Future work could explore the use of feature attribution techniques to enhance interpretability and support clinical decision-making more effectively.

5. Conclusions

In this study, we addressed two challenges in multimodal cancer survival analysis: integrating potentially conflicting data sources and developing models that provide clinically actionable survival probabilities rather than just relative risk rankings. We introduced MADSurv, a novel deep-learning framework that incorporates an uncertainty-aware attention mechanism. This module enables the model to dynamically weigh various data modalities by estimating the confidence of its own modality-specific experts, resulting in more robust and personalized predictions. Additionally, MADSurv is designed to provide a discrete-time survival function for each yearly milestone, offering survival probability estimates at critical clinical points. Our extensive experiments across five large-scale TCGA datasets demonstrated that MADSurv achieves superior performance in patient risk ranking, survival probability prediction, and patient risk stratification. We believe that our framework represents a significant advancement towards a more personalized and actionable cancer prognosis. In the future, we aim to extend this framework to additional external datasets and feature types such as WSI and pathological reports, facilitating AI-based end-to-end prognosis and decision-making to better support clinicians.

CCS Concepts.

• Computing methodologies → Machine learning; Supervised learning; Survival analysis.