Deep learning for prognostic stratification and biomarker exploration in upper tract urothelial carcinoma: a multicenter retrospective cohort study

Abstract

Background:

Upper tract urothelial carcinoma (UTUC) necessitates precise prognostic assessment for personalized therapy due to its aggressive nature and the limitations of traditional methods. An objective, interpretable pathological evaluation system is urgently needed to improve UTUC risk stratification and patient outcomes.

Methods:

In this multicenter retrospective study, we developed a prior knowledge-guided deep learning system using whole-slide images from 805 UTUC patients who underwent radical nephroureterectomy. The UCSegNet tile classifier was employed for eight-category tissue segmentation, and the CONCH vision-language model was used for tumor patch categorization, creating integrated probability heatmaps. Two prognostic networks, MacroContextNet and PGCA-Net, were then trained to predict the overall survival (OS) by integrating these multiscale pathological features. AI-derived quantitative pathological biomarkers were explored, including tissue fraction scores and co-localization scores. Model performance was evaluated using the concordance index (C-index), time-dependent area under the receiver operating characteristic curve (AUC), and multivariable Cox regression.

Results:

The UCSegNet tile classifier achieved excellent multiclass tissue classification accuracy across all cohorts (AUC range: 0.9916–0.9948). The primary prognostic model, PGCA-Net, demonstrated superior prognostic performance (C-index range: 0.672–0.795 across validation cohorts) compared to MacroContextNet (C-index range: 0.643–0.730) and outperformed current state-of-the-art models. After adjustment for clinical covariates, PGCA-Net effectively stratified patients into high-risk and low-risk groups for OS, with hazard ratios (HRs) for the high-risk group ranging from 4.93 to 9.38 across different cohorts. Seven AI-derived quantitative pathological biomarkers were identified and validated; notably, high tumor-muscle co-localization (Coloc_M) scores (HR up to 5.06) and high tumor-renal parenchyma co-localization (Coloc_R) scores (HR up to 4.52) consistently predicted increased mortality risk.

Conclusions:

This prior knowledge-guided deep learning system significantly improves OS prediction and risk stratification in UTUC by integrating multiscale pathological features. The AI-driven, interpretable tool offers an objective approach for prognostic assessment and biomarker discovery, with strong potential to refine personalized UTUC management and enhance prognostic accuracy.

HIGHLIGHTS

• A prior knowledge-guided deep learning system was developed and validated for predicting OS and exploring biomarkers in upper tract urothelial carcinoma (UTUC) patients using whole-slide images from a multicenter cohort.

• The UCSegNet tile classifier achieved excellent multiclass tissue classification, while the prognostic networks, MacroContextNet and PGCA-Net, effectively predicted OS.

• We identified and validated several AI-derived quantitative pathological biomarkers, providing an objective and interpretable tool that has strong potential to refine personalized UTUC management and enhance prognostic accuracy.

Introduction

Upper tract urothelial carcinoma (UTUC), a type of urothelial carcinoma (UC), primarily arises in the renal pelvis, calyces, and ureter[1]. Although both UTUC and bladder urothelial carcinoma (BUC) are subtypes of UC, UTUC has historically been understudied[2]. Although UTUC constitutes only 5–10% of all UC cases, its incidence is rising due to improved diagnostic methods and an aging population^[1,3]. The hidden anatomical position and unique biological traits frequently lead to understated first clinical manifestations. Consequently, roughly two-thirds of patients are diagnosed after muscle invasion, a stage that is significantly correlated with inferior results^[4–6]. Radical nephroureterectomy (RNU) is the standard treatment for UTUC. However, UTUC is more aggressive than BUC, with a much higher incidence of muscle-invasive disease and lymph node metastases. Moreover, patients with UTUC are susceptible to postoperative renal function deterioration, and their overall treatment results and survival rates are lower than those of patients with BUC^[1,4,7–9]. These unique characteristics not only complicate early detection and accurate staging but also underscore the urgent need for personalized treatment strategies and improved patient outcomes.

Currently, the pathological Tumor Node and Metastasis (TNM) staging system is the cornerstone of prognostic evaluation, and histological grade remains a crucial prognostic indicator within the UTUC pathological evaluation system^[1,10]. Nevertheless, various research have disclosed considerable constraints in conventional assessment approaches. Conventional prognostic evaluations frequently exhibit flaws, such as neglecting the geographical context of tissue constituents and depending on subjective pathological interpretation due to the absence of quantifiable biomarkers^[11–13]. For instance, while current UTUC staging predominantly depends on the degree of tumor invasion, considerable prognostic variability is shown even among patients with identical disease stages^[1,14]. Accurate prognostic assessment is essential for guiding treatment and predicting outcomes, underscoring the critical need for an objective, reproducible, and interpretable pathological evaluation system.

In recent years, the advent of computational pathology has provided a novel approach to addressing the subjectivity and constraints associated with traditional pathological evaluations^[15,16]. By employing deep learning and advanced image processing techniques, researchers have developed neural network models capable of autonomously extracting complex image features from whole-slide images (WSIs) for applications such as tumor classification, tissue segmentation, and prognostic prediction^[17–19]. Studies have shown that integrating existing pathology data into a deep learning model enhances data efficiency and aids in the identification of diagnostically significant tissue areas[20]. In existing frameworks, WSI analysis is typically formulated as a multiple instance learning (MIL) scenario, which normally entails extracting features from all patches and later consolidating them into a slide-level representation for the final prediction^[21,22]. However, current research based on MIL has not fully explored the prognostic information inherent in the spatial distribution of tissue components and lacks the integration of prior pathological knowledge for more accurate prognostic assessment and biomarker discovery.

To address the aforementioned limitations, this study proposes a novel deep learning framework that integrates prior pathological knowledge to improve the prognostic evaluation of UTUC. Initially, we employed a UCSegNet tile classifier, which is based on the ResNeXt50 architecture, to accurately generate tissue classification probability heatmaps and quantitatively assess key biomarkers[23]. We subsequently utilized a pretrained vision-language model (CONCH), which incorporates text prompts to guide the extraction of global semantic features and to generate probability heatmaps for tumor categorization[24]. By combining tissue classification probability heatmaps with tumor categorization probability heatmaps, we constructed the MacroContextNet to analyze the spatial distribution patterns at the whole-slide level and to extract macrotopological features closely linked to prognosis. Finally, we integrated microlevel tissue features from the UNI self-supervised network with the macrospatial features provided by MacroContextNet via the pregating and contextual attention gate (CAG) multimodal fusion mechanism. This integration led to the construction of a pregated contextual attention network (PGCA-Net), which was enhanced by a multihead contextual attention transformer (MHCAT) to optimize cross-modal information interactions, thereby improving model interpretability and its ability to manage uncertainty in prognostic predictions^[25–27]. Furthermore, by utilizing tissue segmentation heatmaps and MacroContextNet attribution heatmaps, we investigated and confirmed several potential prognostic biomarkers. Ultimately, this research aims to develop an integrated prior knowledge-based AI prognostic system with enhanced clinical applicability, offering a more objective and potentially more accurate approach to UTUC prognosis.

Methods

Retrospective patient cohort and ethical statement

This retrospective, multicenter prognostic study adhered strictly to the TRIPOD guidelines and has been reported in line with the STROCSS 2025 guidelines^[28,29] (Supplemental Digital Content 3, available at: http://links.lww.com/JS9/F261and Supplemental Digital Content 4, available at: http://links.lww.com/JS9/F262).

We enrolled patients with UTUC who underwent RNU at four institutions: CMUFH, CMUSH, CMUTH, and YCH. The study cohort was stratified by institutional source: patients from CMUFH were randomly allocated at a 7:3 ratio to form an internal training set (n = 398) and validation set (n = 171), while patients from CMUSH, CMUTH, and YCH constituted the external validation cohort (n = 125, 45, and 66, respectively). Baseline characteristics, including demographics, clinical parameters, preoperative computed tomography urography imaging, perioperative details, cystoscopic follow-up data, and pathological reports, were systematically extracted from electronic medical records. This retrospective study included patients treated between 1 December 2012 and 30 December 2024, with the final follow-up conducted on 31 January 2025. Further data preprocessing and verification were carried out between February 2025 and March 2025. The exclusion criteria included nonurothelial malignancies, incomplete clinical records, and poor-quality WSIs exhibiting ≥ 20% artifacts or resolution < 20 μm/pixel. The detailed inclusion/exclusion criteria are illustrated in Supplemental Digital Content Fig. S1, available at: http://links.lww.com/JS9/F25. Ethical approval was obtained from our institutional review boards, and the requirement for informed consent was waived. The study was registered on the ClinicalTrials.gov network.

Multicenter WSIs acquisition and standardization

Hematoxylin and eosin-stained histopathological sections from UTUC patients across four participating institutions were digitized via three distinct WSI systems: KF-PRO-020 and KF-PRO-005-EX (Ningbo Jiangfeng Biotechnology Co., Ltd, China) and SQS-600P (Shenzhen Qiangsheng Technology Co., Ltd, China). All WSIs were acquired at a 40 × objective magnification with standardized protocols. The spatial resolution characteristics of each scanner were as follows: KF-PRO-020: 0.246 × 0.246 μm/pixel, KF-PRO-005-EX: 0.252 × 0.252 μm/pixel, and SQS-600P: 0.206 × 0.206 μm/pixel.

Local pathology knowledge-guided patch representation

To facilitate AI-driven biomarker discovery, prognosis modelling with MacroContextNet, and attribution analysis, we implemented the UCSegNet tile classification framework to decompose WSIs into tissue probability heatmaps and segmentation maps. The UCSegNet training process involves annotating tiles into eight distinct histological categories: tumor tissue, connective tissue, muscularis tissue, non-ROI, renal tissue, lymphoid tissue, adipose tissue, and empty area. Annotation precision and efficiency were enhanced through the integration of the segment analysis model within QuPath software[30], enabling rapid pathological markup. Annotation precision and efficiency were enhanced through an expert-led, multitiered quality control system. Two board-certified pathologists independently annotated diagnostically challenging tiles. All discrepancies were adjudicated by a senior pathologist with over 30 years of experience to establish a definitive ground-truth label.

The UCSegNet architecture ( $f_{UCSegNet\mathrm{\_}conv}$), which used ResNeXt50 as its backbone and modified fully connected layers ( $f_{UCSegNet\mathrm{\_}fc}$), outputted eight class probabilities. During inference, WSIs were systematically tiled while preserving coordinates. Background elimination was conducted via the OTSU method on tissue thumbnail images. The extraction of subsequent patch images (256 × 256 pixels) was guided by non-background regions identified in 20 × WSI thumbnails. The quantitative descriptors included: $p(x,y)$ represents the classification probability for the patch at coordinates $(x,y)$, $s(x,y)$ represents the final classification outcome for patch $(x,y)$, and $i(x,y)$ represents the raw input patch data with spatial indexing. The $p(x,y)$ and $s(x,y)$can be defined as follows:

$$p(x,y)=softmax(f_{UCSegNet\mathrm{\_}fc}(f_{UCSegNet\mathrm{\_}conv}(i(x,y))))$$ (1)

$$s(x,y)=argmax(p(x,y))$$ (2)

By utilizing the UCSegNet’s continuous patch inference, $p(x,y)$ and $s(x,y)$ were combined to generate the corresponding tissue probability heatmaps $P$and tissue segmentation maps $S$.

Globally pathological knowledge-guided patch representation

To address the challenges of traditional pathological grading, which depends on pathologists’ subjective experience and is constrained by the quality of annotations, we developed an intelligent tumor categorization method for tumor patches informed by globally integrated pathological knowledge. The CONCH model was utilized to classify tumor patches extracted from WSIs. The tumor categorization includes three classes: high-grade urothelial carcinoma (HG), low-grade urothelial carcinoma (LG), and urothelial carcinoma with histologic variants (HV). Specifically, the CONCH model was initialized with the CoCa (Contrastive Captioners) pretraining framework, which employs a dual-tower architecture to process the image encoder (ViT-B/16) and text decoder in parallel. Its innovation lies in utilizing a vision-language model pretrained on large-scale image-text pairs to extract visual features through text prompts[31]. For WSIs, the OpenSlide and DeepZoomGenerator were utilized to extract 448 × 448 pixel RGB image patches. Each patch subsequently underwent normalization preprocessing before being input into the ViT-B/16 encoder. For each tile, the model computed image embeddings from the image encoder and text embeddings from class descriptive texts. The probability distribution for tumor pathological categorization was derived based on their cosine similarity, enabling automated and high-precision tumor categorization. Here, $v_i$ represents image feature embeddings, $v_t$ represents text feature embeddings, $e^{logit\mathrm{\_}scale}$ is a learnable scaling factor, $p_t(x,y)$ represents the categorization probability of a tumor patch, $s_t(x,y)$ represents the categorization result of a tumor patch, and $i(x,y)$ represents the tumor patch associated with specific coordinates. The $p_t(x,y)$ and $s_t(x,y)$ can be defined as follows:

$$p_t(x,y)=softmax(v_i\cdot v_t^T\cdot e^{logit\mathrm{\_}scale})$$ (3)

$$s_t(x,y)=argmax(p_t(x,y))$$ (4)

Utilizing continuous patch inference, $p_t(x,y)$ and $s_t(x,y)$ are combined to generate the corresponding tumor categorization probability heatmaps $P_t$and tumor categorization segmentation maps $S_t$. Figure 1b illustrates the comprehensive procedure for pathological knowledge-guided patch representation.

Figure 1.

Overview of the UTUC OS prognostic framework construction. (a) Patients were enrolled in four cohorts: CMUFH, CMUSH, CMUTH, and YCH. (b) Pathology knowledge-guided WSI decoupling and sparsification workflow. The study utilized the UCSegNet tile classifier to encode local pathological knowledge for tissue classification, while employing the pretrained vision-language model (CONCH) to integrate global pathological knowledge for tumor patch grading. (c) AI-driven prognostic pathological biomarker research workflow. The final biomarker exploration and validation were achieved, guided by the highlighted regions indicated in the saliency attribution heatmap of MacroContextNet. (d) Detailed architecture of PGCA-Net. The network utilized macro embeddings as queries to direct the aggregation of micro embeddings. (e) Core functionalities and clinical implementation of the UTUC prognostication system. UTUC, upper tract urothelial carcinoma; OS, overall survival; PG, pregating; CAG, contextual attention gate.

Macroscopic prognostic feature encoding network (MacroContextNet) construction

After obtaining the tissue probability heatmaps $P$ and the tumor pathological categorization probability heatmap $P_t$, we generated the final image patch probability heatmaps $M$ through the fusion of these two heatmaps. The construction of MacroContextNet $f_{MacroContext}$ comprised three core modules: a macroscopic tissue probability heatmap encoding module $f_{MacroContext\mathrm{\_}enco}$, a feature compression and stabilization module $f_{MacroContext\mathrm{\_}stab}$, and a prediction module $f_{pred}$. To achieve survival prediction using macroscopic tissue information from WSIs, we utilized the ResNeXt50 network to encode the $M$ into 2048-dimensional low-dimensional feature vectors. To ensure compatibility with the channel dimensions of subsequent modules for feature fusion, we adjusted the input channel number of ResNeXt50 to 12. The $f_{MacroContext\mathrm{\_}stab}$ module was used to further compress the coded macro feature vector $K_{MacroContext}\in {\mathbb{R}}^{2048}$ to $S_{MacroContext}\in {\mathbb{R}}^{32}$ and enhance the robustness of $f_{MacroContext}$. The final patient-level risk score $R{S}_{MacroContext}$ was then derived from $S_{MacroContext}$ using module $f_{pred}$. Related equations are as follows:

$$K_{MacroContext}=f_{MacroContext\mathrm{\_}enco}(M)$$ (5)

$$S_{MacroContext}=f_{MacroContext\mathrm{\_}stab}(K_{MacroContextNet})=ReLU(BN(FC(K_{MacroContext})))$$ (6)

$$R{S}_{MacroContext}=f_{pred}(S_{MacroContext})$$ (7)

Pregated contextual attention network (PGCA-Net) construction

To enhance the accuracy of prognosis prediction, we designed a weakly supervised deep learning framework, termed the pregated contextual attention network (PGCA-Net). This framework utilized the multihead context-aware transformer (MHCAT) mechanism to effectively integrate macroscopic features and microscopic features, addressing the limitations of models that rely solely on macroscopic features. Drawing inspiration from multimodal co-attention transformers, which learn interactions between histological instances and genetic data, our model was designed to capture the interplay between macroscopic and microscopic features for prognostic prediction[27].

Initially, we employed a self-supervised learning model (UNI) to extract patch-level microscopic features ( $K_{MicroContext}$) in WSIs. After processing by their respective encoder layers, the macroscopic features $K_{MacroContext}$ and $K_{MicroContext}$ were passed into the pregated attention (PGA) module for calculating attention weights. The comprehensive PGA module can be expressed as:

$$Q=Linear(K_{MacroContext}),K=Linear(K_{MicroContext}),V=Linear(K_{MicroContext})$$ (8)

$$attention\mathrm{\_}weights=softmax\left(\frac{Q{K}^T}{\sqrt{d_k}}\right)$$ (9)

$$Q_{hat}=attention\mathrm{\_}weights\cdot V$$ (10)

The module returns $Q$, $Q_{hat}$, and attention weights. The $Q$ and $Q_{hat}$ were passed into MHCAT for further processing. The comprehensive MHCA equations can be expressed as:

$$Q=X{W}_q+b_q,K=X{W}_k+b_k,V=X{W}_v+b_v$$ (11)

$$Attention\left(Q,K,V\right)=softmax\left(\frac{Q{K}^T}{\sqrt{d_k}}\right)V$$ (12)

$$MultiHead\left(Q,K,V\right)=Concat\left(h_1,h_2,\dots ,h_H\right)W_O$$ (13)

$$c=CAG(Q,\stackrel{⌢}{q})$$ (14)

$$Output=MultiHead\left(Q,K,V\right)+c$$ (15)

Subsequently, the co-attended features from each modality were processed by their respective transformer encoders. These encoders comprised several layers of multihead self-attention processes, which allowed the model to capture complex dependencies within each feature set. After post-encoding, the transformed features were passed through attention heads to produce pooled representations, which summarize the most prevalent information from the transformer outputs. The pooled representations from the micro and macro modalities were subsequently fused. Finally, the corresponding feature vector was passed through a linear classifier, which generates raw logits. These logits were then transformed into probability distributions using the sigmoid and softmax functions. Comprehensive analyses regarding model dimension selection, the effects of various fusion procedures, and ablation experiments are available in Supplemental Digital Content Table S1, available at: http://links.lww.com/JS9/F259. Figure 1d depicts the comprehensive structure of PGCA-Net.

AI-inspired prognostic biomarker quantification

Tissue fraction score calculation: Given the segmentation map $S$, the tissue fraction score is expressed as follows:

$$P_t=\frac{N_t}{N-N_{empty}-N_{non-ROI}}$$ (16)

$N_t$ represents the pixel count within class $t$, $N_{empty}$ represents the pixel count within empty class, $N_{non-ROI}$ represents the pixel count within non-ROI, and $N$ represents the total number of pixels within $S$. $S$ was obtained by applying the argmax function to the tissue probability heatmaps ( $P$). Using the $P_t$ formula, we defined the Tumor Fraction Score ( $TFS$) and Infiltrating Lymphocytes Score ( $IFS$) as follows:

$$TFS=\frac{N_{TUM}}{N-N_{empty}-N_{non-ROI}}$$ (17)

$$IFS=\frac{N_{INF}}{N-N_{empty}-N_{non-ROI}}$$ (18)

$N_{TUM}$ represents the pixel count of the tumor region and $N_{INF}$ represents the pixel count of the lymphocyte region.

Tumor-infiltrating lymphocytes ( $TILs$): we evaluated $TILs$ based on the lymphocyte co-localization score. To quantify $TILs$, we divided $S$ into $m\times n$ grids of uniform size, with each grid measuring 10 pixels in dimension. Next, we established the lymphocyte co-localization score $M$ utilizing the Morisita–Horn index.

$$M=\frac{2\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n\left(p_{ij}^{INF}\times p_{ij}^{TUM}\right)}{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n{(p_{ij}^{INF})}^2+\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n{(p_{ij}^{TUM})}^2}$$ (19)

$p_{ij}^{INF}$and $p_{ij}^{TUM}$represent the proportions of inflammation and tumor areas within the grid-cell ( $i,j$). Based on this foundation, the quantified $TILs$ can be defined as:

$$TILs=\left\{{\hspace{0.17em}}_{\begin{array}{c}1,\sum _{i=1}^m\sum _{j=1}^n\left(p_{ij}^{TUM}\right)\le 0\end{array}}^{\frac{M}{2}\times \frac{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n(p_{ij}^{INF})}{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n(p_{ij}^{TUM})},\sum _{i=1}^m\sum _{j=1}^n\left(p_{ij}^{TUM}\right)>0}\right)$$ (20)

Tumor-Muscle Infiltration Fraction ( $TIM$) and Tumor-Muscle Co-localization ( $Coloc\mathrm{\_}M$) score: The spatial architecture of inflammation within the tumor microenvironment was assessed through analysis of $TILs$. Building upon this, we developed quantitative metrics $TIM$ and $Coloc\mathrm{\_}M$ to evaluate the spatial location of tumor and muscularis boundaries, reflecting the spatial relationship between muscularis and tumor. Specifically, increased values of $TIM$ and $Coloc\mathrm{\_}M$ correlate with a greater degree of tumor infiltration into the muscle layer, a characteristic associated with more aggressive and advanced-stage tumors. The mathematical formulation of $TIM$ and $Coloc\mathrm{\_}M$ is as follows:

$$Coloc\mathrm{\_}M=\frac{2\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n\left(p_{ij}^{MUS}\times p_{ij}^{TUM}\right)}{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n{(p_{ij}^{MUS})}^2+\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n{(p_{ij}^{TUM})}^2}$$ (21)

$$TIM=\left\{{\hspace{0.17em}}_{\begin{array}{c}0,\sum _{i=1}^m\sum _{j=1}^n\left(p_{ij}^{TUM}\right)\le 0\end{array}}^{\frac{Coloc\mathrm{\_}m}{2}\times \frac{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n(p_{ij}^{MUS})}{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n(p_{ij}^{TUM})},\sum _{i=1}^m\sum _{j=1}^n\left(p_{ij}^{TUM}\right)>0}\right)$$ (22)

$p_{ij}^{MUS}$and $p_{ij}^{TUM}$ represent the proportions of muscularis and tumor regions within the grid-cell ( $i,j$).

Tumor-Renal Parenchyma Infiltration Fraction ( $TIR$) and Tumor-Renal Parenchyma Co-localization ( $Coloc\mathrm{\_}R$) score: In accordance with the definitions of $TIM$ and $TILs$, $TIR$ is used to measure the spatial distribution between tumor margins and renal parenchyma margins. The quantified $TIR$ and $Coloc\mathrm{\_}R$ can be defined as:

$$Coloc\mathrm{\_}R=\frac{2\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n\left(p_{ij}^{RPA}\times p_{ij}^{TUM}\right)}{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n{(p_{ij}^{RPA})}^2+\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n{(p_{ij}^{TUM})}^2}$$ (23)

$$TIR=\left\{{\hspace{0.17em}}_{\begin{array}{c}0,\sum _{i=1}^m\sum _{j=1}^n\left(p_{ij}^{TUM}\right)\le 0\end{array}}^{\frac{Coloc\mathrm{\_}R}{2}\times \frac{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n(p_{ij}^{RPA})}{\sum {\hspace{0.17em}}_{i=1}^m\sum {\hspace{0.17em}}_{j=1}^n(p_{ij}^{TUM})},\sum _{i=1}^m\sum _{j=1}^n\left(p_{ij}^{TUM}\right)>0}\right)$$ (24)

$p_{ij}^{RPA}$and $p_{ij}^{TUM}$ represent the proportions of renal parenchyma and tumor regions within the grid-cell ( $i,j$).

Survival-oriented loss function engineering

To train the UCSegNet tile classifier, we utilized the cross-entropy loss function. The negative Cox partial log-likelihood loss, being mini-batch dependent, poses difficulties when applied to mini-batch training on pathology WSIs, primarily due to the heterogeneous nature of microscopic patch features across different WSIs. For the survival prediction tasks using MacroContextNet and PGCA-Net, we adopted a category-based negative log-likelihood loss function^[32,33]. Based on the quartiles of uncensored patients’ survival times in the training set, we partitioned the continuous overall survival (OS) time $T_s$ into four discrete intervals: [ $t_0$, $t_1$], [ $t_1$, $t_2$], [ $t_2$, $t_3$], and [ $t_3$, $t_4$]. Subsequently, the survival time $T_i$ for the $i-th$ patient was discretized into one of these predefined intervals.

(25)

The prediction layer constructed the hazard function $f_h$ based on the patient’s final fused representation $F_{final}$, and the survival function $f_s$ can be expressed as:

$$f_h\left(n\left|F_{final}\right)\right)=P\left(T_i=n\left|T_i\ge n,F_{final}\right)\right)$$ (26)

$$\begin{array}{r}{f}_s\left(n\left|F_{final}\right)\right)=P\left(T_i>n\left|F_{final}\right)\right)\\ =\prod _{s=1}^n\left(1-f_h\left(s\left|F_{final}\right)\right)\right)\end{array}$$ (27)

And the discrete survival log-likelihood function can be expressed as:

$$\begin{array}{r}{L}_i=-c_{i}log\left(f_s\left(Y_i\left|F_{final}\right)\right)\right)\\ -\left(1-c_i\right)log\left(f_s\left(Y_i-1\left|F_{final}\right)\right)\right)\\ -\left(1-c_i\right)log\left(f_h\left(Y_i\left|F_{final}\right)\right)\right)\end{array}$$ (28)

$Y_i$ represents the truth label for the $i$-th individual, whereas $ci$ represents the binary censoring result ( $ci$=1 if the patient was still alive at the end of the follow-up, and $ci$=0, otherwise). To ensure that uncensored patient data contributed meaningfully to model training, we computed the final loss as a weighted sum.

Statistical analysis

We evaluated the predictive accuracy for OS using the concordance index (C-index) and the area under the receiver operating characteristic (ROC) curve (AUC). OS was defined as the time from surgery to death from any cause. The theoretical optimal risk stratification threshold was determined through maximum rank statistics applied to the training cohort data. This established a binary risk model: a high-risk group (score ≥ threshold) and a low-risk group (score < threshold). Kaplan–Meier survival curves with log-rank tests were used to compare group differences. Multivariable Cox proportional hazards regression was implemented using the survival R package to identify independent prognostic factors.

Continuous variables were presented as median (interquartile range, IQR), and categorical variables as count (percentage). Statistical comparisons used χ² tests (or Fisher’s exact test for small samples) for categorical variables and ANOVA (or the Kruskal–Wallis test for non-parametric distributions) for continuous variables, with all tests conducted as two-tailed analyses and statistical significance defined as P < 0.05.

Data availability statement

The research datasets generated during this study, including de-identified original imaging materials and follow-up records, are available from the corresponding author upon reasonable request. Access to these datasets requires prior approval from the institutional review boards of all four participating medical institutions. Due to the sensitive nature of the clinical imaging data and longitudinal patient follow-up information, which could potentially compromise participant confidentiality if disclosed without proper ethical oversight. The source code for this study is available online (https://github.com/jacobhqh1997/UTUC_MCA).

Results

Patient clinical baseline characteristics

In this study, we established a multicenter research cohort by integrating comprehensive clinical data and histopathological images from 805 UTUC patients. Data were retrospectively collected from four medical centers: CMUFH (n = 569), CMUSH (n = 125), CMUTH (n = 45), and YCH (n = 66). For the CMUFH cohort, which spans December 2012 to December 2024, patients were randomly split into a training set (n = 398) and an internal validation set (n = 171) at a 7:3 ratio. External validation was performed using three additional cohorts: CMUSH (May 2013–December 2024), CMUTH (May 2016–December 2023), and YCH (January 2016–December 2024), as illustrated in Figure 1a. Baseline characteristics, including patient demographics, tumor staging, and lymphovascular invasion status, were analysed across cohorts (Table 1). Specifically, the training cohort exhibited a median follow-up of 11.95 months (IQR: 4.10–31.61), the internal validation cohort 10.61 months (IQR: 4.03–22.51), the CMUSH cohort 8.44 months (IQR: 3.04–30.14), the CMUTH cohort 8.65 months (IQR: 5.08–16.42), and the YCH cohort 17.50 months (IQR: 10.07–30.00).

Table 1.

Baseline patients’ characteristics

	CMUFH (n=569)		CMUSH	CMUTH	YCH
	Training (n=398)	Validation (n=171)	(n=125)	(n=45)	(n=66)
Age
≤60 years	94 (23.6%)	37 (21.6%)	27 (21.6%)	9 (20.0%)	11 (16.7%)
61–70 years	143 (35.9%)	64 (37.4%)	43 (34.4%)	17 (37.8%)	20 (30.3%)
71–80 years	120 (30.2%)	54 (31.6%)	42 (33.6%)	14 (31.1%)	23 (34.8%)
> 80 years	41 (10.3%)	16 (9.4%)	13 (10.4%)	5 (11.1%)	12 (18.2%)
Gender
Male	235 (59.0%)	109 (63.7%)	61 (48.8%)	31 (68.9%)	37 (56.1%)
Female	163 (41.0%)	62 (36.3%)	64 (51.2%)	14 (31.1%)	29 (43.9%)
Pathological T stage
(Ta, T0, Tis)	82 (20.6%)	37 (21.6%)	18 (14.4%)	7 (15.6%)	20 (30.3%)
Tx	3 (0.8%)	3 (1.8%)	–	–	–
T1	123 (30.9%)	51 (29.8%)	39 (31.2%)	14 (31.1%)	18 (27.3%)
T2	106 (26.6%)	44 (25.7%)	31 (24.8%)	9 (20.0%)	12 (18.2%)
T3	76 (19.1%)	33 (19.3%)	33 (26.4%)	13 (28.9%)	14 (21.2%)
T4	8 (2.0%)	3 (1.8%)	4 (3.2%)	2 (4.4%)	2 (3.0%)
Pathological N stage
N0	25 (6.3%)	7 (4.1%)	23 (18.4%)	20 (44.4%)	7 (10.6%)
N+	13 (3.3%)	6 (3.5%)	4 (3.2%)	2 (4.4%)	1 (1.5%)
Nx	360 (90.5%)	158 (92.4%)	98 (78.4%)	23 (51.1%)	58 (87.9%)
Pathological M stage
M0	397 (99.7%)	169 (98.8%)	124 (99.2%)	44 (97.8%)	66 (100.0%)
M1	1 (0.3%)	2 (1.2%)	1 (0.8%)	1 (2.2%)	–
Lymph vascular invasion
No	372 (93.5%)	157 (91.8%)	110 (88.0%)	36 (80.0%)	57 (86.4%)
Yes	26 (6.5%)	14 (8.2%)	15 (12.0%)	9 (20.0%)	9 (13.6%)
Follow-up (median [IQR])	11.95 [4.10, 31.61]	10.61 [4.03, 22.51]	8.44 [3.04, 30.14]	8.65 [5.08, 16.42]	17.50 [10.07, 30.00]

Data are median (interquartile range) or n (%).

Knowledge-guided tissue probability heatmaps generation via UCSegNet and vision-language model

On the basis of the ResNeXt50 architecture, this study successfully constructed the UCSegNet tile classification network to achieve eight-category tissue structure parsing in WSIs, including the tumor area, connective tissue area, muscularis area, renal area, non-Region of Interest (non-ROI), adipose tissue area, empty area, and immune cells area. The experimental results demonstrated exceptional classification performance across all cohorts, with ROC-AUCs ranging from 0.9916 [95% confidence interval (CI): 0.9896–0.9935] to 0.9948 (95% CI: 0.9941–0.9954) (Fig. 2a, Supplemental Digital Content Fig. S2, available at:http://links.lww.com/JS9/F259), indicating that UCSegNet has highly reliable discriminative ability.

Figure 2.

Classification performance of UCSegNet and time-dependent AUCs of MacroContextNet and PGCA-Net. (a) ROC curves for the multiclass patch classification model (UCSegNet). (b) Time-dependent AUCs for MacroContextNet across all enrolled cohorts. (c) Time-dependent AUCs for PGCA-Net across all enrolled cohorts. ROC, the receiver operator characteristic curve; AUC, area under the ROC.

Following the construction of UCSegNet, we applied it to perform patch-level classification across all the samples. By integrating the spatial coordinates of image patches, we first generate tissue classification probability heatmaps, which reflect local pathological knowledge-guided representations. To further incorporate global pathological semantics, we innovatively integrated the CONCH vision-language foundation model, which employs contrastive learning to jointly model histopathological images and textual descriptions, achieving state-of-the-art performance in multiple pathological analysis tasks[24]. Leveraging its pretrained image encoder, we extracted semantic feature distributions of tumor pathological knowledge and generated tumor categorization probability heatmaps by corresponding patch coordinates, thereby establishing knowledge-guided representations. Ultimately, the tissue probability heatmaps and the tumor categorization heatmaps were concatenated to generate the final integrated probability heatmaps.

MacroContextNet and PGCA-Net construction and validation

Using concatenated probability heatmaps, we trained and validated the MacroContextNet framework for macroscopic prognostic prediction. This framework comprises three cascaded modules: the macro heatmap encoding module for low-dimensional feature mapping; the feature compression module, which extracts robust prognostic features through channel attention mechanisms; and the prediction module, which establishes associations between multiscale features and survival risk. The results demonstrated that MacroContextNet achieved C-index values of 0.643 (95% CI: 0.496–0.79) in the CMUFH training cohort and 0.712 (95% CI: 0.518–0.906) in the CMUFH internal validation cohort. In the external validation cohorts of CMUSH, CMUTH, and YCH, the C-index values were 0.661 (95% CI: 0.418–0.904), 0.730 (95% CI: 0.365–1.095), and 0.695 (95% CI: 0.572–0.810), respectively (Table 2).

Table 2.

Performance of MacroContextNet and PGCA-Net in different cohorts

	CMUFH training		CMUFH validation		CMUSH		CMUTH		YCH
	MacroContextNet	PGCA-Net	MacroContextNet	PGCA-Net	MacroContextNet	PGCA-Net	MacroContextNet	PGCA-Net	MacroContextNet	PGCA-Net
Low risk (n)	347	328	154	130	113	77	32	34	47	51
High risk (n)	51	70	17	41	12	48	13	11	19	15
Univariate Cox analysis	4.28 (2.69–6.83)	7.57 (4.71–12.15)	2.43 (1.10–5.37)	4.54 (2.33–8.86)	3.83 (1.52–9.61)	6.39 (2.66–15.32)	4.21 (1.12–15.85)	3.70 (1.04–13.18)	2.75 (1.23–6.14)	5.24 (2.01–13.67)
Adjust for age, gender	5.5 (3.35–9.05)	9.38 (5.65–15.59)	2.32 (1.01–5.3)	4.93 (2.46–9.9)	5.77 (2.07–16.08)	7.75 (3.08–19.49)	4.95 (1.24–19.8)	2.54 (0.55–11.85)	2.73 (1.23–6.05)	8.04 (2.77–23.37)
Adjust for age, gender, T stage, tumor grade	3.11 (1.79–5.39)	5.94 (3.25–10.84)	1.66 (0.71–3.87)	3.03 (1.37–6.7)	3.65 (1.04–12.75)	7.36 (2.53–21.37)	4.68 (0.73–30.22)	1.13 (0.13–9.53)	2.60 (1.16–5.86)	8.43 (2.79–25.52)
C-index	0.643 (0.496–0.79)	0.767 (0.655–0.879)	0.712 (0.518–0.906)	0.725 (0.544–0.906)	0.661 (0.418–0.904)	0.795 (0.667–0.923)	0.730 (0.365–1.095)	0.739 (0.452–1.026)	0.695 (0.572–0.810)	0.672 (0.457–0.887)
AUC at 1 years	0.702 (0.565–0.839)	0.780 (0.665–0.895)	0.616 (0.475–0.757)	0.686 (0.518–0.855)	0.613 (0.424–0.803)	0.779 (0.661–0.897)	0.665 (0.360–0.969)	0.782 (0.589–0.974)	0.696 (0.495–0.898)	0.769 (0.611–0.927)
AUC at 3 years	0.644 (0.550–0.739)	0.820 (0.750–0.890)	0.659 (0.500–0.818)	0.701 (0.550–0.851)	0.672 (0.517–0.827)	0.893 (0.800–0.986)	0.676 (0.363–0.989)	0.792 (0.500–1.083)	0.713 (0.523–0.902)	0.665 (0.479–0.852)
AUC at 5 years	0.675 (0.573–0.778)	0.818 (0.740–0.897)	0.605 (0.406–0.805)	0.671 (0.501–0.842)	0.609 (0.416–0.803)	0.803 (0.623–0.984)	0.757 (0.437–1.077)	0.917 (0.730–1.104)	0.615 (0.373–0.858)	0.654 (0.412–0.897)

C-index, concordance index; AUC, area under the receiver operator characteristic curve.

To explore deeply the interaction between microscopic histopathological features and macroscopic features and to comprehensively predict prognostic outcomes while revealing interpretable associations between them, we designed a weakly supervised deep learning framework, termed pregated contextual attention network (PGCA-Net), which incorporates two key mechanisms: pregating and CAG. The pregating mechanism directly controls cross-modal interaction generation prior to multihead attention, whereas the CAG mechanism leverages contextual information to evaluate the relevance of generated attention vectors following multihead attention (Fig. 1d). PGCA-Net achieved C-index values of 0.767 (95% CI: 0.655–0.879) in the CMUFH training cohort and 0.725 (95% CI: 0.544–0.906) in the CMUFH internal validation cohort. In the external validation cohorts of CMUSH, CMUTH, and YCH, the C-index values were 0.795 (95% CI: 0.667–0.923), 0.739 (95% CI: 0.452–1.026), and 0.672 (95% CI: 0.457–0.887), respectively. Furthermore, ablation studies confirmed that PGCA-Net outperformed several state-of-the-art models, including AttMIL, TransMIL, and Patch-GCN, in prognostic prediction tasks (Supplemental Digital Content Table S1, available at: http://links.lww.com/JS9/F259). Time-dependent AUC analyses for MacroContextNet and PGCA-Net across all cohorts are illustrated in Figure 2b and c, respectively.

Prognostic performance of MacroContextNet and PGCA-Net across enrolled cohorts

On the basis of the maximally selected rank statistics calculated in the training cohort, the cutoff values for MacroContextNet and PGCA-Net were determined to be −2.985543 and −2.939310, respectively. Across all enrolled cohorts, both models consistently demonstrated robust risk stratification performance. For MacroContextNet, the hazard ratios (HRs) for the high-risk group ranged from 2.43 (95% CI: 1.10–5.37; P = 0.029) to 16.71 (95% CI: 3.44–81.1; P < 0.001). Similarly, PGCA-Net exhibited HRs ranging from 3.7 (95% CI: 1.12–4.47; P = 0.022) to 7.57 (95% CI: 4.71–12.15; P < 0.001) (Figs 3–4).

Figure 3.

Kaplan–Meier survival analysis for prognostic system in CMUFH cohort. Kaplan–Meier survival analysis was performed for MacroContextNet (a–b), PGCA-Net (c–d) across the CMUFH training and validation cohort.

Figure 4.

Kaplan–Meier survival analysis for prognostic system in CMUSH, CMUTH, and YCH cohort. Kaplan–Meier survival analysis was performed for MacroContextNet (a–c), PGCA-Net (d–f) across the CMUSH, CMUTH, and YCH cohort.

To further validate the risk stratification performance of MacroContextNet and PGCA-Net after incorporating clinical covariates, multivariable Cox regression analyses were performed. After adjusting for covariates such as age and sex, the HRs for MacroContextNet ranged from 2.32 (95% CI: 1.01–5.30) to 5.77 (95% CI: 2.07–16.08) across all cohorts. PGCA-Net exhibited HRs ranging from 4.93 (95% CI: 2.46–9.90) to 9.38 (95% CI: 5.65–15.59) (Table 2). When jointly adjusted for T stage and tumor grade, the HRs for MacroContextNet ranged from 1.66 (95% CI: 0.71–3.87) to 4.68 (95% CI: 0.73–30.22) across all cohorts. PGCA-Net exhibited HRs ranging from 1.13 (95% CI: 0.13–9.53) to 8.43 (95% CI: 2.79–25.52) (Table 2).

The predictive performance of both models was further validated via the C-index and AUC metrics. MacroContextNet achieved C-index values ranging from 0.643 to 0.730, whereas PGCA-Net demonstrated superior discriminative ability with C-index values ranging from 0.672 to 0.795. Both models exhibited excellent AUCs for 1-, 3-, and 5-year survival prediction (Table 2). These findings confirm that both models retain stable prognostic efficacy after the integration of clinical variables. Detailed HR risk and Kaplan–Meier curve results for each subgroup across all enrolled cohorts are displayed in Supplemental Digital Content Figs S3–12. available at: http://links.lww.com/JS9/F259. Furthermore, we assessed the clinical applicability of both models. For 3-year OS, both models demonstrated reasonable calibration and showed a positive net benefit across a range of clinically relevant thresholds in decision curve analysis, with PGCA-Net generally showing superior performance (Supplemental Digital Content Fig. S13, available at: http://links.lww.com/JS9/F259).

AI-based exploration and validation of prognostic pathological biomarkers in UTUC

To elucidate the risk prediction mechanisms of MacroContextNet and identify potential biomarkers, this study employed the integrated gradient attribution method to generate saliency attribution heatmaps (overlapping with pathological segmentation heatmaps, Fig. 1c), visualizing key regions driving model decisions. The results revealed that the high-risk groups presented heatmap concentrations at tumor-muscularis interfaces (e.g., muscularis invasion margins) and tumor-renal parenchyma interfaces, focusing on the areas between renal parenchyma and muscle tissue, whereas the low-risk groups presented lymphocyte-rich stromal regions, suggesting that spatial heterogeneity drives prognostic divergence. On the basis of these findings, we defined seven quantitative pathological metrics spanning three dimensions: tumor invasion patterns, the immune microenvironment, and tumor–host spatial interactions. These included Tumor-Muscle Invasion Score (TIM), Tumor-Renal Parenchyma Invasion Score (TIR), Tumor-Infiltrating Lymphocytes (TIL), Tumor Fraction Score (TFS), Inflammatory Fraction Score (IFS), Tumor-Muscle Co-localization Score (Coloc_M), and Tumor-Renal Co-localization Score (Coloc_R). Cox regression and Kaplan‒Meier analyses across cohorts confirmed their prognostic reliability, with Coloc_M and Coloc_R prioritized for cross-cohort validation owing to their consistent performance (Fig. 5, Supplemental Digital Content Fig. S14, available at: http://links.lww.com/JS9/F259). Coloc_M demonstrated stable risk stratification across the CMUFH training cohort (HR = 4.1, 95% CI: 2.31–7.3), CMUFH internal validation set (HR = 3.4, 1.15–9.99), CMUSH cohort (HR = 5.02, 1.18–21.3), and combined CMUTH-YCH external validation set (HR = 5.06, 1.2–21.43), suggesting that tumor-muscularis interface disruption is a prognostic predictor. For pyelocaliceal carcinoma with parenchymal invasion, integration of the CMUFH validation, CMUSH, CMUTH, and YCH cohorts revealed that Coloc_R maintained significant prognostic associations in the training set (HR = 4.52, 1.88–10.86) and combined validation cohorts (HR = 2.24, 1.12–4.47), indicating that renal parenchyma invasion is a potential prognostic determinant. These findings systematically unravel key spatial histopathological features underlying UTUC prognosis.

Figure 5.

Kaplan–Meier survival analysis for AI-inspired biomarker exploration and validation. Kaplan–Meier survival analysis was performed for Coloc_M (a–d) and Coloc_R (e–f) across the cohorts. Coloc_M, tumor-muscle co-localization score; Coloc_R, tumor-renal parenchyma infiltration fraction.

Model interpretability through attention-based visualization

In addition to achieving a higher C index and improved risk stratification (Table 2), PGCA-Net offered enhanced interpretability by visualizing how multimodal features interact and contribute to prognostic predictions. Attention-based visualization illustrates the interplay between macro- and micro-level modalities, whereas global interpretability highlights differences in potential prognostic biomarkers across PGCA-Net-defined risk groups. Representative analyses from the CMUFH training and internal validation cohorts are shown in Figures 6 and 7, with the results from the external cohorts provided in Supplemental Digital Content Figs S15–17, Available at:http://links.lww.com/JS9/F259. A visual demonstration of the AI-driven prognostic system is available in the supplementary movie (Supplemental Digital Content Fig. S1e, available at: http://links.lww.com/JS9/F260).

Figure 6.

Instance-level and global-level interpretability analyses of the PGCA-Net in the CMUFH training cohort. The highlighted regions in the saliency attribution heatmaps reveal the focal areas of MacroContextNet, while the activated zones in the co-attention heatmaps represent ROIs guided by the macroscopic features. (a) Low-risk prediction example, (b) high-risk prediction example, (c) global-level distribution differences of potential biomarkers between high- and low-risk groups. ROI, regions of interest; Coloc_M, tumor-muscle co-localization score; Coloc_R, tumor-renal parenchyma co-localization score; IFS, inflammatory fraction score; TFS, tumor fraction score; TIL, tumor-infiltrating lymphocytes; TIM, tumor-muscle infiltration fraction score; TIR, tumor-renal parenchyma infiltration fraction.

Figure 7.

Instance-level and global-level interpretability analyses of the PGCA-Net in the CMUFH validation cohort. The highlighted regions in the saliency attribution heatmaps reveal the focal areas of MacroContextNet, while the activated zones in the co-attention heatmaps represent ROIs guided by the macroscopic features. (a) Low-risk prediction example, (b) high-risk prediction example, (c) global-level distribution differences of potential biomarkers between high- and low-risk groups. ROI, regions of interest; Coloc_M, tumor-muscle co-localization score; Coloc_R, tumor-renal parenchyma co-localization score; IFS, inflammatory fraction score; TFS, tumor fraction score; TIL, tumor infiltrating lymphocytes; TIM, tumor-muscle infiltration fraction score; TIR, tumor-renal parenchyma infiltration fraction.

Discussion

While the EAU guidelines provide a risk stratification framework to guide treatment selection for UTUC patients, more precise OS prediction is needed to better support personalized therapeutic decisions[1]. In this study, we developed and validated a deep learning-based prognostic system that integrates multiscale pathological features with prior domain knowledge for accurate survival risk assessment in UTUC patients. Our framework combines the local structural parsing ability of UCSegNet with the global semantic understanding of visual-language foundation model, while the PGCA-Net also enables the effective fusion of pathological information at the micro- and macro-levels.

To our knowledge, this study also represents one of the largest AI cohorts for UTUC to date. The central innovation of our work is the construction of a multiscale feature interaction framework. Unlike previous studies that focused on single-modality features[34], we implemented a pregated co-attention (PGCA) mechanism to integrate both macroscopic and microscopic pathological features. The integration model, PGCA-Net, achieved strong cross-cohort predictive performance, with a C-index ranging from 0.672 to 0.795. Ablation studies further confirmed its superior risk stratification capability compared to current state-of-the-art models such as AttMIL and TransMIL^[35,36], underscoring the importance of multiscale feature integration in prognostic modeling^[37,38]. This advancement offers a powerful tool for individualized risk stratification, potentially improving the accuracy of prognostic predictions after surgery. Interestingly, consistent with PGCA-Net, the integration of macroscopic pathological features into the AttMIL, TransMIL, and Patch-GCN resulted in improved performance in corresponding models, demonstrating the critical importance of integrating macroscopic pathological features for enhancing the performance of the prognostic models[39]. These findings align with prior research on bladder cancer, which emphasized the importance of multiscale integration strategies[40]. This integrated approach is particularly crucial given the prognostic instability we observed in baseline models reliant on T stage and Grade alone in our multicenter analysis, reinforcing the need for a more robust method to overcome the limitations of conventional markers.

Guided by attribution heatmaps derived from MacroContextNet, we identified and validated seven quantifiable biomarkers directly from WSIs. Among these, the Coloc_M and Coloc_R demonstrated superior stratification consistency and distributional separability across all enrolled cohorts, indicating their potential as robust prognostic markers for UTUC. This consistent prognostic value across independent cohorts aligns with the aggressive infiltration patterns commonly observed in UTUC[4]. In contrast, the prognostic utility of other identified biomarkers was less robust, possibly due to differences in data distributions between the training and validation sets, as well as potential interactions among the biomarkers themselves. Further analyses of the attribution heatmaps indicated that high-risk regions were predominantly located in tumor-infiltrating muscular areas, whereas low-risk regions were enriched in lymphoid stromal response zones^[41–43], enhancing model interpretability and offering new insights into the spatial pathology of UTUC. Future work may involve integration with spatial transcriptomics to further elucidate signaling dynamics within the tumor microenvironment.

From a clinical perspective, our knowledge-based prognostic system offers three key advantages. First, it facilitates comprehensive, end-to-end analysis of WSIs, thus fulfilling the requirements for rapid prognostic evaluation[44]. Second, the system produces attention heatmaps that visually emphasize key prognostic areas, potentially aiding pathologists in pinpointing diagnostic blind spots and diminishing subjective bias[45]. Third, it quantifies multiple latent biomarkers from WSIs, providing objective metrics to inform personalized treatment strategies[46]. Nonetheless, prospective clinical studies are needed to further validate the system’s efficacy in real-world decision-making.

Our study has several limitations. First, its retrospective design is subject to inherent selection bias, although this was mitigated by the inclusion of multicenter cohorts. Second, standardized data on postoperative adjuvant therapy were not collected due to historical treatment heterogeneity, representing a potential unmeasured confounder. Third, the limited sample size precluded in-depth analysis of rare UTUC subtypes, indicating the need for larger-scale studies to assess the generalizability of our biomarkers. Fourth, the median follow-up in our cohorts was relatively short, a common challenge in studies of an aggressive disease like UTUC. However, to mitigate concerns of exaggerated short-term performance, our time-dependent AUC analysis confirmed the model’s robust long-term prognostic stability, with high predictive accuracy maintained at both 3- and 5-year time points across the cohorts (Table 2). Additionally, the current system requires substantial computational resources, which may impede its deployment in resource-limited settings. Future work should concentrate on developing lightweight models suitable for broader clinical adoption. Lastly, our system has not yet incorporated UTUC-relevant genomic features (e.g., FGFR3 mutation status and p53 alterations), although previous studies in BUC have shown that multiomics integration can significantly enhance the predictive accuracy of prognostic models^[47,48]. Future efforts should therefore aim to integrate molecular and radiomic data to construct a comprehensive, multimodal prognostic framework that captures the biological complexity of UTUC.

In conclusion, we have developed and validated a deep learning-based prognostic system that integrates macro- and micro-level pathological information for accurate OS prediction in UTUC patients. This system offers clinicians and pathologists a reliable tool for survival risk stratification and could guide individualized treatment decisions in the management of UTUC.

Supplementary Material

js9-112-1402-001.docx ^{(88.6KB, docx)}

js9-112-1402-002.docx ^{(37KB, docx)}

js9-112-1402-003.docx ^{(75.1KB, docx)}

js9-112-1402-004.docx ^{(9.3MB, docx)}

Download video file ^{(12.3MB, mp4)}

Ethical approval

This study was approved by the Ethics Committee of The First Affiliated Hospital of Chongqing Medical University (Approval No. K2024-187-07), and the requirement for informed consent was waived due to the retrospective nature of the study.

Consent

Not applicable.

Sources of funding

This study was supported by the Medical Health Care Ecosystem Innovation Team of the First Affiliated Hospital of Chongqing Medical University (CYYY-DSTDXM-202409), Chongqing Municipal Education Commission’s 14th Five-Year Key Discipline Support Project (No. 20240101), and Annual Research Projects of Chongqing Research Association for Science Popularization (2024CQKPB006).

Author contributions

X.P., H.T., B.X.X., Y.J.L., Q.H.H., W.Y.H., and M.Z.X. conceived and designed the study; X.P., H.T., B.X.X., Y.W.T., F.X.Y., and Q.H.H. collected the pathological slide images and follow-up data; B.X.X., H.T., and X.P. evaluated the images; Y.W.T., B.L., Y.D.C., and Y.J.L. labeled the pathological slide images; X.P., H.T., W.L.Z., and Q.H.H. trained and developed the AI system; X.P., H.T., and Q.H.H. analyzed and interpreted the data and wrote the original draft of the manuscript; Q.H.H., W.Y.H., and M.Z.X. revised the manuscript and completed the comparative experiments, while Y.J.L., Q.H.H., W.Y.H., and M.Z.X. supervised and directed the study.

Conflicts of interest disclosure

The authors declare that they have no competing interests.

Research registration unique identifying number (UIN)

(1) Name of the registry: AI-UTUC. (2) Unique identifying number or registration ID: NCT06993779. (3) Hyperlink to your specific registration (must be publicly accessible and will be checked): https://clinicaltrials.gov/study/NCT06993779.

Guarantor

Mingzhao Xiao, Weiyang He, Quanhao He, and Yingjia Liu.

Provenance and peer review

Not commissioned, externally peer-reviewed.

Data availability statement

The original data are not publicly available, but are available from the corresponding author upon reasonable request after IRB approval in accordance with the institute’s policy.