Enhancing foundation model transfer for prostate cancer detection with patch-level contrastive learning

Abstract

Accurate localization of prostate cancer on magnetic resonance imaging (MRI) remains challenging due to the subtle appearance of cancer, resulting in missed clinically significant cancers, large inter-reader variability and large numbers of confounders that require biopsy confirmation. Vision foundation models have shown promise, but direct transfer to prostate MRI is challenging due to the substantial domain gap between natural images and prostate MRI and the subtle appearance of prostate cancer. We therefore developed prostate vision contrastive networks (ProViCNet), a weakly supervised model utilizing patch-level contrastive learning on MRI to support MRI-based screening, biopsy targeting and focal treatment planning. ProViCNet was trained and validated (using 4401 patients across six cohorts) as a prostate cancer detection model on MRI. Training labels were biopsy-confirmed radiologist annotations, while the evaluation labels included both biopsy and surgery-confirmed lesions. ProViCNet demonstrated consistent detection and segmentation performance across multiple internal and external validation cohorts, with area under the receiver operating characteristic curve (AUROC) values ranging from 0.875 to 0.966, and outperforming radiologists in the MRI expert reader study (0.907 versus 0.805, p < 0.01). We also integrated ProViCNet with serum PSA to develop a virtual screening test, which preserved high sensitivity for detecting clinically significant cancers while more than doubling specificity from 15% to 38% (p < 0.001) among men with PSA ≥ 4 ng/mL, thereby potentially reducing unnecessary biopsies. These findings highlight ProViCNet’s potential for enhancing the accuracy of prostate cancer diagnosis and reducing unnecessary biopsies.

Introduction

Prostate cancer is the most common malignancy in American men and the second leading cause of cancer-related death^1,2. Magnetic resonance imaging (MRI) has become an integral component of prostate cancer diagnosis, as MRI is required prior to biopsy in many countries (e.g., in Europe) or is strongly recommended in others (e.g., the United States of America). MRI offers great soft tissue contrast, allowing the visualization of cancer, and enabling men to avoid unneeded biopsy when suspicious cancerous lesions are not identified^3–5. MRI-guided biopsies improve the detection of clinically significant prostate cancer (csPCa, defined as International Society of Urologic Pathology (ISUP) Grade Group [GG] ≥2), which is particularly important given that early detection of prostate cancer is associated with excellent 5-year survival rates (>98%)^6,7. However, interpretation of prostate imaging remains challenging^8–10. In patients undergoing biopsy, radiologists interpreting MRI miss 12% of biopsy-detected clinically significant prostate cancers. Among patients undergoing radical prostatectomy¹¹, radiologists missed 34% of clinically significant and 81% of indolent cancers^12,13. Additionally, MRI interpretation demonstrates significant inter-observer variability across radiologists¹⁴, and specificity is relatively low—between 21.9% and 68.5%, depending on diagnostic criteria¹⁵. These diagnostic limitations can lead to delayed detection and intervention, potentially compromising survival outcomes, as the 5-year survival rates drop to 34% in advanced stages of the disease¹⁶. To overcome these diagnostic limitations and the associated decline in survival outcomes, accurate interpretation of MRI is essential to avoid unneeded biopsies in men without csPCa, to biopsy all lesions suspicious for cancer and localize the full cancer extent to facilitate treatment planning and focal treatment.

Recent developments in artificial intelligence (AI) have demonstrated significant potential in medical image analysis¹⁷. In particular, vision foundation models, which are pre-trained on large-scale datasets and can be adapted to various downstream tasks, have recently emerged, thereby further accelerating the progress in computer vision tasks^18,19. These vision foundation models have shown improved performance across various domains by learning generalizable representations from vast amounts of diverse medical imaging modalities²⁰. They not only serve as powerful feature extractors but also demonstrate enhanced generalization capabilities with limited data and offer robust transfer learning abilities across different medical imaging domains^21,22. In the field of prostate cancer, deep learning-based approaches for MRI analysis have shown promising results^12,23,24, with recent studies exploring self-supervised pretraining directly applied to prostate cancer imaging^25,26. However, these studies failed to design single foundation models to enable multiple tasks associated with prostate cancer diagnosis (screening, biopsy targeting and treatment planning) while lacking validation across MRI vendors, institutions and populations. Such a single foundation model has greater potential for adoption as it may improve MRI interpretation across the broad clinical variability of prostate MRI.

In this study, we introduce ProViCNet, a model developed to investigate whether a weakly supervised approach leveraging a patch-level contrastive learning strategy with pseudo-labels could improve the detection and localization of prostate cancer in MRI (Fig. 1a). Our approach integrates the Vision Foundation Model’s general vision capabilities with prostate-specific anatomical knowledge through a specialized training strategy. The framework was designed to process MRI imaging data, incorporating domain-specific features using label-guided, patch-based self-supervised learning while maintaining the advantages of foundation models. This approach aims to enhance diagnostic precision, to reduce false positives requiring biopsy confirmation, and to decrease inter-observer variability. To evaluate the clinical utility and generalization of ProViCNet, we conducted extensive validation in different tasks using five cohorts comprised of patients from different vendors and institutions (including three external ones). Specifically, we tested ProViCNet for classification tasks related to prostate cancer screening using MRI, for segmentation tasks related to MRI-guided biopsy targeting and identification of the full extent of cancer for MRI-guided focal treatment planning. In addition, we performed comparative analysis against board-certified radiologists interpreting MRIs for clinical care. We further evaluated the model’s performance for biopsy decision support by comparing ProViCNet against PSA-based screening and the PI-RADS scoring system. These results demonstrate the model’s capabilities and its promise in enhancing clinical decision-making for prostate cancer diagnosis, potentially improving patient care and outcomes.

Fig. 1. Development and validation of ProViCNet for prostate cancer detection.

a Overview of the proposed foundation model architecture. ProViCNet processes MRI sequences (T2W, ADC, highest b-value in DWI) to detect all cancer or clinically significant cancers. b Lesion-level performance evaluation framework. The prostate is divided into six regions (sextants), with performance metrics calculated using the 90th percentile of prediction values for each region. Right: Performance metrics across different classification thresholds. c Study cohorts and population characteristics. Internal training cohort (n = 1404), biopsy-confirmed test (C1, n = 352), and radical prostatectomy (C2, n = 120) cohorts. External validation was performed on three independent cohorts: PI-CAI (C3, n = 1497), UCLA (C4, n = 1154), and UCSD (C5, n = 292). Pie charts show the distribution of csPCa, indolent cancer, and normal cases. d Technical components of ProViCNet: (1) preprocessing of 3D prostate imaging data, (2) 3D-enhanced foundation model with transformer encoder, (3) patch-level contrastive learning strategy, and (4) final cancer detection module.

Results

Study cohorts

We included radiology images from 4401 patients across six cohorts (Fig. 1 and Table 1), using biparametric MRI (T2-weighted, highest b-value from the diffusion weighted imaging(DWI) and apparent diffusion coefficient (ADC)). Of these, 1404 patients constituted the development dataset, D, and were randomly split 80:20 into training and internal validation sets (“Methods”). This development dataset was used exclusively for model training and parameter tuning and not for testing. Model performance was evaluated on the five independent cohorts (C1–C5). The cohorts, C1 (n = 352) and C2 (n = 120), are internal test sets and do not overlap with the development set D, comprising a biopsy-confirmed cohort and a radical prostatectomy (RP) cohort with whole-mount pathology-based ground truth, respectively. The cohorts C3 (n = 1497) and C4 (n = 1154) are publicly available external datasets^23,27, and the cohort C5 (n = 292) is an external validation set. Table 1 presents an overview of PSA distributions, grade groups, and lesion characteristics across cohorts. Detailed information about patient selection criteria, clinical characteristics and imaging protocols can be found in the “Methods” section.

Table 1.

Patient and dataset characteristics across internal and external cohorts

Description	Internal cohort			External cohorts
	Development dataset (D)	Test (C1)	Test (C2)	PICAI (C3)	UCLA (C4)	UCSD (C5)
Source procedure	Biopsy	Biopsy	RP	Biopsy	Biopsy	Biopsy
Image modality	MRI	MRI	MRI	MRI	MRI	MRI
Population
Patients	1404	352	120	1473	760	292
PSA mean ± SD [quantile 25% - 75%]	9.7 ± 14.0 [5.0–10.5]	9.0 ± 6.9 [5.0–10.5]	9.1 ± 5.3 [5.6–11.0]	11.9 ± 15.0 [5.9–13.0]	8.9 ± 10.5 [4.6–10.0]	9.9 ± 12.6 [4.6–9.7]
Age mean ± SD [quantile 25% - 75%]	65.7 ± 7.4 [61–71]	65.2 ± 7.4 [60–71]	63.6 ± 7.9 [60–69]	65.6 ± 7.2 [61–70]	65.5 ± 7.5 [61–71]	68.8 ± 8.29 [62–74]
Scans	1404	352	120	1497	1154	292
GG = 0 (%)	609 (42.3%)	155 (44.0%)	1 (0.8%)	1091 (72.9%)	387 (33.5%)	139 (47.6%)
GG = 1 (%)	245 (17.0%)	43 (12.2%)	23 (19.2%)	-	286 (24.8%)	56 (19.2%)
GG ≥ 2 (%)	550 (40.7%)	154 (43.8%)	96 (80.0%)	406 (27.1%)	481 (41.7%)	43 (14.7%)
Lesions
Cancer	1137	271	-	436	972	-
Sign Cancer	730	187	-	232	622	114
GG = 1 (%)	407 (35.8%)	84 (31.0%)	-	204 (46.8%)	350 (36.0%)	-
GG = 2 (%)	354 (31.1%)	100 (36.9%)	-	138 (31.7%)	392 (40.3%)	50 (43.9%)
GG = 3 (%)	205 (18.0%)	55 (20.3%)	-	54 (12.4%)	120 (12.3%)	27 (23.7%)
GG = 4 (%)	97 (8.5%)	14 (5.2%)	-	20 (4.6%)	60 (6.2%)	17 (14.9%)
GG = 5 (%)	74 (6.5%)	18 (6.6%)	-	20 (4.6%)	50 (5.1%)	20 (17.5%)
MRI Characteristics
Prostate volume (mL) ± SD	56.7 ± 33.6	56.2 ± 34.2	28.7 ± 14.1	58.9 ± 31.6	48.5 ± 25.7	49.3 ± 38.3
Cancer volume (mL) ± SD	2.7 ± 13.6	2.4 ± 6.2	1.2 ± 3.4	1.51 ± 7.0	2.1 ± 6.8	1.1 ± 4.8
Number of axial slices	33.2 ± 4.4	33.4 ± 4.5	35.5 ± 4.2	20.0 ± 0.0	30.0 ± 1.3	32.5 ± 1.1

Architecture of the model

We developed ProViCNet, a prostate-specific model leveraging weakly supervised, patch-level contrastive learning with pseudo-labels, designed to detect and localize csPCa in MRI (Fig. 1a). We performed a lesion-level evaluation, where the prostate was divided into six regions (Fig. 1b and Supplementary Fig. 1a), and AUROC, AUPRC, sensitivity, and specificity were calculated using the 90th percentile of prediction values with thresholds determined during internal validation (“Methods”). ProViCNet employs a 3D-enhanced vision transformer pretrained on the DINOv2 model, coupled with patch-level contrastive learning that effectively distinguishes cancer tissue from normal tissue even near ambiguous lesion boundaries (Fig. 1d, “Methods”). Each MRI sequence is processed through a dedicated decoder to generate probability maps, which are then fused to capture complementary anatomical and functional details (Supplementary Fig. 2). Additional implementation details, including contrastive pair sampling and model training protocols, are provided in the Methods section.

Diagnostic performance of the AI model for csPCa

In the internal biopsy-confirmed test dataset C1, ProViCNet with MRI sequences achieved strong discriminative performance with an average AUROC of 0.923 and a high sensitivity of 0.895 while maintaining clinically relevant specificity to 0.778 (Fig. 2a). The corresponding Area Under the Precision-Recall Curve (AUPRC) was 0.879, indicating robust performance even with class imbalance. The RP cohort C2, which provided histopathology-derived ground truth labels, achieved an AUROC of 0.875, an AUPRC of 0.822, with a sensitivity of 0.819 and specificity of 0.730 for clinically significant cancer detection on MRI. The dice similarity coefficient (DSC) for both C1 and C2 was 0.425 and 0.389, respectively (Fig. 2b). Detailed metrics for these cohorts, including sensitivity, specificity, PPV, NPV, DSC, and accuracy, can be found in Supplementary Table 1.

Fig. 2. Performance evaluation of ProViCNet across multiple cohorts and comparison with board-certified radiologists.

a Performance metrics for csPCa detection on MRI across internal (C1, C2) and external (C3, C4, C5) cohorts. Bar plots show AUROC, AUPRC, sensitivity, specificity, and DSC with error bars indicating 95% confidence intervals. b Lesion-level receiver operating characteristic curves for four cohorts, with corresponding AUROC values and 95% confidence intervals. c Comparative analysis between ProViCNet and expert radiologists on 93 radical prostatectomy cases, showing statistical significance in AUROC (p < 0.01, DeLong’s test). d Representative case showing multi-parametric MRI prediction integration. Left: ground truth annotation; Right: model predictions from individual MRI sequences (T2-weighted, DWI, and ADC) and their fusion. Color map indicates cancer probability. e Example cases demonstrating varying model performance, arranged by increasing DSC (0.115 to 0.603). Top row: original T2-weighted images; bottom row: corresponding model predictions overlaid with ground truth annotations.

Qualitative analysis of the model predictions revealed the complementary nature of different MRI sequences (Fig. 2d). While T2-weighted images provided detailed anatomical information, DWI and ADC sequences contributed distinctive functional characteristics of the tissue. The integration of these complementary features enabled comprehensive MRI predictions. Representative cases with varying segmentation performance are shown in Fig. 2e, displaying the axial slice with the largest cancer extent for each case, illustrating the model’s performance across different scenarios from low to high DSC (0.115–0.603). Metrics for each individual MRI sequence in the C1 and C2 cohorts can be found in Supplementary Table 2. For the internal C1 cohort, the AUROC for T2, DWI, and ADC sequences was 0.899, 0.885, and 0.851, respectively. For the C2 cohort, the AUROC values for T2, DWI, and ADC sequences were 0.824, 0.866, and 0.827, respectively, showing slightly lower performance compared to the C1 cohort. This decrease in performance can be attributed to the use of histopathology-derived ground truth labels in the C2 cohort.

External validation performance for csPCa

The model was evaluated across multiple external cohorts. In cohort C3, the MRI model achieved its highest performance with an AUROC of 0.966 and AUPRC of 0.933, along with a sensitivity of 0.953 and specificity of 0.761 for clinically significant prostate cancer detection on MRI. The C4 cohort showed consistent performance with an AUROC of 0.920 and AUPRC of 0.854, maintaining a comparable sensitivity of 0.846 and specificity of 0.766. In cohort C5, which had the highest proportion of non-csPCa, the MRI model demonstrated a performance AUROC of 0.946, and the highest specificity 0.951 among all cohorts.

Detection performance for all prostate cancer lesions

The MRI model’s performance with respect to detecting all prostate cancers, including both indolent and clinically significant cases, is detailed in Supplementary Table 3. Upon including indolent cancers in the analysis, AUROC values showed modest improvements of 0.5% to 3.2% across cohorts, with the highest increase observed in the C2 cohort (AUROC 0.907 to 0.936). However, this broader detection scope resulted in decreased specificity across all cohorts due to increased false positive predictions, which was particularly notable in C4, where specificity dropped from 0.766 to 0.636, with an average decrease of 9.9% (range: 2.6-13.1%).

Comparative diagnostic performance of the AI and the radiologist

We compared the performance of ProViCNet with radiologists and PI-RADS scoring in a subset of 93 patients in cohort C2 with clinically significant cancer who underwent radical prostatectomy (Fig. 2c). The AI model demonstrated a significantly higher AUROC of 0.907 compared with the expert radiologist readers’ 0.805 (p < 0.01, DeLong’s test, PI-RADS ≥ 3). The AI model showed sensitivity and specificity of 0.880 and 0.654, respectively, while expert radiologist readers achieved 0.825 and 0.799 (PI-RADS ≥ 3). The DSC was 0.396 for the AI model and 0.347 for the expert radiologist readers. Additionally, diagnostic performances with PI-RADS are detailed in Supplementary Table 4.

Feature representation analysis

To evaluate the discriminative capabilities of learned features, we performed visualization analysis on the internal test cohort C1. Features were extracted from the pretrained vision transformer backbone of ProViCNet using small patches from T2-weighted MRI sequences. Up to 10 patches per label category were sampled within the prostate of each subject. The high-dimensional features were reduced to three components using Uniform Manifold Approximation and Projection (UMAP) for visualization. In the three-dimensional feature space, patches were color-coded according to their tissue labels: background (gray), normal prostate gland (green), non-csPCa (orange), and csPCa (red) (Fig. 3a). Visualization revealed distinct clustering patterns corresponding to different tissue types, suggesting that patch-level representation learning captured discriminative features for distinguishing normal prostate tissue, indolent cancer, and csPCa.

Fig. 3. ProViCNet feature representation analysis and application for screening to reduce unnecessary biopsies.

a Visualization of learned feature representations using dimensionality reduction. ProViCNet features extracted from image patches of the C1 cohort were projected into 3D space using UMAP dimensional reduction, showing clear separation between normal prostate tissue (green), indolent cancer (orange), and aggressive cancer (red) regions. b Receiver operating characteristic curves comparing ProViCNet’s performance (blue) against PSA screening (yellow) for detecting csPCa at the patient level (AUROC: 0.798–0.875 vs. 0.666–0.668, p < 0.001). c Performance metrics comparing PSA ≥ 4 ng/mL screening alone (left) versus PSA ≥ 4 ng/mL with AI assistance (right) for biopsy decision support. AI assistance maintained high sensitivity (0.937) while significantly reducing unnecessary biopsies through improved specificity (0.147 to 0.378, 157% increase).

In addition to presenting segmentation results, we performed feature-space visualization using principal component analysis (PCA) to examine feature patterns across different MRI sequences (Supplementary Fig. 3). The first three PCA components showed distinct spatial patterns for T2-weighted, DWI and ADC sequences within the prostate gland. While individual sequences showed some false positive regions, the final integrated prediction combining across sequences showed reduced false positive signals, particularly in distinguishing clinically significant cancer from normal prostate tissue.

Improving specificity in PSA-based biopsy screening with ProViCNet

Prostate-specific antigen (PSA) is a widely used tool for prostate cancer screening. PSA ≥ 4 ng/mL is the most commonly utilized threshold for recommending biopsy, primarily due to its high sensitivity. However, its low specificity results in a significant number of unnecessary biopsies in patients without csPCa. To address this limitation, we evaluated the ability of MRI-based ProViCNet predictions to distinguish patients with csPCa from those without, comparing its performance to PSA ≥ 4 ng/mL by analyzing lesion-specific maximum predicted values (Fig. 3b). Across the C1, C3, and C4 cohorts (comprising 259, 1456, and 983 patients with available PSA data, respectively), PSA achieved AUROCs ranging from 0.666 to 0.688. ProViCNet predictions yielded significantly higher AUROCs of 0.843, 0.875, and 0.798, respectively (p < 0.001, DeLong’s test), demonstrating its ability to distinguish between tissue types more effectively.

Next, we evaluated the potential of MRI-based ProViCNet predictions to enhance specificity among men with PSA ≥ 4 ng/mL while matching the sensitivity of the PSA ≥ 4 ng/mL threshold (Fig. 3c). Across the combined cohorts (C1, C3, and C4), the PSA ≥ 4 ng/mL threshold achieved a sensitivity of 0.937 but a specificity of only 0.147. By integrating ProViCNet’s AI predictions, specificity improved to 0.378, representing a relative increase of 157%, while sensitivity was maintained. This improvement also increased the overall accuracy from 0.319 to 0.500. These findings highlight the potential for combining MRI-based AI predictions with PSA screening to potentially reduce the number of unnecessary biopsies without compromising the diagnostic performance. We also assessed PSA density using a threshold selected to match the sensitivity of PSA ≥ 4 ng/mL, yielding nearly identical specificity and overall performance.

Comparison with existing segmentation models

We conducted a comprehensive comparative analysis between ProViCNet and eight established segmentation models to evaluate the relative performance in prostate cancer detection (Fig. 4a)^28–35. To ensure standardized comparison, all models were evaluated using only T2-weighted MRI sequences from the C1 cohort, without multi-sequence fusion. AUROC evaluation demonstrated that ProViCNet achieved superior performance (AUROC 0.899) compared with other models, with nnUNet showing the second-highest performance (AUROC 0.863). The remaining models achieved AUROC values ranging from 0.848 to 0.710, with particularly notable differences in performance in cases with small lesions and complex anatomical structures (detailed performance metrics in Supplementary Table 5).

Fig. 4. Comparative analysis of model performance with other segmentation models and its correlation with clinical variables.

a Comparison of model performance against other established deep learning architectures. Bar plots show AUROC values with 95% confidence intervals for nine different models evaluated on T2-weighted MRI sequences from the C1 cohort. ProViCNet achieves superior performance (AUROC 0.899) compared to other architectures. b Qualitative comparison of cancer probability predictions across different models on a representative case. Ground truth annotations (yellow outline) and predicted cancer probability maps are shown for each model, demonstrating ProViCNet’s improved cancer localization accuracy. Clinical correlates of model performance across different cohorts: c DSC stratified by lesion volume quartiles showing improved performance for larger lesions, d DSC across different GG demonstrating consistent performance across cancer grades, and e DSC stratified by PSA level quartiles indicating model performance is largely independent of PSA levels.

Lesion-level performance evaluation using DeLong’s test revealed significant differences in AUROC between ProViCNet and nnUNet (p < 0.001). This performance advantage was consistent across different prostate zones and tumor sizes. Probability heatmap visualization (Fig. 4b) showed the comparison between predicted cancer regions from different models on the same case used in Fig. 2d. Evaluation of T2-weighted MRI sequences revealed that ProViCNet exhibited higher performance in detecting clinically significant lesions (AUROC 0.899, sensitivity 0.774, specificity 0.874) compared with nnUNet (AUROC 0.863, sensitivity 0.476, specificity 0.974; Supplementary Table 6).

Morphological correlates of model performance

Quantitative analysis revealed significant correlations between model performance and morphological characteristics of prostate cancer (Fig. 4c–e). The DSC showed moderately positive correlations with cancer lesion volume (Spearman’s ρ = 0.514, p < 0.001) and grade group (ρ = 0.416, p < 0.001), while prostate volume did not show a significant correlation with DSC (ρ = 0.05, p = 0.620) in the C1 cohort. Analysis of lesion volume quartiles demonstrated a consistent trend across all cohorts, with larger lesions being associated with higher DSC. The model’s prediction confidence also showed positive correlation with lesion volume (ρ = 0.368, p < 0.001). This relationship between lesion size and detection accuracy was maintained across different grade groups, with particularly robust performance in higher-grade lesions.

Ablation study of model components

We performed a systematic ablation study using all MRI sequences to evaluate the contribution of each architectural component (Table 2). The baseline ViT architecture achieved an AUROC of 0.747 and showed a performance comparable to that of conventional architectures such as SwinUNet, UNet, and LeViTUNet (Supplementary Table 5)^33–35. Integration of the DINOv2 pre-trained weights substantially improved model performance (AUROC: 0.877), demonstrating the significant impact of transfer learning from vision foundation models. Alternative approaches, such as using frozen DINOv2 weights with Low-Rank Adaptation (LoRA), showed inferior performance (AUROC: 0.824)³⁶. While this performance exceeded that of ViT models trained without pre-trained weights, it suggests that some degree of backbone fine-tuning is necessary for optimal performance on downstream tasks. During fine-tuning, we found that applying a small learning rate (10%) to the backbone yielded optimal model performance. The 3D-enhanced positional embedding tokens further increased the AUROC to 0.918. The final model incorporating patch-level contrastive learning achieved the highest performance (AUROC: 0.930), demonstrating the cumulative benefit of each component.

Table 2.

Ablation study

Model name	DINOv2 backbone weights	Low-rank adaptation (LoRA)	Positional embedTokens (3D enhanced)	Patch-level contrastive learning	AUROC	Sensitivity	Specificity
ViT					0.747	0.635	0.758
DINOv2 ViT	✓				0.877	0.763	0.903
DINOv2 LoRA ViT	✓ (frozen)	✓			0.824	0.735	0.708
ProViDNet	✓		✓		0.918	0.883	0.702
ProViCNet (Ours)	✓		✓	✓	0.930	0.872	0.808

Discussion

Accurate detection and localization of clinically significant prostate cancer remains a critical challenge in clinical practice, impacting millions of men worldwide. In this study, we developed ProViCNet, a prostate-specific model leveraging weakly supervised, patch-level contrastive learning with pseudo-labels to effectively adapt foundation model principles to MRI data, demonstrating robust performance in detecting and localizing prostate cancer. Our extensive multi-institutional validation confirmed that ProViCNet outperforms both radiologists performing routine MRI interpretation and conventional AI methods. By combining domain-specific learning strategies with large-scale vision model training, ProViCNet adeptly distinguishes subtle lesion boundaries across imaging modalities, thus offering a promising avenue to enhance prostate cancer diagnosis and reduce inter-observer variability.

ProViCNet addresses several critical challenges in prostate cancer diagnosis. The model exceeds the performance of radiologists (AUROC 0.907 vs. 0.805, p < 0.01), with notably higher sensitivity (0.880 vs. 0.825, p < 0.001) in identifying csPCa. Given the increasing adoption of prostate MRI as a primary diagnostic tool, these performance improvements could be particularly valuable for clinical practice. The model generates lesion probability maps that could aid in biopsy targeting decisions. Additionally, when integrated with PSA screening, ProViCNet offers a practical approach for improving the current diagnostic paradigm. By increasing specificity from 0.147 to 0.378, while maintaining sensitivity of 0.937 at PSA ≥ 4 ng/mL, the model could potentially reduce unnecessary biopsies significantly, without compromising cancer detection rates. This improvement is especially relevant considering the pain, risks, psychological burden, and healthcare costs associated with unnecessary invasive procedures. Taken together, these evaluations go beyond aggregate metrics by demonstrating consistent performance across external cohorts, for multiple tasks related to cancer diagnosis, including avoiding unnecessary biopsies through MRI-guided screening, improving detection of all cancers for MRI-guided targeting, and effective treatment of the full extent of cancer for focal treatment.

The methodological advancements in ProViCNet contribute significantly to its robust performance. While conventional deep learning approaches have shown promise in prostate cancer detection, our approach effectively leverages pretrained vision models through a weakly supervised, patch-level contrastive learning strategy with pseudo-labels. This strategy allows the model to inherently focus on cancer-specific features, overcoming inherent uncertainties and limitations in radiologist-provided annotations, and thereby enabling more generalizable and discriminative feature learning. This strategy proved particularly effective, as demonstrated by our feature representation analysis and ablation studies, enhancing the model’s ability to distinguish clinically significant cancers from other prostate tissue. The integration of multiple MRI sequences through sequence-specific decoders allows for comprehensive capture of both the anatomical and functional characteristics of prostate tissue. Interestingly, direct self-supervised learning on prostate imaging did not yield significant improvements in cancer detection performance. We hypothesize that this limitation stems from the characteristics of prostate cancer imaging—the low frequency of cancer regions within images and their diffuse boundaries pose challenges for multi-view contrastive learning approaches like DINOv2, which typically benefit from clear object boundaries. Our findings suggest that while general vision foundation models provide valuable initialization, generating discriminative feature representations for subtle and sparse cancer regions within 3D medical images requires an approach closer to supervision. This is exemplified by our label-guided patch-level contrastive learning strategy, which effectively addresses the challenges of learning from ambiguous cancer boundaries and low tumor-to-background ratios typical in prostate imaging.

Recent large-scale efforts, including the PI-CAI challenge and specialized frameworks such as FocalNet, CorrSigNIa, and SPCNet, have made substantial progress in MRI-based prostate cancer detection ^12,23,24,37. ProViCNet builds upon these advances while exploring a different technical direction through the use of a vision foundation model and pseudo-label guided patch-level contrastive learning. Through the DINOv2-pretrained vision transformer architecture, ProViCNet effectively captures the contextual relationships essential for identifying sparse and indistinct cancer regions in prostate imaging. The 3D-enhanced positional embedding tokens further strengthen the model’s ability to learn volumetric structures, while the label-guided patch-level contrastive learning in the ViT backbone refines these embeddings, mitigating uncertainties from radiologist-defined lesion borders and enhancing generalization. Collectively, these design elements complement existing approaches by providing more robust feature representations, potentially enabling advanced downstream applications such as treatment outcome, recurrence and survival prediction.

Several limitations of our study should be considered. Our current 2D vision foundation model backbone with 3D positional encoding exhibits strong performance. However, this architecture may not be optimal for truly volumetric imaging modalities like ultrasound and CT, where depth information is as significant as width and height. Although studies suggest minimal performance differences between 2D and 3D approaches in MRI due to large inter-slice distances, future development of native 3D vision transformers could potentially enhance feature extraction from volumetric data. Nonetheless, the 2D backbone offers the advantage of being easily adaptable to 3D architectures and can be applied to a broader range of tasks³⁸. Second, while our study included multiple external validation cohorts, there were notable differences in clinical characteristics, particularly in the proportion of clinically significant cancers, across datasets. Additionally, scanner manufacturers varied significantly between cohorts—our internal cohort predominantly used GE scanners (84.55%; Supplementary Table 7), while most external datasets (C3, C4) were acquired exclusively using Siemens and Philips Medical Systems, though the UCSD cohort (C5) used GE scanners. Despite these variations in scanner manufacturers, patient characteristics, and evaluation approaches, our model’s robust performance across multiple cohorts demonstrates its potential generalizability in real-world clinical settings. Third, PI-RADS–based comparisons were performed in cohort C2, where ground truth cancer from whole mounts is available (see subsection “Ground truth cancer labels ”); yet such analysis was not possible in the other cohorts, where ground truth cancer labels are biopsy-confirmed radiologist labels. Additionally, although we demonstrated improved performance compared with conventional methods and radiologist interpretation, prospective clinical trials are needed to validate whether these improvements translate to better patient outcomes. In particular, our model provides probability heatmaps indicating regions most likely to contain cancer, potentially guiding more precise needle placement during biopsy procedures. Nonetheless, its impact on biopsy yield and clinical decision-making requires further investigation. Future work should include reader studies to quantify how ProViCNet’s assistance affects radiologists’ detection performance and its potential role in reducing unnecessary biopsies in real-world clinical settings.

In conclusion, ProViCNet represents a significant advancement in imaging analysis for prostate cancer detection, effectively adapting vision foundation model strategies to clinical imaging tasks. The model’s ability to process MRI data and provide interpretable cancer probability maps could enhance diagnostic precision and biopsy guidance. Future work should focus on prospective clinical validation to quantify its impact on radiologists’ performance and patient outcomes, ultimately establishing its role in improving clinical decision-making for prostate cancer diagnosis.

Methods

Study design and datasets

This retrospective multi-center study was approved by the Institutional Review Board at Stanford University (Protocol: IRB-44998), which waived the requirement for written informed consent. We analyzed MRI sequences in 1876 patients to develop and internally validate our model (Fig. 1c and Table 1). The development dataset, D (1404 scans from 1404 patients), was randomly split into training (80%) and internal validation (20%) sets. Model performance was evaluated on two internal test cohorts: a biopsy-confirmed cohort (C1, 352 scans from 352 patients) and a radical prostatectomy (RP) specimen cohort (C2, 120 scans from 120 patients).

For external validation, we used two public datasets and one independent institutional cohort. The public datasets included the Prostate Imaging: Cancer AI (PI-CAI) challenge dataset (C3, 1497 scans from 1473 patients) and the UCLA prostate cancer dataset (C4, 1154 scans from 760 patients) ^23,27. Additional external validation was performed using data from UCSD (C5, 292 scans from 292 patients). These external cohorts, representing different institutions and geographical regions, provided diverse patient populations and imaging protocols to evaluate model generalizability.

Ground truth cancer labels

The main goal of ProViCNet is to detect and localize the extent of clinically significant prostate cancer (csPCa), defined based on the International Society of Urologic Pathology (ISUP) grade group (GG) ≥ 2. In addition to csPCa, we also evaluated ProViCNet for its ability to detect all cancers (GG ≥ 1, including indolent cancer, GG = 1). Ground truth labels for cancer and csPCa are pathology confirmed for each cohort, with different confirmation approaches based on the available pathology information. For cohorts D, C1, C3-5, the cancer ground truth labels correspond to regions of interest (ROIs) outlined by radiologists as part of clinical care and having Prostate Image Reporting and Data System (PI-RADS) Scores ≥3, which are targeted during an MRI-ultrasound fusion-guided biopsy. Radiology lesions that show benign pathology are disregarded.

Moreover, to allow a direct comparison between ProViCNet and radiologist interpretation of MRI exams, we included cohort C2, which comprises patients who underwent radical prostatectomy and also have cancer ground truth labels derived from whole-mount histopathology images. We refer to this study as the Reader study and to the radiologist lesions with PI-RADS scores ≥3 as the Expert Reader. To generate cancer ground-truth labels in this cohort, we applied a commercially available AI model that identifies Gleason Patterns on digitized histopathology images (Deep Bio, Seoul, Republic of Korea)³⁹. The AI model has been extensively validated as part of our prior work^23,27, in a comparison with three pathologists that showed a correlation coefficient κ = 0.91 and sensitivity = 0.997 for cancer identification and a correlation coefficient of κ = 0.89 for grade-group classification⁴⁰. Moreover, predictions of this AI model that were outside the manual outlines of cancer provided by an expert pathologist (CAK, >10 years of experience reading prostate pathology images) were discarded. Lesions were considered to be csPCa if at least 1% of pixels had Gleason Patterns 4 or 5, otherwise the lesion was considered indolent. To map cancer ground truth labels from the whole mount histopathology images on MRI, we performed the following three steps: (1) we sectioned the excised prostate using patient-specific 3D printed molds to facilitate slice correspondences between T2 weighted MRI and whole mount histopathology⁴¹; (2) a radiology-pathology image expert (MR, >12 years prostate radiology-pathology expertise) identified slice correspondences between T2w MRI and histopathology images; (3) we used our MRI pathology registration framework to register the whole mount histopathology onto corresponding T2w MRI (validation data showed misalignment errors within 2–3 mm)⁴². The affine and deformable transformations derived from registering the pathology and MRI images were then applied onto the cancer labels to project them from histopathology onto MRI and create the ground truth cancer labels for patients in C2.

Baseline characteristics of patients and image datasets

For internal cohorts, MRI-ultrasound fusion-guided biopsy was performed using the Artemis System (Eigen Health, Grass Valley, California), equipped with Hitachi Ultrasound Devices. Suspicious lesions (PI-RADS scores ≥ 3) were targeted and projected onto ultrasound images via the fusion system.

Patient characteristics showed distinct patterns across cohorts (Table 1). The average of PSA levels ranged from 8.9 ± 10.5 ng/mL in the C4 cohort to 11.9 ± 15.0 ng/mL in the C3 dataset. The distribution of maximum ISUP grade group (GG) revealed distinct population characteristics: C2, which consists of the radical prostatectomy (RP) patients, showed the highest proportion of GG ≥ 2 (80.0%), reflecting the selection bias inherent in surgical candidates who typically have more aggressive disease. Meanwhile, the C4 cohort closely matched our development dataset (41.7% vs. 40.7% GG ≥ 2). In contrast, the C5 cohort demonstrated a notably lower frequency of GG ≥ 2 (14.7%), while the C3 cohort showed a moderate prevalence (27.1%). Among external cohorts, the C4 cohort had the highest mean cancer volume (2.1 ± 6.8 mL).

Image acquisition and preprocessing

The MRI protocols included T2-weighted, diffusion-weighted imaging (DWI comprised of b-values {800, 1200, 1400}), apparent diffusion coefficient (ADC) sequences, along with other sequences (e.g., Dynamic Contrast Enhanced). For our internal cohorts for model development and test, MRI examinations were predominantly (84.55%) performed on 3 T scanners (GE Healthcare, Chicago, IL) with a surface coil (Supplementary Table 7).

ProViCNet takes three inputs: the T2-weighted MRI, the highest b-value DWI sequence and the ADC. For standardization across institutions, T2-weighted MRI sequences served as the reference for spatial normalization. All MRI sequences were resampled to a standardized voxel spacing of 0.5 × 0.5 × 3.0 mm, where 0.5 × 0.5 mm corresponds to the in-plane axial resolution and 3.0 mm corresponds to the out-of-plane distance between slices. Images were center-cropped to 256 × 256 pixels in the axial plane. Image intensities were normalized using the mean and standard deviation calculated from prostate-specific regions. All image processing was performed using the SimpleITK library on NIfTI format (nii.gz) data.

ProViCNet architecture

ProViCNet is a prostate-specific foundation model designed to detect cancer locations and distinguish csPCa from multi-parametric MRI sequences of prostate cancer patients (Fig. 1a). It consists of several key components (Fig. 1d): (1) input data preprocessing, which utilizes three consecutive axial slices to maintain spatial context; (2) a ViT backbone pretrained using the vision foundation model DINOv2¹⁹; and (3) weakly supervised learning with patch-level contrastive learning guided by radiologist annotations. The 3D-enhanced vision transformer incorporates the relative axial positions of consecutive slices through positional embedding, creating a unified token representation that preserves spatial relationships. The patch-level contrastive learning strategy enhances the model’s ability to differentiate cancer tissue from normal tissue through careful pair selection. Positive pairs are created from patches sharing similar characteristics (either both cancer or both normal tissue, with ≥95% overlap), while negative pairs contrast cancer patches with normal tissue patches. To account for potential annotation uncertainties at cancer boundaries, normal patches in negative pairs are sampled at least one patch width away from cancer regions. This mitigates the impact of inherent limitations in radiologist-defined boundaries. This approach also creates robust feature representations that are less sensitive to annotation ambiguities while maintaining strong discriminative power. The model processes three MRI sequences (T2-weighted, DWI, and ADC) independently through sequence-specific decoders that generate pixel-level probability maps for the prostate and cancer. These MRI features are integrated through a multi-sequence fusion module to enable comprehensive detection and assessment of the prostate gland, cancer, and csPCa (Supplementary Fig. 2).

Patch-level contrastive learning

We implemented a patch-level contrastive strategy to reinforce the model’s discriminative capacity while accommodating uncertainties in radiologist-defined lesion boundaries (Fig. 1d). Each token from the ViT final feature map corresponds to a 14 × 14 pixel region in the original image, and we superimpose the ground-truth label to determine whether that token is dominantly cancerous or non-cancerous. Specifically, for a patch ρ we define

$${\mathrm{\rho }}_c\left(p\right)=\frac{N_c\left(p\right)}{N},{\mathrm{\rho }}_g\left(p\right)=\frac{N_g\left(p\right)}{N}$$ 1

where $N_c\left(p\right)$ and $N_g\left(p\right)$ represent the numbers of cancer and prostate-gland pixels (respectively) within patch $p$ and $N$ is the total number of pixels in that patch. A patch is classified as cancer if ${\mathrm{\rho }}_c\left(p\right)\ge 0.95$ or normal gland if ${\rho }_g\left(p\right)\ge 0.95$.

During training, anchor patches are drawn from strongly cancer-positive areas (i.e., high ${\mathrm{\rho }}_c$), then positive pairs are formed with spatially adjacent patches that share a similarly high ${\rho }_c$. Meanwhile, negative pairs are formed by comparing anchor patches to patches with a minimal cancer proportion (i.e., ${\mathrm{\rho }}_g\left(p\right)\ge 0.95$) located at least one patch-distance away. This ensures that boundary regions—where labeling may be uncertain—are excluded from negative pairs, thereby reducing mislabeled examples.

Let $f_a$ and $f_t$ be the feature embeddings (extracted by a contrastive projection head) for an anchor patch $p_a$ and its target patch $p_t$. We compute the cosine similarity $s$ as:

$$s\left(f_a,f_t\right)=\frac{f_a\cdot f_t}{\mathrm{\Vert }{f}_a\mathrm{\Vert }\mathrm{\Vert }{f}_t\mathrm{\Vert }}$$ 2

where $\Vert f\Vert$ denotes the $L_2$ norm, defined as:

$$\mathrm{\Vert }f\mathrm{\Vert }=\sqrt{{\sum }_i{f}_i^2}$$ 3

The patch-level contrastive loss $L_c$ is computed based on the cosine similarity between feature embeddings of anchor and comparison patches. Positive pairs (spatially adjacent cancer regions) are encouraged to have higher similarity, while negative pairs (regions with minimal cancer proportion) are pushed apart. The loss is defined as follows:

and the patch-level contrastive loss $L_c$ follows:

$${\mathrm{L}}_{\mathrm{c}}=\left\{\begin{array}{cc}1-\mathrm{s}\left({\mathrm{f}}_{\mathrm{a}},{\mathrm{f}}_{\mathrm{t}}\right),& \left(\mathrm{positive\; pair}\right)\\ \max \left(0,\mathrm{s}\left({\mathrm{f}}_{\mathrm{a}},{\mathrm{f}}_{\mathrm{t}}\right)-m\right)& \left(\mathrm{negative\; pair}\right)\end{array}\right\}$$ 4

where $m$= 0.5 is a margin threshold ensuring that negative pairs do not become too similar. Specifically, when the cosine similarity of a negative pair exceeds $m$= 0.5, a penalty is applied, encouraging their embeddings to be more distinct in feature space.

While positive pairs were constructed using spatially adjacent patches, negative pairs were initially sampled from neighboring regions. However, during loss computation, a distant negative selection strategy was applied, where negative pairs were selected from patches at least one additional patch-distance away. This adjustment ensures that negative pairs used for contrastive learning originate from more spatially distinct regions, mitigating the potential influence of subtle annotation uncertainties at cancer boundaries while preserving the integrity of the original spatial relationships.

To further encourage high-fidelity embeddings, we employ a projection head that expands each 384-dimensional ViT token embedding into a higher-dimensional vector with 65,536 dimensions via:

$$h=\mathrm{MLP}\left(x\right),h^{\prime }=\frac{h}{\mathrm{\Vert }h\mathrm{\Vert }},z=W{h}^{\prime }$$ 5

where $x$ is the ViT output token, “$\text{MLP}$” is a three-layer perceptron with batch-normalization and GELU activation, and $W$ is a weight-normalized linear transformation. The feature vector $h$ is normalized to unit length using its $L_2$ norm, ensuring stable training and improved feature discrimination. This high-dimensional projection fosters fine-grained discrimination between subtle normal–cancer differences, while the normalization layers stabilize training. The patch-level contrastive loss is then combined with standard segmentation loss (e.g., cross-entropy or Dice) in an end-to-end fashion:

$$L_{\text{final}}=\left(1-\mathrm{\alpha }\right)L_{\text{seg}}+\mathrm{\alpha }{L}_{\text{contrastive}}$$ 6

where $\mathrm{\alpha }$ controls the trade-off between the losses and was set to 0.05 in our experiments. This hybrid objective ensures robust localization of lesions (via segmentation) while learning more discriminative features for cancerous vs. non-cancerous tissue under imperfect boundary annotations. Full implementation details, including code for sampling patch pairs and under-sampling negative vs. positive pairs to avoid class imbalance, are provided in the Supplementary Methods.

Feature visualization and analysis

To visualize learned feature representations, we employed both global and local visualization strategies. Patch-level features were spatially interpolated from the model’s native patch resolution to match the original image dimensions, enabling direct comparison with input images. For analyzing the distribution of features across different tissue types, we extracted patches from all patients and applied UMAP for dimensionality reduction⁴³. This allows visualization of the relationships between normal tissue, indolent cancer, and aggressive cancer patches in a common embedding space⁴³.

To examine feature distributions specifically within prostate tissue, we focused on patches contained entirely within the prostate gland. These features were analyzed using PCA, with the top three components visualized using a jet colormap to highlight spatial patterns of learned features. This approach revealed distinct organizational patterns of features between normal and cancerous regions while maintaining anatomical context.

Model training and optimization

We employed a multi-task learning strategy with carefully controlled learning dynamics to balance feature adaptation and cancer detection. The ViT backbone, initialized with DINOv2 pre-trained weights, was fine-tuned at a reduced learning rate (10% of the base rate) to preserve the foundational visual features while allowing adaptation to prostate-specific characteristics. This differential learning rate strategy proved crucial for maintaining the generalization capabilities of the vision foundation model while enabling domain-specific optimization.

The model was trained end-to-end using the Adam optimizer with an initial base learning rate of 2 × 10⁻⁴ and weight decay of 1 × 10⁻⁵. The loss function combined segmentation loss with patch-level contrastive learning loss at a ratio of 9:1, allowing the model to focus primarily on accurate cancer detection while benefiting from the improved feature representations induced by contrastive learning. Each training batch consisted of 32 sets of three consecutive axial slices, with training continued until convergence, typically requiring 100 epochs. The best model was selected based on the average AUROC in the internal validation dataset. The model was implemented in PyTorch (version 2.0.0).

For comparative analysis, eight established segmentation models (nnUNet, UNet, UNet++, Swin-UNet, TransUNet, LeViT-UNet, MISSFormer, and UCTransNet)^28–35 were trained and evaluated using the T2-weighted MRI sequences from the same training dataset as ProViCNet. All models were trained with the same loss function as ProViCNet.

Evaluation metrics

Model performance was evaluated using lesion-level metrics as previously described by us¹², and summarized next. The prostate was split into six distinct regions (referred to as sextants) to create a balance of positive and negative samples, as cancer is present in only as many as 5% of the pixels in positive patients. To create the sextants, the prostate is split into left and right sides, and subsequently apex, midgland and base within the axial view following the transrectal biopsy template (see Fig. 1b). For negative regions (without cancer), the region-level probability was defined as the 90th percentile of prediction probability values across all pixels within the region. For positive samples, the score was defined as the 90th percentile of prediction probability values computed only within the ground-truth cancer mask in that region (Supplementary Fig. 1a). Our evaluation ensures that ground truth lesions are detected if at least a 0.10 DSC is observed between the predicted lesion and ground truth lesion, which is consistent with prior studies from our team¹² and others^23,27. This sextant level split is particularly important in patients in cohort C2, where all patients have cancer and undergo radical prostatectomy, as no negative patients (without cancer) will undergo such a procedure. This lack of negative samples at the patient level limits our ability to compute metrics, such as specificity at the patient level. AUROC, AUPRC, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy were computed at the lesion level using sextants for each patient and are then aggregated at the population level by averaging.

Sensitivity, specificity, PPV, NPV, and accuracy were also calculated, using the best threshold determined from internal validation on the developmental set (Supplementary Fig. 1b). The same evaluation procedure was applied to radiologist assessments for comparison. Clinically significant prostate cancer (csPCa) was defined as GG ≥ 2. We also evaluated model performance stratified by lesion volume and GG to assess the impact of tumor characteristics on detection accuracy. The DSC was used to assess spatial overlap at the patient level, computed between the predicted mask and ground-truth cancer outlines.

Statistical analysis

Statistical analyses were performed using Python (version 3.8.19). Confidence intervals (95%) for AUPRC, sensitivity, specificity, and DSC were estimated via patient-level non-parametric bootstrap resampling. DSC was summarized per patient. For AUROC, probability scores and labels from all patients were concatenated to estimate 95% confidence intervals using DeLong’s method and to perform paired AUROC comparisons using DeLong’s test across models, radiologists, PSA, and PSA + AI. Correlations between performance and morphological characteristics were assessed using Spearman’s rank correlation. P values < 0.05 were considered statistically significant.

Development of combined PSA-AI screening model

To improve screening specificity while maintaining the sensitivity of PSA ≥ 4 ng/mL, we developed a stacking ensemble model using a logistic regression model to integrate the binary PSA threshold status with AI-derived predictions. The model was formulated as:

$$\text{Outcome}\sim {\beta }_0+{\beta }_1\left(\text{PSA}\ge 4\frac{\mathrm{ng}}{\mathrm{mL}}\right)+{\beta }_2\left(\text{AI}\text{prediction}\right)$$ 7

where Outcome is a binary variable indicating the presence (1) or absence (0) of clinically significant prostate cancer, $\text{PSA}\ge 4$ ng/mL is a binary indicator of PSA status, and AI prediction represents ProViCNet’s predicted probability of csPCa. The coefficients ${\beta }_{\mathit{0}}$, ${\beta }_1$, and ${\beta }_2$ were estimated using logistic regression. The model was trained and evaluated on 2698 patients with available PSA data from cohorts C1 (n = 259, with 236 having PSA ≥ 4 ng/mL), C3 (n = 1456, with 1325 having PSA ≥ 4 ng/mL), and C4 (n = 983, with 794 having PSA ≥ 4 ng/mL). This stacking architecture enables bidirectional risk reclassification by leveraging complementary information from both clinical biomarkers and AI-derived imaging features. The ensemble approach allows for (1) identification of high-risk cases with PSA < 4 ng/mL through strong AI predictions, and (2) reclassification of PSA ≥ 4 ng/mL cases as low-risk based on AI predictions. We calibrated the model’s decision threshold to match the sensitivity of conventional PSA ≥ 4 ng/mL screening and evaluated performance through standard diagnostic metrics, including sensitivity, specificity, PPV, NPV, and overall accuracy.

Author contributions

J.H.L. conceived and conducted the study, designed the machine learning models, and wrote the manuscript. C.X.L., H.J., I.B., S.V., L.Z., and S.S. contributed to data preprocessing. Y.S. and T.M.S. supported external validation. C.K. performed pathology image validation. M.H.C., S.J.C.S., S.R.Z., and E.R.S. performed data annotation. R.F. and P.G. contributed to data collection. G.A.S. assisted with data collection and annotation. M.R. supervised the project and provided critical revisions to the manuscript. All authors reviewed and approved the final version of the manuscript.

Data availability

The PI-CAI (C3) and UCLA (C4) datasets used for external validation are publicly available through their respective data portals. Due to privacy regulations, the internal cohort data from Stanford University and external validation data from UCSD (C5) are not publicly available. However, qualified researchers may request access to the internal dataset for academic purposes through appropriate institutional data sharing agreements. Example data and trained model weights sufficient to reproduce our main findings are available in the code repository.

Code availability

The complete implementation of ProViCNet (initial commit January 2025) is freely available at https://github.com/pimed/ProViCNet. All analyses were performed using Python version 3.8.19. The deep learning models were developed using PyTorch version 2.0.0, with additional dependencies including SimpleITK for image processing and matplotlib version 3.7.5 for visualization. Detailed documentation, including model architecture specifications, training protocols, and inference procedures, is provided in the GitHub repository README file. The source code is released under the MIT license to encourage broad academic and research use. The pre-trained model weights are accessible through the Hugging Face model repository (https://huggingface.co/pimed/ProViCNet).

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41746-026-02831-y.