From Pixels to Prognosis: A Survey on AI-Driven Cancer Patient Survival Prediction Using Digital Histology Images

Abstract

Survival analysis is an integral part of medical statistics that is extensively utilized to establish prognostic indices for mortality or disease recurrence, assess treatment efficacy, and tailor effective treatment plans. The identification of prognostic biomarkers capable of predicting patient survival is a primary objective in the field of cancer research. With the recent integration of digital histology images into routine clinical practice, a plethora of Artificial Intelligence (AI)-based methods for digital pathology has emerged in scholarly literature, facilitating patient survival prediction. These methods have demonstrated remarkable proficiency in analyzing and interpreting whole slide images, yielding results comparable to those of expert pathologists. The complexity of AI-driven techniques is magnified by the distinctive characteristics of digital histology images, including their gigapixel size and diverse tissue appearances. Consequently, advanced patch-based methods are employed to effectively extract features that correlate with patient survival. These computational methods significantly enhance survival prediction accuracy and augment prognostic capabilities in cancer patients. The review discusses the methodologies employed in the literature, their performance metrics, ongoing challenges, and potential solutions for future advancements. This paper explains survival analysis and feature extraction methods for analyzing cancer patients. It also compiles essential acronyms related to cancer precision medicine. Furthermore, it is noteworthy that this is the inaugural review paper in the field. The target audience for this interdisciplinary review comprises AI practitioners, medical statisticians, and progressive oncologists who are enthusiastic about translating AI-driven solutions into clinical practice. We expect this comprehensive review article to guide future research directions in the field of cancer research.

Introduction

Cancer, a complex disease with no definitive cause, has witnessed significant advancements in research in recent decades [1]. Diagnosis, prognosis, and treatment play crucial roles in effective cancer management [2]. Prognostic information provided during the initial diagnosis holds significant importance for guiding treatment decisions and for the subsequent monitoring process [3, 4]. Prognosis refers to predicting outcomes such as tumor recurrence, metastasis, or mortality based on clinical findings [5]. Multiple factors, known as prognostic factors, influence the prognosis of cancer, including grade, metastasis, tumor characteristics, lymph node involvement, biomarkers, genetic mutations, age, and treatment options [6, 7]. Identifying prognostic biomarkers, including histopathological, genetic, gene expression, and protein markers, is a major goal in cancer precision medicine [8]. whole slide images (WSIs) have a crucial role in the assessment of histopathological biomarkers and are widely recognized as the benchmark in this domain [9].

Active research efforts are underway in the field of computational image analysis of whole slide images (WSIs) utilizing machine learning and deep learning algorithms [10]. The ongoing research domain aimed at improving survival predictions by learning survival informative features associated with patient outcomes [11]. Deep learning has achieved remarkable success in addressing various challenges in computer vision, such as computational pathology [12]. In fact, models achieve performance comparable to expert pathologists [13, 14]. However, extracting survival-related features from gigapixel WSIs poses challenges due to their size and diverse tissue appearances [15]. As a result, Patch-based techniques have emerged to tackle these obstacles and identify meaningful patches linked to the advancement of cancer in whole slide images (WSIs).

Many computational methods have been developed over time for the automated analysis of large whole slide images, aiming to establish connections between the extracted morphological and histopathological characteristics and a patient’s survival. These computational methods can be broadly stratified into two major sections. The first group of algorithms leverages intermediate computer vision tasks for feature extractions. These methods require survival informative ROI annotations. However, manual patch-level annotations are labor-intensive and subject to pathologist’s interpretation [16]. The second group of algorithms uses direct feature extraction methods without intermediate computer vision tasks. These algorithms have been designed to automatically extract features from gigapixel WSIs without prior knowledge of tissue regions or appearance characteristics [17]. Various strategies exist in the literature for patch selection and aggregation techniques to extract informative features related to patient survival using learning algorithms [18, 19].

Prognosis assessment in oncology relies on the statistical analysis of data collected over many years from a cohort of cancer patients. Predicting the time until certain events occur, such as cancer recurrence or mortality, significantly influences decision-making within the field of oncology [20]. The field of survival analysis is of great importance to patients, clinicians, researchers, and policymakers [21, 22]. Different statistical metrics can be employed to assess survival duration, encompassing disease-specific, relative, overall, and disease-free survival. The extensive body of research encompasses various approaches, with examples like Kaplan-Meier (KM) plots [23], log-rank tests, and Cox (proportional hazards) regression [24], to assess and measure survival statistics linked to characteristics extracted from whole slide images (WSIs).

The main objective of this survey paper is to offer an extensive review and summary of the existing research on the application of artificial intelligence for predicting the survival of cancer patients by analyzing digital histology images. This paper aims to demonstrate how AI-powered algorithms, leveraging whole slide imaging (WSI), can be used to predict survival outcomes based on whole slide images. This paper discusses the methodologies used in the literature, evaluates their performance metrics, identifies ongoing challenges, and suggests potential solutions for future advancements. This comprehensive review is targeted toward individuals in the realms of artificial intelligence, and healthcare, encompassing both researchers and practitioners. It also targets medical professionals, oncologists, biomedical informatics experts, and computational scientists involved in analyzing digital histology images. This survey will also provide value to individuals within academia, students, healthcare decision-makers, professionals in the healthcare technology industry, and researchers engaged in related domains like precision medicine and computational pathology. Additionally, it is relevant to the general scientific community interested in the intersection of AI, digital pathology, and cancer prognosis. We have created Table 1 that describes frequently utilized terminologies to provide help and guidance.

Table 1.

Compilation of acronyms and abbreviations employed in the paper

Acronyms	Words	Acronyms	Words
TIL	Tumor-infiltrating lymphocyte	TMA	Tissue microarray
DFS	Disease-specific survival	OS	Overall survival
OSCC	Oral squamous cell carcinoma	ccRCC	Clear cell renal cell carcinoma
GBM	Glioblastoma	LUSC	Lung squamous cell carcinoma
DCIS	Ductal carcinoma in situ	KICH	Kidney chromophobe
TMB-H	Tumor mutational burden-high	BCRA	Breast invasive carcinoma
BLCA	Bladder carcinoma	ADC	Antibody-drug conjugates
LGG	Low-grade glioma	LUAD	Lung adenocarcinoma
PRAD	Prostate adenocarcinoma	NLST	National Lung Screening Trial
COAD	Colon adenocarcinoma	STAD	Stomach adenocarcinoma
LIHC	Liver hepatocellular carcinoma	OV	Ovarian carcinoma
READ	Rectum adenocarcinoma	HCC	Hepatocellular carcinoma
HNSC	Head-neck squamous cell carcinoma	ICC	Intrahepatic cholangiocarcinoma
KIRP	Kidney renal papillary cell carcinoma	KIRC	Kidney renal clear cell carcinoma

This review article is organized into seven distinct sections. In the “ Introduction” section, there is a concise exploration of the significance behind enhanced prognostics through survival analysis, alongside the contribution of deep learning in advancing survival prediction. Additionally, the scope of this review is outlined. The “Survey Methodology” section delves into the methodology employed for selecting the articles under review, as well as their distribution across various years, conferences, and journals. Moving on to the “Features Extraction Techniques” section, the fundamental understanding of computational techniques for medical professionals is established. The “Cultivating the Constructs of Survival Analysis” section demonstrates the techniques available in the literature for quantification and evaluation of survival statistics. In the “Compendium of Computational Methods for Predicting Cancer Patients’ Survival” section details about the reviewed techniques are categorized based on methods adopted for feature extraction. It also describes the performance of these methodologies in tabular format. The “Discussion” section highlights limitations in the reviewed articles and discusses the findings and future direction. Finally, the “Conclusion” section concludes the contribution of this review article, research gaps, findings, and the pathway for future research.

Scope of the Review

The objective of this review is to provide a thorough overview of the present research state regarding the application of artificial intelligence in predicting the survival of cancer patients by analyzing digital histology images Notably, there is a dearth of existing review papers addressing this specific topic. Our primary objective is to summarize the various methods, challenges, and advancements in this field, laying the groundwork for future research to improve the accuracy of prognosis and the effectiveness of cancer treatment.

To accomplish this objective, we conducted an extensive literature search across reputable peer-reviewed journals and prominent conferences, utilizing platforms such as PubMed, Springer, IEEE Xplore, and Google Scholar. We meticulously curated literature published between 2018 and 2022, scrutinizing the methodologies employed in each paper. Alongside showcasing innovative techniques discussed in these works, we also pinpoint areas requiring further investigation and potential avenues for future exploration. In addition to shedding light on prevalent methodologies, we identify research gaps and propose directions for future inquiries. Remarkably, no prior reviews have been found on this exact topic, possibly making this paper the inaugural review on survival analysis of cancer patients. We anticipate that this review will effectively bridge the gap between the computational community and medical experts, creating a conducive environment for subsequent research and development in the field of survival analysis of cancer patients using whole slide images (WSIs).

Our target audience for this interdisciplinary review includes AI practitioners, medical statisticians, and forward-thinking oncologists interested in integrating AI-powered solutions into clinical practice. The ultimate goal is to provide invaluable insights and guidance, thereby influencing the direction of future research in the realm of precision medicine for cancer.

Survey Methodology

This section outlines the criteria used to select relevant research articles for this review paper. Furthermore, it demonstrates how the reviewed articles are categorized based on their publication venues (journals or conferences) and publication years.

Papers Selection

We conducted a comprehensive search for papers on various platforms such as PubMed, Springer, IEEE Xplore, and, finally, Google Scholar. Our search resulted in a total of 4160 papers. Of these, 2100 were excluded due to duplication or their focus on other computational pathology matters unrelated to survival analysis. Subsequently, we rigorously reviewed the remaining 2060 papers and identified 260 articles that met the criteria of relevance for this survey. The other 2000 papers were found to fall short of legitimacy criteria, as some exclusively addressed survival prediction or prognosis, unrelated to cancer patients using histopathology images. Among the remaining 260 articles, a further assessment was conducted to ensure legitimacy. Consequently, 200 papers were excluded, either due to their focus on extracting features beyond histological attributes (e.g., gene mutation and protein biomarkers) or their application of techniques outside the realm of deep learning or machine learning, such as relying solely on expert pathologists for problem-solving. The research articles included in this review are outlined using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework. The selection process of the research articles, as depicted in Fig. 1, is summarized by PRISMA.

Fig. 1.

PRISMA—Visualization of the process for selecting research articles in the review

Papers Acquisition Method

To gather research articles for this review, we conducted searches across multiple platforms including PubMed, Springer, IEEE Xplore, and Google Scholar. Our focus was on selecting documents from peer-reviewed journals and conference proceedings. Employing diverse keywords relevant to this review, we employed logical operators such as ’AND’ and ’OR’ to enhance search accuracy and yield more precise results. The keywords employed encompassed:

• Whole slide images, whole slide images, WSI, histopathology images, histology images

• Survival analysis, survival prediction, outcome prediction, prognosis

• Artificial intelligence techniques, machine learning, deep learning, and convolutional neural networks (CNNs)

We derived the keywords from all the included research articles encompassed within our review, and these were utilized to construct a word cloud, visually represented in Fig. 2. The word cloud visually encapsulates the frequently encountered terms within the body of literature concerning the prediction of cancer patients’ survival through the utilization of whole slide images. Given our principal objective of comprehensively presenting the existing body of literature in the domain of survival prediction leveraging whole slide images, alongside the extraction of histological attributes through the machine and deep learning algorithms, the word cloud prominently features interconnected terms including histopathology, survival analysis, WSI, tumor analysis, morphological features, and other relevant terms.

Fig. 2.

A graphic showing the most frequent keywords from articles under review

The initial stage of paper selection involved scrutiny of paper titles, with matches against the predefined criteria determining selection. In cases where the title did not meet these criteria, we proceeded with a more in-depth assessment by reviewing the abstract, conclusion, and model diagram to guide our ultimate choice. Details outlining the criteria for paper inclusion and exclusion can be found in Table 2.

Table 2.

Criteria for the inclusion and exclusion of legitimate papers

Inclusion criteria	Exclusion criteria
Articles focused on utilizing whole slide images to predict the survival outcomes of cancer patients	Articles that primarily deal with predicting survival but do not specifically concentrate on cancer patients or the application of whole slide images.
Articles that use machine learning and deep learning techniques for survival prediction	Articles that are not using features through expert pathologists
Articles that work on whole slide images with or without other data types such as gene expression and DNA sequencing	Articles exclusively focusing on data types other than whole slide images, such as gene expression and DNA sequencing
Articles from Journals and Conferences mentioned in Fig. 4	Articles that are not peer-reviewed

Papers Distribution

This section provides an overview of how published papers are distributed across different journals, conferences, and years. Its purpose is to offer a summary of the articles featured in this review, shedding light on the volume of literature available in peer-reviewed publications and the impact of the research.

Figure 3 depicts how the reviewed articles are spread out over different years. The graph visually illustrates the progressive expansion of the literature, on survival prediction utilizing whole slide images and artificial intelligence, from 2018 to 2022. Similarly, the analysis reveals an increasing body of work on survival analysis of cancer patients through histological features extracted from whole slide images, beginning in 2018 and particularly flourishing in 2022, during which 20 research articles were published.

Fig. 3.

An orderly listing of survival analysis research publications using cancer patients’ WSIs

Figure 4 illustrates the number of included research articles about survival prediction using whole slide images across various journals and conferences. Each venue is represented by an arrow in the same color as the corresponding segment in the pie chart. The statistics depicted in the graph indicate that the majority of papers, a total of 8 articles, were sourced from MICCAI. Additionally, 6 papers were sourced from Nature Scientific Reports.

Fig. 4.

Distribution of included research articles among journals and conferences according to publication venue

Features Extraction Techniques

Effectively identifying and extracting meaningful features from whole slide images (WSIs) holds significance in constructing survival prediction models for cancer patients. These features encompass details about image structure, dimensions, morphology, texture, as well as the arrangement of distinct cell categories [25]. There are multiple techniques that can be used for feature extraction. These feature extraction techniques range from simple handcrafted feature extraction methods like thresholding to more complex methods like deep learning-based approaches. Using the extracted characteristics, various methods are utilized to predict survival outcomes, spanning from traditional statistical techniques to state-of-the-art machine learning and deep learning approaches. The most relevant features correspond to heightened prediction precision, ultimately enhancing the quality of patient care. This section introduces the commonly used techniques in medical computer vision for extracting survival informative features from WSIs.

Handcrafted Feature Extraction

Handcrafted feature extraction implies the manual depiction and extraction of features from data. In the context of whole slide images (WSI), this procedure involves identifying relevant features such as dimensions, morphology, texture, size, tissue structures, cells, patterns, and the arrangement of elements within a WSI [26]. This method of feature extraction requires domain knowledge to discern the most important features. Although handcrafted features were extensively utilized in the past, Recent developments in convolutional neural networks (CNNs) and deep learning have significantly reduced their need. However, in some cases, handcrafted features might still have value, especially when dealing with limited data or when specific domain knowledge is crucial for interpreting the results. In modern digital pathology, a combination of both handcrafted features and deep learning techniques can be applied to leverage the strengths of each approach.

Artificial Neural Networks (ANN)

In the recent era of technological advances, all manual tasks are being automated by Machine learning. Artificial Neural networks [27] constitute a subset of machine learning methodologies designed to discern concealed patterns and construct representations of the information contained within input data. Learning algorithms [28] learn by seeing the data, predicting the output, evaluating the output, and updating the learned information accordingly, this process repeats until it builds the representations present in the data.

Convolutional Neural Network (CNN)

CNN, which stands for Convolutional Neural Network (CNN) [29], is a specialized neural network architecture created for the purpose of analyzing and processing data in a grid-like format, such as images and videos. This type of network has demonstrated significant effectiveness in core computer vision tasks, such as the classification of images, detection of objects, segmentation of regions in images, and more.

In the CNN, an input image is utilized to learn distinctive features through the assignment of weights. This is accomplished by the CNN as it goes through multiple layers and employs diverse filters, resulting in the extraction of discriminative features. CNN arranges convolutional layers, pooling layers, and fully connected layers in specific sequences to create various architectures. In recent years, a multitude of CNN architectures, including the likes of AlexNet, have been developed and refined [30], VGGNet [31], GoogleNet [32], ResNet [33], ResNeXt [34], Squeeze & Excitation Net [35], DenseNet [36], and EfficientNet [37].

Attention Mechanism in CNN

In computer vision, attention mechanism refers to the process of selecting relevant features from the image, that a learning algorithm should focus on. This concept mirrors the cognitive attention observed in human perception [38]. In the context of CNNs, the deeper layers are confined to perceiving only what has been conveyed by the preceding layers. Consequently, they might miss out on the comprehensive context of the features. The attention mechanism is employed to capture image regions that demand the CNN’s focus and subsequently transmit them to the deeper layers

Cultivating the Constructs of Survival Analysis

The need for survival analysis arises when studying events that occur over time and vary based on changing conditions or circumstances. This form of analysis finds applications in various realms of data examination, including the field of medicine. It involves employing a range of statistical methods to explore the duration it takes for a particular event of significance to occur or become noticeable. The overarching term used to characterize this phenomenon is “survival time,” which is employed in a broad context. It signifies the duration a patient being monitored endures without experiencing the event of interest. This event could encompass scenarios such as the probability of a patient’s tumor recurring, the likelihood of distant metastasis occurring, or the likelihood of mortality. Typically, the event of interest is not documented for all individuals by the end of the observation period, and their time until the event remains undetermined. Further survival data contain more early events than late events, which makes it hugely imbalanced and data is rarely normally distributed rather it is skewed. These challenges associated with survival data evoke the need of special techniques for survival analysis. In this section, we have discussed some terminologies, the statistical inference approaches, and concepts associated with these approaches that are commonly used for survival analysis.

Splendid Terminologies Enriching the Domain of Survival Analysis

There are some terminologies that are commonly used in the survival analysis domain, that need to be understood first, before diving into the survival analysis.

Survival Function

The survival function calculates the probability of an individual or any object of interest surviving until a specific time point. This concept is also referred to as the survivor function, dependability function, and survival probability. In engineering, the phrase dependability function is frequently employed, although the phrase survival function is utilized in a wider range of applications, including human mortality. The survival function is the lifetime’s complementary cumulative distribution function. Complementary cumulative distribution functions are often referred to as generic survival functions.

Hazard Function

The hazard function is a technique to describe the distribution of data in a survival study. This function is most frequently employed for predicting an individual’s risk of mortality in relation to age. Nonetheless, it can also be applied to simulate various other time-dependent events of interest. The hazard function provides a more precise simulation of whether certain time intervals exhibit the highest or lowest probabilities of an event taking place.

Hazard Ratio

The hazard ratio is a quantitative representation that illustrates the difference in event occurrence frequencies between two groups over time. Hazard ratios are frequently used in clinical trials for cancer research to compare the overall survival of a group of patients who have received one treatment to a control group with a different or a placebo. When the hazard ratio equals one, there exists no distinction in survival outcomes between the two groups. A hazard ratio of more than one or less than one indicates that one group’s chance of surviving was higher than the other.

Censoring

Censoring is a term used to describe situations where a study concludes before all enrolled participants experience the event of interest, or when a subject exits the study before the event takes place. This occurrence represents a form of missing data issue. Censoring is commonly encountered in survival analysis. The term censoring is frequently used in survival analysis.

Quantification of Survival Analysis

Survival techniques are generally of three types, nonparametric, semi-parametric, and parametric techniques. Various statistical methods are available in the literature developed for approximating survival functions. These include comparing survival curves between two groups and employing regression methods to model survival data. In general, when conducting survival analysis, it is essential for all approaches to account for a censoring mechanism in order to draw meaningful statistical conclusions.

Kaplan-Meier (KM) Estimator

The Kaplan-Meier estimator [39] is a nonparametric statistical method employed to estimate survival probabilities based on observed events, encompassing both censored and uncensored data. It operates without the need to assume an underlying probability distribution [40]. To conduct Kaplan-Meier analysis, three key variables must be taken into account: the chronological time of observation for each subject, their status (whether an event occurred or they were censored) at the conclusion of their observed time, and the specific study group to which they belong.

Log-Rank Test

The log-rank test, another nonparametric statistical approach, is utilized to compare the distribution of time until a significant event transpires in distinct groups. This test assesses the null hypothesis that there is no difference in the probability of an event (such as a death) occurring between the populations. However, it might be any other event that happens just once in a person. The most frequent occurrences of relevance are death or the start of a tumor. Even when there is no mortality involved in the event, the event time-often referred to as survival time-is the amount of time that has passed from the initial treatment or observation.

Cox Regression Hazard Model

The Cox proportional hazards model, as documented in [5], serves as a semi-parametric regression method extensively employed in medical studies for investigating the connection between patient’s survival duration and one or more predictor variables. This approach is frequently employed when the primary objective is to investigate the association between survival time and various risk factors. The Cox model’s adaptability extends to the inclusion of both numerical and categorical predictor variables. Furthermore, Cox regression allows for a thorough assessment of the influence of multiple risk factors on the duration of survival.

(AFT) Model

When there is an assumption or predetermined choice of an appropriate distribution for survival data, a parametric approach becomes suitable. Within this parametric framework, various methods are accessible, including the accelerated failure time (AFT) model, aligned with the presumed survival distributions.

Compendium of Computational Methods for Predicting Cancer Patients’ Survival

WSIs contain valuable information regarding cancer disease and its clinical outcome. Therefore, these are considered the gold standard for histopathological and morphological biomarkers. Over the years, numerous computational techniques have emerged for the automated assessment of large whole slide images, aiming to establish connections between the extracted morphological and histopathological characteristics and the patient’s survival outcome. These automated deep learning and machine learning-based computational methods are becoming increasingly complex because of the various traits of the whole slide images such as gigapixel slides, diverse tissue appearance, high inter-observer variability, and uncertainty about the region of tissue mostly associated with patient’s survival. Different prognostic markers are used to extract the features that are informative of the patient’s survival.

This section provides a summary of the literature that employs computational approaches on whole slide images (WSIs) for survival prediction. Broadly, there are two types of approaches in the literature aimed at addressing this challenge. Firstly, some studies utilize Regions of Interest (ROIs) that have been annotated by pathologists to predict patient survival. These algorithms leverage intermediate computer vision tasks to achieve accurate prognostic analysis (“Enhanced Survival Prediction: Leveraging Intermediate Computer Vision Tasks for Accurate Prognostic Analysis” section). The second type of algorithm extracts direct insights without the need for intermediate computer vision tasks (“Seamless Survival Prediction: Direct Insights without Intermediate Computer Vision Tasks” section).

Enhanced Survival Prediction: Leveraging Intermediate Computer Vision Tasks for Accurate Prognostic Analysis

These kinds of algorithms depend on the pathologist’s annotated ROIs and handcrafted features. Pathologists are experts at studying diseases, so their annotations can contain valuable information for the model to learn. A survival prediction model using annotations from pathologists can provide more accurate predictions about how long a person might live after a medical diagnosis [41]. However, there are challenges too. The model’s accuracy heavily relies on the correctness of pathologist’s annotations, which might sometimes be subjective or prone to errors. Moreover, the process of manually annotating patches is labor-intensive and susceptible to variations in interpretation by pathologists. In this section, we have summarized all the research articles that exploit the potential of utilizing intermediate image analysis for survival prediction. We have also illustrated their performance in a tabular form. These research articles utilize different computer vision techniques as intermediate feature extractors including image segmentation, detection, nuclei instance segmentation, and image classification for enhanced prognostic analysis.

Features Extraction Using Image Segmentation

In survival prediction, segmentation-based methods are used to extract features by outlining different regions or segments on whole slide image (WSI) [42]. These regions can correspond to different tissue structures or cellular components within the image. Applying various methods ranging from handcrafted methods to deep learning methods, features are learned from these regions [43], and these features are passed to computational models that predict patient survival outcomes.

Bhargava et al. [44] introduced a methodology that utilized convolutional neural network-based techniques to extract stroma and nuclei, resulting in a confidence map indicating pixel affiliation with either nuclei or stromal areas. Each patient’s histological images yielded 242 quantitative features, derived from the stromal and nuclei compartment boundaries. This model exhibited significant improvements in performance for two validation sets of radical prostatectomy patients, with an AUC of 0.87, HR of 4.71 with 95% CI: 1.65–13.4, p = 0.003, and an AUC of 0.77, HR of 5.7 with 95% CI: 1.48–21.90, p = 0.01. Peng et al. [45] developed a model that extracted texture features from global and nucleus regions of interest after segmenting them. They obtained 224 nucleus features and 56 global features and computed a risk score by combining these features with corresponding coefficients. This risk score was utilized in a univariate Cox proportional hazards model. The model’s performance was optimized, achieving a c-index of 0.772/0.029 and a time-dependent AUC of 0.785/0.038. Wang et al. [46] presented a deep learning framework for detecting lymph nodes and tumor regions, calculating the tumor area-to-MLN area ratio (T/MLN). Their approach involved three key steps: segmentation, classification, and T/MLN calculation. The framework’s performance was assessed using WSIs of LNs from CH Hospital and JX Cancer Hospital, focused on gastric cancer patients, achieving a c-index of 0.694. Jiao et al. [47] developed a model that segmented the tissue region from whole slide images (WSIs) and categorized tissue types using two classification models, distinguishing among nine classes. Kaplan-Meier (K-M) analyses were conducted for each feature, resulting in a hazard ratio of 1.665 (95% CI: 1.110$\sim$2.495, p = 0.014) for patients with colon adenocarcinoma sourced from TCGA. Wulczyn et al. [48] developed a deep learning system (DLS) for predicting disease-specific survival in colorectal cancer patients, achieving 5-year disease-specific survival AUC values of 0.70 (95% CI: 0.66–0.73) and 0.69 (95% CI: 0.64–0.72) for two datasets. Xie et al. [49] devised a system for predicting survival in intrahepatic cholangiocarcinoma (ICC) patients, with a p-value of 0.0475 and a Hazard Ratio (HR) of 2.90 (95% CI: 1.01–8.32) for Disease-Free Survival (DFS). Klimov et al. [16] introduced a predictive pipeline for recurrence-free survival in ductal carcinoma in situ (DCIS) patients, achieving a hazard ratio (HR) of 11.6 (95% CI: 5.3–25.3), an accuracy (Acc) of 0.87, and a sensitivity (Sn) of 0.71, with specificity (Sp) at 0.91. Shaban et al. [50] presented a methodology for calculating the Tumor-Infiltrating Lymphocyte Abundance (TILAb) score, achieving a c-index of 0.87 with a 95% confidence interval (CI) of 7.5–9.9 for the tissue region classifier TRC-1. Tabibu et al. [18] introduced a model that classifies patches into cancerous and normal categories and further differentiates them into three sub-classes of cancer type, yielding a Hazard Ratio (HR) of 2.265 (95% CI: 1.5343–3.343) with a p-value of 3.87e-5. Yamashita et al. [51] developed HCC-SurvNet, a deep learning-based system achieving a c-index of 0.724 on the internal test cohorts of the hepatocellular carcinoma (HCC) dataset. Xu et al. [52] suggested a pipeline that classified CD3 and CD8 stained whole slide images (WSIs), segmented these regions, and calculated stroma-immune scores. They have achieved a hazard ratio of 55.7% and 80.8% for high vs. low risk, with a confidence interval [CI] of 0.24–0.63, and a significance of P < 0.001. Wang et al. [53] presented CGSignature, which employed multiplexed immunohistochemistry (mIHC) images and transformed these images into Cell-Graphs for analyzing the tumor microenvironment (TME). CGSignature achieved an Area Under the Receiver Operating Characteristic curve of 0.960 ± 0.0 in binary or ternary classification for gastric cancer patient survival prediction.

Feature Extraction Using Nuclei Instance Segmentation

Nuclei instance segmentation can be used as a feature extraction method [54] for survival prediction by specifically identifying and segmenting the nuclei within a whole slide image (WSI). Nuclei are important indicators of cell proliferation and can provide valuable information [55] about patient survival outcomes. Nuclei instance segmentation provides a more precise analysis of the WSI, allowing for the extraction of fine-scale features that may be relevant to patient survival. By adopting this approach, we can enhance the precision of the survival prediction model by supplying additional pertinent data to the machine learning-based survival prediction algorithm. There exists a substantial body of literature that employs nuclei instance segmentation as a means of extracting features for survival prediction.

Chen et al. [56] introduced the concept of Pathomic Fusion. This innovative approach is a multimodal and interpretable strategy designed to fuse histopathological images with genomic features, aiming to predict patient survival outcomes effectively. At the heart of this approach, the Kronecker product is harnessed to encapsulate interactions between features in various modalities, considering them in pairs. Feature extraction from whole slide images involves CNNs and GCNs, with CNNs using a pre-trained VGG19 model and GCNs employing a graph-based approach to capture the tumor microenvironment. GCN captures the features in four steps: spatially localizing and identifying abnormal cell features using nuclei instance segmentation, using K-Nearest Neighbors to find adjacent cell connections, and calculating manual and deep learning features. The preservation of feature clarity is attained by implementing a gating-based attention mechanism. This process involves understanding and pinpointing the importance of these features. An evaluation conducted on glioma and clear cell renal cell carcinoma datasets from TCGA demonstrates a notably high c-index of 0.720 in both low and high-risk groups. In their work, Tian et al. [57] introduced a two-tier Fuhrman grading system based on machine learning. They extracted 1855 Regions of Interest (ROIs) from 395 whole slide images of clear cell renal cell carcinoma (ccRCC) patients from the TCGA dataset. These ROIs were partitioned into 2000 by 2000 pixel segments, with each segment characterized by 72 histomic features, including 48 texture-based features, 15 intensity-based features, and 9 morphological features. Employing seven different machine learning classifiers, they used these histomic features to classify patients into low or high-grade categories. Remarkably, Lasso regression emerged as the best-performing technique, featuring inherent feature selection capabilities. To assess the prognostic value of the predicted grade, overall survival was predicted using both crude and adjusted Cox proportional hazard models, resulting in a model that achieved an HR of 1.66 with a 95% CI ranging from 0.97 to 2.83. Alsubaie et al. [58] proposed a model that quantitatively assesses tumor nuclei by summarizing the statistics of nuclear pleomorphism. Morphometric features of tumor nuclei were utilized to characterize the heterogeneity of lung adenocarcinoma (LUAD). The Cox proportional hazard model was employed to extract the most informative features related to patient survival. Heatmap analysis revealed significant correlations between global nuclear morphometric features and overall survival in LUAD. Through multivariate analysis, the proposed model yielded a hazard ratio (HR) of 5.9, a 95% CI of 2.43–14.46, and a p-value of 0.0001. Wang et al. [59] introduced an advanced deep learning framework that utilizes a graph-based hierarchical representation. This approach explores multi-scale topological structures within whole slide images, examining pathomic features at the patch and cellular levels while disregarding their spatial connections. When presented with a whole slide image (WSI), patches were selected through a tumor detection model, forming a patch graph where each node corresponds to a cell graph. The model acquires hierarchical graph representations of whole slide images (WSIs), from patch to nuclei level. The efficacy of the proposed model was validated across diverse datasets, including the UCLA prostate biopsy dataset, Cedars-Sinai dataset, and the TCGA-PRAD dataset, achieving an impressive peak c-index of 0.7934. Chen et al. [60] introduced a computational pipeline to create embedded maps for flexible profiling of cell populations in whole slide images (WSIs). This included dividing whole slide images (WSIs) into blocks, carrying out segmentation and classification of nuclei for each block, and encoding WSIs using information about the location and classification of nuclei. This approach enables the study of tumor cells and the microenvironment using manageable embedded maps instead of large WSIs. The pipeline was applied to analyze texture patterns in the TCGA-LAUD dataset. The model achieved an optimal C-index of 0.70 during testing.

Feature Extraction Using Image Classification

Classification can be used as a feature extraction method for survival prediction by assigning each pixel or region in a whole slide image (WSI) to a specific class based on certain characteristics such as color, intensity, or texture [61]. The classes can correspond to different tissue structures or cellular components within the image [62]. After classifying the image, attributes like dimensions, morphology, texture, and luminance can be derived from each category [63]. These attributes can then serve as inputs for a machine learning framework dedicated to forecasting patient survival results. In this section, we have summarized literature that uses image classification for extraction features associated with patient’s survival.

Wulczyn et al. [64] approached the task of survival prediction from a different angle by treating it as a classification problem, rather than dealing with regression or ranking. Instead of working with continuous event time their approach discretizes time into intervals and trains modals to predict the specific occurrence of intervals. Their proposed deep learning system (DLS) predicts disease-specific survival across 10 different cancer types sourced from TCGA. This DLS mainly consists of multiple CNN modules with shared weights, along with an average pooling layer that aggregates the computed image features. Each CNN module received a randomly selected image patch from the slides as input. The patches were processed through the deep learning system in a manner that ensured at least one patch among multiple patches provided probabilistic information about the patient outcome. Cox regression hazard is applied for survival prediction and (95% CI 1.0–6.5) is recorded for 5 of 10 cancer types. Zadeh Shirazi et al. [65] introduced a classifier that classified the survival rate into 4 classes. These four classes are I for 0–6 months, II for 6–12 months, class III for 12–24 months, and class IV for more than 24 months. For each of these classes 217, 210, 277, and 145 ROIs are selected respectively. ROI patches are extracted for different sizes and compared to get the most features from ROIs. For classification, the most popular 5 DCNNs are considered, and after the comparison, the best-performing model is selected. DeepSurvNet yielded precision rates of 0.99 and 0.8 on the TCGA brain cancer dataset. Liao et al. [66] established an automated pipeline for extracting quantitative image features from HCC histopathological slides, enabling the training of machine learning models that accurately distinguished HCC from adjacent normal tissue (AUC 0.988) and predicted patient survival. Using IF model risk scores, patients were stratified into high or low-score groups, effectively separating them in the training (p < 0.0001) and test sets (p = 0.013), with strong prognostic performance in external validation (p = 0.013). Importantly, the study’s prognostic model exhibited comparable accuracy to traditional TNM staging systems. In their study, Kather et al. [17] presented a deep learning model designed to calculate the deep stroma score, a prognostic marker. This research employed four cohorts of colorectal cancer patients with HE-stained tissue samples. To achieve this, they trained a VGG-19 model that had been pre-trained on Imagenet using 86 meticulously delineated single-tissue regions from colorectal cancer cases. The model’s objective was to classify these regions into nine distinct tissue classes. The model was utilized to automate tissue decomposition in multi-tissue HE images extracted from 862 slides, encompassing 500 patients diagnosed with colorectal cancer in stages I to IV. The deep stroma score is computed utilizing the output of CNN. This factor is subsequently utilized as a prognostic variable in the multivariable Cox proportional hazard model. Remarkably, this model yielded significant results, including a hazard ratio [HR] of 1.99 with a 95% confidence interval [CI] of 1.27–3.12 and a p-value of 0.0028. These findings were corroborated using an independent dataset consisting of 409 colorectal cancer patients at stages I-IV from the Darmkrebs cohort. In their work, Zheng et al. [67] introduced two deep learning-based weakly supervised models: BlcaMIL for diagnosis and MibcMLP for prognostication. The dataset comprised 926 whole slide images (WSIs) obtained from 412 bladder cancer (BLCA) patients, with an additional 250 WSIs from 150 BLCA patients for external validation. The authors employed ResNet-50 for feature extraction across all patches, followed by dimensionality reduction through an Autoencoder. Subsequently, the resulting features were used to classify patches into negative and positive categories. The BlcaMIL model exhibited an impressive accuracy of 98.7% on the external validation set of BLCA. MibcMLP assigned risk scores to patches using patch-level features and survival data through an iterative learning process, achieving OS prediction with C-index values of 0.631 and 0.622 on the internal and external validation sets, respectively. Courtiol et al. [68] introduced a five-step model architecture called MesoNet for predicting patient outcomes. In the feature extraction step, the model extracts the foreground area, followed by tiling the matter area of the whole slide image into 10,000 small tiles of size 224x224 in the second step. The third step involves feature extraction, where 2048 features are extracted using ResNet50. In the fourth stage, a score is computed for each tile using positive and negative instances, determined through a weighted sum across all 2048 features. The final layer consists of a multilayer perceptron that translates the calculated scores into predictions. This model achieved a c-index of 0.656 for the TCGA Malignant mesothelioma dataset and 0.643 for the MESOBANK dataset. Knuutila et al. [69] suggested an architecture that first identifies metastatic tumors using clinical annotations. Primary tumors with metastasis were categorized into two subgroups based on the speed of metastatic progression, either occurring rapidly or slowly following the detection of the primary tumor. Combining AI predictions with traditional factors such as Clark’s level and tumor diameter in a risk factor model yielded a higher AUROC of 0.917 for assessing metastasis risk in primary cutaneous squamous cell carcinoma. This indicates that AI can recognize novel morphological features, enhancing the clinical evaluation of metastasis risk and prognosis.

Feature Extraction Using Image Detection

Histopathological features in whole slide images, analyzed through computational methods offer valuable insights into prognosis and patient survival outcomes. By detecting and analyzing specific structures, cells, or biomarkers, researchers aim to uncover meaningful patterns that can contribute to the prediction of patient survival outcomes. These detection approaches enable the identification and quantification of important features within the whole slide images [70], ultimately enhancing our understanding of disease progression and prognosis. This section highlights the literature that uses image detection techniques for learning features related to patient survival.

Xu et al. [71] presented a computational algorithm designed for the automatic prediction of tumor mutational burden (TMB) and the identification of TILs or tumor-infiltrating lymphocytes. The workflow involves two primary steps. Firstly, tumor detection is executed, and features are extracted by employing a pre-trained Xception model on representative tiles. Subsequently, TMB is classified using a Support Vector Machine (SVM). Additionally, TILs within tumor regions of whole slide images are identified. The spatial heterogeneity and structure of regions on WSIs containing TMB-H tumor cells and TILs are investigated for their prognostic significance. This computational pathology approach’s efficacy was evaluated using the TCGA-BLCA dataset, resulting in an AUROC of 0.752 (95% CI: 0.694–0.810). Lu et al. [72] developed a pipeline for lymphocyte detection on whole slide images, starting with a color classification-based field of view (FOV) outline, followed by a U-Net model to identify lymphocytic regions. The model was trained in a cascade manner, incorporating feedback from domain experts and pathologists to ensure robustness. Initial lymphocyte detection used a small dataset, and subsequent training incorporated the pathologist’s feedback to generate a global tumor-infiltrating lymphocyte (TIL) map. Spatial features were estimated on the TIL maps, capturing local TIL cluster patterns, and statistical features were extracted from these clusters. Survival analysis performed on the TCGA-BRCA dataset, using the Cox proportional hazard model, yielded significant results with a p-value of 0.00027 for univariate analysis and a p-value of 0.000369 for multivariate analysis. Chen et al. [73] proposed a computational model that initially identifies cell nuclei and cytoplasm. In the second step, features such as pixel intensity distributions, shapes, sizes, textures, and proximity relations of primary and secondary objects are extracted, and their means are calculated. Disease-specific survival was calculated on the TCGA-ccRCC dataset, achieving a hazard ratio of 9.50 and a p-value of 0.0091 for the validation cohort. Comes et al. [74] designed a model that predicts 1-year disease-free survival (DFS) in cutaneous melanoma patients. whole slide images from a cohort of 43 patients were annotated by pathologists and divided into sections for training and validation using a 5-fold cross-validation approach. Subsequently, the model was tested on an independent cohort of 11 patients. The model attained a median AUC of 69.5% and a median accuracy of 72.7% on the public cohort. These findings hold promise and lay the foundation for future investigations involving larger patient cohorts (see Table 3).

Table 3.

Survival prediction of patients using whole slide images

S#	Publication	MAG	States	Organ	Datasets Sources	Top Performance
1	Bhargava et al. [44]	40x	DFS	Prostate	Hospital of the University of Pennsylvania	AUC = 0.87 HR = 4.71 95% CI = 1.65–13.4 P-value = 0.003
					New York Presbyterian Weill Cornell Medical Center University Hospitals Cleveland Medical Center	AUC = 0.77 HR = 5.7 95% CI = 1.48–21.90 P-value = 0.01
2	Peng et al. [45]	20x, 40x	DFS	Skin	TCGA-SKCM	C-index = 0.772 AUC = 0.785
3	Wang et al. [46]	×20	OS	Gastric cancer	CH Hospital and JX Cancer Hospital	C-index = 0.694
4	Jiao et al. [47]	20x	OS	Colorectal	TCGA-COAD	HR = 1.665 95%CI = 1.110–2.495 p = 0.014
5	Wulczyn et al. [48]	20x	DFS	Colorectal	Institute of Pathology and the BioBank at the Medical University of Graz	AUC [95% CI] = 0.70 [0.66–0.73]
6	Xie et al. [49]	100x	OS	Liver	Nanjing Drum Tower Hospital	AUC = 0.74 ± 0.06
7	Klimov et al. [16]		DFS	Breast	Nottingham City Hospital	HR = 11.6 95% (CI) = 5.3–25.3 Accuracy = 0.87 Sensitivity = 0.71 Specificity = 0.91
8	Shaban et al. [50]	40×	DFS	head & neck	Shaukat Khanum Memorial Cancer Hospital and Research Centre (SKMCH &RC)	C-index = 0.87 95% CI 7.5–9.9
9	Tabibu et al. [18]	20x, 40x	OS	Kidney	TCGA-KIRC TCGA-KIRP TCGA-KICH	HR = 2.265 95% CI = 1.5343–3.343 P-value = 3.87e-5
10	Yamashita et al. [51]	40x	OS	Liver	TCGA-HCC	HR = 6.52 95% CI = (1.83, 23.2) P-value = 0.0038
					Stanford-HCC	HR = 3.72 95% CI = (2.17, 6.37) P-value = 0.0001
11	Xu et al. [52]	20x 40x	OS	Colorectal	Guangdong Provincial People’s Hospital	HR (95% CI) = 71.5% (66.9–76.5)
					Sixth Affiliated Hospital of Sun Yat-sen University	HR (95% CI) = 82.0% (74.4–90.4%)
12	Chen et al. [56]	40x	OS	Brain	TCGA-GBMLGG	C-index = 0.702
				Kidney	TCGA-KIRC	C-index = 0.702
13	Tian et al. [57]		OS	Kidney	TCGA ccRCC	HR = 1.66 95% CI = 0.97–2.83
14	Alsubaie et al. [58]	40x	OS	Lung	TCGA-LUAD	HR = 5.9 95% CI = 2.43–14.46 P-value = 0.0001
15	Wang et al. [59]	40x	OS	Prostate	TCGA-PRAD	C-index = 0.7934 ± 0.082
16	Wulczyn et al. [64]		DSS	Bladder	TCGA-BLCA	HR = 0.75 95% CI = 0.45–1.24 P-value = 0.2636
				Breast	TCGA-BRCA	HR = 2.86 95% CI = 1.42–5.76 P-value = 0.0034
				Colon	TCGA-COAD	HR = 4.03 95% CI = 1.92–8.44 P-value = 0.0002
				Head & Neck	TCGA-HNSC	HR = 2.32 95% CI = 1.11–4.88 P-value = 0.0257
				Kidney	TCGA-KIRC	HR = 1.88 95% CI = 1.23–2.87 P-value = 0.0035
				Liver	TCGA-LIHC	HR = 2.74 95% CI = 1.54–4.86 P-value = 0.0006
				Lung	TCGA-LUAD	HR = 1.35 95% CI = 0.87–2.08 P-value = 0.1824
					TCGA-LUSC	HR = 1.97 95% CI = 0.90–4.32 P-value = 0.0894
				Ovary	TCGA-OV	HR = 1.24 95% CI = 0.95–1.63 P-value = 0.1157
				Stomach	TCGA-STAD	HR = 1.50 95% CI = 0.85–2.62 P-value = 0.1602
17	Zadeh et al. [65]	20x	OS	Brain	TCGA-GBM	Precision = 0.65 MCC = 0.62 AUC = 0.87
18	Kather et al. [17]		OS	Colorectal	TCGA-COAD TCGA-READ	HR = 1.99 95% CI = 1.27–3.12 P-value = 0.0028
					Darmkrebs: Chancen der Verhütung durch Screening (DACHS)	HR = 1.63 95% CI = 1.14–2.33 P-value = 0.008
			DFS	Colorectal	Darmkrebs: Chancen der Verhütung durch Screening (DACHS)	HR = 1.92 95% CI = 1.34–2.76 P-value = 0.0004
19	Zheng et al. [67]	20x		Bladder	RHWU	HR = 2.414 95%CI = 0.98 p = 0.001
20	Courtiol et al. [68]	40x	OS	Mesothelioma	TCGA-MM	C-index = 0.656
21	Xu et al. [71]	20×	OS	Bladder	TCGA	AUROC = 0.752 95% CI = 0.694–0.810
22	Lu et al. [72]	40x		Breast	TCGA	P-value = 0.000369
23	Chen et al. [73]	20x	DFS	Kidney	Shanghai General Hospital	HR = 9.50 P-value =.0091
24	Comes et al. [74]	20x	DFS	Skin	CPTAC-CM	AUC = 0.695
25	Wang et al. [53]	20x	OS	Lung	TCGA-LUAD	AUC = 0.960 ± 0.01
26	Liao et al. [66]	20x	OS	Liver	HCC obtained cBioPortal for Cancer Genomics	p < 0.0001
					West China Hospital (WCH)	p < 0.0001
27	Knuutila et al. [69]	20x	DSS	skin	Turku University Hospital	AUROC = 0.747
28	Chen et al. [60]	20x	OS	Liver	TCGA-LAUD	C-index = 0.70

Seamless Survival Prediction: Direct Insights without Intermediate Computer Vision Tasks

In the medical image analysis field, seamless survival without intermediate computer vision tasks has opened a promising direction for accurate, and fastened prediction survival prediction. In this section, we have discussed approaches that omit the requirement for intermediate image analysis steps for feature extraction and offer direct survival prediction insights instead. It leverages different approaches and does survival prediction in an unsupervised, self-supervised, and weakly supervised manner via embodying domain-related understanding. These approaches can significantly reduce the annotation effort and costs associated with training models on WSIs while still achieving reasonable performance in detecting and analyzing survival-specific information within the slides. It aims at a direct prediction process and enables accurate and efficient survival forecasts.

Survival Feature Extractions Using Weakly Supervised Techniques

Predicting survival in a weakly supervised manner involves the challenge of estimating the survival outcome or predicting the time-to-event for individuals within a dataset. This task becomes particularly challenging when the available supervision is either limited or incomplete. In weakly supervised learning, the training data is labeled at a higher level or with less granularity compared to fully supervised learning. The reason behind this is primarily due to the availability of event and time-to-event data at the patient level, which provides us with valuable information. However, acquiring the same information at the patch level proves to be expensive or time-consuming. By leveraging weak supervision and using techniques like multiple instance learning, weakly supervised learning for WSIs allows for training models on large-scale datasets without requiring precise annotations for every patch. In this section, we have summarized all the papers that extract the survival informative features in a weakly supervised manner.

Bychkov et al. [75] introduced an image analysis workflow that begins with an RGB image of one TMA spot per patient, each measuring 35,000 x 35,000 in size. These images are initially divided into patches of size 224 x 224. Subsequently, a 4096 dimentional feature vector is extracted by passing the image patches through a pre-trained VGG-16 model. Finally, a risk score for five-year disease-specific survival is predicted by passing a series of 4096-bin feature vectors through a recurrent neural network known as Long Short-Term Memory (LSTM). A dataset of 420 colorectal cancer patients from the Helsinki University Central Hospital is used to model’s performance. The modal achieved a 95% confidence interval of 1.79–3.03, and an AUC of 0.69 with a hazard ratio of 2.3, surpassing both histological grade and the Visual Risk Score. Hao and colleagues [76] introduced PAGE-Net, a deep learning model that combines histopathological images and genomic data to offer a comprehensive biological interpretation. This model uses an innovative convolutional neural network to merge texture-based patches and capture globally significant survival-related features. For the genome-specific layers, they employ a sparse deep neural network called Cox-PASNet based on pathways. The model achieved a c-index of 0.702 on the TCGA-GBM dataset. Turkki et al. [77] developed a model that utilizes a pre-trained VGG-16 model to extract image features for each TMA spot. These features are then combined into a single image descriptor using improved Fisher vector (IFV) encoding. The model was tested on a breast cancer cohort from Helsinki University Central Hospital, which included 1299 tissue spots (one per patient) from two datasets: the FinProg series and the single-center series. Patients were divided into low and high digital risk score groups for disease-specific survival. The survival analyses, employing a Cox regression model, revealed a hazard ratio of 2.10 (95% CI 1.33–3.32, p = 0.001) for univariate analysis and a hazard ratio of 2.04 (95% CI 1.20–3.44, p = 0.007) for multivariate analysis. Chen et al. [78] introduced a model that employs instance-level feature extraction to represent WSIs and genomic data. Small image patches from WSIs are processed through ResNet50 to extract these features. A Genomic-Guided Co-attention layer is introduced to map features from whole slide images (WSI) and genomic data by learning densely connected co-attention mappings. The GCA layer also reduces the space complexity of WSI bags. Survival outcome prediction is performed using a set-based transformer. The model’s performance is evaluated across five different datasets of lung, breast, brain, bladder, and uterine cancers from the TCGA database, with the best performance observed in the brain cancer dataset (GBMLGG), achieving a c-index of 0.817 ± 0.021. Jiang et al. [79] proposed a model that initially divides whole slide images (WSI) into k patches. These patches are then processed through a pre-trained ResNet-18 model to extract relevant features. The features obtained from the k patches are subsequently averaged and used to estimate risk via a two-layer neural network. The final output of the model represents the risk score. The model’s performance is assessed using a grade 2 diffuse glioma dataset from TCGA, achieving a c-index of 0.715 (95% CI: 0.569, 0.830) for prognosis prediction and an AUC of 0.667 (95% CI: 0.532, 0.784) for predicting IDH mutations. Agarwal et al. [80] proposed a novel technique that initially trains a survival prediction model by collecting the whole slide feature map (WSFM). Features from WSIs are extracted by passing image tiles through the Inceptionv3 model, followed by PCA to reduce feature dimensions. The WSFM captures features from the entire tissue along with adjacency information for each tile. A Siamese survival convolutional neural network (SSCNN) is developed, predicting the survival score using WSFM and multivariable clinical variables. The proposed methodology achieved a c-index of 0.6221 for the TCGA Glioblastoma multiforme (GBM) dataset. Sandarenu et al. [81] introduced a computational framework designed to predict the survival outcomes of clinically aggressive triple-negative breast cancer using hematoxylin and eosin-stained tissue microarray images. They employed a multi-instance fully convolutional network with attention-based features aggregation, sharing weights to estimate the risk of mortality in patients with breast cancer. The model achieved an average concordance index of 0.616 and displayed autonomous prognostic importance in both univariate and multivariate analyses. Through qualitative analysis of heatmaps generated by the model, the authors associated high-risk predictions with specific tissue features, providing explainability for their method in triple-negative breast cancer. Li et al. [82] introduced a hierarchical-based multimodal transformer framework that learns features from whole slide images to predict patient survival. The model reduces space complexity by learning co-attention mappings between imaging and genomics data. Wetstein et al. [83], introduced a deep learning model designed for grading breast cancer using whole slide images. The training dataset comprised images from 706 patients (below 40 years) diagnosed with invasive breast cancer, along with corresponding tumor grades. The model’s performance was assessed on an independent test set consisting of 686 patients. Impressively, the model achieved a Cohen’s Kappa score of 0.59, indicating 80% accuracy in distinguishing between low/intermediate and high tumor grades compared to assessments by expert pathologists. Furthermore, the model’s predictions for low/intermediate and high-grade categories demonstrated significant differences in overall survival (OS) and disease/recurrence-free survival (p < 0.05) in survival analysis, using univariate Cox hazard regression analysis. Chen et al. [84], introduced a context-aware graph convolutional model named patch-GCN, which leverages spatially resolved patches to hierarchically integrate instance-level features. These extracted features play a pivotal role in capturing both local and global topological patterns within the tumor microenvironment. The patch-GCN model achieved an overall c-index of 0.636 across multiple cancer types in TCGA datasets, including BLCA, BRCA, GBMLGG, and LUAD, highlighting its substantial predictive capabilities. Li et al. [85] introduced a graph convolutional neural network (graph CNN) that models WSI as a graph. The graph CNN incorporates an attention mechanism to enhance feature learning by optimizing the graph representation. The effectiveness of their model was demonstrated on real lung and brain carcinoma datasets from TCGA and NLST, achieving the highest accuracy of 0.7066 for NLST. Di et al. [86] introduced a method comprising two significant stages. The first stage, named the Big-Hypergraph Factorization Neural Network, addresses information loss through dense sampling. This is followed by Encoding Patches to establish an incidence matrix capturing intricate high-order correlations among vertices and hyperedges, facilitating operations such as feature transfer, information propagation, and representation aggregation. This approach generates a comprehensive global representation for each input whole slide image (WSI) from a collection of WSIs using a big-hypergraph factorization neural network. In the second stage, named Multi-Level Ranking Survival Prediction, hazard scores are assigned to individual WSIs. The method was evaluated across three datasets: TCGA-LUSC, TCGA-GBM, and NLST, achieving C-index scores of 0.6734 ± 0.09, 0.6738 ± 0.012, and 0.6911 ± 0.011, respectively. Xu et al. [87] introduced an approach that merges risk and time prediction methodologies together using risk prediction features as a guide for predicting survival time. Features from tumor patches are extracted through a pre-trained classifier. They utilized a graph convolutional network for aggregating information from these patches. The C-index values for risk prediction were 0.834, 0.627, and 0.563 for BLCA, BRCA, and GBM, respectively. Lu et al. [88] presented a method that utilized weakly-supervised attention multiple instance learning and differential techniques for survival prediction. The authors also stratified patients based on whole slide images, demonstrating the efficacy of their approach within a federated setting. Federated learning demonstrated the ability to generate accurate weakly-supervised deep learning models through the use of distributed data, removing the necessity for direct data sharing and its associated complexities. Additionally, the approach maintained privacy by incorporating randomized noise generation techniques, ensuring the preservation of privacy in their methodology. Tu et al. [89] analyzed cancer prognosis with whole slide images (WSIs) by suggesting a method encompassing two curriculums. The first one is named saliency-guided weakly-supervised instance encoding with cross tiles where the model has 3 input branches, each one with a different magnification. The second one is counteractive-enhanced Soft-bag Prognosis Inference. The proposed model is tested using three datasets from TCGA, i.e., COAD, LIHC, and BLCA achieving a c-index of 0.717, 0.705, and 0.672 respectively. Xie et al. [90] introduced GC-SPLeM, comprising three components: WSI feature extraction, modal-fusing network, and a GNN-based predictor. A ResNet50 pre-trained on ImageNet is used as a feature extractor. A patient-level feature matrix is generated by concatenating 1024-dimensional feature vectors from WSI patches. Modal fusion network fuses together the feature matrix and gene expression data matrix. Modal fusion network generates a 64-dimensional vector which is utilized by a GNN-based predictor to construct a KNN affinity graph. The proposed model achieved a c-index of 0.622. Mackenzie et al. [91] suggested a model that utilizes a graph neural network. The modal uses survival scores to perform pairwise ranking of graph representations of WSIs. They produce survival scores by translating spatially localized deep features along with their spatial context to a graph neural network. The proposed model achieved a c-index of 0.672 ± 0.058. Benkirane et al. [92] proposed HyperAdaC, an adaptive clustering-based hypergraph representation designed to capture high-order correlations among various regions in whole slide images (WSIs). This compact model aims to facilitate the generalization of graph neural networks in survival prediction scenarios. The model achieved a c-index of 0.667 on BLCA, 0.592 on BRCA, 0.778 on GBMLLG, LAUD on 0.595 and UCEC on 0.667.

Survival Features Extraction Using Unsupervised Techniques

Survival feature extraction using unsupervised techniques refers to estimating the survival outcome or time-to-event information for individuals or subjects without relying on labeled or annotated survival data. In unsupervised learning, the objective is to identify patterns or structures within the data without possessing any pre-existing patch-level information. Major approaches for survival prediction in an unsupervised manner include clustering and dimensionality reduction techniques including PCA for reducing the dimensionality of the data.

Zhu et al. [93] put forward with the aim of exploiting the distinctive patterns inherent in patients’ tumor morphology. The model’s efficacy was assessed across three publicly available cancer survival datasets: NLST containing 1104 slides for 404 patients, TCGA-LUSC encompassing 485 slides distributed among 121 patients, and TCGA-GBM comprising 255 slides spanning 126 patients. This encompassed the extraction of multiple patches from each WSI through adaptive sampling. These image patches were subsequently allocated to distinct clusters, from which patient-level predictions were derived using a Deep Convolutional Survival (DeepConvSurv) aggregation model. The ensuing survival analysis relied on various iterations of Cox regression methodologies. Notably, the proposed approach established its dominance by achieving a c-index of 0.703 for NLST via Lasso-Cox, 0.638 for LUSC through Lasso-Cox, and 0.645 for GBM using Cox-Log. Yao et al. [94] introduced an attention-driven strategy named DeepAttnMISL (Attention-based Multiple Instance Survival Learning) to intricately capture features connected to a patient’s survival. The effectiveness of this approach is evaluated across expansive datasets encompassing colorectal and lung cancer cases. The process initiates by sampling patches from whole slide images, which are subsequently organized into 10 distinct phenotype groups. This categorization is facilitated by initially extracting features via a pre-trained model and subsequently utilizing k-means clustering. To create patient-level representations, the model employs attention-based Multiple Instance Learning (MIL) pooling, aggregating features from each cluster. Subsequently, the model predicts the patient’s hazard risk, achieving a c-index of 0.606 across 6 phenotype groups. Shao et al. [14] introduced a weakly supervised deep ordinal Cox model (BDOCOX) that encompasses three key steps: patch extraction, bag generation, and the design of the BDOCOX model. Each whole slide image (WSI) is divided into 1000 patches, and a 4096-dimensional feature vector is extracted from each patch using a pre-trained ResNet128. These patches are then grouped into 5 distinct phenotype categories through K-means clustering. The weakly supervised deep ordinal Cox model is constructed by replacing the linear component of the Cox model with a non-linear, fully connected network operating in-depth. N bags are generated from WSIs and bag-level representation is obtained by averaging patch-level features in the bag. The model is evaluated on three datasets from TCGA and the best performance (CI 0.726 ± 0.022, AUC 0.751 ± 0.034) is achieved for the lung cancer dataset (LUSC). Tang et al. [15] developed a novel capsule network named CapSurv comprised of three sequential steps introducing a new survival loss function. Initially, image patches are selected in a random manner from the tissue region of the image, excluding background whitespace at 20x magnification. Afterward, utilizing a pre-trained VGG16 model, features are extracted from these patches. This representation, rich in features, is then employed to classify the patches into various clusters. Among these clusters, one that exhibits particularly strong predictive potential is identified and chosen. Subsequently, the CapSurv model, as proposed, is trained to perform survival prediction. Notably, the model achieves a c-index of 0.67 for the GBM dataset and 0.673 for the LUSC dataset. Yao et al. [19] introduced a deep multiple instances learning approach aimed at uncovering all potential hidden patterns within the image patches that contain informative data related to patient survival. The proposed model ability is evaluated on Lung (Lung-ADC) and Brain (GBM) tumors whole slide pathological images datasets to predict the risk score of patients. The first step of the methodology divides the extracted patches from the images into different phenotype groups. By subjecting the image patches to a pre-trained VGG model, 4096 features are extracted, following which k-means clustering is employed to create distinct phenotype clusters for each patient. Then a Multi-Instance Fully Convolutional Network (MI-FCN) is used for learning local representation from different phenotypes which are eventually aggregated for patient-level representation. The aggregated features from each phenotype group make a final survival prediction as the patient’s survival score. The method demonstrated notable results, achieving a c-index of 0.678 for Lung-ADC and 0.657 for GBM Muhammad et al. [95] devised the EPIC-survival model, which seamlessly integrates patient survival modeling by connecting feature encoding and feature aggregation. They introduced a stratification-boosting approach to enhance the ranking and differentiation of risk groups. The model’s performance was assessed using whole slide images (WSI) from patients with intrahepatic cholangiocarcinoma (ICC) obtained from Memorial Sloan Kettering Cancer Center (MSKCC), Erasmus Medical Center-Rotterdam (EMC), and the University of Chicago (UC), achieving an impressive concordance index of 0.880. Liu et al. [96] introduced EOCSA, a framework that randomly extracts patches from whole slide images (WSIs) and creates multiple clusters using these patches. Following this, they created a survival prediction model known as DeepConvAttentionSurv (DCAS), which adeptly extracts features at the patch level, removes less discriminative clusters, and precisely predicts survival in epithelial ovarian cancer (EOC). The model incorporates channel, spatial, and neuron attention mechanisms to enhance feature extraction performance. Patient-level features are derived using the proposed weight calculation method. They estimated the survival time using the LASSO-Cox model. The model achieved an impressive C-index of 0.980 on TCGA-EOC. Sun et al. [97] presented an unsupervised deep learning network that merges a variational autoencoder and generative adversarial network. This network is designed to generate a signature for whole slide images (WSI) to predict disease-free survival (DFS) and overall survival (OS) in patients. They developed an integrated nomogram to evaluate the supplementary value of the deep learning signature (DLS) in conjunction with the TNM stage for personalized outcome predictions. The model demonstrated a c-index of 0.748 for DFS and 0.794 for OS.

Survival Features Extracting Using Self-supervised Techniques

In self-supervised learning for survival feature extraction, the aim is to predict individuals’ or subjects’ survival outcomes using self-supervised learning techniques. Traditional survival analysis, on the other hand, involves modeling time-to-event data, such as time until death or failure, using various statistical and computational methods. Self-supervised learning seeks to learn representations or predictive models from unlabeled data without explicit guidance. When applied to survival prediction, self-supervised learning methods harness the inherent structure or patterns in the data to acquire representations that capture essential features related to survival outcomes.

Chang et al. [98] introduced the Hybrid Aggregation Network (HANet), which aggregates multiple whole slide images (WSIs) of a patient to obtain patient-level information for survival analysis. A self-supervised convolutional neural network is employed to extract features from WSIs. These features are then fused using two proposed aggregation modules. To predict patient-level survival risk, region representations are obtained from patients’ WSIs using the self-aggregation module. The methodology’s effectiveness is assessed using the lung cancer dataset NLST and TCGA-LUSC, resulting in c-indices of 0.734 and 0.668, respectively, for both datasets. Fan et al. [99] proposed a survival prediction model designed to enhance performance by leveraging patient-level heterogeneous features. They preprocess images through colorization using self-supervised techniques and subsequently train a Convolutional Neural Network (CNN) for feature extraction. The model predicts survival by combining multiple WSIs at the patient level with consistency and ranking losses. Model performance is evaluated using TCGA-GBM and TCGA-LUSC datasets, achieving c-indices of 0.6654 and 0.6772, respectively, for both datasets. Fan et al. [100] present a novel survival prediction framework tailored for histopathological whole slide images (WSIs). This comprehensive framework includes patch sampling, feature extraction, and patient-level survival prediction. To facilitate WSI feature extraction, self-supervised learning techniques such as colorization and cross-channel tasks are employed as pretext tasks to train convolutional-based models. Patient-level survival prediction is carried out by harmonizing features from multiple WSIs using consistency and contrastive losses. Experimental analyses conducted on TCGA-GBM, TCGA-LUSC, and NLST datasets demonstrate the framework’s outstanding performance, evident through concordance indices of 0.670, 0.679, and 0.711, respectively. Vale-Silva and Rohr [101] introduced an architectural framework consisting of three fundamental modules: a feature representation module, a multimodal data fusion layer, and an output sub-model. In the feature representation module, individual sub-models dedicated to distinct data modalities generate fixed-size hidden data representations. The subsequent multimodal data fusion layer combines these modality-specific sub-models to yield a unified representation. Finally, the output sub-model translates the fused representation into discrete-time conditional survival probability predictions. Performance evaluation across 33 cancer types results in a remarkable peak c-index score of 0.822. Shen et al. [102] proposed a novel survival analysis model that utilizes a Vision Transformer (ViT) backbone coupled with convolution operations, ensuring effective feature extraction from whole slide images (WSIs) related to cancer diagnosis. This model maximizes the utilization of comprehensive WSI information, eliminating the need to exclude crucial morphological insights. The authors also introduce a post hoc explainable technique aimed at identifying significant patches and distinctive morphological attributes, significantly enhancing the model’s interpretability. Evaluations conducted on two extensive cancer datasets, NLST and small cell lung cancer (CHCAMS), validate the remarkable efficacy of this approach, resulting in C-index scores of 0.730 and 0.707, respectively. These outcomes robustly underscore the model’s potential not only in survival prediction but also in aiding critical cancer treatment decisions. Di et al. [103] presented RankSurv, a survival prediction method for whole slide images (WSIs) that incorporates ranking information into the learning process. The proposed approach involves two steps. Firstly, a hypergraph representation is introduced to predict hazards for each WSI, capturing the higher-order correlation between patches within the WSI. Subsequently, a prediction process based on rankings is conducted using pairwise survival data. The effectiveness of RankSurv is evaluated through experiments on three publicly available carcinoma datasets, namely LUSC, GBM, and NLST (see Table 4).

Table 4.

Survival prediction of patients using whole slide images

S#	Publication	MAG	States	Organ	Datasets Sources	Top Performance
1	Bychkov et al. [75]		DSS	Colon	Helsinki University Central Hospital	HR = 2.3 95%CI = 1.79–3.03 AUC = 0.69
2	Hao et al. [76]	20x	OS	Brain	TCGA-GBM	C-index = 0.702
3	Turkki et al. [77]		DSS	Breast	Helsinki University Central Hospital	HR = 2.04 95% CI = 1.20–3.44 P = 0.007
4	Chen et al. [78]	20x	OS	Bladder	TCGA-BLCA	C-index = 0.624 ± 0.034
				Breast	TCGA-BRCA	C-index = 0.580 ± 0.069
				Brain	TCGA-GBMLGG	C-index = 0.817 ± 0.021
				Lung	TCGA-LUAD	C-index = 0.620 ± 0.032
				Uterine	TCGA-UCEC	C-index = 0.622 ± 0.019
5	Yao et al. [94]	20x	OS	Lung	NLST	C-index = 0.6963 AUC = 0.7143
				Colorectal	MCO	C-index = 0.606 AUC = 0.644
6	Jiang et al. [79]	10x	OS	Brain	TCGA-LGG	C-index [95% CI] = 0.784 [0.655, 0.880]
						AUC [95% CI] = 0.739 [0.613, 0.856]
7	Agarwal et al. [80]	20x, 5x, 1.25x	OS	Brain	TCGA-GBM	C-index = 0.6221
8	Sandarenu et al. [81]	40x			TNBC	HR = 2.28 95%CI = 1.24–4.18 p = 0.01
9	Li et al. [82]	20x	OS	Bladder	TCGA-BLCA	C-index = 0.660 ± 0.021
				Breast	TCGA-BRCA	C-index = 0.606 ± 0.028
				Brain	TCGA-GBMLGG	C-index = 0.823 ± 0.019
				Lung	TCGA-LUAD	C-index = 0.616 ± 0.016
				Uterus	TCGA-UCEC	C-index = 0.658 ± 0.047
10	Wetstein et al. [83]	20x	OS	Breast	YBC	HR = 1.84 95% CI = 1.24–2.72 P-value = 0.025
11	Shao et al. [14]	20x	OS	Kidney	TCGA-KIRC	C-index = 0.699 ± 0.027 AUC = 0.714 ± 0.031
				Liver	TCGA-LIHC	C-index = 0.701 ± 0.041 AUC = 0.727 ± 0.039
				Lung	TCGA-LUSC	C-index = 0.726 ± 0.022 AUC = 0.751 ± 0.034
12	Chen et al. [84]	20x	OS	Bladder	TCGA-BLCA	C-index = 0.560 ± 0.034
				Breast	TCGA-BRCA	C-index = 0.580 ± 0.025
				Brain	TCGA-GBMLGG	C-index = 0.824 ± 0.024
				Lung	TCGA-LUAD	C-index = 0.585 ± 0.012
				Uterine	TCGA-UCEC	C-index = 0.629 ± 0.052
13	Li et al. [85]		OS	Lung	TCGA-LUSC	C-index = 0.6606
					TCGA-NLST	C-index = 0.7066
				Brain	TCGA-GBM	C-index = 0.6215
					MESOBANK	C-index = 0.643
14	Di et al. [86]	20x	OS	Lung	TCGA-LUSC	C-index = 0.6734 ± 0.09
					TCGA-NLST	C-index = 0.6911 ± 0.011
				Brain	TCGA-GBM	C-index = 0.6738 ± 0.012
15	Xu et al. [87]	20x	OS	Bladder	TCGA-BLCA	C-index = 0.834
				Breast	TCGA-BRCA	C-index = 0.627
				Brain	TCGA-GBM	C-index = 0.563
16	Zhu et al. [93]			Lung	NLST	C-index = 0.703
		20x	OS		TCGA-LUSC	C-index = 0.638
				Brain	TCGA-GBM	C-index = 0.645
17	Tang et al. [15]	20x	OS	Lung	TCGA-LUSC	C-index = 0.673
				Brain	TCGA-GBM	C-index = 0.67
18	Yao et al. [19]	5x	OS	Lung	Lung-ADC (NLST)	C-index = 0.678
				Brain	TCGA-GBM	C-index = 0.657
19	Muhammad et al. [95]	20x	OS	Bile duct	(ICC) patients’ WSI from Memorial Sloan Kettering Cancer Center (MSKCC), Erasmus Medical Center-Rotterdam (EMC), and the University of Chicago (UC)	C-index = 0.880
20	Chang et al. [98]	10x	OS	Lung	TCGA-LUSC	C-index = 0.668
					NLST	C-index = 0.734
21	Fan et al. [100]	20x	OS	Lung	TCGA-LUSC	C-index = 0.679 ± 0.015
					TCGA-NLST	C-index = 0.6711 ± 0.015
				Brain	TCGA-GBM	C-index = 0.02
22	Vale-Silva and Rohr [101]	40x	OS	-	Pan Cancer (All 33 cancer types of TCGA)	C-index= 0.822
23	Fan et al. [99]		OS	Brain	TCGA-GBM	C-index = 0.6654
				Lung	TCGA-LUSC	C-index = 0.6772
24	Shen et al. [102]	20x		Lung	NLST	C-index = 0.730
					CHCAMS	C-index = 0.707
25	Di et al. [103]	20x	OS	Lung	TCGA-LUSC	C-index = 0.66 ± 0.011
					TCGA-NLST	C-index = 0.7011 ± 0.015
				Brain	TCGA-GBM	C-index = 0.05
26	Lu et al. [88]	20x	OS	Breast	TCGA-BRCA	0.842±0.022
				Renal cell	CCRCC	C-index = 0.985±0.004
27	Tu et al. [89]	20x,	OS	Colon	TCGA-COAD	C-index = 0.717
		10x,		Liver	TCGA-LIHC	C-index = 0.705
		5x		Bladder	TCGA-BLCA	C-index = 0.672
28	Xie et al. [90]	20x	OS	Liver	TCGA-LAUD	C-index = 0.622
29	Mackenzie et al. [91]	20x	DSS	Liver	TCGA-BRCA	C-index = 0.672 ± 0.058
30	Benkirane et al. [92]	20x	OS	Bladder	TCGA-BLCA	C-index = 0.564 ± 0.034
				Breast	TCGA-BRCA	C-index = 0.592 ± 0.025
				Brain	TCGA-GBMLGG	C-index = 0.778 ± 0.024
				Lung	TCGA-LUAD	C-index = 0.595 ± 0.012
				Uterine	TCGA-UCEC	C-index = 0.667 ± 0.022
31	Liu et al. [96]	20x	OS	Liver	TCGA-BRCA	C-index = 0.672 ± 0.058
32	Sun et al. [97]	20x	OS	Colorectal	TCGA-CRC	C-index = 0.794

Discussion

The articles surveyed in this review paper emphasize the potential of employing computational techniques with whole slide images (WSIs) for predicting the survival of cancer patients. Analyzing WSIs to identify histopathological biomarkers for survival prediction and enhancing prognostic insights is an active and rapidly evolving research area. whole slide images (WSIs) possess unique attributes, such as their considerable size, high resolution, and diverse tissue appearances. The critical task of selecting survival-representative features from these WSIs that correlate with patient outcomes is essential when developing survival prediction models. Over time, a number of computational techniques have emerged to improve prognostic capabilities using WSIs. These computational methods generally fall into two categories. Direct feature extraction methods that do not require intermediate computer vision tasks. Techniques such as self-supervised, weekly-supervised, and unsupervised learning, multiple instance learning, clustering, and graph convolutional neural networks are utilized for learning features directly from WSIs. Other approaches that leverage intermediate computer vision tasks such as image segmentation, nuclei instance segmentation, image classification, and handcrafted features are utilized for extracting the potential features for survival prediction. Each of these methods offers distinctive perspectives on the factors influencing patient outcomes, underscoring the need for further research to enhance and broaden the utilization of WSIs in predicting survival and improving patient care.

The reviewed methodologies have shown promising advancements in improving patient outcomes and advancing the understanding of cancer morphology. While there has been advancement in the field, there is still no universally accepted optimal approach for analyzing cancer morphology and predicting patient survival. Furthermore, the growing complexity of these computational methods demands substantial resources for the effective training of survival prediction models. As a result, there is a need for further investigation into the efficient utilization of deep learning techniques in the context of survival prediction. The reviewed research papers demonstrate the potential of using whole slide images to predict cancer patient survival. These papers have introduced intricate algorithms and a variety of methodologies to extract survival-relevant features from WSIs. Nevertheless, the examined articles indicate a clear trend toward further research to achieve survival prediction with resource efficiency in this endeavor. Ongoing research and innovation will continue to play a pivotal role in understanding of cancer morphology, advancing personalized medicine, and raising the standards of patient care as the search for the optimal approach continues to evolve.

Conclusion

Survival analysis finds extensive application in developing prognostic indices for mortality or disease recurrence, assessing treatment effectiveness, and tailoring effective treatment plans. One of the key goals of precision medicine in cancer is to discover prognostic biomarkers that can reliably forecast patient survival. whole slide images are regarded as the definitive method for identifying histopathological biomarkers that provide information about survival outcomes. This review article targets the latest approaches proposed in the last 5 years for survival analysis, explores the approaches utilized in existing research, performance metrics, and existing obstacles, and points toward prospective solutions for future research. The reviewed paper compiles different techniques researchers have adopted to extract features associated with patient survival from WSIs, including handcrafted features, transfer learning, image segmentation, nuclei instance segmentation, and image classification. The selection of the most impactful WSI features can lead to the most accurate patient outcomes and improved patient care. However, there is no agreed-upon best method yet, and each technique offers a distinct understanding of factors impacting patient survival. The field is constantly expanding, and there is still much untapped potential in the realm of deep learning techniques for predicting the survival of cancer patients.

Author Contributions

All authors contributed equally to the study’s conception, design, material preparation, data collection, and documentation.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Data Availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Declarations

Ethics Approval

This study provides a summary of previously published research articles sourced from open-access platforms such as PubMed, Google Scholar, and Science Direct.

Consent to Participate

All papers included in this review article are collected from open source platforms. I confirm that all previously published research articles included in this review paper have been properly cited and are used in accordance with copyright and fair use guidelines.

Consent to Publish

The articles in this review paper are included in accordance with applicable copyright and citation guidelines.

Competing Interests

The authors declare no competing interests.