Application of Deep Learning in Cancer Prognosis Prediction Model

Abstract

As an important branch of artificial intelligence and machine learning, deep learning (DL) has been widely used in various aspects of cancer auxiliary diagnosis, among which cancer prognosis is the most important part. High-accuracy cancer prognosis is beneficial to the clinical management of patients with cancer. Compared with other methods, DL models can significantly improve the accuracy of prediction. Therefore, this article is a systematic review of the latest research on DL in cancer prognosis prediction. First, the data type, construction process, and performance evaluation index of the DL model are introduced in detail. Then, the current mainstream baseline DL cancer prognosis prediction models, namely, deep neural networks, convolutional neural networks, deep belief networks, deep residual networks, and vision transformers, including network architectures, the latest application in cancer prognosis, and their respective characteristics, are discussed. Next, some key factors that affect the predictive performance of the model and common performance enhancement techniques are listed. Finally, the limitations of the DL cancer prognosis prediction model in clinical practice are summarized, and the future research direction is prospected. This article could provide relevant researchers with a comprehensive understanding of DL cancer prognostic models and is expected to promote the research progress of cancer prognosis prediction.

Introduction

The incidence of cancer has now become one of the highest in the world. According to the cancer data published by Ca magazine in 2020, the number of new cancer cases in the world reached 19.29 million in 2020, and the number of death cases was 9.96 million. ¹ The number of deaths due to cancer each year accounts for one-sixth of the total number of deaths. ² Accurate cancer prognosis could help improve the cure rate and overall survival of patients with cancer.

In recent years, due to the improvement in computer performance and model architecture and the growth of clinical data, Deep learning (DL) has made some substantial breakthroughs. ³ Through the deep superposition of artificial neural networks in the traditional field of machine learning, the extracted low-level image features can be iterated layer by layer into more abstract high-level features, thus achieving the prediction task. ⁴ DL has flourished in the field of medicine, especially in the prediction of cancer, because of its unique advantages of autonomous learning.^5–7

At present, cancer prognosis prediction methods mainly include traditional machine learning, radiomics, and DL. Among them, machine learning methods mainly use some classical classification algorithms, such as random forest, logistic regression, and support vector machine, to train and predict cancer-related data sets. ⁸ These methods are mainly applied to numerical data, they usually cannot directly process image and text data, and meeting the needs of clinical application in terms of speed and prediction accuracy is difficult. Radiomics focuses on the processing and analysis of image data. Through image processing technology, imaging features of medical images, such as ultrasound, CT, or MRI, are extracted, and these features are used to predict cancer prognosis. ⁹ This method is more interpretable to the model, but when the tumor does not change much, the prediction accuracy of the model may decrease. Meanwhile, DL is a relatively novel method that uses structures superimposed by multilayer neural networks to train and predict cancer-related data sets. ¹⁰ It can process various information, such as numerical value, text, and image, at the same time, and it can extract features and build models more accurately, with better prediction accuracy and stability. In addition, the DL model has the advantages of automatic feature extraction, strong generalization ability, and wide application range, and it is gradually becoming an emerging technology for cancer prognosis prediction, which is the development trend in the field of cancer prognosis prediction in the future.^11–13

In the last 5 years, thousands of articles on “Deep learning cancer prediction” have been published on PubMed. However, reviews on DL in cancer prognosis prediction models are relatively few. Therefore, in the present paper, the application of DL in cancer prognosis prediction models was systematically reviewed. The following aspects of DL prognostic models were mainly reviewed: (1) input data type; (2) performance evaluation indicators; (3) model construction process; (4) research progress of mainstream models in recent years; (5) key factors affecting performance and enhancing performance; and (6) main limitations of clinical application and future research trends.

Data Types for DL Cancer Prediction Model

1. Numerical and categorical data. Numerical data, including patient age, tumor size, and various pathological and biochemical indicators, can be used for numerical calculation and analysis. Categorical data, including patient gender, clinical stage, and pathological type, are often used to group and label the data. These data are stored in numerical or categorical form, which can provide an intuitive reference, reflect the correlation with clinical endpoints, and enhance the predictive power of DL models when the amount of data is large. For example, numerical and categorical data, such as age, tumor size, and pathological stage, are very important features when predicting the survival time of patients with cancer. ¹⁴

2. Text data. Text data mainly refer to medical records, image reports, and other text information stored in the form of natural language. The benefit of text data is that they can provide a professional physician's experience, provide a more complete picture of key details in treatment, and train and optimize DL models to extract more important features. For example, when predicting the survival of patients with metastatic cancer, the text information in medical record reports and prescriptions can be used to understand the patient's diagnosis, treatment plan, etc., thus introducing important features into the model. ¹⁵

3. Image data. Image data, including ultrasound, CT, MRI, x-ray, and other image types, refer to the medical image data taken during patient visit. They can provide the specific location, size, shape, and other information of the lesion, which has a better diagnostic effect. Through DL algorithms, image features can be automatically extracted and analyzed to achieve fast and accurate cancer prognosis prediction. For example, the use of ultrasound images to predict the efficacy of neoadjuvant chemotherapy in patients with breast cancer or the use of MRI images to predict axillary lymph node metastasis of breast cancer needs to rely on image data to extract key features.^16,17

In addition, the fusion of numerical and categorical data, text data, and image data can more comprehensively consider various information in the course of cancer treatment, forming a multimodal data set, which helps improve the accuracy and robustness of the model and provide enhanced decision support for cancer prognosis prediction.

Construction Process of DL Cancer Prognostic Model

The construction steps of the DL cancer prognostic model can be divided into the following parts. (1) Determine specific prognostic problems (survival rate, recurrence rate, mortality rate, etc.). (2) Data acquisition and preprocessing: relevant data, including patient clinical information and image data, are obtained, and data cleaning, normalization, and other preprocessing work are conducted. (3) Model selection and design: the appropriate DL model is selected in accordance with the type of data and specific clinical tasks while optimizing and adjusting the model. (4) Partition data set: the preprocessed data are divided into training, validation, and test sets, which are not repeated. (5) Model training and tuning: through the input of training set and validation set data into the model for multiple rounds of iterative training and validation, the parameters of the model are constantly adjusted to determine the best model. (6) Model evaluation: the prediction performance and generalization ability of the model in the test set are evaluated. The specific process is shown in Figure 1.

Figure 1.

Dl cancer prognostic model construction process.

Evaluation index of the Accuracy of Predicting Cancer Prognosis

• (1) The area under the receiver operating characteristic curve (AUC) is used to evaluate the effect of binary classifiers; the closer it is to 1, the better the classification effect.

  $$\mathrm{AUC}={\displaystyle \frac{{\sum }_{\mathrm{in}{\mathrm{s}}_{\mathrm{i}}\in \mathrm{positive}\mathrm{class}}\mathrm{ran}{\mathrm{k}}_{\mathrm{in}{\mathrm{s}}_{\mathrm{i}}}-{\displaystyle \frac{\mathrm{M}\times (\mathrm{M}+1)}{2}}}{\mathrm{M}\times \mathrm{N}}}$$

where $\mathrm{ran}{\mathrm{k}}_{\mathrm{in}{\mathrm{s}}_{\mathrm{i}}}$ is the serial number of sample i (probability scores are ranked in order from smallest to largest), M is the number of positive samples, N is the number of negative class samples, and $\sum _{\mathrm{in}{\mathrm{s}}_{\mathrm{i}}\in \mathrm{positive}\mathrm{class}}\mathrm{ran}{\mathrm{k}}_{\mathrm{in}{\mathrm{s}}_{\mathrm{i}}}$ : add the serial numbers of the positive sample.

• (2) Accuracy (ACC) refers to the probability that all the samples predict correctly.

  $$\mathrm{ACC}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FN}+\mathrm{FP}}}$$

where TP is the number of samples predicted to be positive, TN is the number of samples of negative class predicted into negative class, FN is the number of samples of positive class predicted into negative class, and FP indicates the number of samples in which a negative class is predicted to be a positive class.

• (3) Sensitivity (SEN) refers to the proportion of all truly positive samples identified as positive.

  $$\mathrm{SEN}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$
• (4) Specificity (SPE) refers to the proportion of all truly negative samples that are identified as negative.

  $$\mathrm{SPE}={\displaystyle \frac{\mathrm{TN}}{\mathrm{FP}+\mathrm{TN}}}$$
• (5) Concordance index (C-index) is used to evaluate the predictive power of the survival model; the closer it is to 1, the better.

  $$\mathrm{C}-\mathrm{index}={\displaystyle \frac{\mathrm{K}}{\mathrm{M}}}$$

where K is the number of pairs in which the predicted result agrees with the actual result when all samples are matched, and M is the number of pairs that cannot be determined to be consistent.

• (6) Positive predictive value (PPV) refers to the proportion of all samples predicted to be positive that are actually positive.

  $$\mathrm{PPV}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$
• (7) Negative predictive value (NPV) refers to the proportion of all samples predicted to be negative that are actually negative.

  $$\mathrm{NPV}={\displaystyle \frac{\mathrm{TN}}{\mathrm{TN}+\mathrm{FN}}}$$
• (8) F1-Score reconciles PPV and SEN indicators.

  $$\mathrm{F}1-\mathrm{Score}={\displaystyle \frac{\mathrm{PPV}\times \mathrm{SEN}}{\mathrm{PPV}+\mathrm{SEN}}}$$
• (9) Matthew's correlation coefficient (MCC) refers to the correlation between the actual category and the predicted category number; the closer the MCC value is to 1, the better the classifier.

  $$\mathrm{MCC}={\displaystyle \frac{\mathrm{TP}\times \mathrm{TN}-\mathrm{FP}\times \mathrm{FN}}{\sqrt{(\mathrm{TP}+\mathrm{FP})(\mathrm{TP}+\mathrm{FN})(\mathrm{TN}+\mathrm{FP})(\mathrm{TN}+\mathrm{FN})}}}$$

Progress of DL in Cancer Prognosis Prediction

Survival prediction is one of the most widely used areas in DL cancer prognosis analysis. It helps avoid patient overtreatment, provide decision support, and facilitate clinical research and drug development to improve treatment outcomes and quality of life for patients with cancer. At present, the vast majority of cancer prognosis research based on DL focus on predicting the survival of patients with cancer. The following focused on this specific prognostic question and analyzed the models that DL most commonly uses to predict cancer prognosis.

Table 1 summarizes the relevant studies on DL predicting cancer prognosis in the past 5 years, including study author (year of publication), cancer type, data type, network model, model performance, clinical endpoint of prognosis, and number of patients.

Table 1.

Recent Advances in DL-Based Cancer Prognosis Models.

Author (year)	Cancer type	Data type	Model	Performance	Clinical endpoint	Number of patients
Bhambhvani et al (2021) 18 and	Genitourinary rhabdomyosarcoma	Clinical and pathological data	DNN	AUC: 0.930	5-year survival	277
Li et al (2023) 19	Nonsmall cell lung cancer	Clinicopathological、Inflammatory、 Radiomics features (CT)	DNN	C-index: 0.712	Progression free survival	272
Chang et al (2023) 20	Diffuse large B-cell lymphoma	Clinical、Laboratory、 Metabolic parameters	DNN	ACC: 0.760 AUC: 0.720	10-year survival	122
Huang et al (2021) 21	Gastric cancer	H&E-stained pathological images	CNN	AUC: 0.657	Overall survival	1411
Wessels et al (2022) 22	Clear cell renal cell carcinoma	Haematoxylin and eosinstained slides	CNN	ACC: 0.655 AUC: 0.700	5-year survival	353
Ben et al (2022) 23	High-grade glioma	MRI image	CNN	ACC: 0.740	Overall survival	209
Li et al (2022) 24	Osteosarcoma	Clinical data	DBN	ACC: 0.793	5-year survival	1094
Wang et al (2022) 25	Esophageal squamous cell carcinoma	Clinical data,blood indicators	DBN	ACC: 0.897	5-year survival	298
Han et al (2022) 26	Lung cancer	CT image,clinical data	DRN	ACC: 0.825	disease-specific survival	198
Pham et al (2022) 27	Rectal cancer	Immunohistochemistry image	DRN	ACC: 0.875	5-year survival	80
Zhang et al (2023) 28	Nasopharyngeal carcinoma	MRI image,genetic data	DRN	AUC: 0.880	1-year progression free survival	151
Lo et al (2023) 29	Colorectal cancer	Colonoscopy image features Clinical features	ViT	ACC: 0.940 AUC: 0.930	Overall survival	437
Lian et al (2022) 30	Non-small cell lung carcinomas	CT image	ViT	AUC: 0.785	Overall survival	1705

Cancer Prognostic Model Based on Deep Neural Network

Deep neural networks (DNNs) are multilayer unsupervised neural networks consisting of an input layer, a hidden layer, and an output layer. ³¹ The nodes are fully connected between layers, the output features of the upper layer are used as the input of the next layer for feature learning, and the upper layer features are extracted implicitly by merging the lower layer features for cancer prognosis assessment.

Given the successful application of DL in other fields, Bhambhvani et al ¹⁸ attempted to develop and compare the performance of DNN with the traditional statistical model Cox proportional hazards in predicting the 5-year survival of rhabdomyosarcoma of the genitourinary system in children. Various clinical and pathological data, including age, sex ratio, and race, were included, and the final AUC of the DNN model was as high as 0.93, much better than that of the Cox proportional hazard model of 0.82, which initially confirmed the great potential of DNN to predict cancer prognosis. Li et al ¹⁹ trained a new multidimensional DNN model on the basis of clinicopathological, inflammatory, and radiomic features to predict the survival of patients with nonsmall cell lung cancer. DNN exhibited a better effect in the test cohort than the established Cox proportional hazard and random survival forest machine-learning models, with C-index values of 0.712, 0.665, and 0.679, respectively. It can be used as a noninvasive method to assist doctors in developing personalized treatment plans. Chang et al ²⁰ systematically compared the performance of DNNs with other machine-learning (logistic regression, random forest, and support vector machine) and DL (fuzzy neural network) models to evaluate 10-year survival in diffuse large B-cell lymphoma. A 10-times fivefold cross-validation approach was used to avoid randomness in the partitioning of the dataset and better reflect the true predictive performance of each model. By using clinical and metabolic parameters as input for each model, the final DNN demonstrated the highest prediction ACC of 0.76. However, it was not statistically significant compared with the other four models.

The above study shows that DNNs can better integrate multiomics data, including but not limited to clinical, pathological, and radiomics data, and have higher accuracy than traditional machine learning models in predicting cancer survival. ³² Therefore, DNNs are preferred for building cancer prognostic models when the input data type is numerical or categorical.

Cancer Prognostic Model Based on Convolutional Neural Network

Convolutional neural networks (CNNs) are usually composed of a convolution layer, a pooling layer, and a fully connected layer, among which the convolution layer is mainly used for image feature extraction. ³³ The first convolution layer can only extract some low-level features, such as edge, size, and shape, and more abstract features can be iterated only by combining the deep convolution layer. The pooling layer adopts the subsampling form, which is mainly used to reduce the dimension of the feature and maintain the feature shape unchanged to the maximum extent to improve the generalization performance of the network. The fully connected layer usually exists only in the last few layers of the network. The information sampled under the pooling layer is fully connected with the current layer, which is responsible for interpreting global feature information and performing advanced inference functions. N-dimensional vectors are often used as the output form, and the classification model analyzes N-dimensional vectors and outputs the classification results. The core idea is to apply the end-to-end learning mechanism, fully excavate the underlying features of the image, and then combine the loss function to modify the learned features repeatedly to achieve the best classification effect. ³⁴

In recent years, CNN has been widely used in the field of medical imaging to predict the prognosis of cancer patients. Huang et al ²¹ designed a new CNN-based model MIL-GC to achieve overall patient survival prediction directly from digital hematoxylin and eosin (H&E)-stained pathology images. The main idea is to first slice the pathology images and use CNN to initially diagnose the benign and malignant probability of each slice, select the suspicious slices as feature vectors output to a multilayer perceptron to calculate the survival probability of each slice, and then merge the probability values of all suspicious slices in the input images and use the average value as the final survival probability. The AUCs of the model on the internal and external validation sets were 0.671 and 0.657, respectively, indicating that the model can effectively predict the overall survival of patients with gastric cancer and provide a basis for clinical treatment plan selection with high robustness. Wessels et al ²² concluded that CNN-based image analysis has the potential to improve risk stratification in clear-cell renal cell carcinoma and investigated CNN-based extraction of relevant image features from H&E-stained slides to predict the 5-year overall survival in clear-cell renal cell carcinoma. The mean accuracy after 10-fold cross-validation was 0.655, with an AUC of 0.700. In addition, incorporating CNN predictions as an independent predictor into the multivariate logistic regression model can further improve the prediction performance. The results suggest that CNN image-based survival prediction is promising and could be integrated with existing clinical data. Ben et al ²³ applied 3D CNN to multisequence MRI (T1CE, T2, and Flair) data to predict the overall survival in patients with high-grade glioma. This method overcomes the limitation of small sample size of labeled medical images by fusing multisequence images. The final accuracy of the multisequence fusion CNN was 0.740, which was significantly better than those of T1CE (0.700), T2, (0.630) and Flair (0.650). The results showed that multisequence MRI images can improve the predictive performance of the model and provide reference for the development of multimodal CNN models.

The above study showed that CNN can be a good guide for survival prediction of cancer, and it can learn detailed features from image data that cannot be recognized by human eyes; the prognosis results of CNN can also be used as independent prognostic indicators. ³⁵ The feasibility of combining CNN prediction results with clinical and other data to improve the prediction performance was also verified. Multisequence image fusion is an effective means to improve the prediction performance of CNN.

Cancer Prognostic Model Based on Deep Belief Network

A deep belief network (DBN) consists of multiple unsupervised RBMS and a supervised back propagation neural network. ³⁶ In specific applications, DBN is usually divided into two stages: unsupervised feature learning and supervised fine-tuning. In the unsupervised feature learning stage, DBN takes the output of the bottom RBM as the input of the top RBM, uses the unsupervised greedy algorithm to train the RBM layer by layer from the bottom to the top layer, and realizes the transformation from the primary feature to the high-level abstract feature. In the supervised fine-tuning stage, the back-propagation neural network fine-tunes the advanced features obtained in the previous stage to achieve global optimization. Then, it uses these features as the initial input of DBN to fine-tune each network from the top layer to the bottom layer for supervised training to achieve the best classification effect of the model. The main idea of DBN is to initialize the model parameters layer by layer by using a greedy algorithm and then optimize the global parameter weights with supervised learning. ³⁷ This method effectively overcomes the problem of deep model training and improves the learning efficiency of the model.

Many methods to predict cancer prognosis based on DBN have been proposed, either entirely on the basis of DBN or in combination with other methods to obtain satisfactory results. Li et al ²⁴ compared the performance of DBN with six machine-learning models (decision tree, gradient boosting machine, logistic regression, random forest, parsimonious Bayes, and XGBoost) in predicting lung metastasis in patients with osteosarcoma on the basis of clinical data. The results showed that DBN significantly outperformed the machine-learning model in predicting lung metastasis, with an accuracy of 0.888. Meanwhile, the accuracy of the best among the six machine-learning models (XGBoost) was only 0.665. Thus, DBN was found to be very effective in detecting lung metastasis in patients with osteosarcoma. Subsequently, the DBN-based lung metastasis prediction model was integrated into the Cox proportional risk model as a parameter to predict the overall survival of patients with osteosarcoma. The predicted AUCs for 1-year, 3-year, and 5-year survivals were 0.851, 0.806, and 0.793, respectively. Wang et al ²⁵ proposed a novel DL network IAOA-DBN combining DBN and Archimedean optimization algorithm for predicting the 5-year survival of patients with esophageal squamous cell carcinoma. First, a minimum redundancy maximum correlation algorithm was used to screen statistically significant features. Then, DBN was introduced to predict the survival probability of patients. The design adopted an improved Archimedean optimization algorithm to optimize the learning rate and batch size of DBN to remove its influence, which is easily limited by the number of parameters. Initialization and perturbation functions were added to the improved Archimedean optimization algorithm to further improve the optimization efficiency and global search capability. Based on this, a survival prediction model for patients with esophageal squamous cell carcinoma was constructed and completed. IAOA-DBN demonstrated a higher accuracy of 0.897 than the relevant baseline network and the latest research model.

The combination of unsupervised and supervised learning gives DBNs a great deal of flexibility. Currently, DBNs and their derived models have shown powerful capabilities in cancer prognosis survival prediction, whether constructing prediction models based on macro features or combining macro and micro features. Compared with other models, DBN can perform unsupervised learning, which is a huge advantage in prediction tasks that may require labeled data. In the future, DBNs should be expanded with more complex network structures and richer node functions to facilitate the development of “DBN + cancer” prediction models.

Cancer Prognostic Model Based on Deep Residual Network

The basic constituent unit of a deep residual network (DRN) is a residual unit, which is stacked sequentially by a volume layer, a batch normalized layer, and a nonlinear active function layer. ³⁸ In the standard application of DRN, the input images successively go through the Conv layer to extract features, the BN layer to normalize the feature distribution, and the ReLU layer to add nonlinear factors. Then, they enter the N residual units to optimize the feature weights, map the features to the hidden space, and finally map the learned features to the sample space through the N fully connected layers and output the results. In specific clinical tasks, parameters such as convolution step size and convolution kernel size are sometimes adjusted in accordance with needs, and sometimes, a pooling layer is added after the residual unit to reduce feature dimension. DRN introduces the idea of identity mapping into the network to increase the transmission path of features. ³⁹ While ensuring that the effect of the deep network is not weaker than that of the shallow layer, DRN cleverly alleviates the problem of performance degradation so that the depth of the network could be superimposed to hundreds of layers.

After the CNN-based method became mainstream, the improved DRN on the basis of CNN has also been favored by an increasing number of scholars. Han et al ²⁶ proposed an innovative multicycle data fusion DRN to predict the survival of patients with lung cancer by integrating longitudinal CT images and clinical data at different periods. The results showed that the prediction accuracy of three-cycle fusion DRN was as high as 0.825, which was better than 0.799 (average accuracy) of single-cycle DRN and 0.803 of clinical data. This difference reflected that the fusion of multicycle data can provide enhanced results in terms of accuracy and help patients with lung cancer predict survival more effectively. Pham et al ²⁷ emphasized the importance of adding nonlinear dynamics and long short-term memory to the original DRN. A feature-optimized DRN was designed to predict the 5-year survival of patients with rectal cancer. The results showed that the proposed ResNet-FRP-LSTM is superior to the comparison experiments that were based on ResNet alone and DenseNet and NASNet, with accuracies of 0.875, 0.750, 0.750, and 0.750, respectively. The study confirmed the great potential of DRN and its derived models for predicting survival in patients with rectal cancer. Zhang et al ²⁸ used DRN to develop a prediction model for 1-year progression-free survival after intensity-modulated radiotherapy in patients with nasopharyngeal carcinoma and compared it with the conventional texture analysis. The final AUC of the DRN model based on MRI images was 0.85, and the AUC was improved to 0.88 after combining genetic data, both higher than that of texture analysis at 0.76. This approach provides accurate survival prediction for patients with nasopharyngeal carcinoma prior to radiotherapy, and it does not require clinicians to manually identify tumor target areas, thereby avoiding human effect on prediction performance.

DRNs have been a hot research topic in the field of cancer prognosis prediction, and it has been widely used in 3D medical images. They have better fitting effect for multiperiod image data fusion than for single-period image data. Its special residual structure gives DRN good portability, which provides convenience for combining with other modules. Furthermore, in terms of improving model prediction performance, the combination of DRN-based image DL features with genetic features provides ideas for future research. However, further confirmation by multicenter and large sample prospective studies is still needed.

Cancer Prognostic Model Based on Vision Transformer

Vision transformer (ViT) uses the original Transformer architecture (a continuously stacked 6-layer encoder–decoder architecture), and it is based entirely on the self-attention mechanism. ⁴⁰ In ViT, the input image is converted into a series of blocks, the spatial position of each block is encoded separately by 6 layers of encoders to provide spatial information, and classification flag bits are introduced to represent global information. The spatial and positional information is then fed together to the decoder layer to calculate multiple attention and output the learned block information. Finally, a multilayer perceptron is used to integrate the block information for prediction.

The development of ViT provides a new research scheme for the prognosis prediction of cancer. Lo et al ²⁹ predicted the overall survival by using 1729 colonoscopy images of 437 patients with colorectal cancer on the basis of the powerful global feature extraction capability of ViT. The proposed feature ensemble classifier FEViT combines the colonoscopy features and clinical features of the pretrained ViT to build a prognostic model. The results showed that the accuracy and AUC of FEViT in predicting patient survival were 0.940 and 0.930, better than those of the traditional TNM staging (0.900 and 0.830, respectively). ViT is proven to have good accuracy in predicting survival and expected to be an effective tool for survival prognosis of patients with colorectal cancer. Although pure ViT has been able to achieve satisfactory results, research scholars have invested a considerable amount of time to explore ViT in combination with other networks to capture more complex data distributions or obtain better performance. Lian et al ³⁰ proposed a model combining ViT and graph neural network to predict the overall survival of patients with nonsmall cell lung cancer. This model combines the advantages of ViT and graph neural network in tracking the underlying features of images and capturing advanced information across channels. The results showed that the AUCs of ViT on the internal and external test sets were 0.785 and 0.695, respectively, better than those of the classical TNM (AUC = 0.690, 0.634) and ResNet-Graph (AUC = 0.730, 0.626) models. Therefore, feasibility of using ViT to generate features of cancer prognosis survival analysis was demonstrated.

The above studies demonstrated the powerful performance of building a ViT cancer survival prognostic model. The self-attentive mechanism in ViT comes with long-range dependence capability, which makes it more efficient and scalable than the traditional DL models and show a ride on large-scale datasets. Moreover, its integration with other models can further improve the prediction accuracy. Other prognostic factors, such as common genetic and histological data, were rarely studied to integrate them into ViT. However, with the growth of time and cancer cases, subsequent studies could combine the two, and ViT with combined genetic or histological data is believed to be more helpful for the accurate prediction of patient prognosis.

Comparison of Different DL Cancer Prognostic Models

Table 2 shows the proposed time, advantages and disadvantages, applicable data types, and development trends of five mainstream DL cancer prognosis prediction models: DNN, CNN, DBN, DRN, and ViT. Choosing the appropriate model in accordance with the specific clinical task is convenient for relevant researchers.

Table 2.

Comparison of Different DL Cancer Prognosis Models.

Model	Time of presentation	Advantage	Shortcoming	Applicable data type	Development trend
DNN	1969	More detailed and efficient representation of complex nonlinear problems; Autonomous learning	Changes in time series cannot be modeled; Easy to overfit	Various omics data and numerical data	Reduce model complexity and operation time; Explore explainability
CNN	1998	Sparseconnection; Parameter sharing; Good feature extraction ability	Gradient disappearance; Input-output fixation; More training data is required	Image data such as CT; Time series numerical data such as EEG data; Textual data (medical records, documents, etc)	Develop deeper CNN; Reduce network computing time
DBN	2006	Can learn data distribution well; Insensitive to input dimensions	Long training time; Models are not explanatory	Image data; Numerical data such as gene expression data; Text data with missing value classes	Improve the model structure and increase the training speed; Combined with advanced models
DRN	2015	Able to train deeper neural networks; Improve network performance; Avoid gradient vanish	Increase the amount of computation and training time; Many parameters, easy to overfit	Image data; Numerical data such as genetic data	Combining attention mechanisms and other techniques to improve model performance; Explore explainability
ViT	2017	Greater long-range dependence and scalability; Feature extraction can be carried out using global information	Large amount of calculation; Need more training data	Image data; Numerical data such as genetic data	Integration of advanced mechanisms; Developed specifically for medical image models

Key Factors Influencing and Enhancing the Performance of DL Prognostic Model

In this article, an in-depth analysis of existing DL cancer prognostic prediction models identified a series of public factors that can directly or indirectly affect prediction performance, which is difficult to address through individual efforts and require the development of identical criteria to eliminate them. Scattered performance enhancement techniques from existing studies were summarized to help researchers in this field make more robust cancer prognostic prediction models. These factors are explained in detail below.

Public Factors Affecting Performance

Medical imaging: All kinds of medical imaging equipment generate noise to a greater or lesser extent, which can cause interference during the image acquisition process and physical artifacts to appear in the acquired images. Motion artifacts can also occur if the examined object moves during image acquisition. However, these noises and artifacts are unavoidable, and no standard method is available to eliminate them. ⁴¹

Datasets: Currently, few publicly available, freely accessible online datasets on cancer prognosis prediction can be found, leading to the majority of studies being based on private datasets, which are often of variable quality and cannot be shared for various reasons, including ethical reasons. This limitation restricts the comparison of model performance across datasets.

Machine hardware: DL models needing to train a large amount of data, which is associated with millions of parameters, places a high demand on the memory capacity of the GPU. However, the same model may have different convergence results under different GPU performances, greatly affecting the design and performance of the model.

Enhanced Performance Factors

Multitechnology, multimodal image, or multiomics data fusion: Using multitechnology (such as DL + machine learning), multimodal images (such as CT + MRI or ultrasound + CT + MRI), or multi-omics data (such as clinical data, genetic data, and image imaging features) fusion to predict cancer prognosis during training can further improve prediction accuracy and reduce overfitting. ⁴²

Transfer learning: Transfer learning is an effective method to train networks in the case of small data sets. Previous studies conducted pretraining on a large number of natural images or other modal images and then directly transferred to their own research images for retraining, achieving remarkable results. ⁴³

Ensemble learning: Ensemble learning averages the accuracy of multiple prediction models to obtain the final result. Research showed that some simple prediction models can show the same prediction accuracy as complex architectures. ⁴⁴

Regularization: Small sample learning is a common problem in the medical field. Most DL models have only a few hundred patient samples. In these studies, overfitting problems and hard convergence problems of DL networks often arise. Regularization is an effective solution; for example, early stopping and dropout layers can prevent model overfitting, and batch normalization can improve the convergence speed of the model.^45,46

Summary and Prospect

The research of DL has achieved remarkable progress and gradually began to improve people's daily life. In the medical field, many clinical studies combined with DL and achieved great success. DL models have revolutionized the prognosis prediction of cancer. However, applying DL prognostic models to clinical practice still has limitations.

1. Interpretability is a prerequisite for clinical application. However, the current interpretation for DL model decisions remains at the macro level, such as heat map, which has not yet reached the requirement of clinical application. Therefore, the research of strong interpretable methods will be a continuous and unchanging hot spot in the field of DL cancer prognostic models.

2. Most of the current cancer prognosis prediction studies were conducted on single-center data only, and achieving similar prediction levels on other central data sets is difficult. However, in clinical practice, the acquisition of various data can vary greatly due to different equipment, parameters, and operators, resulting in low model generalization ability. Therefore, how to improve the generalization performance of the model and ensure that stable prediction levels are still maintained in data sets with large individualized differences, such as different hospitals and different populations, are the directions that future research should focus on.

In this article, the recently published studies on DL in cancer prognosis were summarized. Most of them showed that the performance of the DL model is the best at present. With the continuous improvement and refinement of the technology, the use of DL to predict cancer prognosis is believed to have a broad prospect and gradually move towards clinicalization. The research of the DL cancer prognostic model could better cure patients with cancer.