Deep Learning for Cancer Detection Based on Genomic and Imaging Data: A Comprehensive Review

Abstract

Cancer is a major global health challenge, and early detection is critical to improving survival rates. Advances in genomics and imaging technologies have made the integration of genomic and imaging data a common practice in cancer detection. Deep learning, especially Convolutional Neural Networks (CNNs), demonstrates substantial potential for early cancer diagnosis by autonomously extracting valuable features from large-scale datasets, thus enhancing early detection accuracy. This review summarizes the progress in deep learning applications for cancer detection using genomic and imaging data. It examines current models, their applications, challenges, and future research directions. Deep learning introduces innovative approaches for precision diagnosis and personalized treatment, facilitating advancements in early cancer screening technologies.

Introduction

Cancer remains one of the leading causes of disease burden and mortality globally, posing a significant public health challenge. Early detection is vital for improving survival rates, as it allows for timely intervention and more effective treatment strategies. The rapid development of high-throughput technologies has made genetic and imaging data essential for cancer detection and diagnosis. Combining these data types offers a comprehensive perspective, ranging from the molecular to the structural level. In recent years, deep learning techniques, particularly convolutional neural networks (CNNs),¹ have demonstrated considerable potential in cancer detection, significantly enhancing early detection accuracy and efficiency by autonomously extracting complex features from large-scale datasets.

Role of Genomic and Imaging Data in Cancer Detection

The Role of Genetic Data in Cancer Detection

Whole Genome Data (WGD) encompasses the complete DNA sequence of an individual and identifies genetic variants associated with cancer, including mutations, copy number variants, and structural variants.^2,3 These variants can be quantified using the following equation:

(1)

Where denotes the effect function of the mutation and is the weight of the mutation location. This formula helps to assess the contribution of different mutations to cancer development.

This information can be utilized for early cancer detection and risk assessment. For example, mutations in BRCA1 and BRCA2 are strongly associated with an elevated risk of breast and ovarian cancer.⁴ Somatic mutation data helps identify specific molecular features of cancers, guiding the selection of targeted therapies.^5,6

The Role of Imaging Data in Cancer Detection

Imaging techniques are crucial for the early detection, diagnosis, and treatment monitoring of cancer. CT and X-ray are commonly used to screen for lung, bone, and other cancers, providing high-resolution images that help identify tumor location, size, and morphology.^7,8 In contrast, MRI is advantageous for soft-tissue imaging and is commonly used to detect brain tumors, prostate cancer, and breast cancer.^9,10 Pathology images, derived from tissue biopsies, are the gold standard for cancer diagnosis. The advent of digital pathology has facilitated the storage, sharing, and analysis of these images.¹¹

Application and Development of Deep Learning in Cancer Detection

Improvements in deep learning techniques, particularly convolutional neural networks (CNNs), have shown significant advantages over traditional methods in cancer detection. Deep learning can automatically extract features from data, reduce human intervention, and enhance detection accuracy. By integrating genetic and imaging data, deep learning provides more effective support for accurate cancer detection.^12,13 In addition to CNNs, emerging deep learning models, such as the Transformer and graph neural networks (GNNs), demonstrate great potential in cancer detection. These models can better capture global features and topological relationships within complex data.^14,15

Advantages and Challenges of Deep Learning

One of the primary advantages of deep learning is its strong model adaptability, which allows it to be applied to various cancer detection tasks through transfer learning, thereby reducing the reliance on large-scale labeled data.¹⁶ However, challenges remain in the application of deep learning for cancer detection, such as data quality, model interpretability, and clinical feasibility.^17,18 Although Transformer and GNN models have demonstrated significant performance improvements, their interpretability and computational complexity remain key areas for future research.^19,20

In conclusion, both genetic and imaging data offer unique advantages in cancer detection, and deep learning technologies provide a promising approach for accurate diagnosis and treatment. With the continued optimization and implementation of deep learning models, these technologies will play an increasingly critical role in cancer diagnosis and treatment in the future.

Current Challenges and Future Plans for Deep Learning in Cancer Detection

Deep learning demonstrates significant potential in cancer detection using genomic and imaging data. However, several challenges remain, including difficulties in data acquisition, data heterogeneity that affects model generalization, lack of model interpretability that limits clinical applications, complexity in multimodal data fusion, and issues with validation and application in real clinical settings. Further research and planning are required.

Data Quality and Quantity

High-quality, large-scale labeled data is essential for training deep learning models. However, access to medical data is restricted by privacy protections, ethical standards, and data-sharing mechanisms, resulting in data scarcity.^21,22 Additionally, data heterogeneity, such as variations in imaging equipment and gene sequencing platforms across hospitals, can lead to differences in data distribution, thereby affecting the generalization ability of models.¹⁴

Model Interpretability and Transparency

Deep learning models are often considered “black boxes” and lack interpretability, which limits their application in clinical settings.²³ Both doctors and patients must understand the model’s decision-making process to build trust and ensure the reliability of the diagnosis.²⁴ Therefore, it is essential to develop models with interpretability features or visualization tools that support decision-making.

Multimodal Data Fusion

The effective fusion of genomic and imaging data can provide more comprehensive information for cancer detection. However, feature extraction and fusion strategies for these different data types are not yet fully developed, which may lead to information loss or the introduction of noise, ultimately affecting model performance.^17,25

Clinical Validation and Application

Although deep learning models have shown promising results in research, their validity and reliability in real clinical settings still need to be validated.²⁶ These models must undergo rigorous clinical trials to ensure their applicability across different populations and environments. Additionally, the deployment of models must consider factors such as computational resources, cost, and physician acceptance.²⁷

Future Planning

To address the challenges outlined above, the following areas should be prioritized in the future:

Data Sharing and Standardization: Establish a secure, compliant data-sharing platform and promote multicenter collaboration to obtain diverse and high-quality data. Additionally, develop standardized protocols for data collection and labeling to reduce the impact of data heterogeneity on model performance.^21,22

Model Interpretability Research: Develop interpretable deep learning models or combine traditional machine learning methods to enhance model transparency and improve clinical acceptability.^23,24

Multimodal Fusion Methods: Explore effective strategies for multimodal data fusion to fully leverage the complementary information from genomic and imaging data, thereby improving the accuracy and robustness of cancer detection.^17,25

Clinical Translation and Validation: Strengthen the clinical validation of models by conducting multicenter, large-scale clinical trials to assess their practical applications and facilitate their integration into clinical practice.^26,27

In conclusion, while deep learning holds significant promise for cancer detection, continued efforts are required to overcome challenges related to data, models, and clinical applications. Multi-party collaboration is essential to drive technological advances and achieve the goal of precision medicine.²⁸

Scope and Organization of the Review

This paper reviews deep learning methods for cancer detection based on genomic and imaging data, with a focus on their application in early screening, diagnosis, and prognosis prediction. The paper is organized as follows: Chapter 2 introduces the basic principles of deep learning techniques and their applications in medicine, emphasizing common models such as convolutional neural networks (CNNs). Chapter 3 provides a detailed analysis of the progress in deep learning for cancer detection using combined genomic and imaging data, exploring their application in early screening and diagnosis. Chapter 4 discusses the application of deep learning in multimodal data fusion, highlighting recent research on integrating genomic and imaging data. Finally, Chapter 5 addresses the current challenges of deep learning techniques in cancer detection, including issues related to data quality, model interpretability, and clinical feasibility, and outlines future research directions. Through these discussions, this paper aims to provide a reference for advancing precision medicine.

Deep Learning Methodologies

Overview of Deep Learning Architectures

Deep learning, a key branch of machine learning, has made significant advancements in cancer detection in recent years. Various deep learning architectures have been proposed and widely applied for early diagnosis, prognosis assessment, and treatment selection in cancer care. These architectures include Convolutional Neural Networks (CNNs),²⁹ Recurrent Neural Networks (RNNs) and their variants (eg, Long Short-Term Memory Networks (LSTMs)³⁰ and Gated Recurrent Units (GRUs)), Generative Adversarial Networks (GANs),³¹ Transformer Networks,³² and Graph Neural Networks (GNNs).³³ In cancer detection, these models can process complex genomic and medical imaging data to automatically extract valuable features and enhance diagnostic accuracy.

Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) are the most widely used class of deep learning architectures, particularly prominent in the field of image processing. CNNs automatically extract key features from images, such as edges, textures, and shapes, by locally sensing the input data through convolutional layers. The formula for the convolution operation can be expressed as follows:

(2)

Where is the input image, is the filter, and is the offset of the convolution. This local sensing mechanism enables the CNN to effectively capture spatial features in the image.

In addition, the pooling operation is acrucial step in CNNs used to reduce the dimensionality of the feature map. Pooling operations extract the most salient features from an image while reducing computational complexity and preventing overfitting. Common pooling techniques include Max Pooling and Average Pooling. The formulas are as follows:

(3)

(4)

The main advantage of CNNs is that they do not rely on manual feature extraction and can automatically learn more discriminative features from large datasets. In cancer detection, CNNs are widely used in medical image analysis. For instance, in CT image analysis for lung cancer, CNNs can identify and classify lung nodules to determine whether they are malignant.^34,35 In early breast cancer screening, CNNs are used for the automatic analysis of mammogram images, enabling the detection of small lumps and improving diagnostic accuracy.³⁶

Recurrent Neural Networks (RNNs) and Their Variants

Recurrent Neural Networks (RNNs) are well-suited for processing sequence data and are characterized by their ability to model temporal dependencies, preserving information from previous time steps. This makes RNNs particularly advantageous for processing genetic data, medical records, and other time-series data.

Standard RNNs suffer from the vanishing gradient problem, which limits their effectiveness in processing long sequences. To address this issue, Long Short-Term Memory Networks (LSTMs) and Gated Recurrent Units (GRUs) have been introduced. These variants mitigate the vanishing gradient problem by incorporating agating mechanism. LSTMs and GRUs are widely used in genomics, particularly in cancer prediction and progression analysis.^37,38 For instance, LSTMs are used to predict the occurrence and progression of cancer based on gene expression data,³⁹ while GRUs are employed to detect cancer-associated mutations and analyze temporal patterns in gene sequences.⁴⁰ The update formula of LSTMs can be expressed as follows:

(5)

Where , , denote the input gate, forget gate and output gate respectively, and is the candidate cell state, which can effectively deal with the timing dependence in sequence data.

Generative Adversarial Networks (GANs)

Generative Adversarial Networks (GANs) are atype of generative model consisting of agenerator and adiscriminator, which mutually enhance each other through adversarial training. The generator produces synthetic data, while the discriminator evaluates the authenticity of the data. GANs have significant applications in medical imaging, particularly in image enhancement and data generation.

In cancer detection, GANs are widely used for medical image generation and enhancement. For example, GANs can generate high-quality CT or MRI images, thereby improving the diagnosis of low-quality images.^41,42 Additionally, GANs can be employed for data augmentation, generating more labeled images to help train more accurate deep learning models and enhance the performance of cancer detection.⁴³

Transformer Network

The Transformer network represents asignificant breakthrough in deep learning in recent years. It employs aself-attention mechanism to capture dependencies between positions in asequence. Unlike traditional RNNs, the Transformer can process the entire sequence in parallel, which enhances computational efficiency and yields remarkable results across several tasks.

In cancer detection, the Transformer is widely used to analyze both images and genetic data. The Visual Transformer (ViT) is used to analyze pathology images, improving the accuracy of image classification by dividing the image into multiple segments and capturing the relationships between these segments using the self-attention mechanism.^44,45 Additionally, the Transformer has been applied to genomic data analysis, particularly in cancer risk prediction, to effectively capture long-term dependencies between gene mutations.⁴⁶

Graph Neural Networks (GNNs)

Graph Neural Networks (GNNs) are aclass of deep learning models designed for processing graph-structured data. In cancer detection, GNNs are widely used to model molecular-level cancer information.^47,48 GNNs can learn the relationships between nodes and edges to identify complex interactions between genes, proteins, and cancer phenotypes.⁴⁹

GNNs have demonstrated success in predicting cancer-related gene mutations. For instance, GNNs are used to analyze gene interaction networks and identify key genes and biomarkers associated with cancer.^50,51 Additionally, GNNs can be applied to medical image analysis by segmenting images into multiple regions and capturing the relationships between these regions through graph structures, thereby improving the accuracy of tumor detection and classification.^52,53

Feature Extraction and Representation Learning in Deep Learning

Within the framework of deep learning, feature extraction and representation learning are critical steps for understanding and processing data. Unlike traditional machine learning methods, which rely on manually designed features, deep learning automatically extracts features from raw data through multi-layer neural networks and learns effective representations of the data. This significantly enhances the efficiency and accuracy of cancer detection. Particularly in the processing of genetic and imaging data, deep learning technologies provide robust support for early cancer diagnosis by automatically learning the intrinsic structure of the data and identifying potential correlations.

Feature Extraction and Representation Learning for Genetic Data

Gene data, particularly gene expression data and genomic sequence data, are characterized by high dimensionality, sparsity, and complexity. Traditional feature engineering approaches typically rely on domain knowledge for manually extracting features and selecting appropriate models for prediction. However, the complexity and diversity of genetic data make manual feature extraction challenging, as it may fail to capture all important information, potentially leading to overfitting and bias.

Deep learning addresses these challenges through automatic feature extraction and representation learning. The application of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) in genetic data analysis, especially in gene expression and DNA sequence data, has shown significant advantages. By training multi-layer networks, deep learning can automatically uncover complex patterns and potential relationships in the data, leading to more accurate identification of cancer-related genes.

For instance, CNNs can capture local dependencies in gene expression data through their convolutional layers and extract features that are discriminative for cancer prediction.^54,55 RNNs and their variants (eg, LSTM and GRU) can efficiently capture temporal dependencies in gene sequences, offering anovel approach to detecting cancer-related mutations.^56,57 Additionally, Graph Neural Networks (GNNs) have made notable progress in genetic data processing in recent years. GNNs efficiently represent gene interaction networks by treating genes as nodes in agraph, extracting relationships and dependencies between genes through graph convolution operations. GNNs have shown strong performance in predicting cancer gene mutations and are adept at capturing the complex interaction patterns between genes.^58,59

Feature Extraction and Representation Learning for Image Data

Medical images, particularly CT, MRI, and pathology images, play acrucial role in cancer detection. Traditional image processing methods rely on hand-designed feature extraction algorithms, such as edge detection and texture analysis. Although effective, these methods often fail to extract sufficiently informative features when dealing with complex medical image data, leading to low classification and diagnostic accuracy.

The application of deep learning, especially in Convolutional Neural Network (CNN) architectures, has significantly advanced medical image processing technology. CNNs can effectively identify critical information, such as tumor morphology, size, and location, by automatically learning features from low-level to high-level image data. CNNs have achieved remarkable results in tumor detection, segmentation, and classification. For instance, CNNs can automatically recognize lung nodules in CT images and determine their malignancy.^60,61 In breast cancer screening, CNNs are used to analyze mammography images, extracting features of tiny masses, thereby improving early diagnostic accuracy.^62,63

Deep learning not only excels at extracting visual features from medical images but also plays apivotal role in multimodal data fusion. By integrating genetic and image data, deep learning can extract features from multiple dimensions, providing more comprehensive information for cancer diagnosis. For example, by combining genetic data with CT images, deep learning can more accurately predict cancer progression and metastasis.^64,65

Deep Learning vs Traditional Feature Engineering Methods

Traditional feature engineering methods typically rely on expert domain knowledge to manually extract features and select appropriate models for training. For instance, atraditional approach might involve selecting features manually and representing them with the following formula:

(6)

Where, is the representation of the manually extracted features, is the features extracted from the domain knowledge and is the weights of the features.

In contrast, deep learning can automatically learn efficient representations of data through multi-layer neural networks, bypassing the limitations of manual feature extraction. For instance, Convolutional Neural Networks (CNNs) can automatically extract the morphological features of tumors from images. Traditional methods are susceptible to human bias during the feature selection process and are less efficient in processing complex data, particularly when dealing with high-dimensional and complex datasets, such as genomic data and medical images. Their effectiveness is often limited.^44,45 For example, in genomic data processing, traditional methods typically extract only a limited set of features and may not effectively identify various types of gene mutations.⁶⁶ In contrast, deep learning automatically learns effective feature representations from large datasets through the training of multi-layer neural networks, eliminating the constraints of manually designed features.⁶⁷ Deep learning can not only extract low-level features (eg, edges and textures in images) but also learn high-level semantic representations of the data, enabling it to capture complex patterns more effectively in cancer detection. For example, the application of CNNs in medical image analysis automatically extracts morphological features of tumors, avoiding the need for manual feature extraction.⁶⁸ Moreover, deep learning excels at handling large-scale datasets, particularly in the fusion of multimodal data, where it offers significant advantages. For example, combining genetic and medical imaging data through joint training of deep learning models can substantially enhance the accuracy of early cancer diagnosis.⁶⁴ In multimodal data fusion, deep learning further improves prediction accuracy and provides more precise information for personalized treatment by automatically learning the associations between datasets.³⁹

Overall, deep learning outperforms traditional feature engineering methods in terms of accuracy and efficiency, particularly when handling large-scale and complex data. Its advantages become increasingly evident as the data complexity grows.^34,69 With the continuous advancement of deep learning technology, it is expected to further accelerate progress in cancer detection.⁷⁰

Training and Optimization of Deep Learning Models

In deep learning, model training and optimization are crucial for achieving efficient performance. The training process involves selecting appropriate loss functions, optimization algorithms, and regularization techniques, all of which directly influence the convergence speed and generalization capability of the model. Additionally, effective training strategies, such as data augmentation and transfer learning, are particularly important in addressing issues like data insufficiency and class imbalance. This section will examine these factors in detail and explore how they can be leveraged to improve the performance of deep learning models in cancer detection applications.

Selection of the Loss Function

The loss function is a crucial component in deep learning model training, as it determines the optimization direction and goal of the model. In cancer detection, commonly used loss functions include Cross-Entropy Loss (CEL)⁷¹ for classification tasks and Mean Squared Error (MSE)⁷² for regression tasks.

For binary classification tasks, such as distinguishing between benign and malignant tumors, cross-entropy loss is used due to its ability to effectively measure the difference between predicted values and actual labels.⁷³ It is widely applied, and its formula is as follows:

(7)

Where is the True Label (0 or1) and is the Probability Predicted by the Model.

In multi-category classification problems, weighted cross-entropy loss (Equations 1 and 2) is commonly used to address class imbalance. It is formulated as follows:

(8)

Where is the number of categories, is the weight of category , is the true label, and is the probability predicted by the model.

Weighted cross-entropy loss enables the model to enhance its ability to recognize less frequent classes by assigning different weights to each class, making it particularly useful in medical image classification.^74,75

Optimization Algorithms

The choice of optimization algorithm directly affects the training speed and convergence of deep learning models. Commonly used optimization algorithms include stochastic gradient descent (SGD)⁷¹ and its variants, such as Adam⁷⁶ and RMSprop,⁷⁷ among others.

Stochastic Gradient Descent (SGD) is the most basic optimization method, updating model parameters by calculating the gradient of each training sample. Despite its simplicity and effectiveness, SGD can be slow to train and prone to converging to local optima. Therefore, SGD is often used in conjunction with Momentum to accelerate the convergence process.⁷⁸

The Adam optimization algorithm (Adaptive Moment Estimation) is one of the most widely used optimization algorithms. Adam combines the advantages of gradient descent by calculating both the first-order andsecond-order moment estimates of the gradient (ie, momentum and adaptive learning rate), which allows the model to converge faster on complex datasets and improves its robustness, especially when dealing with sparse data.⁷⁹ In cancer detection, the Adam optimization algorithm has become the preferred choice for processing medical image data (eg, MRI and CT images) due to its strong performance and stability.⁸⁰

Regularization Techniques

The complexity of deep learning models makes them susceptible to overfitting, particularly when the available data is insufficient. To address this issue, regularization techniques are commonly used to enhance the generalization ability of the models. Common regularization methods include L1 and L2 regularization, as well as Dropout. The formula for regularization is as follows:

(9)

Where is the Model Parameter, λ is the Regularization Parameter, and is the Total Number of Parameters.

(10)

Where is the Model Parameter, λ Is the Regularization Parameter, and is the Total Number of Parameters.

(11)

Where is the dropout rate (ie, the probability of dropping a neuron during each training session), and is the output after Dropout processing.

L1 regularization reduces feature redundancy by introducing an L1 penalty term, which sparsifies the model parameters.⁸¹ L2 regularization stabilizes the model by introducing an L2 penalty term, improving the convergence speed and making the model more stable.⁸² L1 and L2 regularization are often used in combination to achieve better results in various tasks.

Dropout is another common regularization technique that reduces the risk of overfitting by randomly “dropping” a portion of neurons during each iteration. This prevents the model from relying too heavily on specific features.⁸³ Dropout is widely used in deep convolutional neural networks (CNNs) for cancer image analysis, particularly in tumor classification and segmentation tasks, and effectively enhances the model’s generalization ability.⁸⁴

Cancer Detection Model Assessment Metrics

Evaluating cancer detection models is a critical step in ensuring their validity and reliability. Common evaluation metrics include Accuracy, Sensitivity, Specificity, F1 Score, and Area Under the Receiver Operating Characteristic (ROC) Curve (AUC). These metrics provide a comprehensive assessment of the model’s performance across different dimensions and help researchers evaluate its practical applicability.

Accuracy

Accuracy is the most intuitive assessment metric and is defined as the ratio of correctly predicted samples to the total number of samples. The formula is as follows:

(12)

In cancer detection, accuracy measures the overall predictive performance of the model. However, accuracy is sensitive to class imbalance and, in some cases, may not accurately reflect the model’s true performance.⁸⁵

Sensitivity and Specificity

Sensitivity (also known as recall) is the proportion of actual positive samples that the model correctly predicts as positive. It measures the model’s ability to detect true positives (eg, malignant tumors). The formula is as follows:

(13)

Specificity, on the other hand, is the proportion of actual negative samples that the model correctly predicts as negative. It measures the model’s ability to rule out false positives (eg, benign tumors). Sensitivity is particularly important in cancer detection because it determines whether a tumor can be detected in time.^86,87 The formula is as follows:

(14)

F1 Values

The F1 score is the harmonic mean of sensitivity and precision, commonly used to evaluate models on unbalanced datasets. It integrates both the model’s detection and accuracy rates. The formula is as follows:

(15)

(16)

F1 scores are particularly important in cancer detection, especially when testing for minority classes (eg, rare cancer types), as they help avoid the bias associated with relying on accuracy alone.⁸⁸

ROC Curves and AUCs

The Receiver Operating Characteristic (ROC) curve is used to evaluate the performance of binary classification models. It illustrates the relationship between sensitivity and the false positive rate (1-specificity) at various thresholds. The formula is as follows:

(17)

The area under the curve (AUC) is the integral of the ROC curve. A larger AUC value indicates better model performance across various classification thresholds. The AUC provides a comprehensive measure of the model’s classification performance and is a crucial evaluation metric in cancer detection.^89,90

Cancer Detection Using Genomic Data

Whole Genome Data Analysis

Deep learning can automatically extract features from a large number of genetic markers by building multi-layer neural networks to identify genetic variants associated with cancer. Convolutional Neural Networks (CNNs) have been widely used in genomic data analysis. CNNs extract cancer-associated features by learning spatial relationships between genetic markers. The CNN model proposed by Wu et al (2020) successfully identified multiple SNP loci associated with breast cancer.⁹¹ In addition, Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTMs) also perform well in Genome-Wide Association Studies (GWAS), particularly in processing gene sequence data. These models capture temporal dependencies in gene expression, improving the prediction accuracy of cancer-related genes.^92,93

Graph Neural Networks (GNNs), as an emerging deep learning method, effectively model complex relationships between genes by representing gene interactions as graph structures. For example, Xie et al (2021) used GNNs to analyze gene interaction networks and successfully improved the recognition accuracy of cancer-related genes.⁹⁴ GNNs can discover potential cancer-related genetic variants by learning the relationships between nodes (genes).

Advantages of Deep Learning in GWAS

Compared to traditional statistical methods (eg, linear regression), the application of deep learning in Genome-Wide Association Studies (GWAS) offers significant advantages. First, deep learning can automatically extract features from data, overcoming the limitation of traditional methods that rely on manual feature selection. While traditional GWAS often requires expert input to select genetic markers, deep learning models can automatically learn important features from training data, thus improving the efficiency and accuracy of the models.⁹⁵

Second, deep learning effectively handles high-dimensional and complex data. In GWAS, genetic data typically contains millions of SNP markers, while the number of samples is often limited. Traditional statistical methods are easily affected by the curse of dimensionality, leading to overfitting. Deep learning addresses the issue of high-dimensional data by reducing dimensionality and effectively extracting useful information through multi-layer neural networks.⁹⁶ The formula is as follows:

(18)

Finally, deep learning can capture complex non-linear relationships between genetic markers. Traditional methods assume a linear relationship between genes and disease, but the actual relationship is often more complex. Deep learning better fits these complex relationships through its non-linear structure, improving the accuracy of identifying cancer-related genetic variants.^97,98 The formula is as follows:

or

(19)

Deep Learning Models for Cancer-Related Gene Marker Identification

The occurrence of cancer is closely linked to genetic variation, particularly the types of SNP (single nucleotide polymorphism) markers and mutations associated with cancer. Traditional GWAS methods rely on statistical techniques and often require manual selection of specific genetic markers to analyze their association with cancer. In contrast, deep learning can recognize complex genetic patterns by automatically extracting features from large-scale data, thereby improving the accuracy of identifying cancer-related genetic markers (Table 1).

Table 1.

Deep learning models for identifying cancer-related gene markers

Model Type	appliance	study
Convolutional Neural Network (CNN)	Local relationship learning for genetic markers, eg breast cancer GWAS data analysis	Successful identification of multiple genetic markers associated with breast cancer.99
Recurrent Neural Network (RNN) and LSTM	For gene sequence data analysis to capture timing dependencies and identify lung cancer-related genes	Successful identification of key genes associated with lung cancer.100,101

Deep Learning in Cancer-Related Biological Pathway Mining

In addition to genetic markers, cancer development is closely linked to a complex network of biological pathways. Traditional GWAS primarily focuses on the role of individual genes and often overlooks the biological pathways where multiple genes act synergistically. In contrast, deep learning models can uncover cancer-related biological pathways by exploring the interrelationships between genes (Table 2).

Table 2.

Deep learning models for mining cancer-related biological pathways

Model Type	Appliance	Study
Graph Neural Networks (GNN)	For modeling cancer gene interaction networks, discovery of PI3K-Akt and MAPK pathways	Revealing important cancer-related biological pathways102
Generative Adversarial Networks (GAN)	For generating cancer gene mutation networks and mining gastric cancer-related pathways	Validation of important pathways associated with gastric cancer103

Deep learning models can not only identify cancer-related genetic markers and biological pathways but also provide new insights into the pathogenesis of cancer. By automatically learning from genes and biological pathways, deep learning can reveal interactions between different genetic variants and explore the key factors contributing to cancer development. For example, the deep learning framework proposed by Zhang et al successfully integrated genomic and phenotypic data, uncovering the genetic basis of multiple cancer types.^104,105

Additionally, deep learning can aid in understanding cancer heterogeneity and facilitating individualized treatment. Cancer exhibits diverse genetic characteristics among different patients, and even within the same patient, distinct genetic variants and pathway alterations can emerge at different stages of cancer progression.^106,107 Deep learning models can provide personalized diagnostic and therapeutic recommendations by learning from large-scale genetic data of patients. For instance, Yu et al employed a deep learning model to analyze cancer genomic data and proposed a personalized treatment method based on genetic variations, which is expected to have clinical applicability¹⁰⁸ (Figure 1).

Figure 1.

Deep learning in cancer.

Somatic Mutation Data Analysis

Somatic mutations are genetic differences between cancer cells and normal cells, with common types including single nucleotide variants (SNVs) and insertions/deletions (Indels). These variants play a crucial role in cancer development, progression, and metastasis. Therefore, accurate detection of somatic cell variants is essential for early diagnosis and treatment of cancer. In recent years, deep learning (DL) techniques have shown great potential as a powerful tool for detecting somatic cell variants, particularly in the accurate identification of SNVs and Indels from tumor tissue sequencing data. This section will discuss the application of deep learning in somatic cell variant detection, compare the performance of different algorithms, and analyze the challenges and future directions.

Deep Learning in SNV and Indel Detection

Somatic cell variant detection typically relies on high-throughput sequencing (NGS) technology, which identifies cancer-related variants through whole-genome or whole-exome sequencing of tumor samples. While traditional variant detection methods, such as GATK and Samtools, have been widely used, they still face limitations in detecting complex structural variants, low-frequency variants, and controlling false positives. Deep learning methods, on the other hand, can automatically extract features from raw sequencing data and capture complex relationships between genetic markers, significantly enhancing the accuracy and sensitivity of variant detection. Deep learning techniques, particularly Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and their variants (eg, LSTM and GRU), have been successfully applied to variant detection and prediction (Table 3).

Table 3.

Deep Learning in SNV and Indel Detection

Model Type	Appliance	Study
Convolutional Neural Network (CNN)	For automated extraction of local features from genetic data, detection of SNVs and Indels	Improved accuracy and recall of variant detection109
Randomized neural networks (RNNs) and LSTMs	For capturing temporal dependence in gene expression, especially in low-frequency mutation detection110,111	Excellent performance in low-frequency mutation detection
Variable Auto-Encoder (VAE)	Detection of cancer-associated SNVs and Indels by generating patterns of potential variants	Successful identification of multiple mutations associated with cancer111

Predicting Cancer Susceptibility and Progression Based on Somatic Mutations

By identifying cancer-associated somatic mutations, deep learning can predict cancer susceptibility and progression. Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs) have excelled in cancer susceptibility prediction, particularly in extracting meaningful mutation patterns and temporal information from genetic mutation data (Table 4).

Table 4.

Models for Predicting Cancer Susceptibility and Progression Based on Somatic Mutations

Model Type	Appliance	Study
Convolutional Neural Network (CNN)	Extracting mutation patterns from genetic data to predict cancer susceptibility	Improved accuracy of cancer susceptibility prediction109
Long Short-Term Memory Network (LSTM)	Breast and lung cancer susceptibility analysis by time-series modeling	Mutation patterns associated with cancer susceptibility identified110
Ensemble Learning	Combining mutation data and clinical information to predict cancer progression	Significantly improved cancer progression prediction performance112
Graph Neural Networks (GNN)	Modeling gene interactions to reveal mechanisms of cancer progression	Mutations Linked to Cancer Metastasis and Drug Resistance Revealed102

Cancer occurrence and progression is a multifactorial process, and deep learning models can provide comprehensive risk assessments for cancer susceptibility and progression by integrating multidimensional information, such as gene mutation data, epigenetic data, and clinical data. Multimodal learning (ML) is widely used in cancer risk assessment and can effectively integrate information from various data sources to enhance prediction accuracy.

Zhang et al proposed a multimodal deep learning framework that combined gene mutation data with clinical information to successfully predict the susceptibility and progression of multiple cancer types. Experimental results showed that the multimodal learning-based model exhibited high prediction accuracy across multiple datasets.¹¹³ Additionally, transfer learning (TL) has been widely employed in cancer risk assessment. By leveraging models pre-trained on large-scale datasets, transfer learning can effectively improve performance on small sample datasets. For example, Kim et al (2020) achieved high prediction accuracy on a small-sample cancer genome dataset using a transfer learning approach.⁶⁸

Deep learning methods show great potential in predicting cancer susceptibility and progression based on somatic mutation data. By integrating multidimensional mutation information, deep learning can provide more accurate risk assessments for early diagnosis and personalized cancer treatment.¹¹⁴ However, challenges such as high-dimensional data, insufficient samples, and model interpretability remain in the field. With the continuous optimization of algorithms and the expansion of datasets, the application of deep learning in cancer genomics holds great promise, and it is expected to provide more powerful technical support for personalized cancer treatment and precision medicine in the future (Figure 2).

Figure 2.

Application of deep learning model in somatic cell mutation detection.

Cancer Detection Using Imaging Data

CT and X-Ray Imaging

Lung Cancer Detection and Diagnosis

Lung cancer is one of the leading causes of cancer-related deaths worldwide, and early detection is crucial for improving survival rates. With the advancement of computer vision and deep learning (DL) technologies, significant progress has been made in the early diagnosis and screening of lung cancer in recent years. In particular, deep learning methods have shown great potential in the recognition and classification of lung nodules, early lung cancer screening, and CT image processing. However, lung cancer detection still faces many challenges, such as detecting small nodules and controlling the false positive rate. Lung nodules are among the earliest manifestations of lung cancer, and their accurate identification and classification are essential for early diagnosis and treatment. Traditional lung nodule detection methods rely on the experience of doctors and manual labeling; however, this approach suffers from high time costs and a high misdiagnosis rate. In recent years, deep learning methods, particularly Convolutional Neural Networks (CNNs), have made significant progress in lung nodule detection.

CNNs can automatically extract high-dimensional features from CT images and perform nodule identification and classification. The following studies provide insight into related work (Table 5 and Figure 3).

Table 5.

CNN and Multiscale Network Based Lung Cancer Detection Study

Explore	Methodologies	Findings
Zhou et al (2020)	CNN-based lung nodule detection system	Improved detection accuracy, AUC from 0.85 to 0.92115
Xu et al (2019)	Multiscale Convolutional Networks for Lung Nodule Detection of Different Sizes	Improved accuracy in detecting small nodules116
Chen et al (2021)	A lung cancer screening model incorporating multiple deep learning networks	Improved detection of small nodules117
He et al (2020)	Multi-layer convolutional neural network for deep analysis of CT images	Significantly improved sensitivity for early lung cancer screening118

Figure 3.

Performance comparison of deep learning models in lung nodule detection.

Bone Cancer and Other Applications

Deep Learning (DL) techniques are increasingly being used in cancer detection, particularly in the detection of bone cancer and other cancer types, and have shown significant potential. Conventional imaging analysis methods, such as X-rays and CT scans, are effective but still rely on physician expertise and are prone to false positives and false negatives. Deep learning, especially Convolutional Neural Networks (CNNs), has emerged as a key technology to improve the accuracy of imaging detection.

Early diagnosis of bone cancer is crucial, particularly for osteosarcoma and metastatic bone cancer. X-ray images are commonly used for bone cancer screening, but their sensitivity to early cancerous lesions is limited, and they can easily miss these lesions. Deep learning methods, especially CNNs, have been widely applied for the early detection and classification of bone cancer. The following are some related studies (Table 6 and Figure 4).

Table 6.

CNN and Deep Learning Based Cancer Detection Research

Explore	Methodologies	Findings
Liu et al (2020)	CNN-based bone cancer detection system	Ability to accurately recognize malignant nodules in x-ray images119
Yang et al (2021)	Bone Cancer Diagnosis Based on Multiscale Convolutional Networks	Enhanced detection of small bone nodules120
Zhang et al (2020)	CNN-based CT image analysis of liver and pancreatic cancer	Successful detection of liver and pancreatic cancer121
Zhou et al (2021)	Deep learning-based early bowel and pancreatic cancer detection	Improved sensitivity and specificity for bowel and pancreatic cancer122

Figure 4.

Detection accuracy of deep learning models across different cancer types.

Deep learning has not only made significant progress in the detection of bone cancer and abdominal tumors but has also been applied to the detection of common cancers, such as breast and lung cancer, further promoting the widespread adoption of early cancer screening technology. Through automatic feature extraction and pattern recognition, deep learning has greatly enhanced the accuracy and efficiency of cancer detection, providing strong clinical support. It has demonstrated significant advantages in the detection of bone cancer and other cancer types, particularly in the processing and analysis of image data. With the continuous optimization of algorithms and the expansion of datasets, deep learning will play an increasingly important role in early cancer screening and personalized treatment. In the future, deep learning is expected to help clinicians improve diagnostic efficiency, reduce misdiagnosis rates, and enable early detection and treatment.

MRI Imaging

Brain Tumor Detection and Segmentation

Brain tumors are among the most common malignant tumors in the nervous system, and their early detection and accurate segmentation are crucial for clinical diagnosis and treatment. In recent years, Deep Learning (DL) techniques, especially Convolutional Neural Networks (CNNs), have made significant progress in the automatic detection and segmentation of brain tumors. Magnetic Resonance Imaging (MRI), a commonly used imaging tool for brain tumor detection, has become a key area of application for Deep Learning in medical image analysis. Deep learning not only enables automatic tumor segmentation and classification but also improves the accuracy of differential diagnosis of tumor types and other brain diseases through multimodal MRI data fusion.

Automatic segmentation and classification of brain tumors are core tasks for deep learning in medical image analysis. Traditional segmentation methods rely on manual labeling and simple image processing algorithms, which are inefficient and susceptible to physician bias. In contrast, deep learning methods, particularly CNNs, achieve accurate tumor segmentation and classification by automatically learning image features.

U-Net is a deep learning architecture widely used in medical image segmentation. It extracts the spatial features of the tumor by performing multi-layer convolutional operations on the input image and restores the resolution of the image through an inverse convolutional operation. In brain tumor segmentation, U-Net is used for precise localization and segmentation of tumor regions. The following are related studies (Table 7 and Figure 5).

Table 7.

U-Net Based Lung Cancer Segmentation Modeling Study

Explore	Methodologies	Findings
Zhou et al (2020)	A U-Net-based model for automatic brain tumor segmentation	Outperforms conventional methods in tumor region identification and segmentation accuracy123
Zhang et al (2021)	V-Net based multi-scale brain tumor segmentation model	Effective improvement of segmentation accuracy in small tumor regions124
Xu et al (2020)	Deep Learning-based Multi-Classification Brain Tumor Model	Significantly better than traditional medical image analysis methods125

Figure 5.

Accuracy of different brain tumor segmentation models.

Deep learning methods, particularly multimodal learning based on convolutional neural networks (CNNs), can automatically fuse information from multiple MRI images to improve the accuracy of tumor segmentation and classification. Li et al proposed a multimodal CNN model capable of fusing MRI images from different modalities, significantly enhancing both the detection and segmentation accuracy of brain tumors.¹²⁶ Additionally, Zhou et al employed a deep learning multi-task learning framework, combined with T1 and T2-weighted images, to successfully improve brain tumor segmentation and enhance the ability to distinguish brain tumors from other brain lesions.¹²⁷

Prostate and Breast Cancer Applications

Deep Learning (DL) techniques have shown great potential for application in MRI detection of prostate and breast cancers. MRI, a commonly used imaging technique for prostate and breast cancers, provides important information for the early diagnosis and treatment of these cancers. In recent years, significant progress has been made in applying deep learning methods to these fields, particularly in the localization and grading of prostate cancer, the early detection of breast cancer, and the use of dynamic contrast-enhanced MRI (DCE-MRI) data to improve detection accuracy.

Prostate cancer is a common malignant tumor in men, and early diagnosis and accurate grading are crucial for effective treatment. MRI plays a key role in diagnosing prostate cancer, especially in tumor localization and grading. Traditional MRI analysis relies on physicians’ experience and manual annotation, which is inefficient and prone to human error. Deep learning, particularly Convolutional Neural Networks (CNNs), can more accurately identify and localize prostate cancer lesions by automating the analysis of MRI images. CNNs are widely used in prostate cancer detection. The following are related studies (Table 8).

Table 8.

CNN and Deep Learning Based Cancer Detection Modeling Study

explore	methodologies	Findings
Zhao et al (2020)	CNN-based model for automatic prostate cancer detection	High accuracy and sensitivity to successfully identify prostate cancer lesions128
Wang et al (2021)	Deep learning-based grading model for prostate cancer	Successfully differentiating between high and low risk patients129

Breast cancer is one of the most common cancers in women worldwide, and its early diagnosis is crucial for improving survival rates. MRI, an important tool for breast cancer screening, provides effective diagnostic information, especially when other imaging methods (eg, mammography) fail to detect tumors. Deep learning methods, particularly CNN-based models, have been successfully applied to the early detection of breast cancer, achieving remarkable results. These methods can automatically extract features from breast MRI images and accurately identify the morphology, size, and location of tumors. Zhu et al proposed a deep convolutional neural network-based breast cancer detection method, which successfully enabled early screening by analyzing MRI images.¹³⁰ Compared to traditional methods, the model significantly improved the sensitivity of detecting small masses, facilitating early breast cancer detection.

Dynamic Contrast-Enhanced MRI (DCE-MRI) provides information about tumor blood flow and vascular permeability, making it an important tool for diagnosing prostate and breast cancer. With contrast-enhanced MRI images, physicians can obtain more detailed information about the tumor, which aids in precise localization and grading. Deep learning methods can further enhance the accuracy of DCE-MRI in cancer detection by integrating multimodal data. Li et al proposed a deep learning-based multimodal data fusion method that combines DCE-MRI with conventional T1-weighted and T2-weighted MRI images, improving the accuracy of breast and prostate cancer detection.¹³¹ Additionally, Liu et al introduced a new breast cancer detection method by combining DCE-MRI data with a deep convolutional neural network, significantly improving the sensitivity and specificity of tumor detection.¹³²

Pathological Image Analysis

Pathology image classification plays a crucial role in cancer diagnosis, with the primary goal of identifying cancer cell types and determining tumor grading and staging by analyzing tissue section images. This information provides a foundation for precise treatment decisions. In recent years, deep learning techniques, particularly Convolutional Neural Networks (CNNs), have been widely applied in pathology image analysis due to their exceptional performance in automatic feature extraction and image classification. This section summarizes the current status of deep learning applications in pathology image classification, the challenges it faces, and potential solutions.

Histopathological Image Classification and Diagnosis

Deep learning models have shown excellent performance in cancer cell type recognition. Here are some related studies (Table 9 and Figure 6).

Table 9.

CNN and Deep Learning Based Cancer Cell Recognition and Classification Study

Explore	Methodologies	Findings
CNN-based cell recognition in breast cancer pathology sections	Automatic recognition of tumor cells from normal cells133	Excellent performance in cell type identification in breast cancer
Multi-task learning approach in pathology image analysis	Simultaneous prediction of tumor type and regional boundaries to improve classification accuracy and efficiency134	Improved classification accuracy and efficiency through multi-task learning
Deep learning based tumor grading model	Capture microscopic level of detail features such as nucleus size, shape and density distribution16	Demonstrated accuracy comparable to or better than pathologists in grading and staging of cancers such as breast and prostate135,136
Migration Learning and Pretraining Models for Pathology Images	Improving Model Performance by Migrating Pre-trained Models on the ImageNet Dataset137	Significantly reduce data requirements and improve model performance

Figure 6.

Accuracy of different pathology image classification methods.

Digital Pathology and Computational Pathology

The combination of Digital Pathology (DP) and Computational Pathology (CP) has become a significant research focus in modern medicine, offering new tools for cancer diagnosis and treatment decisions. With the rapid advancement of Whole Slide Image (WSI) technology and deep learning, research in this field has greatly enhanced the efficiency and accuracy of pathology. This section summarizes the key technologies, current applications, and future directions for integrating digital pathology with deep learning.

Whole Slide Images are one of the core technologies in digital pathology, enabling the display of complete tissue sections at ultra-high resolution. However, the high resolution and large data volume of WSIs present challenges for automated analysis. Deep learning provides an effective solution for automating WSI analysis through its powerful feature extraction capabilities. Deep learning models have demonstrated strong performance in tumor detection and segmentation. For example, CNNs are widely used to identify cancerous regions in WSIs, assisting pathologists in efficiently screening cancer cases.¹³⁸ The following are related studies (Table 10 and Figure 7).

Table 10.

Study of Cancer Region Localization and Classification Based on U-Net and Deep Learning

Explore	Methodologies	Findings
He et al (U-Net based segmentation model)	Precise localization of cancer cell areas in breast cancer sections139,140	Segmentation performance is superior to traditional methods
Chen et al (Modeling Attention Mechanisms for Multi-Instance Learning)	Effective localization of key lesion areas in WSI141	Improved accurate localization of lesion areas

Figure 7.

Application of deep learning models in digital pathology.

Pathology reports are critical for cancer diagnosis, but traditional report generation relies on the pathologist’s experience, which is subjective and time-consuming. Deep learning-based automatic pathology report generation techniques facilitate efficient conversion from image to text. These techniques typically combine image feature extraction with Natural Language Processing (NLP) methods in deep learning. The following are related studies (Table 11 and Figure 8).

Table 11.

Transformer and BERT Based Cancer Detection and Diagnosis Modeling Study

Examine	Methodologies	Findings
Li et al (2021)	Transformer-based multimodal model to extract key features from WSI and generate structured pathology reports142	Ability to improve the accuracy of pathology report generation to aid in clinical diagnosis
Li et al (2021)	Transformer-based multimodal model to extract key features from WSI and generate structured pathology reports142	Ability to improve the accuracy of pathology report generation to aid in clinical diagnosis
Li et al (2021)	Combining the BERT model for semantic understanding and standardization, extracting terminology and generating normalized reports143	Improved accuracy and professionalism of report generation
Kather et al (2020)	Combining WSI with Patient Clinical Data to Predict Cancer Recurrence Probability and Treatment Effectiveness Using Deep Learning Models144,145	Effectively improves the accuracy of cancer prognosis prediction
Zhu et al (2021)	Proposing a weakly supervised learning method to analyze WSI with high quality with only a small amount of labeled data146	Reduced annotation data requirements and improved WSI analysis quality

Figure 8.

Application of WSI combined with genomic data.

Ultrasound Imaging

Liver and Thyroid Cancer Detection

Ultrasound imaging is widely used for the early screening and diagnosis of hepatocellular carcinoma and thyroid cancer due to its noninvasive, real-time, and cost-effective nature. However, ultrasound image quality is often compromised by noise, artifacts, and resolution limitations, which present challenges for tumor localization, size measurement, and benign/malignant classification. In recent years, the rapid development of deep learning techniques has provided new approaches to address these issues. The following are some related studies (Table 12 and Figure 9):

Table 12.

U-Net and Deep Learning Based Liver Cancer Detection and Segmentation Study

Examine	Methodologies	Findings
Xu et al (2020)	Multiscale U-Net model for tumor segmentation of liver ultrasound images147	Effective Improvement of Liver Tumor Segmentation Performance, Especially for Low Contrast Problems
Li et al (2021)	DenseNet-based nodule detection framework for high-precision nodule size measurement148	Improved accuracy of nodule size measurement and morphologic analysis
Wang et al (2020)	Combining CNN and Attention Mechanisms for Classifying Liver Cancer Benign and Malignant149	Classification accuracy of more than 90%, effectively differentiating benign and malignant liver cancer
Zhang et al (2020)	A multimodal deep learning framework fuses imaging and non-imaging data to improve benign and malignant classification accuracy of thyroid cancer150	The AUC value of 0.95 significantly improved the accuracy of classification of benign and malignant thyroid cancer

Figure 9.

U-Net and Deep Learning Based Liver Cancer Detection and Segmentation Study.

Noise and artifacts in ultrasound images significantly impact model performance. To minimize the interference of noise on detection results, researchers have employed image preprocessing techniques such as filtering, contrast enhancement, and Adversarial Generative Networks (GANs) to generate high-quality images.^151,152

The limited amount of data is another challenge in ultrasound image analysis. Transfer learning and data augmentation techniques help mitigate this issue to some extent. For example, Chen et al significantly improved the generalization ability of their model by scaling up a small sample dataset using a transfer learning approach.¹⁵³

Obstetric and Gynecological Cancer Applications

Obstetrics and gynecology cancers (eg, cervical cancer, ovarian cancer) are among the major diseases threatening women’s health, and ultrasound plays a crucial role in early screening due to its noninvasive, real-time, and cost-effective nature. In recent years, the introduction of deep learning techniques has significantly improved the diagnostic accuracy of ultrasound images for cancer screening in obstetrics and gynecology. This section reviews the application of deep learning in ultrasound screening and discusses how detection performance has been enhanced by combining morphological and hemodynamic information.

Ultrasound screening for cervical cancer typically relies on identifying structural abnormalities in the cervical region. Deep learning-based automated diagnostic models enable efficient detection of early lesions by extracting morphological features from cervical ultrasound images. The following is a related study (Table 13 and Figure 10).

Table 13.

Deep Learning-Based Early Cancer Detection and Classification Study

Explore	Methodologies	Findings
Zhang et al (2020)	CNN-based automatic cervical cancer early lesion identification model154	Classification accuracy of more than 92%, successful identification of early cervical cancer lesions
Liu et al (2020)	LSTM-based model for dynamic ultrasound sequence analysis155	Significantly improved early screening for cervical cancer
Wang et al (2021)	A multimodal deep learning framework combining 3D ultrasound images and blood flow signals156	AUC value of 0.95 improves the performance of benign-malignant classification of ovarian cancer
Zhu et al (2020)	Accurate Classification of Ovarian Cancer Based on Multiple Instance Learning (MIL)157	Accurate classification of ovarian cancer images successfully achieved

Figure 10.

Application of deep learning models in cancer screening in obstetrics and gynecology.

The combination of morphological and hemodynamic features is crucial for ultrasound screening of obstetric and gynecological cancers. Deep learning techniques have significantly improved diagnostic performance by jointly analyzing these two types of features. For example, Chen et al developed an Attention Mechanism (AM) model that focuses on both the structural features of tumors and blood flow patterns, achieving a sensitivity of 94% in cervical cancer detection.¹⁵⁸ To enhance the model’s ability to capture complex blood flow features, Generative Adversarial Networks (GANs) were used to generate high-quality blood flow images, improving the model’s training effectiveness and classification performance.¹⁵⁹ Additionally, a Transformer-based multimodal learning framework was employed to fuse morphological and blood flow information, offering new approaches to further improve diagnostic accuracy.¹⁶⁰

Multi-Modal Data Integration

Multimodal data fusion is a key technology in cancer detection. Genetic data provide molecular-level information about cancer, while medical images offer insights into the shape, size, and spatial distribution of tumors. A single data source often cannot fully capture the complexity of cancer, so combining multimodal data (eg, genomic and medical image data) can enhance the precision and accuracy of cancer detection. This section explores how to integrate genetic and image data, introduces the architectural design and training strategies of deep learning models for multimodal data fusion, and discusses the challenges related to data compatibility and model complexity.

Combining Genomic and Imaging Data

Genetic and image data reveal cancer characteristics from different dimensions. Genomic information captures the molecular features of a tumor, such as mutations and gene expression, while image data provide insights into the shape, size, and spatial distribution of the tumor. Unimodal data often fail to fully capture the diversity and heterogeneity of tumors, which can lead to biased diagnoses. For example, genetic data cannot provide information on the spatial distribution of tumors, while imaging data may lack sensitivity to molecular-level abnormalities.¹⁶¹ Therefore, multimodal data fusion, combining genetic and imaging data, has become a crucial tool for enhancing cancer detection and diagnosis. By integrating these data types, the limitations of individual data sources can be mitigated, providing more comprehensive diagnostic information. For instance, in breast cancer detection, integrating genomic data with mammography images has been shown to significantly improve both the sensitivity and specificity of early detection.¹⁶² Furthermore, multimodal data fusion can aid in the development of personalized treatment strategies, supporting treatment plan selection by comprehensively analyzing both genomic characteristics and imaging features¹⁶³(Table 14 and Figure 11).

Table 14.

Common Multimodal Data Fusion Methods and Their Specific Applications

Integration Methods	Descriptive	Sample Application
Multi-input model	The gene data and image data are fed into two separate neural network branches for feature extraction, and finally feature fusion is performed through a fully connected layer.	Huang et al proposed a dual-stream CNN-based framework that ultimately achieves feature fusion through a fully connected layer to improve performance in multimodal cancer classification tasks.164
Joint Encoder	Multi-modal data is processed by sharing weights, and deep fusion of data is realized in the process of feature extraction.	Zhang et al proposed a Transformer architecture that is able to learn complex relationships between data through a multi-head attention mechanism.165
Graph neural network	Capturing higher-order relationships between gene data and image data, the fusion of features is realized by graph convolution operation.	The graph embedding-based model developed by Li et al achieves feature fusion through graph convolution operations with an AUC value of 0.92166 in the cancer subtype prediction task.
Migration learning with pre-trained models	Utilizing pre-trained CNNs on ImageNet for image feature extraction and incorporating specific models for genomic data significantly improves training efficiency.	The combination of migration learning and pre-trained models significantly improves the training efficiency of multimodal data fusion.167
Joint Optimization Strategy	Simultaneous training of feature extraction networks and classification networks for multimodal data to enhance synergy between data through adversarial loss.	The adversarial training method proposed by Wang et al improves the F1 value in the breast cancer detection task by 8% through adversarial loss enhancement.168
Self-supervised learning	Improve the performance of multimodal models by generating pre-training tasks such as data reconstruction and feature prediction.	Chen et al used a comparative learning approach to construct positive-negative sample pairs using unlabeled gene data and image data, thus improving the robustness of feature extraction.169

Figure 11.

Proportion of different multimodal data fusion methods.

Clinical Data Integration and Decision Support Systems

Integrating deep learning with clinical data (eg, patient history, treatment records) provides crucial support for cancer diagnosis and treatment. By combining multimodal data and developing intelligent Decision Support Systems (DSS), the efficiency of diagnosis, the development of personalized treatment plans, and the interpretability of clinical decisions can be enhanced. This section reviews methods for combining deep learning with clinical data, discusses the current status and prospects of deep learning-based decision support systems for cancer diagnosis, and highlights the importance of clinical utility and interpretability.

Patient history and treatment records contain rich time-series data and structured information, which are valuable sources for predicting disease progression and treatment response. Deep learning-based natural language processing (NLP) techniques, such as BERT and GPT models, are employed to automatically extract key information from Electronic Health Records (EHR). The fusion of clinical data with imaging and genomic data is a key approach to improving diagnostic accuracy (Table 15).

Table 15.

Multimodal Deep Learning for Convergent Applications in Cancer Diagnosis

Fusion Data Types	Application Cases	Effect
EHR + CT images + genomic data	Integrating EHR, CT images and genomic data into a multimodal deep learning framework for improved detection sensitivity in lung cancer detection170	Improved sensitivity and accuracy
EHR + Genetic Data	BERT-based modeling to extract risk factors and treatment response in cancer patients171	Outperforms traditional machine learning methods
EHR + Image Data	The DSS system developed by Lee et al combines CT images and clinical data to predict the pathological stage of lung cancer172	Provide a visual explanation of staging

DSS systems are capable of integrating multimodal data from patients to provide decision support for personalized treatment. For instance, Kim et al developed a deep learning-based DSS to recommend the most effective targeted drugs for lung cancer patients by analyzing their genetic mutations and treatment history¹⁷³ (Table 16).

Table 16.

Deep Learning DSS Systems in Personalized Therapy

Application Areas	Method Description	Major Application	Findings
Personalized Medication Recommendations	Deep learning-based DSS to analyze patient mutations and treatment history	Recommendation of the best targeted drug	Improving the Effectiveness of Targeted Drug Therapy173
Early screening and risk prediction	Cancer risk prediction by combining patients’ age, lifestyle habits and imaging data	Risk prediction for breast cancer	Helping Physicians Develop More Rational Screening Programs174

In clinical applications, the interpretability of deep learning models is crucial for enhancing physicians’ trust. For example, through heatmaps or attention mechanisms, physicians can understand the basis of model predictions, thereby increasing the transparency of decision-making.¹⁷⁵

The heterogeneity of clinical data and imbalance in sample size are major challenges in DSS design. The following are relevant solutions (Table 17 and Figure 12):

Table 17.

Migration Learning Based Cancer Diagnosis and Dataset Enhancement Study

Concern	Prescription	(for) Instance	Effect
Heterogeneity of clinical data	Transfer learning	Improving Diagnostic Performance Using Migration Learning on Scarce Datasets176	Improved cancer diagnosis performance on scarce datasets
Uneven sample size	Small Sample Learning Techniques	A small sample learning framework based on contrast learning176	Improved model performance on small sample datasets
Real-time decision support	Edge computing and cloud computing based DSS system	Supporting Real-Time Clinical Decision Making177	Improved efficiency and accuracy of real-time decision making

Figure 12.

Schematic representation of the interpretability of the deep learning DSS system.

Decision Support Systems (DSS) for cancer diagnosis that combine deep learning with clinical data have great potential to improve diagnostic efficiency, support personalized treatment, and enhance transparency in clinical decision-making. Future research should focus on data sharing and privacy protection, model interpretability, and real-time performance to facilitate the widespread adoption of DSS in clinical practice.

Multi-Input Multi-Output Neural Networks, Transformer, and Other Models for Integrating Genetic and Imaging Data

Multi-Input Multi-Output Neural Networks (MIMO) and Transformer models have been widely used in cancer detection research that integrates genetic and medical imaging data in recent years, owing to their advantages in processing multimodal data. This section explores the specific applications, methodological innovations, and challenges faced by these models in fusing genetic and imaging data.

Multi-input multi-output neural networks process different modalities of data (eg, genetic data and imaging data) separately by designing independent branching networks, and subsequently achieve data integration through a feature fusion layer. The formula is as follows:

or

(20)

For example, in cancer diagnosis, image data branching networks are typically based on Convolutional Neural Networks (CNNs), while genetic data branching networks are typically based on Fully Connected Networks (FCNs) or autoencoders¹⁷⁸ (Table 18).

Table 18.

Deep Learning Based Breast and Lung Cancer Detection Study

Application Areas	Method Description	Performance Enhancement	Findings
Breast Cancer Diagnosis	Integration of breast cancer gene expression data with breast MRI imaging features	AUC increased from 0.85 to 0.92 in unimodal mode	A two-branch multi-input network designed by Li et a[179]
Lung Cancer Detection	Combining CT Imaging and Gene Mutation Data for Cancer Staging Prediction	Superior to traditional methods	Zhang et al. Multi-input neural network applications[180]

Feature fusion is a key aspect of multi-input neural networks. Common strategies include simple feature concatenation, weighted fusion, and fusion methods based on attention mechanisms. For instance, feature fusion based on the attention mechanism can dynamically assign modal weights to emphasize key features.¹⁸¹

The Transformer model is widely used in multimodal data analysis due to its global attention mechanism, which captures long-range dependencies. Compared to traditional neural networks, the Transformer exhibits stronger modeling capabilities when integrating high-dimensional genetic data with imaging data¹⁸² (Table 19 and Figure 13).

Table 19.

Transformer-Based Cancer Image Analysis and Diagnosis Study

Application Areas	Method Description	Performance Enhancement	Findings
Liver Cancer Detection	Transformer-based model integrates gene expression data with CT image features	The AUC reaches 0.94, which is significantly higher than the traditional CNN model	Transformer-based multimodal modeling proposed by Xu et al183
Prostate Cancer Staging	Joint learning of genetic data and pathology image features for modeling via Transformer	Achieve 92% accuracy	Multimodal Transformer model developed by Chen et al184

Figure 13.

Performance comparison between MIMO and Transformer models.

Challenges and Future Directions

Data Quality and Quantity Issues

The use of genetic and image data in cancer detection is gaining increasing attention. However, challenges related to data quality (eg, sequencing errors and image artifacts) and data volume (eg, small sample sizes and data imbalance) significantly impact the performance of deep learning models. This section addresses these issues and discusses strategies for tackling small-sample learning, data imbalance, and the application of Explainable AI (XAI) techniques (Tables 20–22 and Figure 14).

Table 20.

Data Quality Issues

Concern	Descriptive	Solution Strategy
Sequencing errors in genetic data	Sequencing errors mainly include base substitution errors, insertion/deletion errors, etc, which may interfere with mutation detection and gene expression analysis.185	Deep learning-based error repair algorithm for detecting and repairing sequencing errors using deep generative models.186
Artifacts and Noise in Image Data	Artifacts (eg, motion artifacts, metal artifacts) and noise degrade the model’s accuracy in detecting tumor boundaries187	Using GANs to remove artifacts and improve image quality188

Table 21.

Data Volume Issues

Concern	Descriptive	Solution Strategy
Small sample size (statistics)	Small sample data may lead to overfitting of the deep learning model, which reduces the generalization ability.189	Migration learning: by migrating knowledge from large publicly available datasets such as ImageNet.190 Data enhancement: techniques such as rotation, flipping and cropping.191 Small sample learning: meta-learning based methods that enable rapid adaptation to new tasks with limited samples.192
Data imbalance issues	Data imbalance is another challenge with cancer testing data, especially when there are significantly fewer samples in minority classes (eg, malignant tumors) than in majority classes (eg, benign tumors), and the model is susceptible to bias in favor of the majority class.193	Resampling methods: such as oversampling (SMOTE) and undersampling to balance the data distribution.194 Adjusting the loss function: using techniques such as weighted cross-entropy or Focal Loss.195 Generating Adversarial Networks: expanding the minority class sample set by generating synthetic data.196

Table 22.

Application of Explanatory Techniques (XAI)

Concern	Descriptive	Solution Strategy
Importance of XAI	The introduction of interpretability techniques can help researchers understand the decision basis of deep learning models.197	Enhance model transparency with SHAP and Grad-CAM technologies.
XAI in genetic data	SHAP approach to quantify the contribution of mutant loci to cancer risk prediction.197	SHAP-based interpretations help develop individualized treatment plans.
XAI in Medical Imaging	Grad-CAM highlights areas of interest in medical images through heat maps.198	Grad-CAM shows tumor areas in CT images of lung cancer.
Application of XAI to multimodal data integration	Chen et al visualize the weight distributions of different modal features through the attention mechanism.199	Use XAI to show the contribution of different modal features to the final prediction results.

Figure 14.

Impact of data quality and data volume on model performance.

Interpretability of Deep Learning Models

Deep learning models demonstrate excellent performance in cancer detection, but their “black-box” nature makes the decision-making process difficult to interpret, limiting their widespread application in clinical practice. Understanding the decision-making process of these models, identifying key features, and developing interpretable methods are crucial to enhancing model credibility and facilitating clinical use. This section explores the interpretability of deep learning models and reviews relevant research progress.

Deep learning models typically have highly nonlinear and complex structures, making their decision-making processes difficult to understand intuitively. To increase model transparency, researchers have proposed various techniques. For example, Grad-CAM (Gradient-weighted Class Activation Mapping) explains a model’s classification decisions by generating a heatmap that highlights the regions the model focuses on in an image.²⁰⁰ In breast and lung cancer detection, Grad-CAM is widely used to analyze key regions of model decisions and assist physicians in validating the model’s rationale²⁰¹(Tables 23–26).

Table 23.

Identification of Key Features

Examine	Methodologies	Findings
Kim et al (2020)	Identification of key lesion regions in CT images of lung cancer based on attention mechanism	Significantly improves the detection accuracy of the model202
Application of SHAP to genetic data	SHAP-based analysis of gene mutation patterns	Mutation Patterns Significantly Associated with Breast Cancer Development203

Table 24.

Development of Interpretable Methods

Methodologies	Descriptive
Visualization technology (LIME)	Interpreting model predictions on specific samples by local linear approximation204
Attention mechanism	Weighting the importance of features to visualize the model’s region of interest182,205
Comparative learning	Explaining Model Decisions by Comparing Characteristic Differences between Positive and Negative Samples206
Methods of inferring cause and effect	Analyzing the causal relationship between model outputs and input variables through causal diagrams207

Table 25.

The Significance of Interpretability for Clinical Applications

Examine	Methodologies	Findings
Importance analysis of features generated by SHAP	For breast cancer risk prediction and to guide clinical decision making	Enhanced credibility in clinical decision making208
Identification of Attentional Mechanisms Responsive to Targeted Therapies	Identifying potential patient response characteristics to targeted therapies	Optimized individualized treatment plans209
Heat map to aid in data labeling	Used to indicate lesion areas and speed up the labeling process	Improved data labeling efficiency210

Table 26.

Challenges and Future Directions

Challenge	Future Direction
Standardized evaluation indicators	Development of a Common Criteria for Quantitative Interpretability211
Interpretability studies of multimodal data	Development of multimodal interpretable methods for fusion analysis of imaging and genetic data212
Real-time Interpretation and Usability Enhancement	Combining Edge Computing to Enhance Real-Time Clinical Use213

Ethical and Regulatory Considerations

The application of deep learning in cancer detection has revolutionized medical image analysis and disease prediction. However, the widespread adoption of this technology has raised several ethical issues and regulatory challenges, including data privacy protection, algorithmic bias, and model validation and approval criteria. This paper explores these key issues and proposes potential solutions to ensure the responsible and safe application of the technology (Tables 27–31 and Figure 15).

Table 27.

Data Privacy Protection

Concern	Methodologies	Study
Data Sharing and Privacy Breaches	Federated Learning (FL) avoids centralized storage through distributed training214,215	Reduced risk of privacy breaches
Data de-identification techniques	De-identification techniques such as Differential Privacy216,217	Maintain high data utilization and ensure privacy

Table 28.

Algorithmic Bias

Sources and Effects of Prejudice	Cure	Study
Imbalance or insufficiency of training data218	Data Enhancement, Stratified Sampling, Adjustment of Loss Function	Reduced bias in diagnosis see219
Algorithmic Bias Triggers Widening Health Disparities220	A multi-task learning framework incorporating fairness constraints	Significantly improved performance equalization across races219

Table 29.

Model Validation and Approval Criteria

Validation Difficulties	Regulatory Standards	Study
Lack of harmonized assessment criteria221	FDA and MDR require evidence of reproducibility of model performance222,223	Improved model reliability222
Complexity of the model validation process221	Strict approval processes are required to ensure security and effectiveness	Ensures the safety of the model for clinical applications223

Table 30.

Safeguards for Rational and Safe Application of Technology

Safeguard	Methodologies
Transparency and interpretability	Enhancing Transparency through Visualization Techniques (eg Grad-CAM) and Causal Inference224
Multi-party collaboration and standardization	Promoting open data-sharing platforms and harmonized assessment benchmarks225
Continuous monitoring and updating	Dynamic Learning and Online Training Technologies226

Table 31.

Future Directions

Directional	Goal
Construction of an ethical framework	Ensuring Equal Benefits for All Patient Groups in Model Use227
Internationalized regulatory cooperation	Harmonization of model approval criteria and promotion of cross-border applications228

Figure 15.

Programs for data privacy protection.

Future Research Directions and Potential Breakthroughs

Deep learning has shown great potential in cancer detection, but many scientific questions and technical challenges remain to be addressed. Future advancements in the field are expected through the development of more advanced deep learning architectures, the integration of multi-omics data, the enabling of personalized cancer detection, and breakthroughs in early cancer screening and liquid biopsy techniques. This section explores future research directions and potential breakthroughs (Table 32 and Figure 16).

Table 32.

Developing More Advanced Deep Learning Architectures

Examine	Methodologies	Study
Wang et al (2020)	Graph Embedding Based Cancer Detection Model Combining GNN and Attention Mechanisms	Significantly improved cancer detection model performance229,230
Zhang et al (2020)	Meta-learning based framework for breast cancer detection	New ideas for scarce datasets231

Figure 16.

Research results of deep learning architecture.

Integration of Multi-Omics Data

The occurrence and progression of cancer involve genomic, transcriptomic, proteomic, and other multi-omics data. Integrating these data with imaging data provides a more comprehensive understanding of the biological mechanisms of cancer (Tables 33–36).

Table 33.

Deep Learning-Based Cancer Imaging and Genetic Data Analysis Study

Explore	Methodologies	Study
Kim et al (2021)	A multimodal framework combining gene expression profiling and MRI imaging=	Excels in Ovarian Cancer Detection232
Li et al (2021)	Deep learning models based on single-cell RNA sequencing data	Successfully used to predict tumor metastasis233

Table 34.

Personalized Cancer Testing

Explore	Methodologies	Study
Zhang et al (2020)	Individualized diagnostic advice based on attention mechanisms	Individualized treatment recommendations were provided234,235
Chen et al (2021)	AI model-based prediction of people at high risk of breast cancer	Model sensitivity of 94%236

Table 35.

Early Cancer Screening and Liquid Biopsy

Explore	Methodologies	Study
He et al (2021)	A ctDNA methylation analysis-based framework for lung cancer detection	Demonstrates Excellence in Early Lung Cancer Detection237
Chen et al (2020)	AI-based modeling for screening people at high risk of breast cancer	Model has 94% sensitivity in screening236

Table 36.

Future Directions

Orientations	Goal
Multi-centric data sharing and standardization	Improving the generalizability and credibility of models238
Interpretability and Ethical Considerations	Enhancing Model Transparency and Clinical Trust239
Edge Computing and Real-Time	Implementing Deep Learning Models in Real Time240

Conclusion

Deep learning demonstrates significant potential for cancer detection using genetic and imaging data.²⁴¹ Through efficient feature extraction and multimodal data fusion, deep learning models play a crucial role in enhancing cancer detection accuracy, supporting early diagnosis, and informing personalized treatment decisions.²⁴² However, the application of current technology still faces numerous challenges.²⁴³ The current status of its application, major issues, and future directions are summarized below (Tables 37–39 and Figure 17).

Table 38.

Challenges

Challenge	Solution Strategy	Study
Insufficient data quality and quantity	Enhanced data collection and data enhancement	Alleviates the problem of scarce data248
Model Complexity and Computational Resource Requirements	Development of lightweight model architectures and optimization algorithms	Improved efficiency of model calculations249
Insufficient interpretability	Introduction of interpretable technologies (eg Grad-CAM and SHAP)	Enhanced model credibility250
Privacy and Ethical Issues	Using de-identification techniques and federated learning	Improved data privacy protection251

Table 37.

Application Status

Application Areas	Methodologies	Study
Precise feature extraction	Convolutional Neural Networks (CNN) and Graph Neural Networks (GNN) for Genetic Data and Medical Image Analysis	Improved accuracy of cancer detection244
Multimodal data fusion	Integrating imaging data with genetic data to improve cancer diagnosis accuracy	Significantly Improved Accuracy of Cancer Diagnosis245
Support for individualized treatment	Analyzing the relationship between genetic mutations and treatment response	Developing Individualized Treatment Strategies246
Promoting early diagnosis	Deep learning model based on liquid biopsy data and imaging data	Significantly reduced misdiagnosis and underdiagnosis247

Table 39.

Future Research Directions

Directional	Goal
Development of lightweight models	Reducing dependence on computing resources252
Data Enhancement and Small Sample Learning	Extending the training dataset to alleviate data deficiencies253
Deep fusion of multimodal data	Improving Data Fusion Efficiency and Increasing Detection Accuracy254
Enhancing the interpretability of models	Enhancing the transparency and credibility of models255
Promoting data sharing and standardization	Enhancing model training and extending applicability256

Figure 17.

Directions for future research.

Deep learning techniques have revolutionized cancer detection by integrating genomic data and medical images, showing significant potential, particularly in enhancing detection accuracy and supporting personalized treatment. However, challenges such as data quality, model complexity, and interpretability remain to be addressed. Future research should focus on developing efficient, transparent, and lightweight models to facilitate the deep integration and clinical application of multimodal data, providing robust technical support to achieve the goal of precision medicine.

Funding Statement

No funding was received for this study.

Use of Artificial Intelligence Tools

During the preparation of this paper, no artificial intelligence tools were used.

Data Sharing Statement

The data of this study are sourced from relevant papers on PubMed and Google Scholar. The data involved in this paper can all be obtained from the corresponding papers on the above platforms. Since the data were not directly generated by the authors, no additional access method is required.

Ethical Approval and Consent to Participate

This study does not involve human or animal experiments, so no ethical approval or consent to participate is required.

Patient Consent for Publication

This study does not contain patient-related data, so no patient consent for publication is required.

Author Contributions

As the corresponding author and the first author, Wang Xinyu was responsible for the overall conception, design, writing and revision of the study, and was accountable for the integrity and accuracy of the research. Su Can, as the second author, assisted in literature research, data collation and other work, providing important support for the paper. All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

All authors declare that they have no competing interests.