The application of artificial intelligence and radiomics in lung cancer

Abstract

Lung cancer is one of the most leading causes of death throughout the world, and there is an urgent requirement for the precision medical management of it. Artificial intelligence (AI) consisting of numerous advanced techniques has been widely applied in the field of medical care. Meanwhile, radiomics based on traditional machine learning also does a great job in mining information through medical images. With the integration of AI and radiomics, great progress has been made in the early diagnosis, specific characterization, and prognosis of lung cancer, which has aroused attention all over the world. In this study, we give a brief review of the current application of AI and radiomics for precision medical management in lung cancer.

Background

Lung cancer is one of the most common types of cancer in the world.^1,2 However, most patients with lung cancer do not feel the existence of a tumor until severe clinical symptoms appear, which in part leads to poor clinical outcomes. As a result, early detection in the high-risk population for pulmonary cancer is important. Meanwhile, different types of therapy have brought challenges in selecting eligible patients for proper treatment strategies. A great deal of research has been conducted in solving the practical problems in lung cancer management.

According to universal clinical guidelines, computed tomography (CT) is highly recommended in lung cancer screening, which could help in various steps such as patients’ follow-up, early detection, and prognosis.^3,4 The frequent usage of imaging machines produces large amounts of imaging data. The traditional way to illustrate these raw data largely depends on human intervention. Visible features such as tumor density, the composition of tumor (including the presence of blood vessel, the existence of necrosis, and mineralization), regularity of tumor margins, anatomic relationship to the surrounding tissues (such as pleura and mediastinum), and whether these structures are being invaded⁵ are considered priorities for evaluating the property of the tumor and then influence the clinical decision. However, this conventional time-consuming practice of reading images is not enough in modern medical systems because it not only includes inter- and intra-observer variabilities, but also leaves lots of hidden information undiscovered. Due to the limitations mentioned before, artificial intelligence (AI) and radiomics have been trusted by scientists to further dig out the potentially valuable data under all these images.⁶⁻¹⁰ Figure 1 shows the main workflow of AI and radiomics for lung cancer.

Figure 1.

The workflow of AI and radiomics in lung cancer.

AI is considered to be a system's ability to extract information through outside data and arrive at solutions for specific goals based on learning.¹¹ Machine learning is a way to achieve AI. The basic theory of machine learning is that a computer generates a mathematical model by learning lots of training data to make decisions and predictions in the real world.¹² Deep learning is a branch of machine learning that is formed based on an artificial neural network with many levels of algorithms, each level providing a different interpretation of the data it conveys.^13,14

Deep learning

Deep learning is fed with raw images and extracts higher-level features through multiple layers progressively. Humans are freed from the time-consuming work through transferring traditional human-aided steps, such as feature extraction and selection, to the neural network.

Deep neural network

A deep neural network (DNN) with more than one hidden layer mimics the way that the human brain works. One layer means a collection of several neurons and each layer has its own activation function. The associated neuron in the brain is a mathematical operator that applies a specific function on input.¹⁵ A DNN has one input layer and one output layer as usual, and the design of hidden layers changes depending on the behavior the designers want.

When medical images are fed to a DNN, the question may arise as to whether these images show the existence of lung cancer. Through the process of multiple layers, the primary question can be broken down into whether these images show the same shapes in a specific direction and, finally, questions are simple and answerable at the level of a single pixel.

For better understanding, the introduction of a convolutional neural network (CNN), one of the most used variants of deep learning architectures is presented as follows.

Convolutional neural network

As for the typical structure of the DNN, the CNN also has input and output layers, and the hidden layer of CNN typically consists of convolutional layers, pooling layers, fully connected layers, and normalization layers. The localized features are filtered by the learnable kernels of convolutional layers. The pooling layers are employed to efficiently reduce the data dimensions and hence control overfitting for models. Finally, neurons of fully connected layers connect to all activation neurons of the former layer for integration and high-level reasoning. Mostly, deep CNN is recommended in image processing.

Since 2010, a competition called "The ImageNet Large Scale Visual Recognition Challenge (ILSVRC)" has been held annually, which aims to discover better algorithms to detect objects and classify images.¹⁶ During this competition, CNN models have made great progress and aroused attention throughout the world. For instance, in 2012, AlexNet achieved a top 5 error rate of 15.3% that consisted of five convolutional layers, some of them followed by max-pooling layers and three fully connected layers.¹⁷ Furthermore, CNN-based models such as GoogLeNet/Inception,¹⁸ ResNet,¹⁹ VGGNet,²⁰ and DenseNet²¹ also have also functioned well in the competition. The application of these CNN models and their derivatives in lung imaging is prevalent today.²²⁻³⁴ Table 1 presents the features of some popular models and their applications in lung cancer imaging.

Table 1.

The summary of several CNN models.

CNN architecture	Year	Test top 5 error rate (%)	Main improvement	Examples of exploit in lung cancer (ref.)
AlexNet	2012	15.32	A “template” for CNN models by 5 conv, 5 pooling, and 3 fc with the first use of ReLU as activation function	Detecting potential malignant lung nodules26,27
Inception-V1	2014	6.67	A wider network by introducing inception block with different size filters in same layer	Detecting potential malignant lung nodules26,28,33 Discriminating the histological subtypes29
VGG	2014	7.32	A deeper network by increasing conv layers with small size (3*3) filters	Segmenting lung cancer image30
ResNet	2015	3.57	An even deeper network trying to solve the vanishing gradient problem by residual blocks with skip connections	Predicting malignancy of lung nodules26,32
DenseNet-264	2016	5.17	A logic extension of ResNet by making every layer densely connected to the previous layers	Classifying benign and malignant pulmonary nodules34
SeNet	2017	2.25	Proposing the novel “Squeeze-and-Excitation” (SE) block to improve the interdependencies of channels	Detecting pulmonary nodules31

The top 5 error rate was reported with test data in ILSVRC. Abbreviations: conv, convolutional layers; fc, fully connected layers; ReLU, rectified linear unit.

Conventional machine learning and deep learning algorithms mostly serve one specific goal, which leads to a heavy demand for data. It is time- and energy-consuming to label medical images that require professional knowledge for detecting specific structures. To solve this question, transfer learning is widely used in model construction, which gains knowledge from solving one problem and utilizes what it learned to resolve another different but related problem. For example, ImageNet is a useful resource with thousands of natural images, which could be a valuable dataset for the CNN medical image processing training dataset with the help of transfer learning. As we mentioned, participants involved in ILSVRC developed models for this competition and often release their final model under a permissive license for reuse.¹⁶ These initial deep learning models are built and trained on more than 1 million pictures in the ImageNet dataset first. And then the model can be fully or partially reused and fine-tuned for the new task regarding lung cancer.³⁵ Moreover, a medical-to-medical transfer is also available with the model pretrained in a medical image dataset.^22,36 However, in the application of transfer learning, Raghu et al. pointed out that pretrained CNN models with transfer learning under ImageNet dataset did not outperform small conventional architecture significantly. They explored further and found that most meaningful features were concentrated in the lowest layers, which gave a hint that hybrid approaches could be generated according to transfer learning, such as reusing the lower stages and redesigning the top layers of the existing CNN model to establish a more efficient one.³⁷ Overall, transfer learning enables researchers to build more robust models for a wide variety of tasks of interest efficiently without establishing a specific dataset for training and further improve the reproducibility of the models.

Radiomics

Radiomics is a method extracting useful features to uncover potential information about diseases through medical images.³⁸⁻⁴⁰ In contrast to the deep learning mentioned before, radiomics belongs to traditional machine learning algorithms depending on feature engineering. The overview of radiomics workflow is illustrated as follows.

Image acquisition

Depending on the disease type, raw image data is generated with specific radiological machines such as CT, magnetic resonance imaging, and positron emission tomography (PET). Taking CT images as an example, CT acquisition parameters such as peak kilovoltage (kVp), milliampere-second (mAs), reconstruction kernel, and reconstruction slice thickness could vary in different institutions.^41,42 Previous studies showed that standardization of reconstruction methodology, especially reconstruction kernel and reconstructed slice thickness, was necessary for generating reproducible radiomic features.^43,44 Li et al. found that thin slice thickness (1 mm) with a smooth convolutional kernel (B30f/B31f/B31s) constructing a better model for prediction of epidermal growth factor receptor (EGFR) status in nonsmall cell lung cancer.⁴⁵ Unifying reconstruction outcomes for various radiological parameters is still a challenge for the generalization of models in clinical work. Furthermore, due to the different CT parameters, image preprocessing prior to segmentation is required to increase the quality of images, and enhance the accuracy and interpretability of features. For instance, smoothing, voxel size resampling, and gray-level normalization were applied to initial CT images produced by different scans.⁴⁶⁻⁴⁹

Image segmentation

Images are segmented and reduced to the volume of interest (VOI, tumor with or without peri-tumor environment) in this step. During this process, different studies opt for different ways, such as inspection by experts, and automated and semi-automated algorithms.⁵⁰ For lung cancer segmentation, the difficulty depends highly on the density of the tumor and the anatomic relationship with surrounding structures. Compared with the clear boundary of solid nodules, pure ground-glass nodules (pGGNs) and mixed ground-glass nodules (mGGNs) have an obscure margin that is hard to define. Meanwhile, manual segmentation costs lots of time and human resources in extracting the VOI by human inspection when the tumor is surrounded by mediastinum or connects with pleura or blood vessels, which also obviously increase intra- and inter-operator variability.^51,52 Thus, semi- or fully automated methods are selected for reducing inter-operator variability. There are lots of segmentation tools available now such as 3DSlicer (www.slicer.org),⁵³ itk-SNAP (www.itksnap.org),^54,55 and MIM software (www.mimsoftware.com).⁵⁶ Most of these applications offer some manual or semi-automated segmentation choices. It depends on researchers to select the software that is most suitable for their studies. However, visual inspection is still required after software processing because experience has told us that sometimes the software can fail.

Feature extraction

Many radiomic features can be extracted from VOI with high-throughput computing. Those features are generally classified into two types. The ones that directly explain radiographic images such as shape, size, calcification, vascularity, necrosis, and so on are characterized as semantic features, while the quantitative features derived mathematically are defined as agnostic features.³⁸ Moreover, agnostic features can be graded as first-, second-, and higher-order statistic features. First-order statistics, such as energy, mean, and variance, describe the gray-level intensity distribution based on a single voxel or pixel, and second-order features concerning the spatial patterns of gray-level intensities of a region of interest.⁵⁷ Higher-order features are derived from the computational process of adding filters or mathematical transformations to the images.⁵⁸

Feature selection and data analysis

Different from the aim of feature extraction to find as many features as possible, feature selection is performed to avoid overfitting and to find reproducible and repeatable features.^7,59 Feature selection can be classified into three main schemes.

The first is the filter method where features are removed with low variance under univariate analysis.⁶⁰ It calculates a certain statistical index of each feature that informs their correlation to the outcome variable, and then selects those with high ranks. Because this method does not consider the relationship among variables, it could include irrelevant features.

The second is the wrapper method, which repeatedly evaluates the importance of one subset of features. A machine learning algorithm architecture is built to evaluate the subset with a chosen metric each time, and finally a model with the best performance is selected.⁶¹ One example of the wrapper method is recursive feature elimination. It recursively removes the weakest features per loop, and finally reachs the specified number of features. Obviously, due to repeated learning steps and cross-validation, this type of method increases computational time and has a high overfitting risk when the number of observations is insufficient.

The third is the embedded method, which is similar to the wrapper method in finding a suitable subset of features.⁶¹ The difference is that the embedded technique trains one machine learning model and selects the optimal features automatically during learning, which reduces the computational complexity compared with the wrapper technique. One popular embedded method is LASSO (least absolute shrinkage and selection operator) regression or L1 regularization.62,−64 In the linear model, regularization adds a penalty to the coefficient of each feature and for LASSO, some of the coefficients shrunk to zero, which means the correlated features can be removed for the final model. This method has often been used in lung cancer radiomic study. For example, Huang et al. performed the LASSO Cox regression model to select the most valuable features for the prediction of prognosis of early-stage nonsmall cell lung cancer.⁶⁴ Five features with nonzero coefficient were included: CE_kurtosis_0, CE_uniformity_0_0, CE_homogeneity_45_0, UE_uniformity_45_1.0, and CE_uniformity_0_1.5. The following constructed radiomic signature could successfully give an estimation of a patient’s prognosis.

Studies have been carried out to evaluate which feature selection method is suitable for different aims.^65,66 Parmar et al. compared 14 different radiomic feature selection methods with a dataset of 464 images from patients with lung cancer for survival prediction to investigate their capability in dealing with high-throughput data mining problems. Through assessment by using the area under receiver operator characteristic curve (AUROC), the Wilcoxon test-based feature selection method outperformed others with the highest median AUROC.⁶⁶

Meanwhile, the radiomics quality score (RQS) was first mentioned by Lambin et al. for the evaluation of feature reproducibility, and now the compliance of RQS is required in radiomic studies to achieve high scientific quality.⁷

Once the radiomics features have been selected, they are analyzed to build a model for achieving specific goals in clinical work. Therefore, the analysis method varies according to the different goals and clinical outcomes of the studies, which ranges from statistical methods to AI or machine learning algorithms such as the support vector machine (SVM),⁶⁷ random forests,⁶⁸ and neural networks.²² However, the choice of modeling methods affects the prediction ability of radiomics⁶⁶ and inherent shortcomings such as individual assumption in logistic regression⁷ followed with different models. To test the reliability of the model, validation is required after constructing the model and receiver-operator-characteristic (ROC) curves are most frequently chosen for assessment of prediction accuracy.⁶⁹

Integration of deep learning and radiomics

As mentioned, radiomics and deep learning share a different path for medical image processing. Since traditional radiomics requires a lot of anticipation from human users, questions that whether deep learning could be applied in the workflow of radiomics to smooth the process of radiomics have been explored widely.

During the preprocessing of images in radiomics, different reconstruction kernels could affect the following feature selection. In one retrospective study, images scanned for one patient underwent two different reconstructing kernels: soft kernel (B30f) and sharp kernel (B50f) reconstruction with or without contrast-enhanced by one CT scanner.⁷⁰ And the CNN model formed by residual learning with an end-to-end way was constructed to convert kernels in the image preprocessing step of traditional radiomic workflow.⁷⁰ The results showed that CNN reduced the effect of different reconstruction kernels with the CCC (concordance correlation coefficient) between two readers improved to 0.84 compared with the CCC of 0.38 without CNN conversion.⁷⁰ Meanwhile, Park et al.proposed a deep learning algorithm based on CNN model for converting 3- or 5-mm-thick CT images into 1-mm slices, which significantly improved reproducibility with mean CCCs increasing from 0.27–0.65 to 0.45–0.72 (P < 0.001) for all comparisons in three pairings (slice thicknesses of 1- and 3-mm, 1- and 5-mm, and 3- and 5-mm).⁷¹ These articles showed that the combination of deep learning and radiomics could diminish the influence of disunified reconstruction parameters in clinical work for models and, in the future, more studies focused on this may improve the reproducibility of models built.

In the part of feature extraction, deep features were fused with quantitative radiomic features in some trials.^72,73 In one study predicting the malignancy of lung nodules, with the combination of traditional radiomic features and deep features extracted through the last fully connected layer of pretrained CNN models by transfer learning, AUC increased for predicting malignant nodules compared with the results from the traditional radiomic model. In this study, the authors found that the correlation between deep and radiomic features was low (in [0.5, −0.5]), which gave hope for the future improvement of image analysis.⁷² Thus, the feature extracted through DL and radiomics could be integrated to form a more detailed and multi-level model to illustrate the medical images better.

However, compared with traditional radiomics, deep learning depends on larger training data and substantial computer power such as high-performance GPUs for accuracy, and correspondingly more cost is inevitable. Thus, which algorithms are suitable for the aim of research should be evaluated carefully, and the integration of deep learning and radiomics is deserved for more explorations.

Clinical use of AI and radiomics for lung cancer

The techniques mentioned before are now prevalent in the field of lung cancer management. Taking the PubMed dataset as an example, we searched studies concerning AI and radiomics in lung cancer, and the overall trend of this topic has been on the rise over the last 10 years (Fig. 2). Studies of AI in lung cancer management started relatively later than those of radiomics but have attracted great attention recently. Both topics share quicker development trajectories, especially since 2017, which may result from the promotion of interdisciplinary study, the increasing awareness of precision medical care, and improvements in technical and computational facilities. Table 2 gives a summary of several examples focusing on solving clinical problems in lung cancer with AI and radiomics in different ways.

Figure 2.

Number of published papers by year.

Table 2.

Examples of applications of AI and radiomics in lung cancer.

Clinical application	Author and year	Tumor type	Image modality	Algorithm	Outcome
Early detection
Classify cancerous nodules	Hawkins 201668	Benign and malignant nodules	CT	Random forest classifier	AUC: 0.83
	Ardila D 201982	Benign and malignant nodules	CT	Three-dimensional CNN model	AUC: 0.944
	Baldwin 202025	Benign and malignant nodules in 5 to 15 mm	CT	CNN model	AUC: 0.896
Characterization of lung cancer
Classify histology subtype	Linning 201892	AD, SCLC, SCC	CT	SVM	AUC: 0.741 and 0.822 for SCLC and NSCLC, AD and SCLC etc.
	Wu 201693	AD, SCC	CT	Naive Bayes' classifier	AUC: 0.72
	Wang 202094	AD	CT	CNN model combined with radiomic features	AUC: 0.861
Classify somatic mutations	Velazquez 201796	NSCLC	CT	Random forest classifier	AUC: 0.80 and 0.69 for EGFR+ and KRAS+, and EGFR+ and EGFR− etc.
	Wang 201997	AD	CT	CNN model derived from DenseNet	AUC: 0.81 for EGFR− and EGFR+
Prognosis prediction
Predict outcomes after surgery or radiation therapy	Wu 2016107	NSCLC	PET/CT	LASSO with Cox survival model	Prognostic CI: 0.71
	Hosny 201822	NSCLC	CT	3D CNN model	AUC: 0.70 and 0.71 for surgery and radiotherapy
Predict response to chemotherapy	Wei 2019108	SCLC	CT	Regression	AUC: 0.797
Predict response to targeted therapy	Song 2018112	NSCLC	CT	Cox regression	AUC: 0.71
Predict response to immunotherapy	Sun 2018120	Advanced solid malignant tumor	CT	Regression	AUC: 0.67 (95% CI: 0.57–0.77)
	He 2020121	Advanced NSCLC	CT	3D DenseNet for feature extraction and fully connected network as classifier	OS: HR: 0.54, 95% CI: 0.31–0.95

Abbreviations: CT, computed tomography; AUC, area under curve; CNN, convolutional neural network; AD, adenocarcinoma; SCC, squamous cell carcinoma; SCLC, small cell lung cancer; NSCLC, nonsmall cell lung cancer; EGFR⁻/EGFR⁺, epidermal growth factor receptor negative/positive; PET/CT, positron emission tomography/computed tomography; LASSO, least absolute shrinkage and selection operator.

Early detection of lung cancer

Since 2002, the National Lung Screening Trial (NLST) has included participants with a high risk of lung cancer. Low-dose helical CT compared with traditional chest radiography was applied for their lung cancer screening. The results showed that lose-dose helical CT outperformed chest radiography in terms of sensitivity (93.8% versus 73.5%, respectively).⁷⁴ Therefore, regular lung cancer screening is recommended for the high-risk population for early diagnosis of lung cancer.⁷⁵ However, as mentioned in the NLST, only 3.6% of the nodules detected were cancerous.⁷⁴ The overdiagnosis of nodules in the clinical practice may lead to extra treatment for noncancerous nodules, which would have a negative influence on the cost-effectiveness of a lung cancer screening program.⁷⁶⁻⁷⁸

Accordingly, improving the present methods to detect nodules and discriminate lung cancer from benign nodules is now urgent. In 2014, Lung CT Screening Reporting & Data System (Lung-RADS) was first proposed as a tool for standardizing lung cancer screening CT reporting and management based on the traditional radiologist's lexicon such as size, calcification, and tumor density. However, when implemented in clinical work, controversies appeared⁷⁹ such as decreased sensitivity⁸⁰ and increased inter-observer variability.

As a result of interdisciplinary efforts, radiomics and further AI are chosen for compensation. For example, in the LUNA16 challenge, participants developed several ways to automatically detect pulmonary nodules in CT scans with high accuracy.⁸¹ In the aspect of recognizing the nature of nodules, one study included the NLST population and predicted the tendency of nodules detected at baseline with radiomics.⁶⁸ The population included patients showing nodules not being diagnosed as cancer at baseline, and after one- or two-time follow-up screenings, patients were divided based on two outcomes (nodule positive with cancer negative and nodule positive with cancer positive). Through radiomic workflow, 219 features were extracted. After two feature selection algorithms (relief-f and correlation-based feature subset selection) were performed, the random tree classifier with 23 features had the best performance, with an overall accuracy of 80% to predict the outcome of nodules (cancer or noncancer) at baseline.⁶⁸ In 2019, Google AI and collaborators built an end-to-end approach by using three-dimensional deep learning convolutional neural networks, with the functions of both detecting and characterizing lung cancer risk by using the input CT data only.⁸² Similarly, a risk prediction model called the Lung Cancer Prediction CNN (LCP-CNN) was constructed to estimate the malignancy risk of lung nodules and it reached an AUC of 89.6%.²⁵ According to the phenomena where different resolution CTs could be applied to one patient, Xu et al. produced an AI-based system called the DeepLN, which detected lung nodules in both low- and high-resolution CT screening images in order to solve the problem of multiple resolution of CT images in the real world.^83,84

Great achievements have been made in the field of the early detection of lung cancer, however, limitations and challenges remain in clinical work.^85,86 As mentioned before, different models and algorithms have been selected for developing the final decision-making approach, and the training sample was recruited from a population of specific nationalities being scanned with various CT scanning parameters, which induces low reproducibility and marketability. Future research with larger samples and protocols for comparing different models is expected.

Characterizing lung cancer

Before choosing the optimal treatment for a lung cancer patient, various invasive tools are suggested for uncovering the characterizations of lung cancer, such as histology and genetic mutation.

Lung cancer is classified into two major histological types: small cell lung carcinoma (SCLC, around 15% of all lung cancers) and non-SCLC (NSCLC, around 85% of all lung cancers). NSCLC can be further divided into several subtypes such as adenocarcinoma, squamous cell carcinoma, and large cell carcinoma.⁸⁷⁻⁸⁹ Radiomics gives a noninvasive way to identify lung cancer histological subtypes on images.^90,91 Linning et al. established a model for the classification of SCLC and NSCLC with an AUC of 0.741. Two features included in the model (Law-15 and Contrast) had higher values in NSCLC than those in SCLC, whereas the value of “Uniformity” in NSCLC was lower than that in SCLC, which further indicates the higher heterogeneity of NSCLC than SCLC.⁹² In another study, 440 features were extracted, and multivariate machine learning models were trained for classification of NSCLC, which shows the strong power of radiomics and AI in precision medicine.⁹³ Furthermore, in a study classifying subtypes of lung ADC, the proposed model combing both deep learning and radiomic features outperformed other pure radiomic and deep learning models with an accuracy of 0.966.⁹⁴ Classifying histology with medical images before surgery could help doctors choose neoadjuvant therapy efficiently and improve the life quality of patients by avoiding invasive procedures such as biopsy. And depending on the previous studies, the combination of deep learning and radiomics deserves further exploration.

Precision medicine also emphasizes the involvement of genetic changes in patients, and lung cancer is one of the few cancers with high genetic alteration;⁹⁵ thus the combination of genomic and radiomic biomarkers has the potential to enhance the management of lung cancer. With an integrated analysis between somatic mutation testing for EGFR and KRAS mutations and radiomic features of CT images, Velazquez et al. developed three radiomic signatures predictive of mutational status for classifying: (i) between EGFR⁺ and EGFR⁻, (ii) between KRAS⁺ and KRAS⁻, and (iii) between EGFR⁺ and KRAS⁺.⁹⁶ The 3D Slicer was chosen for feature extraction followed by minimum redundancy maximum relevance (MRMR) algorithm for feature selection, and the top 20 MRMR ranked features were selected to train the random forest classifier.⁹⁶ With all the efforts, the radiomic signature distinguishing between EGFR mutated and KRAS mutated tumors showed a great performance with an AUC of 0.80. Meanwhile, signature discriminated between EGFR⁺ and EGFR⁻ cases with an AUC of 0.69, and with the integration of clinical information such as age and smoking status, the combined signature increased accuracy with an AUC of 0.75. Similar outcomes showed in KRAD⁺/KRAS⁻ radiomic signature and clinical-combined one (AUC = 0.63 versus 0.75, respectively).⁹⁶ Deep learning models are also employed in this area. Wang et al. pretrained the first 20 convolutional layers by transfer learning through the natural images from ImageNet database, and the last four convolutional layers were trained by CT images of lung adenocarcinoma.⁹⁷ Their results showed a significant difference between the deep learning scores of EGFR-mutant and EGFR-wild groups in both training and validation cohorts.⁹⁷ Such results were displayed in recent studies with different AI-based techniques for predicting the different gene mutations.^98,99 However, due to the lack of a unified validation dataset, it is hard to compare the outcomes shown in these articles. Therefore, the models are still in experimental stages instead of being applied immediately in the real clinical world.

Predicting clinical outcomes after treatment for lung cancer

Prognosis is an important part of precision medical care for lung cancer because even patients sharing the same TNM staging with similar therapies exhibited various outcomes. To standardize the measurement of tumor response, the Response Evaluation Criteria in Solid Tumors (RECIST) guideline was introduced in 2000.¹⁰⁰ With the popularization of this criteria, vast quantities of images were produced, which allows AI to facilitate in predicting prognosis through mining data of images during follow-up.^{22, 101−103}

Surgeries for resection and radiotherapy are preferred for lots of NSCLC patients. In one proposed study, CT radiomic signature of tumor and peritumoral lung parenchyma was applied for risk stratification of post-surgery relapse or death, which successfully stratified patients into two groups with significant differences in the 3-year disease-free survival.¹⁰⁴ Similarly, a radiomic signature was constructed for the estimation of prognosis in patients with early-stage NSCLC.⁶⁴ Depending on the image biomarker, low- and high-risk groups were set with significant differences in the 3-year disease-free survival and independent of clinical-pathologic risk factors.⁶⁴ Another study focused on predicting the 2-year overall survival of patients undergoing either surgery or radiotherapy by using AI-based deep learning networks.²² The CNN model was first constructed for predicting 2-year overall survival of patients receiving radiotherapy, and subsequently, transfer learning was applied to achieve the same goal for patients undergoing surgery, and the CNN models presented significant prognostic performance for radiotherapy [(AUC) = 0.70 (95% CI 0.63–0.78), P < 0.001] and surgery [AUC = 0.71 (95% CI 0.60–0.82), P < 0.001] patients, respectively.²² Based on PET/CT, CT density measurements of LNs may serve as an available radiomic feature for differentiating between malignant and benign LNs for N status.¹⁰⁵ In terms of metastasis of lung cancer, PET radiomics features were able to differentiate between primary and metastatic lung lesions.¹⁰⁶ Moreover, a proposed study found that by combing the quantitative radiomic approach with histology information, PET radiomic signature showed the ability of prediction of the risk of distant metastasis.¹⁰⁷ Different image modalities are available in lung cancer management, while under the processing of AI and radiomics, they show the similar outcomes. Therefore, some of the medical examination items could be reduced and the medical fees for patients could be lowered with the application of AI in clinical work.

Chemotherapy regimens such as Cisplatin and Paclitaxel are also prescribed in the management of lung cancer. A radiomic signature formed by 21 features was established to predict the response of chemotherapy for SCLC patients.¹⁰⁸ However, negative results also existed. In one study, a radiomic method was employed in the prediction prognosis of patients with advanced lung cancer receiving Cisplatin-based chemotherapy, and no features were significantly associated with survival in both training and validation groups.¹⁰⁹ As mentioned before, different training data could influence the capability of models, thus the need for a unified training dataset is urgent.

Based on the mutational status of the pulmonary tumor, targeted drugs are ordered accordingly, however, patients with the same genetic mutation do not have the same extent of sensitivity to drugs mostly due to drug resistance.^110,111 Therefore, AI and radiomics are also expected to predict the prognosis for patients taking target therapy. One radiomic signature consisting of 12 CT features showed the capability for discriminating patients with rapid and slow progression to EGFR-TKI therapy.¹¹² The combination of genetic study and radiomics is now popular and the coined term “radiogenomics” is becoming highly frequent.

Currently, immune checkpoints such as programmed cell death protein 1 (PD-1) and programmed death-ligand 1 (PD-L1) have been explored in lung cancer treatment that revolutionized the therapeutic options for patients with lung cancer.^113,114 Until now, the PD-L1 expression level on tumor cells is treated as the best available biomarker for choosing patients who will benefit from receiving anti-PD-1/PD-L1-based therapies.¹¹⁵ However, some studies showed that some patients with PD-L1 negativity also benefited from the anti-PD-1/PD-L1-based therapies, thus the predictive value of PD-L1 expression may be limited.^116,117 Moreover, the value of other biomarkers such as tumor-infiltrating CD8 cells and tumor mutation burden were excavated in accumulating articles, while no standardized criteria were launched for selecting eligible cancer patients to receive immunotherapy.^115,118,119 Imaging features were exploited to serve as the biomarker for predicting prognosis with immunotherapy in several studies as well. In one study, the abundance of CD8 cells in the training samples were aligned with the images to build a radio-genomic signature by elastic-net regularized regression method.¹²⁰ And this signature successfully predicted the clinical outcomes for validation groups consisting of patients with cancer who had been treated with anti-PD-1 and PD-L1.¹²⁰ Similarly, He et al. constructed one deep learning model depending on the level of tumor mutation burden to predict the clinical outcomes of advanced NSCLC patients receiving anti-PD1/PD-L1 therapies.¹²¹ Instead of invasive examination, imaging could serve as the biomarkers for selecting eligible patients for immunotherapy in the future.

Our research team designed the pulmonary nodule/lung cancer comprehensive management mode, which combined the AI techniques not only in radiology for lung cancer screening but also in managing a biomedical information big data platform and follow-up system (Fig. 3). Low-dose computed tomography was detected and evaluated by our multi-disciplinary team with the combination of biomedical examinations to explore the risk of modules. Then patients with no nodules or low- or intermediate-risk nodules were included in the follow-up system and doctors were systemically allocated for each time follow-up, while patients with high-risk nodules would undergo interventions such as micro-invasive surgery and further standard treatment for lung cancer. All the clinical examination and Health-Related Quality-of-Life questionnaire results were updated for our big data platform. A combined system like this helps us improve the management of lung nodules and further enhance the efficiency and precision of clinical work.

Figure 3.

The workflow of the pulmonary nodule/lung cancer comprehensive management mode.

Challenge

In these days, AI has improved clinical work with its huge potential. However, challenges still exist.

First of all, without the uniform data, the outcomes could be different for various types of AI algorithm and radiomics, even with a similar aim. Therefore, to compare the efficiency of AI and radiomic models fairly, a unified measurement and public validation dataset is needed.

Second, a huge amount of data are required for training to establish a more stable and accurate model. However, it is not always possible for one single clinical center to collect all the data they want, especially in collecting standardized data with regular follow-up (e.g. for treatment evaluation or prognostic analysis), for data-hungry methods such as deep neural networks. In terms of AI model constructing, semisupervised learning, which mines features from large amounts of unlabeled data, is preferred and has already showed success in natural image analysis, so it deserves our attention in the application of lung cancer. Another solution is to encourage the intellectual collaboration that institutions, regions, and even countries all agree to share their data for AI algorithm setup and improvement. However, this kind of collaboration remains problematic. One is the legality of sharing the protected health information for everyone. The other is that data from different races and countries may lead to bad generalization of trained AI algorithms. And different institutions may choose various radiological machines with different parameters.

Moreover, the interpretability, reproducibility, and generalization of AI algorithms in clinical work is now being discovered. For AI systems to get wider acceptance and commercial usage, it is imperative to provide transparent explanations of the hidden black box of AI, because the model is generated with a multilayer nonlinear structure without specific medical knowledge.¹²² Meanwhile, a deep learning algorithm constructed with many parameters and weights made it hard to reproduce.

Prospect

The field of precision medical care in lung cancer shows a remarkable growth with the progress in techniques. In the aspect of radiology, AI and radiomics have a mutual influence on each other. Radiomic approach bridges the gap between images and other available data and thus generates AI systems, which aims for improving the process of medical management of lung cancer.

In addition, imaging is not only one measurement for diseases. As AI has a broad definition, the usage of AI in lung cancer is not only restricted in radiology for diagnosis and prognosis but also in other applications such as wearables, testing tools,¹²³ and so on.^124,125 For example, swarm-intelligence was applied to efficiently select RNA biomarker panels from platelet RNA-sequencing libraries, which discriminated patients with NSCLC from noncancerous patients.¹²⁶ Another study used AI sensor for measuring the nonlinear heart rate variability in patients with lung cancer for uncovering their performance status, which may assist in detecting and prognosis for lung cancer.¹²⁷

In prospect, AI could be the detector by integrating the information of health records with data of physiological status, economic status, and even social networks through digging information from wearables, social media, or other devices of digital applications, and finds the individuals who have a high risk of disease. Before the arrival of patients, AI could make the appointments with doctors and organize the daily schedule. Once patients have undergone a radiological scan, AI could help in processing and analyzing the raw images combined with other relevant reports, which assists in decision-making for doctors. For follow-up patients, AI has the potential to detect micro-changes that may be neglected by human eyes, and this ability gives the chance for early intervention, which leads to a better prognosis further on. The integration of AI in medicine will reduce time-consuming and repetitive work, which could enhance the efficiency and accuracy of daily clinical work, accomplish the goal of precision medicine to improve the life quality of patients, and finally save social resources and improve life expectancy.

Conclusion

In this article, we gave a brief introduction of AI and radiomics, as well as their latest and meaningful implementation in the field of lung cancer imaging. In the future, more studies utilizing the updated AI-based technique are expected to improve the management of lung cancer; meanwhile, standardization for those trials also calls for better promotion of AI systems to different populations and clinical centers.

Author contributions

Writing: the original draft preparation, Y. Z.; review and editing, X. X., C. W. and L. S.; visualization, C.W. and J. G.; supervision and conception, Z.Y. and W. L. All the authors read the article and approved the final version.

Conflict of interest

None of the listed authors declared conflicts of interest.