Application of Radiomics and Artificial Intelligence for Lung Cancer Precision Medicine

Abstract

Medical imaging is the standard-of-care for early detection, diagnosis, treatment planning, monitoring, and image-guided interventions of lung cancer patients. Most medical images are stored digitally in a standardized Digital Imaging and Communications in Medicine format that can be readily accessed and used for qualitative and quantitative analysis. Over the several last decades, medical images have been shown to contain complementary and interchangeable data orthogonal to other sources such as pathology, hematology, genomics, and/or proteomics. As such, “radiomics” has emerged as a field of research that involves the process of converting standard-of-care images into quantitative image-based data that can be merged with other data sources and subsequently analyzed using conventional biostatistics or artificial intelligence (AI) methods. As radiomic features capture biological and pathophysiological information, these quantitative radiomic features have shown to provide rapid and accurate noninvasive biomarkers for lung cancer risk prediction, diagnostics, prognosis, treatment response monitoring, and tumor biology. In this review, radiomics and emerging AI methods in lung cancer research are highlighted and discussed including advantages, challenges, and pitfalls.

Medical imaging is the standard-of-care for early detection, diagnosis, treatment planning, monitoring, and image-guided interventions of lung cancer patients. Most medical images are stored digitally in a standardized Digital Imaging and Communications in Medicine (DICOM) format that can be readily accessed and used for qualitative and quantitative analysis. Computerized image-based feature extraction from medical images has been an area of research over the last several decades to create computer-aided diagnosis (CAD) systems (Dhawan et al. 1986; Giger et al. 1988; Kawata et al. 1998; Hardie et al. 2008). However, in today's conventional clinical radiology practice, only a few quantitative metrics are used to describe the phenotype of nodules or tumors. This limited list of quantitative metrics with respect to pulmonary lesions includes (1) computed tomography (CT)-based largest diameter of nodules detected incidentally or in a screening setting following the Fleischner Society (Bueno et al. 2018), or Lung-RADS (ACR 2014) guidelines respectively, (2) CT-based largest diameter of tumors following the Response Evaluation Criteria in Solid Tumors (RECIST) (Eisenhauer et al. 2009), (3) standardized uptake value (SUV)-derived metrics from positron emission tomography (PET), and (4) percent enhancement of lesions on magnetic resonance imaging (MRI).

Over the last decade, there has been emerging evidence that medical images contain complementary and interchangeable data orthogonal to other sources such as pathology, hematology, genomics, and/or proteomics (Gillies et al. 2016). “Radiomics” is an emerging field of research that involves the process of converting standard-of-care images into a quantitative image-based biomarker that can be merged with these orthogonal clinical and other “omic”-based data and subsequently analyzed using conventional biostatistics or artificial intelligence (AI) methods. As radiomic features capture biological and pathophysiological information from these regions-of-interest (ROIs), these quantitative radiomic features have shown to provide rapid and accurate noninvasive biomarkers for lung cancer risk prediction, diagnostics, prognosis, treatment response monitoring, and tumor biology. In this review, radiomics and emerging AI methods in lung cancer research are highlighted and discussed including advantages, challenges, and pitfalls.

THE RADIOMICS PIPELINE

Medical images can be analyzed by either extracting quantitative features from identified and delineated ROIs or by analyzing an entire image or image series. The former is often referred to as “conventional” radiomics where quantitative image features are extracted from a segmented ROI and analyzed while the latter is often driven by emerging deep learning (DL) methods that do not always depend on segmentation of an ROI. With respect to conventional radiomics, there are five fundamental steps: (1) image acquisition and digitization, (2) ROI selection and segmentation, (3) quantitative feature extraction, (4) biomarker discovery/training, and (5) test and validation (Fig. 1A).

Figure 1.

Image-based biomarker modeling pipelines. (A) Conventional radiomics pipeline. (B) Deep learning pipeline. (ROI) Region of interest, (IHC) immunohistochemistry.

The first step involves image-data acquisition using standard-of-care medical images. Because imaging is widely used in oncology care, radiomics has substantial utility with the readily available image data that exist in every radiology clinical PACS server. However, image-acquisition protocols vary widely within and across radiology practices. Although it is advantageous to have homogeneous image-acquisition parameters in an analytical cohort, there are downstream image-processing steps and analytical methods that can be applied to address heterogeneous image acquisitions, often resulting in more robust predictive models than those obtained with a single homogeneous acquisition protocol (Mu et al. 2019). For instance, resampling all image voxels in a data set to a specific voxel size is a well-known and applied technique in a majority of radiomic studies to increase robustness of features that are subsequently extracted (Shafiq-Ul-Hassan et al. 2018). When texture features are calculated, a common practice of relative discretization (i.e., clustering of the pixels in the image to a fixed number of bins) or absolute discretization (i.e., clustering of the pixels in the image to a fixed bin size) have also shown to substantially impact the reproducibility of PET- (Leijenaar et al. 2015), CT- (Larue et al. 2017; Shafiq-Ul-Hassan et al. 2017), and MRI- (Duron et al. 2019) derived features. Normalization of radiomic features (e.g., by total number of ROI voxels, a single voxel volume, etc.) that are sensitive to voxel size and image-acquisition parameters have also shown to improve the robustness of radiomics across different scanners as well of eliminating the dependence on volume (Shafiq-Ul-Hassan et al. 2017, 2018).

The second step involves the ROI selection and segmentation. An ROI may include one or more “index” nodules or tumors, metastatic lesions, or whole organs in three-dimensional space. As such, it creates an advantage of radiomics over tissue-based biomarkers due to capturing the entire ROI in three-dimensional space (e.g., tumor and surrounding parenchyma) compared to just a portion of tumor(s) that was captured on a biopsy. Although there are clear guidelines for “index” lesion selection (Eisenhauer et al. 2009), a radiologist or a very experienced imaging scientist is ideally best suited for ROI selection. Additionally, overreads of 5%–10% of all images should be considered and conducted by an independent radiologist and this can result in changing the identity of the “index” lesions. Segmentation of the ROI can be performed using manual, semiautomated, or fully automated approaches. Manual segmentation is laborious and although it can be very accurate when conducted by an experienced radiologist, interreader studies show poor reproducibility with Dice coefficient often <80% (Alilou et al. 2017). Semiautomated (e.g., using an initial parameter such as a single click on the center of a lesion) or fully automated segmentation approaches are faster and more repeatable (Kalpathy-Cramer et al. 2016b; Tunali et al. 2019c) but may still require manual verification and corrections if the segmentations algorithm fails on difficult cases. As some radiomic features are extremely sensitive to segmentation, a segmentation algorithm that produces accurate, reproducible, and consistent segmentations in an automated manner and requires minimal user input is a critical requirement to increase stability of image-based features (Kalpathy-Cramer et al. 2016a; Tunali et al. 2019c).

The third step is feature extraction where image-based features are calculated from the ROI. The various classes of features have been described in detail elsewhere (Gillies et al. 2016). Briefly, radiomic features are classified into first-, second-, and higher-order features. First-order features include shape- and size-based features and features that describe the distribution of values of individual voxels (i.e., volumetric pixels) without concern for spatial relationships. Shape- and size-based features consist of the larger-scale form of the nodule (or tumor) such as sphericity, compactness, surface area, surface-to-volume ratio, and volume. Nodule/tumor voxel intensities describe the histogram of intensities within the CT image through basic metrics such as mean, median, maximum, minimum, uniformity, or randomness (entropy) of the intensities in the image, as well as the skewness (asymmetry) and kurtosis (flatness) of the histograms of values. Second-order image features are described as “texture” features and they describe statistical interrelationships between voxels with similar (or dissimilar) contrast values. Texture features consist of image consistency and patterning textures that quantify intranodule (or intratumor) heterogeneity differences observable within the segmented nodule, tumor, or stromal volume. In practice, there are numerous methods and formulas that can be used to calculate texture features, resulting in hundreds of values, far too many to elaborate here. Higher-order features are also described as “texture” features and impose filter grids on the image to extract repetitive or nonrepetitive patterns. These include fractal analyses, Minkowski functionals, wavelets, and Laplacian transforms of Gaussian bandpass filters. These generally describe enhanced textural information within the tumor/nodule volume through an iterative process of decomposing the original image into different frequency ranges/scales and then feature extraction occurs from the decomposed images to quantify textural and spatial intensity. To enable interoperability of radiomic signatures (i.e., repeatable of signatures), differences in terminology, algorithms, software implementations, and other methodological facets must be explained distinctly. Recently, an international collaboration initiative called the image biomarker standardization initiative (IBSI) has worked toward standardizing the extraction of image biomarkers to address part of this interoperability problem by providing image biomarker nomenclature and definitions, benchmark data sets, and benchmark values to verify image processing and image biomarker calculations, as well as reporting guidelines (Zwanenburg et al. 2018).

The fourth step is the analytical steps of training or discovery of a model to predict the desired dependent variable (e.g., risk, prognosis, treatment response, etc.) or phenotype (e.g., gene mutation[s], molecular signature, protein expression, etc.) using conventional biostatistics or machine learning (ML) methods (Fig. 2). The decision of what analytical method(s) to be deployed is dependent on many different factors including sample size (e.g., ML approaches may overfit smaller sample sizes), study end point (e.g., dichotomous vs. continuous or time-dependent), and the statistic or metric of interest (e.g., P-value-driven point estimate vs. performance statistics such as the area under the curve [AUC]). There are no set rules and this aspect of radiomics is an area of intense research and debate. Regardless, it is often advantageous that the training model be integrated to include orthogonal information including patient data, clinical data, and other 'omic information such as driver mutations, immunohistochemistry (IHC) proteomic data, or circulating biomarkers, when available. Image-based features that are nonreproducible can be eliminated (Balagurunathan et al. 2014a,b; Kalpathy-Cramer et al. 2016a; Tunali et al. 2019c) and principal component analysis or clustering methods could be used to reduce dimensionality of the created models to avoid overfitting (Hosny et al. 2018).

Figure 2.

End points for radiomics modeling. Radiomics can predict molecular marker and also predict patient outcomes such as overall survival.

The critical final step is the testing and validation of the model with independent data from internal and/or external data sets. A successful external validation can demonstrate the potential generalizability of the model as it may include different acquisition protocols and patient population(s). Defining a successful validated model is dependent on the statistic or metric of interest. For example, if the model was trained to find an integrated model with the highest AUC, then AUC obtained in the test and validation cohorts should not be significantly different from the trained model.

ARTIFICIAL INTELLIGENCE IN MEDICAL IMAGING

AI is an umbrella term technology that includes ML and DL and enables machines to mimic human intelligence and consequently have a downstream effect on transforming industries such as medicine. DL is a subset of ML, while ML is a subset of AI (Fig. 3). AI has unprecedented success due to advances in central processing units (CPUs) and graphics processing units (GPUs), and the availability of larger data sets due to increased storages and digitization. AI methods have made substantial advances on interpreting complex and multidimensional data in various applications from stock trading to computer vision and natural language processing to self-driving cars. This advance got its share within health care on various applications such as patient risk management, drug discovery, patient diagnosis and prognosis, and medical image interpretations. Specifically, DL methods (e.g., convolutional neural networks) excel at image pattern recognition and thus may quantify image information that is subtle to humans in the setting of medical imaging.

Figure 3.

Artificial intelligence, machine learning, and deep learning.

Generally, AI in medical image analyses can be applied in two ways: (1) radiomic features extracted from ROIs can be input into ML algorithms to develop classifiers (i.e., “conventional radiomics”), or (2) an entire image or image series can be an input into a DL network to develop classifiers. DL methods are currently being explored in detection, characterizing, and monitoring of cancers. Lung cancer is one of the most prominently researched cancer types by AI due to its medical importance, an abundance of CT or PET/CT images, and the high-contrast, high-resolution inherent in CT images (Hosny et al. 2018). Although DL and conventional ML radiomic approaches have similar end points in terms of medical image analysis, they differ substantially in terms of training methods. As previously described, “conventional radiomics” requires accurate segmentation of the ROI, human-engineered feature extraction, and selection for optimized model creation, whereas DL methods do not require ROI annotation but rather use a single seed point or bounding box that can be identified by an expert human observer or a separately trained AI detection system.

AI methods have already been deployed with great success in radiomics studies. However, to successfully use AI methods into clinical settings, several challenges remain. First, large patient and/or image data sets are crucial to the success of any AI approach. This is especially the case with the DL model, wherein a large number of layers are used (Fig. 1B). Small data sets are prone to being overfitted, ungeneralizable, and nonreproducible. Because medical imaging is widely deployed in developed countries, access to large imaging data sets should be a real-world reality. However, a critical barrier to access is the time required to annotate and curate such data sets, which is well recognized (Gillies et al. 2016). For DL approaches, data can be “augmented” with a number of approaches (Napel et al. 2018). One of the most widely used approaches is “transfer learning,” where a neural network model is first trained on a different problem to identify features such as edges, sharpness, etc., and afterward the network is either reused (partly or completely) or tuned to be adapted for the new task. Another approach is using a large public repository such as The Cancer Imaging Archive (TCIA) or the National Lung Screening Trial (NLST) with annotated medical images to pretrain a neural network and tune it for a specific task. An alternative approach to data augmentation is to artificially inflate the training data set size by warping, rotating, or inverting the images, which theoretically can be used to overcome the lack of large sample sizes. A more recent development is to augment image data using generative adversarial networks (GANs) that create new data by generating images with a different modality, such as contrast-enhanced to nonenhanced CT, or MRI to CT (Sandfort et al. 2019). However, even with augmentation, data can be insufficient when rare diseases or small data sets are being modeled. An alternative way to tackle this is through a centralized database or using a distributed learning platform where the “code” is shared instead of the data (Lambin et al. 2017). One other potential limitation with DL methods is that these systems are a “black box” in that there is a lack of transparency as to how the networks perform the various tasks. However, Chartrand et al. (2017) argues that a highly accurate opaque system is preferable to an inaccurate transparent one, that users may never understand how these networks work, and AI will likely identify patterns that humans cannot interpret. Another limitation is that most DL methods are optimized for binary classifications rather than time-dependent end points based on a continuous scale, such as survival-related outcomes.

RADIOMICS AND AI IN EARLY DETECTION/LUNG CANCER SCREENING

Worldwide, lung cancer is the most common diagnosed cancer and leading cause of cancer-related deaths (Bray et al. 2018). Because of the large number of diagnoses, even incremental improvements in patient outcomes would have a profound impact. There is a vast difference in survival outcomes between early-stage versus late-stage patients, where a localized lung cancer has a 56% 5-year overall survival (OS) and only 5% for distant metastasized disease (Siegel et al. 2019). Thus, detecting lung cancers early, when they are manageable, or even curable, is a critical need. Asymptomatic lung cancers are detected either incidentally (e.g., when a patient receives an imaging study for another indication) or via a screening program that is designed for individuals who are at high risk. In 2011, results from the NLST demonstrated a 20% relative reduction in lung cancer mortality for individuals screened by low-dose computed tomography (LDCT) compared with standard chest radiography in a high-risk population of 53,454 current and former smokers ages 55–74 (National Lung Screening Trial Research Team et al. 2011; Silva et al. 2017; De Koning et al. 2018; Pastorino et al. 2019). More recently, in 2018, the initial results of the Nederlands-Leuvens Longkanker Screenings Onderzoek (NELSON) trial also showed significant declines in lung cancer mortality (De Koning et al. 2018). Moreover, in 2019, two additional randomized trials conducted in Italy called the Multicentric Italian Lung Detection (MILD) trial (Pastorino et al. 2019) and in Germany called The German Lung Cancer Screening Intervention (LUSI) trial (Becker et al. 2020) were published, providing additional validation of lung cancer screening value. As such, the cumulative evidence based on the results of three published trials and anticipated publication of the NELSON trial has demonstrated substantial beneficial mortality reductions associated with LDCT screening.

Despite the mortality benefit associated with lung cancer screening, there are many limitations of early detection by LDCT (De Koning et al. 2018) including detection of a large number of indeterminate pulmonary nodules and overdiagnosis of indolent neoplasms that may not otherwise cause clinical symptoms or death (Schabath and Gillies 2015). Additionally, although small pulmonary nodules (<6 mm) are considered to be “negative” in lung cancer screening, prior analyses have shown that NLST participants who had baseline negative screens and were diagnosed with incidental lung cancer in follow-up screening had poorer survival rates compared to screening participants who had a baseline positive screen that developed incidence of lung cancer (Schabath et al. 2015; Patz et al. 2016).

Developing biomarkers for diagnostic discrimination between malignant tumors and benign nodules is not unique to imaging research and studies have been conducted for decades in this space (Brzakovic et al. 1990; Hadjiiski et al. 1999; Wei et al. 2005). With the emergence of lung cancer screening, there has been a substantial increase in efforts to develop noninvasive image-based classifiers. As such, after the NLST was publicly available, many studies were used from patient medical images on this immense cohort. Following is just a few examples of the research that this data set has made possible (Table 1). One of the earliest studies conducted by Hawkins et al. (2016) used baseline LDCT scans from the NLST to predict which baseline indeterminate (4–12 mm) pulmonary nodules would subsequently be diagnosed as an incident lung cancer in follow-up screening intervals. The authors developed an ML model of 23 features that yielded a radiomic signature with an AUC of 0.81 for predicting development of cancer in 1 year, which was far superior to volume alone (AUC = 0.72) and is the most widely used predictive marker in the clinic. Peikert et al. (2018) created a radiomics model to compare malignant and benign screen-detected indeterminate lung nodules used in the NLST data set. Using least absolute shrinkage and selection operator (LASSO) multivariable analysis, they reported that an AUC of 0.939 could be reached with only eight nonredundant radiomic features. Huang et al. (2018) used the NLST to perform a matched case-control study to identify CT image features to increase the positive predictive value (PPV) and reduce the false positive rates (FPRs) compared to thoracic radiologist evaluations. Cherezov et al. (2018) used images and data from the NLST and improved malignancy prediction accuracy from 74.7% to 81.0% by implementing nodule size–specific models. In their study, they used the Synthetic Minority Oversampling Technique (SMOTE) to overcome the class imbalances that are inherent in these data sets. Chae et al. (2014) used texture features to differentiate preinvasive lesions from invasive pulmonary adenocarcinomas that are marked as part-solid, ground-glass opacities (GGOs) on chest CT scans. Their artificial neural network (ANN) model showed an excellent performance using five radiomic features with an AUC of 0.981 on 86 part-solid GGOs. Liu et al. (2017) extracted semantic features (i.e., radiological traits quantified by radiologists) from baseline nodules in the NLST and developed a model that predicted which participants would be diagnosed with lung cancer 1–2 years (AUC = 0.80) after the baseline screen. Ardila et al. (2019) conducted a DL network to predict the risk of lung cancer using the NLST data set, and their model achieved an AUROC of 94.4%, which was validated on an independent validation data set. Also using images from cancer patients in the NLST, Morales et al. (2019) identified two radiomic features that stratified patients into three risk groups (e.g., low, intermediate, and high) with an AUC of 0.878. One of the features was found to be associated with FOXF2 expression, which has been shown to be associated with poor prognosis. Dhara et al. (2016) used 891 nodules from the Lung Image Database Consortium and Image Database Resource Initiative database and classified malignant versus benign nodules using a support vector machine (SVM). Their models reached an AUC of 0.951, which outperformed methods that required manual segmentation of a trained radiologist. Other studies have tried addressing the issue of overdiagnosis by using quantitative features. Maldonado et al. (2013) developed a decision algorithm called CANARY that binned pulmonary nodules as aggressive or indolent. Their algorithm had a validation sensitivity of 98.7% and a specificity of 63.6%. Finally, Lu et al. (2019) identified a model that included features that were extracted from the tumor and the “difference region” (i.e., the part-solid region of the tumor) and yielded an AUC of 0.846 to discriminate aggressive versus indolent nodules. The summary of these studies can be found in Table 1.

Table 1.

Radiomics and artificial intelligence (AI) studies in early detection and lung cancer screening

References	Imaging modality	Major end point	N (training + test cohorts)	Number of features in the model	Evaluation metric	Analytical method used	Study description	Validation type
Hawkins et al. 2016	CT	Diagnosis	176 + 152	10	AUC = 0.81	Random forest classifier	Predicting benign vs. malignant nodules on a screening cohort	Internal validation with a separate data set
Peikert et al. 2018	CT	Diagnosis	726	8	AUC = 0.939	LASSO	Differentiating benign vs. malignant from indeterminate nodules	Internal validation using bootstrapping
Huang et al. 2018	CT	Diagnosis	140 + 46	5 radiomic + 1 clinical	AUC = 0.9154	Random forest classifier	Differentiating benign vs. malignant from small nodules (≤20 mm)	Internal validation with a separate data set
Cherezov et al. 2018	CT	Diagnosis	255 + 212	Ranging from 5 to 10	AUC = 0.76–0.86	Machine learning	Differentiating benign vs. malignant based on nodule size	Internal validation with a separate data set
Chae et al. 2014	CT	Diagnosis	86	2 and 5	AUC = 0.981	Logistic regression + ANN	Differentiating preinvasive lesions from invasive pulmonary adenocarcinomas	No validation
Liu et al. 2017	CT	Diagnosis	102 + 70	4 semantic features	AUC = 0.74 and 0.80	Liner classifier	Differentiating benign vs. malignant from incidentally identified nodules	Internal validation with a separate data set
Dhara et al. 2016	CT	Diagnosis	891	Unknown	AUC = 0.8488–0.9505	SVM	Predicting benign vs. malignant nodules	No validation
Maldonado et al. 2013	CT	Tumor behavior	54 + 86	Unknown	Sensitivity = 98.7%	Logistic regression	Categorizing pulmonary nodules as aggressive or indolent	Internal validation with a separate data set
Lu et al. 2019	CT	Tumor behavior	114	5	AUC = 0.846	Logistic regression	Predicting tumor growth (VDT) in lung cancer screening using multi-CT windows	No validation
Ardila et al. 2019	CT	Diagnosis	6716 + 1139	Deep features	AUC = 0.944	Deep learning	Predicting risk of lung cancer	External validation
Morales et al. 2019	CT	Tumor behavior	161 + 73	2	AUC = 0.878	Classification and regression tree	Stratifying incident lung cancer patients into risk groups of survival	Internal validation with a separate data set

(CT) Computed tomography, (AUC) area under the curve, (LASSO) least absolute shrinkage and selection operator, (ANN) artificial neural network, (SVM) support vector machine, (VDT) volume doubling time.

RADIOMICS AND AI IN PROGNOSTICATION

Although pathologic staging remains to be the most important prognostic factor for lung cancer survival (Mirsadraee et al. 2012), there is marked variability in patient outcomes and survival among patients with the same stage of disease suggesting that other factors contribute to lung cancer survival, progression, and recurrence (Birim et al. 2006; Ries 2007; Kachroo et al. 2008; Dela Cruz et al. 2011; Pao and Girard 2011; Schabath et al. 2014). Accurately classifying the aggressiveness of tumors is critical as it can help physicians determine the potential prognosis of a patient beyond just stage alone and thus provide options to choose between curative and palliative treatments as well as the aggressiveness of the therapy and follow-up. Radiomic studies have shown that image-based classifiers have the potential to complement staging and improve prognostication of lung cancer. Aerts et al. (2014) analyzed non-small-cell lung cancer (NSCLC) and head and neck patients and validated a CT radiomic signature that had better prognostic performance than tumor-node metastasis (TNM) staging and volume with a concordance index of 0.65. They found associations between their signature features and gene-expression patterns using gene-set-enrichment analysis where the most informative features were correlated with cell-cycling pathways. Grove et al. (2015) developed two CT features, convexity (hazard ratio [HR] = 0.31) and entropy ratio (HR = 2.36), which were significantly associated with OS of patients diagnosed with primary lung adenocarcinoma using two independent cohorts. Tunali et al. (2017) assessed the same cohorts and developed novel radiomic features generated from radial gradient (RG) and radial deviation (RD) maps that also predict OS (HR = 0.40). Coroller et al. (2015) built a combined model of CT radiomics and clinical predictors to predict the development of distant metastasis (CI = 0.61) and Wu et al. (2016a) used fluorine 18 (¹⁸F) PET/CT-based radiomic features to also predict the development of distant metastasis (CI = 0.71). Huang et al. (2016) found radiomic signatures that correlated with disease-free survival (HR = 1.77). Several other studies (Huynh et al. 2017; Li et al. 2017; Oikonomou et al. 2018) investigated the prognostic performance of CT radiomic features for distant metastasis and locoregional recurrence after stereotactic body radiation therapy. Win et al. (2013) showed that heterogeneity on both CT and PET components of PET/CT were significant predictors of survival. Chae et al. (2014) and She et al. (2018) found CT radiomic signatures that differentiate indolent versus invasive lung adenocarcinoma with an AUROC of 0.98 and 0.95, respectively. The summary of these studies can be found in Table 2.

Table 2.

Radiomics and artificial intelligence (AI) studies in prognostication

References	Imaging modality	Major end point	N (training + test cohorts)	Number of features in the model	Evaluation metric	Analytical method used	Study description	Validation type
Aerts et al. 2014	CT	OS	422 + 225	4	Concordance index = 0.65	Cox proportional hazards regression	Prognostic power of radiomic features and the underlying gene-expression patterns	External validation
Grove et al. 2015	CT	OS	61 + 47	1	HR = 0.31 and 2.36, P-value: 0.008	Cox proportional hazards regression	Prognostic power of newly developed radiomic features	External validation
Tunali et al. 2017	CT	OS	61 + 47	1	HR = 0.40, P-value: 0.014	Cox proportional hazards regression	Prognostic power of newly developed radiomic features	External validation
Coroller et al. 2015	CT	OS and distant metastasis	98 + 84	3	Concordance index = 0.61	Cox proportional hazards regression	Predicting distant metastasis	External validation
Wu et al. 2016a	PET	Distant metastasis	70 + 31	2	Concordance index = 0.71	Cox proportional hazards regression	Predicting distant metastasis	Internal validation with a separate data set
Huang et al. 2016	CT	DFS	141 + 141	5	HR: 1.77 concordance index = 0.691	LASSO	Predicting DFS in early-stage patients	Internal validation with a separate data set
Huynh et al. 2017	CT	OS	131	13	AUC = 0.667	Correlation analysis (Spearman's correlation coefficient)	Early-stage disease recurrence prediction of patients treated with SBRT	No validation
Li et al. 2017	CT	OS	92	2 radiomic + 1 clinical + 1 semantic	Log-rank P-value = 0.0002	Cox proportional hazards regression	Early-stage disease survival prediction of patients treated with SBRT	No validation
Oikonomou et al. 2018	CT + PET	OS	150	7	Log-rank P-value = 0.002	Cox proportional hazards regression	Predict clinical outcome in lung cancer patients treated with SBRT	No validation
Win et al. 2013	FDG-PET/CT	OS	56 + 66	2 radiomic + 1 clinical	Cox regression P-value < 0.001	Cox proportional hazards regression	Predicting OS	External validation
Chae et al. 2014	CT	Prognostic	86	2	AUC = 0.981	ANN	Differentiate preinvasive lesions from invasive pulmonary adenocarcinomas	No validation
She et al. 2018	CT	Prognostic	207 + 195	5	AUC = 0.95	Logistic regression	Differentiate indolent from invasive pulmonary adenocarcinomas	Internal validation with a separate data set

(CT) Computed tomography, (DFS) disease-free survival, (SBRT) stereotactic body radiation therapy, (LASSO) least absolute shrinkage and selection operator, (ANN) artificial neural network, (OS) overall survival, (HR) hazard ratio, (FPR) false positive rate, (FNR) false negative rate.

RADIOMICS AND AI IN TREATMENT RESPONSE

Advances in lung cancer treatments (e.g., checkpoint blockade immunotherapy and tyrosine kinase inhibitors [TKIs]) have decreased lung cancer mortality rates and improve patient survival outcomes. Early assessment of a therapeutic efficacy and predicting treatment outcomes would aid decision support for which treatment has the potential to have optimal benefit for the individual patient. This could eliminate unnecessary treatments, reduce toxicities and costs, and increase patient survival. However, biomarkers that are highly predictive of both positive and negative responses that can be used prior to initiation of therapy are an unmet clinical need. As such, accurate, generalizable, and ideally noninvasive biomarkers are needed as diagnostic companions for different treatment types (Teng et al. 2018).

Immunotherapy that blocks inhibitory checkpoint signals has shown to yield durable responses. However, a substantial subset of patients do not respond to immunotherapy and, in some instances, patients experience rapid and lethal immunotherapy-induced hyperprogressive disease (HPD). Several studies have used CT radiomics to address this recent clinical unmet need of identifying immunotherapy response. Trebeschi et al. (2019) developed models to predict outcomes of both NSCLC and melanoma patients in the setting of immunotherapy. Their lesion-level performance in NSCLC patients yielded an AUROC of 0.83, whereas the best patient-level prediction models had an AUROC of 0.76. Sun et al. (2018) developed models that assess CD8 cell tumor infiltration and used this model to predict immunotherapy response of NSCLC patients. Their radiomic signature of CD8 cells consisted of eight radiomic features, which yielded an AUC of 0.76 to discriminate inflamed tumors from immune-desert tumors. Tunali et al. (2019a) developed radiomics clinical models to predict rapid disease progression, including HPD, and among NSCLC patients treated with immunotherapy found modest-to-high AUCs of 0.81–0.85. Following up on this work, Tunali et al. (2019c) developed and validated a clinical-radiomic risk model that identified a very high-risk group of patients associated with extremely rapid and poor survival outcomes (HR for OS = 5.35) compared to the low-risk group (HR = 1.00). Lastly, Mu et al. (2019a) developed a PET/CT radiomic signature to predict patients who are likely to achieve durable clinical benefit from immunotherapy that yielded an AUC of 0.81 in the validation cohort.

Similar to immunotherapy, only a subset of patients benefit from treatment with TKIs (Shepherd et al. 2005). As such, studies have been conducted using radiomics to predict patient outcomes and TKI treatment response. Cook et al. (2015) created models using textural features from fluorine 18 (¹⁸F) fluorodeoxyglucose (FDG) PET images to predict outcomes among patients treated with Erlotinib (an Epidermal Growth Factor Receptor [EGFR] TKI). Their model discriminated patients into high versus low OS (26.6 mo vs. 13.1 mo, P = 0.006). Ravanelli et al. (2018) identified CT texture features that predicted 6-month progression with an AUROC of 0.80 and 1-year progression with 0.76. Park et al. (2018) identified texture features from pretreatment FDG-PET/CT to predict early EGFR TKI failure. After adjusting for clinical parameters, high gray level co-occurrence matrix (GLCM) entropy was associated with worse survival (HR = 4.86).

Studies have also investigated the utility of radiomics for treatment responses to chemotherapy or radiation therapy. Coroller et al. (2017) used CT radiomic features extracted from primary lung tumors and lymph nodes to predict pathological complete response (pCR) and gross residual disease (GRD) after neoadjuvant chemoradiation before surgery. They identified a pCR radiomics feature (AUC = 0.75) that significantly outperformed total primary tumor and lymph node volume (AUC = 0.58) and a GRD radiomic clinical model that had an AUC of 0.73. Yu et al. (2018) trained and validated cohorts of a CT radiomic model to predict metastasis (HR = 1.27) among NSCLC patients treated with surgery or stereotactic ablative radiotherapy (SABR). Mattonen et al. (2016) identified an ML-based radiomics model to detect local recurrence after SABR. Their radiomic signature consisted of five features that discriminated local recurrence from fibrosis with an AUC of 0.85 and had a significantly lower false negative rate compared to an expert physicians assessment (99% vs. 23%). Khorrami et al. (2019) used peri- and intratumoral CT radiomic features to predict pemetrexed-based chemotherapy response and showed that peritumoral features were predictive for time-to-progression (AUC = 0.77). Fave et al. (2017) used delta radiomics (i.e., changes in radiomic features in time) and showed that radiomic feature alterations after radiation therapy were associated with tumor response (C-index = 0.558). The summary of these studies can be found in Table 3.

Table 3.

Radiomics and artificial intelligence (AI) studies in treatment response

References	Imaging modality	Major end point	N (training + test cohorts)	Number of features in the model	Evaluation metric	Analytical method used	Study description	Validation type
Trebeschi et al. 2019	CT	Response to immunotherapy	133 + 70	2	AUC = 0.83, P < 0.001	Machine learning	Identifying radiomic biomarkers for immunotherapy response	Internal validation with a separate data set
Sun et al. 2018	CT	Response to immunotherapy	135 + 256	8	AUC = 0.76	Machine learning	Identifying radiomic markers to assess tumor infiltrating CD8 cells and immunotherapy response	External validation
Tunali et al. 2019a	CT	Rapid disease progression	228	4 radiomic + 4 clinical	AUC = 0.804	Logistic regression	Identifying rapid disease progression phenotypes in patients treated with immunotherapy	Internal validation with bootstrapping
Tunali et al. 2019c	CT	Response to immunotherapy	180 + 90 + 58	1 radiomic + 2 clinical	Log-rank P < 0.001	Classification and regression tree	Identifying survival risk groups for patients treated with immunotherapy	External validation
Mu et al. 2019a	FDG-PET/CT	Response to immunotherapy	194 + 47 + 48	8	AUC = 0.81	LASSO	Predicting durable clinical benefit from immunotherapy	Internal validation with a separate data set
Cook et al. 2015	FDG-PET	Response to Erlotinib	47	1 delta radiomica	P = 0.01	Wilcoxon signed-rank test	Identifying baseline radiomic biomarkers and delta-radiomic biomarkers of treatment response and OS of patients treated with EGFR TKI	No validation
Ravanelli et al. 2018	CT	Response to Erlotinib	50	5	AUC = 0.8	LASSO	Determining high- and low-risk groups of patients treated with EGFR TKI	Cross-validation and bootstrapping
Park et al. 2018	FDG-PET/CT	Response to Erlotinib	161 + 21	8 univariable features	C-index range = 0.630–0.669	Harrell's C-index	Determining overall survival risk groups of patients treated with EGFR TKI	Internal validation with a separate data set
Coroller et al. 2017	CT	pCR	85	1	AUC = 0.75, P = 0.01	Random forest	Predicting pathological response after neoadjuvant chemoradiation	No validation
Yu et al. 2018	CT	Response to surgery or SABR	147 + 295	2	HR = 1.27, P < 2 × 10−16	Random survival forests	Predicting OS of stage I NSCLC patients	Internal validation with a separate data set
Mattonen et al. 2016	CT	Local recurrence	45	5	FPR = 24.0%, FNR = 23.1%	Machine learning	Assess physician ability to detect timely local recurrence and to compare physician performance with a radiomics tool	No validation
Khorrami et al. 2019	CT	OS and TTP	72 + 53	7	AUC = 0.77	Minimum redundancy, maximum relevance	Discriminative ability of radiomic features on response to chemotherapy	Internal validation with a separate data set
Fave et al. 2017	CT	Local recurrence	107	1 delta radiomica	Log-rank P-value = 0.269, C-index = 0.558	Cox proportional hazards regression	Assessing radiation therapy response by using delta radiomics	No validation

(HR) Hazard ratio, (OS) overall survival, (EGFR) Epidermal Growth Factor Receptor, (ANN) artificial neural network, (FPR) false positive rate, (FNR) false negative rate, (TTP) time-to-progression, (AUC) area under the curve, (pCR) pathological complete response, (SABR) stereotactic ablative radiotherapy, (LASSO) least absolute shrinkage and selection operator, (NSCLC) non-small-cell lung cancer.

^aDelta-radiomics is the difference in radiomic features between multiple scans.

RADIOMICS AND AI IN RADIOGENOMICS

Radiogenomics is the study of the relationship between imaging features and genomic phenotype(s) (e.g., gene expression, gene mutations, etc.) to inform the potential underlying cellular pathophysiology of a tumor. It should be noted that in some publications, “radiogenomics” refers to genomic prediction of radiation-response patterns, which is not reviewed herein. Cancers are heterogeneous across a wide range of spatial and temporal scales, which results in habitat variations in metabolism, vasculature, oxygenation, and gene expression (Gatenby et al. 2013; Yip and Aerts 2016). Genomic heterogeneity, molecular, and microenvironmental events reflect tumor aggressiveness and therapy response. Tumors with same stage and histology still have unique biological underpinnings such as driver mutations, proteomic profiling, genomic heterogeneity, and/or microenvironments that reflect and can impact aggressiveness and therapy response. Technical advances allow extensive molecular characterization of tumor cells in each individual patient that enables personalized cancer treatment such as targeted treatments of TKIs. Targeted therapies such as Erlotinib and Gefitinib are used to treat patients with positive EGFR mutations (Riely et al. 2006). However, a single arbitrary sample taken from the tumor using needle biopsy can only represent a small subregion of the tumor region, which may result in misleading diagnoses. Meanwhile, many radiogenomic studies have shown that radiomic features can capture the link between the cancer genomics and tumor phenotype.

CT imaging is routinely used in the management of lung cancer patients. Thus, CT radiomics could be used to predict mutational status of clinically actionable mutations using noninvasive information. Such an approach would be particularly beneficial as a clinical predictor in patients with unresectable lung cancer, among patients in whom biopsy is unable to be performed, or to minimize additional biopsy in a molecular test is indeterminate. Additionally, since radiomics can be extracted immediately and in real time, data can be captured longitudinally from an ROI to pinpoint potential phenotypic transformations and activation of alternative pathways to recognize potential acquired resistance to therapies earlier. Rios Velazquez et al. (2017) developed a clinical radiomics signature to differentiate between EGFR and KRAS mutations (AUC = 0.70), the most common somatic mutations in lung adenocarcinomas. Gevaert et al. (2017) used semantic features to predict EGFR and KRAS mutations; however, their models were only able to predict for EGFR mutations accurately (AUC = 0.87). Liu et al. (2016) used CT radiomics to predict EGFR mutation status (AUC = 0.709) in an Asian cohort who had surgically resected peripheral lung adenocarcinomas. Weiss et al. (2014) identified CT texture features that discriminated between KRAS mutant tumors from pan-wild-type tumors (%89.6 accuracy) and Yamamoto et al. (2014) combined clinical covariates and CT-based features to characterize tumors with anaplastic lymphoma kinase (ALK+)-rearranged NSCLC. Yoon et al. (2015) identified clinical covariates and CT and PET radiomics to predict for ALK/ROS1/RET fusion-positive lung adenocarcinoma (Sensitivity = 0.73, Specificity = 0.70) and Zhou et al. (2018) combined semantic CT features with next-generation RNA sequencing data and validated 10 metagenes annotated by functional gene-enrichment analysis that were significantly associated with semantic features. Wu et al. (2016b) identified texture CT features that were associated with NSCLC tumor histology (AUC = 0.72). The summary of these studies can be found in Table 4.

Table 4.

Radiomics and artificial intelligence (AI) studies in radiogenomics

Reference	Imaging modality	Major end point	N (training + test cohorts)	Number of features in the model	Evaluation metric	Analytical method used	Study description	Validation type
Wu et al. 2016b	CT	Tumor histology	350	5	AUC = 0.72, P = 2.3 × 10−7	Machine learning	Finding association between radiomic features and the tumor histologic subtypes	No validation
Rios Velazquez et al. 2017	CT	EGFR mutation	353 + 352	21	AUC = 0.70	Random forest	Discriminating between EGFR+ and EGFR− cases	External validation
Gevaert et al. 2017	CT	EGFR mutation	186	5 semantic features	AUC = 0.87	Multivariate decision tree	Discriminating between EGFR+ and EGFR− cases	No validation
Liu et al. 2016	CT	EGFR mutation	298	4 radiomic + 2 clinical	AUC = 0.709	Logistic regression	Discriminating between EGFR+ and EGFR− cases	No validation
Weiss et al. 2014	CT	KRAS mutation	48	2	Accuracy = 89.6%	Recursive decision tree	Differentiation between KRAS mutation and pan-wild-type	No validation
Yamamoto et al. 2014	CT	ALK mutation	59 + 113	3 radiomic + 1 clinical	Accuracy = 78.8%	Random forest	Discriminating between ALK+ and ALK– cases	External validation
Yoon et al. 2015	PET/CT	ALK/ROS1/RET mutations	539	4 radiomics + 1 qualitative image features + 2 clinical	Sensitivity = 0.73, Specificity = 0.70	Chi-squared test and Student's t-test	Predictors of tumors with ALK, ROS1, or RET fusions	Cross-validation

(CT) Computed tomography, (AUC) area under the curve, (EGFR) Epidermal Growth Factor Receptor.

LIMITATIONS AND RECOMMENDATIONS

Radiomics have shown promise for providing noninvasive biomarkers to predict diagnosis, prognosis, and treatment response, and for longitudinal monitoring of lung cancer treatment. Despite the compelling results from these studies, there are still many limitations that need to be addressed that result in few reproducible findings and potentially spurious results.

There are many reasons that can be attributed to generating nonreproducible results including heterogeneous image acquisition and segmentation and inappropriate use of statistical methods (e.g., overfit models). Standard-of-care image-acquisition parameters have a wide range of parameters that include inter alia: pixel spacing and slice thickness, reconstruction kernel, kVp, washout periods on PET scans, administration of contrast agent, echo time, and repetition time on MRI scans. Intra- and interscanner variabilities affect these parameters that cause radiomic feature distributions to change. To overcome this issue, constant effort needs to be expended to ensure that acquisition and reconstruction protocols are either standardized or correctable. On the other hand, if the radiomic features are extracted from heterogeneous image-acquisition parameters, features that are less sensitive to these parameters should be used and decisions should be made to consider eliminating the sensitive features (Shafiq-Ul-Hassan et al. 2017). This does assume that the reproducible features contain as much diagnostic information as those that may be considered for elimination. Meanwhile, computational radiomic feature calculations involve many critical processing steps that include preprocessing, spatial interpolation, and intensity discretization. The IBSI (Zwanenburg et al. 2018) provides standardized algorithms for radiomic feature calculation and gives consensus and benchmarks on the most common radiomic features and image-processing steps before feature extraction.

Another important factor that affects the reproducibility is the segmentation of the ROI (i.e., tumor parenchyma or peritumoral region). Manual segmentations are particularly time consuming and often lead to intraobserver variations. To overcome this, segmentation of the tumors can be done by semiautomated algorithms that involve minimal user variations such as simple initializations (e.g., seed point), followed by a computer-derived delineation of the ROI. However, to tackle this issue further, segmentation algorithms across institutions have to be standardized to achieve consistent delineations. Nevertheless, many of the features are not reproducible even when acquired within a few minutes using the same image-acquisition parameters (Balagurunathan et al. 2014b) or when the same segmentation algorithms are being used (Kalpathy-Cramer et al. 2016a). Hence, researchers are encouraged to choose reproducible features by using test–retest data sets such as RIDER, and stable features by using multiple segmentation data sets such as Moist run (Kalpathy-Cramer et al. 2016b; Tunali et al. 2019b) or by performing multiple segmentations on a subset of images from their own data set of interest.

Another issue is poor study design that can result in false positive findings (Yip and Aerts 2016). With the potential wide range of hyperparameters such as number of filters, feature categories, and other adjustable parameters, theoretically there are unlimited numbers of radiomic features available for analysis. Studies often analyze a large number of features without accounting for multiple testing errors, which leads to selection bias and false positive results. Chalkidou et al. (2015) suggests using a minimum of 10–15 observations (i.e., patients) per predictor variable (i.e., radiomic feature) to realistically reduce false discovery rates. Another potential application is to correct significant P values for multiple testing using methods such as the Bonferroni–Holm or Benjamini–Hochberg methods (Holm 1979; Benjamini and Hochberg 1995; Bland and Altman 1995). If estimates of predictive performance are conducted from a cohort of a single institution, multiple-fold, repeat cross-validation should be considered to minimize overfitting. Using one or more validation cohorts is the optimal method to validate findings to avoid spurious findings. However, one potential limitation is how to handle differences in demographic and clinical covariates across the training and validations cohorts. As such, another potential approach is to combine patient data sets from different institutions and randomly split all patients into training, test, and validation sets so that the trained model includes heterogeneity. Another suggestion is to assess whether the validated model can be applied on a distinct patient population (e.g., TKI-treated patients only) or reflect a “pan-signature” that can be used across multiple patient populations.

Medical image analysis and radiomics have shown to have the ability to characterize phenotypic and biological underpinning of ROIs (Aerts et al. 2014). Linking radiomics to biology is critical to evolve from mere statistical associations to characterizing tumor biology. Moreover, characterizing the biological underpinnings of radiomics will likely ensure the clinical uptake of these models. However, the link between radiomics and the tumor biology are often unknown due to the lack of availability of data sets that include both radiomics and genomic/biological information. Although studies have linked radiomics to biology (Aerts et al. 2014; Grossmann et al. 2017; Morales et al. 2019; Tunali et al. 2019c), consistent efforts in this domain are needed to expedite the transition of radiomics into clinical practice.

To assess the quality of a radiomics study, a radiomic quality score (RQS) has been developed by Lambin et al. (2017), which evaluates radiomic studies by a series of questions on internal consistency, reproducibility, and clinical applicability. The RQS score makes no claims regarding the significance of the study evaluated; rather, the RQS quantifies the proper study design and scientific utility. We highly suggest researchers to evaluate their studies by RQS and try to maximize their score for more repeatable and quality science in the field of radiomics.

CONCLUSIONS

Radiomics is a noninvasive tool designed for clinical decision support, for both radiologists and oncologists, designed to use routinely available standard-of-care images. Radiomics have been shown to have utility across the lung cancer care continuum including risk prediction, early detection, diagnosis, prognosis, and treatment response. Despite the aforementioned limitations, radiomics and the radiomic community have evolved, in many ways, faster than other 'omics fields because the radiomics field was able to leverage from numerous existing disciplines. Although the radiomic community has embraced the rigorous training, testing, and validation of radiomic models, such models have yet to impact clinical practice. However, with the standardization of medical imaging and image-based features and the use of emerging technologies such as DL, prospective trials to test the clinical utility of radiomics will be emerging in the near future.

COMPETING INTEREST STATEMENT

R.J.G. is an investor and member of the Advisory Board at HealthMyne, Inc., and has Research support from Helix Biopharma. No potential conflicts of interest were disclosed by the other authors.