Abstract
With the ongoing advances in imaging techniques, increasing volumes of anatomical and functional data are being generated as part of the routine clinical workflow. This surge of available imaging data coincides with increasing research in quantitative imaging, particularly in the domain of imaging features. An important and novel approach is radiomics, where high-dimensional image properties are extracted from routine medical images. The fundamental principle of radiomics is the hypothesis that biomedical images contain predictive information, not discernible to the human eye, that can be mined through quantitative image analysis. In this review, a general outline of radiomics and artificial intelligence (AI) will be provided, along with prominent use cases in immunotherapy (e.g. response and adverse event prediction) and targeted therapy (i.e. radiogenomics). While the increased use and development of radiomics and AI in immuno-oncology is highly promising, the technology is still in its early stages, and different challenges still need to be overcome. Nevertheless, novel AI algorithms are being constructed with an ever-increasing scope of applications.
Key words: radiomics, artificial intelligence, radiogenomics, imaging markers, immunotherapy, precision medicine
Highlights
-
•
A radiological image contains data that encodes the morphological profile.
-
•
Radiomic features can be linked to clinical endpoints and tumour biology.
-
•
Combining radiomics with artificial intelligence creates algorithms to support clinical decisions.
-
•
Radiomics can be used to predict response or adverse events to immunotherapy.
-
•
Radio(geno)mics carries great promise for precision medicine and immunotherapy.
Introduction
Radiological imaging plays a vital role in the detection of cancer,1,2 disease staging3, 4, 5 and monitoring treatment response.6,7 With emergence of the RECIST criteria in 2000,8 a standardized set of rules was constructed that enabled clinicians to better categorize tumour response during treatment. The field of oncology, however, is changing rapidly, primarily driven by increasing understanding of the underlying tumour biology. The emergence of novel immunotherapy using immune checkpoint inhibitors and genetically driven targeted therapy has improved the prognosis of several tumour types dramatically.9,10 While the clinical benefits provided to patients by these new biological treatments is undeniable, medical imaging has faced a new set of challenges in attempts to characterize the morphological patterns of response, progression and adverse events to targeted therapy and immunotherapy.11, 12, 13 Classical RECIST criteria, for example, proved suboptimal in the context of immunotherapy as patients treated with checkpoint inhibitors responded very differently from patients who received conventional chemo(radio)therapy. This shortcoming led to the creation of the iRECIST guidelines as a possible solution.14 The iRECIST criteria were undoubtedly steps in the right direction, especially for patients who would be classified as progressive disease under previous RECIST criteria.15 However, considering that only 20%-40% of patients respond to immunotherapy, there is a pressing need for reliable predictive biomarkers.16
In comparison with other diagnostic modalities (e.g. pathological biopsies), imaging offers longitudinal insight into the patient's condition. This feature has brought radiology to the centre of therapeutic response and adverse event monitoring. Alongside the abovementioned iRECIST criteria, a number of studies have examined the use of positron emission technology (PET) imaging to assess response (e.g. PECRIT/PERCIMT criteria) and detect immune-related side effects.17, 18, 19, 20 Outside the scope of this review, it is believed that molecular imaging will prove undoubtedly helpful as the field of oncology moves towards precision medicine.
Significant advances in oncologic imaging have been witnessed in the quantitative aspects of radiology, particularly the domain of imaging features. Radiomic research is currently enjoying a rapid boom to identify non-invasive imaging markers/phenotypes capable of being linked with clinical/biological outcomes. The aim of this review is to provide a broad overview of imaging features, primarily radiomics, and highlight applications relevant to immunotherapy (response prediction) and targeted therapy (radiogenomics).
Imaging features
Imaging features, obtained from either anatomical or functional images, can be broadly divided into qualitative (or semantic) features and quantitative features. Semantic features are acquired by an experienced reader (i.e. radiologist) who scores different tumour characteristics (lesion shape, number of lesions, lesion intensity, etc.) during the evaluation of medical images. Quantitative features are derived by applying advanced mathematical algorithms to the images (as in the case of radiomics).2,21,22 Semantic and quantitative features can be used in statistical or artificial intelligence (AI) models to predict specific clinical endpoints. Individual radiomic features can be combined to form imaging ‘signatures’ or ‘phenotypes’. Table 1 provides an overview of the three major types of imaging feature approaches: semantics, handcrafted radiomics and deep learning radiomics.
Table 1.
A key summary of the advantages and disadvantages of the different features
| Advantages | Disadvantages | |
|---|---|---|
| Semantics |
|
|
| Handcrafted radiomics |
|
|
| Deep learning radiomics |
|
|
Semantic features
Semantic features reflect immediately intuitive and often clinically reported image properties such as lesion shape, boundary type, density characteristics, intensity and others (see Figure 1).2,21,23, 24, 25, 26 These features, or image properties, provide additional information to the radiologist during the detection and diagnosis of disease as well as during the monitoring of response to treatment.23 Semantic traits were proven to be useful in non-invasive discrimination between malignant and benign computed tomography (CT) lung nodules.27 In non-small cell lung cancer (NSCLC), semantic features could differentiate between patients with epidermal growth factor receptor (EGFR), anaplastic lymphoma kinase (ALK) and Kirsten rat sarcoma viral oncogene homologue (KRAS) mutations.28,29 EGFR-mutated nodules tend to be smaller,28,29 and display pleural retraction28 and speculation.29 KRAS-mutated nodules are associated with a rounder shape,28 the presence of nodules in non-tumour lobes28 and the presence of multiple small nodules.29 Larger volume multifocal thoracic lymphadenopathies on CT imaging were found to be indicative of ALK mutations.30
Figure 1.
A schematic representation of different semantic features that can be extracted from medical images.
The different features can be denoted in a feature vector. Feature vectors between the two lesions can differ due to changes in tumour properties.
Semantic features, however, come with their own set of drawbacks and caveats, most chiefly standardization. Different readers can score the same lesion very differently, resulting in a high degree of variability for semantic features. Features derived using such a subjective method may lead the multidisciplinary team to draw incorrect conclusions regarding the diagnosis or treatment response.2 Moreover, there is a learning curve for readers to be able to generate reliable semantic features, highlighting the risk of inter- and intra-observer variability.31 Inter- and intra-observer variability is often cited as a point of concern in such radiological studies.32, 33, 34, 35, 36, 37
Another limitation of semantic features is the need for visual assessment by human readers. The tumour description is inherently limited to what is immediately discernable to human eyes, meaning that high-dimensional, and possibly valuable, imaging features will not be taken into account.32,38, 39, 40, 41 Early attempts were made to automatically predict qualitative features directly from the image; however, this approach has not seen wide adoption, especially with the rise of fully-automated quantitative features.21,42 Although humans excel at pattern recognition, the execution of complex quantitative assessments is often inadequate, making the semantic observations an ideal ‘stepping stone’ for further research to obtain a more quantitative, objective and automatic analysis.43,44
The quantitative field of radiomics thus presents itself as an automatic and objective method to extract these features from medical images, overcoming many of the shortcomings that plagued semantic features.
Radiomic features
The fundamental concept driving the field of radiomics is that biomedical images contain useful predictive information, beyond what is visible to the naked eye, that can be decoded employing quantitative image analysis.24,45 Routine medical images are converted into high-dimensional mineable data, in the form of radiomic features, using mathematical formulae that describe image properties quantitatively.38,46, 47, 48, 49, 50, 51 Radiomic feature extraction methods yield a far greater number of features compared with human-derived semantic features, with only a fraction of the effort.
As almost all patients undergo scans for diagnosis or treatment response assessment, a potentially vast database of sources exists for both retrospective and prospective research. From these digital images, a large number of quantitative features can be extracted and analysed for hypothesis generation or experimentation.52 Radiomic data can be used alone or in combination with semantic features to improve the understanding of underlying biological, therapeutic and post-therapeutic changes, and treatment-related adverse events.40
The introduction of radiomics into the imaging analysis workflow adds a promising new tool to better sub-stratify patients and customize treatment plans. Quantitative analysis of images can help to differentiate different types of cancer, predict disease outcomes and determine response to treatment.53, 54, 55, 56, 57, 58
Based on the method of extraction, radiomic features can be divided into two main approaches: handcrafted (or classical) radiomics and deep learning radiomics (see Figure 2).22,24,59
Figure 2.
An overview of the two main radiomic pipelines.
The top path shows classical radiomics, with tumour delineation and manual feature engineering and extraction. After feature extraction, statistical models or machine learning can be used to link the features to the endpoint. The lower path shows deep learning radiomics, where the deep learning network extracts features without human interference to link them to the desired endpoint.
Handcrafted features are derived from pre-engineered formulas based on visualized aspects of the image.22,24,59 The handcrafted features are often based on intensity histograms, shape attributes and textures, encoding morphological, phenotypic characteristics.39 Before handcrafted features can be computed, regions of interest (ROIs) within the radiological image must first be delineated (segmented). The combination of delineation followed by handcrafted feature extraction is a defining characteristic of classical radiomics.24 The extracted features can then be used in either statistical or machine learning models. Delta radiomics, a specific subgroup within classical radiomics, compares (and ultimately subtracts) pre- and post-treatment radiomic features (i.e. feature changes) to generate a new set of ‘delta radiomic’ features meant to reflect treatment-induced morphological changes.60,61
As a human-dependent process, manual delineation/segmentation is vulnerable to inter- and intra-observer variability.62 Furthermore, because handcrafted features are extracted mathematically by human-defined and curated quantitative formulas, bias can occur.63,64 New AI-driven methodologies have emerged to mitigate many of the perceived pitfalls of classical radiomics.
The second methodological approach to medical image analysis is deep learning, a novel technique that is considered a potential ‘game-changer’ for the field of radiology. Deep learning is a broad term used to describe a collection of neural networks which generate their own features and perform classification without human involvement.24 It has been shown that deep learning has greater accuracy for specific tasks compared with traditional computer vision techniques.65
While an in-depth explanation of deep neural networks is beyond the scope of this review, a small introduction can be given. Many different types of deep learning networks exist, but all are based on the same premise: an artificial neuron. Artificial neurons model our biological neurons and form the basic building block of neural networks. Each neural network contains multiple neurons arranged in different layers. With each neuron obtaining a different value, combinations of these neurons form features such as lines, shapes, patterns and anatomical structures. To obtain the underlying feature extraction and classification parameters, the network needs to learn; this task is commonly referred to as ‘training’. During training, backpropagation, a phenomenon where the neurons are tuned to learn the data, occurs. Backpropagation ensures that the neuron combinations form different features.
In the field of medical image analysis, the convolutional neural network (CNN) is the class of deep learning networks employed most often, as their feature design and extraction are free from human interference. CNNs can generate quantitative features several orders of magnitude greater than the methods applied during semantic feature extraction.23 Additionally, many deep learning approaches do not require expert delineation/segmentation, further mitigating human bias. With deep learning, feature extraction, selection and classification all occur end-to-end automatically and impartially from human interference.
Radiomic features aim to quantify the morphological phenotype, but they cannot be used for prediction on their own. Rather, the features are combined together, and subsequently, specific predictive radiomic signatures are identified within statistical models/machine learning algorithms. Statistical models, such as univariate or multivariate analysis, are more commonly used in early proof-of-concept studies, where the objective is merely to find mathematical associations between the radiomic features and the relevant clinical endpoint. Machine learning algorithms provide the ability to construct predictive systems that learn directly from the data. Machine learning and deep learning algorithms provide a set of advantages over classic statistical modelling, namely the fact that there are fewer mathematical assumptions and minimal human involvement is required. AI algorithms may be classifiers that try to predict binary/categorical outcomes (KRAS-mutated/wild-type), regression models that give outputs as a continuous number (i.e. body mass index = 15.2), or survival analysis models designed specifically to predict the probability of survival. When designing the AI aspect of a radiomic study, care should be taken when determining which type of machine learning/deep learning algorithm would be optimal for the data and endpoint in question.
Applications of radiomics and AI
Radiomics for prediction of response and adverse events in immunotherapy
Immunotherapy, as an emerging pillar in cancer treatment, has been researched extensively at the basic, translational and clinical levels.66, 67, 68 To date, publications on deep learning/radiomics in the context of immunotherapy are limited. However, as the possibility for synergy between non-invasive imaging markers and immunotherapy gains further recognition, it is anticipated that this will become a major line of medical imaging research.
Broadly classified radiomic research in immunotherapy has focused on either prediction of immune-related adverse events or response, using either RECIST or survival metrics as the endpoint.
With the aim of predicting the immunotherapeutic response, Trebeschi et al. extracted radiomic features from CT images of patients with metastatic NSCLC and melanoma who had received anti-PD1 immune checkpoint blockade. Lesions that displayed a more heterogeneous morphologic imaging phenotype with non-uniform density patterns and compact borders were associated with a better response to immunotherapy.69 Imaging markers linked to vessel tortuosity were also used to differentiate between responders and non-responders among patients with NSCLC after nivolumab treatment [area under curve (AUC) = 0.79].70 Subsequent studies have shown similar associations between specific radiomic features and post-immune checkpoint inhibitor response.71, 72, 73, 74, 75, 76 Tunali et al. demonstrated that combining radiomic features with clinical data enhanced the predictive power of the radiomic algorithm (AUC = 0.80-0.87 with radiomic features and AUC = 0.77-0.78 without radiomic features).77 A combination of RNA sequence data and CT-derived radiomic features was also able to identify immune infiltration in solid tumours.78 CT-derived radiomic features combined with delta radiomics was also predictive of an early response to immunotherapy (AUC = 0.857).79 Overall survival and progression-free survival were also linked to tumour skewness in pre-treatment CT scans of patients with metastatic melanoma treated with pembrolizumab.80 Magnetic resonance imaging (MRI) radiomic features showed an association with overall survival in patients with melanoma brain metastases receiving immune checkpoint inhibitors.81
Some radiomic projects also focused on adverse events seen in cohorts treated with immunotherapy, such as pneumonitis,82 sarcoid-like granulomatous disease of the lung83 and, most notably, pseudoprogression.84, 85, 86, 87, 88 Distinguishing pseudo-progression from genuine progressive disease is an essential yet challenging task, often requiring serial imaging to make the distinction. Elshafeey et al. used radiomic features paired with a support vector machine to classify pseudo-progression from progressive disease in patients with glioblastoma (AUC = 0.89).84 Similar results were obtained by Hu et al., whose work used imaging features derived from eight different MRI sequences (AUC = 0.94).85 The combination of PET/CT radiomic features with blood parameters was associated with early differentiation of pseudo-progression in patients with metastatic melanoma (AUC = 0.82).89
For deep learning, few papers that differentiate pseudo-progression from progressive disease have been published. An intriguing approach is that of Li et al., who combined two novel deep learning techniques (a generative adversarial network and AlexNet) (AUC = 0.92).86 Other research in the prediction of pseudo-progression with deep learning achieved comparable results (AUC = 0.83 and 0.80).87,88
Radiogenomics for non-invasive insight into tumour biology for precision medicine
Radiogenomics is the field of research where imaging phenotypes are linked to genetic characteristics (such as gene expression and gene mutation).90,91 The goal is to associate imaging features with previously determined gene expression profiles. In the past years, the term ‘radiogenomics’ has expanded to encompass more than ‘just genomics’. Radiogenomics currently aims to link imaging markers with other biological parameters, such as proteomics, metabolomics and microenvironmental data.22
While still a field of research in its infancy, radiogenomics has the potential to add significant value to the future clinical workup of a patient. In an era where targeted therapies are being used increasingly, knowing the genetic profile of the tumour can help in the formulation of a customized treatment plan. Radiogenomics offers the ability to profile the tumour non-invasively and mitigate several challenges facing traditional biopsy-based approaches. In radiogenomics, the full tumour burden could be analysed non-invasively and cost-effectively, accounting for both intra- and inter-lesional heterogeneity.92,93 Radiogenomics is also able to analyse patient images at several different time points (e.g. longitudinal imaging of the genomic landscape) and tumour sites; this is not always possible with traditional biopsy-based diagnostic modalities.94
Radiogenomics has been applied in many different tumour types. In the field of breast cancer, Yamamoto et al. found correlation between quantitative features extracted from dynamic contrast-enhanced MRI scans and early metastasis and metastasis-free survival biomarkers in breast cancer.95 Additionally, several studies have used non-invasive imaging features to predict the molecular subtype of the tumour (i.e. luminal A96 and luminal B93). Luminal A subtype breast cancer is less aggressive and has a better response to therapy than luminal B subtype breast cancer.93 Being able to gain insight into tumour biology non-invasively can impact the treatment plan significantly.
In lung cancer, radiomic research discovered a relation between specific hypoxia markers (vascular endothelial growth factor and EGFR) and quantitative texture features derived from cholangiocarcinoma CT scans. The presence of this imaging phenotype could then be used to indicate a hypoxic microenvironment non-invasively.94 Similarly, radiomic features have been used to predict the presence of many clinically relevant mutations in NSCLC, namely EGFR,97, 98, 99, 100 KRAS101, 102, 103 and ALK.104
Challenges in radiomics
Despite its promising results, the field of radiomics also faces several challenges that need to be overcome.
Generalizability and reproducibility
A requirement for the implementation of biomarkers in the clinic, imaging or otherwise, is that they are independent, informative and reproducible.105 Imaging biomarkers that meet these criteria are often derived from large heterogeneous datasets and validated on independent test sets. Reproducibility, however, may be hampered due to differences in radiomic values resulting from differences in scanning equipment,106 image acquisition protocols,107 reconstruction algorithms,108,109 image pre-processing110 and feature extraction methods.62 Essentially, this results in a situation where, based on the healthcare centre and scanning equipment, different radiomic signatures can be found to be prognostic/predictive for the same endpoint.111 Standardization of scan protocols between different institutions could aid in the generalizability and reproducibility of radiomic features/signatures.112,113 Initiatives such as the Quantitative Imaging Network initiative have started to address this concern by putting forward a set of standards and best practices for image acquisition and feature extraction.114 Lambin et al. have even proposed a radiomic quality score where specific practices within the methodology of an article would be rewarded or penalized.115 For example, a study with external validation, biological correlation and multiple segmentations per image would receive additional points for its quality score.
Radiomic workflow
The radiomic pipeline can be divided into four main steps: image acquisition, ROI delineation/segmentation, feature extraction/selection, and statistical analysis/AI training. Errors can occur at any point in this pipeline. The build-up of these errors is generally termed ‘propagation of the error’.
As a human-dependent step, ROI delineation/segmentation presents many challenges, most notably inter- and intra-observer variation.48 Any variation in the radiomic workflow can result in massive changes to the radiomic feature values.116 Kocak et al. demonstrated that each step in the pipeline was prone to a change of segmentation margin of 2 mm; a margin that can be of high impact and should be taken into account.117
Additionally, as expert radiologists are required for accurate delineations, time and availability constraints can often serve as ‘bottlenecks’ for data inclusion in radiomic studies.118,119 Automatic AI-driven ROI delineation approaches have therefore become a growing field of research within radiology, but the technology remains in its infancy as it suffers from its own set of caveats.120
Future directions
Notwithstanding the challenges mentioned above, the field of radiomics has a bright future, particularly with the research currently being carried out in the following domains.
The rise of deep learning
As indicated in the Introduction, classical radiomics is characterized by a mandatory tumour delineation step, followed by a handcrafted feature extraction step. Delineation is vulnerable to inter- and intra-observer variability as well as human bias, limiting reproducibility. AI methods, particularly deep learning, can mitigate many of the limitations in classical radiomics. Alongside the potential for automatic ROI segmentation, CNNs learn image features internally and link these features to the desired outcomes, all without human involvement. The use of automatic AI methods removes many of the potential sources of human-induced bias from the radiomic pipeline.
Less data-hungry AI
Generalizable deep learning models typically require a large number of well-annotated samples as training data. AI training datasets used by Google, for example, consist of hundreds of thousands of images.121 One of the significant challenges in medical AI, especially in the case of patients treated with immunotherapy, is the limited amount of publically available high-quality well-annotated medical/imaging data.122,123 Current AI approaches, especially deep learning, achieve better predictive performance when trained on large amounts of well-annotated data. Obtaining this high-quality medical data remains challenging, as evidenced by the fact that many of the articles highlighted in this review had low sample sizes (N < 100).
Data augmentation can be used to artificially increase the amount of usable training data to overcome this limitation. Being able to use unlabelled data would unlock even more of the hospital's data for AI development. Towards this, unsupervised neural networks have gained significant interest in medical AI research. In unsupervised learning, the network learns features in the data without having access to the ground truth.
Model explainability techniques: understanding the ‘black box’
Something that raises concern in all of society is ‘fear of the AI uprising’. This sentiment, which is not well founded, reflects the concern that society has with AI. A significant part of this fear stems from the perceived inexplicability of the predictions, regardless of the accuracy (i.e. the ‘black box’). In the context of medicine, clinicians will not accept and adopt technology that seems to behave without any fundamental basis. However, this fear has already existed for more than 30 years, and new methods have been developed to shed light on the inner workings of neural networks.
One approach that has been gaining traction in AI radiological research is saliency mapping. A saliency map is an intensity map that represents the individual importance of each individual pixel/voxel in the generation of the model's prediction. In essence, saliency maps allow for the creation of heat maps that can be overlaid on the anatomical image to indicate which parts of the image were deemed important/relevant by the algorithm for its prediction.
Tumour delineation
Within the radiomic pipeline, tumour segmentation is considered a ‘bottleneck’ as reliable segmentation is time-consuming and requires experienced radiologists.124 AI has two approaches to mitigate this limitation: automatic segmentation algorithms and non-segmentation-dependent deep learning.
In automatic segmentation, algorithms are created to automatically detect ROIs (based on prior training) and generate delineations/segmentations. Subsequently, features can be extracted from the automatically segmented region. As there is minimal human involvement in this approach, inter- and intra-observer variability can be minimized. One caveat is that in order to create an automatic segmentation algorithm, the AI model must first be trained using human-generated segmentations. It is advised that great care should be paid to the training data to prevent any bias in the data that would lead to a bias in the model.125 Another mitigation strategy for the segmentation ‘bottleneck’ is the use of CNNs to detect either ‘global’ imaging markers (on the entire non-delineated image instead of an ROI) or have readers generate broad ROIs, termed ‘bounding boxes’. With bounding boxes, the radiologist does not need to delineate the tumour in detail, but can instead indicate the general area of the tumour, for example, in the image.
Multimodal AI models/integrated healthcare systems
Within the routine clinical workflow, patients undergo a wide array of tests and analysis ranging from blood tests to different types of imaging to organ-specific function tests. In essence, from the moment of admission to the moment of discharge, a single patient can generate large volumes of clinical, pathological, radiological and genetic data (among others), all encoding potentially complementary information. AI methods allow for the integration of these different data types to drastically enhance predictive performance. It is envisaged that, in the future, integration systems will be established in hospitals where different diagnostic data are combined (see Figure 3).126, 127, 128 The resulting clinical decision support system will then aid the multidisciplinary tumour boards to design the best possible management plan on an individual patient basis.
Figure 3.
A schematic representation of multimodal artificial intelligence (AI) models.
With the increase in available data and knowledge, a combination of different diagnostic data into one integrated diagnostic system can improve AI model outcomes. The outcomes of the integrated diagnostic models can be discussed in multidisciplinary tumour boards to optimize the treatment management plan for patients.
Conclusions
Radiomics is a promising research field with future potential for implementation in the clinical workflow. Whether it is in the context of prognostication, response prediction or tumour biology assessment, radiomic features have been studied extensively with a large number of proof-of-concept studies. Notwithstanding the challenges that need to be addressed, significant progress has been made, especially with the implementation of novel AI methods. It is believed that imaging markers, in general, and radiomics, in particular, will find their place within an integrated diagnostics system that harnesses multimodal patient data.
Acknowledgments
Funding
This research did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors.
Disclosure
The authors have declared no conflicts of interest.
References
- 1.Fass L. Imaging and cancer: a review. Mol Oncol. 2008;2(2):115–152. doi: 10.1016/j.molonc.2008.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kisilev P., Walach E., Hashoul S., Barkan E., Ophir B., Alpert S. Proceedings of the British Machine Vision Conference (BMVC) BMVA Press; London: 2016. Semantic description of medical image findings: structured learning approach; pp. 171.1–171.11. [Google Scholar]
- 3.Bergan, RC, Raza A, Mundle SD, et al. Imaging in Oncology. (Blake MA, Kalra MK, eds.); 2008.
- 4.Altun E, Jewells VS, Fielding JR. Imaging in Oncology. 2nd ed. (Husband J, Reznek RH, eds.). Taylor and Francis Group; 2015.
- 5.Reilly R.M. Wiley; Hoboken, NJ: 2019. Medical Imaging for Health Professionals: Technologies and Clinical Applications. [Google Scholar]
- 6.Thoeny H.C., Ross B.D. Predicting and monitoring cancer treatment response with DW-MRI. J Magn Reson Imaging. 2010;32(1):2–16. doi: 10.1002/jmri.22167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Eisenhauer E.A., Therasse P., Bogaerts J., et al. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1) Eur J Cancer. 2009;45(2):228–247. doi: 10.1016/j.ejca.2008.10.026. [DOI] [PubMed] [Google Scholar]
- 8.Therasse P., Arbuck S.G., Eisenhauer E.A., et al. New guidelines to evaluate the response to treatment in solid tumors. J Natl Cancer Inst. 2000;92(3):205–216. doi: 10.1093/jnci/92.3.205. [DOI] [PubMed] [Google Scholar]
- 9.Esfahani K., Roudaia L., Buhlaiga N., Del Rincon S.V., Papneja N., Miller W.H. A review of cancer immunotherapy: from the past, to the present, to the future. Curr Oncol. 2020;27(S2):87–97. doi: 10.3747/co.27.5223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Sambi M., Bagheri L., Szewczuk M.R. Current challenges in cancer immunotherapy: multimodal approaches to improve efficacy and patient response rates. J Oncol. 2019;2019:4508794. doi: 10.1155/2019/4508794. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dromain C., Beigelman C., Pozzessere C., Duran R., Digklia A. Imaging of tumour response to immunotherapy. Eur Radiol Exp. 2020;4(1):1–15. doi: 10.1186/s41747-019-0134-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Erinjeri J.P., Fine G.C., Adema G.J., et al. Immunotherapy and the interventional oncologist: challenges and opportunities-a society of interventional oncology white paper. Radiology. 2019;292(1):25–34. doi: 10.1148/radiol.2019182326. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Galldiks N., Kocher M., Ceccon G., et al. Imaging challenges of immunotherapy and targeted therapy in patients with brain metastases: response, progression, and pseudoprogression. Neuro Oncol. 2020;22:17–30. doi: 10.1093/neuonc/noz147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Seymour L., Bogaerts J., Perrone A., et al. iRECIST: guidelines for response criteria for use in trials testing immunotherapeutics. Lancet Oncol. 2017;18(3):e143–e152. doi: 10.1016/S1470-2045(17)30074-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Park H.J., Kim G.H., Kim K.W., et al. Comparison of RECIST 1.1 and irecist in patients treated with immune checkpoint inhibitors: a systematic review and meta-analysis. Cancers (Basel) 2021;13(1):1–14. doi: 10.3390/cancers13010120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Sharma P., Hu-Lieskovan S., Wargo J.A., Ribas A. Primary, adaptive, and acquired resistance to cancer immunotherapy. Cell. 2017;168(4):707–723. doi: 10.1016/j.cell.2017.01.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Cho S.Y., Lipson E.J., Im H.-J., et al. Prediction of response to immune checkpoint inhibitor therapy using early-time-point 18F-FDG PET/CT imaging in patients with advanced melanoma. J Nucl Med. 2017;58(9):1421–1428. doi: 10.2967/jnumed.116.188839. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Anwar H., Sachpekidis C., Winkler J., et al. Absolute number of new lesions on 18F-FDG PET/CT is more predictive of clinical response than SUV changes in metastatic melanoma patients receiving ipilimumab. Eur J Nucl Med Mol Imaging. 2018;45(3):376–383. doi: 10.1007/s00259-017-3870-6. [DOI] [PubMed] [Google Scholar]
- 19.Sachpekidis C., Anwar H., Winkler J., et al. The role of interim 18F-FDG PET/CT in prediction of response to ipilimumab treatment in metastatic melanoma. Eur J Nucl Med Mol Imaging. 2018;45(8):1289–1296. doi: 10.1007/s00259-018-3972-9. [DOI] [PubMed] [Google Scholar]
- 20.Unterrainer M., Ruzicka M., Fabritius M.P., et al. PET/CT imaging for tumour response assessment to immunotherapy: current status and future directions. Eur Radiol Exp. 2020;4(1):63. doi: 10.1186/s41747-020-00190-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Yip S.S.F., Liu Y., Parmar C., et al. Associations between radiologist-defined semantic and automatically computed radiomic features in non-small cell lung cancer. Sci Rep. 2017;7:3519. doi: 10.1038/s41598-017-02425-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bodalal Z., Trebeschi S., Nguyen-Kim T.D.L., Schats W., Beets-Tan R. Radiogenomics: bridging imaging and genomics. Abdom Radiol. 2019;44(6):1960–1984. doi: 10.1007/s00261-019-02028-w. [DOI] [PubMed] [Google Scholar]
- 23.Rahul P., Schabath M., Balagurunathan Y., et al. Explaining deep features using radiologist-defined semantic features and traditional quantitative features. Tomography. 2019;5(1):192–200. doi: 10.18383/j.tom.2018.00034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Rogers W., Thulasi Seetha S., Refaee T.A.G., et al. Radiomics: from qualitative to quantitative imaging. Br J Radiol. 2020;93(1108):20190948. doi: 10.1259/bjr.20190948. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Opulencia P., Channin D.S., Raicu D.S., Furst J.D. Mapping LIDC, RadLexTM, and lung nodule image features. J Digit Imaging. 2011;24(2):256–270. doi: 10.1007/s10278-010-9285-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Langlotz C.P. RadLex: a new method for indexing online educational materials. Radiographics. 2006;26(6):1595–1597. doi: 10.1148/rg.266065168. [DOI] [PubMed] [Google Scholar]
- 27.Liu Y., Balagurunathan Y., Atwater T., et al. Radiological image traits predictive of cancer status in pulmonary nodules. Clin Cancer Res. 2017;23(6):1442–1449. doi: 10.1158/1078-0432.CCR-15-3102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Rizzo S., Petrella F., Buscarino V., et al. CT radiogenomic characterization of EGFR, K-RAS, and ALK mutations in non-small cell lung cancer. Eur Radiol. 2016;26(1):32–42. doi: 10.1007/s00330-015-3814-0. [DOI] [PubMed] [Google Scholar]
- 29.Lv J., Zhang H., Ma J., et al. Comparison of CT radiogenomic and clinical characteristics between EGFR and KRAS mutations in lung adenocarcinomas. Clin Radiol. 2018;73(6):590.e1–590.e8. doi: 10.1016/j.crad.2018.01.009. [DOI] [PubMed] [Google Scholar]
- 30.Halpenny D.F., Riely G.J., Hayes S., et al. Are there imaging characteristics associated with lung adenocarcinomas harboring ALK rearrangements? Lung Cancer. 2014;86(2):190–194. doi: 10.1016/j.lungcan.2014.09.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Li Q., Balagurunathan Y., Liu Y., et al. Comparison between radiological semantic features and lung-RADS in predicting malignancy of screen-detected lung nodules in the National Lung Screening Trial. Clin Lung Cancer. 2018;19(2):148–156.e3. doi: 10.1016/j.cllc.2017.10.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Wu W., Pierce L.A., Zhang Y., et al. Comparison of prediction models with radiological semantic features and radiomics in lung cancer diagnosis of the pulmonary nodules: a case-control study. Eur Radiol. 2019;29(11):6100–6108. doi: 10.1007/s00330-019-06213-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Charbonnier J.P., Chung K., Scholten E.T., et al. Automatic segmentation of the solid core and enclosed vessels in subsolid pulmonary nodules. Sci Rep. 2018;8(646):1–11. doi: 10.1038/s41598-017-19101-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Wetzel S.G., Cha S., Johnson G., et al. Relative cerebral blood volume measurements in intracranial mass lesions: interobserver and intraobserver reproducibility study. Radiology. 2002;224(3):797–803. doi: 10.1148/radiol.2243011014. [DOI] [PubMed] [Google Scholar]
- 35.Van Riel S.J., Sánchez C.I., Bankier A.A., et al. Observer variability for classification of pulmonary nodules on low-dose CT images and its effect on nodule management. Radiology. 2015;277(3):863–871. doi: 10.1148/radiol.2015142700. [DOI] [PubMed] [Google Scholar]
- 36.Chen S., Qin J., Ji X., et al. Automatic scoring of multiple semantic attributes with multi-task feature leverage: a study on pulmonary nodules in CT images. IEEE Trans Med Imaging. 2017;36(3):802–814. doi: 10.1109/TMI.2016.2629462. [DOI] [PubMed] [Google Scholar]
- 37.Ridge C.A., Yildirim A., Boiselle P.M., et al. Differentiating between subsolid and solid pulmonary nodules at CT: inter- and intraobserver agreement between experienced thoracic radiologists. Radiology. 2016;278(3):888–896. doi: 10.1148/radiol.2015150714. [DOI] [PubMed] [Google Scholar]
- 38.Sun K.Y., Hu H.T., Chen S.L., et al. CT-based radiomics scores predict response to neoadjuvant chemotherapy and survival in patients with gastric cancer. BMC Cancer. 2020;20(1):468. doi: 10.1186/s12885-020-06970-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Aerts H.J.W.L., Velazquez E.R., Leijenaar R.T.H., et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014;5:4406. doi: 10.1038/ncomms5006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Cook G.J.R., Siddique M., Taylor B.P., Yip C., Chicklore S., Goh V. Radiomics in PET: principles and applications. Clin Transl Imaging. 2014;2(3):269–276. [Google Scholar]
- 41.Chicklore S., Goh V., Siddique M., Roy A., Marsden P.K., Cook G.J.R. Quantifying tumour heterogeneity in 18F-FDG PET/CT imaging by texture analysis. Eur J Nucl Med Mol Imaging. 2013;40(1):133–140. doi: 10.1007/s00259-012-2247-0. [DOI] [PubMed] [Google Scholar]
- 42.Depeursinge A., Kurtz C., Beaulieu C., Napel S., Rubin D. Predicting visual semantic descriptive terms from radiological image data: preliminary results with liver lesions in CT. IEEE Trans Med Imaging. 2014;33(8):1669–1676. doi: 10.1109/TMI.2014.2321347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Miller G.A. The magical number two, plus or minus one: some limits on our capacity for processing musical information. Psychol Rev. 1994;101(2):343–352. doi: 10.1037/0033-295x.101.2.343. [DOI] [PubMed] [Google Scholar]
- 44.Wang Y.-X.J., Ng C.K. The impact of quantitative imaging in medicine and surgery: charting our course for the future. Quant Imaging Med Surg. 2011;1(1):1–3. doi: 10.3978/j.issn.2223-4292.2011.09.01. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Hectors S.J., Lewis S., Besa C., et al. MRI radiomics features predict immuno-oncological characteristics of hepatocellular carcinoma. Eur Radiol. 2020;30(7):3759–3769. doi: 10.1007/s00330-020-06675-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.van Griethuysen J.J., Fedorov A., Parmar C., et al. Computational radiomics system to decode the radiographic phenotype. Cancer Res. 2017;77(21):e104–e107. doi: 10.1158/0008-5472.CAN-17-0339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Shaver M.M., Kohanteb P.A., Chiou C., et al. Optimizing neuro-oncology imaging: a review of deep learning approaches for glioma imaging. Cancers (Basel) 2019;11(6):1–14. doi: 10.3390/cancers11060829. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Rizzo S., Botta F., Raimondi S., et al. Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp. 2018;2(1):36. doi: 10.1186/s41747-018-0068-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Schweitzer A.D., Chiang G.C., Ivanidze J., Baradaran H., Young R.J., Zimmerman R.D. Computer-extracted texture features to distinguish cerebral radionecrosis from recurrent brain tumors on multiparametric MRI: a feasibility study. Am J Neuroradiol. 2017;38(3):2231–2236. doi: 10.3174/ajnr.A5019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Peng L., Parekh V., Huang P., et al. Distinguishing true progression from radionecrosis after stereotactic radiation therapy for brain metastases with machine learning and radiomics. Int J Radiat Oncol Biol Phys. 2018;102(4):1236–1243. doi: 10.1016/j.ijrobp.2018.05.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Zwanenburg A., Vallières M., Abdalah M.A., et al. The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. 2020;295(2):328–338. doi: 10.1148/radiol.2020191145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Gillies R.J., Kinahan P.E., Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2016;278(2):563–577. doi: 10.1148/radiol.2015151169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Song S.H., Park H., Lee G., et al. Imaging phenotyping using radiomics to predict micropapillary pattern within lung adenocarcinoma. J Thorac Oncol. 2017;12(4):624–632. doi: 10.1016/j.jtho.2016.11.2230. [DOI] [PubMed] [Google Scholar]
- 54.Coroller T.P., Agrawal V., Huynh E., et al. Radiomic-based pathological response prediction from primary tumors and lymph nodes in NSCLC. J Thorac Oncol. 2017;12(3):467–476. doi: 10.1016/j.jtho.2016.11.2226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Coroller T.P., Grossmann P., Hou Y., et al. CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiother Oncol. 2015;114(3):345–350. doi: 10.1016/j.radonc.2015.02.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Huang Y., Liu Z., He L., et al. Radiomics signature: a potential biomarker for the prediction of disease-free survival in early-stage (I or II) non-small cell lung cancer. Radiology. 2016;281(3):947–957. doi: 10.1148/radiol.2016152234. [DOI] [PubMed] [Google Scholar]
- 57.Huang Y.Q., Liang C.H., He L., et al. Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J Clin Oncol. 2016;34(18):2157–2164. doi: 10.1200/JCO.2015.65.9128. [DOI] [PubMed] [Google Scholar]
- 58.Wibmer A., Hricak H., Gondo T., et al. Haralick texture analysis of prostate MRI: utility for differentiating non-cancerous prostate from prostate cancer and differentiating prostate cancers with different Gleason scores. Eur Radiol. 2016;25(10):2840–2850. doi: 10.1007/s00330-015-3701-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Afshar P., Mohammadi A., Plataniotis K.N., Oikonomou A., Benali H. From handcrafted to deep-learning-based cancer radiomics: challenges and opportunities. IEEE Signal Process Mag. 2019;36(4):132–160. [Google Scholar]
- 60.Mazzei M.A., Nardone V., Di Giacomo L., et al. The role of delta radiomics in gastric cancer. Quant Imaging Med Surg. 2018;8(7):719–721. doi: 10.21037/qims.2018.07.08. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Fave X., Zhang L., Yang J., et al. Delta-radiomics features for the prediction of patient outcomes in non-small cell lung cancer. Sci Rep. 2017;7(1):1–11. doi: 10.1038/s41598-017-00665-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Traverso A., Wee L., Dekker A., Gillies R. Repeatability and reproducibility of radiomic features: a systematic review. Int J Radiat Oncol Biol Phys. 2018;102(4):1143–1158. doi: 10.1016/j.ijrobp.2018.05.053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Chalkidou A., O'Doherty M.J., Marsden P.K. False discovery rates in PET and CT studies with texture features: a systematic review. PLoS One. 2015;10(5):1–18. doi: 10.1371/journal.pone.0124165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Hosny A., Aerts H.J., Mak R. Handcrafted versus deep learning radiomics for prediction of cancer therapy response. Lancet Digit Health. 2019;1:e106–e107. doi: 10.1016/S2589-7500(19)30062-7. [DOI] [PubMed] [Google Scholar]
- 65.O'Mahony N., Campbell S., Carvalho A., et al. Deep learning vs. traditional computer vision. Adv Intell Syst Comput. 2020;943:128–144. [Google Scholar]
- 66.Ribas A., Wolchok J.D. Cancer immunotherapy using checkpoint blockade. Science. 2018;359(6382):1350–1355. doi: 10.1126/science.aar4060. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Meiliana A., Wijaya A., Sriwijaya U. Cancer immunotherapy: a review. Indones Biomed J. 2016;8(1):1–20. [Google Scholar]
- 68.Zhang H., Chen J. Current status and future directions of cancer immunotherapy. J Cancer. 2018;9(10):1773–1781. doi: 10.7150/jca.24577. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Trebeschi S., Drago S.G., Birkbak N.J., et al. Predicting response to cancer immunotherapy using noninvasive radiomic biomarkers. Ann Oncol. 2019;30(6):998–1004. doi: 10.1093/annonc/mdz108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Velcheti V., Alilou M., Khunger M., Thawani R., Madabhushi A. Changes in computer extracted features of vessel tortuosity on CT scans post-treatment in responders compared to non-responders for non-small cell lung cancer on immunotherapy. J Clin Oncol. 2017;35(15_suppl):11518. [Google Scholar]
- 71.Nardone V., Tini P., Pastina P., et al. Radiomics predicts survival of patients with advanced non-small cell lung cancer undergoing PD-1 blockade using Nivolumab. Oncol Lett. 2020;19(2):1559–1566. doi: 10.3892/ol.2019.11220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Alilou M., Bera K., Vaidya P., et al. Quantitative vessel tortuosity radiomics on baseline non-contrast lung CT predict response to immunotherapy and are prognostic of overall survival. Med Imaging. 2019;109501F [Google Scholar]
- 73.Khorrami M., Prasanna P., Gupta A., et al. Changes in CT radiomic features associated with lymphocyte distribution predict overall survival and response to immunotherapy in non-small cell lung cancer. Cancer Immunol Res. 2020;8(1):108–119. doi: 10.1158/2326-6066.CIR-19-0476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Mu W., Tunali I., Gray J.E., Qi J., Schabath M.B., Gillies R.J. Radiomics of 18F-FDG PET/CT images predicts clinical benefit of advanced NSCLC patients to checkpoint blockade immunotherapy. Eur J Nucl Med Mol Imaging. 2020;47(5):1168–1182. doi: 10.1007/s00259-019-04625-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.He B., Dong D., She Y., et al. Predicting response to immunotherapy in advanced non-small-cell lung cancer using tumor mutational burden radiomic biomarker. J Immunother Cancer. 2020;8(e000550):1–10. doi: 10.1136/jitc-2020-000550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Rundo F., Spampinato C., Banna G.L., Conoci S. Advanced deep learning embedded motion radiomics pipeline for predicting anti-PD-1/PD-L1 immunotherapy response in the treatment of bladder cancer: preliminary results. Electronics. 2019;8(10):1134. [Google Scholar]
- 77.Tunali I., Gray J.E., Qi J., et al. Novel clinical and radiomic predictors of rapid disease progression phenotypes among lung cancer patients treated with immunotherapy: an early report. Lung Cancer. 2019;129:75–79. doi: 10.1016/j.lungcan.2019.01.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Sun R., Limkin E.J., Vakalopoulou M., et al. A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study. Lancet Oncol. 2018;19(9):1180–1191. doi: 10.1016/S1470-2045(18)30413-3. [DOI] [PubMed] [Google Scholar]
- 79.Wang Z., Mao L., Zhou Z., Si L., Zhu H., Chen X. Pilot study of CT-based radiomics model for early evaluation of response to immunotherapy in patients with metastatic melanoma. Front Oncol. 2020;10(1524):1–8. doi: 10.3389/fonc.2020.01524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Durot C., Mulé S., Soyer P., Marchal A., Grange F., Hoeffel C. Metastatic melanoma: pretreatment contrast-enhanced CT texture parameters as predictive biomarkers of survival in patients treated with pembrolizumab. Eur Radiol. 2019;29(6):3183–3191. doi: 10.1007/s00330-018-5933-x. [DOI] [PubMed] [Google Scholar]
- 81.Bhatia A., Birger M., Veeraraghavan H., et al. MRI radiomic features are associated with survival in melanoma brain metastases treated with immune checkpoint inhibitors. Neuro Oncol. 2019;21(12):1578–1586. doi: 10.1093/neuonc/noz141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.Colen R.R., Fujii T., Bilen M.A., et al. Radiomics to predict immunotherapy-induced pneumonitis: proof of concept. Invest New Drugs. 2018;36(4):601–607. doi: 10.1007/s10637-017-0524-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Nguyen-Kim T.D.L., Trebeschi S., Pouw J.E.E., et al. Deep learning radiomics distinguishes intrapulmonary disease from metastases in immunotherapy-treated melanoma patients. Ann Oncol. 2019;30:v529. [Google Scholar]
- 84.Elshafeey N., Kotrotsou A., Hassan A., et al. Multicenter study demonstrates radiomic features derived from magnetic resonance perfusion images identify pseudoprogression in glioblastoma. Nat Commun. 2019;10(1):1–9. doi: 10.1038/s41467-019-11007-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.Hu X., Wong K.K., Young G.S., Guo L. Support Vector Machine (SVM) multi-parametric MRI identification of pseudoprogression from tumor recurrence in patients with resected glioblastoma. J Magn Reson Imaging. 2011;33(2):296–305. doi: 10.1002/jmri.22432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 86.Li M., Tang H., Chan M.D., Zhou X., Qian X. DC-AL GAN: pseudoprogression and true tumor progression of glioblastoma multiform image classification based on DCGAN and AlexNet. Med Phys. 2020;47:1139–1150. doi: 10.1002/mp.14003. [DOI] [PubMed] [Google Scholar]
- 87.Jang B.S., Jeon S.H., Kim I.H., Kim I.A. Prediction of pseudoprogression versus progression using machine learning algorithm in glioblastoma. Sci Rep. 2018;8(1):1–9. doi: 10.1038/s41598-018-31007-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 88.Bacchi S., Zerner T., Dongas J., et al. Deep learning in the detection of high-grade glioma recurrence using multiple MRI sequences: a pilot study. J Clin Neurosci. 2019;70:11–13. doi: 10.1016/j.jocn.2019.10.003. [DOI] [PubMed] [Google Scholar]
- 89.Basler L., Gabryś H.S., Hogan S.A., et al. Radiomics, tumor volume, and blood biomarkers for early prediction of pseudoprogression in patients with metastatic melanoma treated with immune checkpoint inhibition. Clin Cancer Res. 2020;26(16):4414–4425. doi: 10.1158/1078-0432.CCR-20-0020. [DOI] [PubMed] [Google Scholar]
- 90.Rutman A.M., Kuo M.D. Radiogenomics: creating a link between molecular diagnostics and diagnostic imaging. Eur J Radiol. 2009;70(2):232–241. doi: 10.1016/j.ejrad.2009.01.050. [DOI] [PubMed] [Google Scholar]
- 91.Kuo M.D., Jamshidi N. Behind the numbers: decoding molecular phenotypes with radiogenomics – guiding principles and technical considerations. Radiology. 2014;270(2):320–325. doi: 10.1148/radiol.13132195. [DOI] [PubMed] [Google Scholar]
- 92.Lee J., Narang S., Martinez J.J., Rao G., Rao A. Associating spatial diversity features of radiologically defined tumor habitats with epidermal growth factor receptor driver status and 12-month survival in glioblastoma: methods and preliminary investigation. J Med Imaging. 2015;2(4):1–9. doi: 10.1117/1.JMI.2.4.041006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93.Grimm L.J., Zhang J., Mazurowski M.A. Computational approach to radiogenomics of breast cancer: luminal A and luminal B molecular subtypes are associated with imaging features on routine breast MRI extracted using computer vision algorithms. J Magn Reson Imaging. 2015;42(4):902–907. doi: 10.1002/jmri.24879. [DOI] [PubMed] [Google Scholar]
- 94.Sadot E., Simpson A.L., Do R.K.G., et al. Cholangiocarcinoma: correlation between molecular profiling and imaging phenotypes. PLoS One. 2015;10(7):1–12. doi: 10.1371/journal.pone.0132953. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 95.Yamamoto S., Han W., Kim Y., et al. Breast cancer: radiogenomic biomarker reveals associations among dynamic contrast-enhanced MR imaging, long noncoding RNA, and metastasis. Radiology. 2015;275(2):384–392. doi: 10.1148/radiol.15142698. [DOI] [PubMed] [Google Scholar]
- 96.Mazurowski M.A., Zhang J., Grimm L.J., Yoon S.C., Silber J.I. Radiogenomic analysis of breast cancer: luminal B molecular subtype is associated with enhancement dynamics at MR imaging. Radiology. 2014;273(2):365–372. doi: 10.1148/radiol.14132641. [DOI] [PubMed] [Google Scholar]
- 97.Sacconi B., Anzidei M., Leonardi A., et al. Analysis of CT features and quantitative texture analysis in patients with lung adenocarcinoma: a correlation with EGFR mutations and survival rates. Clin Radiol. 2017;72(6):443–450. doi: 10.1016/j.crad.2017.01.015. [DOI] [PubMed] [Google Scholar]
- 98.Caramella C., Bluthgen M.V., Rosellini S., et al. Prognostic value of texture analysis and correlation with molecular profile in EGFR mutated/ALK rearranged advanced non-small cell lung cancer (NSCLC) Eur J Cancer. 2015;51:S647–S648. [Google Scholar]
- 99.Aerts H.J.W.L., Grossmann P., Tan Y., et al. Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC. Sci Rep. 2016;6(33860):1–8. doi: 10.1038/srep33860. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100.Ozkan E., West A., Dedelow J.A., et al. CT gray-level texture analysis as a quantitative imaging biomarker of epidermal growth factor receptor mutation status in adenocarcinoma of the lung. Am J Roentgenol. 2015;205(5):1016–1025. doi: 10.2214/AJR.14.14147. [DOI] [PubMed] [Google Scholar]
- 101.De Jong E.E.C., Van Elmpt W., Hendriks L.E.L., Leijenaar R.T.H., Dingemans A.M.C., Lambin P. Radiomic CT features for evaluation of EGFR and KRAS mutation status in patients with advanced NSCLC. Radiother Oncol. 2016;119:S290–S291. [Google Scholar]
- 102.Weiss G.J., Ganeshan B., Miles K.A., et al. Noninvasive image texture analysis differentiates K-ras mutation from pan-wildtype NSCLC and is prognostic. PLoS One. 2014;9(7):e100244. doi: 10.1371/journal.pone.0100244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 103.Rios Velazquez E., Parmar C., Liu Y., et al. Somatic mutations drive distinct imaging phenotypes in lung cancer. Cancer Res. 2017;77(14):3922–3930. doi: 10.1158/0008-5472.CAN-17-0122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 104.Yoon H.J., Sohn I., Cho J.H., et al. Decoding tumor phenotypes for ALK, ROS1, and RET fusions in lung adenocarcinoma using a radiomics approach. Medicine (Baltimore) 2015;94(41):1–8. doi: 10.1097/MD.0000000000001753. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 105.McShane L.M. In pursuit of greater reproducibility and credibility of early clinical biomarker research. Clin Transl Sci. 2017;10(2):58–60. doi: 10.1111/cts.12449. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 106.Fave X., MacKin D., Yang J., et al. Can radiomics features be reproducibly measured from CBCT images for patients with non-small cell lung cancer? Med Phys. 2015;42(12):6784–6797. doi: 10.1118/1.4934826. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 107.Lecler A., Duron L., Balvay D., et al. Combining multiple magnetic resonance imaging sequences provides independent reproducible radiomics features. Sci Rep. 2019;9(2068):1–8. doi: 10.1038/s41598-018-37984-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 108.Zhao B., Tan Y., Tsai W.Y., et al. Reproducibility of radiomics for deciphering tumor phenotype with imaging. Sci Rep. 2016;6(23428):1–7. doi: 10.1038/srep23428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 109.Kim H., Park C.M., Lee M., et al. Impact of reconstruction algorithms on CT radiomic features of pulmonary tumors: analysis of intra- and inter-reader variability and inter-reconstruction algorithm variability. PLoS One. 2016;11(10):1–11. doi: 10.1371/journal.pone.0164924. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 110.Moradmand H., Aghamiri S.M.R., Ghaderi R. Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma. J Appl Clin Med Phys. 2020;21(1):179–190. doi: 10.1002/acm2.12795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 111.Zwanenburg A. Radiomics in nuclear medicine: robustness, reproducibility, standardization, and how to avoid data analysis traps and replication crisis. Eur J Nucl Med Mol Imaging. 2019;46(13):2638–2655. doi: 10.1007/s00259-019-04391-8. [DOI] [PubMed] [Google Scholar]
- 112.Boellaard R. Standards for PET image acquisition and quantitative data analysis. J Nucl Med. 2009;50(SUPPL. 1):11S–20S. doi: 10.2967/jnumed.108.057182. [DOI] [PubMed] [Google Scholar]
- 113.Öllers M., Bosmans G., van Baardwijk A., et al. The integration of PET-CT scans from different hospitals into radiotherapy treatment planning. Radiother Oncol. 2008;87:142–146. doi: 10.1016/j.radonc.2007.12.025. [DOI] [PubMed] [Google Scholar]
- 114.Kumar V., Gu Y., Basu S., et al. QIN ‘radiomics: the process and the challenges’. Magn Reson Imaging. 2013;30(9):1234–1248. doi: 10.1016/j.mri.2012.06.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115.Lambin P., Leijenaar R.T.H., Deist T.M., et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14(12):749–762. doi: 10.1038/nrclinonc.2017.141. [DOI] [PubMed] [Google Scholar]
- 116.Yip S.S.F., Aerts H.J.W.L. Applications and limitations of radiomics. Phys Med Biol. 2016;61(13):R150–R166. doi: 10.1088/0031-9155/61/13/R150. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 117.Kocak B., Ates E., Durmaz E.S., Ulusan M.B., Kilickesmez O. Influence of segmentation margin on machine learning–based high-dimensional quantitative CT texture analysis: a reproducibility study on renal clear cell carcinomas. Eur Radiol. 2019;29:4765–4775. doi: 10.1007/s00330-019-6003-8. [DOI] [PubMed] [Google Scholar]
- 118.Beam C.A., Layde P.M., Sullivan D.C. Variability in the interpretation of screening mammograms by us radiologists: findings from a national sample. Arch Intern Med. 1996;156(2):209–213. [PubMed] [Google Scholar]
- 119.Armato S.G., III, McNitt-Gray M.F., Reeves A.P., et al. The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Acad Radiol. 2007;14(11):1409–1421. doi: 10.1016/j.acra.2007.07.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 120.Hesamian M.H., Jia W., He X., Kennedy P. Deep learning techniques for medical image segmentation: achievements and challenges. J Digit Imaging. 2019;32(4):582–596. doi: 10.1007/s10278-019-00227-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 121.Nichols J.A., Herbert Chan H.W., Baker M.A.B. Machine learning: applications of artificial intelligence to imaging and diagnosis. Biophys Rev. 2019;11(1):111–118. doi: 10.1007/s12551-018-0449-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 122.Shaikh F.A., Kolowitz B.J., Awan O., et al. Technical challenges in the clinical application of radiomics. JCO Clin Cancer Informatics. 2017;(1):1–8. doi: 10.1200/CCI.17.00004. [DOI] [PubMed] [Google Scholar]
- 123.Colen R.R., Fujii T., Bilen M.A., et al. Radiomics to predict immunotherapy-induced pneumonitis: proof of concept. Invest New Drugs. 2018;36(4):601–607. doi: 10.1007/s10637-017-0524-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 124.Khan S, Yong SP. A comparison of deep learning and hand crafted features in medical image modality classification. 2016 3rd Int Conf Comput Inf Sci. Published online 2016:633-638.
- 125.Nanni L., Ghidoni S., Brahnam S. Handcrafted vs. non-handcrafted features for computer vision classification. Pattern Recognit. 2017;71:158–172. [Google Scholar]
- 126.Lu M., Zhan X. The crucial role of multiomic approach in cancer research and clinically relevant outcomes. EPMA J. 2018;9(1):77–102. doi: 10.1007/s13167-018-0128-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 127.Bodalal Z., Trebeschi S., Beets-Tan R. Radiomics: a critical step towards integrated healthcare. Insights Imaging. 2018;9:911–914. doi: 10.1007/s13244-018-0669-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 128.Zanfardino M., Franzese M., Pane K., et al. Bringing radiomics into a multi-omics framework for a comprehensive genotype-phenotype characterization of oncological diseases. J Transl Med. 2019;17:1–21. doi: 10.1186/s12967-019-2073-2. [DOI] [PMC free article] [PubMed] [Google Scholar]



