A review on radiomics and the future of theranostics for patient selection in precision medicine

Abstract

The growing complexity and volume of clinical data and the associated decision-making processes in oncology promote the advent of precision medicine. Precision (or personalised) medicine describes preventive and/or treatment procedures that take individual patient variability into account when proscribing treatment, and has been hindered in the past by the strict requirements of accurate, robust, repeatable and preferably non-invasive biomarkers to stratify both the patient and the disease. In oncology, tumour subtypes are traditionally measured through repeated invasive biopsies, which are taxing for the patient and are cost and labour intensive. Quantitative analysis of routine clinical imaging provides an opportunity to capture tumour heterogeneity non-invasively, cost-effectively and on large scale. In current clinical practice radiological images are qualitatively analysed by expert radiologists whose interpretation is known to suffer from inter- and intra-operator variability. Radiomics, the high-throughput mining of image features from medical images, provides a quantitative and robust method to assess tumour heterogeneity, and radiomics-based signatures provide a powerful tool for precision medicine in cancer treatment. This study aims to provide an overview of the current state of radiomics as a precision medicine decision support tool. We first provide an overview of the requirements and challenges radiomics currently faces in being incorporated as a tool for precision medicine, followed by an outline of radiomics’ current applications in the treatment of various types of cancer. We finish with a discussion of possible future advances that can further develop radiomics as a precision medicine tool.

Introduction

Background

Technological advances have led to an abundance of novel diagnostic techniques and imaging modalities available to oncology.¹ Additional complexity is added by genetic² and micro environmental³ heterogeneity of tumours and between patients.⁴ Due to the large volumes and complexity of modern data,⁵ new methods to facilitate clinical decision-making are required.

Precision (or personalised) medicine describes preventive and treatment procedures that take into account an individual patient’s characteristics together with their specific disease(s).⁶ A common approach to precision medicine is data mining, i.e. discovering patterns in large databases of diversified cohorts using powerful computational tools such as machine learning. Patterns can be discovered within the variability of patient populations that allow for the stratification of patient groups and the identification of the ideal treatment for the individual patient,⁷ thus improving patient outcome.^8–10 However, this requires large databases of patients in order to cover as much of the variations within a population as possible.

An important source of large-scale data that could be used are radiological images derived during routine oncological examinations. Tumours exhibit phenotypical differences which can be visualised through routine medical imaging,¹¹ which in turn allows for visualisation of the entire tumour volume or subregions on a macroscopic level, at baseline and longitudinally. However, imaging in a clinical setting is primarily used qualitatively, and clinical decision-making is based on visual assessments of the tumour by radiologists. Radiomics offers a quantitative alternative to assess tumour heterogeneity quantitatively. Radiomics is an advanced image feature analysis methodology, which formats standard clinical images from CT, MRI and/or positron emission tomography (PET) into a multidimensional source for data mining.¹² A large number of image features are extracted from imaging data using various mathematical algorithms. These features, together with gold standard information, are used by machine-learning algorithms, computational methods that “learn” correlations from data, creating models that automate and improve classification of tumour phenotype and genomic profile^13–15 as imaging biomarkers.

Radiomics-based imaging biomarkers have shown to outperform common prognostic models based on clinical parameters such as the Tumor-Node-Metastasis staging system (TNM).¹³ However, radiomics does not intend to replace current clinical decision-making, but rather aims to provide a supplement to current measures such as clinical, treatment and genomic data, all incorporated into a decision support system.¹⁶ To do so, a robust, repeatable and cost-effective method to clinically implement radiomics is required.

Radiomics workflow

A typical radiomics analysis starts with data selection: choosing the image to analyse, the imaging protocol to use and the correlated outcome. The image typically contains the primary tumour volume, which is analysed and linked to a certain outcome, such as tumour type, overall survival, or tumour recurrence. Proper data selection is important to create useful models, as it needs to be reproducible and applicable across sizeable cohorts. Large heterogeneous datasets are required to provide enough data to validate the model on different subsamplings of the data (cross-validation).¹⁷ In addition, the quality of the data is dependent on the image acquisition protocols used in clinical centres, which can often vary extensively, as well as the imaged site, scanner properties, reconstruction methods and motion artefacts.^{18, 19} Guidelines for image acquisition and standardised protocols are therefore beneficial for producing large, high-quality datasets.²⁰ In the case of non-standardised imaging protocols, sharing of imaging protocols should be encouraged.

After image acquisition and volume reconstruction, a region of interest is defined, usually, but not necessarily, through slice-by-slice delineation of the tumour in the case of three-dimensional images. This is a labour-intensive process, and the variance caused by inter- and intra-operator variability is an issue.^{21, 22} A (semi-) automatic segmentation method to reduce workload and uncertainty caused by human error is therefore preferred. Besides operator variability, image segmentation, protocol standardisation, slice interval, reconstruction method, time-point and respiratory motion have all been found to have effects on feature reproducibility.^23–35 Methods to improve reproducibility include multiple segmentations by different clinicians and phantom studies to determine the effects caused by different scanners.

Since the values of extracted features (mostly mathematical formulas using pixel/voxel intensities as input) depend on image reconstruction and pre-processing methods, proper reporting of methods such as filtering techniques, intensity discretisation and voxel resampling is critical for inter-operability of the radiomics features. Many of the extracted features are noise driven and need to be removed to improve model performance. The same applies to features that are highly correlated with other features or existing clinical parameters, as they do not provide any meaningful addition to the model. Test-retest studies which repeat the imaging processes after a short period of time are indispensable, as they measure the amount of variation inherent in the measurements. Stability and correlation tests can be used to make a selection based on the most robust, repeatable and non-redundant features.^{36, 37}

The extracted features are fed into machine-learning methods together with clinical outcomes or pathology results to construct classification, predictive, or prognostic models. Prognostic models aim to predict a certain outcome regardless of therapy, while predictive models provide information about the effects of a certain therapeutic intervention. However, the number of extracted features is often larger than the number of patients included in a cohort, which risks overfitting the model. The best solution to prevent overfitting is to increase the number of samples used to train the model. While clinical data is abundant compared to research trial data, sharing between institutes has proven to be difficult due to various ethical, political and administrative issues.³⁸ An alternative to large datasets is to reduce the number of features to a subset of the most relevant features. Various filtering-based techniques for feature selection can be used, such as the univariate Fisher score and Gini index tests, or multivariate algorithms such as mutual information or Conditional infomax feature extraction,³⁹ which identify and select a subset of features based on predictive power. Valid predictive modelling requires separate independent datasets for training and validation.

Various different machine-learning models are available, such as neural networks, decision trees, support vector machines and multiple regression techniques. The modelling procedure has been shown to affect performance of prediction models based on radiomics features.³⁹) Common measures of predictive performance of models are discrimination and calibration.⁴⁰ Discrimination is a measure of the model to assign a higher risk-prediction to patients positive to a certain outcome, compared to patients without the outcome, which can be quantified using the sensitivity, specificity, or through the area under the curve (AUC) of the receiver operating characteristic. The AUC is equal to the probability that a positive event is correctly labelled as a positive event, and is given in the range of 0 to 1. Alternatively, the Concordance Index (CI), a measure of goodness of fit for classification models with binary outcome ranging from 0 to 1, can be used. Both AUC and CI show a perfect predictive performance at 1, while at 0.5 the predictive performance is completely random. Calibration is an internal measure of the model’s agreement between observed outcomes and predicted outcomes. The calibration is usually assessed through a calibration slope, where different resamplings of observed outcomes are plotted against predicted outcomes. If 100% agreement between these two is found at multiple samplings, then the calibration slope will be 1. Finally, a log-rank test is usually used to test the significance of the difference between survival curves of two patient groups. This is used when separating patients in low- and high-risk groups based on radiomics features.

These measures of predictive performance are used to internally and externally validate the model. Internal validation is necessary to estimate and reduce the optimism in model performance, which is the degree a trained model fits worse on new data than it does on the data used to train the model. Internal validation uses the data used to train the model, and can be performed through methods such as bootstrap analysis or cross-validation.⁴¹ External validation uses an independent, external dataset to validate the accuracy of the predictive model, and to assess the generalisability of a predictive model.⁴¹ Figure 1 shows an overview of the steps involved to train and validate a predictive model.

Figure 1.

Overview of the steps involved to train and validate a predictive model.

Effective and transparent radiomics studies require rigorous compliance with several guidelines, including effective validation. The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) initiative is a set of guidelines made for studies creating and/or validating prediction models.⁴² There are guidelines for the source and specific information of data, the type of predictive model, procedures for building the model and the method for internal validation and measurements of model performance. Whereas the TRIPOD initiative covers prediction models in general, the Radiomics Quality Score (RQS)⁴³ is being developed specifically for radiomics studies. The RQS assesses the quality of a study using a checklist and reports compliance as a percentage. Some of the guidelines include robust segmentations, test-retest stability of the determined features, the standardisation or thorough description of imaging protocols used, valid feature selection and internal/external validation.⁴⁴ An overview of the different steps and the RQS criteria is shown in Figure 2.

Figure 2.

Overview of steps of a Radiomics analysis (top) with corresponding RQS score criteria for each step (bottom). RQS, Radiomics Quality Score.

The aim of these guidelines is to provide key details of model development and validation, which in turn allows for better reproducibility and critical appraisal of predictive models. For future and past studies, authors should check the RQS score and TRIPOD initiative to determine the quality of their methodology and allow the field of radiomics to mature. The ultimate objective of precision medicine is to link the tumour phenotype to a certain clinical endpoint, with the goal of improving clinical decision-making. Therefore, the next section will describe the use of radiomics in various studies and their efficacy in determining clinical endpoints.

Role in precision medicine

Aerts et al¹³ performed a radiomics analysis on a large CT dataset (N = 1019) of lung- and head and neck (h-n) cancer patients. Using a feature selection algorithm to reduce the number of features from 440 to a prognostic signature of 4 features, they found that a model built using this signature was significantly more prognostic of overall survival (OS) than a measure of tumour volume, and combining the radiomics signature with tumour volume also provided a better prognostic ability. The model was validated on different patient groups and cancer types.¹³ The radiomics signature showed slightly higher prognostic performance when validated in an external lung dataset than TNM or tumour volume (CI of 0.65 vs 0.63 and 0.60 respectively). For two external h-n cohorts, the signature showed higher performance compared to volume or TNM in one case (CI of 0.69 vs 0.65 and 0.66 respectively), and similar performance in the other (CI = 0.69 vs 0.68 and 0.69 respectively). This radiomics signature was also externally validated in a study by Leijenaar et al.⁴⁵ on a large set of oropharyngeal squamous cell carcinoma patients (N = 542).⁴⁵ The signature showed good discrimination and calibration (CI = 0.628 and calibration slope of 0.855), and after the population were split in two groups using the median value of the signature score, significant differences in OS (long rank p-value = 2e-5) could be observed.

Furthermore, CT radiomics features have been shown to be prognostic of distant metastasis and 12-month survival in glioblastoma,⁴⁶ and pathological response to treatment,⁴⁷ local recurrence,^{48, 49} histology subtype,^{50, 51} OS^{36, 50} and prediction of radiation induced pneumonitis^{52, 53} for lung cancer. In h-n squamous cell carcinomas, radiomics has proven to improve the prediction overall and progression free survival, and determining Human papillomavirus (HPV) status.⁵⁴

Delta-radiomics is an alternative analysis which measures the change of radiomics features longitudinally. Certain features have been proven to change during treatment, indicating that this may be an additional source of information.⁵⁵ Delta-radiomics on CT has shown to be prognostic in non-small cell lung cancer (NSCLC) for OS, local recurrence and distant metastasis.⁵⁶ For h-n cancer patients, delta-radiomics features have proven to be a predictive and prognostic biomarker, as well provide additional information of the presence of HPV for patient stratification.

An additional source of routine medical images for radiomics analysis are cone-beam CT (CBCT) images, often used in radiotherapy for daily positioning before treatment. Van Timmeren et al⁵⁷ have used CBCT data of NSCLC patients to validate a previously constructed CT radiomics signature. The signature was found to be predictive of OS in three different independent CBCT datasets (CI = 0.59–0.66), indicating CBCT could potentially be a useful source of information for radiomics analysis.⁵⁷

Fludeoxyglucose-PET (FDG-PET)-based quantitative image analysis shows promise in improving prognosis in pancreatic cancer. A study by Cui et al used quantitative imaging features to predict OS, and showed better prognostic compared to the use of conventional prognostic variables of tumour volume and maximum standardised uptake value  (CI of 0.66 vs 0.48-0.64).⁶⁰ FDG-PET-based radiomics features correlate to mortality, local failure and distant metastasis for pancreatic cancer,⁶¹ and have also shown to be predictive in oesophageal cancer,⁶² tumour response in cervical cancer^{63, 64} and local control⁶⁵ and OS⁶³ in h-n cancer.

MRI-based radiomics has shown promise in prostate cancer: a study by Shoshana et al. (2016) used T ₂ weighted MRI radiomics features to differentiate between peripheral and transition zone prostate tumours (AUC = 0.61–0.71), in a patient dataset from three different institutions.⁶⁶ Furthermore, a study by Vallières et al⁶⁷ use a combination of FDG-PET and MRI texture features to predict the lung metastasis in soft-tissue sarcomas. They found that a multivariable model was highly predictive of lung metastasis in soft-tissue sarcomas (AUC = 0.98), validated through bootstrapping procedures. However, the study lacked external validation for a valid conclusion.⁶⁷ In the context of glioblastoma, several studies using MRI data have shown that a radiomics model may accurately detail the molecular subtype of the tumour,^68–70 OS^69–71 and predict short v s long-term survival.⁷² Finally, for an imaging method outside of radiology, Zhang et al.⁷³ proposed a radiomics approach to ultrasound elastography, to use the density of tumour tissue for classification as benign or malignant. A signature of seven features, out of a total of 364 extracted features, was able to accurately (AUC = 0.92) discriminate between benign and malignant tumour tissue.⁷³

To reduce inter- and intra-observer delineation variability and to reduce workload, a number of (semi-) automatic segmentation methods have been proposed and tested in radiomics studies in recent years. Several studies have shown that (semi-) automatic segmentation methods reduce inter-observer delineation variability compared to manual segmentation of lung lesions.^74–77 For example, a study by Parmar et al.⁷⁷ compared the robustness of 56 radiomics features derived with manual segmentation of tumour volume by five experts to a semi-automatic method performed two times by three experts, and showed that semi-automatically derived features have significantly higher reproducibility compared to manually derived features.⁷⁷ Full automatic segmentation of tumours is also a possibility, as shown by Li et al.⁷⁸ This study used radiomics features in a random forest model to classify tumour tissue on a voxel level. The algorithm was trained and tested on publically available datasets, and showed promising accuracy in classifying tumour tissue, necrosis, normal tissue and oedema.⁷⁸

Semantic features, unique qualitative characteristics that provide information about the prognosis and (sub) type of lesions, are an alternative method to describe tumour (sub) type, and could be useful in improving prediction of certain endpoints. Some examples of semantic features are the presence of cavitation or calcification in the tumour, or features describing the roundness or spiculation of the tumours. In a study on NSCLC, Yip et al⁷⁹ studied 9 semantic features, consisting of 3 binary features and 6 categorical classifiers, and 57 radiomics features describing NSCLC cancer phenotypes. To study the correlation between features they used Spearman’s Rank-Order Correlation, which is a measure of the strength and direction of association between two variables. Spearman’s rank ranges from −1 to 1, with both extremes signifying perfect correlation between two variables. The study found significant association between radiomics features and binary semantic features (AUC = 0.56–0.76), but no or weak correlation was found between classification semantic and radiomics features (Spearman’s correlation = 0.002–0.65). This indicates that radiomics and semantic features have complementary but distinct roles in outcome prediction, as they have both been proven to be able to significantly improve prediction outcomes.⁷⁹

Lastly, deep learning tools, such as convolutional neural networks (CNNs), could be a method to augment radiomics analysis. Deep learning algorithms are able to learn features from imaging data without much manual input, provided that a large amount of data is available. Deep learning has been successfully implemented in a number of different studies using medical imaging data.^{80, 81} Orlando et al⁸² used a combination CNN-learned and hand-crafted discriminative features to detect red lesions (a collective term for micro aneurysms and haemorrhages), one of the earliest signs in diabetic retinopathy. The combination of features was used in a random forest classifier to discriminate between true and false red lesion candidates, and compared against either set of features separately. The combination achieved higher AUC values compared to the separate feature prediction models (AUC of 0.89 vs 0.79/0.73 for CNN and hand-crafted features respectively). Recently published articles have already shown that radiomics analysis may benefit from incorporating deep learning methods.^(82-85) For example, Lao et al⁸³ used a combination of hand-crafted and deep radiomics features to predict OS for Glioblastoma Multiforme patients on MRI images. After feature selection, a radiomics signature was created, using exclusively deep-learned features, that was able to accurately predict OS in an independent validation dataset (AUC = 0.71). Deep learning augmented radiomics analysis has also been reported to be effective in assessing treatment response in bladder cancer,⁸⁴ where conversely a signature built solely on hand-crafted features was found to have better prognostic performance. These results indicate that deep learning will have an increasingly important role in predictive modelling,⁸² and have a complementary role with hand-crafted features in a radiomics analysis framework.

Discussion

Radiomics has been shown to be suitable for classification, prediction and prognosis of various clinical endpoints and tumour types. Many studies show a clear improvement over conventional measures predicting clinical endpoints, although variation in feature stability due to different scanners, imaging protocols and tumour motion still leaves a lot of room for improvement.^{13, 60} The segmentation of tumours also proves to be a small but persistent obstacle, as it is a labour- and time-intensive process and is heavily influenced by inter- and intra- segmentation variation.^{21, 22} However, numerous studies have reported methods to allow for a more automatic approach to segmentation,^74–78 which in turn could lead to a more robust radiomics analysis.

Combining radiomics features with deep learning features or semantic features may be able to further improve prognostic performance. Several studies have proven the effectiveness of using these features independently in predictive modelling.^80–87 In studies comparing the prognostic performance of these features to hand-crafted radiomics features, results were found to be mixed, indicating these methods may have distinct and complementary roles in improving prognosis.

A larger hurdle for radiomics is the transition to clinical implementation. While routine delineation is already in place in radiotherapy settings, a clinical platform to easily perform radiomics analysis during routine check-up/treatment is not. The main challenge of precision treatment is to correctly integrate various sources of data quantitatively and subsequently use this data to provide specific clinical predictions that accurately and robustly estimate outcomes as a function of the possible decisions. Numerous methods, besides radiomics, are currently in use that make use of novel biomarkers, as well as conventional clinical factors. However, many of these methods lack external validation of their legitimacy, reproducibility, or clinical validity.⁸⁸ Radiomics offers a solution that integrates multiple measures into one prediction of outcome, with the added benefit of automation, which could save time and money in a clinical environment.

While many radiomics studies include external validation steps, sharing of clinical data is still an issue.⁸⁹ The difficulty in sharing data may be overcome through a centralised database, or conversely through decentralised distributed learning platforms.⁹⁰ To facilitate a centralised database, data has to be made available in accordance with the FAIR principles: Findability, Accessibility, Inter-operability and Reusability.⁸⁹ An example of an effort to increase data shareability is through the development of ontologies to describe radiomics features.²⁹

The distributed learning method instead aims to solve the problem of sparse data by avoiding the numerous ethical, legal and administrative issues involved in sharing data between institutes. Instead of the images being collected from numerous institutes in one central location, the model is sent and trained on site without any data leaving a particular institute. The trained models are then collected, analysed and integrated into a single model. Several proof-of-concept studies have proven that a distributed learning approach is feasible using clinical parameters,^90–92 and the next step would be to integrate radiomics features, by sending a platform to extract radiomics features on-site in conjunction with the untrained predictive model. This way, a distributed learning method could provide the necessary volume and variety in data to achieve a machine-driven approach to medicine.

Conclusion

In conclusion, radiomics provides a novel non-invasive method of assessing tumour subtype, using the mostly untapped source of data of routine clinical images. The technique is often hampered by studies with small sample sizes and lack of external validation. In addition, variability in features caused by differences in imaging modality, protocols and respiratory motion, and a lack of inter-operability, may decrease the generalisability of the created radiomics models. In the future, research should be informed by guidelines such as RQS and TRIPOD, which improve the validity of radiomics as a clinically accepted field. The clinical value of the technique has already been described for a wide range of tumours and a number of different clinical outcomes. The added fact that the analysis can be performed in an automated fashion makes the technique attractive for clinical implementation to reduce workload. Performing studies on different tumour sites/types in future research may prove the generalisability of the method, and consequently lead to radiomics becoming a standard method clinically. In the future, larger volumes of data will be available for use in Radiomics studies by means of centralised, publically accessible datasets and distributed learning. Combining radiomics with other parameters will lead to high-quality decision support systems, and deep learning and semantic feature approaches may be combined with radiomics analyses to increase predictive accuracies of these models even further. Radiomics has a way ahead before full implementation in clinic is a reality, but may prove to be invaluable in realising precision medicine in cancer treatment.