Radiomics: Extracting more information from medical images using advanced feature analysis

Abstract

Solid cancers are spatially and temporally heterogeneous. This limits the use of invasive biopsy based molecular assays but gives huge potential for medical imaging, which has the ability to capture intra-tumoural heterogeneity in a non-invasive way. During the past decades, medical imaging innovations with new hardware, new imaging agents and standardised protocols, allows the field to move towards quantitative imaging. Therefore, also the development of automated and reproducible analysis methodologies to extract more information from image-based features is a requirement. Radiomics – the high-throughput extraction of large amounts of image features from radiographic images – addresses this problem and is one of the approaches that hold great promises but need further validation in multi-centric settings and in the laboratory.

1. Introduction

The use and role of medical imaging technologies in clinical oncology has greatly expanded from primarily a diagnostic tool to include a more central role in the context of individualised medicine over the past decade (Fig. 1). It is expected that imaging contains complementary and interchangeable information compared to other sources, e.g. demographics, pathology, blood biomarkers, genomics and that combining these sources of information will improve individualised treatment selection and monitoring.¹

Fig. 1.

Different sources of information, e.g. demographics, imaging, pathology, toxicity, biomarkers, genomics and proteomics, can be used for selecting the optimal treatment.

Cancer can be probed in many ways depending on the non-invasive imaging device used or the mode by which it operates (Fig. 2). Classically, anatomical computed tomography (CT) imaging is a often used modality, acquiring images of the ‘anatome’ in high resolution (e.g. 1 mm³). CT imaging is now routinely used and is playing an essential role in all phases of cancer management, including prediction, screening, biopsy guidance for detection, treatment planning, treatment guidance and treatment response evaluation.^2,3 CT is used in the assessment of structural features of cancer but it is not perceived to portray functional or molecular details of solid tumours. Functional imaging concerns physiological processes and functions such as diffusion, perfusion and glucose uptake. Here, commonly used methodologies are dynamic contract enhanced-magnetic resonance imaging (DCE-MRI), assessing tumour perfusion and fluoro-2-deoxy-D-glucose (FDG) positron emission tomography (PET) imaging, assessing tumour metabolism, which both often are found to have prognostic value.^4–6 Finally, another modality is molecular imaging, visualising at the level of specific pathways or macro-molecule in vivo. For example, there are molecular markers assessing tumour hypoxia or labelled antibodies, assessing receptor expression levels of a tumour.^1,7

Fig. 2.

Multilevel imaging: anatomical, functional, and molecular imaging.

Over the past decades, medical imaging has progressed in four distinct ways:

• Innovations in medical devices (hardware): This concerns improvements in imaging hardware and the development of combined modality machines. For example, in the last decade we moved from single slice CT to multiple slices CT and CT/PET. More recent developments are dual-source and dual-energy CT. These techniques significantly increase the temporal resolution for 4-D CT reconstructions allowing visualisation of fine structures in tissues, also in several stages in the cardiac or respiration phase. Moreover, dual-energy CT can be used to improve identification of tissue composition and density.

• Innovations in imaging agents: Innovations in imaging agents (or imaging biomarker, imaging probe, radiotracer), i.e. molecular substances injected in the body and used as an indicator of a specific biological process occurring in the body. This is achieved by contrast agents, i.e. an imaging agent using positive emission tomography (radiotracer). A common use is to find indications of pathological processes, e.g. hypoxia markers using PET imaging.

• Standardised protocol allowing quantitative imaging: Historically radiology has been a qualitative science, perhaps with the exception of the quantitative use of CT based electron densities in radiotherapy treatment planning. The use of standardised protocols like common MRI spin-echo sequences helps to allow multicentric use of imaging as well as transforming radiology to a more quantitative, highly reproducible science.

• Innovations in imaging analysis: The analysis of medical images has a large impact on the conclusions of the derived images. More and more software is becoming available, allowing for more quantification and standardisation. This has been illustrated by the development of the computer-assisted detection (CAD systems) that improves the performance of detecting cancer in mammography or in lung diseases.⁸

Radiomics focuses on improvements of image analysis, using an automated high-throughput extraction of large amounts (200+) of quantitative features of medical images and belongs to the last category of innovations in medical imaging analysis. The hypothesis is that quantitative analysis of medical image data through automatic or semi-automatic software of a given imaging modality can provide more and better information than that of a physician. This is supported by the fact that patients exhibit differences in tumour shape and texture measurable by different imaging modalities (Fig. 3).

Fig. 3.

(A) Two representative 3-D representations of a round tumour (top) and spiky tumour (bottom) measured by computed tomography (CT) imaging. (B) Texture differences between non-small cell lung cancer (NSCLC) tumours measured using CT imaging, more heterogeneous (top) and more homogeneous (bottom). (C) Differences of FDG-PET uptake, showing heterogeneous uptake.

2. The workflow of Radiomics: a (semi) high-throughput approach

Fig. 4 depicts the processes involved in the Radiomics workflow. The first step involves the acquisition of high quality and standardised imaging, for diagnostic or planning purposes. From this image, the macroscopic tumour is defined, either with an automated segmentation method or alternatively by an experienced radiologist or radiation oncologist. Quantitative imaging features are subsequently extracted from the previously defined tumour region. These features involve descriptors of intensity distribution, spatial relationships between the various intensity levels, texture heterogeneity patterns, descriptors of shape and of the relations of the tumour with the surrounding tissues (i.e. attachment to the pleural wall in lung, differentiation). The extracted image traits are then subjected to a feature selection procedure. The most informative features are identified based on their independence from other traits, reproducibility and prominence on the data. The selected features are then analysed for their relationship with treatment outcomes or gene expression. The ultimate goal is to provide accurate risk stratification by incorporating the imaging traits into predictive models for treatment outcome and to evaluate their added value to commonly used predictors.

Fig. 4.

The Radiomics workflow. On the medical images, segmentation is performed to define the tumour region. From this region the features are extracted, e.g. features based on tumour intensity, texture and shape. Finally, these features are used for analysis, e.g. the features are assessed for their prognostic power, or linked with stage, or gene expression.

3. The Radiomics hypothesis: inferring proteo-genomic and phenotypic information from radiological images

The underlying hypothesis of Radiomics is that advanced image analysis on conventional and novel medical imaging could capture additional information not currently used, and more specifically, that genomic and proteomics patterns can be expressed in terms of macroscopic image-based features. If proven, we can infer phenotypes or gene–protein signatures, possibly containing prognostic information, from the quantitative analysis of medical image data.

This hypothesis is supported by image-guided biopsies, which demonstrated that tumours show spatial differences in protein expressions.⁹ More specifically, it has been demonstrated that major differences in protein expression patterns within a tumour can be correlated to radiographic findings (or radiophenotypes) such as contrast-enhanced and non-enhanced regions based on CT data.¹⁰ The authors suggest that image-guided proteomics holds promise for characterising tissues prior to treatment decisions and without imaging there is indeed a risk that the optimum treatment decision could be neglected (i.e. the use or not of a targeted agent). Also, Kuo et al. reported the association of CT-derived imaging traits with histo-pathologic markers, and several pre-defined gene expression modules on liver cancer. ^11,12 In ovarian carcinoma, an imaging feature describing the enhancement fraction as proportion of enhancing tumour tissue on a pre-treatment CT scan, was found predictive for outcome after first line chemotherapy. ¹³ In lung cancer, CT derived information has been limited to pre-treatment assessment of tumour volume and as response evaluation defined as tumour size reduction.¹⁴ For PET imaging, the maximum and median FDG uptake has often been investigated, indicating strong prognostic power.^6,19 However, more complex descriptions of FDG uptake are only investigated on a limited scale. There was a study of El Naqa et al.¹⁵, investigating the predictive power of intensity–volume histogram (IVH) metrics, shape and texture features to assess response to treatment of a limited set of patients with head and neck and cervix cancers. Tixier et al. also explored the potential of SUV based, shape and texture features extracted from baseline FDG-PET, images, to assess response to therapy and prognosis in order to predict response to combined chemo-radiation treatment in oesophageal cancer.¹⁶ Also, textural features in FDG PET images exhibited small variations due to different acquisition modes and reconstruction parameters.¹⁷ These examples open the question of whether quantitative extraction of additional imaging features on conventional imaging improves the ability of currently used parameters to predict or monitor response to treatment.

Furthermore, Radiomics can be linked with the concept of radio-genomics, which assumes that imaging features are related to gene signatures. An interesting finding in recent literature is that tumours with more genomic heterogeneity are more likely to develop a resistance to treatment and to metastasise.¹⁸ This links to the concept that more heterogeneous tumours have a worse prognosis. According to the Radiomics hypothesis, the genomic heterogeneity could translate to an expression in an intra-tumoural heterogeneity that could be assessed through imaging and that would ultimately exhibit worse prognosis. This hypothesis has been sustained by Jackson et al.¹⁹ and as well as by Diehn et al.²⁰ who quite convincingly showed that proliferation and hypoxia gene expression patterns can be predicted by mass effect and tumour contrast enhancement, respectively. They also showed that a specific imaging pattern could predict overexpression of epidermal growth factor receptor (EGFR), a known therapeutic target. Moreover, in their analysis the presence of certain image features was highly predictive of outcome. The authors concluded that imaging in this case MR provided an ‘in vivo portrait’ of genome-wide gene expression in glioblastoma multiform. Similar findings have been found in hepatocellular carcinomas by Segal et al.²¹, showing that the combination of only 28 imaging traits was sufficient to reconstruct the variation of 116 gene expression modules.

These types of studies will need to be extended, by including more patients with external validation datasets, more tumour types that exhibit phenotypes such as invasiveness. This will be the focus of the QuIC-ConCePT consortium, to confirm experimentally the Radiomics hypothesis, namely to establish a causal relationship between gene expression patterns and image features.

4. Conclusions

Solid cancers have extraordinarily spatial and temporal heterogeneity at different levels: genes, proteins, cells, microenvironment, tissues and organs. This limits the use of biopsy based molecular assays but in contrast gives a huge potential for non-invasive imaging, which has the ability to capture intra-tumoural heterogeneity in a non-invasive way. Medical imaging innovations with new hardware, new imaging agents and standardised protocol now allow for quantitative imaging but require the development of ‘smart’ automated software to extract more information from image-based features. Radiomics – the high-throughput extraction of image features from radiographic images – is one approach that holds great promises but needs further validation in a multi-centric setting and in the laboratory.

Appendix A

QuIC-ConCePT Consortium participants include: AstraZeneca, European Organisation for Research and Treatment of Cancer (EORTC), Cancer Research UK, University of Manchester, Westfälische Wilhelms-Universität Münster, Radboud University Nijmegen Medical Center, Institut National de la Santé et de la Recherche Médical, Stichting Maastricht Radiation Oncology ‘Maastro Clinic’, VUmc Amsterdam, King’s College London, Universitair Ziekenhuis Antwerpen, Institute of Cancer Research – Royal Cancer Hospital, Erasmus Universitair Medisch Centrum Rotterdam, Imperial College of Science Technology and Medicine, Keosys S.A.S., Eidgenössische Technische Hochschule Zürich, Amgen NV, Eli Lilly and Company Ltd., GlaxoSmithKline Research & Development Limited, Merck KGa, Pfizer Limited, F. Hoffmann – La Roche Ltd., Sanofi–Aventis Research and Development.