A Role for FDG PET Radiomics in Personalized Medicine?

Abstract

Radiomics describes the extraction of multiple features from medical images, including molecular imaging modalities, that with bioinformatic approaches, provide additional clinically relevant information that may be invisible to the human eye. This information may complement standard radiological interpretation with data that may better characterize a disease or that may provide predictive or prognostic information. Progressing from predefined image features, often describing heterogeneity of voxel intensities within a volume of interest, there is increasing use of machine learning to classify disease characteristics and deep learning methods based on artificial neural networks that can learn features without a priori definition and without the need for preprocessing of images. There have been advances in standardization and harmonization of methods to a level that should support multi-center studies. However, in this relatively early phase of research in the field, there are limited aspects that have been adopted into routine practice. Most of the reports in the molecular imaging field describe radiomic approaches in cancer using ¹⁸F-fluorodeoxyglucose positron emission tomography (¹⁸F-FDG-PET). In this review, we will describe radiomics in molecular imaging and summarize the pertinent literature in lung cancer where reports are most prevalent and mature.

Introduction

Nuclear medicine imaging, and especially positron emission tomography (PET), already provides additional functional and molecular information on disease processes compared to predominantly morphological imaging methods such as radiographs, ultrasound, computed tomography (CT), and magnetic resonance imaging (MRI). However, it is likely that only a fraction of the information available in an image may be visible to the human eye and that further clinically useful information may be mined using radiomic analysis, whereby image features can be extracted using computational approaches.^1–5

Radiomics is evolving, and the natural extension of radiomics into the fields of artificial intelligence (AI) and machine learning (ML) with the availability of increased and affordable computing power and improved access to large imaging datasets, is generating enormous interest from clinicians, radiologists, computer scientists, academics and industry.^6,7 It cannot have escaped attention that imaging journals and conferences almost all include increasing amounts of work in the fields of radiomics and AI.

There is hope, in addition to the standard metrics a radiologist or nuclear medicine physician reports on a macroscopic scale (eg, length, volume, standardized uptake value (SUV)), that access to data held within an image that is not visible with the naked eye, will allow an ability to extract valuable additional clinically relevant data. These data may improve characterization and phenotyping, segmentation, prediction of treatment response, and prognostication. Radiomic features may be used alone, in a model to provide a radiomic signature, or with other clinical or -omics data to provide better characterization, prediction or prognostication, particularly in the field of oncologic imaging.

Early reports have hinted at the potential power of radiomics to personalize patients’ management and improve clinical outcomes. However, there are few if any applications that have been adopted into the day to day workflow of most clinical departments. Nevertheless, intense research is being carried out by academics and industry to optimize workflows and use of imaging data for nuclear medicine physicians and radiologists for patient benefit.

In this review, we describe how radiomics, ML, deep learning (DL), and AI relate. We summarize the literature that is available for the application of radiomics in PET imaging with a focus on oncologic aspects and give examples of where it may be applied in non-small cell lung cancer (NSCLC), this being the most frequently reported tumor type in this field.

Definitions

Radiomics involves the extraction of multiple features from medical images that can be used to provide additional information, often using bioinformatic approaches. Most of the features may be invisible to the human eye but are generally derived by mathematical formulae that provide data on the intensity, position, and relationship of voxels to other voxels in the image. Categories of radiomic features include parameters that describe texture, heterogeneity or shape within an image.⁸

The underlying hypothesis of radiomics is that intensity values from individual voxels in an image and their spatial distribution may reflect underlying genetic, molecular, and cellular processes on a relatively macroscopic scale. Heterogeneity in an image of a malignant tumor may be influenced by a number of underlying biological processes including angiogenesis, fibrosis, cellularity, hypoxia, and receptor expression. Variability in these factors is known to be associated with more aggressive behavior, worse prognosis, and resistance to treatment⁹ and may reflect the underlying genetic heterogeneity derived from clonal variation within and between tumors in the same patient. Intratumoral genetic heterogeneity is associated with morphologic heterogeneity of nuclei within tumor cells and is associated with a poor prognosis.^10–12 Genetically heterogeneous tumors are more likely to form treatment-resistant clones and have greater metastatic potential.¹³

There are a number of potential advantages of using imaging to determine tumor heterogeneity or other biological characteristics. Whole tumors, their metastases, and their microenvironment can be sampled, something that is not possible with biopsy specimens. Imaging can also be performed noninvasively and serially during treatment. A disadvantage is that imaging heterogeneity or other parameters is on a macroscopic scale and it remains unclear exactly what underlying biological or genetic features the various imaging modalities and their extracted radiomic features may reflect.

The most common type of radiomic features that have been described are statistical parameters that can be derived from the voxel intensity histogram and describe global characteristics within a volume of interest (VOI), so-called first-order features. Examples include SUVmean, kurtosis (the “peakedness” of the histogram compared to a normal distribution), skewness (the asymmetry or deviation from a normal distribution), and first-order entropy (a measure of the randomness of voxel intensities within a VOI). First-order features describe global characteristics but do not describe the spatial relationships between voxels.⁸

Second-order statistical features describe the relationship between intensity and position of pairs of voxels. Examples include features calculated from gray level co-occurrence matrices such as second-order entropy (randomness), second-order energy (uniformity), homogeneity and contrast (local intensity variation), all defined by specific mathematical formulae.¹⁴

High-order features describe relationships between 3 or more voxels in different planes. These are calculated from neighborhood gray tone difference matrices and include parameters such as coarseness (large basic patterns making up a texture), contrast (large differences between neighboring regions), and busyness (rapid changes in intensity between neighboring voxels).¹⁵ Other types of high-order features include those that can be calculated to give information on similarity of voxel intensities in runs (gray level run length matrices)¹⁶ or similar zones of intensity (gray level size zone matrices).¹⁷

The nomenclature of statistical features can be confusing as some first-, second- or high-order features have the same name but different formulae, for example entropy can be first or second-order, or some features have different names, for example, energy, uniformity and angular second moment, but have the same formula. Statistical features are often called texture or heterogeneity parameters as they describe the distribution of tracer within a VOI. Types of the more commonly used statistical radiomic features are described in Table 1.

Table 1. Statistical Texture Features.

Order	Description	Examples
First-order	Gray level frequency distribution from histogram analysis	SUV, minimum, mean, maximum, standard deviation, skewness, kurtosis, entropy, energy
Second-order	Gray level co-occurrence matrices	entropy, energy, contrast, homogeneity, dissimilarity, correlation
High-order	Neighborhood gray tone difference	coarseness, busyness, contrast, complexity
	matrices Voxel alignment matrices	run length nonuniformity, run percentage, short run emphasis, long run emphasis, gray level nonuniformity
	Gray level size zone matrices	size zone nonuniformity, zone percentage, zone variance, zone entropy, low gray level zone emphasis, high gray level zone emphasis, low gray level short zone emphasis, high gray level short zone emphasis, low gray large zone emphasis, high gray level large zone emphasis

Fractal analysis is an example of model-based features. Fractals are self-similar structures of repeating patterns at different scales whereby fractal parameters such as fractal dimension and lacunarity describe the spatial complexity and heterogeneity of voxels within an image.^18,19 Transform-based methods analyze voxel intensities in a different space and may be used to measure heterogeneity but have not been used as frequently for PET data analysis as the statistical methods which predominate in the literature. Other features related to the shape of a lesion VOI, for example, a tumor, can also be calculated that describe the geometry or how much the geometry deviates from a regular spheroid or ellipsoid shape.^8,20,21

The use of radiomics is evolving. Early research has concentrated on the selection of predefined image features (often texture features that describe voxel intensity heterogeneity) that can be used alone in a relatively simple way to classify pathology in images or as predictive or prognostic markers. In a more complex form, the radiomic features can be used in combination as inputs into ML classifiers. AI broadly describes the use of computers to perform tasks that would normally require human-level intelligence and ML is a branch of AI that involves algorithms that are programmed to learn from observations. ML algorithms, including methods such as support vector machines (SVM), random forest (RF), artificial neural networks (ANN) that reflect biological neuronal systems, and principle component analysis (PCA), are trained to learn features and patterns from data. Details are beyond the scope of this review but are well described elsewhere.^6,7,22 Supervised learning refers to ML algorithms that are trained with data that has known outputs (SVM, ANN, RF) whereas unsupervised learning describes algorithms in which the outcomes are not known (PCA). Deep learning (DL) is a type of ML that utilizes neural networks with several layers and convolutional neural networks (CNN) are most frequently used in imaging.

Radiomic features may be obtained without a priori definition using DL data-driven methods (representation learning). These methods use several layers that are applied to ANNs to form deep neural networks (DNN) that require less human input on training. CNNs are a type of DNN particularly suited for image analysis, operating directly on the unprocessed images to extract imaging features that are relevant to the specific task (Fig 1).²³ They consist of an input layer (the image voxel values), intermediate “hidden” layers (computed representations feeding the process), and the output layer (outcome classification, eg, tumor vs nontumor). The latter DL methods have the advantage of not requiring preprocessing of image data or predefinition of image features, offering the ability to overcome some of the confines and problems associated with the current radiomics workflow.

Figure 1. From [23].

Convolutional neural network architecture for esophageal cancer ¹⁸F-FDG PET data in a vector composed from 4 convolutional (U) and 4 max pooling (V) layers. Different colour arrows in the first convolutional layer represent different learnable weight matrices. Coloured squares in the feature maps represent elements that include local spatial information from the previous layer. h – hidden layer, y_i – responder, y_k – nonresponder.

Challenges to Radiomic Analysis

One of the greatest challenges in the introduction of radiomics is the harmonization of scan acquisition, image processing, and software for analysis, so that results may be reproduced across centers and become practical and reliable for multicenter studies.

It has been shown that radiomic features vary in their dependency on technical acquisition and postprocessing factors with some features being more robust to these variations than others. Factors that can affect radiomic feature measurement include voxel size, VOI size, reconstruction algorithm, postreconstruction smoothing, scan acquisition time postinjection and quantization or binning and segmentation method, amongst others.^24–27

Some features, for example, first-order entropy, are relatively robust with little variation between different reconstruction parameters, remaining relatively independent of segmentation method²⁵ and showing good reproducibility, equivalent to or better than that of SUVs.²⁸ However, other features have been reported to show >30% variation when calculated after the use of 5 different reconstruction parameters.²⁴ Van Velden et al. investigated the repeatability of 105 intensity, shape and texture radiomic features from ¹⁸F-FDG-PET in patients with NSCLC scanned on 2 occasions and determined the effects of segmentation (manual vs threshold) and reconstruction (point spread function vs European Association of Nuclear Medicine guideline) methods²⁹. Sixty-three features showed an intraclass correlation coefficient > 90%. Twenty-five features were sensitive to segmentation method and only 3 were sensitive to reconstruction method. A further study also repeated ¹⁸F-FDG-PET scans in patients with NSCLC and of 91 radiomic features test-retest repeatability was high in 71% and inter-observer variation was 91%.³⁰

To some extent differences in scan acquisitions can be harmonized retrospectively to overcome multicenter effects. A method that was derived from a previously described genomic harmonization method has been reported to be an efficient method of doing this.³¹

PET is associated with relatively large voxel sizes compared to CT and MRI leading to greater challenges in measurement of radiomic features without bias or dependence on volume. Using probability theory, Brooks et al. estimated a volume of at least 45cm³ would be required to avoid dependence on volume when calculating second-order entropy derived from ¹⁸F-FDG-PET scans in patients with cervical cancer.³² However, another study of ¹⁸F-FDG-PET in patients with NSCLC showed dependence of second-order entropy at volumes less than 10cm³, suggesting lower tumor volumes may be applicable for radiomic analysis on modern scanners.³³

Several groups have recognized the risks of variation of radiomic methods in the literature and have published recommendations on radiomic analysis in PET and other imaging modalities in attempts to harmonize efforts in this field.^34,35 An international image biomarker standardization initiative has successfully standardized 164 PET radiomic features across 25 research teams using different software and is a successful initiative to enhance reproducibility in multiple centers.³⁶

Some publications have noted the weaknesses in some of the early radiomic literature, particularly with regards to statistical analysis^37,38 with the potential for false discovery. Other issues that have been raised include the potential for publication bias with studies showing positive correlations between radiomics and nonimaging characteristics predominating,³⁹ although negative reports do exist.⁴⁰ Recent guidance on minimum requirements for radiomic and AI-related imaging research has been published to ensure more robust data and confidence in this area of research.⁴¹ Notable inclusions are the requirement for independent training (against a reference standard), validation (fine tuning), and test (diagnostic performance) sets of data with a preference for data from another institution to be used for final testing of AI algorithms. One of the challenges with developing ML algorithms in PET is that large sets of image data with relevant information on tumor type and relevant clinical outcome data are uncommon and more difficult to acquire than in higher volume imaging modalities such as CT and MRI. Methods that may mitigate this challenge include transfer learning where DL models that have been developed and validated for another application can be used in a separate task to build the first layers of a CNN, saving time on training and building neural networks.⁴² Ethical issues and data protection are further challenges to obtaining large enough datasets for ML algorithm development.

Determining Biological and Molecular Phenotype With Radiomics

Underlying biological and molecular mechanisms of cancer (and other diseases) provide insight on tumor phenotype and how it may behave with regards aggressiveness and response or resistance to therapy. Depending on the specific tracer, PET can give macroscopic information on a particular aspect, for example, glucose metabolism with ¹⁸F-FDG or cellular proliferation with ¹⁸F-fluorothymidine. While global measures of molecular tumor characteristics provide information that is of clinical utility in every-day practice, data are emerging that the radiomic signature of ¹⁸F-FDG (and some other tracers) distribution within a tumor may reflect other hallmarks of tumor molecular phenotype, biology or genetics.

In preclinical models of head and neck cancer, the spatial distribution of ¹⁸F-FDG has been shown to be associated with tumor cellular density, stromal tissue, and necrosis⁴³ and in pancreatic and hepatoma models, the heterogeneity of ¹⁸F-FDG reflects glucose transporter and hexokinase distribution.^44,45 Similarly, in orthotopic breast cancer models the spatial distribution and density of cells on histology correlates with radiomic features derived from autoradiographic images of ¹⁸F-FDG distribution.⁴⁶ Heterogeneity of a therapeutic target, such as carcinoembryonic antigen (CEA) to ¹³¹I-anti-CEA antibody, can affect response to therapy as shown in colorectal tumor models of differing CEA expression heterogeneity as measured by multifluorescence.⁴⁷ This microscopic heterogeneity was also possible to determine in the same tumor models on a macroscopic scale by using 125I-A5B7 anti-CEA nanoSPECT imaging and second-order texture features, including entropy, energy, contrast, and homogeneity.⁴⁸ Using the same colorectal tumor model, it has been shown that change in radiomic features describing the variation in size of isometabolic patches of ¹⁸F-FDG activity are better predictors of response to antiangiogenic therapy (bevacizumab) than conventional size or global activity parameters.⁴⁹ In man, there have been reports of associations between radiomic features and mutational status in colorectal cancer including KRAS, TP53 and APC mutations.⁵⁰

Examples of the Use of Radiomics in NSCLC

A number of radiomic features have been shown to correlate with histopathological characteristics in NSCLC. Histopathological measurement of cell density has been shown to correlate positively with SUVmean and total lesion glycolysis (TLG) and inversely with skewness and kurtosis, and a correlation reported between fractal lacunarity (gaps between clusters of ¹⁸F-FDG activity) measured by both texture analysis and histopathologically, despite the difference in scales (Fig 2).⁵¹ Another small feasibility study showed that areas of relatively low ¹⁸F-FDG uptake in NSCLC showed larger areas of fibrosis.⁵² A study of multiple radiomic features in NSCLC showed an ability to predict tumor stage better than standard SUV, metabolic tumor volume and TLG metrics and that there was a correlation between some texture features and tumor proliferation measured by Ki67 immunohistochemistry.⁵³

Figure 2. ¹⁸F-FDG PET/CT of a highly metabolically active NSCLC (adenocarcinoma).

Specific values of several relevant variables are shown in the inset (top). The Hematoxylin slide (bottom left) shows a dense tumor cell population corresponding to the high SUVmean and negative skew of the ¹⁸F-FDG PET image. Binary image (bottom right) thresholded to differentiate high and low mean cell density (MCD) regions shows low pathological lacunarity (path-lac) as smaller sized black regions of low cellularity between white regions of high cellularity.

Attempts have also been made to differentiate NSCLC histological subtypes by ¹⁸F-FDG PET/CT radiomic characteristics. Studies have shown the ability to differentiate squamous cell carcinomas from adenocarcinomas with squamous cell tumors tending to show higher heterogeneity.^54,55

Additional radiomic data has been extracted from mediastinal lymph nodes in ¹⁸F-FDG PET scans of patients with NSCLC. A comparison of a CNN and 4 ML methods that used 13 standard (eg, SUV, size) and 82 texture features, showed similar overall accuracy in differentiating benign from malignant nodes between the CNN and best ML methods that used a combination of standard and texture features⁵⁶. The CNN method was more sensitive than radiologists’ interpretation (84% vs 73%) with similar specificity (88% vs 90%) with an additional advantage that the CNN method did not require segmentation for feature definition.

There is an increasing clinical need for imaging methods to stratify patients for checkpoint inhibitor blockade immunotherapy given the recognized limitations of immunohistochemical analysis of target positivity by biopsy and the high level of immune-related toxicities associated with this class of drugs. Imaging has advantages of being able to assess a whole primary tumor as well as its metastases. Specific SPECT and PET tracers are being developed to report directly on relevant targets such as PD1/PDL1^{57, 58} but radiomic methods from more ubiquitous ¹⁸F-FDG PET scans have been reported that may give relevant predictive information.

First-order entropy of ¹⁸F-FDG distribution has been shown to correlate with high a density of CD8 tumor-infiltrating lymphocytes in surgical specimens of NSCLC.⁵⁹ By using radiomic features extracted from ¹⁸F-FDG PET/CT in patients with NSCLC scheduled for checkpoint blockade immunotherapy, Mu et al. used a multiparametric radiomic signature derived by the least absolute shrinkage and selection operator (LASSO) method to predict those who would have a durable clinical benefit as well as progression-free and overall survival.⁶⁰ The same group also used a radiomic score from pretherapy ¹⁸F-FDG PET/CT scans to predict immunotherapy toxicity with an area under the receiver operating characteristic (AUROC) of 0.88 in a prospective validation cohort.⁶¹

Other studies have evaluated the ability of ¹⁸F-FDG PET radiomics to predict the molecular profile and genetic mutational state of NSCLC. This is of potential clinical utility as, for example, EGFR mutations are associated with an improved response to certain tyrosine kinase inhibitor (TKI) drugs. Radiomic signatures from ¹⁸F-FDG PET in NSCLC have been shown to be associated with mutation status^62–64 that could potentially be used to stratify patients for treatment. If imaging methods could predict mutational status in the primary tumor and metastases, they could complement tissue molecular profiling, particularly if serial measurements are required that would guide decisions during therapy.

A radiogenomic study between ¹⁸F-FDG PET and NSCLC in 25 patients reported a prognostic metagene signature (involved in cell cycle, proliferation, death, self-cognition pathways) that was associated with a multivariate ¹⁸F-FDG uptake feature, including SUVmax, SUV(variance) and SUV (principle components, including skewness and kurtosis). Both the metagene and radiomic signature were highly correlated with survival in external and validation cohorts tested.⁶⁵ Advanced tumor stage, grade, and invasion were associated with NF-κB protein expression that was in turn specifically associated with high ¹⁸F-FDG activity in a further report from the same group.⁶⁶

Radiomics of ¹⁸F-FDG PET/CT has also been investigated to determine the predictive ability with regards to treatment response and likelihood of tumor recurrence. Two studies have evaluated texture parameters in predicting response to TKIs in EGFR-mutated patients. The first showed that pretreatment heterogeneity predicted earlier progression and treatment failure⁶⁷ and another showed that a reduction in ¹⁸F-FDG PET first-order entropy at 6 weeks predicted CT response at 12 weeks and OS better than standard SUV metrics.⁶⁸ A further study of ¹⁸F-FDG PET scans from 93 patients who underwent curative resection for NSCLC used ML methods to develop a radiomic model that predicted recurrent disease (AUROC 0.956).⁶⁹ In patients with early-stage NSCLC treated with stereotactic radiotherapy, a radiomic model was developed that predicted local recurrence with high accuracy (91%).⁷⁰ A further study of 100 patients who had received stereotactic radiotherapy used a ML method with 722 radiomic features from ¹⁸F-FDG PET scans with the ability to predict nodal failure and survival better than clinical variables.⁷¹

Several other studies have used radiomic methods to predict disease- or progression-free and overall survival. Some studies used individual predefined texture features⁷² (Fig 3) and others developed radiomic models.⁷³ In a study of 201 datasets and 43 textural features using the LASSO method, it was possible to identify a single textural feature (SumMean) as an independent predictor of overall survival in large tumors treated with chemoradiotherapy⁷⁴.

Figure 3. From [72].

ROC curves for baseline ¹⁸F-FDG PET primary tumor coarseness, contrast, busyness, and complexity for identification of responders vs nonresponders in patients treated with chemoradiotherapy for NSCLC. [This research was originally published in JNM. Cook GJ, Yip C, Siddique M et al. Are Pretreatment ¹⁸F-FDG PET Tumor Textural Features in Non–Small Cell Lung Cancer Associated with Response and Survival After Chemoradiotherapy? J Nucl Med. 2013;54:19-26. © by the Society of Nuclear Medicine and Molecular Imaging, Inc.]

Combining radiomic features derived from tumor, tumor penumbra, and bone marrow was found to predict disease-free survival better than clinical factors in a further study.⁷⁵ However, not all investigators were able to find a successful prognostic model using preselected radiomic features or ML methods.^76,77

Conclusion

The science of radiomic analysis of medical images is progressing rapidly. From the origins of predefined texture features being used to complement standard image metrics, there is now much work using AI methods including DL that recognize and learn features within an image without a priori feature definition. There is increasing standardization and harmonization within the field that will allow multicentre studies to progress. With better access to large curated datasets and increasing computing power, there is hope that AI methods may complement radiological interpretation and provide additional information that will better characterize a disease process and increase predictive and prognostic ability. This has been shown across different cancer types, but research is most mature in NSCLC. However, despite the large amount of radiomic literature, the reported methodologies remain diverse and while as yet there is little impact in the clinic, there is hope and effort within academia and industry to translate radiomic tools into clinical use.