Radiomics in Oncological PET/CT: a Methodological Overview

Abstract

Radiomics is a medical imaging analysis approach based on computer-vision. Metabolic radiomics in particular analyses the spatial distribution patterns of molecular metabolism on PET images. Measuring intratumoral heterogeneity via image is one of the main targets of radiomics research, and it aims to build a image-based model for better patient management. The workflow of radiomics using texture analysis follows these steps: 1) imaging (image acquisition and reconstruction); 2) preprocessing (segmentation & quantization); 3) quantification (texture matrix design & texture feature extraction); and 4) analysis (statistics and/or machine learning). The parameters or conditions at each of these steps are effect on the results. In statistical testing or modeling, problems such as multiple comparisons, dependence on other variables, and high dimensionality of small sample size data should be considered. Standardization of methodology and harmonization of image quality are one of the most important challenges with radiomics methodology. Even though there are current issues in radiomics methodology, it is expected that radiomics will be clinically useful in personalized medicine for oncology.

Introduction

Tumors become heterogenous at both genetic and phenotypic levels during progression via branched evolution, which is called intratumoral heterogeneity (ITH) [1]. ITH is clinically important because it is one of the main causes of resistance to treatment due to bottle neck effect [2, 3]. Further, ITH can cause underestimation of mutational burden of tumors due to analyzing only small specimens containing limited amount of tumor cells [4]. Despite larger and multiple biopsies are needed to evaluate mutational characterization accurately, tumor sampling procedure is invasive and is limited to practice enough [5]. Therefore, the role of imaging is expected in personalized medicine, which can perform total region analysis and can be repeatedly performed by less invasive.

¹⁸F-Fluorodeoxyglucose (¹⁸F-FDG) positron emission tomography/computed tomography (PET/CT) imaging is successfully used in clinical practice, especially for oncology [6, 7]. Visual assessment of ¹⁸F-FDG PET/CT imaging by image specialists such as radiologists and nuclear medicine physicians can make diagnosis of tumor, staging, and response assessment after chemo- or radiotherapy [8]. The subjectivity of visual assessment causes large intra- and inter-observer variations of interpretation in medical imaging [9]. Quantification of imaging is increasingly highlighted to achieve objective interpretation [10]. Many studies have suggested clinical usefulness of standardized uptake value (SUV)-based PET parameters such as SUV_max, SUV_peak, and SUV_avg as well as metabolic tumor volume (MTV) and total lesion glycolysis (TLG) [11–17]. However, those parameters do not reflect information of metabolic ITH which can be estimated via evaluation of spatial distribution of metabolic activity in a tumor.

One of the most highlighted methods to quantify ITH from images is texture analysis. Different mathematical methods including texture analysis has been developed for pattern recognition of images since the early 1970s [18]. Rapid development of image analysis based on high-throughput computing enables to extract many kinds of quantitative phenotype features about shape, edge, and texture from digitalized medical images including ultrasonography, CT, magnetic resonance imaging, and PET/CT [19]. This field of medical study has been termed as ‘radiomics’ which aims maximizing the information extracted from medical images and building a model using machine learning to achieve better patient management [20]. For example, radiogenomics is a subfield of radiomics, which aims to predict genomic information of tumors using macroscopic phenotypes expressed on images, and is expected to do a complementary role in personalized medicine for oncology [20]. ¹⁸F-FDG PET/CT imaging shows spatial distribution of metabolic activity in a tumor, which is thought to be related to tumor phenotype and tumor microenvironment. Recently, many studies with ¹⁸F-FDG PET/CT imaging in oncology have focused on quantification of metabolic ITH and its clinical usefulness in prediction of treatment response or prognosis as well as diagnosis.

This review aims to help researchers who are interested in starting radiomics study to carry out high quality research. Researchers should understand details of radiomics methodology, because radiomics research is more methodologically complex than conventional PET/CT imaging research. This review provides an overview of workflow in radiomics study and critical insights into the emerging methodological issues which affect results of radiomics studies.

Analysis Pipeline

The conventional PET/CT analysis approach consists of the following: (1) PET/CT imaging: image acquisition and reconstruction; (2) preprocessing: tumor segmentation; (3) quantification: feature extraction (SUV_max, MTV, TLG, etc.); and (4) analysis: statistical analysis (Fig. 1a). The pipeline of radiomics analysis on ¹⁸F-FDG PET/CT is similar to the conventional PET/CT analysis but needs a few more steps such as intensity quantization during preprocessing and texture matrix design before feature extraction in quantification (Fig. 1b). Several user-friendly software packages provide functions for tumor segmentation and radiomics feature extraction [21–24]. Although it depends on the purpose of research, radiomics research uses machine learning in many cases while using statistical analysis. Given each step of radiomics analysis contains various options which affect results of radiomics features, it is essential to understand the method correctly and analyze them in a way that is appropriate for the purpose of the study.

Fig. 1.

The pipeline of radiomics analysis. The conventional PET/CT analysis approach is composed of imaging (acquisition and reconstruction), preprocessing (tumor segmentation), quantification (PET feature extraction: SUV, MTV and TLG) and statistical analysis (a). The pipeline of radiomics analysis is composed of imaging (acquisition and reconstruction), preprocessing (segmentation and quantization), quantification (histogram or texture matrix design and feature extraction), and analysis (statistics and/or machine learning) (b)

PET/CT Imaging

Influences of Image Acquisition

So far, most radiomics studies have been conducted in a single institution with retrospective manners. Each institution acquires PET/CT images using different machines with different imaging methods. Regardless of successive results for construction of prognosis-predicting model using texture features extracted from more than one PET/CT acquisition system [25, 26], texture features are known as being affected by settings of image acquisition and reconstruction methods. A study focusing on image acquisition setting of dual-time points, with peripheral nerve sheath tumors which had relatively low ¹⁸F-FDG uptake, showed that many texture features were changed significantly between early (101.5 ± 15.0 min) and late (251.7 ± 18.4 min) scans [27].

Several articles reported that respiratory motion affected values of the texture features significantly [28–31]. In addition, among different phase bins such as the end of inhalation, inhalation-to-exhalation, mid exhalation, end of exhalation, and exhalation-to-inhalation, the difference was low to moderate as coefficient of variation (CV) < 10% [28]. The influence of respiratory motion to the texture features of lung tumors was different according to the lesion location: e.g., minimal influence in upper lobe lesions and more significant influence in lower lobe lesions. Despite the significant difference of values in the texture features between whether respiratory gating or not, the impact on the prognostic values was not significant [30].

Influences of Image Reconstruction

The sensitivity to image reconstruction settings is feature-dependent [32–34]. Robust texture features in reconstruction settings can be considered as good parameters in radiomics analysis (Table 1) [34].Texture features were more affected by grid size than iteration number and full-width half-maximum filtration size [33]. While partial volume correction (PVC) had a smaller impact on texture features than delineation method, regional texture features were more affected by whether PVC or not than local texture features [35]. Area under curve of cumulative SUV-volume histogram (AUC-CSH) could contain more information after PVC [35, 36].

Table 1.

Robustness of texture features to reconstruction

Classification	Texture matrix	CV ≤ 5%	5% < CV ≤ 10%	10% < CV ≤ 20%	CV > 20%
1st order	Histogram	SUVavg/SULpeak/TLG/MTV/surface area/asphericity/entropy	SUVmax/SD/kurtosis	Skewness	Variance
2nd order	GLCM	Entropy/homogeneity/dissimilarity	Energy/contrast/IDM/LNE
Higher order	NGTDM		Coarseness/contrast/strength	Busyness/complexity
	GLRLM	SRE/LRE/RLNU/RP	HGRE/HGSRE	GLNUr/HGLRE	LGRE/LGSRE/LGLRE
	GLSZM	SZE/GLNUs/ZP	ZLNU/HGZE/HGSZE	LZE/LGZE/HGLZE	LGSZE/LGLZE

Modified from Shiri (2017) [31] with formal permission

CV coefficient of variation, GLCM gray-level co-occurrence matrix, GLNUr gray-level non-uniformity for run, GLNUs gray-level non-uniformity for size, GLRLM gray-level run-length matrix, GLSZM gray-level size-zone matrix, HGLRE high gray-level long-run emphasis, HGLZE high gray-level long-zone emphasis, HGRE high gray-level run emphasis, HGSRE high gray-level short-run emphasis, HGSZE high gray-level short-zone emphasis, HGZE high gray-level zone emphasis, IDM intensity difference moment, IFH intensity frequency histogram, LGLRE low gray-level long-run emphasis, LGLZE low gray-level long-zone emphasis, LGRE low gray-level run emphasis, LGSRE low gray-level short-run emphasis, LGSZE low gray-level short-zone emphasis, LGZE low gray-level zone emphasis, LRE long run emphasis, LZE long-zone emphasis, MTV metabolic tumor volume, NGTDM neighborhood gray-tone difference matrix, RLNU run-length non-uniformity, RP run percentage, SD standard deviation, SRE short-run emphasis, SUV standardized uptake value, SZE short-zone emphasis, TLG total lesion glycolysis, ZLNU zone-length non-uniformity, ZP zone percentage

Issues: the Need for Harmonization

Harmonization is crucial for multicenter trials or obtaining compatible research results because the image reconstruction conditions of different research institutes are different. PSF reconstruction with Gaussian filtering harmonizes texture features with those obtained ordered subset expectation maximization. Since loss of information occurs in the filter process, it should be considered that the unfiltered images can be more discriminative in radiomics analysis [37]. A recent study demonstrated that using the ComBat method, initially developed for solving the problem of batch effect in genomic analysis, successfully harmonized the data of different reconstructions settings by removing the estimated center effect without loss of the pathophysiological information [38].

Tumor Segmentation

Recognition and Delineation

The process of tumor segmentation can be categorized into two sequential phases; recognition and delineation [39]. In the recognition phase, a target lesion with high uptake is identified and distinguished from other target lesion-like entities in the PET image. In the delineation phase, the target lesion is precisely separated from the background and non-significant object. The segmentation in the PET image is affected from intrinsic and extrinsic factors including spatial resolution and noise as well as shape, texture, and location of pathologies [40]. Low resolution and high smoothing make it difficult to define the precise segmentation of the tumor in PET images. Even though recent explosive use of hybrid imaging including PET/CT and PET/MR which improves tumor segmentation in the PET image mainly via improvement in the recognition phase, the challenging issues in the delineation phase are still unresolved.

Tumor Delineation Methods for PET Images

There are a wide variety of tumor delineation methods such as those which are manual, thresholding-based, stochastic and learning-based, and boundary based, which make different results [40]. So far, none have been agreed as the best tumor delineation methods for PET radiomics research. This is because, as discussed later, there is no consensus on whether to include the necrotic portion in PET radiomics studies. The manual delineation method is an intuitive and simple process, being the most common way to obtain surrogate truths if experts carry out the delineation process [41, 42]. However, it is a highly subjective and laborious method, related to high intra- and inter-operator variations as well as less reproducible products [43–47]. The thresholding methods, including fixed- and adaptive thresholding delineations, are rapid as well as simple and intuitive. A predefined cutoff value of SUV of 2.5 is one of the most commonly used fixed thresholding values for malignant tumor delineation [48–51]. Fixed relative thresholding allowing voxels with higher intensities than 40% of the SUV_max is another most commonly used fixed thresholding method in a clinical setting [48, 49, 51]. Fixed thresholding does not consider background intensities, which leads to inaccurate delineation for tumors with small volume and low metabolic activity. To correct the background-related errors in tumor delineation, it has been suggested to use various adaptive thresholding methods, which were based on phantom data. [48, 52–55]. The most commonly discussed adaptive thresholding method in the radiomics field was the Nestle’s method, determining threshold (T) by a function of tumor metabolic activity and background intensities [48, 51, 56, 57]

$$T=\left(\beta \times {\mathrm{SUV}}_{\text{tumor}\_\text{mean}\_70\%\text{SUVmax}}\right)+{\mathrm{SUV}}_{\text{background}\_\text{mean}},\text{with}\mathrm{\beta }=0.15\text{or}0.30.$$

Regardless of simplicity and general applicability, the thresholding methods are known to be underestimating tumor volume as well as susceptible to contrast variation, noise, and heterogeneity [48, 58, 59]. Several algorithm-based delineation methods including fuzzy C-Means (FCM) [60] and fuzzy locally adaptive Bayesian (FLAB) [61] have been proposed for advanced automatic tumor delineation which showed better accuracy as well as less dependency on noise and image quality. The FLAB algorithm showed superior delineation results in small sized lesions less than 2 cm than the FCM algorithm [61]. Another advanced method is a gradient-based tumor delineation approach which showed more accurate and robust results than those of the thresholding methods [62, 63]. Meanwhile, CT-based anatomical tumor delineation showed significantly larger tumor volume than those of PET-based methods [59]. An example of different tumor delineation results is presented in Fig. 2.

Fig. 2.

Tumor segmentation using different delineation methods. Different results are presented by four different methods of tumor delineation, including gradient (PETedge, purple), fixed absolute thresholding with SUV of 2.5 (red), fixed relative thresholding with 40% of SUV_max (T_40%, blue), and adaptive thresholding (the Nestle’s method, green) methods

Issues: Inclusion or Exclusion of Necrotic Portion

One specific issue regarding the tumor delineation process in radiomics research is about inclusion/exclusion of inner necrotic areas with low ¹⁸F-FDG uptake similar to background activity. For instance, contouring-based delineations such as the gradient-based methods include inner non-hypermetabolic areas [62, 64]. On the other hand, clustering-based methods exclude inner non-hypermetabolic areas [64, 65]. Thresholding-based methods exclude inner non-hypermetabolic areas while the threshold is sufficiently high. Given that more aggressive tumor makes more necrotic portions during proliferation, inclusion of necrotic portions would be helpful to assess aggressiveness of a tumor via texture analysis. On the other hand, those necrotic portions are no more metabolically active, and the approach focusing on hypermetabolic areas may be also reasonable.

Issues: Sensitive Results of Radiomics Features to Delineation Methods

Radiomics features are sensitive to different delineation methods [35]. Therefore, one tumor delineation method may not be sufficient for radiomics analysis. The recent trend of tumor segmentation in radiomics research is multiple-segmentation which uses various tumor delineation methods from manual delineation to various automatic delineations [66]. Different delineation methods can be used as a validation method as far as delineation related sensitivity issue of radiomics features [67]. To accomplish advanced level of radiomics analysis, it is recommended to applicate multiple segmentations and clarify how segmentation is performed.

Intensity Quantization

Relative and Absolute Quantization

Voxel intensity quantization, also known as SUV discretization or resampling, is the routine preprocessing step before second or higher order texture feature extraction in radiomics analysis. Quantization reduces the effect of noise in radiomics analysis, by changing the substantially continuous voxel intensity to a discontinuous value. In radiomics analysis using ¹⁸F-FDG PET/CT images, two different approaches have been suggested for SUV quantization, namely the relative quantization (RQ) known as the fixed bin number (FBN) method and the absolute quantization (AQ) also called as the fixed bin size (FBS) method.

The RQ method which resamples a voxel value, I(x), via discretization of the SUV range (SUV_max − SUV_min) inside the segmented tumor by a predetermined bin number (D; e.g., 16, 32, 64, 128, 256) to obtain a resampled voxel value (I_RQ(x)) as the following formula [68]:

$$I_{\mathit{RQ}}\left(x\right)=\mathit{\text{round}}\left(D\times \frac{I\left(x\right)-{\mathit{SUV}}_{\mathit{min}}}{{\mathit{SUV}}_{\mathit{max}}-{\mathit{SUV}}_{\mathit{min}}+1}\right)$$

One of the most appropriate and widely used bin numbers for the RQ is 64, because intensity resolution would be sufficiently fine, less than 0.25 of SUV in a case with the SUV range of a tumor from 4 to 20 [68, 69]. Another study has suggested that 32 numbers of bins are appropriate because values of several texture features including entropy calculated in gray-level co-occurrence matrix (Entropy_GLCM) are substantially affected by a number of bins, especially when the FBN is less than 32 [51].

Recently, the alternative quantization method, the AQ, has been suggested not to discard the absolute ¹⁸F-FDG uptake information [70, 71]. The AQ method, resampling I(x), intensities of voxel x, to I_AQ(x) with a FBS (B; e.g., 0.1, 0.25, 0.5, or 1 of SUV), uses the following formula [71]:

$$I_{\mathrm{AQ}}\left(x\right)=\mathit{\text{round}}\left(\frac{I\left(x\right)}{B}\right)-\mathit{min}\left(\mathit{\text{round}}\left(\frac{I\left(x\right)}{B}\right)\right)+1$$

Another option for the AQ process has been introduced using a high bound (HB; e.g., 15, 20, 25 SUV unit) with discrete values (D; e.g., 64) as [72]:

$$I_{\mathit{AQ}2}\left(x\right)=\mathit{\text{round}}\left(D\times \frac{I\left(x\right)}{\mathit{HB}}\right)$$

For example, when choosing a high bound of 15, 20, or 25 SUV unit with 64 discrete values with this formula, the intensity resolution is approximately 0.2, 0.3, or 0.4, respectively.

The RQ and AQ methods mostly result in discordant values of texture features to each other [71]. The RQ method treats PET images as dimensionless, which induces heterogenous intensity resolution dependent on the SUV range within tumors on different PET images. In these cases, the voxel–intensity differences within the tumor with relatively low SUV_max would be exaggerated compared to those in the tumor with higher SUV_max (Fig. 3). Therefore, values of texture features based on the RQ method may not be directly comparable with each other in different PET images. The AQ method induces identical intensity resolution among different PET images, which enables direct comparison of values of texture features in different PET images, even in a clinical response setting that the SUV range of a tumor varies during treatment [71]. While the values of texture features from the RQ method were counter-intuitive to the visual assessment of heterogeneity, those from the AQ method were more correlated in the simulated and clinical data of breast cancer patients [73]. For these reasons, the AQ method seems to be more appropriate than the RQ method for metabolic radiomics analysis using ¹⁸F-FDG PET/CT images. The AQ method also had better repeatability and lower sensitivity of texture features to the methods of delineation and reconstruction in a prospective study with double baseline ¹⁸F-FDG PET/CT scans with a 3-day interval [74].

Fig. 3.

Examples of resampled PET images using RQ and AQ methods. The 16-bin resampled voxel intensities within tumors are presented as different colors (see color bars). Even though the tumor in case II has relatively lower SUVs than the tumor in case I, the tomography of case II (RQ) is exaggerated and looks similar to that of case I (AQ or RQ). Applying the AQ method, the tumors in cases I and II show different tomographies. AQ absolute quantization, I_AQ resampled intensity by the AQ method, I_RQ resampled intensity by the RQ method, RQ relative quantization

Issues: Dependence on SUV_max and/or MTV

The influence of the quantization methods on values of texture features is highly related to the dependency to the conventional PET parameters, tumor SUV_max and/or MTV. Several studies commonly found a trend that most texture features based on the RQ method have the dependency of their values on MTV [64, 72, 75–77]. Because the dependence on MTV is much sensitive particularly for tumors with relatively small size, MTVs of 10 cm³, 45 cm³, or 60 cm³ have been suggested as cutoffs to find valuable complementary information to MTV in radiomics analysis using the RQ method [72, 75, 76]. Meanwhile, it has been highlighted that the dependence of AQ-based texture features on not only MTV but also SUV_max (Fig. 4) [50, 64, 72, 77]. As the higher mountain topography is more tough and complicated, the dependence on SUV_max in tumor is considered to be one of the unique characteristics of the texture features of PET images. Actually, the RQ method is regarded as a kind of intensity normalization to the maximum intensity within a tumor. Therefore, it seems artificial to normalize and eliminate the dependence on SUV_max by using the RQ method. It might be more effective to find additive meaning of texture features based on the AQ method in multivariable analysis.

Fig. 4.

Dependence on SUV_max and MTV of AQ-based texture features. The correlogram of 109 texture features shows dependence on conventional PET parameters, including SUV_max and MTV (p < 0.0005). AQ absolute quantization, MTV metabolic tumor volume, SUV_max maximum standardized uptake value, TLG total lesion glycolysis. Note: Color bar, Pearson correlation coefficient. Modified from Ha (2017) [47], which is licensed under a Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/)

Texture Matrix Design and Feature Extraction

The First Order: Histogram-Based

Texture features are categorized into first, second, and higher orders by the number of voxels involved in their texture matrix design [78]. The first-order texture features are mainly calculated from intensity frequency histogram which represents the frequency distribution of one-voxel intensity in a delineated tumor. Therefore, the first-order texture features describe global characters of a tumor on PET images. The first-order texture features include CV, skewness, kurtosis, entropy_histo, and energy as well as conventional PET parameters like the SUV_max, SUV_avg, MTV, and TLG (Table 2). A commonly used heterogeneity-associated parameter, the CV which is defined as the standard deviation (SD) over SUV_avg in a delineated tumor [79], is also the first-order texture features derived from intensity frequency histogram (IFH) (Fig. 5a, b). There is another option to calculate global heterogeneity using the CSH, also known as the intensity volume histogram (IVH), having a decline curve form (Fig. 5a, c) [80]. The CSH describes percent of total tumor volume above percent threshold of SUV_max, which threshold varies from 0 to 100%. Lower AUC-CSH indicates more heterogeneity in a tumor [36].

Table 2.

First-order texture features

Feature	Definition	Description
SUVmax, SUVavg, MTV, TLG	–	Conventional PET parameters
CV	Standard deviation of SUVs/SUVavg	A global heterogeneity marker. Higher values indicate more heterogeneity
AUC-CSH	Percent of total tumor volume above percent threshold of SUVmax, which threshold varies from 0 to 100%.	A global heterogeneity marker. Lower values indicate more heterogeneity
Skewness	$\frac{\frac{1}{E}{\sum }_i{\left(\mathit{\text{HISTO}}\left(i\right)-\overline{\mathit{\text{HISTO}}}\right)}^3}{{\left(\sqrt{\frac{1}{E}{\sum }_i{\left(\mathit{\text{HISTO}}\left(i\right)-\overline{\mathit{\text{HISTO}}}\right)}^2}\right)}^3}$ (with E = the total number of voxels in the VOI, $\overline{\mathit{\text{HISTO}}}$= the average of gray levels in the histogram)	The asymmetry of the gray-level distribution in the intensity frequency histogram
Kurtosis	$\frac{\frac{1}{E}{\sum }_i{\left(\mathit{\text{HISTO}}\left(i\right)-\overline{\mathit{\text{HISTO}}}\right)}^4}{{\left(\frac{1}{E}{\sum }_i{\left(\mathit{\text{HISTO}}\left(i\right)-\overline{\mathit{\text{HISTO}}}\right)}^2\right)}^2}$ (with E = the total number of voxels in the VOI, $\overline{\mathit{\text{HISTO}}}$ = the average of gray levels in the histogram)	The sharpness of the peak of the gray-level distribution in the intensity frequency histogram
Entropyhisto	$\sum _{i}p\left(i\right)\bullet log\left(p\left(i\right)+\epsilon \right)$	The randomness of the gray-level distribution in the intensity frequency histogram

AUC-CSH, area under curve of cumulative SUV-volume histogram; CV, coefficient of variation; MTV, metabolic tumor volume; SUV, standardized uptake values; SUV_avg, average of SUVs; SUV_max; maximum of SUVs; TLG, total legion glycolysis; VOI, volume of interest

Fig. 5.

2D example of texture matrix design. Using a 2D sample matrix with four bins of pixel intensity (4 × 4 size (a)), various texture matrices including IFH (b), CSH (c), GLCM (d), NGTDM (e), GLRLM (f), and GLSZM (g) were computed. CSH cumulative SUV-volume histogram, d distance, GLCM gray-level co-occurrence matrix, GLRLM gray-level run-length matrix, GLSZM gray-level size-zone matrix, IFH intensity frequency histogram, NGTDM neighborhood gray-tone difference matrix, W window size

The Second Order: GLCM-Based

The second order texture features are calculated based on GLCMs also known as spatial gray-level dependence matrices (SGLDMs) which compute for the second order joint probabilities, namely the local association of intensities of 2 voxels in a delineated tumor [78]. The information on GLCMs is about how many pairs of voxels of resampled intensities (i, j) which satisfy a certain relationship of a defined direction and distance (Fig.5a, d) [81]. Theoretically, multiple GLCMs could be computed for a single PET case with various distances and directions. The one-voxel distance has been usually chosen to compute GLCMs in PET radiomics analysis [68, 71]. Given 26 voxels surrounding a voxel in 3D space, 13 directions are usually used to compute GLCMs in 3D space. The difference of “symmetry” and “asymmetry” methods is whether the number of paired voxels including in the reverse direction is counted or not. Two methods have been suggested for GLCM-based 3D texture feature extraction; one is using 13 GLCMs for each direction in 3D space to calculate texture features independently and then average them; the other one is using 1 GLCM in consideration of neighbor voxels for all 13 directions directly in 3D space (Fig. 6) [64, 76]. In regard to the issue of volume-dependence of the RQ-based texture features, the single GLCM method showed less volume dependence than the 13 GLCMs with averaging method [64, 76]. The normalized GLCM can be computed as following formula [23]:

$${\text{GLCM}}_{d_x,d_y}\left(i,j\right)=\frac{1}{\mathit{\text{Pairs}}}\sum _{p=1}^{n-d_x}\begin{array}{c}\sum _{q=1}^{m-d_y}\left\{\begin{array}{c}1\mathit{\text{if}}I\left(p,q\right)=i\mathit{\text{and}}I\left(p+d_x,q+d_y\right)=j\\ 0\mathit{\text{otherwise}}\end{array}\right)\\ \end{array}$$

Fig. 6.

Approach to extract the multi-directional GLCM-based texture feature. Two ways extracting the GLCM-based texture feature in multi-directions are presented. The first method, with averaging, extracts a multi-directional texture feature (TF_13GLCMs) by averaging every texture feature on each direction. The second method, without averaging, extracts another multi-directional texture feature (TF_1GLCM) from a multi-directional GLCM. GLCM gray-level co-occurrence matrix, TF a texture feature

Table 3 summarizes GLCM-based texture features including homogeneity, energy, contrast, correlation, entropy, and dissimilarity.

Table 3.

GLCM-based features

Name	Definition	Description
Normalized GLCM	Normalized ${\text{GLCM}}_{d_x,d_y}\left(\mathrm{i},\mathrm{j}\right)=\frac{1}{\text{Pairs}}\sum _{p=1}^{n-d_x}\begin{array}{c}\sum _{q=1}^{m-d_y}\left\{\begin{array}{c}1\mathit{\text{if}}I\left(p,q\right)=i\text{and}I\left(p+d_x,q+d_y=j\right)\\ 0\text{otherwise}\end{array}\right)\\ \end{array}$	2nd-order texture matrix
HomogeneityGLCM	$\sum _i\sum _j\frac{\text{GLCM}\left(i,j\right)}{1+\mid i-j\mid }$	The homogeneity of gray-level voxel pairs
EnergyGLCM	$\sum _i\sum _j\text{GLCM}{\left(i,j\right)}^2$	The uniformity of gray-level voxel pairs
ContrastGLCM	$\sum _i\sum _j\text{GLCM}{\left(i-j\right)}^2\bullet \text{GLCM}\left(i,j\right)$	The local variations in the GLCM
CorrelationGLCM	$\sum _i\sum _j\frac{\left(i-{\mu }_i\right)\bullet \left(j-{\mu }_j\right)\bullet \text{GLCM}\left(i,j\right)}{{\sigma }_i\bullet {\sigma }_j}$	The linear dependency of gray levels in GLCM
EntropyGLCM	$-\sum _i\sum _j\text{GLCM}\left(i,j\right)\bullet log\left(\text{GLCM}\left(i,j\right)+\epsilon \right)$	The randomness of gray-level voxel pairs
DissimilarityGLCM	$\sum _i\sum _j\mid i-j\mid \bullet \text{GLCM}\left(i,j\right)$	The variation of gray-level voxel pairs

GLCM gray-level co-occurrence matrix

The Higher Order: NGTDM-Based

The higher order texture features are calculated from several texture matrices computed based on interrelationships of 3 or more voxels [78]. Neighborhood gray-tone difference matrices (NGTDMs) are one of the higher order texture matrices, which compute for differences between each voxel and neighboring voxels within a certain distance (Fig. 5a, e). When a distance for NGTDMs are 1 voxel, a centered voxel and 8 surrounding voxels are constructing a 3 × 3 neighborhood group, namely 3 voxels as the window size. NGTDMs calculate sum of the absolute difference between centered voxel level and average of the neighboring voxels for g. The window size (W) of 3 voxels is mostly used [82, 83]. The NGTDM can be computed as follows [23]:

$$\mathit{\text{NGTDM}}\left(i\right)=\sum _p\sum _q\left\{\begin{array}{c}\left|\overline{M}\left(p,q\right)-i\right|\mathit{\text{if}}I\left(p,q\right)=i\\ 0\mathit{\text{else}}\end{array}\right)$$

$$\left(\mathit{\text{with}}\overline{M}\left(p,q\right),\mathit{\text{the average of the neighbour voxel intensities}}\right)$$

NGTDM-based texture features including coarseness, contrast, and busyness, which represent local texture within a delineated tumor, and the measures showed good correspondences with human’s visual perception [84]. Table 4 summarizes NGTDM-based features.

Table 4.

NGTDM-based features

Name	Definition	Description
NGTDM	$\text{NGTDM}\left(i\right)=\sum _p\sum _q\left\{\begin{array}{c}\left|\overline{M}\left(p,q\right)-i\right|\text{if}I\left(p,q\right)=i\\ 0\text{else}\end{array}\right)$ (with $\overline{M}\left(p,q\right),$ the average of the neighbor voxel intensities)	Higher order, local texture matrix
CoarsenessNGTDM	$\frac{1}{{\sum }_{i}p\left(i\right)\bullet \text{NGTDM}\left(i\right)}$	The level of spatial rate of change in intensity
ContrastNGTDM	$\left[\sum _i\sum _{j}p\left(i\right)\bullet p\left(j\right)\bullet {\left(i-j\right)}^2\right]\bullet \frac{{\sum }_i\text{NGTDM}\left(i\right)}{E\bullet G\bullet \left(G-1\right)}$ (with E = the number of voxels in VOI, G = the number of gray levels)	The intensity difference between neighboring regions of voxels
BusynessNGTDM	$\frac{{\sum }_{i}p\left(i\right)\bullet \text{NGTDM}\left(i\right)}{{\sum }_i{\sum }_j\mid i\bullet p\left(i\right)-j\bullet p\left(j\right)\mid }$ (with p(i) ≠ 0, p(j) ≠ 0)	The spatial frequency of changes in intensity

NGTDM neighborhood gray-tone difference matrix

The Higher Order: GLRLM and GLSZM-Based

Gray-level run-length matrices (GLRLMs) are statistical higher order texture matrices and measure for runs (l) of voxels with same gray-level (i) along a given direction (Fig. 5a, f) [85]. The counting process of GLRLMs is symmetrical between a given direction and its reverse; only 13 directions in 3D space are needed to be considered, consequently [86]. Extracting GLRLM-based texture features in 3D space is similar to that of GLCM. One way to obtain a value of GLRLM-based texture feature in 3D images is that averaging the texture values separately calculated in each matrix in 13 directions [87]. The other way is that simultaneously considering 13 directions to compute 1 GLRLM [88]. Gray-level size zone matrices (GLSZMs) are alternative to GLRLMs, considering the size (s) of continued voxels with same gray level (i) rather than the run length of voxels with same gray level in a certain direction (Fig.5a, g) [89]. Therefore, the GLSZM has a variable number of columns determined by the size of the largest zone [90]. Computing GLSZMs does not need to consider directions like computing GLCMs or GLRLMs. Tables 5 and 6 summarize GLRLM and GLSZM-based features.

Table 5.

GLRLM-based features

Name	Definition	Description
GLRLM	GLRLM(i, j), the number of runs with intensity i of homogenous runs of j voxels in some direction (where H corresponds to the number of homogenous runs)	Higher order, regional texture matrix
Short-run emphasis (SRE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLRLM}\left(i,j\right)}{j^2}$	The distribution of short runs, which is larger for fine textures
Long-run emphasis (LRE)	$\frac{1}{H}\sum _i\sum _j\text{GLRLM}\left(i,j\right)\bullet j^2$	The distribution of long runs, which is larger for coarse textures
Low gray-level run emphasis (LGRE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLRLM}\left(i,j\right)}{i^2}$	The distribution of low gray-level runs
High gray-level run emphasis (HGRE)	$\frac{1}{H}\sum _i\sum _j\text{GLRLM}\left(i,j\right)\bullet i^2$	The distribution of high gray-level runs
Short-run low gray-level emphasis (SRLGE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLRLM}\left(i,j\right)}{i^2\bullet j^2}$	The distribution of the short homogeneous runs with low gray levels
Short-run high gray-level emphasis (SRHGE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLRLM}\left(i,j\right)\bullet i^2}{j^2}$	The distribution of the short homogeneous runs with high gray levels
Long-run low gray-level emphasis (LRLGE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLRLM}\left(i,j\right)\bullet j^2}{i^2}$	The distribution of the long homogeneous runs with low gray levels
Long-run high gray-level emphasis (LRHGE)	$\frac{1}{H}\sum _i\sum _j\text{GLRLM}\left(i,j\right)\bullet i^2\bullet j^2$	The distribution of The long homogeneous runs with high gray levels
Gray-level non-uniformity for run (GLNUr)	$\frac{1}{H}\sum _i{\left(\sum _j\text{GLRLM}\left(i,j\right)\right)}^2$	The non-uniformity of the gray levels of the homogeneous runs
Run length non-uniformity (RLNU)	$\frac{1}{H}\sum _j{\left(\sum _i\text{GLRLM}\left(i,j\right)\right)}^2$	The non-uniformity of the length of the homogeneous runs
Run percentage (RP)	$\frac{H}{{\sum }_i{\sum }_j\left(j\bullet \text{GLRLM}\left(i,j\right)\right)}$	The homogeneity of the homogeneous runs

GLRLM gray-level run-length matrix

Table 6.

GLSZM-based features

Name	Definition	Description
GLSZM	GLSZM(i, j), the number of homogeneous zones of j voxels with the intensity i (where H corresponds to the number of homogenous zones)	Higher order, regional texture matrix
Short zone emphasis (SZE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLSZM}\left(i,j\right)}{j^2}$	The distribution of the short homogeneous zones
Long-zone emphasis (LZE)	$\frac{1}{H}\sum _i\sum _j\text{GLSZM}\left(i,j\right)\bullet j^2$	The distribution of the long homogeneous zones
Low gray-level zone emphasis (LGZE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLSZM}\left(i,j\right)}{i^2}$	The distribution of the low gray-level zones
High gray-level zone emphasis (HGZE)	$\frac{1}{H}\sum _i\sum _j\text{GLSZM}\left(i,j\right)\bullet i^2$	The distribution of the high gray-level zones
Short-zone low gray-level emphasis (SZLGE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLSZM}\left(i,j\right)}{i^2\bullet j^2}$	the distribution of the short homogeneous zones with low gray levels
Short-zone high gray-level emphasis (SZHGE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLSZM}\left(i,j\right)\bullet i^2}{j^2}$	the distribution of the short homogeneous zones with high gray levels
Long-zone low gray-level emphasis (LZLGE)	$\frac{1}{H}\sum _i\sum _j\frac{\text{GLSZM}\left(i,j\right)\bullet j^2}{i^2}$	the distribution of the long homogeneous zones with low gray levels
Long-zone high gray-level emphasis (LZHGE)	$\frac{1}{H}\sum _i\sum _j\text{GLSZM}\left(i,j\right)\bullet i^2\bullet j^2$	The distribution of the long homogeneous zones with high gray levels
Gray-level non-uniformity for zone (GLNUz)	$\frac{1}{H}\sum _i{\left(\sum _j\text{GLSZM}\left(i,j\right)\right)}^2$	The non-uniformity of the gray levels of the homogeneous zones
Zone length non-uniformity (ZLNU)	$\frac{1}{H}\sum _j{\left(\sum _i\text{GLSZM}\left(i,j\right)\right)}^2$	The non-uniformity of the length of the homogeneous zones
Zone percentage (ZP)	$\frac{H}{{\sum }_i{\sum }_j\left(j\bullet \text{GLSZM}\left(i,j\right)\right)}$	The homogeneity of the homogeneous zones

GLRLM gray-level run-length matrix

Issues: Reproducibility of Texture Features

Several studies have explored the issues of the repeatability of texture features using test–retest data sets using the same setting of image acquisition, reconstruction, segmentation, and quantization. A test–retest study, with the RQ preprocessing, proved that most GLCM texture features including entropy, homogeneity, and dissimilarity, and some GLSZM texture features including intensity variability and size zone variability had comparable reproducibility to SUV_max. However, several GLSZM-features including small area emphasis and low intensity emphasis had low reproducibility [91]. Another test–retest study, using the AQ preprocessing with 0.5 SUV intensity resolution, proved that the majority of texture features had a high reproducibility, although GLSZM texture features showed the overall lowest reproducibility compared with global and GLCM texture features [70]. A greater number of GLCM- and GLRLM texture features had better repeatability with the AQ preprocess compared with that of the RQ preprocess [74]. CV and AUC-CSH, which are ITH-related global texture features, had moderate-to-high reproducibility. Other global texture features, such as skewness and kurtosis, had various levels of reproducibility from low to high [70, 91, 92].

Analysis for Radiomics Data

Statistics

The traditional approach to the clinical image research is driven by medical statistics. This approach is still valuable while focusing on the clinical usefulness of ITH-related parameters from images. For this purpose, several ITH-related parameters carefully selected by researchers would be tested to identify certain clinical importance such as the role of prognostic markers for drug resistance [93, 94]. Multiple comparison problems should be considered when using multiple parameters such as radiomics features. To correct the multiple comparison problems, P values in statistics should be adjusted by post-hoc analysis for the multiple tested parameters [95]. The Bonferroni correction, having type I error rate α/m for each of m tests, is the most conservative and commonly used method to correct the multiple comparison issue. Sometimes, the Bonferroni’s correction is too conservative to find the clinical significance within given samples. Various methods have been suggested, such as less conservative post-hoc analyses, e.g., the Holm’s procedure and false-discovery rate, which use sequentially adjusted P values [96, 97]. In addition, multivariable analysis is recommended to evaluate additional usefulness and independent significance of texture features compared with SUV_max and MTV, on which the parameters have dependence, as well as already established clinical parameters.

Machine Learning

Machine learning methods used in radiomics researches are classified as supervised learning and unsupervised learning [98]. Supervised learning is used to solve problems of regression predicting continuous variables, and classification predicting categorical variables. Supervised learning trains a function f_w based on a labeled training data set consisting of a paired input (x) and desired output (y). There are various types of supervised learning algorithms including support vector machines, linear regression, logistic regression, linear discriminant analysis, decision trees, k-nearest neighbor algorithms, and neural networks. The typical data structure of supervised learning is composed of three subsets: the training set fitting models, the validation set estimating prediction error for model selection, and the test set assessing the generalization error of the final chosen model [99]. The validation set is also regarded as a part of model training and a test set should be separated from training parts. Enough samples are needed for each set to get the best learning and testing results. If the data set is not enough to split into training and testing parts, the cross-validation method could be more efficient for use in order to validate the established model. In k-fold cross-validation, the data set D is randomly partitioned into k mutually exclusive subsets D₁, D₂, …, D_k of equal size. In each of learning experiments, one of those subsets is picked as a test set and the remaining subsets then as a training set. Learning experiments and testing performances are performed k separate times and the performance of k-fold cross-validation is obtained by averaging the test results from the k different learning experiments [100]. Unsupervised learning is used for unlabeled data to find hidden structure of data set. Typical algorithms include k means clustering, hierarchical clustering, etc. To perform clustering, it is important to determine how represent a cluster of more than one point and how calculate the distance between clusters. A cluster center called centroid is often used to represent each cluster. Most commonly used distance functions are Euclidean distance and Manhattan distance.

Issues: High Dimensionality in Machine Learning with Radiomics Data

One thing should be considered in radiomics research using machine learning is the high-dimensionality of radiomics feature space. Although hundreds of radiomics features could be extracted from cases, most radiomics studies usually have a restricted number of cases for analysis usually no more than 200. The small sample size but high dimensionality causes lower information density in radiomics research, which results in the higher risk of an overfitted model. This is related to one of well-known issues in the machine learning field, namely “The Curse of Dimensionality” [101]. As the number of features or dimensions grows, the amount of data needed to generalize accurately grows exponentially. As the dimensionality grows over the optimal number of features without increasing the number of training samples, the classifier’s performance decreases. To avoid the curse of dimensionality, training machine with carefully selected features is the most commonly used strategies in radiomics field [87, 102]. Further, highly correlated features provide redundant rather than complementary information; it can be used as a basis for selecting features in radiomics analysis [28, 103]. Non-linear classifiers are more vulnerable to overfitting due to high dimensionality than linear and simple classifiers [104]; therefore, non-linear classifiers are more restricted in feature number. Another approach like dimensionality reduction such as the principal component analysis, by combining the original feature values to have the largest variance, would help to manage the problems [105]. However, combined features with the largest variance do not guarantee having the most discriminative information.

Conclusions

Radiomics studies have been conducted in two perspectives: ITH assessment and high-throughput extraction of information. Radiomics feature extraction is less intuitive than conventional PET parameters and served as an entry barrier to this field of study. All the steps including PET/CT imaging, preprocessing, matrix design and feature extraction, and statistics/modeling affect the quality of radiomics analysis. Therefore, it is important to understand the issues of metabolic radiomics including tumor segmentation, quantization, robustness/repeatability, and dependence on SUV_max/MTV. Further, statistical validity should be ensured with cares in analyzing hundreds of parameters. Based on the successful validation of the clinical role of radiomics by individual studies, multicenter research or meta-analysis is expected to be actively pursued in a near future. In this regard, standardization of methodology and harmonization of radiomics results will become more important in later studies. Despite the pending issues of methodology, radiomics is expected to be clinically useful in precision medicine for oncology.

Abbreviations

AQ

Absolute quantization

AUC-CSH

Area under curve of cumulative SUV-volume histogram

CSH

Cumulative SUV-volume histogram

CV

Coefficient of variation

FBN

Fixed bin number

FBS

Fixed bin size

FCM

Fuzzy C-means

FDR

False-discovery rate

¹⁸F-FDG

¹⁸F-fluorodeoxyglucose

FLAB

Fuzzy locally adaptive Bayesian

GLCM

Gray-level co-occurrence matrix

GLRLM

Gray-level run-length matrix

GLSZM

Gray-level size zone matrix

IFH

Intensity frequency histogram

ITH

Intratumoral heterogeneity

IVH

Intensity volume histogram

MTV

Metabolic tumor volume

NGTDM

Neighborhood gray-tone difference matrix

PET/CT

Positron emission tomography/computed tomography

PVC

Partial volume correction

RQ

Relative quantization

SAM

Second angular moment

SGLDM

Spatial gray-level dependence matrix

SUV

Standardized uptake value

TLG

Total lesion glycolysis

Conflict of Interest

Seunggyun Ha, Hongyoon Choi, Jin Chul Paeng, and Gi Jeong Cheon declare no conflict of interest.

Ethical Statement

This work does not contain any studies with human participants or animals performed by any of the authors. For this type of study, formal consent is not required.

Informed Consent

Not applicable.