Distinguishing Granulomas from Adenocarcinomas by integrating stable and discriminating radiomic features on non-Contrast CT scans

Mohammadhadi Khorrami; Kaustav Bera; Rajat Thawani; Prabhakar Rajiah; Amit Gupta; Pingfu Fu; Philip Linden; Nathan Pennell; Frank Jacono; Robert C Gilkeson; Vamsidhar Velcheti; Anant Madabhushi

doi:10.1016/j.ejca.2021.02.008

. Author manuscript; available in PMC: 2022 May 1.

Published in final edited form as: Eur J Cancer. 2021 Mar 17;148:146–158. doi: 10.1016/j.ejca.2021.02.008

Distinguishing Granulomas from Adenocarcinomas by integrating stable and discriminating radiomic features on non-Contrast CT scans

Mohammadhadi Khorrami ¹, Kaustav Bera ¹, Rajat Thawani ², Prabhakar Rajiah ³, Amit Gupta ⁴, Pingfu Fu ⁵, Philip Linden ⁶, Nathan Pennell ⁷, Frank Jacono ⁸, Robert C Gilkeson ⁴, Vamsidhar Velcheti ⁹, Anant Madabhushi ^1,¹⁰

PMCID: PMC8087632 NIHMSID: NIHMS1676206 PMID: 33743483

Abstract

Objective:

To identify stable and discriminating radiomic features on non-contrast CT scans to develop more generalizable radiomic classifiers for distinguishing granulomas from adenocarcinomas.

Methods:

In total, 412 patients with adenocarcinomas and granulomas from three institutions were retrospectively included. Segmentations of the lung nodules were performed manually by an expert radiologist in a 2D axial view. Radiomic features were extracted from intra- and perinodular regions. 145 patients were employed as part of the training set (S_tr) whereas 205 patients were used as part of test set I ( $S_{t e}^{1}$ ) and 62 patients were employed as part of independent test set II ( $S_{t e}^{2}$ ). To mitigate the variation of CT acquisition parameters, we defined ‘stable’ radiomic features as those for which the feature expression remains relatively unchanged between different sites, as assessed using a Wilcoxon rank-sum test. These stable features were employed to develop more generalizable radiomic classifiers that were more resilient to variations in lung CT scans. Features were ranked based on two criteria, firstly based on discriminability (i.e. maximizing AUC) alone and subsequently based on maximizing both feature stability and discriminability. Different machine learning classifiers (LDA, QDA, Support vector machines and random forest) were trained with features selected using the two different criteria and then compared on the two independent test sets for distinguishing granulomas from adenocarcinomas, in terms of area under the receiver operating characteristic curve.

Results:

In the test sets, classifiers constructed using the criteria involving maximizing feature stability and discriminability simultaneously achieved higher AUC compared to the discriminating alone criteria ( $S_{t e}^{1}$ (n=205): maximum AUCs of 0.85 vs. 0.80; p-value = 0.047 and $S_{t e}^{2}$ (n=62): maximum AUCs of 0.87 vs. 0.79; p-value = 0.021). These differences held for features extracted from scans with < 3 mm slice thickness (AUC = 0.88 vs. AUC = 0.80; p-value = 0.039, n=100) and for the ≥ 3 mm cases (AUC = 0.81 vs. AUC = 0.76; p-value = 0.034, n=105). In both experiments, shape and peritumoral texture features had a higher stability compared to intratumoral texture features.

Conclusions:

Our study suggests that explicitly accounting for both stability and discriminability results in more generalizable radiomic classifiers to distinguish adenocarcinomas from granulomas on non-contrast CT scans. Our results also showed that peritumoral texture and shape features were less affected by the scanner parameters compared to intratumoral texture features, however, they were also less discriminating compared to intratumoral features.

Keywords: Lung Cancer, NSCLC, Radiomics, Machine learning, Stability, Malignant Nodule

Introduction

Computed-Tomography (CT) can non-invasively capture tumor phenotypic characteristics and is the most widely used imaging modality in lung cancer diagnosis and assessment of treatment response. Radiomics, a term referring to comprehensive quantification of tumor phenotypes on radiographic scans seeks to identify subtle image-based attributes that may not be visually discernible by a human expert reader [1]. Radiomic features have been shown to be useful in not only identifying the presence of disease on radiographic scans but also in helping identify cues relating to disease outcome and treatment response. A number of radiomic approaches have been presented so far for diagnosis and characterization of lung [1-5], breast [6] and brain [7, 8] cancers using various imaging modalities ranging from CT, MRI and PET. Specifically, in the context of lung cancer, radiomic approaches have been employed for discriminating benign and malignant nodules [9, 10], predicting outcome and treatment response to chemotherapy [11, 12] and immunotherapy [13, 14].

However, the quality of these CT scans often varies depending on the type of scanner employed across institutions [15] due to differences in equipment and acquisition parameters including slice thickness [16], contrast enhancement, choice of reconstruction kernel [17], temporal variations, as well as due to scanning position and patient specific characteristics [18]. A number of recent studies have been exploring the effect of these different imaging parameters on the performance of radiomic features in malignancy diagnosis [19], EGFR prediction [20] and prognosis [12]. These sources of variation could affect the quality of scans which in turn may affect the quality of the radiomic parameters and lead to a marked difference in the predictive performance. To construct generalizable classifiers, one needs to identify features that are both predictive and stable across sites and scanners.

In general, there are two approaches to dealing with the problem of acquisition related sources of variance when it comes to the radiomics [21]. One approach is to try to explicitly correct the source of variance, i.e. trying to understand, model, and subsequently subtract the source of variance [21, 22]. The other approach is to try to identify stable features, i.e. those that are robust to variations in CT scanners and acquisition parameters [23-26]. The approach described in this study belongs to the latter category, where we attempt to explicitly define the concept of feature stability for pruning radiomic features. This is defined in terms of how often the expression for a feature remains relatively unchanged between different sites, as assessed using a Wilcoxon rank-sum test. A few studies have attempted to evaluate radiomic features on test/retest imaging exams as a pre-processing step to identify stable features [27, 28].

Radiomic features are used to capture lesion heterogeneity through quantitative descriptors. Morphologic heterogeneity is an attribute that is closely associated with malignant nodules, in turn reflecting the intermixed area of high cell density and necrosis and can be captured by radiomic textural analysis. Recently there has been interest in radiomic investigation of the peritumoral region, the area immediately surrounding the tumor mass. Interest in the understanding of the biology of the tumor microenvironment and habitat has been driven by the hypothesis that this region might harbor valuable disease specific prognostic cues. Vascular invasion, infiltration of tumor lymphocytes and angiogenesis are often seen around the tumor microenvironment and their presence has been implicated in higher risk of recurrence as well as shortened disease specific survival in NSCLC patients [29-31]. Our group has previously shown that peritumoral features tend to be more stable with respect to changes in CT images scanners compared to the intranodular texture features for predicting recurrence in early stage NSCLC [23].

In addition, there has also been growing body of research in the use of deep learning (DL) approaches for discriminating malignant and benign nodules. The main idea with deep learning approaches is to expose the network to every possible combination of imaging acquisition parameters. However, identifying and corralling potentially thousands of scans to capture the plurality in imaging variance can be a significant challenge. The approach in this study takes an alternative approach, focusing instead on the use of quantitative criteria to first identify stable features and subsequently only selecting those radiomic features for inclusion within a machine learning model that simultaneously maximize both discriminability (as assessed by criteria like Area under the curve (AUC)) and stability. This approach offers the advantage of a much smaller discovery set with scans represented from different sites and scanners, enabling the identification of discriminating features that are also resilient to sources of acquisition variance.

The aim of this study was to extend and apply the stability-discriminability feature selection approach for a challenging diagnostic problem in chest CT scans, distinguishing granulomas (benign tumor confounders) from adenocarcinomas at baseline non-contrast CT. We hypothesize that differences in the feature values of cancerous tumor is a reflection of site-specific imaging variance since it is unlikely that lesion texture would vary across sites without site-specific confounding effects. By integrating stability measurements in tandem with discriminability-based feature selection approaches that typically only focus on feature discriminability [32], we sought to develop more generalizable classifiers that are more robust to variations in lung CT scans across sites and acquisition parameters.

Materials and Methods

Patient cohort

This study was compliant with Health Insurance Portability and Accountability Act (HIPAA) and approved by the institutional review board. A total number of 405 patients with adenocarcinomas or granulomas between January 1, 2007 and December 31, 2013 were retrospectively documented from University Hospitals Cleveland Medical Center (UHCMC, Site I) and Cleveland Clinic Foundation (CCF, Site II). All patients that met the following criteria were included: (a) availability of pathology report via surgical resection/biopsy, (b) presence of a screening or diagnostic thoracic CT scan in axial view and (c) presence of a solitary pulmonary nodule. We applied the exclusion criteria of removing scans with CT artifacts (n = 48), presence of imaging contrast (n = 37) and patients who underwent biopsy prior to imaging (n = 30) that resulted in the exclusion of 115 patients. The final cohort had a population of 290 patients, where 145 of them were randomly selected as a training set (S_tr) that in turn consisted of 73 adenocarcinomas and 72 granulomas. The patients were selected in a manner that ensured an equal number of adenocarcinomas and granulomas were included in the training set. The remaining 145 patients as well as 60 patients with adenocarcinomas from the National Lung Screening Trial (NLST) dataset were employed as part of test set I ( $S_{t e}^{1}$ ) (in total 205 patients) with 132 adenocarcinomas and 73 granulomas. In addition, 62 patients from University Hospitals Cleveland Medical Center between May 22, 2019 and September 23, 2020 diagnosed with adenocarcinomas and granulomas were collected and used as independent test set II ( $S_{t e}^{2}$ ). None of these patients had to be excluded since they all met our inclusion criteria. The test set II ( $S_{t e}^{2}$ ) consisted of 62 patients with 52 adenocarcinomas and 10 granulomas. Moreover, detailed clinical and demographic statistics and disease types (types of adenocarcinoma and granuloma) for the patients considered in this study are summarized in Supplementary Table S4.

CT Acquisition

The CT scan images were acquired from either Siemens, GE Medical Systems, Philips or Toshiba machine scanners. Further details regarding image acquisition are provided in the Supplementary Section. The dataset also had images acquired from multiple reconstruction kernels. It is known that the different reconstruction kernels of the CT image acquisition affect the radiomic features. Therefore, precaution was taken to equally distribute cases in the training and validation sets and to account for variability as shown in Supplementary Table S1. The pixel sizes ranged from 0.42 x 0.42mm to 0.97 x 0.97mm with an average size of 0.73 x 0.73mm.

Nodule Segmentation and Feature Extraction

The index lesions were identified by an expert radiologist matched with patient reports and lesions were segmented on 3D-Slicer® software where a free hand tool was used to annotate the outer boundary of the nodule. The perinodular compartment around the nodule was defined via the use of morphological operations (dilation) as a region extending radially from the nodule boundary up to roughly 5 mm. The choice of perinodular compartment size was determined based on the findings in our prior study [9], where the most discriminating features were found to be within an immediate distance of 5 mm from the nodule. The region corresponding to air was eliminated from within the perinodular compartment and radiomic texture features were subsequently extracted.

Two-dimensional texture features were extracted slice by slice containing the whole volume of the nodule. The set of features were selected so as to capture textural structure of intra- and peritumoral regions. In this study, we extracted 13 Haralick features from a gray-level cooccurrence matrix (GLCM) that enables for capture of a textural pattern that in turn encodes for the variation in tumor microarchitecture, heterogeneity and local appearance of nodules. In addition, we extracted 25 Law, Law-Laplacian and 48 Gabor features from the intra- and peritumoral regions. Law and Law-Laplacian are filter-based descriptors that can capture combinations of 5 textural patterns, including levels (L), edges (E), spots (S), waves (W), or ripples (R) which can in turn capture patterns of heterogeneous enhancement and abnormal structure in the nodule. The Gabor filter bank was used to capture texture responses at 6 different spatial frequencies (f = 0, 2, 4, 8, 16, or 32) within the image at 8 different directional orientations (θ = 0, π/8, π/4, 3π/8, π/2, 5π/8, 3π/4, 7π/8), enabling possible capture of changes in tumor microarchitecture, tumor lymph angiogenesis in adenocarcinomas and lymph histiocytic inflammatory response in granulomas. Statistics (mean, median, SD, skewness, kurtosis and variance) for each feature were computed within the intra- and peri-tumoral region, resulting in a total of 666 features. Radiomic feature extraction was performed with MATLAB 2018b (Mathworks, Natick, MA, USA) using a toolbox developed in-house. A total of 24 computerized quantitative 3D space shape features to capture attributes like convexity, compactness, area and volume of the nodule were also employed. All feature values were then normalized (mean of 0 and a standard deviation of 1). More information on the features can be found in Supplementary Table S2 and in Supplementary Data.

Feature Stability

To assess feature stability, we employed the concept of the preparation-induced instability score (PI), a concept introduced by Leo et al [24]. The extracted radiomic and shape features from both intranodular and perinodular regions were ranked based on their stability in S_tr and were assessed via the PI. The stability measure involved cross-dataset comparisons for quantifying the frequency at which a feature was found to be differentially expressed between datasets from different sites. A high PI suggests that a feature is frequently different between sites, because it is likely affected by variations in CT scanners and acquisition parameters whereas a low instability score shows that features are not affected by variations in CT scanners. Every feature with a PI > 0.1 was considered as highly unstable. A PI > 0.1 indicates that a radiomic feature associated with cancerous nodule was significantly different for more than 10% of comparisons in different sites. More information regarding feature stability can be found in Supplementary Data.

Feature Discriminability

To determine the discriminability of each feature, the area under the receiver operating characteristic (ROC) curve (AUC) was calculated. Feature AUC was calculated via 100 iterations of 3-fold cross validation (CV) using twelve machine learning classifiers in S_tr. We used twelve machine-learning classifiers from nine families of classifiers to evaluate the discriminability of features in S_tr. The MATLAB module classification toolbox was used to classify adenocarcinomas vs. granulomas (See Supplementary Table S3). These classifiers were also used to assess the discriminability of features in $S_{t e}^{1}$ and $S_{t e}^{2}$ . The mean AUC across twelve classifiers was then used to calculate the final AUC for each feature in S_tr. A feature was considered discriminating if its associated AUC was greater than a predetermined threshold of 0.67. This threshold ensured that the performance of our classifiers in S_tr is always greater than 0.8.

Each feature had two values, a PI and mean AUC. Thus, each feature was defined by a unique position in the PI-AUC space. Those features that have an AUC > 0.67 were considered as highly discriminating features. In addition, only features with a PI up to 0.1 were considered as stable features. The features identified simultaneously as stable and discriminating were considered as highly stable-discriminating features and used in the experiments.

To avoid the issue of having too many features within the classifier and hence reduce the curse of dimensionality and the risk of overfitting, we imposed the constraint that the number of features should not exceed the number of events by an order of 10, based on the Harrell guideline [33]. To do this, feature selection methods were employed to ensure that the number of features met the guideline. We applied four different feature selection methods (sequential feed forward selection (FFS), Wilcoxon rank-sum (WLCX), minimum redundancy maximum relevance (mRMR), and student t test (TT)) to select features that fell into the (1) stable-discriminating and (2) discriminating alone feature groups. All the feature selection methods used in this study identified features based on maximizing differences between malignant and benign nodules (p-value <0.05) in the training set.

Experimental Design

Experiment 1: Distinguishing adenocarcinomas from granulomas on lung CT by evaluating the effect of site variability

In the first experiment, the effect of site variability with and without controlling for slice thickness was evaluated. Stable and discriminating features were identified and a subset of these features were then selected by using four feature selection methods and subsequently used to construct corresponding machine learning classifiers. Seven classifiers were trained with selected features in S_tr and then evaluated in $S_{t e}^{1}$ and $S_{t e}^{2}$ .

Experiment 2: Distinguishing adenocarcinomas from granulomas on lung CT by evaluating the effect of site variability by controlling for slice thickness

The effect of site variability by controlling slice thickness of lung CT for distinguishing adenocarcinomas from granulomas was also evaluated. To do this, we first retained only cases with CT slice images < 3 mm in both S_tr, $S_{t e}^{1}$ . In this case, 102 patients were identified in S_tr, while 100 patients remained in $S_{t e}^{1}$ . Then we retained only cases with CT slice thickness ≥ 3mm in both S_tr, $S_{t e}^{1}$ . In this case, 43 patients were identified in S_tr and 105 patients remained in $S_{t e}^{1}$ . Since all scans in $S_{t e}^{2}$ had CT slice images < 3 mm, the result for this experiment will not change compared to experiment 1.

Experiment 3: Evaluating resilience of radiomic features via Umap Embedding

Umap embedding [34] was performed on S_tr to assess the inter-site variation of the most stable-discriminating features compared to the discriminating features alone, for distinguishing adenocarcinomas from granulomas. Umap is a dimensionality reduction method that allows for reducing the multi-dimensional feature space map to a smaller number of dimensions (only two dimensions) for evaluating the clustering between different categories of entities. If distinct clusters appeared in the Umap space and those clusters corresponded to patients from a specific site, that would reflect the presence of site-specific attributes or vendor effects. On the other hand, if the images from all sites were more homogeneously distributed in the Umap space, it would suggest that the set of radiomic features were resilient to the source of variance in CT images. Umap was performed on both the (1) discriminating features alone and then (2) stable-discriminating radiomic features.

Results

Distinguishing adenocarcinomas from granulomas on lung CT by evaluating the effect of site variability

Figure 1 shows the PI-AUC space for 1356 features based on inter-site variability for S_tr for 145 patients. The X-axis corresponds to the PI values and Y-axes corresponds to the AUC, each circle in the plane shows a pair AUC and PI. The center of each circle is the average AUC (calculated from 12 classifiers) and its radius is the standard deviation of AUC calculated from 12 classifiers. Different colors correspond to different features family as indicated in figure 1.

41 features satisfied the stability-discriminability threshold criteria. From these, 40 features were identified as intratumoral and 1 feature was peritumoral. Additionally, 139 features were identified based on the discriminability threshold alone, of these 126 were intratumoral and 13 were peritumoral features. The average PI for intranodular and perinodular features on lung CT by evaluating the effect of site variability was 0.26 and 0.17, respectively. The representative AUC values for each combination of machine learning classifiers and feature selection methods on $S_{t e}^{1}$ (n = 205) and $S_{t e}^{2}$ (n = 62) are illustrated in Table 1 and Supplementary Table S5, respectively. As may be seen, classifiers constructed using the most discriminating-stable criteria achieved higher AUC compared to those identified based on the discriminating criteria alone ( $S_{t e}^{1}$ : maximum AUCs of 0.85 vs. 0.80 using QDA classifier; p-value = 0.047 and $S_{t e}^{2}$ : maximum AUCs of 0.87 vs. 0.79 using LDA classifier and WLCX feature selection; p-value = 0.021).

Table 1.

Classifier accuracy (AUC) for feature selection (in rows) and classification (in columns) for test set I (n = 205).

	Feature Selection	KNN	LDA	QDA	SVM	SVM RBF	SVM Poly	Random Forest
Stability-Accuracy PI < 0.1 and AUC > 0.67	mRMR	0.71	0.77	0.73	0.73	0.71	0.72	0.69
	TT	0.65	0.75	0.72	0.73	0.73	0.73	0.70
	WLCX	0.67	0.78	0.85	0.81	0.74	0.79	0.80
	FFS	0.77	0.74	0.78	0.75	0.73	0.76	0.70
Accuracy alone AUC > 0.67	mRMR	0.69	0.72	0.71	0.74	0.59	0.71	0.70
	TT	0.60	0.70	0.70	0.72	0.69	0.74	0.68
	WLCX	0.67	0.76	0.80	0.80	0.74	0.71	0.77
	FFS	0.67	0.72	0.72	0.71	0.67	0.72	0.68

Open in a new tab

Figure 2 visually illustrates two texture from the Law-Laplacian family. The feature map in Figure 2(b) illustrates stable intranodular Law-Laplacian L × L feature (PI = 0.06, AUC = 0.74) and 2(c) illustrates discriminating alone intranodular Law-Laplacian L × L feature (PI = 0.8, AUC = 0.74).

Distinguishing adenocarcinomas from granulomas on lung CT by evaluating the effect of site variability by controlling for slice thickness

Figure 3 shows the PI-AUC space for 1356 features for N = 102 patients based on inter-site variability with control on slice thicknesses < 3 mm, for diagnostic problem. 63 features passed stability-discriminability threshold criteria and 204 features were identified based on the discriminability threshold alone. From those 63 features that met stability-discriminability threshold criteria, 50 features were identified as intratumoral and 13 were peritumoral. The average PI for intranodular and perinodular features from lung CT was 0.22 and 0.10, respectively. The representative AUC values for each combination of machine classifiers and feature selection methods on $S_{t e}^{1}$ (n = 100) are illustrated in Table 2. The QDA classifier had a higher performance in discriminating malignant from benign nodules when it was trained with the most discriminating-stable features compared to the discriminating features alone (AUC = 0.88 vs. AUC = 0.80; p-value = 0.039, n=100).

Table 2.

Classifier accuracy (AUC) for feature selection (in rows) and classification (in columns) for test set I (n = 100) with controlling for slice thicknesses < 3 mm.

	Feature Selection	KNN	LDA	QDA	SVM	SVM RBF	SVM Poly	Random Forest
Stability-Accuracy PI < 0.1 and AUC > 0.67	mRMR	0.76	0.76	0.79	0.76	0.88	0.65	0.85
	TT	0.86	0.62	0.85	0.67	0.79	0.67	0.77
	WLCX	0.72	0.59	0.82	0.60	0.77	0.55	0.77
	FFS	0.74	0.63	0.88	0.73	0.80	0.78	0.74
Accuracy alone AUC > 0.66	mRMR	0.60	0.64	0.76	0.70	0.73	0.59	0.71
	TT	0.61	0.62	0.80	0.68	0.71	0.66	0.72
	WLCX	0.58	0.58	0.77	0.54	0.60	0.50	0.69
	FFS	0.66	0.64	0.73	0.66	0.71	0.70	0.69

Open in a new tab

Figure 4 visually illustrates two texture from the Haralick feature family. The feature map in Figure 4(b) illustrates stable perinodular Haralick entropy feature (PI = 0.02, AUC = 0.68) and 4(c) illustrates discriminating alone perinodular Haralick entropy feature (PI = 0.3, AUC = 0.68).

Figure 5 shows the PI-AUC space for 1356 features for N = 43 patients based on inter-site variability with control on slice thicknesses ≥ 3 mm. 62 features passed stability-discriminability threshold criteria, of which there were 38 intratumoral and 24 peritumoral features. Also 147 features were identified based on the discriminability threshold alone, of these 130 features were intratumoral and 17 were peritumoral features. The average PI for intranodular and perinodular features from lung CT in this case was 0.34 and 0.18, respectively. The representative AUC values for each combination of machine learning classifiers and feature selection methods on $S_{t e}^{1}$ (n = 105) are illustrated in Table 3. The QDA classifier trained with the most discriminating-stable features had a higher performance compared to the discriminating features alone (AUC = 0.81 vs. AUC = 0.76; p-value = 0.034, n=105).

Table 3.

Classifier accuracy (AUC) for feature selection (in rows) and classification (in columns) for test set II (n=105) with controlling for slice thicknesses ≥ 3 mm.

	Feature Selection	KNN	LDA	QDA	SVM	SVM RBF	SVM Poly	Random Forest
Stability-Accuracy PI < 0.1 and AUC > 0.67	mRMR	0.71	0.77	0.79	0.74	0.69	0.79	0.73
	TT	0.70	0.76	0.81	0.81	0.70	0.76	0.69
	WLCX	0.65	0.75	0.80	0.71	0.60	0.62	0.72
	FFS	0.66	0.78	0.76	0.76	0.65	0.67	0.65
Accuracy alone AUC > 0.67	mRMR	0.68	0.73	0.74	0.73	0.67	0.75	0.68
	TT	0.71	0.72	0.76	0.75	0.68	0.74	0.63
	WLCX	0.61	0.69	0.70	0.67	0.62	0.60	0.68
	FFS	0.61	0.71	0.72	0.71	0.62	0.65	0.65

Open in a new tab

Resilience of radiomic features via Umap Embedding

The results of Umap embedding of S_trare shown in Figure 6. While the cancerous (malignant) and non-cancerous (benign) patients from each site tended to separate in the discriminating features embedding, no site formed a distinct cluster in the stable-discriminating feature space. This suggests that while the radiomic features varied considerably in discriminating features across different sites, however the stable-discriminating features were not adversely affected by variations in scanner acquisition parameters across the different sites.

Discussion

Radiomic based feature analysis has allowed for non-invasive capture of many aspects of biological perturbations occurring within the tumor and tumor microenvironments at the organ and multi-organ system level and could possibly reflect attributes on the scans relating to the tumor biology such as molecular subtype and gene mutation or expression [29-31, 35, 36]. In addition, integrating radiomic approaches with machine learning driven diagnostic assistance has potential implications for lung cancer screening for early interventions for malignant adenocarcinomas while at the same time potentially preventing unnecessary biopsies for benign granulomas.

More recently, a large number of studies have been exploring the sensitivity of radiomic features towards varying acquisition parameters [37- 40]. While several groups have studied the sensitivity of radiomics for different cases and applications, relatively few studies have tackled the specific problem of trying to develop generalizable radiomic classifiers by integrating stability and discriminability considerations across multiple different sites and slice thicknesses. Our group has previously looked at the issue of radiomic feature stability in the context of other problems. For instance, we previously showed that machine learning classifiers involving stable and predictive radiomic features could help to improve the prognosis of cancer recurrence in early stage (I, II) NSCLC on non-contrast CT [23].

For CT texture analysis to be adopted into broad clinical practice across multiple imaging centers, the methodology must be largely robust to variations in acquisition parameters. To build and test such an approach, we focused on one of the most challenging dilemmas as differentiating malignant from benign nodules, due to the similar appearance of both on standard CT images. We analyzed the texture features inside and outside the nodule which appears to substantially help in discriminating benign from malignant nodules.

In our first experiment we found that shape, Laws-Laplacian and Gabor texture features tend to be more stable while Haralick and Laws texture features tend to be more unstable. The reason for this probably relates to the fact that Haralick and Laws texture features involve extracting higher order derivatives and joint statistics from CT images, higher order derivatives are known to be noisy and therefore more unstable [43]. In contrast, Gabor and Laplacian features tend to involve low pass filtering coupled with feature extraction and hence are likely more resilient. Lesion shape features that tend to be less dependent on CT specific acquisition parameters, tend to be more stable compared to texture features which are dependent on image intensities.

The specific problem that was studied in this work was identifying radiomic features to distinguish adenocarcinomas from granulomas on CT scans, a challenging problem for human readers to visually diagnose. A number of studies have applied radiomic approaches to achieve better distinction between malignant and benign lung nodules to date [9, 10, 19]. However, no single robust method has been established to address this challenging problem. One study by Orooji et al [10] showed that Law, Gradient and Gabor texture features were the most discriminating texture features to identify adenocarcinomas from granulomas. Another study by Beig et al [9] revealed that the combination of perinodular and intranodular Gabor and Law texture features from non-contrast CT images, increased the performance of the classifiers for distinguishing adenocarcinomas from granulomas, compared to the use of intranodular texture features alone. However, unlike our study, in both of those studies, the authors did not explicitly consider stability as part of feature selection. In fact, the control experiment in our study (i.e. using discriminating features) was essentially the approach used in those two studies; one that we showed was inferior in terms of generalizability on the test set compared to considering stability and discriminability as part of the feature selection process (see Supplementary Data for more information).

In the second experiment, we considered the effect of slice thickness of the CT images on the performance of classifiers. Previous studies have shown that textural features are most affected by changes in slice thickness [41]. Similarly, the previous study by He et al. [17] showed that quantification of CT image features could be significantly affected by slice thickness and demonstrated that thinner slices were better than thicker slices for texture analysis. Our study is consistent with these previous findings and revealed that intratumoral and peritumoral texture features as well as shape features were more stable across CT slice thicknesses < 3 mm. Most of the intranodular Laws-Laplacian and perinodular Laws texture features tend to be more unstable with slice thicknesses ≥ 3 mm. Thicker slices introduces more partial volume artifacts as compared to thinner sectioned CT scans [44, 45]. The increased partial volume artifacts in turn affects the stability of the radiomic features more severely.

Finally, the results of Umap embedding showed that while discriminating radiomic features are extremely affected by the variation in CT images across different sites, considering the stability criteria tends to result in more resilient features across scanners and sites.

In addition, despite a lot of advantages of deep learning approaches, these approaches are largely “black-box” and provide little insight into the features driving the prediction [46, 47]. In previous studies [9, 48] we have shown that the hand-crafted radiomic features are able to directly capture more attributes relating to the pathobiology of the disease (e.g., angiogenesis [48], immune environment [9]). In addition, in previous studies by our group [9] for the problem of discriminating benign from malignant nodules, we have found that hand-crafted radiomic features have tended to outperform DL strategies, when using the same sized training set. It may well be that the DL approaches were limited by the size of the training set (n<200) and on a significantly larger training set, the DL classifier might be non-inferior to the hand-crafted based approach.

We acknowledge the limitations of our study. Firstly, there is no explicit analysis of reconstruction kernels nor explicit consideration of radiation dose, tube voltage or tube current in our study. However, the results of previous studies have suggested that texture features tend to be resilient to tube voltage, tube current and convolution kernels [17, 49, 50]. Also, all CT images in this study were acquired from non-contrast images and no explicit study was done on contrast CT images. While previous studies showed that radiomic features are dependent on the manual annotation of the nodule, the CT annotation in this study was limited to a single reader instead of multiple readers. However, because we used statistics of the features as opposed to pixel-wise classification, we do not believe that the lesion contours substantially affected the texture features. Moreover, our results showed no significant difference for the shape features between two readers (see Supplementary Data) and so pixel-wise annotation does not appear to adversely affect the results of this study. Despite the aforementioned limitations, we believe that our finding that a combination of stable and discriminating radiomic features could allow for creation of more generalizable classifiers for distinguishing granulomas from adenocarcinomas is an important one. The findings presented herein could allow for more intelligent construction of generalizable radiomic classifiers for lung cancer diagnosis on CT scans.

Conclusions

With additional multi-site evaluation, the radiomic machine classifier presented in this work could potentially serve as a decision support tool for thoracic radiologists to distinguish malignant from benign nodule in non-contrast CT scans. Our findings suggest that accounting for both feature stability and discriminability results in more generalizable and accurate radiomic classifiers for the challenging problem of discriminating granulomas from adenocarcinomas. With additional validation, the approach presented in this study could be developed into a clinical decision support tool to help reduce the number of surgical resections and biopsies on account of benign nodules that are currently identified as suspicious or intermediate risk on non-contrast CT scans.

Supplementary Material

NIHMS1676206-supplement-1.pdf^{(296.2KB, pdf)}

NIHMS1676206-supplement-2.pdf^{(343.8KB, pdf)}

Highlights.

Intramodular Gabor and Law features are highly discriminating-stable radiomic features
Shape features are highly stable but less accurate for distinguishing malignant from benign nodules
Perinodular Gabor features are highly discriminating-stable radiomic features in CTs with slice thickness < 3mm
In slice CTs with thickness > 3mm, most radiomic features tend to become unstable

Acknowledgments

Funding

Research reported in this publication was supported by the National Cancer Institute under award numbers 1U24CA199374-01, R01CA202752-01A1, R01CA208236-01A1, R01CA216579-01A1, R01CA220581-01A1, 1U01CA239055-01, 1U01CA248226-01, 1U54CA254566-01, National Heart, Lung and Blood Institute 1R01HL15127701A1, National Institute for Biomedical Imaging and Bioengineering 1R43EB028736-01, National Center for Research Resources under award number 1 C06 RR12463-01, VA Merit Review Award IBX004121A from the United States Department of Veterans Affairs Biomedical Laboratory Research and Development Service, the Office of the Assistant Secretary of Defense for Health Affairs, through the Breast Cancer Research Program (W81XWH-19-1-0668), the Prostate Cancer Research Program (W81XWH-15-1-0558, W81XWH-20-1-0851), the Lung Cancer Research Program (W81XWH-18-1-0440, W81XWH-20-1-0595), the Peer Reviewed Cancer Research Program (W81XWH-18-1-0404), the Kidney Precision Medicine Project (KPMP) Glue Grant, the Ohio Third Frontier Technology Validation Fund, the Clinical and Translational Science Collaborative of Cleveland (UL1TR0002548) from the National Center for Advancing Translational Sciences (NCATS) component of the National Institutes of Health and NIH roadmap for Medical Research, and The Wallace H. Coulter Foundation Program in the Department of Biomedical Engineering at Case Western Reserve University. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the U.S. Department of Veterans Affairs, the Department of Defense, or the United States Government.

Footnotes

Conflicts of interest

MK, KB, RT, PR, AG, PF, PL, NP, FJ, and RG disclosed no conflicts of interest. VV is a Consultant for BMS, Genentech, Astrazeneca, Celgene, Foundation Medicine, Taekeda, Merck, Alkermes, and Nektar Therapeutics; institution has grants or grants pending with Astrazeneca, Merck, BMS, Genentech, and Alkermes; is on the speakers bureaus of Novartis, BMS, Celgene, and Foundation Medicine; has received payment for the development of educational presentations from BMS and Foundation Medicine. AM is on the board of and is a consultant to AiForia Inc; institution has three NCI RO1 grants with Inspirata and one ongoing U24 with PathCore; Elucid Bioimaging has licensed some of the institutions’ patents (both Case Western Reserve University and Rutgers University); holds stock equity in Elucid Bioimaging and Inspirata; sponsored research from Astrazeneca, Bristol Myers-Squibb, and Boehringer-Ingelheim.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

1.Swensen SJ, Brown LR, Colby TV, Weaver AL. Pulmonary nodules: CT evaluation of enhancement with iodinated contrast material. 1995. February;194(2):393–8. doi: 10.1148/radiology.194.2.7824716. [DOI] [PubMed] [Google Scholar]
2.Pyenson BS, Henschke CI, Yankelevitz DF, Yip R, Dec E. Offering Lung Cancer Screening to High-Risk Medicare Beneficiaries Saves Lives and Is Cost-Effective: An Actuarial Analysis. American Health & Drug Benefits. 2014. August; 7(5): 272–282. [PMC free article] [PubMed] [Google Scholar]
3.Coroller TP, Grossmann P, Hou Y, et al. CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology. 2015;114(3):345–350. doi: 10.1016/j.radonc.2015.02.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Fried DV, Tucker SL, Zhou S, et al. Prognostic Value and Reproducibility of Pretreatment CT Texture Features in Stage III Non-Small Cell Lung Cancer. International journal of radiation oncology, biology, physics. 2014;90(4):834–842. doi: 10.1016/j.ijrobp.2014.07.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Cook GJ, Yip C, Siddique M, et al. Are pretreatment 18F-FDG PET tumor textural features in non-small cell lung cancer associated with response and survival after chemoradiotherapy? J Nucl Med. 2013. January;54(1):19–26. doi: 10.2967/jnumed.112.107375. [DOI] [PubMed] [Google Scholar]
6.Saha A, Harowicz M, Mazurowski MA, Breast cancer MRI radiomics: An overview of algorithmic features and impact of inter-reader variability in annotating tumors. Medical physics. 2018. July;45(7):3076–3085. doi: 10.1002/mp.12925. [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Gevaert O, Mitchell LA, Achrol AS, et al. Glioblastoma Multiforme: Exploratory Radiogenomic Analysis by Using Quantitative Image Features. Radiology. 2014. October;273(1):168–74. doi: 10.1148/radiol.14131731. [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Verhaak RG, , Hoadley KA, , Purdom E, , et al. Integrated genomic analysis identifies clinically relevant subtypes of glioblastoma characterized by abnormalities in PDGFRA, IDH1, EGFR, and NF1. Cancer Cell. 2010. January 19; 17(1): 98. doi: 10.1016/j.ccr.2009.12.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Beig N, Khorrami M, Alilou M, Perinodular and Intranodular Radiomic Features on Lung CT Images Distinguish Adenocarcinomas from Granulomas. 2019. March;290(3):783–792. doi: 10.1148/radiol.2018180910. [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Orooji M, Alilou M, Rakshit S, et al. Combination of computer extracted shape and texture features enables discrimination of granulomas from adenocarcinoma on chest computed tomography. J Med Imaging. 2018. April; 5(2): 024501. doi: 10.1117/1.JMI.5.2.024501. [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Khorrami M, Khunger M, Zagouras A Combination of Peri- and Intratumoral Radiomic Features on Baseline CT Scans Predicts Response to Chemotherapy in Lung Adenocarcinoma. Radiol Artif Intell. 2019. March 20; 1(2): e180012. doi: 10.1148/ryai.2019180012. [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Khorrami M, Jain P, Bera K, et al. Predicting pathologic response to neoadjuvant chemoradiation in resectable stage III non-small cell lung cancer patients using computed tomography radiomic features. Lung Cancer. 2019. September;135:1–9. doi: 10.1016/j.lungcan.2019.06.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Khorrami M, Prasann P, Gupta A, et al. Changes in CT radiomic features associated with lymphocytic patterns predict overall survival and response to immunotherapy in non-small cell lung cancer. Cancer Immunol Res. 2020. January;8(1):108–119. doi: 10.1158/2326-6066.CIR-19-0476. [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Vaidya P, Bera K, Patil PD, Gupta A, Jain P, Alilou M, Khorrami M, Velcheti V, Madabhushi A. Novel, non-invasive imaging approach to identify patients with advanced non-small cell lung cancer at risk of hyperprogressive disease with immune checkpoint blockade. J Immunother Cancer. 2020. October;8(2):e001343. doi: 10.1136/jitc-2020-001343. [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Stoel BC, Bakker ME, Stolk J, Dirksen A, et al. Comparison of the sensitivities of 5 different computed tomography scanners for the assessment of the progression of pulmonary emphysema: a phantom study. Invest Radiol. 2004. January;39(1):1–7. doi: 10.1097/01.rli.0000091842.82062.a3. [DOI] [PubMed] [Google Scholar]
16.Petrou M, Quint LE, Nan B, Baker LH. Pulmonary nodule volumetric measurement variability as a function of CT slice thickness and nodule morphology. AJR Am J Roentgenol. 2007. February;188(2):306–12. doi: 10.2214/AJR.05.1063. [DOI] [PubMed] [Google Scholar]
17.He L, Huang Y, Ma Z, Liang C, Liang C, Liu Z. Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule. Sci Rep. 2016. October 10;6:34921. doi: 10.1038/srep34921. [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Berenguer R, Pastor-Juan MDR, Canales-Vázquez J, Castro-García M, Villas MV, Mansilla Legorburo F, Sabater S. Radiomics of CT Features May Be Nonreproducible and Redundant: Influence of CT Acquisition Parameters. Radiology. 2018. August;288(2):407–415. doi: 10.1148/radiol.2018172361. [DOI] [PubMed] [Google Scholar]
19.Alilou M, Beig N, Orooji M, et al. An integrated segmentation and shape-based classification scheme for distinguishing adenocarcinomas from granulomas on lung CT. Medical physics. 2017;44(7):3556–3569. doi: 10.1002/mp.12208. [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Aerts HJ, Grossmann P, Tan Y, Oxnard GR, Rizvi N, Schwartz LH, Zhao B. Defining a Radiomic Response Phenotype: A Pilot Study using targeted therapy in NSCLC. Sci Rep. 2016. September 20;6:33860. doi: 10.1038/srep33860. [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Mackin D, Fave X, Zhang L, Fried D, Yang J, Taylor B, Rodriguez-Rivera E, Dodge C, Jones AK, Court L. Measuring Computed Tomography Scanner Variability of Radiomics Features. Invest Radiol. 2015. November;50(11):757–65. doi: 10.1097/RLI.0000000000000180. [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Nelson DA, Tan TT, Rabson AB, Anderson D, Degenhardt K, White E. Hypoxia and defective apoptosis drive genomic instability and tumorigenesis. Genes Dev. 2004. September 1;18(17):2095–107. doi: 10.1101/gad.1204904. [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Khorrami M, Bera K, Leo P, et al. Stable and discriminating radiomic predictor of recurrence in early stage non-small cell lung cancer: Multi-site study. Lung Cancer. 2020. April;142:90–97. doi: 10.1016/j.lungcan.2020.02.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Leo P, Elliott R, Shih NNC et al. Stable and discriminating features are predictive of cancer presence and Gleason grade in radical prostatectomy specimens: a multi-site study. Sci Rep. 2018. October 8;8(1):14918. doi: 10.1038/s41598-018-33026-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Merisaari H, Taimen P, Shiradkar R, et al. Repeatability of radiomics and machine learning for DWI: Short-term repeatability study of 112 patients with prostate cancer. Magn Reson Med. 2020. June;83(6):2293–2309. doi: 10.1002/mrm.28058. [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Chirra P, Leo P, Yim M, Bloch BN, Rastinehad AR, Purysko A, Rosen M, Madabhushi A, Viswanath SE. Multisite evaluation of radiomic feature reproducibility and discriminability for identifying peripheral zone prostate tumors on MRI. J Med Imaging (Bellingham). 2019. April;6(2):024502. doi: 10.1117/1.JMI.6.2.024502. [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Leijenar RTH, Carvalho S, Zquez ERV, et al. Stability of FDG-PET Radiomics features: An integrated analysis of test-retest and inter-observer variability. Acta Oncol. 2013. October;52(7):1391–7. doi: 10.3109/0284186X.2013.812798. [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Shafiq-Ul-Hassan M, Zhang GG, Latifi K, Ullah G, Hunt DC, Balagurunathan Y, Abdalah MA, Schabath MB, Goldgof DG, Mackin D, Court LE, Gillies RJ, Moros EG. Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. Med Phys. 2017. March;44(3):1050–1062. doi: 10.1002/mp.12123. [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Gabor S, Renner H, Popper H, Anegg U, Sankin O, Matzi V, Lindenmann J, Smolle Jüttner FM. Invasion of blood vessels as significant prognostic factor in radically resected T1-3N0M0 non-small-cell lung cancer. Eur J Cardiothorac Surg. 2004. March;25(3):439–42. doi: 10.1016/j.ejcts.2003.11.033. [DOI] [PubMed] [Google Scholar]
30.Alitalo K, Tammela T, Petrova TV. Lymphangiogenesis in development and human disease. Nature. 2005. December 15;438(7070):946–53. doi: 10.1038/nature04480. [DOI] [PubMed] [Google Scholar]
31.Zhang BC, Guan S, Zhang YF, et al. Peritumoral lymphatic microvessel density is related to poor prognosis in lung adenocarcinoma: A retrospective study of 65 cases. Exp Ther Med. 2012. April;3(4):636–640. doi: 10.3892/etm.2012.470. [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Wolf L, Shashua. Feature Selection for Unsupervised and Supervised Inference: The Emergence of Sparsity in a Weight-Based Approach. Journal of Machine Learning Research 6 (2005) 1855–1887. [Google Scholar]
33.Harrell FE Jr., Regression modeling strategies with applications to linear models, logistic and ordinal regression, and survival analysis. Springer-Verlag, 2015. doi: 10.1007/978-3-319-19425-7. [DOI] [Google Scholar]
34.McInnes L, Healy J, Melville J. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426v2. last revised 18 Sep 2020. [Google Scholar]
35.Liu Z, Feng B,2, Li C, Chen Y., et al. Preoperative prediction of lymphovascular invasion in invasive breast cancer with dynamic contrast-enhanced-MRI-based radiomics. J Magn Reson Imaging. 2019. February 17. doi: 10.1002/jmri.26688. [DOI] [PubMed] [Google Scholar]
36.Rathore S, Akbari H, Rozycki M, et al. Radiomic MRI signature reveals three distinct subtypes of glioblastoma with different clinical and molecular characteristics, offering prognostic value beyond IDH1. Sci Rep. 2018. March 23;8(1):5087. doi: 10.1038/s41598-018-22739-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Echegaray S, Gevaert O, Shah R, Kamaya A, Louie J, Kothary N, Napel S. Core samples for radiomics features that are insensitive to tumor segmentation: method and pilot study using CT images of hepatocellular carcinoma. J Med Imaging. 2015. October;2(4):041011. doi: 10.1117/1.JMI.2.4.041011. [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Shafiq-Ul-Hassan M, Zhang GG, Hunt DC, Latifi K, Ullah G, Gillies RJ, Moros EG. Accounting for reconstruction kernel-induced variability in CT radiomic features using noise power spectra. J Med Imaging (Bellingham). 2018. January;5(1):011013. doi: 10.1117/1.JMI.5.1.011013. [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Balagurunathan Y, Gu Y, Wang H, Kumar V, Grove O, Hawkins S, Kim J, Goldgof DB, Hall LO, Gatenby RA, Gillies RJ. Reproducibility and Prognosis of Quantitative Features Extracted from CT Images. Transl Oncol. 2014. February 1;7(1):72–87. doi: 10.1593/tlo.13844. [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Hunter LA, Krafft S, Stingo F, Choi H, Martel MK, Kry SF, Court LE. High quality machine-robust image features: identification in nonsmall cell lung cancer computed tomography images. Med Phys. 2013. December;40(12):121916. doi: 10.1118/1.4829514. [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Buch K, Li B, Qureshi MM, Kuno H, Anderson SW, Sakai O. Quantitative Assessment of Variation in CT Parameters on Texture Features: Pilot Study Using a Nonanatomic Phantom. AJNR Am J Neuroradiol. 2017. May;38(5):981–985. doi: 10.3174/ajnr.A5139. [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Caramella C, Allorant A, Orlhac F, Bidault F, Asselain B, Ammari S, Jaranowski P, Moussier A, Balleyguier C, Lassau N, Pitre-Champagnat S. Can we trust the calculation of texture indices of CT images? A phantom study. Med Phys. 2018. April;45(4):1529–1536. doi: 10.1002/mp.12809. [DOI] [PubMed] [Google Scholar]
43.Prenosil GA, Weitzel T, Fürstner M, Hentschel M, Krause T, Cumming P, Rominger A, Klaeser B. Towards guidelines to harmonize textural features in PET: Haralick textural features vary with image noise, but exposure-invariant domains enable comparable PET radiomics. PLoS One. 2020. March 16;15(3):e0229560. doi: 10.1371/journal.pone.0229560. [DOI] [PMC free article] [PubMed] [Google Scholar]
44.Zhao B, Tan Y, Tsai WY, Schwartz LH, Lu L. Exploring Variability in CT Characterization of Tumors: A Preliminary Phantom Study. Transl Oncol. 2014. February 1;7(1):88–93. doi: 10.1593/tlo.13865. [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Tan Y, Guo P, Mann H, Marley SE, Juanita Scott ML, Schwartz LH, Ghiorghiu DC, Zhao B. Assessing the effect of CT slice interval on unidimensional, bidimensional and volumetric measurements of solid tumours. Cancer Imaging. 2012. October 31;12(3):497–505. doi: 10.1102/1470-7330.2012.0046. [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Blanc D, Racine V, Khalil A, Deloche M, Broyelle JA, Hammouamri I, et al. Artificial intelligence solution to classify pulmonary nodules on CT. Diagn Interv Imaging. 2020. December;101(12):803–810. doi: 10.1016/j.diii.2020.10.004. [DOI] [PubMed] [Google Scholar]
47.Liu H, Cao H, Song E, Ma G, Xu X, Jin R, Liu C, Hung CC. Multi-model Ensemble Learning Architecture Based on 3D CNN for Lung Nodule Malignancy Suspiciousness Classification. J Digit Imaging. 2020. October;33(5):1242–1256. doi: 10.1007/s10278-020-00372-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Quantitative vessel tortuosity: A potential CT imaging biomarker for distinguishing lung granulomas from adenocarcinomas. Alilou M, Orooji M, Beig N, Prasanna P, et al. Sci Rep. 2018. October 16;8(1):15290. doi: 10.1038/s41598-018-33473-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
49.Kalousis A, Prados J & Hilario M Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl Inf Syst 12, 95–116 (2007). doi: 10.1007/s10115-006-0040-8. [DOI] [Google Scholar]
50.Yu L, Ding C, Loscalzo S. Stable Feature Selection via Dense Feature Groups. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD 2008, New York, NY, USA, pp. 803–811. ACM; (2008). doi: 10.1145/1401890.1401986. [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

NIHMS1676206-supplement-1.pdf^{(296.2KB, pdf)}

NIHMS1676206-supplement-2.pdf^{(343.8KB, pdf)}

[R1] 1.Swensen SJ, Brown LR, Colby TV, Weaver AL. Pulmonary nodules: CT evaluation of enhancement with iodinated contrast material. 1995. February;194(2):393–8. doi: 10.1148/radiology.194.2.7824716. [DOI] [PubMed] [Google Scholar]

[R2] 2.Pyenson BS, Henschke CI, Yankelevitz DF, Yip R, Dec E. Offering Lung Cancer Screening to High-Risk Medicare Beneficiaries Saves Lives and Is Cost-Effective: An Actuarial Analysis. American Health & Drug Benefits. 2014. August; 7(5): 272–282. [PMC free article] [PubMed] [Google Scholar]

[R3] 3.Coroller TP, Grossmann P, Hou Y, et al. CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology. 2015;114(3):345–350. doi: 10.1016/j.radonc.2015.02.015. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R4] 4.Fried DV, Tucker SL, Zhou S, et al. Prognostic Value and Reproducibility of Pretreatment CT Texture Features in Stage III Non-Small Cell Lung Cancer. International journal of radiation oncology, biology, physics. 2014;90(4):834–842. doi: 10.1016/j.ijrobp.2014.07.020. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R5] 5.Cook GJ, Yip C, Siddique M, et al. Are pretreatment 18F-FDG PET tumor textural features in non-small cell lung cancer associated with response and survival after chemoradiotherapy? J Nucl Med. 2013. January;54(1):19–26. doi: 10.2967/jnumed.112.107375. [DOI] [PubMed] [Google Scholar]

[R6] 6.Saha A, Harowicz M, Mazurowski MA, Breast cancer MRI radiomics: An overview of algorithmic features and impact of inter-reader variability in annotating tumors. Medical physics. 2018. July;45(7):3076–3085. doi: 10.1002/mp.12925. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R7] 7.Gevaert O, Mitchell LA, Achrol AS, et al. Glioblastoma Multiforme: Exploratory Radiogenomic Analysis by Using Quantitative Image Features. Radiology. 2014. October;273(1):168–74. doi: 10.1148/radiol.14131731. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R8] 8.Verhaak RG, , Hoadley KA, , Purdom E, , et al. Integrated genomic analysis identifies clinically relevant subtypes of glioblastoma characterized by abnormalities in PDGFRA, IDH1, EGFR, and NF1. Cancer Cell. 2010. January 19; 17(1): 98. doi: 10.1016/j.ccr.2009.12.020. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R9] 9.Beig N, Khorrami M, Alilou M, Perinodular and Intranodular Radiomic Features on Lung CT Images Distinguish Adenocarcinomas from Granulomas. 2019. March;290(3):783–792. doi: 10.1148/radiol.2018180910. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R10] 10.Orooji M, Alilou M, Rakshit S, et al. Combination of computer extracted shape and texture features enables discrimination of granulomas from adenocarcinoma on chest computed tomography. J Med Imaging. 2018. April; 5(2): 024501. doi: 10.1117/1.JMI.5.2.024501. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R11] 11.Khorrami M, Khunger M, Zagouras A Combination of Peri- and Intratumoral Radiomic Features on Baseline CT Scans Predicts Response to Chemotherapy in Lung Adenocarcinoma. Radiol Artif Intell. 2019. March 20; 1(2): e180012. doi: 10.1148/ryai.2019180012. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R12] 12.Khorrami M, Jain P, Bera K, et al. Predicting pathologic response to neoadjuvant chemoradiation in resectable stage III non-small cell lung cancer patients using computed tomography radiomic features. Lung Cancer. 2019. September;135:1–9. doi: 10.1016/j.lungcan.2019.06.020. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R13] 13.Khorrami M, Prasann P, Gupta A, et al. Changes in CT radiomic features associated with lymphocytic patterns predict overall survival and response to immunotherapy in non-small cell lung cancer. Cancer Immunol Res. 2020. January;8(1):108–119. doi: 10.1158/2326-6066.CIR-19-0476. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R14] 14.Vaidya P, Bera K, Patil PD, Gupta A, Jain P, Alilou M, Khorrami M, Velcheti V, Madabhushi A. Novel, non-invasive imaging approach to identify patients with advanced non-small cell lung cancer at risk of hyperprogressive disease with immune checkpoint blockade. J Immunother Cancer. 2020. October;8(2):e001343. doi: 10.1136/jitc-2020-001343. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R15] 15.Stoel BC, Bakker ME, Stolk J, Dirksen A, et al. Comparison of the sensitivities of 5 different computed tomography scanners for the assessment of the progression of pulmonary emphysema: a phantom study. Invest Radiol. 2004. January;39(1):1–7. doi: 10.1097/01.rli.0000091842.82062.a3. [DOI] [PubMed] [Google Scholar]

[R16] 16.Petrou M, Quint LE, Nan B, Baker LH. Pulmonary nodule volumetric measurement variability as a function of CT slice thickness and nodule morphology. AJR Am J Roentgenol. 2007. February;188(2):306–12. doi: 10.2214/AJR.05.1063. [DOI] [PubMed] [Google Scholar]

[R17] 17.He L, Huang Y, Ma Z, Liang C, Liang C, Liu Z. Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule. Sci Rep. 2016. October 10;6:34921. doi: 10.1038/srep34921. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R18] 18.Berenguer R, Pastor-Juan MDR, Canales-Vázquez J, Castro-García M, Villas MV, Mansilla Legorburo F, Sabater S. Radiomics of CT Features May Be Nonreproducible and Redundant: Influence of CT Acquisition Parameters. Radiology. 2018. August;288(2):407–415. doi: 10.1148/radiol.2018172361. [DOI] [PubMed] [Google Scholar]

[R19] 19.Alilou M, Beig N, Orooji M, et al. An integrated segmentation and shape-based classification scheme for distinguishing adenocarcinomas from granulomas on lung CT. Medical physics. 2017;44(7):3556–3569. doi: 10.1002/mp.12208. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R20] 20.Aerts HJ, Grossmann P, Tan Y, Oxnard GR, Rizvi N, Schwartz LH, Zhao B. Defining a Radiomic Response Phenotype: A Pilot Study using targeted therapy in NSCLC. Sci Rep. 2016. September 20;6:33860. doi: 10.1038/srep33860. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R21] 21.Mackin D, Fave X, Zhang L, Fried D, Yang J, Taylor B, Rodriguez-Rivera E, Dodge C, Jones AK, Court L. Measuring Computed Tomography Scanner Variability of Radiomics Features. Invest Radiol. 2015. November;50(11):757–65. doi: 10.1097/RLI.0000000000000180. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R22] 22.Nelson DA, Tan TT, Rabson AB, Anderson D, Degenhardt K, White E. Hypoxia and defective apoptosis drive genomic instability and tumorigenesis. Genes Dev. 2004. September 1;18(17):2095–107. doi: 10.1101/gad.1204904. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R23] 23.Khorrami M, Bera K, Leo P, et al. Stable and discriminating radiomic predictor of recurrence in early stage non-small cell lung cancer: Multi-site study. Lung Cancer. 2020. April;142:90–97. doi: 10.1016/j.lungcan.2020.02.018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R24] 24.Leo P, Elliott R, Shih NNC et al. Stable and discriminating features are predictive of cancer presence and Gleason grade in radical prostatectomy specimens: a multi-site study. Sci Rep. 2018. October 8;8(1):14918. doi: 10.1038/s41598-018-33026-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R25] 25.Merisaari H, Taimen P, Shiradkar R, et al. Repeatability of radiomics and machine learning for DWI: Short-term repeatability study of 112 patients with prostate cancer. Magn Reson Med. 2020. June;83(6):2293–2309. doi: 10.1002/mrm.28058. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R26] 26.Chirra P, Leo P, Yim M, Bloch BN, Rastinehad AR, Purysko A, Rosen M, Madabhushi A, Viswanath SE. Multisite evaluation of radiomic feature reproducibility and discriminability for identifying peripheral zone prostate tumors on MRI. J Med Imaging (Bellingham). 2019. April;6(2):024502. doi: 10.1117/1.JMI.6.2.024502. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R27] 27.Leijenar RTH, Carvalho S, Zquez ERV, et al. Stability of FDG-PET Radiomics features: An integrated analysis of test-retest and inter-observer variability. Acta Oncol. 2013. October;52(7):1391–7. doi: 10.3109/0284186X.2013.812798. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R28] 28.Shafiq-Ul-Hassan M, Zhang GG, Latifi K, Ullah G, Hunt DC, Balagurunathan Y, Abdalah MA, Schabath MB, Goldgof DG, Mackin D, Court LE, Gillies RJ, Moros EG. Intrinsic dependencies of CT radiomic features on voxel size and number of gray levels. Med Phys. 2017. March;44(3):1050–1062. doi: 10.1002/mp.12123. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R29] 29.Gabor S, Renner H, Popper H, Anegg U, Sankin O, Matzi V, Lindenmann J, Smolle Jüttner FM. Invasion of blood vessels as significant prognostic factor in radically resected T1-3N0M0 non-small-cell lung cancer. Eur J Cardiothorac Surg. 2004. March;25(3):439–42. doi: 10.1016/j.ejcts.2003.11.033. [DOI] [PubMed] [Google Scholar]

[R30] 30.Alitalo K, Tammela T, Petrova TV. Lymphangiogenesis in development and human disease. Nature. 2005. December 15;438(7070):946–53. doi: 10.1038/nature04480. [DOI] [PubMed] [Google Scholar]

[R31] 31.Zhang BC, Guan S, Zhang YF, et al. Peritumoral lymphatic microvessel density is related to poor prognosis in lung adenocarcinoma: A retrospective study of 65 cases. Exp Ther Med. 2012. April;3(4):636–640. doi: 10.3892/etm.2012.470. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R32] 32.Wolf L, Shashua. Feature Selection for Unsupervised and Supervised Inference: The Emergence of Sparsity in a Weight-Based Approach. Journal of Machine Learning Research 6 (2005) 1855–1887. [Google Scholar]

[R33] 33.Harrell FE Jr., Regression modeling strategies with applications to linear models, logistic and ordinal regression, and survival analysis. Springer-Verlag, 2015. doi: 10.1007/978-3-319-19425-7. [DOI] [Google Scholar]

[R34] 34.McInnes L, Healy J, Melville J. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426v2. last revised 18 Sep 2020. [Google Scholar]

[R35] 35.Liu Z, Feng B,2, Li C, Chen Y., et al. Preoperative prediction of lymphovascular invasion in invasive breast cancer with dynamic contrast-enhanced-MRI-based radiomics. J Magn Reson Imaging. 2019. February 17. doi: 10.1002/jmri.26688. [DOI] [PubMed] [Google Scholar]

[R36] 36.Rathore S, Akbari H, Rozycki M, et al. Radiomic MRI signature reveals three distinct subtypes of glioblastoma with different clinical and molecular characteristics, offering prognostic value beyond IDH1. Sci Rep. 2018. March 23;8(1):5087. doi: 10.1038/s41598-018-22739-2. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R37] 37.Echegaray S, Gevaert O, Shah R, Kamaya A, Louie J, Kothary N, Napel S. Core samples for radiomics features that are insensitive to tumor segmentation: method and pilot study using CT images of hepatocellular carcinoma. J Med Imaging. 2015. October;2(4):041011. doi: 10.1117/1.JMI.2.4.041011. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R38] 38.Shafiq-Ul-Hassan M, Zhang GG, Hunt DC, Latifi K, Ullah G, Gillies RJ, Moros EG. Accounting for reconstruction kernel-induced variability in CT radiomic features using noise power spectra. J Med Imaging (Bellingham). 2018. January;5(1):011013. doi: 10.1117/1.JMI.5.1.011013. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R39] 39.Balagurunathan Y, Gu Y, Wang H, Kumar V, Grove O, Hawkins S, Kim J, Goldgof DB, Hall LO, Gatenby RA, Gillies RJ. Reproducibility and Prognosis of Quantitative Features Extracted from CT Images. Transl Oncol. 2014. February 1;7(1):72–87. doi: 10.1593/tlo.13844. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R40] 40.Hunter LA, Krafft S, Stingo F, Choi H, Martel MK, Kry SF, Court LE. High quality machine-robust image features: identification in nonsmall cell lung cancer computed tomography images. Med Phys. 2013. December;40(12):121916. doi: 10.1118/1.4829514. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R41] 41.Buch K, Li B, Qureshi MM, Kuno H, Anderson SW, Sakai O. Quantitative Assessment of Variation in CT Parameters on Texture Features: Pilot Study Using a Nonanatomic Phantom. AJNR Am J Neuroradiol. 2017. May;38(5):981–985. doi: 10.3174/ajnr.A5139. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R42] 42.Caramella C, Allorant A, Orlhac F, Bidault F, Asselain B, Ammari S, Jaranowski P, Moussier A, Balleyguier C, Lassau N, Pitre-Champagnat S. Can we trust the calculation of texture indices of CT images? A phantom study. Med Phys. 2018. April;45(4):1529–1536. doi: 10.1002/mp.12809. [DOI] [PubMed] [Google Scholar]

[R43] 43.Prenosil GA, Weitzel T, Fürstner M, Hentschel M, Krause T, Cumming P, Rominger A, Klaeser B. Towards guidelines to harmonize textural features in PET: Haralick textural features vary with image noise, but exposure-invariant domains enable comparable PET radiomics. PLoS One. 2020. March 16;15(3):e0229560. doi: 10.1371/journal.pone.0229560. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R44] 44.Zhao B, Tan Y, Tsai WY, Schwartz LH, Lu L. Exploring Variability in CT Characterization of Tumors: A Preliminary Phantom Study. Transl Oncol. 2014. February 1;7(1):88–93. doi: 10.1593/tlo.13865. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R45] 45.Tan Y, Guo P, Mann H, Marley SE, Juanita Scott ML, Schwartz LH, Ghiorghiu DC, Zhao B. Assessing the effect of CT slice interval on unidimensional, bidimensional and volumetric measurements of solid tumours. Cancer Imaging. 2012. October 31;12(3):497–505. doi: 10.1102/1470-7330.2012.0046. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R46] 46.Blanc D, Racine V, Khalil A, Deloche M, Broyelle JA, Hammouamri I, et al. Artificial intelligence solution to classify pulmonary nodules on CT. Diagn Interv Imaging. 2020. December;101(12):803–810. doi: 10.1016/j.diii.2020.10.004. [DOI] [PubMed] [Google Scholar]

[R47] 47.Liu H, Cao H, Song E, Ma G, Xu X, Jin R, Liu C, Hung CC. Multi-model Ensemble Learning Architecture Based on 3D CNN for Lung Nodule Malignancy Suspiciousness Classification. J Digit Imaging. 2020. October;33(5):1242–1256. doi: 10.1007/s10278-020-00372-8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R48] 48.Quantitative vessel tortuosity: A potential CT imaging biomarker for distinguishing lung granulomas from adenocarcinomas. Alilou M, Orooji M, Beig N, Prasanna P, et al. Sci Rep. 2018. October 16;8(1):15290. doi: 10.1038/s41598-018-33473-0. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R49] 49.Kalousis A, Prados J & Hilario M Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl Inf Syst 12, 95–116 (2007). doi: 10.1007/s10115-006-0040-8. [DOI] [Google Scholar]

[R50] 50.Yu L, Ding C, Loscalzo S. Stable Feature Selection via Dense Feature Groups. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD 2008, New York, NY, USA, pp. 803–811. ACM; (2008). doi: 10.1145/1401890.1401986. [DOI] [Google Scholar]

PERMALINK

Distinguishing Granulomas from Adenocarcinomas by integrating stable and discriminating radiomic features on non-Contrast CT scans

Mohammadhadi Khorrami

Kaustav Bera

Rajat Thawani

Prabhakar Rajiah

Amit Gupta

Pingfu Fu

Philip Linden

Nathan Pennell

Frank Jacono

Robert C Gilkeson

Vamsidhar Velcheti

Anant Madabhushi

Abstract

Objective:

Methods:

Results:

Conclusions:

Introduction

Materials and Methods

Patient cohort

CT Acquisition

Nodule Segmentation and Feature Extraction

Feature Stability

Feature Discriminability

Experimental Design

Experiment 1: Distinguishing adenocarcinomas from granulomas on lung CT by evaluating the effect of site variability

Experiment 2: Distinguishing adenocarcinomas from granulomas on lung CT by evaluating the effect of site variability by controlling for slice thickness

Experiment 3: Evaluating resilience of radiomic features via Umap Embedding

Results

Distinguishing adenocarcinomas from granulomas on lung CT by evaluating the effect of site variability

Figure 1.

Table 1.

Figure 2.

Distinguishing adenocarcinomas from granulomas on lung CT by evaluating the effect of site variability by controlling for slice thickness

Figure 3.

Table 2.

Figure 4.

Figure 5.

Table 3.

Resilience of radiomic features via Umap Embedding

Figure 6.

Discussion

Conclusions

Supplementary Material

Highlights.

Acknowledgments

Footnotes

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases