Abstract
Purpose
To evaluate the short-term reproducibility of radiomic features in liver parenchyma and liver cancers in patients who underwent consecutive contrast-enhanced CT (CECT) with intravenous iodinated contrast within 2 weeks by chance.
Methods
The Institutional Review Board approved this HIPAA-compliant retrospective study and waived the requirement for patients’ informed consent. Patients were included if they had a liver malignancy (liver metastasis, n = 22, intrahepatic cholangiocarcinoma, n = 10, and hepatocellular carcinoma, n = 6), had two consecutive CECT within 14 days, and had no prior or intervening therapy. Liver tumors and liver parenchyma were segmented and radiomic features (n = 254) were extracted. The number of reproducible features (with concordance correlation coefficients > 0.9) was calculated for patient subgroups with different variations in contrast injection rate and pixel resolution.
Results
The number of reproducible radiomic features decreased with increasing variations in contrast injection rate and pixel resolution. When including all CECTs with injection rates differences of less than 15% vs. up to 50%, 63/254 vs. 0/254 features were reproducible for liver parenchyma and 68/254 vs. 50/254 features were reproducible for malignancies. When including all CT with pixel resolution differences of 0–5% or 0–15%, 20/254 vs. 0/254 features were reproducible for liver parenchyma; 34/254 liver malignancy features were reproducible with pixel differences up to 15%.
Conclusion
A greater number of liver malignancy radiomic features were reproducible compared to liver parenchyma features, but the proportion of reproducible features decreased with increasing variations in contrast injection rates and pixel resolution.
Keywords: Liver neoplasms, Texture analysis, Computed tomography, Repeatability, Biomarker
Radiomics is a rapidly expanding field where advances in computational image analysis are applied to growing medical imaging databases for the generation of high-dimensional datasets to investigate different clinical questions [1]. Radiomics employs computer-based analysis to extract a large number of quantitative features from digital images, often after segmentation of the region of interest. Multiple investigators have applied radiomics for diagnostic, prognostic, and predictive models in a number of different organs and malignancies, with promising results [2–9].
Despite the potential to develop predictive models [10], biomarkers must be informative, non-redundant, and reproducible prior to use in clinical decision making. The translation of radiomics to clinical practice requires rigorous testing, including validation with independent test data as well as investigations into the reproducibility of omics-based data [11]. Clinical variations in imaging protocols affect the appearance of CT images across and within institutions and consequently may influence the reproducibility of CT-based radiomics [12–16]. In lung cancer, the reproducibility of radiomic features has been investigated in a study using a publicly available dataset consisting of 31 test–retest non-contrast CT scans [17]. In this so-called “coffee break” study, patients underwent repeat non-contrast CT scanning approximately 15 min apart. Radiomic features extracted from manually delineated tumors were tested for repeatability between scans, using intra-class correlation coefficients (ICC) [13, 14]. In another study, extracted lung cancer CT features with higher reproducibility were shown to have higher prognostic performance [18]. However, unlike lung cancers, which are imaged routinely with non-contrast chest CT, most abdominal and pelvic tumors are evaluated on contrast-enhanced CT (CECT), after intravenous administration of iodinated contrast. Thus, the reproducibility of radiomic features for abdominal tumors cannot be established using a similar “coffee break” design due to the dynamic nature of contrast enhancement in the imaging of abdominal tumors.
One approach to a prospective CECT test–retest study appropriate for abdominal tumors would be to administer intravenous contrast in two separate sessions, with enough time between scans to allow for the excretion of contrast, but this carries an associated risk of additional contrast administration [19]. However, patients with abdominal tumors occasionally undergo repeat CT imaging within short time intervals for clinical reasons, for example, to evaluate new symptoms, such as abdominal pain or fever. Thus, in this retrospective study in patients with liver cancer, we aimed to evaluate CECT imaging of liver in an incidentally obtained “test–retest” CECT dataset. Given the large patient population with primary and secondary liver malignancies seen at our institution, we hypothesized that reproducible radiomic features could be identified for liver tumors in the setting of consecutive CECT with variable acquisition and reconstruction parameters and scanner models. The purpose of our study was to explore how routine variations in imaging affect short-term reproducibility of radiomic features derived from liver tumors and liver parenchyma in patients who undergo clinical CECT imaging twice within 2 weeks.
Methods
Patients
This was a HIPAA-compliant retrospective study conducted after a waiver of patient informed consent was obtained through Institutional Review Board approval. A database of consecutive patients at our institution was queried for patients with a diagnosis code of liver malignancy between June 2009 and October 2015 and the list was narrowed to include all patients who underwent two CECT scans within an interval of less than 15 days. Additional inclusion criteria were (1) the presence of a liver tumor confirmed by electronic medical record review: hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), or liver metastasis (with pathologic report of the liver tumor itself or primary cancer in the case of liver metastasis, supporting the diagnosis); (2) size of at least one tumor greater than 2 cm, and (3) DICOM images on Picture Archiving and Computer System (Centricity PACS, GE Healthcare) with CECT obtained during the portal venous phase and with slice reconstruction thickness of 5 mm or less. Exclusion criteria included any systemic or loco-regional treatment (i.e., chemotherapy, surgical procedure, portal vein embolization, local ablation) directed to the tumor during the time interval between the two consecutive CECTs or within one month preceding the first CT, and non-evaluable CECTs (e.g., with imaging artifacts over the liver tumor).
CT imaging
All patients underwent CECT with one of three CT scanner models: Lightspeed 16, Lightspeed VCT, or Discovery CT 750 HD detector scanner (GE Medical Systems, Chicago, IL, USA). All acquisitions were performed at 120 kVp, exposure time: 500–1100 ms and tube current: 133–440 mA. Images were reconstructed at a section thickness varying from 2.5 to 5 mm with a standard convolutional kernel and with a reconstruction diameter range of 360–500 mm. 150 mL iodinated contrast material (Omnipaque 300, GE Healthcare, Chicago, IL, USA) was administered intravenously for each CECT at a rate between 1 and 4 cc/s.
Image processing
CECT scans were downloaded to a workstation and Scout Liver (Pathfinder Technologies Inc., Peabody, MA) was used to semi-automatically segment the non-tumoral liver parenchyma, vessels, and tumors. A research fellow (T.P.) under attending radiologist supervision (R.D.) performed all segmentations. The details of the segmentation approach utilized by Scout Liver and accuracy and repeatability of segmentations were established previously [20, 21]. For patients with more than one set of CECT scans meeting the inclusion criteria (n = 4), only the first pair was included for analysis. In patients with multiple (two or more) liver tumors, the largest tumor was included in the analysis. In cases where confluent liver tumors were present, a lesion with clearly defined tumor was chosen as a representative lesion for analysis.
Image analysis
Two hundred and fifty-four well-known radiomic features, reflecting heterogeneity in enhancement patterns, were extracted from the segmented liver parenchyma and from one liver tumor per patient in both scans. The feature set included 19 features from gray-level co-occurrence matrix (GLCM) [22–24], 11 from run length matrix (RLM) [25], 5 from intensity histogram (IH), 127 from local binary patterns (LBP) [26–28], 54 from fractal dimension (FD) [29, 30], and 19 from each of the angle co-occurrence matrix 1 (ACM1) and ACM2 [31–33]. The details of the features are provided in Appendix 1. All quantitative image analysis was implemented in Matlab (MathWorks, Natick, MA, USA). Acquisition and reconstruction parameters were extracted from the DICOM headers and electronic medical record for each CT study including scanner model, slice thickness, pixel size, kVp, mAs, contrast injection dose, and administration rate.
Statistical analysis
Statistical analysis was performed using SPSS version 22.0 (IBM, Armonk, NY, USA). Two hundred fifty-four features were reported with summary statistics, for both liver parenchyma and tumor. Concordance correlation coefficients (CCCs) were used to quantify the reproducibility of a feature. A reproducible feature was defined by a CCC > 0.9 [34], regarding parameters setting variations (i.e., contrast material injection rate, pixel resolution, and machine consistency). Analyses were also performed for lower CCC threshold (> 0.85, > 0.80). The Wilcoxon signed rank test was used to compare continuous variables between each scan–rescan set [35]. Categorical variables were compared using Chi-square test. A p value < 0.05 was considered a statistically significant difference. The effect of change in contrast injection rate, pixel resolution, and scanner model on radiomic features was assessed for both liver parenchyma and index tumor regions. These three reflect acquisition and reconstruction parameters routinely modified by CT technologists: contrast injection rate is modified based on the type of venous access available, pixel resolution varies depending on the size of the scan field of view, and CT scanner model choice is based on availability of the scanner at the time of scheduling.
Results
A total of 100 scan–rescan paired sets of liver tumors met the initial inclusion criteria. Fifty-two sets of scans were excluded: 47 based on concurrent treatments and 5 due to imaging artifacts (e.g., presence of metallic objects). Of the remaining 48 sets, 4 patients had more than one set within 15 days (10 sets total). Since only the first pair of consecutive CTs for each patient was included in the planned analysis, our final cohort consisted of 38 scan– rescan sets, one set each from 38 patients. The etiologies of liver malignancy in the included patients were liver metastasis (n = 22), ICC (n = 10), and HCC (n = 6). All patients had pathologic diagnosis confirmation except one patient with IHCC who was diagnosed based on imaging. Twenty-four (63.2%) patients had two or more tumors. Two patients had liver cirrhosis and no patient had portal vein thrombus. The mean time interval between the two scans was 9.7 days (range 3–14) (Table 1).
Table 1.
Patient characteristics
| Tumor type (n = 38) | |
|---|---|
| HCC | 6 (16%) |
| IHCC | 10 (26%) |
| Metastases | 22 (58%) |
| Pancreas | 6 |
| NET | 6 |
| Colon | 2 |
| Rectum | 2 |
| Lung | 2 |
| Ovary | 1 |
| Melanoma | 1 |
| Hilar cholangiocarcinoma | 1 |
| Time between scans (days) | 9.7 (3–14) |
| Number of tumors | |
| 1 | 15 (39%) |
| > 1 to < 5 | 15 (39%) |
| ≥ 5 to < 10 | 3 (1%) |
| ≥ 10 | 5 (13%) |
HCC hepatocellular carcinoma, IHCC intrahepatic cholangiocarcinoma, NET neuroendocrine tumor
We compared imaging acquisition and reconstruction parameters and scanner models across the first and second CECT scans (Table 2). Significant differences were found for pixel resolution (p < 0.05), scanner model (p < 0.01), slice thickness (p < 0.05), and exposure (p < 0.05) across CECTs. Twenty-two patients (58%) underwent both CECTs on the same CT scanner model. No statistically significant differences were observed between first and second CECTs for the following: contrast injection rate reconstruction diameter, exposure time, and tube current (Table 2). Matrix sizes and kVp were constant across all CECTs.
Table 2.
Imaging reconstruction and acquisition variables
| Scan 1 | Scan 2 | p value | |
|---|---|---|---|
| Model | |||
| Lightspeed VCT | 10 (26%) | 3 (8%) | p < 0.01 |
| Lightspeed16 | 18 (26%) | 32 (84%) | |
| Discovery | 10 (48%) | 3 (8%) | |
| Pixel resolution | 0.791 (0.703–0.918) | 0.848 (0.703–0.977) | p < 0.05 |
| Thickness (mm) | 4.8 (2.5–5) | 5 (5) | p < 0.05 |
| Reconstruction diameter | 415.9 (360–500) | 423.2 (360–500) | – |
| Exposure time | 773.6 (500–1100) | 815.3 (699–1159) | – |
| Exposure | 34.1 (7–88) | 37.9 (13–96) | p < 0.05 |
| Tube current | 299.3 (133–440) | 315 (156–420) | – |
| Matrix size | |||
| Row | 512 (512) | 512 (512) | – |
| Column | 512 (512) | 512 (512) | – |
| Voltage (kVp) | 120 (120) | 120 (120) | – |
| Contrast agent | |||
| Contrast volume (mL) | 150 (150) | 150 (150) | – |
| Rate (cc/s) | 2.33 (1–4) | 2.18 (1–4) | – |
Influence of contrast injection rate
We evaluated the influence of contrast injection rate against a feature set composed of 254 features: 19 features from gray-level co-occurrence matrix (GLCM) [22–24], 11 from run length matrix (RLM) [25], 5 from intensity histogram (IH), 127 from local binary patterns (LBP) [26–28], 54 from fractal dimension (FD) [29, 30], and 19 from each of the angle co-occurrence matrix 1 (ACM1) and ACM2 [31–33].
Reproducibility of radiomics features was affected by changes in contrast injection rate between CECTs. The mean contrast injection rate was 2.25 cc/s (range 1–4). The mean rate difference between scan and rescan was 0.15 cc/s (range 0–2.5, p = 0.88). The number of reproducible imaging features [having a concordance correlation coefficients (CCC) > 0.9] was highest when the analysis was limited to patients who had scan sets with similar contrast injection rate, and correspondingly, the number of reproducible features decreased as a greater number of scans with larger differences in contrast injection rate was included (Figs. 1, 2).
Fig. 1.
Number of reproducible imaging features derived from liver parenchyma with variations in contrast injection rate.
Fig. 2.
Number of reproducible imaging features derived from liver tumors with variations in contrast injection rate.
For liver parenchyma-derived imaging features, limiting the analysis to sets where the contrast injection rate difference was less or equal to 15% (n = 10 patients) yielded the greatest number of reproducible features: 63/254 features (6 GLCM, 4 RLM, 1 IH, 19 LBP, 18 FD, 8 ACM1, and 7 ACM2). When the analysis included sets with a change in contrast injection rate up to 25% (n = 13 patients), the number of reproducible features dropped to only 4/254 (LBP4, LBP18, LBP71, and LBP72). When allowing for a change in contrast injection rate up to 50% (n = 21 patients), no feature reached a CCC > 0.9 (Fig. 1). A greater number of features met a lower threshold CCC > 0.8, but the number of such features also decreased as the variability in contrast injection rate increased (Fig. 1).
For radiomic features derived from segmented tumors, changes in contrast injection rate had a lower impact on the reproducibility of radiomic features. 68/254 features (8 GLCM, 5 RLM, 1 IH, 20 LBP, 32 FD, 1 ACM1, and 1 ACM2) were reproducible in patients with changes in contrast injection rate less or equal to 15% (n = 10 patients), while 50/254 features (3 GLCM, 5 RLM, 13 LBP, and 29 FD) were reproducible when the changes in contrast injection rate was up to 50% (n = 21) (Fig. 2).
Influence of pixel resolution
The reproducibility of radiomic features also varied with changes in pixel resolution between scan–rescan sets (Figs. 3, 4). The number of reproducible features was higher when the analysis was limited to smaller variations in pixel resolution between scan–rescan sets, for both features derived from liver parenchyma and features derived from tumor. The mean pixel resolution was 0.820 mm (range 0.703–0.977). The mean pixel resolution difference between scan and rescan was 7.27% (range 0–30.8%, p < 0.05).
Fig. 3.
Number of reproducible imaging features derived from liver parenchyma with variations in pixel resolution.
Fig. 4.
Number of reproducible imaging features derived from liver tumor with variations in pixel resolution.
For features derived from the liver parenchyma, when including all sets with pixel resolution variation of 0–5% (n = 18 patients), 20/254 radiomic features were reproducible (20 LBP). When allowing for pixel resolution variation of up to 15% (n = 34 patients), no feature was reproducible; a greater number of features met a lower threshold CCC > 0.8 (Fig. 3).
Compared to liver parenchyma-derived features, a greater number of radiomic features derived from the tumor region were reproducible. For example, with a pixel resolution variation of up to 15% (n = 34), 34/254 features (1 GLCM, 2 RLM, 5 LBP, and 26 FD) were reproducible.
Influence of scanner model
When the reproducibility analysis was limited to patients with scan–rescan sets with the same scanner model (n = 22 patients), 75 features extracted from the tumor region (10 GLCM, 5 RLM, 22 LBP, 31 FD, 4 ACM1, and 3 ACM2) (Fig. 5) and 3 features from the liver parenchyma region were reproducible (LBP72, LBP75, and LBP76) (Fig. 6). The number of reproducible features for the tumor and parenchyma regions decreased to 35/254 and 0/254, respectively, when all the patients were included (n = 38 patients), regardless of scanner model used. Only 12/254 features from the tumor region (2 GLCM and 10 FD) and 10/254 liver parenchyma (9 LBP and 1 ACM2) extracted features were reproducible when the machine was inconsistent across both scans.
Fig. 5.
Number of reproducible imaging features derived from tumor region on similar, different, or mixed (both similar and different) CT scanner models.
Fig. 6.
Number of reproducible imaging features derived from liver parenchyma on similar, different, or mixed (both similar and different) CT scanner models.
Discussion
The results of the current study underscore the impact of routine variation in image acquisition and reconstruction parameters and scanner model on the reproducibility of radiomic features extracted from liver parenchyma and liver tumors. These data highlight potential challenges in the use of quantitative imaging features derived from CT images obtained after intravenous contrast enhancement with different imaging protocols. Notably, variations in contrast injection rate, pixel resolution, and scanner model affected the number of reproducible radiomic features derived from CECTs performed in the same patients within a short time interval. A greater number of reproducible features were found when the same CT scanner model was used for the CECTs, suggesting that consistent imaging protocol yielded similar imaging features at multiple time points; prospective evaluation of reproducibility is still warranted. A smaller number of liver parenchyma-derived imaging features were reproducible compared to liver tumor-derived imaging features, under the same conditions. The greater reproducibility of liver tumor-derived imaging features is intriguing and may be related to the greater heterogeneity in enhancement pattern in liver malignancies. In comparison, liver parenchymal enhancement patterns may have a lower range of enhancement patterns, affecting their reproducibility as measured by CCC. This is a finding that merits further investigation, with larger datasets of patients who are grouped by similar liver pathologies.
The current study is the first to date to investigate the reproducibility of commonly reported CECT liver imaging features in the radiomics literature. Previous studies studying the reproducibility of radiomics features have mainly reported on features derived from non-contrast imaging, finding that many features are unstable between test–retest imaging scans in different organs. For example, Balagurunathan et al. [13, 14] assessed the CCC of 219 radiomic features from test–retest CT in lung cancer patients acquired 15 min apart. Of these features, only 66 were found to have CCC > 0.9, suggesting that a large number of features may be omitted from routine radiomic analysis, which is comparable to our results using CECT in liver tumors. Galavis et al. [36] assessed the reproducibility of 50 texture features in 20 patients with solid tumors in FDG-PET under different acquisition modes and with different reconstruction parameters. Only 4 features (entropy-first order, energy, maximal correlation coefficient, and low gray-level run emphasis) exhibited small vs. large variations due to five different acquisition modes and reconstruction parameters. On the other hand, Leijenaar et al. [4] analyzed the reproducibility of 91 FDG-PET image features within a 1-day interval in 23 patients with lung tumors and reported that the majority of assessed features had both high test–retest (71%) and inter-observer (91%) reproducibility. They also suggested that features that were more reproducible in repeated PET-CT imaging were also more robust against inter-observer variability. In our study, our features exhibited more variability, which may be due to the differences in imaging modalities and the larger mean time gap of 10 days between two CECTs.
Most studies investigating the stability of radiomic features assessed non-contrast-enhanced images; however, diagnostic CT imaging of the liver requires contrast intravenous administration. Texture features for these images have not yet been widely investigated. Recently, a study assessed the reproducibility of 122 radiomic features extracted from CECTs for lung metastases demonstrating that commonly used texture features (87%) were reproducible between different scan times after contrast injection and between different scanning sessions several days apart (2–7 days) [37]. This study included only eight patients with small and well-defined lung metastases using a single machine with a unique imaging protocol.
This study has a number of limitations, some related to the retrospective nature of the study. The number of patients is relatively small at 38, but is comparable in size to the RIDER dataset used in previous radiomic reproducibility studies which includes 32 patients with non-small cell lung cancer. In addition, imaging was performed on different CT scanners for 16 patients, with a variety of image acquisition and reconstruction parameters, precluding a systematic analysis of the effect of individual parameters. Ideally, a single parameter would be varied at a time, with all others are kept constant, to establish how each acquisition parameter or how the segmentation software itself affects reproducibility. We focused on contrast injection rate, pixel spacing, and scanner model consistency, which are often affected by CT technologists. Pixel resolution depends on the field of view prescribed, which usually varies from 40 to 50 cm; contrast injection rate can be modified based on intravenous access and catheter size available at the time of imaging, while scanner model may depend on the availability of the CT scanner at the time of the scan. Despite these limitations, the results show the potential loss of reproducible imaging features as CT imaging acquisition parameters diverge. This study is also limited by the variety of primary and secondary liver malignancies included; due to the small sample size, we avoided further analysis by tumor subtype. Another limitation relates to the small number of patients with confirmed cirrhosis, preventing a subanalysis of this patient population. Finally, due to the heterogeneous group of patients, we could not compare the effect of reproducibility on a predicted patient outcome. Ideally, a homogeneous group of patients in a prospective clinical trial setting would inform how changes in reproducibility of imaging features affect the prediction of a clinical trial outcome. Finally, the choice of a CCC threshold > 0.9 is relatively arbitrary, and a lower threshold may be sufficient for an imaging feature to be used as a biomarker. For this reason, we show additional results with CCC > 0.85 and 0.8. Nevertheless, our study illustrates the potential effects of variable imaging protocols and scanners on imaging feature reproducibility.
Conclusion
Similar to lung cancers, liver tumors may have a limited number of reproducible imaging features available for radiomic studies. Furthermore, since liver tumors are routinely imaged with contrast, variations in imaging parameters rate appear to reduce the number of reproducible imaging features. The effect of diminished imaging feature reproducibility on biomarker development remains uncertain. Nevertheless, our results challenge the use of any single radiomic feature to make clinical prediction before their reproducibility is measured. Further systematic investigations into the reproducibility of liver-derived imaging features on CECT are needed to support the development of liver radiomics into clinically useful biomarkers. As many factors affect radiomic features, further investigation on reproducibility of quantitative imaging features is warranted through prospective studies.
Supplementary Material
Acknowledgments
We thank Joanne Chin for editorial assistance.
Funding This study was funded in part through the 2016 Society of Abdominal Radiology Wylie J. Dodds Research Award and the National Institutes of Health/National Cancer Institute Cancer Center Support Grant P30 CA008748.
Footnotes
Electronic supplementary material The online version of this article (https://doi.org/10.1007/s00261-018-1600-6) contains supplementary material, which is available to authorized users.
Author contributions Study concepts and study design: MG, AS, RKGD; literature search: AM, RY, AS, RKGD; image review: TP, RY, RKGD; clinical information review: TP, RY, TS, RKGD; statistical analysis: AM, MG; manuscript drafting and edition: TP, AM, RY, AS, RKGD; approval of final version of submitted manuscript: all authors.
Compliance with ethical standards
Conflict of interest The authors declare that they have no conflict of interest.
Ethical approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study formal consent is not required. This article does not contain any studies with animals performed by any of the authors.
References
- 1.Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2016;278(2):563–577. doi: 10.1148/radiol.2015151169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gatenby RA, Grove O, Gillies RJ. Quantitative imaging in cancer evolution and ecology. Radiology. 2013;269(1):8–15. doi: 10.1148/radiol.13122697. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hunter LA, Krafft S, Stingo F, et al. High quality machine-robust image features: identification in nonsmall cell lung cancer computed tomography images. Med Phys. 2013;40(12):121916. doi: 10.1118/1.4829514. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Leijenaar RT, Carvalho S, Velazquez ER, et al. Stability of FDG-PET radiomics features: an integrated analysis of test-retest and inter-observer variability. Acta Oncol (Stockholm, Sweden) 2013;52(7):1391–1397. doi: 10.3109/0284186x.2013.812798. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Sadot E, Simpson AL, Do RK, et al. Cholangiocarcinoma: correlation between molecular profiling and imaging phenotypes. PloS One. 2015;10(7):e0132953. doi: 10.1371/journal.pone.0132953. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Segal E, Sirlin CB, Ooi C, et al. Decoding global gene expression programs in liver cancer by noninvasive imaging. Nat Biotechnol. 2007;25(6):675–680. doi: 10.1038/nbt1306. [DOI] [PubMed] [Google Scholar]
- 7.Ganeshan B, Abaleke S, Young RC, Chatwin CR, Miles KA. Texture analysis of non-small cell lung cancer on unenhanced computed tomography: initial evidence for a relationship with tumour glucose metabolism and stage. Cancer Imaging. 2010;10:137–143. doi: 10.1102/1470-7330.2010.0021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ganeshan B, Goh V, Mandeville HC, et al. Non-small cell lung cancer: histopathologic correlates for texture parameters at CT. Radiology. 2013;266(1):326–336. doi: 10.1148/radiol.12112428. [DOI] [PubMed] [Google Scholar]
- 9.Ganeshan B, Panayiotou E, Burnand K, Dizdarevic S, Miles K. Tumour heterogeneity in non-small cell lung carcinoma assessed by CT texture analysis: a potential marker of survival. Eur Radiol. 2012;22(4):796–802. doi: 10.1007/s00330-011-2319-8. [DOI] [PubMed] [Google Scholar]
- 10.Coroller TP, Agrawal V, Narayan V, et al. Radiomic phenotype features predict pathological response in non-small cell lung cancer. Radiother Oncol. 2016;119(3):480–486. doi: 10.1016/j.radonc.2016.04.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Committee on the Review of Omics-Based Tests for Predicting Patient Outcomes in Clinical T, Board on Health Care S, Board on Health Sciences P, Institute of M. Evolution of translational omics: lessons learned and the path forward. National Academies Press (US); Washington (DC): 2012. Copyright 2012 by the National Academy of Sciences. All rights reserved. https://doi.org/10.17226/13297. [PubMed] [Google Scholar]
- 12.Al-Kadi OS. Assessment of texture measures susceptibility to noise in conventional and contrast enhanced computed tomography lung tumour images. Comput Med Imaging Graph. 2010;34(6):494–503. doi: 10.1016/j.compmedimag.2009.12.011. [DOI] [PubMed] [Google Scholar]
- 13.Balagurunathan Y, Gu Y, Wang H, et al. Reproducibility and prognosis of quantitative features extracted from CT images. Transl Oncol. 2014;7(1):72–87. doi: 10.1593/tlo.13844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Balagurunathan Y, Kumar V, Gu Y, et al. Test-retest reproducibility analysis of lung CT image features. J Digit Imaging. 2014;27(6):805–823. doi: 10.1007/s10278-014-9716-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Summers RM. Texture analysis in radiology: does the emperor have no clothes? Abdom Radiol (N Y) 2017;42(2):342–345. doi: 10.1007/s00261-016-0950-1. [DOI] [PubMed] [Google Scholar]
- 16.Solomon J, Mileto A, Nelson RC, Roy Choudhury K, Samei E. Quantitative features of liver lesions, lung nodules, and renal stones at multi-detector row CT examinations: dependency on radiation dose and reconstruction algorithm. Radiology. 2016;279(1):185–194. doi: 10.1148/radiol.2015150892. [DOI] [PubMed] [Google Scholar]
- 17.Zhao B, James LP, Moskowitz CS, et al. Evaluating variability in tumor measurements from same-day repeat CT scans of patients with non-small cell lung cancer. Radiology. 2009;252(1):263–272. doi: 10.1148/radiol.2522081593. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Aerts HJ, Velazquez ER, Leijenaar RT, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014;5:4006. doi: 10.1038/ncomms5006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.American College of Radiology. Committee on drugs and contrast media. ACR Man Contrast Media Version. 2016;10:2. [Google Scholar]
- 20.Hermoye L, Laamari-Azjal I, Cao Z, et al. Liver segmentation in living liver transplant donors: comparison of semiautomatic and manual methods. Radiology. 2005;234(1):171–178. doi: 10.1148/radiol.2341031801. [DOI] [PubMed] [Google Scholar]
- 21.Simpson AL, Geller DA, Hemming AW, et al. Liver planning software accurately predicts postoperative liver volume and measures early regeneration. J Am Coll Surg. 2014;219(2):199–207. doi: 10.1016/j.jamcollsurg.2014.02.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Haralick RM, Shanmugam K, Dinstein I. Textural features for image classification. IEEE Trans Syst Man Cybern. 1973;3(6):610–621. doi: 10.1109/tsmc.1973.4309314. [DOI] [Google Scholar]
- 23.Yang X, Tridandapani S, Beitler JJ, et al. Ultrasound GLCM texture analysis of radiation-induced parotid-gland injury in head-and-neck cancer radiotherapy: an in vivo study of late toxicity. Med Phys. 2012;39(9):5732–5739. doi: 10.1118/1.4747526. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Banik S, Rangayyan RM, Desautels JE. Measures of angular spread and entropy for the detection of architectural distortion in prior mammograms. Int J Comput Assist Radiol Surg. 2013;8(1):121–134. doi: 10.1007/s11548-012-0681-x. [DOI] [PubMed] [Google Scholar]
- 25.Tang X. Texture information in run-length matrices. IEEE Trans Image Process. 1998;7(11):1602–1609. doi: 10.1109/83.725367. [DOI] [PubMed] [Google Scholar]
- 26.Ojala T, Pietikäinen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 1996;29(1):51–59. doi: 10.1016/0031-3203(95)00067-4. [DOI] [Google Scholar]
- 27.Pietikäinen M, Hadid A, Zhao G, Ahonen T. Computer vision using local binary patterns. London: Springer; 2011. Local binary patterns for still images; pp. 13–47. [Google Scholar]
- 28.Mehta R, Egiazarian KO. Rotated local binary pattern (RLBP)-rotation invariant texture descriptor. ICPRAM. 2013:497–502. [Google Scholar]
- 29.Al-Kadi OS, Watson D. Texture analysis of aggressive and nonaggressive lung tumor CE CT images. IEEE Trans Biomed Eng. 2008;55(7):1822–1830. doi: 10.1109/tbme.2008.919735. [DOI] [PubMed] [Google Scholar]
- 30.Costa AF, Humpire-Mamani G, Traina AJM. An efficient algorithm for fractal analysis of textures. Proceedings of the 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) 2012:39–46. [Google Scholar]
- 31.Chakraborty J, Rangayyan RM, Banik S, Mukhopadhyay S, Desautels JL. Statistical measures of orientation of texture for the detection of architectural distortion in prior mammograms of interval-cancer. J Electron Imaging. 2012;21(3):033010–033011. 033010–033013. [Google Scholar]
- 32.Chakraborty J, Rangayyan RM, Banik S, Mukhopadhyay S, Desautels JL. Detection of architectural distortion in prior mammograms using statistical measures of orientation of texture. Proceedings of the SPIE. 2012:831521. [Google Scholar]
- 33.Chakraborty J, Midya A, Mukhopadhyay S, Sadhu A. Automatic characterization of masses in mammograms. IEEE 6th international conference on biomedical engineering and informatics. 2013:111–115. [Google Scholar]
- 34.Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45(1):255–268. [PubMed] [Google Scholar]
- 35.Symons R, Morris JZ, Wu CO, Pourmorteza A. Coronary CT angiography: variability of CT scanners and readers in measurement of plaque volume. Radiology. 2016;281(3):737–748. doi: 10.1148/radiol.2016161670. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Galavis PE, Hollensen C, Jallow N, Paliwal B, Jeraj R. Variability of textural features in FDG PET images due to different acquisition modes and reconstruction parameters. Acta Oncol. 2010;49(7):1012–1016. doi: 10.3109/0284186x.2010.498437. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Yang J, Zhang L, Fave XJ, et al. Uncertainty analysis of quantitative imaging features extracted from contrast-enhanced CT in lung tumors. Comput Med Imaging Graph. 2016;48:1–8. doi: 10.1016/j.compmedimag.2015.12.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.