Radiomics nomogram: A noninvasive tool for preoperative evaluation of the invasiveness of pulmonary adenocarcinomas manifesting as ground-glass nodules

Highlights

• •It is vital to distinguish indolent pulmonary adenocarcinomas from invasive pulmonary adenocarcinomas before surgery.
• •Radiomics is a cutting-edge technology that mines quantitative features from CT images.
• •We designed a nomogram, which incorporated clinical and CT morphological characteristics with the radiomics signature.
• •We applied the radiomics nomogram to preoperatively predict the invasiveness of GGNs.

Abstract

In this study, we aimed to establish a radiomics nomogram that noninvasively evaluates the invasiveness of pulmonary adenocarcinomas manifesting as ground-glass nodules (GGNs). Computed tomography (CT) images of 509 patients manifesting as GGNs were collected: 70% of cases were included in the training cohort and 30% in the validation cohort. The Max-Relevance and Min-Redundancy (mRMR) and the least absolute shrinkage and selection operator (LASSO) algorithm were used to select the radiomics features and construct a radiomics signature. Univariate and multivariate logistic regression were used to select the invasiveness-related clinical and CT morphological predictors. Age, smoking history, long diameter, and average CT value were retained as independent predictors of GGN invasiveness. A radiomics nomogram was established by integrating clinical and CT morphological features with the radiomics signature. The radiomics nomogram showed good predictive ability in the training set (area under the curve [AUC], 0.940; 95% confidence interval [CI], 0.916–0.964) and validation set (AUC, 0.946; 95% CI, 0.907–0.986). This radiomics nomogram may serve as a noninvasive and accurate predictive tool to determine the invasiveness of GGNs prior to surgery and assist clinicians in creating personalized treatment strategies.

Introduction

With an increased awareness of the importance of physical examination, the widespread use of computed tomography (CT) has increased the incidental detection of ground glass nodules (GGNs), defined as hazy increased attenuations of the lung with preservation of the bronchial and vascular margins [1]. Most early-stage lung cancers are adenocarcinomas and manifest as ground-glass nodules (GGNs) on thin-slice CT [2].

In 2011, a new classification system for lung adenocarcinoma was proposed by the International Association for the Study of Lung Cancer, American Thoracic Society, and European Respiratory Society. Lung adenocarcinomas are classified as preinvasive lesions such as atypical adenomatous hyperplasia (AAH) and adenocarcinoma in situ (AIS), minimally invasive adenocarcinomas (MIAs), and invasive pulmonary adenocarcinomas (IPAs) [3]. However, previous studies have reported that the 5-year recurrence-free survival rate of patients with preinvasive lesions and MIAs is nearly 100% [4]. Therefore, preinvasive lesions and MIAs are defined as indolent lesions, and IPAs are defined as invasive lesions. Prognosis also determines clinical treatment strategies. Specifically, published reports recommend follow-up or sub-lobular resection for indolent lesions [5,6], whereas lobectomy with systemic lymph node dissection is generally used for patients with IPAs [7]. Therefore, accurately distinguishing indolent lesions (i.e., AAH, AIS, and MIA) from IPAs before surgery is vital to guide clinical treatment strategies.

In routine clinical work, we often use CT morphological features to differentiate the pulmonary nodules. However, considering the overlapping of features, it remains a challenge for radiologists to evaluate the invasiveness of GGNs prior to operation. It is also a challenge to accurately distinguish indolent lesions from IPAs using intraoperative frozen sections because the final diagnosis requires a rigorous evaluation of the entire tumor to exclude the existence of invasive components [4]. Radiomics is a cutting-edge technology that extracts high-throughput data and mines reproducible quantitative features from CT images [8,9]. It has been applied in lung-disease diagnosis, prediction of prognosis, and evaluation of treatment response as a bridge between medical imaging and personalized medicine [10].

Previously, researchers focused on clinical and CT morphological features to diagnose the invasiveness of GGNs [11], [12], [13], [14]. Further, some studies used CT texture analysis/radiomics to diagnose invasiveness of GGNs [15], [16], [17], [18]. However, these studies were limited to a single methodology, which extracted inadequate features to construct diagnostic models. In recent years, two studies [19,20] constructed a multi-dimensional parametric model that combined clinical and CT morphological features with a radiomics signature to differentiate preinvasive lesions (AAH/AIS) from MIAs/IPAs. However, the prognosis and treatment strategy of MIAs are similar to those of preinvasive lesions, so it is more meaningful to differentiate AAH/AIS /MIA from IPA for clinical decision-making. Based on the above literature review, we intend to integrate clinical and CT morphological features with a radiomics signature to construct a nomogram for differentiating AAH/AIS/MIA from IPA.

Materials and methods

This retrospective study received ethical approval from the First Hospital of Jilin University, and the requirement for informed consent was waived. The workflow is presented in Fig. 1.

Fig. 1.

The workflow of this study.

Patients

First, we retrospectively reviewed the electronic pathological records of patients with AAH, AIS, MIA, and stage T1N0 IPA from January 2016 to October 2018. Following this review, 1037 patients were initially included. Next, we reviewed the last preoperative CT scans of these patients and selected the thin-section plain CT scans manifested as GGNs. Then, we analyzed the radiological characteristics of the GGNs and selected solitary GGNs ≤ 30 mm in diameter that were completely surrounded by aerated lung and that were not associated with atelectasis, mediastinal lymphadenopathy, or pleural effusion [21]. Fig. 2 shows the flow of patient recruitment. Finally, 509 GGNs were included in the study cohort, and their histopathological classification was as follows: AAH group (12 patients), AIS group (58 patients), MIA group (152 patients), and IPA group (287 patients). The patients were randomly divided into the training and validation cohorts in a 7:3 ratio.

Fig. 2.

Flowchart of the patient recruitment pathway. Flowchart shows how selected the study population and its retrospective manner. Numbers in parentheses are numbers of patients. GGN=ground-glass nodule.

Clinical and CT morphological characteristics evaluation

Clinical characteristics, including sex, age, and smoking history, were obtained from the lung cancer database at our hospital. We summarized the CT morphological characteristics used in previous studies [11], [12], [13], [14] and classified them as categorical or continuous variables.

For categorical variables, two radiologists with 12 (reader 1) and 7 (reader 2) years of experience, who were blinded to the pathological results, reviewed each GGN independently on lung window images (level, −500 Hounsfield units [HU]; width, 1500 HU) until they reached a consensus. The categorical variables were as follows: (a) nodule location, (b) type of nodule, pure GGN or mixed GGN, (c) interface of nodule, well-defined and ill-defined, (d) lobulation, (e) vacuole, (f) pleural indention, and (g) vascular convergence. The definition is presented in Supplementary S1.

For continuous variables, two radiologists (readers 1 and 2) independently segmented each GGN using the IntelliSpace Discovery platform (ISD, Philips Healthcare, Best, The Netherlands). The software semi-automatically generates a three-dimensional region of interest (3D-ROI) and automatically calculates the long and short diameters (mm) on the largest axial section, volume (cm3), and average CT value (HU). Differences between the measurements of the two radiologists achieved a satisfactory inter-observer reproducibility of interclass correlation coefficients (ICCs) > 0.8. Finally, we used the results from the senior radiologist (reader 1) for subsequent analysis.

CT image acquisition, segmentation, and radiomics feature extraction

We selected the last thin-section plain CT scan before surgery for each patient. All of the CT images were obtained at full inspiration in the supine position. Details regarding the CT protocol are shown in Supplementary Table S1. CT version and image thickness may have affected the CT phenotype. To test the generalization ability of the radiomics signature, we performed stratification analysis on the subgroups of CT version and image thickness.

The enrolled GGNs were segmented by two radiologists (reader 1 and reader 2) independently using the IntelliSpace Discovery platform (ISD, Philips Healthcare, Best, The Netherlands). The software semi-automatically generated a three-dimensional region of interest (3D-ROI), which was defined as the delineation around the nodule border for every CT axial plane. Thereafter, the inserted plugin-pyradiomics 2.1 [22] of feature extraction automatically extracted more than 1000 radiomics features, including volumetric, histogram, textural, and wavelet features. ICCs were used to assess the interobserver reproducibility of radiomics feature extractions. We retained radiomics features with an ICCs > 0.8.

Radiomics signature building

We compared five different feature selection methods as shown in Supplementary S2 and Table S2. Finally, we used the Max-Relevance and Min-Redundancy (mRMR) and the least absolute shrinkage and selection operator (LASSO) methods to extract features and establish the multi-variate logistic regression (LR) model, which has a good interpretability and judges the contribution of each feature by its weight coefficient. We used mRMR to eliminate the redundant and irrelevant features. The LASSO, which is suitable for the regression of high-dimensional data [23], was used to determine the most useful invasiveness-related radiomics features using 10-fold cross-validation and 100 repetitions in the training cohort. The radiomics signature was quantified as a radiomics score, which was calculated by summing the selected eight features weighted by their coefficients.

Radiomics nomogram construction

Univariate and multivariate logistic regression were used to select the invasiveness-related clinical and CT morphological predictors. A radiomics nomogram was established by integrating the retained clinical and CT morphologicalcharacteristics with the radiomics score.

Performance of the radiomics nomogram

Discrimination, calibration, and clinical use were used to evaluate the performance of the radiomics nomogram. A bar diagram was plotted to exhibit the discrimination performance of the radiomics signature. The receiver-operating characteristic (ROC) curve was used to show the discrimination performance of the radiomics nomogram, and the calibration curve was plotted to assess its calibration. The goodness-of-fit of the nomogram was assessed using the Hosmer–Lemeshow test. P>0.05 was considered well-calibrated. A decision curve analysis (DCA) was conducted to determine the clinical utility of the radiomics nomogram by calculating the net benefits of a range of threshold probabilities for the validation cohort [24].

Statistical analysis

Chi square test or Fisher's exact test was used for the nominal variable. Mann–Whitney test was used for the continuous variable with abnormal distribution. T-test was used for the continuous variable with normal distribution. A two-tailed p-value <0.05 indicated statistical significance. ICCs were used to assess the agreement of the continuous variable measurement and extracting features by two-level radiologists. All statistical analyses for the present study were performed with R (version 3.5.1; http://www.Rproject.org). The detailed used R packages of this paper are listed in the Supplementary S3.

Results

Clinical and CT morphological characteristics

We pathologically confirmed 509 GGNs as lung adenocarcinoma, among which 222 were indolent lesions (AAH, AIS, or MIA) and 287 were IPA at Stage T1N0. The clinical and CT morphological characteristics of the GGNs in the training and validation cohorts are shown in Table 1.

Table 1.

Clinical and CT morphological characteristics of patients in the training and validation cohorts.

Characteristics	Training cohort (n = 357)				Validation cohort (n = 152)
	Indolent lesions		IPAs	P-value	Indolent lesions	IPAs	P-value
Years/Median(25%, 75%)	53.50(46.00, 61.00)		60.00(54.00, 65.30)	<0.001	54.00(47.00, 62.00)	61.00(56.00, 65.00)	<0.001
Sex/No.(%)	Male	41(26.28)	70(34.83)	0.084	12(18.18)	27(31.40)	0.064
	Female	115(73.72)	131(65.17)		54(81.82)	59(68.60)
Smoking/No.(%)	Never	118(75.64)	58(28.86)	<0.001	57(86.36)	24(27.91)	<0.001
	Former/Current	38(24.36)	143(71.15)		9(13.64)	62(72.09)
Location/No.(%)	RUL	60(38.46)	75(37.31)	0.565	27(40.91)	30(34.88)	0.428
	RML	16(10.26)	16(7.96)		1(1.52)	6(6.98)
	RLL	30(19.23)	46(22.89)		12(18.18)	10(11.63)
	LUL	38(24.35)	41(20.40)		20(30.31)	33(38.37)
	LLL	12(7.69)	23(11.44)		6(9.09)	7(8.14)
Type/No.(%)	pGGN	144(92.31)	116(57.71)	<0.001	60(90.91)	44(51.16)	<0.001
	mGGN	12(7.69)	85(42.29)		6(9.09)	42(48.84)
Interface/No.(%)	Ill-defined	21(13.46)	17(8.46)	0.207	8(12.12)	8(9.30)	0.575
	Well-defined	135(86.54)	184(91.54)		58(87.88)	78(90.70)
Lobulation/No.(%)	Absent	124(79.49)	120(59.70)	<0.001	46(69.70)	51(59.30)	0.186
	Present	32(20.51)	81(40.30)		20(30.30)	35(40.70)
Vacuole/No.(%)	Absent	129(82.69)	130(64.68)	<0.001	58(87.88)	60(69.77)	0.008
	Present	27(17.31)	71(35.32)		8(12.12)	26(30.23)
Pleural indention/No.(%)	Absent	117(75.00)	132(65.67)	0.057	56(84.85)	60(69.77)	0.030
	Present	39(25.00)	69(34.33)		10(15.15)	26(30.23)
Vascular convergence/No.(%)	Absent	32(20.51)	22(10.95)	0.003	11(16.67)	9(10.47)	0.322
	Present	124(79.48)	179(89.06)		55(83.34)	77(89.53)
Short_diameter/Median(25%, 75%)	8.20(6.35, 11.01)		12.30(9.80, 15.43)	<0.001	8.40(6.20, 11.31)	12.75(9.80, 15.91)	<0.001
Long_diameter/Median(25%, 75%)	10.60(8.20, 14.05)		16.80(13.10, 20.40)	<0.001	10.95(7.59, 14.01)	16.25(12.90, 20.30)	<0.001
Volume/Median(25%, 75%)	0.46(0.24, 0.99)		1.50(0.86, 3.02)	<0.001	0.52(0.21, 1.03)	1.71(0.83, 2.79)	<0.001
Average CT value/Median(25%, 75%)	−634.70(−689.65, −562.85)		−534.00(−620.30, −456.10)	<0.001	−640.00(−699.05, −560.45)	−535.00(−603.05, −446.60)	<0.001
Radiomics score/Median(25%, 75%)	−0.97(−1.87, −0.09)		1.53(0.50, 2.27)	<0.001	−1.03(−2.41, 0.15)	1.36(0.69, 2.20)	<0.001

Note: Data are the number of patients with percentage in parentheses. Age is shown as mean ± standard deviation. Abbreviations: IPA, invasive pulmonary adenocarcinoma; RUL, right upper lobe; RML, right middle lobe; RLL, right lower lobe; LUL, left upper lobe; LLL, left lower lobe; pGGN, pure ground-glass nodule; mGGN, mixed ground-glass nodule; CT, computed tomography. Bold P values < 0.05.

We found that the median age, smoking history, type of nodule, vacuole, and all continuous variables (including long and short diameter, volume, and average CT value) showed statistically significant differences between the indolent lesions and IPAs, with P <0.05 in both the training and validation cohorts.

Feature selection and radiomics signature building

In the training cohort, the mRMR retained 30 features. The LASSO selected the optimal subset of eight radiomic features when log-λ of –4 was chosen (Fig. 3a–c). Radiomics score was calculated by summing the selected eight features weighted by their coefficients. The formula of the radiomics score and the details of the eight radiomics features were presented in the supplementary S4 and Table S3. The mean value of the radiomics score of IPAs was significantly higher than that of indolent lesions in both the training cohort (1.53 vs. −0.97, respectively; P<0.001) and validation cohort (1.36 vs. −1.03, respectively; P<0.001) (Table 1). The radiomics score for each patient is shown in the bar diagrams (Fig. 4a and b), which exhibit an outstanding discrimination performance. A stratified analysis showed that the performance of radiomics score was not affected by the vendors (GE, Philips, and Siemens) or slice thickness (Supplementary S5 and Fig. S1).

Fig. 3.

Radiomics feature selection using a LASSO regression model. (a) selection of the tuning parameter (λ) in the LASSO model via 10-fold cross-validation based on minimum criteria. Binomial deviances from the LASSO regression cross-validation procedure were plotted as a function of log (λ). (b) The vertical black dotted line drawn at the optimal Log(λ) of −4 resulted in eight non-zero coefficients using 10-fold cross-validation. (c) The y-axis indicates the selected eight radiomics features, and the x-axis represents the coefficient of the features.

Fig. 4.

Bar diagrams of the radiomics scores in the training (a) and validation (b) cohorts. Green and red bars refer to actual invasive and indolent nodules, respectively. Bars going up and down refer to the predicted invasive and indolent nodules, respectively. The green bar pointing up or the red bar pointing down indicates the radiomics score made a correct prediction. Conversely, the green bar pointing down or the red bar pointing up indicates the radiomics score made an incorrect prediction. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Radiomics nomogram construction

We performed univariate logistic analysis on the clinical and CT morphological characteristics in the training cohort, as shown in Table 1. Eleven characteristics associated with invasiveness of GGNs were retained with statistical significance (P <0.05) (Table 2). Then, multivariate logistic analysis was performed on these eleven characteristics. Finally, four characteristics were selected as independent predictors of invasiveness: age, smoking history, long diameter, and average CT value (Table 2). These were incorporated with the radiomics signature to develop a nomogram (Fig. 5).

Table 2.

Predictors for the invasiveness of GGNs in the training cohort.

Characteristics	Univariate logistic regression		Multivariate logistic regression
	OR (95% CI)	P	OR (95% CI)	P
Age	1.065 (1.041–1.091)	<0.001	1.033 (0.999–1.070)	0.0624
Sex	0.667 (0.421–1.057)	<0.001
Smoking	4.676 (3.360–6.509)	<0.001	4.427 (3.013–6.756)	<0.001
Type of nodule	8.793 (4.581–16.877)	<0.001
Lobulation	2.616 (1.618–4.227)	<0.001
Vacuole	2.609 (1.574–4.327)	<0.001
Pleural indention	1.568 (0.985–2.496)	<0.001
Long diameter	1.275 (1.204–1.350)	<0.001	1.234 (1.151–1.332)	<0.001
Short diameter	1.297 (1.211–1.388)	<0.001
Volume	2.863 (2.135–3.839)	<0.001
Average CT value	1.007 (1.005–1.010)	<0.001	1.008 (1.005–1.011)	<0.001

Note: Abbreviations: GGN, ground-glass nodules; OR, odds ratio; CI, confidence interval; CT, computed tomography.

Fig. 5.

The radiomics nomogram. It was constructed based on the integration with long diameter, average CT value, age, smoking history and radiomics score. The probabilities of each predictor can be converted into scores according to the first scale "point" at the top of the graph. The corresponding prediction probabilities are added up, and at the bottom of the sequence is the invasiveness of GGNs.

Evaluating the performance of the radiomics nomogram

The discriminating performances of the radiomics signature and radiomics nomogram are displayed using ROC curves (Fig. 6). The radiomics signature showed favorable predictive efficacy, with an AUC of 0.892 (95% CI, 0.860–0.926) in the training cohort and 0.892 (95% CI, 0.838–0.947) in the validation cohort. The radiomics nomogram, which incorporated the clinical and CT morphological characteristics with the radiomics score, yielded an AUC of 0.940 (95% CI, 0.916–0.964) in the training cohort and 0.946 (95% CI, 0.907–0.986) in the validation cohort. Furthermore, DeLong's test revealed that the AUC of the radiomics nomogram was significantly higher than that of the radiomics signature in the training cohort (0.940 vs. 0.892; P <0.001) and in the validation cohort (0.946 vs. 0.892; P<0.001). The calibration curve (Fig. 7) of the radiomics nomogram showed good agreement in the training cohort (Hosmer–Lemeshow test statistic, P = 0.2414>0.05) and in the validation cohort (Hosmer–Lemeshow test statistic, P = 0.6773>0.05). The DCA for the radiomics score and radiomics nomogram are presented in the validation cohort (Fig. 8). The DCA indicated that using the radiomics score or nomogram to predict the probability of invasiveness added more net benefit than the “treat all” or “treat none” strategies. Additionally, the nomogram added more benefit than using only the radiomics score at any given threshold probability.

Fig. 6.

Receiver-operating characteristic (ROC) curves of the radiomics score (blue lines) and nomogram (red lines) in the training cohort (a) and validation cohort (b). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 7.

Calibration curves of the radiomics nomogram in the training cohort (a) and in the validation cohort (b). The 45-degree gray lines represent perfect predictions. The blue dotted lines represent the predictive performance of the nomogram. The closer fit to the gray line represents a better prediction. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 8.

DCA for the radiomics score and radiomics nomogram. The y-axis represents the net benefit. The x-axis represents the threshold probability. The red line represents the radiomics nomogram. The blue line represents the radiomics signature. The gray line represents the hypothesis that all GGNs are invasive (all). The black line represents the assumption that all GGNs are indolent (none). Using the radiomics nomogram to predict invasiveness adds more benefit than using the radiomics score at any given threshold probability. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Discussion

We developed and validated a radiomics nomogram that incorporated the clinical and CT morphological characteristics with a radiomics signature to preoperatively evaluate the invasiveness of GGNs. The nomogram demonstrated excellent discrimination with an AUC of 0.940 in the training cohort and 0.946 in the validation cohort, which shows it may assist clinicians in accurately evaluating the invasiveness of GGNs prior to operation and taking appropriate treatment measures.

Lung cancer is common among the elderly with a smoking history. A clinical model of lung cancer prediction reported that age and smoking status were independent predictors of nodule invasiveness [25]. Kobayashi et al. [26] reported that a history of smoking was robustly associated with GGN growth. In our study, all GGNs were pathologically confirmed as early-stage lung cancer. There were statistical differences in age and smoking history when identifying the invasiveness of GGNs, which was consistent with the findings of previous studies.

The CT morphological characteristics are usually used in routine clinical practice for predicting the invasiveness of GGNs. Some previous studies [27], [28], [29] have reported that categorical variables, such as lesion size, ill-defined margin, irregular shape, and vascular changes, were predictors of the invasiveness of GGNs. However, these conclusions are not uniform. She et al. [29] reported that lesion size, margin, and shape were the predictive factors of invasiveness. However, Zhao et al. [20] showed that there were no significant differences in CT morphological characteristics between preinvasive lesions and invasive lesions. Therefore, it is not reliable to predict the invasiveness of GGNs based on CT morphological categorical variables. In our study, the multivariate logistic regression model eliminated all categorical variables based on CT morphology. This may be because the GGNs included in this study were all early-stage lung cancer. Usually, the characteristics of early-stage lung cancer are atypical; they rarely exhibit typical malignant signs of lobular, spicule, or pleural indention. Apart from that, the radiologist's experience and ability to recognize the signs also play a key role in diagnosis. Some studies used quantitative CT characteristics to analyze the invasiveness of nodules. To date, the most commonly used quantitative indicators are average CT value and diameter [30], [31], [32]. In our study, the multivariate logistic regression analysis only retained two quantitative radiological characteristics (long diameter and average CT value) as predictors of invasiveness, which was consistent with previous studies [33,34]. Average CT value reflects the heterogeneity of GGNs caused by the infiltration of invasive tumor cells [35]. The Fleischner Society has recommended lesion size for management of lung nodules [36].

Radiomics was introduced in 2012 [37], and it has been increasingly applied in clinical practice, especially for lung cancer. A review by Park et al. [38] showed that radiomics has been widely used to identify the specific driver mutational status of non-small cell lung cancer, predictors of treatment response and prognosis, and the immune phenotype of tumors. Radiomics can extract high-throughput quantitative features, which are not easily recognized visually, to reflect a tumor's heterogeneity. The heterogeneity of invasive nodules is more obvious than that of indolent lesions. According to the hypothesis, radiomics may play an important role in predicting the invasiveness of GGNs.

There have been some recent studies that applied radiomics to predict the invasiveness of GGNs. She et al. [18] developed a model to differentiate AIS/MIA from IPA using gender and a radiomics signature with an AUC of 0.96 in the training set and an AUC of 0.90 in the validation set. Xue et al. [19] constructed a nomogram that combined pure or mixed density and fractal dimension to differentiate preinvasive lesions (AAH/AIS) from invasive lesions (MIA/IPA) with an AUC of 0.76 in the training set and an AUC of 0.79 in the validation set. Zhao et al. [20] used radiomics signature and mean CT value to discriminate sub-centimeter nodules with a C-index of 0.716 in the training set and a C-index of 0.707 in the validation set. In this study, we constructed a nomogram integrating age, smoking history, long diameter and average CT value and the radiomics score to differentiate AAH/AIS/MIA from IPA with a promising performance, an AUC of 0.940 in the training cohort and 0.946 in the validation cohort.

The strengths of this study lie in the following aspects. First of all, we classified the GGNs from the perspective of prognosis, which was more instructive for clinical decision-making. Considering the prognosis and treatment strategy of MIAs is similar to those of preinvasive lesions, so we combined them as indolent lesions and differentiate them from IPA, which was different from the study from Xue et al. and Zhao et al. In indolent lesions group, 12 AAHs confirmed by pathology were included, which were excluded from She et al. study. Though this number was small in our study, it was necessary. AAH usually manifests as GGN, and some are atypical (large size or uneven attenuation), which makes it difficult to distinguish them from IPA. For this reason, we believe it is important to include AAH. Second, we evaluated the CT morphological characteristics, which were not included in She et al. study. Although all of the categorical variables were excluded in our study, this negative result provided important evidence of the inaccurate visual morphological assessment. But we reserved two quantitative morphological indicators, diameter and average CT value, which can reflect the heterogeneity of lesions. Third, we extracted a more comprehensive set of radiomics features based on the plugin of pyradiomics 2.1 [22] installed on ISD, including wavelet features, which were not included in She et al. study. Though the wavelet features are obscure and difficult to interpret, they embody the deep information in CT images that cannot be recognized by the human eye and better reflect the heterogeneity of GGNs. To solve the intractable challenge of interpretability of radiomics features, we normalized the extracted radiomics features and calculated the radiomics score, which is easier to use in clinical work. Furthermore, we compared the effects of different feature selection methods on the diagnostic performance of the model, and found no difference, which indicated that the extracted features have good stability. Fourth, our data were obtained from different CT vendors with different slice thicknesses, which were also presented in the previous studies. However, we performed a stratified analysis to analyze the influence of CT vendor and slice thickness on radiomics signature, which was not carried out in the previous studies. The result showed that the performance of the radiomics score was not affected by the slice thickness and vendors from GE, Philips and Siemen (Delong test P > 0.05), which improved the generalization and robustness of our radiomics signature. Finally, we used semiautomatic segmentation comparing with manual segmentation in the previous studies, and ICC was used to evaluate the consistency of two radiologists, which ensured the reliability and robustness of the extracted features. Based on the above aspects, we believe that our model is more stable, reliable, and robust, and has a higher diagnostic performance in the validation cohort; our model may better assist radiologists in making diagnostic decisions.

To further improve the predictive efficiency of the model, we integrated clinical and CT morphological characteristics with the radiomics score to construct a nomogram. A nomogram is a statistical model that can generate an individual numerical probability of a clinical event by integrating multiple variables; its use spurs the drive toward personalized medicine [39]. The radiomics nomogram yielded a significantly higher AUC than that of the radiomics score. It also demonstrated good calibration and added more benefit than using the radiomics score at any given threshold probability in the DCA curve. These results indicate that this radiomics nomogram may be useful for radiologists in the decision-making process. It may also facilitate individualized precision medical treatment.

This study had several limitations. First, it was retrospective; therefore, our analysis may have been influenced by selection bias. Second, the distribution of pathological subtypes of GGNs was biased, and the number of AAH was small. This was due to the fact that AAH is mostly followed-up and rarely excised by surgery. We will continue to collect data on more AAH with pathological confirmation in the study follow-up. Third, we did not conduct a multi-center study or obtain external validation; therefore, the universality of our findings may be reduced. In the future, we will conduct a multi-center prospective multicenter study to verify our model and enhance its universality.

Conclusion

Radiomics nomogram may serve as a noninvasive and accurate predictive tool to determine the invasiveness of GGNs prior to surgery and assist clinicians in creating personalized treatment strategies.

Funding

There are five funding for this article

• 1.The Science and Technology Development Plan of Jilin Province (Grant Nos. 20170622009JC).
• 2.The Provincial and School Joint Construction Project of Jilin University (Grant Nos. SXGJXX2017-8).
• 3.Foundation of Jilin Provincial Department of Finance, Establishment of standardized database for colorectal cancer and exploration of new diagnosis and treatment model based on big data analysis.
• 4.Jilin Provincial Key Laboratory of Medical imaging & big data (20200601003JC).
• 5.Radiology and Technology Innovation Center of Jilin Province (20190902016TC).

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.