Development and validation of tumor‐to‐blood based nomograms for preoperative prediction of lymph node metastasis in lung cancer

Abstract

Background

To develop and validate tumor‐to‐blood based nomograms for preoperative prediction of lymph node (LN) metastasis in patients with lung cancer (LC).

Methods

A prediction model was developed in a primary cohort comprising 330 LN stations from patients with pathologically confirmed LC, these data having been gathered from January 2016 to June 2019. Tumor‐to‐blood variables of LNs were calculated from positron emission tomography‐computed tomography (PET‐CT) images of LC and the short axis diameters of LNs were measured on CT images. Tumor‐to‐blood variables, number of stations suspected of harboring LN metastasis according to PET, and independent clinicopathological risk factors were included in the final nomograms. After being internally validated, the nomograms were used to assess an independent validation cohort containing 101 consecutive LN stations accumulated from July 2019 to March 2020.

Results

Four tumor‐to‐blood variables (left atrium, inferior vena cava, liver, and aortic arch) and the maximum standardized uptake value (SUVmax) for LNs were found to be significantly associated with LN status (p < 0.001 for both primary and validation cohorts). Five predictive nomograms were built. Of these, one with LN SUVmax/left atrium SUVmax was found to be optimal for predicting LN status with AUC 0.830 (95% confidence interval [CI]: 0.774–0.886) in the primary cohort and AUC 0.865 (95% CI: 0.782–0.948) in the validation cohort. All models showed good discrimination, with a modest C‐index, and good calibration in both primary and validation cohorts.

Conclusions

We have developed tumor‐to‐blood based nomograms that incorporate identified clinicopathological risk factors and facilitate preoperative prediction of LN metastasis in LC patients.

Tumor‐to‐blood parameters improve prediction accuracy of PET/CT for lymph node (LN) metastasis in patients with lung cancer. The LN SUVmax/left atrium SUVmax based nomogram had the best predictive performance. The AUC of the primary N = 330) and validation cohorts (N = 101) was 0.830 and 0.865, respectively.

INTRODUCTION

Lung cancer (LC) remains the most common cancer, ranking first as a cause of cancer‐related death globally. ¹ Accurate identification of lymph node (LN) involvement in lung cancer patients is crucial for successful management. ² , ³ Several predictors of LN metastasis have been identified, including LN diameter, tumor differentiation, and tumor size ⁴ , ⁵ ; however, they are not very accurate. Differentiating uninvolved LNs from those harboring metastases is critical for pretreatment decision‐making regarding the need for adjuvant therapy and the extent of resection. ⁶

18F‐fluorodeoxyglucose positron emission tomography (18F‐FDG PET) plays an important role in identifying LN status: malignant cells accumulate more FDG than normal tissue because they have a higher metabolic rate. ⁷ , ⁸ Several studies have found that the tumor‐to‐blood standardized uptake value (SUV) standard uptake ratio (SUR), ⁹ , ¹⁰ , ¹¹ a normalized value of a PET variable relative to blood pool SUV, better predicts LN status than LN maximum standardized uptake value (SUVmax). Although, various PET/CT imaging criteria for malignancy and new variables have been investigated with the aim of improving accuracy, no single predictor for assessing LN status has thus far proved to be reliable. ¹² , ¹³

Nomograms have been accepted as preferable and reliable tools for quantitative prediction of risks of malignancy and outcomes. ⁴ , ¹⁴ , ¹⁵ Combining a number of risk factors, rather than relying on any single one, has been shown to be a viable and dependable approach to making accurate predictions. A few models have been established for predicting LN metastasis in patients with non‐small cell lung cancer (NSCLC) on the basis of PET/CT‐related variables. ¹⁶ , ¹⁷ However, an optimal nomogram that combines multiple clinical factors with SURs has yet to be established. To the best of our knowledge, no published studies have yet determined whether a SUR‐based nomogram would more accurately predict LN metastasis in such patients.

Therefore, the aim of this study was to develop and validate a SUR‐based nomogram that incorporates clinicopathological risk factors for prediction of LN metastasis in patients with LC.

METHODS

This retrospective study was approved by the institutional review board of, Beijing, China, and the requirement to obtain informed consent was waived (approval number: 2021‐Ke‐21).

Patients

The primary cohort of this study comprised 185 consecutive NSCLC patients with 330 lymph node stations who had undergone PET/CT before resection at the Beijing Chaoyang hospital between January 2016 and June 2019. The model was confirmed using an independent validation cohort comprising 39 patients with 101 resected lymph node stations from July 2019 to March 2020.

The inclusion criteria were as follows: (i) 18F‐fluorodeoxyglucose positron emission tomography‐computed tomography (PET‐CT) performed within the 30 days prior to undergoing resection with curative intent; (ii) no history of other types of cancer; (iii) no neoadjuvant chemotherapy or radiation administered before PET/CT examination; and (iv) systematic lymph node dissection carried out at the second, fourth, seventh, eighth, ninth, 10th and 11th stations for right‐sided lung cancers and at the fifth, sixth, seventh, eighth, ninth, 10th and 11th stations for left‐sided lung cancers. The exclusion criteria were as follows: (i) multiple pulmonary nodules, (ii) pure ground‐glass opacity (GGO) or part‐solid ground‐glass opacity (ps‐GGO) nodule, (iii) sampling lymph node dissection performed during resection, and (iv) pathological diagnosis of a benign nodule.

Baseline clinicopathological data including age, gender, cell type, data of baseline CT, and smoking status were drawn from the participants' medical records. LN status was defined separately for each LN station. Tumors located centrally were defined as located within the proximal third of the hemithorax; all other tumors were defined as peripheral.

FDG PET/CT examination

Within 30 days before surgery, all patients underwent PET/CT using an integrated PET/CT scanner (GE Discovery STE) and a standard protocol. Baseline scans of the chest were performed using 5‐mm‐thick slices. All patients fasted for at least 6 h before the PET/CT examination. The patients' blood glucose concentrations were checked and controlled at <6.8 mmol/L before 18F‐fluorodeoxyglucose injection (3.7 to 4.5 MBq/kg bodyweight), which was administered after micturition and resting for 60 min. Images were obtained from the base of the skull to the mid‐thigh level.

Image analysis

Volume viewer software was used to generate volume of interest (VOI) of the tumors and each lymph node station using the threshold SUV. These factors were then used to calculate the SUVmax, average standardized uptake value (SUVave), and tumor volume by drawing a region of interest around it. The SUVmax was defined as the highest voxel within the VOI. SUVave and tumor volume were automatically measured on the workstation, SUV of 40% or greater of the SUVmax being adopted as the threshold.

For the blood pool SUVmax, VOIs were drawn manually on the left atrium, inferior vena cava, liver (1000 mm³) and aortic arch (Figure 1). The blood pool SUVmax and SUVave were automatically measured on the workstation. The SUR was defined as the LN‐SUVmax/blood pool. As previously stated, LN status was defined by each LN station, not each patient. The number of suspicious LN stations within each patient was also retrieved. The short axis of each lymph node was measured at the mediastinal window of chest CT images window level 40 HU and window width 350 HU. The equations for each SUR were as follows.

$$\text{SURmax}-\mathrm{Lea}=\frac{\mathit{LN}\mathit{\text{SUVmax}}}{\text{Left atrium SUVmax}}$$ (1)

$$\text{SURmax}-\mathrm{Ivc}=\frac{\mathit{LN}\mathit{\text{SUVmax}}}{\text{Inferior vena cava SUVmax}}$$ (2)

$$\text{SURmax}-\mathrm{Liv}=\frac{\mathit{LN}\mathit{\text{SUVmax}}}{\text{Liver SUVmax}}$$ (3)

$$\text{SURmax}-\mathrm{Ara}=\frac{\mathit{LN}\mathit{\text{SUVmax}}}{\text{Arcus Aortae SUVmax}}$$ (4)

FIGURE 1.

Delineation of arcus aortae, left atrium, inferior vena cava and liver. A roughly circular volume of interest (VOI) was delineated on the positron emission tomography (PET) image (a, arcus aortae, diameter = 1.5 cm; b, left atrium, diameter = 2 cm; c, inferior vena cava, diameter = 1.5 cm; d, liver, area = 1000 mm²) and the margin were excluded

Comparison of patient characteristics between LN‐positive and LN‐negative groups and between primary and validation cohorts

Associations between clinicopathological risk factors and LN status and differences in prevalence of affected LNs between the primary and validation cohorts were assessed by univariable analysis.

Feature selection and prediction model building

We used LN SUVmax and four SURs as the main variables for building five separate models (Models 1–5) for predicting lymph node metastasis. Selected clinical and PET/CT variables were introduced as covariates into each model. Univariate and multivariable logistic regression was performed for all models and an odds ratio (OR) with 95% confidence interval (CI) was used to estimate the strength of correlations. Predictors identified by univariable analysis (p < 0.05) were subjected to multivariable regression analysis.

In the primary cohort, multivariable logistic regression analysis began with LN SUVmax, SURmax‐Lea, SURmax‐Ivc, SURmax‐Liv, SURmax‐Ara and the remaining selected predictors. Two‐way stepwise selection was applied by using the likelihood ratio test with Akaike's information criterion as the stopping rule. ¹⁸ , ¹⁹ Numerical and categorical data are presented as mean ± standard deviation and frequency (%), respectively.

Apparent performance and validation of the nomogram

Calibration curves were plotted to assess calibration of the nomograms by the Hosmer–Lemeshow test. ²⁰ All nomograms were subjected to bootstrapping (1000 bootstrap resamples) to calculate the C‐index. The logistic regression formula created in the primary cohort was applied to all patients in the validation cohort; total points for each patient were calculated from factors used in logistic regression in the primary cohort. The C‐index and calibration curve were established.

Decision curve analysis and selection of the optimal nomogram

Decision curves were analyzed to determine the clinical usefulness of all nomograms by quantifying the net benefits at different threshold probabilities in the validation dataset. ²¹ Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated to compare the predictive performance of all models. ²² , ²³ The model with the most modest NRI and IDI was selected as the final model for predicting LN metastasis.

Statistical analysis

For the clinical and pathological factors, numerical data were analyzed using the Mann–Whitney U test and categorical data by Pearson's χ² test. Statistical analysis was conducted with R software (version 4.0.3; http://www.Rproject.org). The reported probabilities are all two‐sided with statistical significance set at 0.05.

RESULTS

On the basis of the selection criteria, a total of 431 LNs in 224 patients were included (Figure 2).

FIGURE 2.

Patient recruitment flowchart

Patient characteristics

Patient characteristics in the primary and validation cohorts are shown in Table 1. The study cohort comprised 185 patients, of whom 127 were LN‐negative and 58 LN‐positive. Overall, the 431 LN stations consisted of 69 N1 and 362 N2 stations. There were no significant differences between the two cohorts in LN prevalence (22.7% vs. 30.7%, p = 0.104).

TABLE 1.

Characteristics of patients in the primary and validation cohorts

Variable	Primary cohort (n = 330)			Validation cohort (n = 101)
	LN− (n = 255)	LN+ (n = 75)	p‐value	LN− (n = 70)	LN+ (n = 31)	p‐value
Gender			0.573			0.668
Male	182 (71.4%)	51 (68.0%)		46 (65.7%)	19 (61.3%)
Female	73 (28.6%)	24 (32.0%)		24 (34.3%)	12 (38.7%)
Age			0.225			0.013
	62.5 (10.8)	61.3 (9.5)		62.7 (8.8)	58.5 (9.0)
Tumor SUVmax			<0.001			0.003
	7.1 (5.1)	11.6 (8.1)		7.8 (6.2)	11.0 (4.1)
Tumor volume			0.004			0.001
	11.2 (11.4)	18.2 (19.4)		16.2 (19.8)	37.6 (33.4)
Tumor location			<0.001			0.006
Peripheral	153 (60.0%)	25 (33.3%)		41 (58.6%)	9 (29.0%)
Central	102 (40.0%)	50 (66.7%)		29 (41.4%)	22 (71.0%)
LN SUVmax			<0.001			<0.001
	2.2 (1.5)	4.8 (3.7)		2.1 (1.7)	4.9 (4.4)
LN SUVave			<0.001			<0.001
	1.7 (0.8)	3.2 (2.2)		1.8 (1.1)	3.2 (2.7)
LN short axis (mm)			<0.001			0.004
	5.8 (1.8)	7.7 (2.9)		5.1 (1.6)	7.1 (3.4)
SURmax‐Lea			<0.001			<0.001
	1.3 (1.0)	3.3 (2.7)		1.2 (1.1)	3.2 (2.9)
SURmax‐Ivc			<0.001			<0.001
	1.4 (1.2)	3.3 (2.7)		1.5 (1.6)	3.8 (3.7)
SURmax‐Liv			<0.001			<0.001
	0.8 (0.6)	1.8 (1.4)		0.7 (0.6)	1.8 (1.8)
SURmax‐Ara			<0.001			<0.001
	1.4 (1.1)	3.2 (2.6)		1.4 (1.4)	3.3 (2.9)
Cell type			0.230			0.297
Adenocarcinoma	153 (60.0%)	47 (62.7%)		35 (50.0%)	13 (41.9%)
Squamous cell carcinoma	72 (28.2%)	15 (20.0%)		29 (41.4%)	12 (38.7%)
Other	30 (11.8%)	13 (17.3%)		6 (8.6%)	6 (19.4%)
Smoking status			0.686			0.489
Current	83 (32.5%)	27 (36.0%)		17 (24.3%)	5 (16.1%)
Former	59 (23.1%)	19 (25.3%)		12 (17.1%)	4 (12.9%)
Never	113 (44.3%)	29 (38.7%)		41 (58.6%)	22 (71.0%)
Suspicious LN metastasis			0.013			0.017
<2 stations	92 (36.1%)	39 (52.0%)		23 (32.9%)	18 (58.1%)
>=2 stations	163 (63.9%)	36 (48.0%)		47 (67.1%)	13 (41.9%)

Note: The p‐value is derived from the univariable association analyses between each of the clinicopathological variables and LN status.

Abbreviations: LN, lymph node; LN−, lymph node negative; LN+, lymph node metastasis; mm, millimeter; SUVave, average standardized uptake value; SUVmax, maximum standardized uptake value.

Nomogram model building and selection of the optimal nomogram

Univariate logistic regression identified LN SUVmax, SURmax‐Lea, SURmax‐Ivc, SURmax‐Liv, and SURmax‐Ara as independent predictors in the primary cohort. Five models (Models 1–5) were constructed using covariates that were identified as independent predictors by multivariable logistic regression analysis.

Model 2 (SURmax‐Lea, LN short axis, tumor volume, suspicious LN stations, tumor location, sex) was found to have a higher C‐index in both the primary and validation cohorts than the other models (Table 2). The predictive performance of Model 2 was significantly superior to that of all four other models according to the NRI and IDI (Supplementary Table S1). Therefore Model 2, which incorporates SURmax‐Lea, a LN metabolism‐related variable, and selected clinical variables, was selected as the optimal model for predicting LN metastases (Figure 3). The results of multivariable logistic regression analysis, C‐index and nomograms of Models 1, 3, 4, and 5 are presented as supplementary data (Supplementary Table S1).

TABLE 2.

Risk factors of all models for lymph node metastasis in lung cancer

Variable	Model 2
	ß	Odds ratio (95% CI)	p‐value
SURmax‐Lea	1.875	1.438–2.445	<0.0001
LN short axis	1.214	1.042–1.415	0.013
Tumor volume	1.023	1.001–1.045	0.042
Tumor location	2.130	1.127–4.023	0.011
Suspicious LN metastasis	2.265	1.204–4.259	0.015
Gender	1.955	1.00–3.822	0.048
C‐index
Primary cohort	0.830	0.774–0.886	0.001
Validation cohort	0.875	0.782–0.948	0.001

Note: ß is the regression coefficient. p‐value, odds ratio and confidence interval were derived from the multivariable regression. In net reclassification index and integrated discrimination improvement analysis, model 1 was regard as reference and compared to other four models.

Abbreviations: CI, confidence interval; LN, lymph node; SUVmax, maximum standardized uptake value.

FIGURE 3.

Developed nomogram of SURmax‐Lea and clinic predictors. The SURmax‐Lea, LN short axis, tumor volume, tumor location, suspicious LN metastasis and gender were incorporated in this nomogram

Performance of all nomograms in primary and validation cohorts

The calibration curve in Model 2 for the probability of LN metastasis demonstrated good agreement between prediction and observation in the primary cohort (Figure 4). The Hosmer–Lemeshow test on all nomograms yielded nonsignificant findings, indicating that there was no departure from perfect fit. The calibration curves for Models 1, 3, 4, and 5 are presented as supplementary data (Supplementary Figure S1).

FIGURE 4.

Calibration curves of the SURmax‐Lea nomograms in the primary cohort. The predictors include SURmax‐lea, LN short axis, tumor volume, tumor location, suspicious LN metastasis and gender. The x‐axis represents the predicted LN metastasis risk. The diagonal dotted line represents a perfect prediction by an ideal model. The solid line represents the performance of the nomogram, of which a closer fit to the diagonal dotted line represents a better prediction

Clinical use

The decision curve for Model 2 is presented in Figure 5. It shows modest prediction of LN metastasis and that using the nomogram to predict LN metastases is more accurate than either the treat‐all‐patients or the treat‐none scheme; however, there were several overlaps between the models. The decision curves for the other four models are presented as supplementary data (Figure S2).

FIGURE 5.

Decision curve analysis for SURmax‐Lea nomograms. The y‐axis measures the net benefit. The red line represents the nomogram. The gray line represents the assumption that all patients have LN metastases. Thin black solid line represents the assumption that no patients have LN metastases

DISCUSSION

We developed and validated five different LN SUVmax‐based nomograms for preoperative prediction of LN metastasis in patients with lung cancer. All nomograms successfully stratified patients according to their risk of LN metastases, SURmax‐Lea being the most accurate of the models. Incorporating variables related to LN metabolism and clinical risk factors into a usable nomogram facilitated preoperative prediction of LN metastasis.

Unlike previous studies ⁴ , ⁵ , ¹⁰ , ²⁴ that reported means of predicting LN metastases in a given individual, we used LN SUVmax of each station to predict metastases in the corresponding LN station when constructing all of our models. Using only the LN station with the highest SUVmax is arbitrary and potentially inaccurate because a single individual may have multiple LNs with abnormal SUVmax. Furthermore, a high LN SUVmax is not diagnostic of the presence of metastasis and vice versa. In the present study, 11 patients (18.9%) with low LN SUVmax had LN metastases, whereas some cases with higher SUVmax did not. Use of only the highest LN SUVmax for prediction would have yielded a greater proportion of false negatives, blunting the accuracy of prediction.

The tumor‐to‐background ratio has been widely used to differentiate between benign and malignant status, especially regarding tumors with low metabolic rates, because it balances each individual's background by incorporating a blood pool value. ⁵ , ¹² , ²⁵ Most tumor‐to‐background ratio studies, including those on colorectal, rectal, and esophageal cancer, have demonstrated a modest ability to differentiate cancer. ⁴ , ¹³ , ²⁶ However, no previous studies have compared different blood pools and identified which is optimal for prediction.

We used the left atrium, inferior vena cava, liver, and aortic arch as blood pools when constructing our models for determining LN metastatic status. The aortic arch and liver have been selected as background blood pools in many previous studies on prediction of LN status, ¹² , ¹³ , ²⁶ , ²⁷ whereas, to the best of our knowledge, no previous studies have used the left atrium or inferior vena cava for this purpose. According to univariable analysis, predictions by LN SUVmax and four SURs were of comparable accuracy; however, the differences between them widened when we integrated all clinical factors into the models. Although there were no differences between the four blood pool baselines, SUR‐Lea combined with multiple clinical factors was superior to all of the other models in classifying LN metastasis status. The reclassification performance was better than that of the other prediction models, with significant NRI and IDI. This superior accuracy may be attributable to the addition of one more predictor, sex, to the SUR‐Lea based model. Among all five models, the ROC showed significant superiority over LN SUVmax for prediction only (NRI, 0.236, 0.256, 0.048, 0.089 and 0.132, p < 0.01).

Some published studies have shown that carcinoembryonic antigen (CEA) can serve as an important predictor of LN metastases. ¹⁷ However, we did not include CEA concentration in our models because we were determining LN status by station, not individually. If we had incorporated it, the false positive rate would have increased for LN stations without metastases and with low SUVmax in patients with high CEA concentrations that were attributable to LN metastases elsewhere. Therefore, adding CEA concentrations to our models would inevitably have augmented bias and reduced the diagnostic accuracy.

Whether the number of LN stations suspected on the basis of radiological findings of harboring metastases is an independent risk factor for LN metastasis is of interest. No previous studies have used this as a predictor. Unsurprisingly, on the one hand, in our study, univariable analysis identified such suspected metastases as a significant predictor (p < 0.001). On the other hand, incorporation of radiologically‐suspected LN metastasis did not improve the reclassification performance in Models 3 and 5. Its inclusion may have introduced multicollinearity between LN SUVmax‐related variables and LN stations suspected of harboring metastases.

In this study, the N1 station was not separated from the N2 station in primary and the validation cohort because of the limited numbers (69 N1 LN stations). It was difficult to distinguish metastatic hilar and interlobar lymph nodes from those which were negative because of the soft‐tissue character and proximity to the bronchus. ²⁸ , ²⁹ The diagnostic accuracy of PET/CT for detecting N1 LN metastasis is fair but has a high false negative rate. ³⁰ In order to increase the diagnostic accuracy of metastatic N1 LNs, endobronchial ultrasound (EBUS) or endoscopic ultrasound fine needle aspiration (EUS‐FNS) combined PET/CT are recommended for the detection of metastatic N1 LNs in CT or PET‐CT negative LNs. ³¹

Limitations of our study include its small size. Only 185 patients with 431 LN stations were included; this may have introduced bias. Furthermore, the SUVmax of N1 station was not routinely measured, being determined only when there was evidence of hypermetabolism, resulting in an imbalance between the number of N1 and N2 stations. In addition, we were unable to include genomic characteristics because this was a retrospective study. In future, identification of reliable gene markers may contribute to building a more accurate prediction model.

In conclusion, this study presents five models that incorporate both metabolism signatures and clinical risk factors. These may facilitate the individualized prediction of LN metastasis in patients with lung cancer.

CONFLICT OF INTEREST

All authors declare that there are no potential conflicts of interest.

Supporting information

Appendix S1. Supporting Information

Fu Y, Xi X, Tang Y, et al. Development and validation of tumor‐to‐blood based nomograms for preoperative prediction of lymph node metastasis in lung cancer. Thorac Cancer. 2021;12:2189–2197. 10.1111/1759-7714.14066