Abstract
Purpose: Previous studies addressing the optimal nodal staging system in patients with resected gastric cancer have shown inconsistent results, and the optimal system for development of prognostic nomograms remains unclear. In this study, we compared prognostic nomograms based on the metastatic lymph node (MLN) count, lymph node ratio (LNR), and log odds of metastatic lymph nodes (LODDS) to predict the 5-year overall survival in patients with resected gastric cancer.
Methods: We analysed 15,320 patients with resected gastric cancer in the Surveillance, Epidemiology, and End Results (SEER) database between 1988 and 2010. Missing data were handled using multiple imputation. When assessed as a continuous covariate with restricted cubic splines, each MLN, LNR, and LODDS variable was incorporated into a nomogram with other significant prognosticators to predict the 5-year overall survival. A two-centre Chinese dataset (1,595 cases) was used as external validation data.
Results: The discriminatory abilities of the MLN-, LNR-, and LODDS-based nomograms were comparable (concordance indices: 0.744, 0.741, and 0.744, respectively, in the SEER set, P > 0.152 for all pairwise comparisons; 0.715, 0.712, and 0.713, respectively, in the Chinese set, P > 0.445 for all pairwise comparisons). The discriminatory abilities of the three nomograms were all superior to the American Joint Committee on Cancer (AJCC) TNM classification (concordance indices: 0.713, P < 0.001 for all in the SEER set; and 0.693, P < 0.001 for all in the Chinese set). The discriminatory abilities of the nomograms were comparable regardless of the number of nodes examined. Moreover, decision curve analyses indicated similar net benefits of using the nomograms.
Conclusion: MLN-, LNR-, and LODDS should be considered equally in the development of multivariate prognostic models and nomograms to refine the prediction of survival among patients with resected gastric cancer.
Keywords: gastric cancer, nomogram, metastatic lymph node (MLN) count, lymph node ratio (LNR), log odds of metastatic lymph nodes (LODDS).
Introduction
Gastric cancer (GC) was the fifth most common malignancy and the third leading cause of cancer-related deaths worldwide, with an especially high incidence in East Asia 1-3. The accuracy of survival prediction for patients with GC is pivotal for postoperative treatment decisions and surveillance. Conventionally, the outcomes of radically resected GC are predicted based on the pathological tumour invasion depth and lymph node status, which are addressed in the American Joint Committee on Cancer (AJCC) TNM classification 4. However, although a minimum of 16 total harvested nodes (THNs) are required by the 7th AJCC TNM classification for accurate nodal staging,3 this system is merely based on the number of metastatic lymph nodes (MLNs) and has been criticized for disregarding the influence of THNs on survival 5, 6. To refine the prediction of prognosis in patients with GC, two ratio-based nodal staging systems have been proposed: the lymph node ratio (LNR) 7, defined as the ratio between MLNs and THNs, and the log odds of metastatic lymph nodes (LODDS) 8, defined as the log of the ratio between the number of MLNs and number of negative lymph nodes (NLNs). However, because of the variation and correctness of the analysis methodology, studies comparing the discriminatory capacities of MLN-, LNR- and LODDS-based staging systems have shown inconsistent results 8-14, and a consensus regarding the optimal system has not yet been reached.
Prognostic nomograms are useful tools that allow for intuitive individual risk evaluation by combining prognostic factors 15. Several nomograms have been developed that incorporate the number of MLNs along with the number of THNs or NLNs and other prognostic factors (e.g., age and depth of invasion) to refine the prediction of outcomes in patients with resected GC 15-20. However, it remains unknown whether using the LNRs or LODDS instead of the MLNs in model development can improve the prognostic performances of these nomograms.
Thus, in the present study, we developed prognostic nomograms incorporating the MLNs, LNR, or LODDS with other significant prognosticators using a population-based cohort from the Surveillance, Epidemiology, and End Results (SEER) database. Thereafter, we performed comparisons of these nomograms among the SEER dataset and an independent Chinese dataset, which allowed for multivariable-adjusted comparisons among the nodal staging systems.
Materials and Methods
Data sets
Using the SEER database, we identified 21,117 patients with GC diagnosed from January 1, 1988 to December 31, 2010. We excluded patients diagnosed after 2010 to allow for a longer median follow-up period. The inclusion criteria were as follows: (1) a single primary gastric carcinoma (ICD-O-3 8010-8231 and 8255-8576) confirmed microscopically; (2) a gastrectomy with at least one node harvested; (3) no radiotherapy at any time; and (4) active follow-up. The following cases were excluded from the analysis: cases with a length of follow-up less than 3 months (3,031 cases), cases of carcinoma in situ (302 cases), and cases of synchronous distant metastasis or unknown status of metastasis (2,204 cases). Stage information was determined according to the 7th AJCC TNM classification. Patients with insufficient information for conversion to the 7th AJCC TNM classification (260 cases) were excluded. The final analytic SEER cohort for the development of nomograms consisted of 15,320 patients.
An independent Chinese cohort consisting of 1,595 patients who had been diagnosed with GC and had undergone curative gastrectomy between 2001 and 2010 at the Sun Yat-Sen University Cancer Centre or Nanfang Hospital was used as a validation data set. The Chinese validation set was collected according to the aforementioned inclusion and exclusion criteria.
Ethical issues
No institutional review was sought because SEER is public-use data; informed consent was waived. The study protocol for the Chinese dataset was approved by the Ethics Committees of the Sun Yat-Sen University Cancer Centre and the Nanfang Hospital.
Statistical analysis
MissForest is a random forest-based method that can efficiently address missing data imputation among multivariate data by producing a single imputed dataset without setting aside test data or performing cross validations 21. For inferring missing data of variables in the SEER set (race: 43 cases, 0.3%; tumour location: 5,237 cases, 34.2%; size: 2,578 cases, 16.8%; and differentiation: 858 cases, 5.6%) and the Chinese dataset (tumour location: 87 cases, 5.5%; size: 35 cases, 2.2%; and differentiation: 178 cases, 11.2%), multiple imputation using missForest19 was performed with the following variables: year of diagnosis (or year of surgery for the Chinese set), patient age, gender, T stage, MLNs, and NLNs.
Overall survival (OS) was the primary outcome of interest. Multivariate Cox regression analyses were used to assess the relationships between clinicopathological factors and OS. To avoid multicollinearity, the MLN, LNR, and LODDS variables were included in three different Cox models 22. Restricted cubic splines were used to examine the functional forms of continuous variables in relation to survival 23. When the relationship was apparently non-linear, transformation using multivariable fractional polynomials was performed 24, and the optimal transformation was obtained based on the Bayesian Information Criteria 25. These methods reduce the bias that can result from empirically selecting cut-off points for continuous variables. For model construction, we began with the following variables: patient age, year of diagnosis, race, tumour location, size, differentiation, T stage, one of the nodal variables (MLNs, LNR, and LODDS), and NLNs (included in the initial MLN-based model only). The final Cox models were obtained using backward stepwise selection (P < 0.050).
Nomograms were generated from the final models based on the MLN, LNR and LODDS variables. Concordance indices (C-indices) were used to compare the capacities of the three nomograms to distinguish patients at low and high risk of death, that is, the discriminative powers of the three nomograms 26. The higher the C-index, the greater the discriminatory capacity of the scheme 26. Additionally, we used calibration to measure how far the nomogram-predictions deviated from the actual outcomes 27. For each nomogram, calibration was performed by reviewing the plot of nomogram-predicted survival probabilities and the Kaplan-Meier-estimated probabilities. A perfect calibration was indicated by the 45-degree reference line 27. Bootstraps with 1,000 resamples were used to quantify the model overfit and calculate Kaplan-Meier estimates. External validation of the nomograms was performed by discrimination and calibration using the Chinese dataset.
In addition to comparisons of the predictive accuracies of the nomograms, we also relied on decision curve analyses, as describe by Vickers et al. 28, to assess the potential of the nomograms for clinical application. Decision curve analyses examine the clinical practical value of a predictive model by quantifying its net benefit according to the threshold probability and the relative weight between false-positive and false-negative results.
For sensitivity analyses, we also constructed Cox regression models based on the MLN, LNR or LODDS variables and repeated the analyses among the imputed Chinese dataset or among patient subsets with complete data for all variables in the SEER or Chinese datasets.
Statistical significance was set as P < 0.050 in a two-tailed test. Statistical analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC, USA), SPSS v.19.0 (SPSS, Chicago, IL, USA), and R v.3.2.3 (http://www.r-project.org).
Results
The clinicopathological characteristics for the SEER set (15,320 cases) and Chinese validation set (1,595 cases) are listed in Table 1 and Table S1, respectively. Patient characteristics for the multiple imputed datasets are listed in Table S2. The median numbers of MLNs and THNs were 1 (interquartile range [IQR]: 0-5) and 11 (IQR: 6-19), respectively, in the SEER dataset, and 2 (IQR: 0-6) and 18 (IQR: 11-26), respectively, in the Chinese dataset. The 5-year Kaplan-Meier OS estimates were 37.6% (95% confidence interval: 37.2%-38.0%) and 61.2% (95% confidence interval: 59.9%-62.5%) in the SEER and Chinese sets, respectively.
Table 1.
Variable | Median (IQR)/N (%) |
---|---|
Year of diagnosis | |
1988-1991 | 1528 (10.0%) |
1992-1995 | 2271 (14.8%) |
1996-1999 | 2324 (15.2%) |
2000-2003 | 3429 (22.4%) |
2004-2007 | 3343 (21.8%) |
2008-2010 | 2425 (15.8%) |
Age, years | 70 (60-78) |
Race | |
White | 9874 (64.5%) |
Black | 1882 (12.3%) |
Other | 3521 (23.0%) |
Unknown | 43 (0.3%) |
Gender | |
Female | 6147 (40.1%) |
Male | 9173 (59.9%) |
Tumour location | |
Cardia | 3455 (22.6%) |
Upper one-third | 469 (3.1%) |
Middle one-third | 1257 (8.2%) |
Lower one-third | 4902 (32.0%) |
Not specified | 5237 (34.2%) |
Tumour size, cm | 4.2 (2.5-6.5) |
Unknown | 2578 (16.8%) |
Tumour differentiation | |
Poorly or undifferentiated | 9940 (64.9%) |
Well or moderately differentiated | 5036 (32.9%) |
Unknown | 858 (5.6%) |
T stage | |
T1 | 3718 (24.3%) |
T2 | 1943 (12.7%) |
T3 | 3856 (25.2%) |
T4a | 4234 (27.6%) |
T4b | 1569 (10.2%) |
MLNs | 1 (0-5) |
THNs | 11 (6-19) |
NLNs | 8 (3-14) |
LNR | 0.08 (0.00-0.50) |
LODDS | -1.61 (-2.94-0.00) |
Adequate nodal evaluation | |
No (≤ 15) | 9,641 (62.9%) |
Yes (> 15) | 5,679 (37.1%) |
SEER, Surveillance, Epidemiology, and End Results database; IQR, interquartile range; MLNs, metastatic lymph nodes; THNs, total harvested lymph nodes; NLNs, negative lymph nodes; LNR, lymph node ratio; LODDS, log ratio of metastatic lymph nodes.
Table 2 shows the MLN-, LNR-, and LODDS-based final Cox models obtained by stepwise backward selection (P < 0.050). The hazard ratios were significantly higher for white and black race, older age, male gender, advanced T stage, increased MLNs, LNR, and LODDS, and decreased NLNs. However, the year of diagnosis was not significantly associated with OS. In the MLN-based model, there was no significant interaction between MLNs and NLNs (Pinteraction = 0.104). Prognostic nomograms were developed from the three final models (Figures 1-3).
Table 2.
Variable | MLN-based model | LNR-based model | LODDS-based model | |||||
---|---|---|---|---|---|---|---|---|
Hazard ratio | P value | Hazard ratio | P value | Hazard ratio | P value | |||
MLNa | 1.333 (1.309-1.356) | < 0.001 | ||||||
LNRb | 6.528 (6.017-7.082) | < 0.001 | ||||||
LODDS | 1.325 (1.300-1.330) | < 0.001 | ||||||
NLNc | 0.734 (0.719-0.749) | < 0.001 | ||||||
Race | < 0.001 | < 0.001 | < 0.001 | |||||
White | 1 | 1 | 1 | |||||
Black | 1.151 (1.083-1.224) | 1.143 (1.075-1.215) | 1.151 (1.085-1.225) | |||||
Other | 0.874 (0.831-0.919) | 0.859 (0.817-0.903) | 0.877 (0.834-0.922) | |||||
Age | 1.020 (1.018-1.022) | < 0.001 | 1.020 (1.018-1.022) | < 0.001 | ||||
Aged | 4.776 (4.239-5.380) | < 0.001 | ||||||
Gender | < 0.001 | |||||||
Female | 1 | 1 | 1 | |||||
Male | 1.083 (1.039-1.128) | 1.09 (1.048-1.137) | 1.082 (1.039-1.128) | |||||
Tumour Location | < 0.001 | < 0.001 | < 0.001 | |||||
Cardia | 1 | 1 | 1 | |||||
Upper one-third | 0.898 (0.817-0.987) | 0.902 (0.821-0.992) | 0.893 (0.813-0.982) | |||||
Middle one-third | 0.787 (0.735-0.842) | 0.787 (0.736-0.843) | 0.783 (0.732-0837) | |||||
Lower one-third | 0.838 (0.798-0.879) | 0.845 (0.805-0.887) | 0.836 (0.797-0.877) | |||||
Tumour sizee | 1.128 (1.087-1.171) | < 0.001 | 1.118 (1.078-1.159) | < 0.001 | 1.131 (1.091-1.174) | < 0.001 | ||
Tumour differentiation | < 0.001 | < 0.001 | < 0.001 | |||||
Poorly or undifferentiated | 1 | 1 | 1 | |||||
Well or moderately differentiated | 0.898 (0.861-0.937) | 0.900 (0.863-0.939) | 0.897 (0.860-0.935) | |||||
T stage | < 0.001 | < 0.001 | < 0.001 | |||||
T1 | 1 | 1 | 1 | |||||
T2 | 1.403 (1.300-1.515) | 1.367 (1.266-1.476) | 1.407 (1.303-1.519) | |||||
T3 | 1.835 (1.710-1.968) | 1.773 (1.654-1.901) | 1.849 (1.725-1.981) | |||||
T4a | 2.249 (2.093-2.416) | 2.220 (2.068-2.384) | 2.268 (2.113-2.435) | |||||
T4b | 3.174 (2.916-3.454) | 3.148 (2.893-3.427) | 3.194 (2.936-3.475) |
MLN, metastatic lymph node; NLN, negative lymph node; LNR, lymph node ratio; LODDS, log ratio of metastatic lymph node.
a. In the MLN-based model, MLN had a non-linear effect on the log hazard of survival. Therefore, loge (MLN+0.5) was included in the model to fit a linear form.
b. In the LNR-based model, LNR had a non-linear effect on the log hazard of survival. Therefore, was included in the model to fit a linear form.
In the MLN-based model, NLN had a non-linear effect on the log hazard of survival. Therefore, loge [(NLN+1)/10] was included in the model to fit a linear form.
c. In the LNR-based model, age had a non-linear effect on the log hazard of survival. Therefore, (Age/100)2 was included in the model to fit a linear form.
d. In the MLN-, LNR-, and LODDS-based models, tumour size had a non-linear effect on the log hazard of survival. Therefore, loge (size/10) was included in the models to fit a linear form.
The unadjusted C-indices for OS prediction for the MLN-, LNR-, and LODDS-based nomograms were 0.744, 0.741, and 0.744, and the bootstrap-corrected C-indices were also 0.744, 0.741, and 0.744, respectively, indicating minimal evidence of overfit for all three models. The C-indices among the three nomograms were comparable and were all equal to two decimal places (P > 0.152 for all pairwise comparisons). All three nomograms showed better discriminatory abilities than the 7th AJCC TNM classification (C-index: 0.713, P < 0.001 for all). When the patients were stratified into subgroups according to THNs (1-10, 11-15 and > 15 THNs), the discriminatory capacities of the three nomograms remained similar among all three subgroups (Table S3). Furthermore, calibration plots showed close agreement between the actual 5-year OS probabilities in the patients of the SEER set and the OS predicted from the three nomograms (Figure S1). Similar calibration results were recorded from the examination of the mean deviations between the actual and predicted OS probabilities, and the mean deviations between the actual survival probabilities and those predicted with the MLN-, LNR-, and LODDS-based nomograms were 0.016, 0.014, and 0.014, respectively (Figures S1A-C).
The C-indices for OS prediction for the MLN-, LNR- and LODDS-based nomograms were comparable for the Chinese dataset (0.715, 0.712, and 0.713, respectively, P > 0.445 for all pairwise comparisons). All three nomograms showed better discriminatory abilities than the 7th AJCC TNM classification (C-index: 0.693, P < 0.001 for all). The calibration was consistent between the actual 5-year OS probabilities and the nomogram-predicted OS among the Chinese set (Figure S2). The mean deviations between the actual and predicted OS probabilities from the three nomograms were also comparable (0.022, 0.017, and 0.023 for the MLN-, LNT-, and LODDS-based nomograms, respectively; Figures S2A-C).
The decision curve analysis indicated that for most of the threshold probabilities for 5-year OS in both the SEER and Chinese sets, the three nomograms yielded a similar net benefit, and all of them achieved a greater net benefit compared with the 7th AJCC TNM classification (Figs. 4A-4B).
As determined through sensitivity analyses performed among the imputed Chinese dataset, patients with complete data for all variables in the SEER set, and patients with complete data for all variables in the Chinese set and using the C-index, calibration, and decision curve analysis as metrics, the MLN-, LNR-, and LODDS-based models exhibited comparable performances (data not shown).
Discussion
An accurate prediction of the prognosis of patients with GC is essential for treatment and follow-up planning. Traditionally, the outcomes of resected GC are predicted based on the AJCC TNM classification, which involves tumour invasion depth and nodal status. However, this system does not account for the number of THNs, which reportedly may influence patient survival 29, 30. Two ratio-based nodal staging systems, namely LNR and LODDS, were proposed to refine survival prediction in patients with resected GC. However, studies comparing these nodal staging systems have proven to be contradictory; whereas some studies have found that the LODDS-based system exhibited the best prognostic performance, 8-12 others did not find a predictive superiority of the LODDS system over the MLN or LNR systems 13, 14, 31.
The inconsistencies among these studies are most likely due to differences in the analysis methodology. Because all three nodal staging systems are closely correlated, it is methodologically inappropriate to use stepwise variable selection to determine the best-performing staging system 32, as was performed in the studies by Sun et al. 8 and Liu et al. 14. In addition, it is necessary to compare the prognostic performances of the nodal staging systems based on multivariate models to account for available prognostic information and reduce potential confounding. However, some of the studies that addressed this issue did not report the specific variables that were included in the multivariate models along with the nodal variables 9, 13. In addition, in a SEER-based study which found a significant predictive superiority of the LODDS-based model compared with the MLN-based model in patients with less than 10 THNs, neither the THNs nor the NLNs were included in the MLN-based model,12 which may have weakened the prognostic performance of this model. In the present study, the number of NLNs was actually identified as an independent prognosticator in the MLN-based model. When the NLNs was removed from the model, the C-index decreased significantly from 0.744 to 0.730 (P< 0.001).
No consensus exists regarding the ideal cut-off values for the three nodal staging systems, and the discrimination of all three systems reportedly improves when they are treated as continuous covariates 9. In this study, multivariate regression models were fitted when the MLN, LNR, and LODDS variables were assessed as continuous covariates to avoid arbitrarily selecting cut-off points for continuous variables, and prognostic nomograms were developed from these models to provide intuitive illustrations. The discriminatory powers of the MLN-, LNR- and LODDS-based nomograms were comparable among the SEER and Chinese datasets. Additionally, the discriminatory abilities of the nomograms were comparable regardless of the number of THNs.
Metrics concerning prognostic accuracy cannot indicate whether a prognostic model is worth using in clinical practice. For example, a scheme with high specificity but low sensitivity may have the same C-index as one with low specificity but high sensitivity. However, in the case of a disease (e.g., death in patients with GC) for which a false-negative result is much more harmful than a false-positive result; the former scheme would be a poorer choice for clinical use. Decision curve analysis addresses this issue, because it can evaluate the clinical practical value of a staging scheme by quantifying its net benefit according to the threshold probability and the relative weight between false-positive and false-negative results 28. For both the SEER and Chinese sets, the three nomograms yielded a similar net benefit over virtually the entire range of the threshold probabilities for the 5-year OS predictions. Therefore, the nomograms showed similar potential for clinical application, suggesting that they were equivalently useful in decision-making regarding postoperative treatment and surveillance.
Several good nomograms have been developed to refine survival prediction among patients with resected GC 15-20. Among these nomograms, only one included LNR rather than MLN along with other prognostic factors to refine the prediction of outcomes,17 and scarce data have addressed whether LNR and LODDS is better than MLN in development of nomograms. Additionally, it is necessary to confirm the generalizability of a nomogram by externally validation using an independent population. In this study, despite the well-recognized differences in tumor epidemiology (e.g., the percentage of proximal disease), treatment patterns, and treatment outcomes between Eastern and Western countries, 33 the proposed nomograms based on MLN-, LNR-, and LODDS exhibited favorable performance in both the SEER and Chinese datasets. Moreover, by using the Chinese set, we also verified the findings from the SEER set that MLN, LNR, and LODDS exhibited equivalent prognostic value after adjustment for other prognostic factors. Therefore, MLN, LNR, LODDS should be equally considered when developing prognostic nomograms.
The use of the SEER data enabled us to draw sound conclusions pertinent to the general clinical practice based on a large sample of patients with GC, which is not possible in single-institution studies. However, the present study has some limitations that should be noted. First, even though the SEER database is checked regularly for discrepancy and reportedly has 95% accuracy, the possibility of coding errors remains. Additionally, the SEER database does not provide data on other possible prognosticators such as lymphovascular invasion, margin status, and patient comorbidities, which may influence the results. Moreover, due to the retrospective nature of the SEER and Chinese datasets, further verification of our findings using prospective data is needed.
In summary, this study demonstrated the comparable predictive abilities and potential for clinical application of the prognostic nomograms based on MLN-, LNR-, and LODDS. Therefore, MLN-, LNR-, and LODDS are interchangeable in the development of multivariate prognostic models and nomograms to refine the prediction of survival among patients with resected GC.
Supplementary Material
Acknowledgments
We thank the staff members of the National Cancer Institute and their colleagues across the United States and at Information Management Services, Inc., who have been involved with the Surveillance, Epidemiology, and End Results (SEER) Program.
Funding
This work was supported by:
1. National High Technology Research and Development Program of China (863 Program), China (No. 2015AA020103).
2. National Natural Science Foundation of China (No.81372570, 81572392).
3. Science and Technology Program of Guangdong Province, China (No. 201508020247).
4. Medical Scientific Research of Guangdong province (No. B201416).
5. China Scholarship Council (No. 201406385034).
6. Science and Technology Planning Project of Guangdong Province (No. 2013A022100023).
Abbreviation
- GC
Gastric cancer
- AJCC
American Joint Committee on Cancer
- THNs
total harvested nodes
- MLNs
metastatic lymph nodes
- LNR
lymph node ratio
- LODDS
log odds of metastatic lymph nodes
- NLNs
negative lymph nodes
- SEER
Surveillance, Epidemiology, and End Results
- OS
Overall survival
- IQR
interquartile range
References
- 1.Torre LA, Bray F, Siegel RL, Ferlay J, Lortet-Tieulent J, Jemal A. Global cancer statistics, 2012. CA Cancer J Clin. 2015;65:87–108. doi: 10.3322/caac.21262. [DOI] [PubMed] [Google Scholar]
- 2.Chen W, Zheng R, Zeng H, Zhang S. The updated incidences and mortalities of major cancers in China, 2011. Chin J Cancer. 2015;34:53. doi: 10.1186/s40880-015-0042-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Chen W, Zheng R, Baade PD. et al. Cancer statistics in China, 2015. CA Cancer J Clin. 2016;66:115–32. doi: 10.3322/caac.21338. [DOI] [PubMed] [Google Scholar]
- 4.Edge SB, Compton CC. The American Joint Committee on Cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM. Ann Surg Oncol. 2010;17:1471–4. doi: 10.1245/s10434-010-0985-4. [DOI] [PubMed] [Google Scholar]
- 5.Lee HK, Yang HK, Kim WH, Lee KU, Choe KJ, Kim JP. Influence of the number of lymph nodes examined on staging of gastric cancer. Br J Surg. 2001;88:1408–12. doi: 10.1046/j.0007-1323.2001.01875.x. [DOI] [PubMed] [Google Scholar]
- 6.Shen JY, Kim S, Cheong JH. et al. The impact of total retrieved lymph nodes on staging and survival of patients with pT3 gastric cancer. Cancer. 2007;110:745–51. doi: 10.1002/cncr.22837. [DOI] [PubMed] [Google Scholar]
- 7.Wang W, Xu DZ, Li YF. et al. Tumor-ratio-metastasis staging system as an alternative to the 7th edition UICC TNM system in gastric cancer after D2 resection-results of a single-institution study of 1343 Chinese patients. Ann Oncol. 2011;22:2049–56. doi: 10.1093/annonc/mdq716. [DOI] [PubMed] [Google Scholar]
- 8.Sun Z, Xu Y, Li de M. et al. Log odds of positive lymph nodes: a novel prognostic indicator superior to the number-based and the ratio-based N category for gastric cancer patients with R0 resection. Cancer. 2010;116:2571–80. doi: 10.1002/cncr.24989. [DOI] [PubMed] [Google Scholar]
- 9.Spolverato G, Ejaz A, Kim Y, Prognostic Performance of Different Lymph Node Staging Systems After Curative Intent Resection for Gastric Adenocarcinoma. Ann Surg; 2015. [DOI] [PubMed] [Google Scholar]
- 10.Calero A, Escrig-Sos J, Mingol F. et al. Usefulness of the log odds of positive lymph nodes to predict and discriminate prognosis in gastric carcinomas. J Gastrointest Surg. 2015;19:813–20. doi: 10.1007/s11605-014-2728-5. [DOI] [PubMed] [Google Scholar]
- 11.Qiu MZ, Qiu HJ, Wang ZQ. et al. The tumor-log odds of positive lymph nodes-metastasis staging system, a promising new staging system for gastric cancer after D2 resection in China. PloS one. 2012;7:e31736. doi: 10.1371/journal.pone.0031736. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Wang X, Appleby DH, Zhang X, Gan L, Wang JJ, Wan F. Comparison of three lymph node staging schemes for predicting outcome in patients with gastric cancer. Br J Surg. 2013;100:505–14. doi: 10.1002/bjs.9014. [DOI] [PubMed] [Google Scholar]
- 13.Xu J, Bian YH, Jin X, Cao H. Prognostic assessment of different metastatic lymph node staging methods for gastric cancer after D2 resection. World J Gastroenterol. 2013;19:1975–83. doi: 10.3748/wjg.v19.i12.1975. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Liu H, Deng J, Zhang R, Hao X, Jiao X, Liang H. The RML of lymph node metastasis was superior to the LODDS for evaluating the prognosis of gastric cancer. Int J Surg. 2013;11:419–24. doi: 10.1016/j.ijsu.2013.03.009. [DOI] [PubMed] [Google Scholar]
- 15.Kattan MW, Karpeh MS, Mazumdar M, Brennan MF. Postoperative nomogram for disease-specific survival after an R0 resection for gastric carcinoma. J Clin Oncol. 2003;21:3647–50. doi: 10.1200/JCO.2003.01.240. [DOI] [PubMed] [Google Scholar]
- 16.Han DS, Suh YS, Kong SH. et al. Nomogram predicting long-term survival after d2 gastrectomy for gastric cancer. J Clin Oncol. 2012;30:3834–40. doi: 10.1200/JCO.2012.41.8343. [DOI] [PubMed] [Google Scholar]
- 17.Kim Y, Spolverato G, Ejaz A. et al. A nomogram to predict overall survival and disease-free survival after curative resection of gastric adenocarcinoma. Ann Surg Oncol. 2015;22:1828–35. doi: 10.1245/s10434-014-4230-4. [DOI] [PubMed] [Google Scholar]
- 18.Song KY, Park YG, Jeon HM, Park CH. A nomogram for predicting individual survival of patients with gastric cancer who underwent radical surgery with extended lymph node dissection. Gastric Cancer. 2014;17:287–93. doi: 10.1007/s10120-013-0270-x. [DOI] [PubMed] [Google Scholar]
- 19.Lai JF, Kim S, Kim K. et al. Prediction of recurrence of early gastric cancer after curative resection. Ann Surg Oncol. 2009;16:1896–902. doi: 10.1245/s10434-009-0473-x. [DOI] [PubMed] [Google Scholar]
- 20.Dikken JL, Baser RE, Gonen M. et al. Conditional probability of survival nomogram for 1-, 2-, and 3-year survivors after an R0 resection for gastric cancer. Ann Surg Oncol. 2013;20:1623–30. doi: 10.1245/s10434-012-2723-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Stekhoven DJ, Buhlmann P. MissForest-non-parametric missing value imputation for mixed-type data. Bioinformatics. 2012;28:112–8. doi: 10.1093/bioinformatics/btr597. [DOI] [PubMed] [Google Scholar]
- 22.Leigh JP. Assessing the importance of an independent variable in multiple regression: is stepwise unwise? J Clin Epidemiol. 1988;41:669–77. doi: 10.1016/0895-4356(88)90119-9. [DOI] [PubMed] [Google Scholar]
- 23.Royston P, Sauerbrei W. Multivariable modeling with cubic regression splines: A principled approach. Stata Journal. 2007;7:45–70. [Google Scholar]
- 24.Sauerbrei W, Royston P. Corrigendum: Building multivariable prognostic and diagnostic models: transformation of the predictors by using fractional polynomials. Journal of the Royal Statistical Society: Series A (Statistics in Society) 2002;165:399–400. [Google Scholar]
- 25.Volinsky CT, Raftery AE. Bayesian information criterion for censored survival models. Biometrics. 2000;56:256–62. doi: 10.1111/j.0006-341x.2000.00256.x. [DOI] [PubMed] [Google Scholar]
- 26.Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4. [DOI] [PubMed] [Google Scholar]
- 27.Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. 2008;26:1364–70. doi: 10.1200/JCO.2007.12.9791. [DOI] [PubMed] [Google Scholar]
- 28.Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26:565–74. doi: 10.1177/0272989X06295361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Smith DD, Schwarz RR, Schwarz RE. Impact of total lymph node count on staging and survival after gastrectomy for gastric cancer: data from a large US-population database. J Clin Oncol. 2005;23:7114–24. doi: 10.1200/JCO.2005.14.621. [DOI] [PubMed] [Google Scholar]
- 30.Schwarz RE, Smith DD. Clinical impact of lymphadenectomy extent in resectable gastric cancer of advanced stage. Ann Surg Oncol. 2007;14:317–28. doi: 10.1245/s10434-006-9218-2. [DOI] [PubMed] [Google Scholar]
- 31.Smith DD, Nelson RA, Schwarz RE. A comparison of five competing lymph node staging schemes in a cohort of resectable gastric cancer patients. Ann Surg Oncol. 2014;21:875–82. doi: 10.1245/s10434-013-3356-0. [DOI] [PubMed] [Google Scholar]
- 32.Wang X, Wan F, Wang JJ. A common misuse of stepwise regression in studies of ratio of metastatic lymph nodes for gastric cancer. Ann Surg Oncol. 2008;15:1805–6. doi: 10.1245/s10434-008-9908-z. [DOI] [PubMed] [Google Scholar]
- 33.Strong VE, Song KY, Park CH. et al. Comparison of gastric cancer survival following R0 resection in the United States and Korea using an internationally validated nomogram. Ann Surg. 2010;251:640–6. doi: 10.1097/SLA.0b013e3181d3d29b. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.