External validation of a nomogram predicting overall survival of patients diagnosed with endometrial cancer

Abstract

Objectives

Nomograms are predictive models that provide the overall probability of a specific outcome. Nomograms have shown better individual discrimination than currently used staging systems in numerous tumor entities. Recently, a nomogram for predicting overall survival (OS) in women with endometrial cancer was introduced by Memorial Sloan-Kettering Cancer Center (MSKCC). The aim of this study was to test the validity of the MSKCC endometrial cancer nomogram using an independent, external patient cohort.

Methods

The MSKCC nomogram is based on five readily available clinical characteristics. A multi-institutional endometrial cancer database was used to test the nomogram’s validity. All consecutive patients treated for endometrial cancer between December 1995 and May 2011 and who had all nomogram variables documented were identified for analysis.

Results

Seven hundred sixty-five eligible patients were identified and used for external validation analysis. In the Austrian patient cohort, median OS was 134 months, and 3-year and 5-year OS rates were 83.8% (95% CI, 80.6–86.5%) and 77.2% (95% CI, 43.5–80.5%), respectively. The nomogram concordance index was 0.71 (SE=0.017; 95% CI, 0.68–0.74). The correspondence between the actual OS and the nomogram predictions suggests a good calibration of the nomogram in the validation cohort.

Conclusion

The MSKCC endometrial cancer nomogram was externally validated and was shown to be generalizable to a new and independent patient population. The nomogram provides a more individualized and accurate estimation of OS for patients diagnosed with endometrial cancer following primary therapy. The nomogram can be used for counseling patients more accurately and for better stratifying patients for clinical trials.

Introduction

Nomograms are predictive models that provide the overall probability of a specific outcome [1]. Several recently constructed nomograms have shown better individual discrimination than current staging systems [2–6]. Endometrial cancer is the most common gynecologic malignancy in the United States [7]. Patient counseling and treatment planning is primarily based on the International Federation of Gynecology and Obstetrics (FIGO) system. The staging system was recently revised for endometrial cancer by modifying the 1988 staging criteria; these changes have been controversial [8,9]. Clinicopathological factors, other than those listed by the FIGO system, may play equally important roles in defining distinctive outcome groups [10–12].

Memorial Sloan-Kettering Cancer Center (MSKCC) recently constructed a nomogram to predict overall survival (OS) in women with endometrial cancer following primary therapy [2]. The nomogram was constructed based on clinicopathologic findings, such as age, stage, grade, histology, and number of negative lymph nodes, which were found to be clinically significant. The model was constructed using a large, single-institution (MSKCC) cohort of patients diagnosed with endometrial cancer. The model was internally validated using cross-validation and bootstrapping methods. Nevertheless, external validation in an independent set of patients is crucial to ensure external applicability to patients from different institutions. The aim of this study was to investigate whether the recently introduced nomogram is generalizable to a new population of patients with endometrial cancer. A database of two large European academic cancer centers was used for external validation.

Materials and Methods

Patients

The institutional review boards of MSKCC, the Medical University of Vienna, and the Medical University of Innsbruck approved this study. Data were abstracted from the institutions’ prospectively maintained endometrial cancer databases following the same inclusion criteria [2]. Electronic medical records and surgery notes were also reviewed. In total, 874 patients with endometrial cancer received primary surgical treatment at the Medical University of Vienna (Vienna, Austria) and the Medical University of Innsbruck (Tirol, Austria) between December 1995 and May 2011. Of these patients, 765 who had all nomogram variables documented were selected for validation analysis. Patients were treated with upfront surgery according to international guidelines. Board-approved pathologists, specialized in gynecological pathology, assessed the pathological specimens. Histological staging and grading was performed according to the 1988 FIGO classification system on the basis of the final evaluation of the pathological specimen. Adjuvant therapy was administered at the physician’s discretion according to international guidelines. Patients were followed in the institutions’ outpatient departments. Recurrent disease was diagnosed by biopsy or imaging studies. If patients did not show for scheduled follow-up visits, they were contacted. Information on cause of death was obtained from the national record section.

Outcomes

Overall survival (OS) was calculated from the date of surgery to either the last follow-up or the date of death. In order to compare the predictions of the nomogram with the actual outcome, OS probabilities were estimated using the Kaplan-Meier method and were compared with the nomogram’s calculated predicted probabilities.

Validation

The performance of the nomogram in the external cohort was assessed through calibration and discrimination. Calibration assesses how far predictions are from actual outcomes, whereas discrimination evaluates whether the model is able to discriminate patients with longer versus shorter survival. The nomogram discriminative ability was estimated on the external cohort by estimating the concordance probability as defined by Gonen [13]. This concordance probability denotes the probability that of two randomly selected patients the patient who survives longer will have a higher survival probability than the patient with shorter survival. The calibration of the nomogram was assessed via a calibration plot by plotting the nomogram’s predicted 3-year survival probability against the patient observed or actual probability as calculated by the Kaplan-Meier method for five sub-cohorts. Each sub-cohort was obtained by ranking patients based on their predicted probabilities and dividing the entire cohort into five equal groups. The calibration plot describes how far predictions are from actual outcomes, ie, the 45-degree line, and how the nomogram fits the external data. Statistical analyses were performed using SAS and SPlus software S-Plus (Version 2000 Professional, Redmond, Washington) with the Design and Hmisc libraries and library PHCPE in R 2.5 [13,14].

Results

Patients

Seven hundred sixty-five eligible patients were included in this study—419 from the Comprehensive Cancer Center Vienna and 346 from the Medical University of Innsbruck. Patient characteristics of the MSKCC cohort and the new validation cohort (the Austrian cohort) are provided in Table 1. Both cohorts were mainly composed of Caucasian women. One of the main differences between the two cohorts was a significantly higher rate of patients with papillary and serous histologies in the MSKCC cohort. The Austrian cohort did not include patients with uterine carcinosarcoma. At the time of surgery, patients from the Austrian cohort were slightly older than their MSKCC counterparts. Additional differences included a higher rate of poorly differentiated tumors and a higher rate of lymphadenectomy in the MSKCC cohort. In the Austrian cohort, lymph node dissection was performed in 352 patients (46%). Of these, 60 (7.8%) were found to have positive lymph nodes. Adjuvant radiotherapy was administered to 382 patients (50.3%) and chemotherapy to 93 patients (12.2%).

Table 1.

Patient characteristics of the Austrian (N=765) and the published MSKCC (N=1735) cohorts

Parameter	Austrian cohort	MSKCC data	P value
Age at Diagnosis
Median (Mean)	66 (65.7)	62 (62.2)	<0.001*
Range	25–93	25–93
Negative Lymph Nodes
Median (Mean)	0 (9.5)	7 (11.6)	<0.001*
Range	0–64	0–92
No Lymphadenectomy	416 (54.4%)	687 (39.6%)
Stage
IA	132 (17.3%)	501 (28.9%)	<0.001
IB	318 (41.6%)	590 (34%)
IC	99 (12.9%)	141 (8.1%)
IIA	32 (4.2%)	36 (2.1%)
IIB	38 (5.0%)	75 (4.3%)
IIIA	47 (6.1%)	116 (6.7%)
IIIB/C	69 (9.0%)	141 (8.1%)
IVA/B	30 (3.9%)	135 (7.8%)
Grade†
1	331 (43.3%)	471 (27.3%)	<0.001
2	284 (37.1%)	622 (36%)
3	150 (19.6%)	634 (36.7%)
Missing	0	8
Histology
Adeno	704 (92%)	1376 (79.3%)	<0.001
MMMT	0 (0%)	100 (5.8%)
PapS/CleC	61 (8%)	259 (14.9%)

^*P values with * were obtained by using Wilcoxon-rank sum test, while other P values were obtained using the Chi-square test.

^†The missing values in “Grade” are ignored in the comparison test.

Median follow-up for survivors was 53 months, and 154 (20.1%) of 765 patients died. Three-year and 5-year OS rates for the Austrian cohort were 83.8% (95% CI, 80.6–86.5%) and 77.2% (95% CI, 43.5–80.5%), respectively. Median OS was 134 months (95% CI, 134–not reached), and Kaplan-Meier OS curves are shown in Figure 1. At the time of last follow-up, 566 patients (74%) were alive without evidence of disease, 23 (3%) had stable disease, 22 (2.9%) had progressive disease, 76 (9.9%) had died of non-disease-related cause, and 78 (10.2%) had died of disease.

Figure 1.

Kaplan-Meier overall survival curve for the Austrian validation cohort (N=765).

Nomogram Validation

The published nomogram is shown in Figure 2. The nomogram included age at diagnosis, 1988 FIGO stage, final FIGO grade, number of negative lymph nodes, and histologic subtype.

Figure 2.

MSKCC endometrial cancer nomogram using 5 readily available clinical characteristics to predict 3-year overall survival (Reprinted from [2] with permission from Elsevier). Instructions: Locate patient’s variable on the corresponding axis. Draw a line to the Points axis. Sum the points. Draw a line from the Total Points axis to the 3-year overall survival probability axis

The nomogram concordance probability was 0.71 (SE=0.017; 95% CI, 0.68–0.74) in the validation cohort compared to 0.75 (SE=0.011; 95% CI: 0.72–0.77 in the MSKCC cohort. This was expected as there is higher predictive accuracy when applying a nomogram to the cohort that was used to develop the model compared to an external cohort [15]. The calibration plot illustrates how the prediction of the nomogram compares with the actual outcomes of the Austrian data set (Figure 3). The x axis shows the prediction calculated with the use of the MSKCC nomogram, and the y axis shows the observed 3-year OS rate. The diagonal line represents the performance of an ideal nomogram, in which predicted outcome would correspond identically with the actual outcome. The line with error bars represents the performance of the MSKCC nomogram applied to the Austrian data set. The correspondence between the actual OS and the nomogram predictions suggests a good calibration of the nomogram in the validation cohort. The OS probabilities determined by the combined prognostic factors expressed through the nomogram broken down by the FIGO 1988 staging groups are shown in Figure 4. Patients with FIGO stage IA show homogenous 3-year OS predictions, with similar distribution of nomogram scores. Patients with stage IIIB-IIIC endometrial cancer are known to have heterogenous prognosis. This is reflected in a wide range of different predictions when the Austrian cohort is broken down by FIGO stage. Overall, the nomogram has better discrimination than staging alone, since it combines additional prognostic factors in a multivariate setting, and thus it is able to discriminate high-, moderate-, and low-risk patients in a more detailed way through individualized predictions [4,5].

Figure 3.

Calibration plot for predicted (using the MSKCC nomogram) and observed 3-year overall survival. Patients were divided in five equal groups based on predicted risk. The 45-degree straight line represents ideal agreement between actual and predicted probabilities. The vertical bars represent the 95% CI of the observed 3-year overall survival rate.

Figure 4.

Predicted 3-year survival probability by 1988 FIGO stage

Discussion

MSKCC recently published a nomogram for predicting OS in women diagnosed with endometrial cancer [2]. The MSKCC endometrial cancer nomogram is based on five readily available clinical patient characteristics and pathologic information, including age at diagnosis, number of negative lymph nodes, stage according to the 1988 FIGO classification, and histologic grade and subtype (Figure 2). Current staging systems do not incorporate these factors [8,9]. As previously discussed, this nomogram was developed for use in clinic at the first postoperative visit [2]. All necessary clinical and surgical information is usually available at that time. The usefulness of a nomogram is that it maps the predicted probabilities into points on a scale from 0 to 100 in a user-friendly graphical interface. The total points accumulated by the various covariates correspond to the predicted probability for a patient [1,16]. Incorporating other important clinical variables, beyond those of the FIGO staging system, is important to get a more accurate prediction of patients' individualized outcomes.

The MSKCC nomogram is useful to accurately predict OS throughout all stages of endometrial cancer. Patients with early stage I endometrial cancer are known to have an excellent prognosis, and 1988 FIGO stages IA and IB were recently combined; however, data from MSKCC [9] suggested that the 1988 FIGO classification of stage I endometrial cancer correctly identified three subgroups of patients (IA, IB, IC) with significantly different OS. Specifically, 1988 FIGO stages IA and IB had distinct oncologic outcomes. In addition, the revised 2009 system eliminated the most favorable group (1988 IA) from the new classification system, and the revised 2009 system for stage I did not improve its predictive ability over the 1988 system. These data highlighted the importance of developing individualized risk-prediction models and nomograms in endometrial cancer and justified the inclusion of the FIGO 1988 system into the current nomogram. The use of the nomogram allows accurate prediction even in the group of patients with early-stage disease. As seen in Figure 4, the predicted risk ranges from 0.6 to 1 for patients with stage IIA or lower. Predictions become more individualized when looking at the group of patients with stage IIIB-IIIC disease known to have heterogenous prognostic outcome [11]. The nomogram allows individualized prediction of OS for these patients in a more detailed way than by FIGO stage alone by incorporating the four additional clinical parameters. The provided prognostic information can be used for patient counseling and for individualized treatment planning.

We externally validated the MSKCC endometrial cancer nomogram and showed that the model is generalizable to a new and independent population of patients. A European multi-center cohort was used for validation. When the nomogram was applied to the Austrian patient cohort the nomogram predictions corresponded very well with the actual observed OS, suggesting a calibrated nomogram in the validation cohort.

Both patient populations mainly consisted of Caucasian women. When patients’ characteristics were compared, we found significant differences in clinical parameters. MSKCC patients were younger than Austrian patients but had a higher rate of high-grade tumors and type II histologies. This seems controversial as non-endometrioid (serous, clear cell carcinoma) histologies are typically more common in elderly patients. The authors think that this finding might be explained by the fact that MSKCC is a tertiary cancer center that sees a high rate of complex cases with aggressive histologies. Each of the two Austrian cancer centers is integrated into an Obstetric and Gynecology department. Therefore, surgeries for general gynecologic disorders are also performed at these institutions. Thus, an incidental diagnosis of (maybe less aggressive) endometrial cancer as a finding after hysterectomy might occur more frequently in this setting than at MSKCC. The Austrian cohort’s OS and the predictions based on the MSKCC nomogram corresponded well even though the patients of the two cohorts had significantly different clinical characteristics and treatment modalities varied among the institutions. The nomogram allows determination of OS for both patients with and without lymphadenectomy. The prognostic value of lymphadenectomy in endometrial cancer staging remains one of the most controversial topics in gynecologic oncology [12]. Treatment modalities vary depending on hospital type and location, and this is reflected in our two independent patient cohorts. In patients treated at MSKCC, lymphadenectomy was more frequently performed than in the validation cohort (Table 1). This is taken into consideration since the nomogram assigns a higher score to patients who had no lymph nodes resected. Patients without lymph node dissection were assigned approximately 20 additional points compared to patients who had extensive lymph node dissection with a high number of removed negative lymph nodes. In conclusion, we found significant differences of clinical parameters between the two cohorts. Our results provide evidence that the MSKCC nomogram is accurate in estimating OS irrespective of patients’ characteristics.

Our study has several limitations as is typical for retrospective analyses, although both institutions prospectively maintain their electronic databases. As is typical for data accumulated over several years, heterogeneity of post-resection adjuvant treatment and lack of comprehensive surgical staging may exist. In the Austrian endometrial cancer cohort, only patients with epithelial histologies were included, and we did not validate the nomogram for mixed carcinomatous histology. As is a common limitation of prognostic nomograms, we observed the highest prognostic accuracy for patients with common characteristics. Patients who deviate from the average patient profile and are rare will tend to have wider variation in their predicted scores compared with patients having common characteristics. We recognize that conveying uncertainty regarding estimates of prognosis is challenging for patients and clinicians. After external validation in an independent European cohort of patients diagnosed with endometrial cancer, the nomogram is ready for application in different institutions. The use of this nomogram improves patient counseling on clinical outcome and allows more individualized planning of postoperative management. The prediction tool can be accessed online on the MSKCC Web site such that when the 5 predictive factors are entered, the OS probability can be calculated automatically (http://nomograms.mskcc.org/Uterine/EndometrialSurvival.aspx).

Research Highlights.

• -The Memorial Sloan-Kettering Cancer Center endometrial cancer nomogram was externally validated.
• -The use of this nomogram allows for more individual patient counseling.