Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Mar 1.
Published in final edited form as: BJU Int. 2011 Jul 1;109(6):855–859. doi: 10.1111/j.1464-410X.2011.10391.x

Evaluating the utility of a preoperative nomogram for predicting 90-day mortality following radical cystectomy for bladder cancer

Jennifer M Taylor 1, Andrew Feifer 2, Caroline J Savage 3, Alexandra C Maschino 4, Melanie Bernstein 5, Harry W Herr 6, S Machele Donat 7
PMCID: PMC3193901  NIHMSID: NIHMS297060  PMID: 21722291

Abstract

OBJECTIVE

  • To evaluate the performance of the Isbarn nomogram for predicting 90-day mortality following radical cystectomy in a contemporary series.

PATIENTS AND METHODS

  • We identified 1141 consecutive radical cystectomy patients treated at our institution between 1995 and 2005 with at least 90 days of follow-up.

  • We applied the published nomogram to our cohort, determining its discrimination, with the area under the receiver operating characteristic curve (AUC), and calibration.

  • We further compared it with a simple model using age and the Charlson comorbidity score.

RESULTS

  • Our cohort was similar to that used to develop the Isbarn nomogram in terms of age, gender, grade and histology; however, we observed a higher organ-confined (≤pT2, N0) rate (52% vs 24%) and a lower overall 90-day mortality rate [2.8% (95% confidence interval 1.9%, 3.9%) vs 3.9%].

  • The Isbarn nomogram predicted individual 90-day mortality in our cohort with moderate discrimination [AUC 73.8% (95% confidence interval 64.4%, 83.2%)].

  • In comparison, a model using age and Charlson score alone had a bootstrap-corrected AUC of 70.2% (95% confidence interval 67.2%, 75.4%).

CONCLUSIONS

  • The Isbarn nomogram showed moderate discrimination in our cohort; however, the exclusion of important preoperative comorbidity variables and the use of postoperative pathological stage limit its utility in the preoperative setting.

  • The use of a simple model combining age and Charlson score yielded similar discriminatory ability and underscores the significance of individual patient variables in predicting outcomes.

  • An accurate tool for predicting postoperative morbidity/mortality following radical cystectomy would be valuable for treatment planning and counselling. Future nomogram design should be based on preoperative variables including individual risk factors, such as comorbidities.

Keywords: bladder cancer, cystectomy, mortality, nomograms, postoperative complications, risk assessment

INTRODUCTION

Radical cystectomy (RC) is associated with a wide spectrum of potential complications, including perioperative mortality [16]. Recent analyses of perioperative complications following RC [6,7] suggest that reported complication rates may be stark underestimates of true outcomes. Contemporary single institution and population-based mortality rates have ranged from 0.3% to 6.8% [16] reflecting significant differences in patient selection or other site-related factors. Postoperative morbidity and mortality following RC have been shown in retrospective series to be associated with surgeon volume, hospital volume and patient-related variables such as age, Charlson comorbidity score, American Society of Anesthesiology (ASA) score, body mass index, estimated blood loss, diversion type and prior radiation [8,9]. These factors are used informally to assess perioperative mortality risk prospectively. However, nomograms have shown increased predictive accuracy over physician judgement or risk classifications for survival outcomes in several malignancies [10,11]. Attention to more accurate estimation of outcomes has prompted the development of a variety of predictive tools for perioperative morbidity/mortality.

An accurate model to predict postoperative mortality would help place in perspective for patients the risks associated with RC vs a multimodality bladder-sparing treatment, in the context of a palliative care approach with which they face an 80% 2-year and 100% 5-year disease-specific mortality [12]. Furthermore, bladder-sparing treatments and the symptoms associated with uncontrolled pelvic disease may be as difficult to tolerate as RC, related to the need for repeat anaesthesia, radiation effects on the bowel, clot retention, irritative bladder symptoms, pelvic pain and other symptoms associated with uncontrolled pelvic disease.

Isbarn et al. [13] recently developed a model to predict 90-day mortality following RC, using data from the national Surveillance, Epidemiology and End Results (SEER) database. We evaluated the performance of this model in a contemporary single institution cohort of patients with prospectively collected 90-day morbidity/mortality data.

PATIENTS AND METHODS

After institutional review board approval, we identified 1141 patients undergoing RC for bladder cancer at Memorial Sloan-Kettering Cancer Center between 1995 and 2005 with a minimum of 90 days follow-up. All patient demographic, pathological and clinical treatment data were prospectively entered into an institutional database from standardized data entry forms completed at each patient visit. Age, year of surgery, gender, weight, height, ASA score, Charlson comorbidity score, clinical and pathological stage and grade, histology, nodal status, type of diversion, perioperative chemotherapy use and follow-up were then retrospectively queried from this database.

Isbarn et al. [13] used age, gender, year of surgery, type of cystectomy, pathological stage, grade and histology to develop their model, and the final predictive model included age, pathological stage, grade and histology. We calculated predicted 90-day mortality rates using the nomogram formula, and compared the predicted with actual 90-day mortality rates using the area under the receiver operating characteristic curve (AUC).

Our first aim was to validate the use of the Isbarn nomogram in our patient population. Of note, the nomogram formula, secondary to the nature of the SEER database, lacked patient comorbidity information and used postoperative pathological variables not available in the preoperative setting for counselling. We therefore compared the discrimination of the Isbarn nomogram with that of a simple model using the combination of age and Charlson comorbidity score, both variables predictive of morbidity/mortality in retrospective series, which are available preoperatively. Since the model using age and Charlson comorbidity score was built with the same data as those with which it was evaluated, we corrected for statistical overfit using repeated 10-fold cross-validation. Confidence intervals were obtained using bootstrap methods with 1000 replications. To illustrate the relationship between age and probability of death within 90 days of RC, we used locally weighted scatterplot smoothing to plot the probability of 90-day mortality, separately by Charlson comorbidity score (<2 and ≥2). All analyses were conducted using STATA 11.1 (STATA Corp., College Station, TX, USA).

RESULTS

Our cohort was similar to the Isbarn cohort with respect to age, gender, grade and histology (Table 1). With respect to stage, 52% of our patients had localized disease (pathological stage ≤T2 with no nodal involvement), compared with 24% in the Isbarn cohort. We recorded 32 deaths within the first 90 days after cystectomy for an overall perioperative mortality rate of 2.8% (95% CI 1.9%, 3.9%), comparatively lower than the rate of 3.9% reported by Isbarn et al. The Isbarn nomogram displayed good discrimination for predicting individual 90-day mortality in our cohort [AUC 73.8% (95% CI 64.4%, 83.2%)].

TABLE 1.

Patient and tumor characteristics

Number of patients 1141
Age, median (interquartile range) 68.1 (60.1, 74.7)
Gender, no. (%)
  Male 862 (75.6)
  Female 279 (24.5)
Year of surgery, no. (%)
  1995–1999 412 (36%)
  1999–2004 633 (55%)
  2005 96 (8%)
Extent of disease, no. (%)
  Localized (≤pT2, N0) 597 (52%)
  Regional (>pT2, any N or any T, N+) 544 (48%)
Grade, no. (%)
  Low/intermediate grade (1–2) 225 (20%)
  High grade (3–4) 827 (72%)
  Unknown grade 89 (8%)
Histology, no. (%)
  TCC or SCC/TCC 1003 (88%)
  SCC 32 (3%)
  Neither TCC nor SCC 33 (3%)
  pT0 73 (6%)

SCC, squamous cell carcinoma.k

The Isbarn et al. nomogram did not appear to be miscalibrated in our cohort. Among those given a lower predicted probability of death within 90 days (0–6%), patients in our cohort tended to have a lower observed mortality rate than that reported by Isbarn et al. (1.3% vs 2.5%). In contrast, those given higher predicted risks experienced a slightly higher rate of 90-day mortality (Table 2). For example, among those given a predicted risk between 6.1% and 12%, the observed rate was 7.1% in our cohort compared with 6.0% in the Isbarn et al. cohort. However, because of the low number of events, the confidence intervals around the observed mortality rates are wide.

TABLE 2.

A comparison of predicted and actual 90-day death rates in the Isbarn cohorts and our present cohort

Observed rate of 90-day mortality
Isbarn et al. Present cohort

Nomogram predictions Development cohort Validation cohort n, % (95% CI)
0–6% 2.4% 2.5% 853, 1.3% (0.6%, 2.3%)
6.1–12% 6.0% 6.0% 281, 7.1% (4.4%, 10.8%)
12.1–18% 14.7% 14.7%   6, 16.7% (0.4%, 64.1%)
18.1% or greater 26.2% 26.2%   1, n/aa
a

Only one patient.

In comparison, a simple model for 90-day mortality using only age and Charlson comorbidity score (<2 vs ≥2) had comparable discrimination [corrected AUC 70.2% (95% CI 67.2%, 75.4%)] to the Isbarn nomogram. Figure 1 illustrates the risk of perioperative mortality by age, stratified by Charlson comorbidity score. Overall risk appeared to increase with older age (P = 0.004); patients with fewer comorbidities tended to have a lower risk of mortality although a formal test of an interaction between age and Charlson comorbidity score was not significant (P = 0.5). Prediction of 90-day mortality in our cohort using ASA score alone yielded an AUC of 56.8% (95% CI 53.3%, 59.4%), which was less predictive than either the combination of age and Charlson comorbidity score (AUC 70.2%) or the Isbarn nomogram (AUC 73.8%).

FIG. 1.

FIG. 1

Probability of 90-day mortality following radical cystectomy by age, stratified by Charlson comorbidity score (grey line <2, black broken line ≥2).

DISCUSSION

A model estimating postoperative morbidity or mortality risk would be valuable to both patients and surgeons considering RC for treatment. However, the construction of a valid nomogram relies on the appropriate selection of variables for the analysis. Previous studies have shown association on multivariate analysis between postoperative mortality and variables such as patient comorbidities, surgeon experience, hospital volume and clinical setting (academic vs community), none of which are readily available in the national SEER database.

Hollenbeck et al. [6] evaluated prognostic factors for perioperative morbidity/mortality associated with RC using a prospective database with over 2500 patients at 123 Veterans Affairs Medical Centers. Their multivariate model associated increased 30-day and 90-day mortality with older age, ASA score ≥3, dependent functional status and low serum albumin. Novara et al. [2] confirmed that ASA score, a subjective assessment of the systemic effect of comorbid conditions, was significantly associated with high-grade complications, including 90-day perioperative death, among 358 patients undergoing RC. Charlson comorbidity score, originally developed using 1-year mortality data on internal medicine patients, has been widely validated in oncology populations undergoing various treatments [14]. We have previously shown the age-adjusted Charlson index to be associated with overall survival in an overlapping patient set from our institution [9]. Other published models for estimating surgical morbidity/mortality such as the surgical Apgar score [15] or Portsmouth POSSUM equation [16] all require intraoperative data not available in the preoperative setting. Although potentially useful in modulating postoperative care of patients, these models have little value in preoperative counselling.

Nomograms have been shown to outperform both physician judgement [10] and risk classification schema [11] in predicting oncological outcomes in urological malignancies. For example, risk classification schema such as the TNM staging system or the D’Amico risk categories in prostate cancer are constructed with the assumption that each risk group represents a homogeneous population. The individual heterogeneity within a risk group will lead to lower predictive accuracy when applying risk classifications to a particular patient [17]. Risk groupings present additional statistical limitations by categorizing data points, whereas nomograms achieve better accuracy by considering data as continuous variables. However, construction of a valid nomogram relies not only on large patient numbers such as those found in the SEER database but on the appropriate selection of variables for inclusion in the analysis.

The nomogram developed by Isbarn et al. was one of the first attempts at an individualized preoperative assessment tool, and we evaluated its potential role in patient counselling prior to RC. An effective preoperative decision-making tool should be based on patient and tumour characteristics available to the surgeon at the time. The model presented by Isbarn et al., however, incorporates postoperative pathological variables and excludes important individual patient risk variables, due to the limitations of the SEER database. The use of postoperative pathological data may further limit the model because of the known significant discrepancy between clinical staging and final pathological stage at cystectomy [18,19].

In addition, the development of a nomogram from registry data has important implications on the applicability of the model. The SEER registry offers advantages of a standardized multicentre data collection methodology with broad demographic distribution, mitigating the potential biases of a predictive tool created from outcomes at a single high-volume institution, whose patient population and results may not be generalizable to the greater community. However, individual comorbidities may be under-represented in SEER if they are not attributed to causing a patient’s hospitalization in claims data [20]. The lack of individual patient detail is a commonly reported drawback of SEER data [21]. The usefulness of a prediction model based on a robust database, such as SEER, is constrained by the range of variables available, and variables not in the source database may be critical to the model’s accuracy. The use of SEER data only as opposed to SEER linked to Medicare claims data also limits longitudinal follow-up and outcome reporting.

We believe it is for these reasons that, in our cohort, the Isbarn nomogram performed only slightly better than a simple model combining age and Charlson comorbidity score (AUC 73.8% vs 70.2%). In addition, as the Apgar and POSSUM models show, patient-specific and site-specific variables influence outcome prediction; models using only preoperative variables may have lower prognostic value due to the unaccounted for influence of surgeon and hospital volume and expertise and intraoperative factors such as blood loss or technical effects of prior treatment. Our comparison model is not intended as an alternative model to nomograms, but rather to illustrate the impact that the inclusion of appropriate variables may have on the accuracy of a predictive model. In other words, bigger is not always better. A robust model with wide applicability requires a large multi-institutional data set, and efforts should be made to standardize detailed patient data reporting across many institutions to provide a well-distributed spectrum of patients with adequate depth of variables.

Although RC has shown a distinct survival benefit in the treatment of muscle-invasive bladder cancer, contemporary studies of practice patterns show that the elderly are less likely to undergo an aggressive therapy, presumably because of the fear of surgical morbidity/mortality [22]. Recent studies have indicated that age alone is not the sole determining factor for morbidity/mortality with surgery, suggesting that individual comorbidities also play a major role [24,7,9,22]. A preoperative predictive model incorporating age as well as individual comorbid conditions would be helpful for counselling patients preoperatively and in weighing the risks/benefits of aggressive treatment vs the alternative of more conservative or palliative treatment. In addition, the ability to predict perioperative morbidity/mortality is an important consideration in determining the sequencing for multimodality treatments, as shown by previous studies indicating that postoperative complications can limit the administration of adjuvant chemotherapy in up to 30% of patients [23].

Although RC in the elderly population has the potential for significant morbidity and mortality, it offers similar disease-specific survival as for the younger population. Therefore, when weighing the potential morbidity/mortality associated with RC compared with that of bladder-sparing modalities, or palliative care, it would be helpful to have a valid predictive model for preoperative counselling and treatment planning. The Isbarn nomogram is one of the first attempts to provide the practising physician with such a model; however, the exclusion of important preoperative patient comorbidity variables, and the use of postoperative pathological details in its construction, limits its utility in the preoperative setting. It showed similar discrimination to a simple model using only age and Charlson score, without any disease-specific information, underscoring the significance of individual patient variables in predicting outcomes.

As both models are ineffective in predicting postoperative mortality in over a quarter of our cohort’s patients, efforts for future nomograms should strive to include a morbidity stratification and identify additional risk factors to improve accuracy. With ongoing efforts to standardize documentation of patient variables and outcomes across institutions, through several collaborative initiatives, the opportunity to combine data from multiple centres and develop a model with these detailed variables not available in SEER may be forthcoming.

ACKNOWLEDGMENTS

We especially thank Dr Pierre Karakiewicz for his assistance and for providing the mathematical formula for the nomogram. This work was supported by the Sidney Kimmel Center for Prostate and Urologic Cancers and a National Institutes of Health NRSA training grant T32-CA82088.

Abbreviations

RC

radical cystectomy

ASA

American Society of Anesthesiology

SEER

Surveillance, Epidemiology and End Results

AUC

area under the receiver operating characteristic curve

Footnotes

CONFLICT OF INTEREST

None declared.

Contributor Information

Jennifer M. Taylor, Email: taylorj3@mskcc.org, Department of Urology, Memorial Sloan-Kettering Cancer Center, New York, USA.

Andrew Feifer, Department of Urology, Memorial Sloan-Kettering Cancer Center, New York, USA.

Caroline J. Savage, Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, New York, USA

Alexandra C. Maschino, Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, New York, USA

Melanie Bernstein, Department of Urology, Memorial Sloan-Kettering Cancer Center, New York, USA.

Harry W. Herr, Department of Urology, Memorial Sloan-Kettering Cancer Center, New York, USA

S. Machele Donat, Department of Urology, Memorial Sloan-Kettering Cancer Center, New York, USA.

REFERENCES

  • 1.Konety BR, Allareddy V, Herr H. Complications after radical cystectomy: analysis of population-based data. Urology. 2006;68:58–64. doi: 10.1016/j.urology.2006.01.051. [DOI] [PubMed] [Google Scholar]
  • 2.Novara G, De Marco V, Aragona M, et al. Complications and mortality after radical cystectomy for bladder transitional cell cancer. J Urol. 2009;182:914–921. doi: 10.1016/j.juro.2009.05.032. [DOI] [PubMed] [Google Scholar]
  • 3.Quek ML, Stein JP, Daneshmand S, et al. A critical analysis of perioperative mortality from radical cystectomy. J Urol. 2006;175:886–889. doi: 10.1016/S0022-5347(05)00421-0. [DOI] [PubMed] [Google Scholar]
  • 4.Chang SS, Cookson MS, Baumgartner RG, Wells N, Smith JA., Jr. Analysis of early complications after radical cystectomy: results of a collaborative care pathway. J Urol. 2002;167:2012–2016. [PubMed] [Google Scholar]
  • 5.Ghoneim MA, el-Mekresh MM, el-Baz MA, el-Attar IA, Ashamallah A. Radical cystectomy for carcinoma of the bladder: critical evaluation of the results in 1,026 cases. J Urol. 1997;158:393–399. [PubMed] [Google Scholar]
  • 6.Hollenbeck BK, Miller DC, Taub DA, et al. The effects of adjusting for case mix on mortality and length of stay following radical cystectomy. J Urol. 2006;176:1363–1368. doi: 10.1016/j.juro.2006.06.015. [DOI] [PubMed] [Google Scholar]
  • 7.Shabsigh A, Korets R, Vora KC, et al. Defining early morbidity of radical cystectomy for patients with bladder cancer using a standardized reporting methodology. Eur Urol. 2009;55:164–174. doi: 10.1016/j.eururo.2008.07.031. [DOI] [PubMed] [Google Scholar]
  • 8.Hollenbeck BK, Miller DC, Taub D, et al. Identifying risk factors for potentially avoidable complications following radical cystectomy. J Urol. 2005;174:1231–1237. doi: 10.1097/01.ju.0000173923.35338.99. [DOI] [PubMed] [Google Scholar]
  • 9.Koppie TM, Serio AM, Vickers AJ, et al. Age-adjusted Charlson comorbidity score is associated with treatment decisions and clinical outcomes for patients undergoing radical cystectomy for bladder cancer. Cancer. 2008;112:2384–2392. doi: 10.1002/cncr.23462. [DOI] [PubMed] [Google Scholar]
  • 10.Ross PL, Gerigk C, Gonen M, et al. Comparisons of nomograms and urologists’ predictions in prostate cancer. Semin Urol Oncol. 2002;20:82–88. doi: 10.1053/suro.2002.32490. [DOI] [PubMed] [Google Scholar]
  • 11.Mitchell JA, Cooperberg MR, Elkin EP, et al. Ability of 2 pretreatment risk assessment methods to predict prostate cancer recurrence after radical prostatectomy: data from CaPSURE. J Urol. 2005;173:1126–1131. doi: 10.1097/01.ju.0000155535.25971.de. [DOI] [PubMed] [Google Scholar]
  • 12.Prout GR, Marshall VF. The prognosis with untreated bladder tumors. Cancer. 1956;9:551–558. doi: 10.1002/1097-0142(195605/06)9:3<551::aid-cncr2820090319>3.0.co;2-2. [DOI] [PubMed] [Google Scholar]
  • 13.Isbarn H, Jeldres C, Zini L, et al. A population based assessment of perioperative mortality after cystectomy for bladder cancer. J Urol. 2009;182:70–77. doi: 10.1016/j.juro.2009.02.120. [DOI] [PubMed] [Google Scholar]
  • 14.Hall WH, Ramachandran R, Narayan S, Jani AB, Vijayakumar S. An electronic application for rapidly calculating Charlson comorbidity score. BMC Cancer. 2004;4:94. doi: 10.1186/1471-2407-4-94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Prasad SM, Ferreria M, Berry AM, et al. Surgical Apgar outcome score: perioperative risk assessment for radical cystectomy. J Urol. 2009;181:1046–1052. doi: 10.1016/j.juro.2008.10.165. [DOI] [PubMed] [Google Scholar]
  • 16.Smaldone MC, Corcoran AT, Hayn M, Konety BR, Hrebinko RL, Jr, Davies BJ. Estimating postoperative mortality and morbidity risk of radical cystectomy with continent diversion using predictor equations. J Urol. 2009;182:2619–2624. doi: 10.1016/j.juro.2009.08.024. [DOI] [PubMed] [Google Scholar]
  • 17.Nguyen CT, Stephenson AJ, Kattan MW. Are nomograms needed in the management of bladder cancer? Urol Oncol. 2010;28:102–107. doi: 10.1016/j.urolonc.2009.04.020. [DOI] [PubMed] [Google Scholar]
  • 18.Shariat SF, Palapattu GS, Karakiewicz PI, et al. Discrepancy between clinical and pathologic stage: impact on prognosis after radical cystectomy. Eur Urol. 2007;51:137–149. doi: 10.1016/j.eururo.2006.05.021. [DOI] [PubMed] [Google Scholar]
  • 19.Ficarra V, Dalpiaz O, Alrabi N, Novara G, Galfano A, Artibani W. Correlation between clinical and pathological staging in a series of radical cystectomies for bladder carcinoma. BJU Int. 2005;95:786–790. doi: 10.1111/j.1464-410X.2005.05401.x. [DOI] [PubMed] [Google Scholar]
  • 20.Klabunde CN, Warren JL, Legler JM. Assessing comorbidity using claims data: an overview. Med Care. 2002;40:26–35. doi: 10.1097/00005650-200208001-00004. [DOI] [PubMed] [Google Scholar]
  • 21.Scosyrev E, Messing J, Noyes K, Veazie P, Messing E. Surveillance Epidemiology and End Results (SEER) program and population-based research in urologic oncology: an overview. Urol Oncol. 2010 doi: 10.1016/j.urolonc.2009.11.005. epub 2 Apr. [DOI] [PubMed] [Google Scholar]
  • 22.Donat SM, Siegrist T, Cronin A, Savage C, Milowsky MI, Herr HW. Radical cystectomy in octogenarians - does morbidity outweigh the potential survival benefits? J Urol. 2010;183:2171–2177. doi: 10.1016/j.juro.2010.02.015. [DOI] [PubMed] [Google Scholar]
  • 23.Donat SM, Shabsigh A, Savage C, et al. Potential impact of postoperative early complications on the timing of adjuvant chemotherapy in patients undergoing radical cystectomy: a high-volume tertiary cancer center experience. Eur Urol. 2009;55:177–185. doi: 10.1016/j.eururo.2008.07.018. [DOI] [PubMed] [Google Scholar]

RESOURCES