Skip to main content
Canadian Urological Association Journal logoLink to Canadian Urological Association Journal
. 2023 May 30;17(8):268–273. doi: 10.5489/cuaj.8231

Real-life benchmarking bladder cancer care: A population-based study

Nicolas Vanin Moreno 1,, Marlo Whitehead 2, D Robert Siemens 1
PMCID: PMC10426426  PMID: 37581551

Abstract

INTRODUCTION

Radical cystectomy (RC) is a complex oncological surgical procedure and population studies of routine surgical care have suggested suboptimal results compared to high-volume centers of excellence. A previous Canadian bladder cancer quality-of-care consensus led to adoption of multiple key quality-of-care indicators, with associated benchmarks created using available evidence and expert opinion to inform and measure future performance. Herein, we report real-life benchmark performance for the management of muscle-invasive bladder cancer (MIBC) relative to expert opinion guidance.

METHODS

This is a population-based, retrospective, cohort study that used the Ontario Cancer Registry (OCR) to identify all incident patients who underwent RC from 2009–2013. Electronic records of treatment from 1573 patients were linked to OCR; pathology records were obtained for all cases and reviewed by a team of trained data abstractors. The primary objective was to describe benchmarks for identified indicators, first as median values obtained across hospitals or providers, as well as a “pared-mean” approach to identify a benchmark population of “top performance,” as defined as the best outcome accomplished for at least 10% of the population.

RESULTS

Overall, performance in Ontario across all indicators fell short of expert opinion-determined benchmarks. Annual surgical volume by each surgeon performing a RC (benchmark >6, percent of institutions meeting benchmark=20%), percent of patients with MIBC referred preoperatively to medical oncology (MO; benchmark>90%, percent of institutions meeting benchmark=2%) and radiation oncology (RO; benchmark>50%, percent of institutions meeting benchmark=0%), time to cystectomy within six weeks of transurethral resection of bladder tumor (TURBT) in patients without neoadjuvant chemotherapy (benchmark <6 weeks, percent of institutions meeting benchmark=0%), percent of patients with adequate lymph node dissection (defined as >14 nodes, benchmark>85%, percent of institutions meeting benchmark=0%), percent of patients with positive margins post-RC (benchmark <10%, percent of institutions meeting benchmark=46%), and 90-day mortality (benchmark<5%, percent of institutions meeting benchmark=37%) fell considerably short. Simply evaluating benchmarks across the province as median performance significantly underestimated benchmarks that were possible by top-performing hospitals.

CONCLUSIONS

Performance through most bladder cancer quality-of-care indicators fall short of benchmarks proposed by expert opinion. Different methodologies, such as a paredmean approach of top performers, may provide more realistic benchmarking.

INTRODUCTION

The management of muscle-invasive bladder cancer (MIBC) is complex, requiring expedited diagnosis, referral to a coordinated multidisciplinary care team, and subsequent delivery of optimized multimodal treatment to maximize survival and maintain quality of life. Previous studies of routine MIBC surgical care in many different jurisdictions have suggested suboptimal outcomes as compared to high-volume centers of excellence. 13 A multidisciplinary committee comprised of representatives from Bladder Cancer Canada (BCC), the Canadian Urological Association (CUA), and the Canadian Urologic Oncology Group (CUOG ) convened during subsequent Bladder Cancer Quality of Care Consensus Meetings (BCQM; 2014, 2016, 2019) to establish the importance of creating specific quality-of-care indicators, with corresponding consensus benchmarks, in order to evaluate and compare bladder cancer performance both within and across institutions and enhance care nationally. Thirteen key quality indicators were identified, nine of which related to MIBC, and benchmarks were created using expert consensus and relevant supporting literature.46

Since its introduction by the Xerox company as a cost-reducing effort, benchmarking principles have been conceptualized in healthcare as a method for continuous quality improvement.7 Various methodologies can be used but the process involves assessing standards, such as quality indicators of care, and then identifying potential areas of improvement. Foundational to these processes is the need for reliable and up-to-date data, sharing information and adopting best practices to modify performance. To assess the real-world performance of MIBC care across Ontario, we herein report a limited number of processes and outcomes relative to the benchmarks set by the consensus meetings described above. Further, we evaluate alternative benchmarking methodologies, including median values across hospitals and the “pared-mean” approach, to identify top performers (The University of Alabama at Birmingham’s Achievable Benchmarks of Care, ABCTM ) with this population-level data.8

METHODS

This is a population-based, retrospective, cohort study used utilized the Ontario Cancer Registry (OCR) to identify all incident patients who underwent radical cystectomy (RC) for localized MIBC from 2009–2013.9 Electronic records of treatment from 1573 patients were linked to OCR. Patient characteristics are as described by Lusty et al.10 Pathology records were obtained for all cases and reviewed by a team of trained data abstractors. Hospital care and surgical intervention data was obtained from the Canadian Institute for Health Information,10 and chemotherapy utilization information was obtained using treatment records from Ontario regional cancer centers and physician billing records, as described by Booth et al.11 These datasets were linked using unique encoded identifiers and analyzed at ICES, Kingston, Ontario. The quality of the ICES database in terms of accuracy and coding has been previously discussed.12 Surgical wait times were measured from time of diagnosis documented within OCR to the time of RC.

The main objectives were to provide a province-wide assessment of these identified quality indicator benchmarks and determine the overall number of institutions meeting these benchmarks over the time frame. After assessing the number of institutions meeting the proposed benchmark, we determined the number of patients that this represented. In other words, we report on the number of patients managed in those institutions attaining the benchmark over the entirety of the five years of the study period. Although a detailed institutional review is not possible with ICES privacy policies, we felt that assessing this over a five-year horizon would give a more global assessment without concern for yearly variations, especially for some centers with relatively low volume.

Further, indicators of care were compared to benchmarks using average/median values across all patients treated in Ontario and the ABCTM method. For the latter, institutions were ranked in descending order of annual performance based on a specific quality indicator. Beginning with the best performing institution, patients in each hospital were then summed until the combined population of this group represented 10% of the total patient pool. As per institutional policy, cells with <6 patients were not reported due to privacy concerns.

Ethics for this study was obtained and approved by the Research Ethics Board of Queen’s University.

RESULTS

A total of 1573 patients from 50 institutions were included in the analysis, representing all patients treated with RC in the province. Data from institutions performing less than 10 procedures over the full five-year time frame were excluded from the benchmarking exercises (n=9). Demographic data, as well as details of the cancer and treatment-related variables have been previously published.10 The quality-of-care indicators and benchmarks for localized MIBC, defined by the BCQM meetings, are listed in Table 1. The number of institutions meeting the expert opinion-based benchmarks across Ontario over the entirety of the five-year period are outlined in Table 2. No institutions consistently met the benchmark for any of the seven observed indicators and only a minority of the institutions were observed to meet some of the expert opinion benchmarks, specifically positive margin rates and 90-day mortality.

Table 1.

Quality of care indicators and benchmarks for MIBC care as defined by expert opinion*

Group Indicator Benchmark

Structure Annual surgical volume of radical cystectomy by each performing surgeon >6

Process Percent of patients with no neoadjuvant chemotherapy who had radical cystectomy within six weeks of last TURBT >90%
For patients with MIBC, percent who received any curative intent definitive therapy (radical cystectomy or radiation-based therapy) >80%
Percent of patients with adequate lymph node dissection defined as >14 nodes >85%
Percent of patients with MIBC being seen by medical oncologist (or discussed at a multidisciplinary tumor board) preoperatively for consideration of neoadjuvant chemotherapy >90%
For patients with MIBC and receiving neoadjuvant chemotherapy, percent who completed a minimum of three cycles of cisplatin-based combination therapy >80%
Percent of patients with MIBC on TURBT being referred to radiation oncology preoperatively for consideration of radiotherapy >50%

Outcome Percent with positive soft tissue margin at radical cystectomy <10%
Percent of patients deceased within 90 days post-cystectomy <5%
*

Quality-of-care indicators, adapted from the 3rd BCQM (2019).6

MIBC: muscle-invasive bladder cancer; TURBT: transurethral resection of bladder tumor.

Table 2.

Real-world performance of quality indicators of radical cystectomy care in Ontario: Percent of institutions meeting expert opinion benchmarks from 2008–2013

Indicator Expert opinion benchmark % of institutions meeting benchmark (total n=41)# % of patients within the whole cohort (n=1573) treated by those institutions meeting benchmark*
Annual surgical volume of radical cystectomy by each surgeon performing this procedure >6 100% 20% 50%
Percent of patients with no neoadjuvant chemotherapy who had radical cystectomy within six weeks of last TURBT >90% 0% 0%
Percent of patients with adequate lymph node dissection defined as >14 nodes >85% 0% 0%
Percent of patients with ≥cT2 seen by medical oncologist preoperatively for consideration of neoadjuvant chemotherapy >90% 2% 0.5%
Percent of patients with ≥cT2 being referred to radiation oncology preoperatively for consideration of radiotherapy >50% 0% 0%
Percent with positive soft tissue margin at radical cystectomy <10% 46% 48%
Percent of patients deceased within 90 days post-cystectomy <5% 37% 31%
*

Any fields with missing data were excluded from the denominator.

#

Only including institutions with less >10 cases performed over the 5 years.

MIBC: muscle-invasive bladder cancer; TURBT: transurethral resection of bladder tumor.

We then assessed these seven indicators for all patients across the province (i.e., not per institution) to understand more globally how patients are managed across the province in routine care. Some proposed strategies have used such averaged values across jurisdictions as a benchmark. Table 3 describes the median value and interquartile range (IQR), including the percent of patients with clinical T2 or greater disease seen by medical oncology (31%) and radiation oncology (10%) prior to surgery across the province over the five years. Finally, we assessed the pared-mean approach to identifying top performers (ABCTM ) as an assessment of potentially attainable benchmarks (Table 3). As one example, no institution met the wait time benchmark for surgery (without preoperative chemotherapy) of >90% of patients having surgery within six weeks. The median wait time for all patients in Ontario was nine weeks (Q1, Q3: 6, 14); however, of the top-performing hospitals from 2009–2013, 56% of patients received surgery within six weeks. Table 3 describes the expert opinion benchmark, the actual data for all patients treated in Ontario, and the attainable benchmark for the top-performing hospitals. Table 4 reports that detailed data for the top-performing hospitals for each indicator.

Table 3.

Exploring different benchmark methodologies for quality-of-care indicators (overall provincial performance and ABCTM benchmark of top institutions#)

Indicator Expert opinion benchmark Overall provincial performance*, median (IQR) ABCTM benchmarks (actual performance observed from top institutions)
Annual surgical volume of radical cystectomy by each surgeon performing this procedure >6 100% 3.8 annual cases (1.8, 10.6) 95%
Percent of patients with no neoadjuvant chemotherapy who had radical cystectomy within six weeks of last TURBT >90% 9.1 weeks (6, 14) 56%
Percent of patients with adequate lymph node dissection defined as >14 nodes >85% 12 nodes (7, 19) 75%
Percent of patients with ≥cT2 seen by medical oncologist preoperatively for consideration of neoadjuvant chemotherapy >90% 31% (20, 44)^ 71%
Percent of patients with ≥cT2 being referred to radiation oncology preoperatively for consideration of radiotherapy >50% 10% (3.9, 14.9)^ 45%
Percent with positive soft tissue margin at radical cystectomy <10% 12% (7.1, 17.6) 1.3%
Percent of patients deceased within 90 days post-cystectomy <5% 8% (3.2, 12.1) 0.2%
*

For the entire province over 5 years. Median (IQR).

#

Institutions with less than 10 cases over the 5 years were excluded (n=9).

^

Percent of cases per institution.

IQR: interquartile range displayed as (Q1, Q3); MIBC: muscle-invasive bladder cancer. TURBT: transurethral resection of bladder tumor.

Table 4.

Benchmarking of top institutions performance for quality indicators (rank order of best performing institutions and actual quality indicator attained over 5 years*)

Expert opinion benchmark Rank order Actual performance observed by top institutions

Percent of patients with adequate lymph node dissection defined as >14 nodes 1 80.0
2 72.7
3 70.8
4 67.0

Percent with positive soft tissue margin at radical cystectomy 1 0.0
2 0.0
3 0.0
4 0.0
5 0.0
6 0.0
7 4.2
8 6.3

Percent of patients with ≥cT2 bladder cancer seen by medical oncologist preoperatively for consideration of neoadjuvant chemotherapy 1 100
2 83.3
3 59.3
4 57.1
5 55.6
6 51.9

Those with ≥cT2 bladder cancer seen by an RO pre-surgery 1 50.0
2 45.2
3 40.0
4 37.0

Percent of patients deceased within 90 days post-cystectomy 1 0.0
2 0.0
3 0.0
4 0.0
5 0.0
6 0.0
7 0.0
8 1.8
9 2.9

Annual surgical volume of radical cystectomy by each surgeon performing this procedure (>6) 1 100
2 94.6
3 90.3

Percent of patients with no neoadjuvant chemotherapy who had radical cystectomy within six weeks of last TURBT 1 57.1
2 55.6
3 48.4
4 45.0
*

Best performing institutions ranked until at least 10% of patients involved in the province from 2009–2103.

RO: radiation oncology; TURBT: transurethral resection of bladder tumor.

DISCUSSION

This report is the first to examine performance of quality indicators of care for patients with non-metastatic MIBC in the province of Ontario relative to benchmarks derived from the BCQM meetings and alternative benchmarking methodologies. When evaluated at first glance, this reveals fairly significant discrepancies between the expert-derived consensus benchmarks and routine care throughout the province.

This is specifically evident by the fact that no institutions in Ontario achieved a wait time to RC within six weeks of transurethral resection of bladder tumor (TURBT) (expert benchmark >90%). Alternate strategies to benchmark this wait time demonstrated that the median wait across the province was 9.14 weeks (Q1, Q3: 6, 14). The top-performing hospitals (involved with at least 10% of the patients treated over the five years) were able meet a benchmark of <6 weeks 56% of the time. As evidence points to an increased risk of bladder cancer mortality with increasing wait times to cystectomy,12 this finding identifies significant gaps in care across the province and highlights the need to identify strategies to streamline cancer care that is guideline-concordant,11,13 despite ongoing stressors in this universal healthcare system. Although this data is somewhat historical, it is likely Ontario hospitals remain similarly stressed today for resources after the pandemic.

The percent of patients with MIBC referred to medical oncology and radiation oncology preoperatively in Ontario fell short of the expert opinion benchmarks, with an observed overall rate from all centers of 31% and 10%, respectively. Evaluation of the top-performing institutions appeared closer to the expert opinion benchmarks, with 71% of patients being referred to medical oncology and 45% to radiation oncology over the five-year timeframe. The cohort described is now over a decade ago but nonetheless, the current guideline-concordant care outlining a multidisciplinary approach to MIBC has been generally accepted during the era that this cohort represents.14,15 Further, more contemporary series from major U.S. centers have reported similar neoadjuvant chemotherapy rates to those reported here.16

This report shows that no institutions across Ontario met the target of adequate lymph node dissection (>14 nodes in >85% of patients), with the top performers achieving these predetermined benchmarks on approximately 75% of their patients. Based on a large amount of evidence supporting the associations of higher lymph node yield and survival, and the performance of the top-performing institutions, it stands to reason that a target of >90% having more than 14 nodes removed is an appropriate goal for urologists to continue to work towards.6 A similar conclusion, that the expert opinion benchmarks are attainable, could be made when the positive margin (1.3%) and 90-day mortality (0.2%) results of top-performing institutions are noted; however, the top performers identified for these two analyses were from lower-volume centers, and the results may not be representative of larger populations, as they may represent significant selection biases.

Finally, although annual surgical volume by each surgeon performing a RC (benchmark >6) was achieved in only 20% of institutions across Ontario during the study period, this represented approximately 50% of the patients receiving RC. The top-performing hospitals, obviously all high-volume centers, were able to ensure the annual surgeon volume was >6 for 95% of their patients treated. With ongoing literature supporting the reduction of morbidity and mortality following major uro-oncological surgery when performed at high-volume centers,2,3 this finding calls into question the need to re-evaluate the allocation of complex uro-oncological procedures to specific high-volume centers. Specifically, the median annual surgeon volume in Ontario over this time period was 3.8 cases (Q1, Q3; 1.8, 10.6). These results hopefully have changed with ongoing passive centralization of care.2

CONCLUSIONS

Overall, this analysis is the first to evaluate RC care across the province based on expert-derived consensus guidelines. By using alternative benchmarking techniques to assess the data, we can provide a more thorough and indepth commentary on real-life performance, appropriateness of these guidelines, and ultimately improve quality of care across the nation. Future studies will aim at addressing the limitations of this current work, which as a retrospective, static picture of care does not provide a commentary on the evolution of performance, the entirety of care indicators, direct quantification of referrals to oncological/multidisciplinary services, nor allow comparison of care against other Canadian provincial systems. Given the inherent limitations of this dataset with incomplete understanding of patient- or disease-related details, our goal for this manuscript was not necessarily to judge performance of institutions but rather to use different strategies to assess attainable performance of quality indicators, allowing us to better consider such benchmarks moving forward. Finally, despite some of the large gaps seen between suggested benchmarks and actual care delivered, as a urological community, it is not unreasonable to set benchmarks high to drive optimized care; however, one of the principles of setting such benchmarks is to set realistically attainable goals to engage surgeons and hospitals to drive processes and structures of care.

ACKNOWLEDGEMENTS

This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH ) and the Ministry of Long-Term Care (MLTC).

Parts of this material are based on data and information provided by Ontario Health (OH ). The opinions, results, view, and conclusions reported in this paper are those of the authors and do not necessarily reflect those of OH. No endorsement by OH is intended or should be inferred.

The authors thank IQVIA Solutions Canada Inc. for use of their Drug Information File. Parts of this material are based on data and information compiled and provided by the Ontario Ministry of Health. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.

Parts of this report are based on Ontario Registrar General (ORG ) information on deaths, the original source of which is Service Ontario. The views expressed therein are those of the author and do not necessarily reflect those of ORG or the Ministry of Public and Business Service Delivery.

The authors thank the Toronto Community Health Profiles Partnership for providing access to the Ontario Marginalization Index.

Parts of this material are based on data and/or information compiled and provided by CIHI and the Ontario Ministry of Health. The analyses, conclusions, opinions, and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.

Footnotes

COMPETING INTERESTS: Dr. Siemens has participated in clinical trials supported by Astellas, Janssen, Merck, and Pfizer. The remaining authors do not report any competing personal or financial interests related to this work.

This paper has been peer-reviewed.

Visit https://www.cua.org/UROpedia to complete the multiple-choice questionnaire associated with this article. This program is an Accredited Self-Assessment Program (Section 3) as defined by the Maintenance of Certification Program of The Royal College of Physicians & Surgeons of Canada, and approved by the Canadian Urological Association. You may claim a maximum of 1 hour of credit.

REFERENCES

  • 1.Kassouf W, Aprikian A, Black P, et al. Recommendations for the improvement of bladder cancer quality of care in Canada: A consensus document reviewed and endorsed by Bladder Cancer Canada (BCC), Canadian Urologic Oncology Group (CUOG), and Canadian Urological Association (CUA) Can Urol Assoc J. 2016;10:E46–80. doi: 10.5489/cuaj.3583. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Siemens DR, Visram K, Wei X, Booth C. Effect of centralization on complex surgical care: A population-based case study of radical cystectomy. Can Urol Assoc J. 2020;14:91–6. doi: 10.5489/cuaj.5998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Grande P, Campi R, Rouprêt M. Relationship of surgeon/hospital volume with outcomes in uro-oncology surgery. Curr Opin Urol. 2018;28 doi: 10.1097/MOU.0000000000000490. [DOI] [PubMed] [Google Scholar]
  • 4.Khare SR, Aprikian A, Black P, et al. Quality indicators in the management of bladder cancer: A modified Delphi study. Urol Oncol. 2017;35:328–34. doi: 10.1016/j.urolonc.2016.12.003. [DOI] [PubMed] [Google Scholar]
  • 5.Kassouf W, Aprikian A, Saad F, et al. Improving patient journey and quality of care: Summary from the second Bladder Cancer Canada-Canadian Urological Association-Canadian Urologic Oncology Group (BCC-CUA-CUOG) bladder cancer quality of care consensus meeting. Can Urol Assoc J. 2018;12:E281–97. doi: 10.5489/cuaj.5246. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kassouf W, Aprikian A, Saad F, et al. Continuing towards optimization of bladder cancer care in Canada: Summary of the third Bladder Cancer Canada-Canadian Urological Association-Canadian Urologic Oncology Group (BCC-CUA-CUOG) bladder cancer quality of care consensus meeting. Can Urol Assoc J. 2020;14:E115–25. doi: 10.5489/cuaj.6378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Willmington C, Belardi P, Murante AM, et al. The contribution of benchmarking to quality improvement in healthcare. A systematic literature review. BMC Health Serv Res. 2022;22:139. doi: 10.1186/s12913-022-07467-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Weissman NW, Allison JJ, Kiefe CI, et al. Achievable benchmarks of care: the ABCTMs of benchmarking. J Eval Clin Pract. 1999;5:269–81. doi: 10.1046/j.1365-2753.1999.00203.x. [DOI] [PubMed] [Google Scholar]
  • 9.Clarke E, Marrett L, Kreiger N. Cancer Registration Principles and Methods. IARC Publ; 1991. Appendix 3 (c) Cancer registration in Ontario: A computer approach; pp. 246–57. [PubMed] [Google Scholar]
  • 10.Lusty A, Doiron RC, Booth CM, et al. No outcome differences after cystectomy between patients with de novo muscle-invasive bladder cancer compared to progressors: A retrospective population-based study. J Urol. 2021;206:260–9. doi: 10.1097/JU.0000000000001715. [DOI] [PubMed] [Google Scholar]
  • 11.Booth CM, Karim S, Brennan K, et al. Perioperative chemotherapy for bladder cancer in the general population: Are practice patterns finally changing? Urol Oncol. 2018. pp. 89–e13. [DOI] [PubMed]
  • 12.Williams J, Young W. Patterns of health care in Ontario: the ICES practice atlas. 2nd ed. Vol. 339. Ottawa: Canadian Medical Association; 1996. A summary of studies on the quality of health care administrative databases in Canada; p. 45. [Google Scholar]
  • 13.Gladstone E, Smolina K, Morgan SG, et al. Sensitivity and specificity of administrative mortality data for identifying prescription opioid-related deaths. Can Med Assoc J. 2016;188:E67–72. doi: 10.1503/cmaj.150349. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Kulkarni GS, Urbach DR, Austin PC, et al. Longer wait times increase overall mortality in patients with bladder cancer. J Urol. 2009;182:1318–24. doi: 10.1016/j.juro.2009.06.041. [DOI] [PubMed] [Google Scholar]
  • 15.Spooner J, Lawen T, Ory J. Triaging urological surgeries to cope with the coronavirus-19 pandemic. Curr Opin Urol. 2022;32:131–40. doi: 10.1097/MOU.0000000000000956. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.International Collaboration of Trialists, Medical Research Council Advanced Bladder Cancer Working Party (now the National Cancer Research Institute Bladder Cancer Clinical Studies Group), European Organisation for Research and Treatment of Cancer Genito-Urinary Tract Cancer Group et al. International phase 3 trial assessing neoadjuvant cisplatin, methotrexate, and vinblastine chemotherapy for muscle-invasive bladder cancer: long-term results of the BA06 30894 trial. J Clin Oncol. 2011;29:2171–7. doi: 10.1200/JCO.2010.32.3139. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Canadian Urological Association Journal are provided here courtesy of Canadian Urological Association

RESOURCES