Abstract
Introduction
We sought to determine the effect of the presence of disseminated disease on perioperative outcomes following radical cystectomy for bladder cancer.
Methods
We identified 4108 eligible patients who underwent radical cystectomy for bladder cancer using the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) database. We matched patients with disseminated cancer at the time of surgery to those without disseminated cancer using propensity scores. The primary outcome of interest was major complications (death, reoperation, cardiac or neurological event). Secondary outcomes included pulmonary, infectious thromboembolic, and bleeding complications, in addition to prolonged length of stay. Generalized estimating equations were used to examine the association between disseminated cancer and the development of complications.
Results
Following propensity score matching and adjusting for the type of urinary diversion, radical cystectomy in patients with disseminated disease was associated with a significant increase in major complications (8.6% vs. 4.0%; odds ratio [OR] 2.50; 95% confidence interval [CI] 1.02–6.11; p=0.045). The presence of disseminated disease was associated with an increase in pulmonary complications (5.8% vs. 1.2%; OR 5.17. 95% CI 1.00–26.66. p=0.049), but not infectious complications, venous thromboembolism, bleeding requiring transfusion, and prolonged length of stay (p values 0.07–0.79).
Conclusions
Patients with disseminated cancer undergoing cystectomy are more likely to experience major and pulmonary complications. The strength of these conclusions is limited by sample size, selection bias inherent in observational data, and a lack of specific oncological detail in the database.
Introduction
Metastatic bladder cancer portends a poor prognosis, with five-year survival less than 10%.1 Since the introduction of cisplatin-based combination chemotherapy more than 20 years ago, there has been little improvement in mortality for patients with metastatic bladder cancer.2
There is growing interest in definitive local therapy for patients with metastatic cancer. The role of cytoreductive nephrectomy, in addition to cytokine therapy, has been well-established in the management of appropriately selected patients with metastatic kidney cancer.3 Recently, there has been interested in a similar approach for patients with metastatic prostate cancer, motivated by population-based, administrative data from the U.S., which have shown a survival benefit to cytoreductive radical prostatectomy.4–6 Two large, multi-institutional trials have been initiated to evaluate this approach [NCT01751438, NCT00268476].
Recently, Seisen et al demonstrated that aggressive local therapy, in addition to chemotherapy, may improve overall survival among patients with metastatic bladder cancer as compared to conservative local therapy.7 While radical cystectomy represents the current gold standard for treatment of muscle-invasive bladder cancer, it is a highly morbid procedure with 30-day postoperative complications of approximately 50%.8,9 We sought to assess the morbidity and mortality associated with radical cystectomy in patients with metastatic bladder cancer. To do so, we used the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) database, a large, multi-institutional, validated registry that has been shown to perform better than administrative databases or institutional series in capturing intraoperative and postoperative complications.10–12
Methods
The study was conducted and reported according to the recommendations of the RECORD statement.13
Study subjects
We used the participant use files of the ACS NSQIP to identify patients undergoing radical cystectomy (common procedural terminology codes 51590, 51595, 51596) with a postoperative diagnosis of bladder cancer (ICD-9 code 188.x) between January 1, 2007 and December 31, 2014. We excluded 33 patients for missing data on important covariates or outcomes: American Society of Anesthesiologists (ASA) physical status classification (n=9), height (n=15), weight (n=11), and length of stay (n=4).
Outcomes
Our primary outcome was major perioperative complications comprising mortality, reoperation, cardiac event, or neurological event.14 Secondary outcomes were pulmonary complications (re-intubation or prolonged ventilation), infectious complications (surgical site infections, pneumonia, urinary tract infection, or sepsis), venous thromboembolism (deep vein thrombosis or pulmonary embolism), bleeding requiring transfusion, and prolonged length of stay comprising hospital stays greater than the median in this cohort (seven days from the date of surgery).
Exposure
Our primary exposure was the presence of disseminated cancer at the time of surgery, a variable routinely collected in the NSQIP dataset. Previous work in colorectal cancer has shown that the disseminated cancer variable has fair-good agreement15 with metastatic stage (Cohen kappa coefficient, 0.454).16 Previous research from NSQIP has shown that patients with disseminated cancer have worse outcomes, including perioperative complications14 and mortality.16
Covariates
We abstracted standard demographic and clinical information for all patients, including age, race, body mass index (BMI), ASA physical status class (categorized as class 1–2 vs. class 3–4), history of cardiac disease, history of neurological disease, history of chronic obstructive pulmonary disease (COPD), history of diabetes (requiring oral agent or insulin), end-stage renal disease (ESRD) requiring dialysis, current smoking status (active smoker within one year), use of preoperative chemotherapy (within 90 days of surgery), use of preoperative radiotherapy (within 90 days of surgery), chronic steroid use, and functional status prior to surgery (independent vs. partially or totally dependent). BMI was categorized in keeping with the World Health Organization stratification [<18.5, 18.5–25, 25–30, >30 (kg/m2)]. We further characterized the type of urinary diversion using CPT codes (51590 and 51595=ileal conduit; 51596=continent).
Statistical analysis
Descriptive statistics were used to compare baseline demographic factors: frequencies and proportions for categorical variables and medians and interquartile ranges (IQR) for continuous variables. Because patients who have disseminated disease at the time of cystectomy may inherently differ from those with localized cancer, we used 1:1 propensity score matching with the greedy algorithm. The propensity score was calculated from a multivariable logistic regression model with independent variables comprising baseline demographic factors of age, race, BMI, ASA class, receipt of preoperative chemotherapy, history of cardiac disease, history of neurological disease, history of COPD, history of diabetes, smoking status, and functional status. Standardized differences were used to compare baseline characteristics of the two groups with differences less than or equal to 0.1 (10%) considered acceptable balance.17 We limited the propensity score to demographic factors and adjusted for the effect of differing urinary diversion types in regression models. We used bivariate analysis to examine the proportion of patients experiencing a complication, with Fisher’s exact test used to compare those with and without disseminated disease at the time of cystectomy. Finally, we examined the association between the presence of disseminated cancer and complications using generalized estimating equations with a logit wing to account for the dependence created by matching, while accounting for the effect of urinary diversion type.
All statistical analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, NC, U.S.).
Results
Baseline characteristics
During the study period, 4108 eligible patients underwent cystectomy for bladder cancer. Of these, 176 (4.3%) had disseminated disease at the time of surgery. While the proportion of patients with disseminated disease varied over the study interval (2.5–7.1%), no significant trend was identified (Cochran-Armitage test for trend p=0.95).
Patients with disseminated disease at the time of cystectomy had lower BMI and higher ASA classification, were less likely to be Caucasian, were more likely to have ESRD requiring dialysis, and were more likely to be partially or totally functionally dependent (Table 1). Following propensity score matching, the groups were acceptably balanced (Table 1). Patients with disseminated disease were significantly less likely to receive a continent urinary diversion (Table 1).
Table 1.
Baseline characteristics of patients undergoing radical cystectomy, before and after propensity score matching
| Before propensity score matching | After propensity score matching | |||||
|---|---|---|---|---|---|---|
|
| ||||||
| No disseminated cancer (n=3932) | Disseminated cancer (n=176) | Std diff | No disseminated cancer (n=174) | Disseminated cancer (n=174) | Std diff | |
| Age, median (IQR) | 69 (62–76) | 70 (62–77) | 0.00 | 71 (63–77) | 70 (62–77) | −0.05 |
| Sex, n (%) | ||||||
| Male | 3241 (82.4) | 141 (80.1) | 0.07 | 136 (78.2) | 141 (81.0) | 0.00 |
| Female | 689 (17.5) | 35 (19.9) | 38 (21.8) | 33 (19.0) | ||
| Missing | 2 (0.1) | 0 | 0 | 0 | ||
| Race, n (%) | ||||||
| Caucasian | 3376 (85.9) | 140 (79.6) | 0.18 | 137 (78.7) | 139 (79.9) | 0.03 |
| African-American | 152 (3.9) | 12 (6.8) | 11 (6.3) | 11 (6.3) | ||
| Other/unknown | 404 (10.3) | 24 (13.6) | 26 (14.9) | 24 (13.8) | ||
| BMI, median (IQR) | 28.0 (24.8–31.6) | 27.3 (23.6–31.3) | −0.14 | 27.4 (23.4–31.2) | 27.3 (23.6–31.2) | 0.02 |
| ASA category, n (%) | ||||||
| 1–2 | 1010 (25.7) | 34 (19.3) | −0.15 | 34 (19.5) | 33 (19.0) | −0.01 |
| 3–4 | 2992 (74.3) | 142 (80.7) | 140 (80.5) | 141 (81.0) | ||
| Cardiac history, n (%) | 182 (4.6) | 8 (4.6) | 0.00 | 5 (2.9) | 8 (4.6) | 0.09 |
| Neurological history, n (%) | 47 (1.2) | 4 (2.3) | 0.08 | 7 (4.0) | 4 (2.3) | 0.09 |
| History of COPD, n (%) | 337 (8.6) | 18 (10.2) | 0.06 | 15 (8.6) | 18 (10.3) | 0.06 |
| Diabetes, n (%) | 792 (20.1) | 35 (19.9) | −0.01 | 29 (16.7) | 35 (20.1) | 0.09 |
| Dialysis, n (%) | 8 (0.2) | 1 (0.6) | −0.13 | 0 | 1 (0.6) | −0.06 |
| Active smoking, n (%) | 971 (24.7) | 48 (27.3) | 0.06 | 43 (24.7) | 47 (27.0) | 0.05 |
| Preoperative chemotherapy, n (%) | 121 (3.1) | 4 (2.3) | −0.05 | 4 (2.3) | 4 (2.3) | 0.00 |
| Preoperative radiotherapy, n (%) | 4 (0.1) | 1 (0.6) | 0.08 | 2 (1.1) | 1 (0.6) | −0.06 |
| Chronic steroid use, n (%) | 129 (3.3) | 13 (7.4) | 0.06 | 12 (6.9) | 13 (7.5) 0.06 | |
| Functional status, n (%) | ||||||
| Independent | 3863 (98.3) | 169 (96.0) | −0.13 | 170 (97.7) | 167 (96.0) | −0.09 |
| Partially/totally dependent | 69 (1.8) | 7 (4.0) | 4 (2.3) | 7 (4.0) | ||
| Urinary diversion | ||||||
| Ileal conduit | 3107 (79.0) | 164 (93.2) | −0.42 | 140 (80.5) | 162 (93.1) | −0.38 |
| Continent diversion | 825 (21.0) | 12 (6.8) | 34 (19.5) | 12 (6.9) | ||
ASA: American Society of Anesthesiologists; BMI: body mass index; COPD: chronic obstructive pulmonary disease; IQR: interquartile range; Std diff: standardized difference.
Bivariate analyses
Among the matched cohort, major complications were more common among patients with disseminated disease (n=15, 8.6%) than those without (n=7, 4.0%), although this was not statistically significant (p=0.08; Table 2). Similarly, there were higher rates of pulmonary complications (5.8% vs. 1.2%; p=0.04; Table 2). There were no significant differences in infectious complications (p=0.12), venous thromboembolic events (p=0.42), bleeding requiring a transfusion (p=0.59), or prolonged length of stay (p=0.91).
Table 2.
Crude rates of complications during radical cystectomy
| Before propensity score matching | After propensity score matching | |||||
|---|---|---|---|---|---|---|
|
| ||||||
| n (%) | No disseminated cancer | Disseminated cancer | p | No disseminated cancer | Disseminated cancer | p |
| Major complication | 354 (9.0) | 16 (9.1) | 0.97 | 7 (4.0) | 15 (8.6) | 0.08 |
| Mortality | 77 (2.0) | 5 (2.8) | 0.41 | 1 (0.6) | 5 (2.9) | 0.21 |
| Reoperation | 225 (5.7) | 11 (6.3) | 0.77 | 5 (2.9) | 11 (6.3) | 0.20 |
| Cardiac complication | 94 (2.4) | 3 (1.7) | 0.56 | 1 (0.6) | 3 (1.7) | 0.62 |
| Neurological complication | 22 (0.6) | 1 (0.6) | 0.99 | 1 (0.6) | 1 (0.6) | 1.00 |
| Pulmonary complication | 152 (3.9) | 10 (5.7) | 0.23 | 2 (1.2) | 10 (5.8) | 0.04 |
| Infectious complication | 1001 (25.5) | 59 (33.5) | 0.02 | 43 (24.7) | 57 (32.8) | 0.12 |
| Sepsis | 503 (12.8) | 35 (19.9) | 0.006 | 22 (12.6) | 33 (19.0) | 0.14 |
| Pneumonia | 131 (3.3) | 10 (5.7) | 0.09 | 2 (1.2) | 10 (5.8) | 0.04 |
| Urinary tract infection | 360 (9.2) | 26 (14.8) | 0.01 | 12 (6.9) | 24 (13.8) | 0.04 |
| Surgical site infection (SSI) | 500 (12.7) | 25 (14.2) | 0.56 | 25 (14.4) | 24 (13.8) | 0.88 |
| Organ space SSI | 206 (5.2) | 15 (8.5) | 0.06 | 6 (3.5) | 15 (8.6) | 0.04 |
| Deep incisional SSI | 80 (2.0) | 2 (1.1) | 0.40 | 5 (2.9) | 2 (1.2) | 0.45 |
| Superficial SSI | 239 (6.1) | 10 (5.7) | 0.83 | 14 (8.1) | 9 (5.2) | 0.28 |
| Venous thromboembolism | 194 (4.9) | 16 (9.1) | 0.01 | 11 (6.3) | 16 (9.2) | 0.42 |
| Deep vein thrombosis | 125 (3.2) | 13 (7.4) | 0.002 | 7 (4.0) | 13 (7.5) | 0.25 |
| Pulmonary embolism | 95 (2.4) | 9 (5.1) | 0.03 | 6 (3.5) | 9 (5.2) | 0.60 |
| Bleeding needing transfusion | 1529 (38.9) | 87 (49.4) | 0.005 | 80 (46.0) | 86 (49.4) | 0.59 |
| Prolonged length of stay | 1949 (49.6) | 94 (53.4) | 0.32 | 94 (54.0) | 93 (53.5) | 0.91 |
Regression analyses
In logistic regression analyses accounting for the clustering created by matching and the effect of the type of urinary diversion, patients with disseminated disease were more likely to experience a major complication (odds ratio [OR] 2.50; 95% confidence interval [CI] 1.02–6.11; p=0.045). There was a statistically significantly increased odds of pulmonary complications for those with disseminated cancer (OR 5.17; 95% CI 1.00–26.66; p=0.049). There was no association between the presence of disseminated disease and infectious complications, venous thromboembolism, bleeding requiring transfusion, and prolonged length of stay (Table 3). The type of urinary diversion (continent vs. ileal conduit) did not significantly affect the occurrence of measured complications (p values 0.18–0.92).
Table 3.
Odds ratios of complications following radical cystectomy for patients with disseminated disease as compared to those without disseminated disease
| Odds ratio | 95% CI | p | |
|---|---|---|---|
| Major complications | 2.50 | 1.02–6.11 | 0.045 |
| Pulmonary complications | 5.17 | 1.00–26.66 | 0.049 |
| Infectious complications | 1.55 | 0.97–2.48 | 0.065 |
| Venous thromboembolism | 1.35 | 0.59–3.08 | 0.48 |
| Bleeding needing transfusion | 1.10 | 0.70–1.72 | 0.68 |
| Prolonged length of stay | 0.94 | 0.61–1.45 | 0.79 |
CI: confidence interval.
Discussion
There is growing interest in the use of aggressive, definitive local therapy (cytoreduction) for patients with metastatic solid organ malignancies. Recently, such an approach has been proposed for metastatic bladder cancer;7 however, there are key differences between cystectomy for bladder cancer and cytoreductive nephrectomy for kidney cancer or radical prostatectomy for prostate cancer. First, metastatic bladder cancer is associated with a poor overall prognosis — even among those who received aggressive local therapy (younger and healthier patients), median overall survival is less than 15 months.7 Second, cystectomy is known to be a morbid operation with significant perioperative morbidity and mortality.9 Thus, the role of aggressive local therapy with significant morbidity in patients with relatively limited life expectancy should be scrutinized. In order to further inform this discussion, we sought to examine the perioperative complications of patients undergoing cystectomy for bladder cancer in the setting of disseminated disease. We found an increase in major complications among patients with disseminated disease as compared to those with localized disease (OR 2.50; 95% CI 1.02–6.11; p=0.045). We found an increase in pulmonary complications, although no association with infectious, thromboembolic, or bleeding complications, nor prolonged length of stay.
Patients with disseminated disease who are selected for aggressive local therapy almost certainly differ from those who are not selected and also from patients who undergo local therapy without disseminated disease. In quantifiable terms, they are likely to be younger and healthier;7 however, they likely also differ in unmeasurable ways and residual bias in observational studies of these patients is probably significant, even after adjustments, including propensity score matching or inverse probability of treatment weighting. We found an increased occurrence of major and pulmonary complications (together comprising death, reoperation, cardiac event, neurological event, reintubation, or prolonged ventilation) among a group of patients who would be favoured by residual bias.
Despite the large, multi-institutional nature of the NSQIP registry, we were limited by the small number of patients with disseminated disease undergoing cystectomy. We used propensity score matching in order to be able to provide multivariable adjusted estimates, as there were insufficient events to allow for conventional multivariable logistic regression.
There are many, well-described advantages to the use of NSQIP data, including the contemporary cohort, generalizability of results, and validated outcome and exposure measures; however, there are significant limitations, particularly with respect to oncological surgery. First, NSQIP lacks important oncological information, including stage, histology, and symptomatic burden of disease. Further, the capture of other oncological therapies, including chemotherapy and radiotherapy, is limited to the 90 days preceding the operative date. Despite the intuitive importance of this information, previous work has shown that they do not significantly improve risk prediction for short-term outcomes, such as those we examined here.16 Further, as with an administrative data source, we are unable to ascertain treatment intent. Therefore, we cannot distinguish between cystectomies undertaken for oncological benefit and those undertaken in a palliative setting for symptom control. There are some data of interest that are not available in NSQIP data, including hospital and surgeon volume, academic status, or fellowship training. Finally, there is no accepted or validated methodology to assign Clavien-Dindo grading to complications within NSQIP. Therefore, we provided complications categorized according to severity and organ system.
Conclusion
The stagnant survival rates for patients with metastatic bladder cancer have prompted efforts to identify novel treatments for these patients. Aggressive local therapy is one option; however, given the relatively short life expectancy for patients with metastatic bladder cancer and the morbidity of cystectomy, caution is warranted. Thus, such treatment may best be reserved for a clinical trial in which morbidity, as well as functional and oncological outcomes can be closely monitored.
Footnotes
See related commentary on page 249.
Competing interests: The authors report no competing personal or financial interests.
This paper has been peer-reviewed.
References
- 1.Abdollah F, Gandaglia G, Thuret R, et al. Incidence, survival, and mortality rates of stage-specific bladder cancer in U.S.: A trend analysis. Cancer Epidemiol. 2013;37:219–25. doi: 10.1016/j.canep.2013.02.002. https://doi.org/10.1016/j.canep.2013.02.002. [DOI] [PubMed] [Google Scholar]
- 2.Zehnder P, Studer UE, Skinner EC, et al. Unaltered oncological outcomes of radical cystectomy with extended lymphadenectomy over three decades. BJU Int. 2013;112:E51–8. doi: 10.1111/bju.12215. https://doi.org/10.1111/bju.12215. [DOI] [PubMed] [Google Scholar]
- 3.Flanigan RC, Mickisch G, Sylvester R, et al. Cytoreductive nephrectomy in patients with metastatic renal cancer: A combined analysis. J Urol. 2004;171:1071–6. doi: 10.1097/01.ju.0000110610.61545.ae. https://doi.org/10.1097/01.ju.0000110610.61545.ae. [DOI] [PubMed] [Google Scholar]
- 4.Culp SH, Schellhammer PF, Williams MB. Might men diagnosed with metastatic prostate cancer benefit from definitive treatment of the primary tumour? A SEER-based study. Eur Urol. 2014;65:1058–66. doi: 10.1016/j.eururo.2013.11.012. https://doi.org/10.1016/j.eururo.2013.11.012. [DOI] [PubMed] [Google Scholar]
- 5.Satkunasivam R, Kim AE, Desai M, et al. Radical prostatectomy or external beam radiation therapy vs. no local therapy for survival benefit in metastatic prostate cancer: A SEER-Medicare analysis. J Urol. 2015;194:378–85. doi: 10.1016/j.juro.2015.02.084. https://doi.org/10.1016/j.juro.2015.02.084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Loppenberg B, Dalela D, Karabon P, et al. The impact of local treatment on overall survival in patients with metastatic prostate cancer on diagnosis: A national cancer data base analysis. Eur Urol. 2017;72:14–9. doi: 10.1016/j.eururo.2016.04.031. https://doi.org/10.1016/j.eururo.2016.04.031. [DOI] [PubMed] [Google Scholar]
- 7.Seisen T, Sun M, Leow JJ, et al. Efficacy of high-intensity local treatment for metastatic urothelial carcinoma of the bladder: A propensity score-weighted analysis from the national cancer data base. J Clin Oncol. 2016 doi: 10.1200/JCO.2016.66.7352. [Epub ahead of print] https://doi.org/10.1200/JCO.2016.66.7352. [DOI] [PubMed] [Google Scholar]
- 8.Gakis G, Efstathiou J, Lerner SP, et al. ICUD-EAU International Consultation on Bladder Cancer 2012: Radical cystectomy and bladder preservation for muscle-invasive urothelial carcinoma of the bladder. Eur Urol. 2013;63:45–57. doi: 10.1016/j.eururo.2012.08.009. https://doi.org/10.1016/j.eururo.2012.08.009. [DOI] [PubMed] [Google Scholar]
- 9.Lavallee LT, Schramm D, Witiuk K, et al. Perioperative morbidity associated with radical cystectomy in a multicentre database of community and academic hospitals. PloS One. 2014;9:e111281. doi: 10.1371/journal.pone.0111281. https://doi.org/10.1371/journal.pone.0111281. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Davenport DL, Holsapple CW, Conigliaro J. Assessing surgical quality using administrative and clinical data sets: A direct comparison of the University Health System Consortium Clinical Database and the National Surgical Quality Improvement Program data set. Am J Med Qual. 2009;24:395–402. doi: 10.1177/1062860609339936. https://doi.org/10.1177/1062860609339936. [DOI] [PubMed] [Google Scholar]
- 11.Cima RR, Lackore KA, Nehring SA, et al. How best to measure surgical quality? Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI) and the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) postoperative adverse events at a single institution. Surgery. 2011;150:943–9. doi: 10.1016/j.surg.2011.06.020. https://doi.org/10.1016/j.surg.2011.06.020. [DOI] [PubMed] [Google Scholar]
- 12.Koch CG, Li L, Hixson E, et al. What are the real rates of postoperative complications: Elucidating inconsistencies between administrative and clinical data sources. J Am Coll Surg. 2012;214:798–805. doi: 10.1016/j.jamcollsurg.2011.12.037. https://doi.org/10.1016/j.jamcollsurg.2011.12.037. [DOI] [PubMed] [Google Scholar]
- 13.Benchimol EI, Smeeth L, Guttmann A, et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 2015;12:e1001885. doi: 10.1371/journal.pmed.1001885. https://doi.org/10.1371/journal.pmed.1001885. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wallis CJ, Bjarnason G, Byrne J, et al. Morbidity and mortality of radical nephrectomy for patients with disseminated cancer — an analysis of the National Surgical Quality Improvement Program Database. Urology. 2016;95:95–102. doi: 10.1016/j.urology.2016.04.055. https://doi.org/10.1016/j.urology.2016.04.055. [DOI] [PubMed] [Google Scholar]
- 15.Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin. 1969;72:323–7. https://doi.org/10.1037/h0028106. [Google Scholar]
- 16.Merkow RP, Kmiecik TE, Bentrem DJ, et al. Effect of including cancer-specific variables on models examining short-term outcomes. Cancer. 2013;119:1412–9. doi: 10.1002/cncr.27891. https://doi.org/10.1002/cncr.27891. [DOI] [PubMed] [Google Scholar]
- 17.Austin PC. Using the standardized difference to compare the prevalence of a binary variable between two groups in observational research. Commun Stat Simul Comput. 2009;38:1228–34. https://doi.org/10.1080/03610910902859574. [Google Scholar]