Abstract
Background and Purpose:
Comorbidity indices (CI) are widely used in retrospective studies. We investigated the value of commonly-used CIs in risk adjustment for postoperative complications after colorectal surgery.
Methods:
Patients undergoing colectomy without stoma for colonic neoplasia at a single institution from 2009–2014 were included. Four CIs were calculated or obtained for each patient using administrative data: Charlson-Deyo (CCI-D), Charlson-Romano (CCI-R), Elixhauser Comorbidity Score (ECS) and American Society of Anesthesiology classification (ASA). Outcomes of interest in the 90-day postoperative period were: any surgical complication, surgical site infection (SSI), Clavien-Dindo (CD) grade 3 or higher complication, anastomotic leak or abscess and nonroutine discharge. Base models were created for each outcome based on significant bivariate associations. Logistic regression models were constructed for each outcome using base models alone, and each index as an additional covariate. Models were also compared using the Delong and Clarke-Pearson method for Receiver-Operator Characteristic curves (ROC), with the CCI-D as the reference.
Results:
1813 patients were included. Postoperative complications were reported in 756 (42%) patients. Only 9% of patients had a CD grade 3 or higher complication, and 22.8% of patients developed an SSI. Multivariable modeling showed equivalent performance of the base model and the base model augmented by the CIs for all outcomes. The ROC curves for the 4 indices were also similar.
Discussion and Conclusions:
The inclusion of CIs added little to the base models, and all CIs performed similarly well. Our study suggests that CIs do not adequately risk adjust for complications after colorectal surgery.
Introduction
Risk adjustment is commonly used to control for hospital-, procedure-, and patient-specific confounders when comparing outcomes between institutions and surgeons.1 Several risk calculators have been developed and validated using information from clinical databases such as ACS NSQIP (American College of Surgeons National Surgical Quality Improvement Program).1–6 However, their use to adjust for risk in observational research requires a manual chart review to collect the data, which can be prohibitively cumbersome when studying a large sample population.7
Patient comorbidity data documented in the electronic medical record can be easily queried from administrative databases and used to calculate several validated comorbidity indices. These indices can be used for risk adjustment without the need for a manual chart review. The most widely used of these, the Charlson comorbidity index (CCI), was introduced in 1987 and has been validated as a predictor of 1-year all-cause mortality.8,9 More recently, the CCI has been modified by Deyo et al.10 and Romano et al.11 for use with ICD-9 (International Classification of Diseases, 9th revision) codes from administrative databases. These indices have been validated for a range of outcomes, from in-hospital mortality to readmissions and health-care utilization.10–15
Few studies have addressed the use of comorbidity indices in risk adjustment for surgical endpoints and postoperative complications.16–23 The indices were generally developed and validated for predicting 1-year mortality or in-hospital death, but complications such as wound infections and anastomotic leaks are more common and of greater relevance to colorectal surgery patients. No standardized methods for risk stratification have been developed specifically for these complications of colorectal surgery, although the CCI and other comorbidity indices have been used without validation.18,21
The objective of this study was to determine the value of comorbidity indices, calculated using data from electronic medical records, in risk adjustment for postoperative outcomes that may occur after colorectal surgery.
Methods
Study Population
We conducted a retrospective analysis of colorectal surgery patients treated at Memorial Sloan Kettering Cancer Center from January 1, 2009, to December 31, 2014. Current Procedural Terminology (CPT) codes were used to identify patients who had undergone colon or rectal resection without a stoma. To minimize variability in surgical risk, we excluded patients on the basis of the following criteria: creation of a stoma prior to or at the time of surgery, prior history of colorectal resections, urgent surgery or transfer from another hospital, primary disease process not of colorectal origin (such as ovarian cancer), receiving hyperthermic intraperitoneal chemotherapy or intraoperative radiation therapy, undergoing a concurrent extra-abdominal procedure (such as video-assisted thoracic surgery), and undergoing treatment aimed at palliation. Patients for whom a stoma was created at the time of the index surgery were excluded because the presence of a stoma likely increases the risk for postoperative complications and nonroutine discharge.24 Finally, patients with fewer than 90 days follow-up were excluded.
Patient demographics and operative details were collected from the electronic medical records (EMR) through a query carried out by the institution’s research and technology management division. The variables of interest were age, sex, smoking history (current, ever, never), insurance status (Medicaid/Medicare versus private), preoperative chemotherapy and radiation, previous abdominal surgery, extraction site, AJCC (American Joint Committee on Cancer) stage, BMI, preoperative ASA (American Society of Anesthesiologists) score, surgical procedure, surgical approach (open, robotic or laparoscopic), surgeon, operative time, the use of a stapler for anastomosis, and concurrent procedures (such as liver resection). This study was approved by the Institutional Review Board of Memorial Sloan Kettering Cancer Center.
Complications
Data on complications that occurred up to 90 days after surgery were collected by two of the authors from patient charts. Inpatient notes, outpatient follow-up notes, and any correspondence from outside doctors or hospitals (including radiology reports) were thoroughly examined. Complications were scored according to the Clavien-Dindo classification system by one reviewer and subsequently examined for accuracy by a second reviewer, with quality checks against two prospectively collected institutional databases that track surgical site infections (SSI) and secondary surgical events. Surgical site infections were diagnosed according to CDC-NSQIP criteria. Anastomotic leak was defined as extravasation of oral contrast on imaging, reoperation with confirmed compromise in anastomotic integrity, or documentation of leak by attending physician or radiologist. Abscess was defined as a rim-enhancing collection on imaging. Complications due to adjuvant chemotherapy or other medical treatments unrelated to surgery were deemed non-attributable and were excluded.
The five outcomes of interest in the 90-day period after surgery were any postoperative complication, SSI, a high-grade postoperative complication, a leak or intra-abdominal abscess, and nonroutine discharge of the patient. A high-grade complication was defined as a patient having a grade 3 or higher complication on the Clavien-Dindo (CD) scale. Nonroutine discharge was defined as either discharge requiring home care services or transfer to a rehabilitation center.
Indices of Comorbidity and Physical Status
Comorbidity data were obtained by an electronic search of the hospital’s administrative data. Only comorbidities present in these records within 6 months prior to and including the day of surgery were used for calculating comorbidity indices.
Three comorbidity indices were calculated for each patient: the Deyo modification of CCI (Charlson-Deyo), the Romano modification of CCI (Charlson-Romano), and the van Walraven modification of the Elixhauser comorbidity score.9–11,15 The unmodified CCI was excluded, as it was originally constructed for use with manually collected comorbidity data, while the Deyo and Romano adaptations of CCI are intended specifically for use with administrative data. The Elixhauser comorbidity score was developed from a set of 30 comorbidities defined by ICD-9-CM (ICD-9 Clinical Modification) codes used in administrative data. The van Walraven modification of the Elixhauser comorbidity score adds weighting to all binary elements by using coefficients from a multivariate logistic regression model for in-hospital mortality. In addition to the three comorbidity indices, the ASA physical status score for each subject was collected from anesthesia records. Cancer-related comorbidity categories were removed from comorbidity score calculations and controlled for separately.25
Models and Statistical Analyses
Bivariate associations between patient or treatment variables and the outcomes of interest were examined using simple logistic regression. Covariates with a significant association (P < .05) were included in the base models. A base model was created for each of the five outcomes. The variables surgeon and age of patient were included in all models regardless of significance.
Logistic regression models were constructed for each outcome using the base model alone and augmented base models with one of the three comorbidity indices or ASA score as an additional covariate. The predictive power of the augmented models was compared with that of the base model by using c-statistics. A difference of >0.02 in predictive power was considered significant.8
The Clarke-Pearson method developed by Delong et al26 was used to compare the unadjusted predictive power of the Charlson-Deyo score to those of the other two comorbidity indices and the ASA score. Receiver-operating characteristic curves were created for each outcome, with the Charlson-Deyo score as the reference, since it is the most widely used comorbidity index.12
Results
A total of 1813 patients met the inclusion criteria. The median patient age at the time of surgery was 62 years, and 50% of the patients had AJCC stage II or III adenocarcinoma of the colon (Table 1). Over the 6-year period of the study, seven surgeons performed a median of 222 colectomies each. Approximately 38% of the colectomies were done with an open approach, and 62% were performed with a minimally invasive approach.
Table 1.
Patient and Treatment Characteristics
| Characteristic | No. of Patients (%) [n = 1813] |
|---|---|
| Age, median (IQR) | 61.7 y (50.8–72.1) |
| Sex | |
| Female | 928 (51) |
| Male | 883 (49) |
| Race or ethnicity, n (%) | |
| Hispanic | 87 (5) |
| Black | 110 (6) |
| Caucasian | 1485 (82) |
| Other | 129 (7) |
| BMI, median (IQR) | 27.9 (24.2–31.7) |
| Insurance, n (%) | |
| Medicare/Medicaid | 847 (47) |
| Private | 966 (53) |
| Disease stage, n (%) | |
| In situ/Benign | 247 (14) |
| I | 357 (20) |
| II | 476 (26) |
| III | 433 (24) |
| IV | 300 (17) |
| Smoking history*, n (%) | |
| Current | 190 (11) |
| Ever | 618(34) |
| Never | 1003 (55) |
| Preoperative chemotherapy, n (%) | 245 (14) |
| Preoperative radiation therapy, n (%) | 26 (1) |
| Previous abdominal surgery, n (%) | 849 (47) |
| Surgical Procedure, n (%) | |
| Right colectomy | 799 (44) |
| Left and Sigmoid colectomies | 402 (22) |
| Low anterior resection | 529 (29) |
| Subtotal colectomy | 83 (5) |
| Surgical approach, n (%) | |
| Open | 693 (38) |
| Laparoscopic | 852 (47) |
| Robotic | 268 (15) |
| Tumor type, n (%) | |
| Adenocarcinoma | 1541 (85) |
| Adenoma | 120 (7) |
| Benign | 77 (4) |
| Other Neoplasiaa | 75 (4) |
| Extraction site, n (%) | |
| Vertical midline | 1614 (89) |
| Other | 199 (11) |
| Comorbidity rate by index, mean (range) | |
| Charlson-Deyo | 1.26 (0–9) |
| Charlson-Romano | 0.63 (0–9) |
| van Walraven | 3.33 (-11–34) |
| ASA | 2.64 (1–4) |
Neuroendocrine tumor, leiomyoma, mucinous cystadenoma, gastrointestinal stromal tumor, or schwannoma.
Two patients have unknown smoking history.
Depending on the index, 38–85% of the patients had at least one recorded comorbidity aside from their cancer-associated diagnoses (Table 1). On the basis of the Charlson-Deyo and Charlson-Romano indices, 73% and 94% of the patients, respectively, had fewer than three comorbidities.
The rates of postoperative complications are shown in Table 2. The overall complication rate within 90 days of surgery was 42%. Four (<1%) patients died within 30 days after surgery, and a total of nine (<1%) patients died within 90 days after surgery.
Table 2.
Postoperative Complications
| Complication | No. of Patients (%) [n = 1813] |
|---|---|
| Any | 756 (42) |
| High grade | 164 (9) |
| Surgical site infectiona | 414 (23) |
| Superficial | 258 (14) |
| Deep | 40 (2) |
| Organ Space | 116 (6) |
| Anastomotic leak/intra-abdominal abscess | 113 (6) |
| Nonroutine discharge | 582 (32) |
Categorized according to the guidelines of the Centers for Disease Control and Prevention.
Surgical approach, tumor stage, and preoperative chemotherapy had significant (P < .05) associations with all five outcomes and were included in the five base models (Table 3). Insurance status, BMI, smoking history, and specimen extraction site were included in the base models for any complication, SSI, and nonroutine discharge. Year of surgery and patient’s sex were included in the model for SSI. History of abdominal surgery was included in the model for nonroutine discharge. Surgical procedure was included in the base models for any complication, high-grade complication, and anastomotic leak or intra-abdominal abscess.
Table 3.
Variables Included in the Base Models
| Any Complication | High-Grade Complication | Surgical Site Infection | Leak/Intra-abdominal Abscess | Nonroutine Discharge |
|---|---|---|---|---|
| Age* | Age | Age | Age | Age* |
| Surgeon* | Surgeon | Surgeon | Surgeon | Surgeon |
| Stage | Stage* | Stage* | Stage* | Stage* |
| Surgical Procedure* | Surgical Procedure | Surgical Procedure* | ||
| Surgical approach* | Surgical approach | Surgical approach* | Surgical approach | Surgical approach* |
| Preop chemo*a | Preop chemoa | Preop chemoa | Preop chemoa | Preop chemoa |
| BMI* | BMI* | BMI* | ||
| Insurance type* | Insurance type* | Insurance type* | ||
| Smoking history | Smoking history | Smoking history | ||
| Extraction site | Extraction site | Extraction site | ||
| Year of surgery | Prior abdominal surgery | |||
| Sex* |
Preoperative Chemotherapy.
Remained significant in multivariable analysis
Several base model covariates remained significant in multivariable analysis. Patients with private insurance had a lower risk of any complication, SSI, or nonroutine discharge (odds ratio [OR], 0.63, 0.69, and 0.63, respectively) when compared to those with Medicare or Medicaid. Patients who had preoperative chemotherapy had a higher risk of any complication as compared to those with no preoperative chemotherapy (OR 1.6, 95% CI: 1.1–2.3). Laparoscopic and robotic surgical approaches were associated with a lower risk of any complication, nonroutine discharge and SSI compared to open surgery. Patients undergoing surgery with stage III or stage IV disease had a significantly higher risk of high-grade complications compared to those with stage in situ or benign disease (OR 2.70 and 2.76, respectively). Patients undergoing left and sigmoid colectomies had a lower risk of anastomotic leak or intra-abdominal abscess compared to those undergoing low anterior resections (OR 0.45, 95% CI:0.23–0.86). Those who had subtotal colectomies or right colectomies were more likely to have any complication compared to those who had low anterior resections (OR 2.19 95% CI: 1.27–3.77; OR 1.41, 95% CI: 1.08–1.84, respectively). Table 4 shows the results of the primary analysis comparing the c-statistics of the base models alone with the c-statistics of the augmented models. A c-statistic of 0.50 signifies that a model is no better at predicting an outcome than random chance. The c-statistics for the base models ranged from 0.686 for the outcome of any complication to 0.759 for the outcome of nonroutine discharge. In the primary analyses for all outcomes, the c-statistics of the base models did not differ significantly from the c-statistics for the augmented models.
Table 4.
C-Statistics of the Base Models and Augmented Models
| Model | C-Statistic | ||||
|---|---|---|---|---|---|
| Any Complication | High-Grade Complication | Surgical Site Infection | Leak/Intra abdominal Abscess | Nonroutine Discharge | |
| Base model | 0.69 | 0.70 | 0.72 | 0.71 | 0.76 |
| Augmented models | |||||
| Charlson-Deyo | 0.69 | 0.71 | 0.72 | 0.72 | 0.76 |
| Charlson-Romano | 0.70 | 0.70 | 0.72 | 0.72 | 0.76 |
| van Walraven | 0.70 | 0.70 | 0.72 | 0.72 | 0.76 |
| ASA | 0.70 | 0.70 | 0.72 | 0.71 | 0.76 |
Figure 1 shows the secondary analyses using the Clarke-Pearson method for the five outcomes. In all cases, the Charlson-Romano index, van Walraven index, and ASA score did not differ significantly from the Charlson-Deyo score (the reference). In addition, the calculated areas under the curve showed poor predictive power between the indices and all outcomes. Models are typically considered reasonable when the c-statistic is higher than 0.7 and strong when the c-statistic exceeds 0.8.
Figure 1.
ROC Curve Analyses for the Five Outcomes
Discussion
Our base models had acceptable predictive powers (c-statistic, 0.69–0.76) for all five outcomes. The addition of the individual comorbidity indices added no predictive power to the base models. The three comorbidity indices performed equally poorly when modeled individually apart from the base models, and the performance of each was equivalent to that of the ASA score. The c-statistics of the comorbidity indices and ASA score were < 0.65 for all five outcomes in univariate analyses.
Similar results have been reported by Dekker et al.,27 who used the Leiden Cancer Registry (a subset of the Netherlands Cancer Registry) to compare the CCI, sum of diseased organ systems, and ASA score in risk adjustment for postoperative mortality, prolonged (>14-day) length of stay, and any surgical complication among 2204 colorectal cancer patients. Those authors found that no single measure of comorbidity or physical status examined outperformed the other measures. Our study confirmed those of Dekker et al, while further evaluating the most commonly-used CIs derived from administrative indices. The findings of our study are applicable to researchers in any institution, and especially those who do not have access to a preexisting database.
Several studies have evaluated the use of comorbidity indices for risk stratification in relation to morbidity and mortality in surgery patients.18,19,23,28 Depending on the study population and outcome of interest, the predictive abilities ranged from poor to excellent. Few studies have focused on specific short-term surgical outcomes. In one such study, Tan et al.18 compared the CCI and ASA score in predicting anastomotic leak after colorectal surgery. Those authors found an association between higher ASA or CCI scores and anastomotic leak, but they did not calculate c-statistics to demonstrate the validity of using these scores for risk adjustment. An association between a comorbidity or physical status score and a specific outcome does not necessarily indicate that inclusion of that score would improve risk stratification. Our analyses specifically address this issue by calculating c-statistics, which indicate that the addition of comorbidity indices provides minimal additional predictive power.
The indices’ lack of predictive power in our study may be due to several factors. First, and most importantly, none of the indices examined were designed to be used in risk adjustment for surgical complications. Since they were originally designed for other endpoints of interest (such as 1-year mortality after hospitalization), they include comorbidities that may have no impact on the risk of postoperative complications. Finally, a considerable proportion (15–62%) of the patients had no comorbidities aside from their cancer-associated diagnoses.
We confirmed an association of higher BMI with any complication, risk of SSI and nonroutine discharge.29–31 In addition to confirming the previously described association between minimally invasive surgery and a lower likelihood of SSI, our models showed that minimally invasive surgery was associated with a lower likelihood of nonroutine discharge.32 Since our study was not aimed at validating these risk factors, their specific effects need to be further investigated in a dedicated analysis.
The limitations of our study include its retrospective nature and potential for missed complications. We captured as many complications of interest as possible by checking the data against two prospective institutional databases as well as all patient correspondence and outside medical records. Another potential limitation lies in the nature of administrative databases themselves. Previous studies have produced conflicting results on concordance between comorbidity indices derived from administrative data and those derived from manually collected data.12,33 We used administrative data because of its increasing use in surgical outcomes research since it minimizes the need for manual chart review. Lastly, since the study was conducted at a comprehensive cancer center, where patients may be more likely to have advanced disease or undergo complex treatment, the high risk associated with the disease itself may have diluted the contribution of the indices in predicting outcomes. However, our study’s distribution of patients by disease stage closely matched the distribution reported by the American Joint Committee on Cancer for the general population of colorectal cancer patients.34
One of the strengths of this study lies in the fact that it was conducted at a single institution, allowing for thorough chart review and manual collection of data on common postoperative complications. Another important strength is the composition of the patient population investigated. The average age in our cohort was 62 years, which is significantly younger than in many studies of comorbidity indices. Many of the patients had no comorbidities apart from cancer-associated diagnoses, and 93% of the patients had fewer than three comorbidities. Given the increasing incidence of colorectal cancer among younger patients, the findings of our study are particularly relevant to the changing population of patients who undergo colorectal cancer surgery.35
Conclusion
The inclusion of the Charlson-Deyo, Charlson-Romano, van Walraven adaptation and ASA indices did not improve the ability of base models to predict postoperative complications in colorectal surgery patients, nor did any index perform better than the others. Inclusion of these comorbidity indices alone does not guarantee adequate risk adjustment in colorectal surgery patients, and it is likely that patient and treatment data from the electronic medical records may suffice.
Synopsis:
We investigated the value of commonly used comorbidity indices for predicting complications after colorectal surgery. Adding these indices to models constructed from patient and treatment variables did not improve risk adjustment for surgical outcomes.
Funding:
NCI grant P30 CA008748.
Footnotes
Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.
The findings of this study were presented as an E-Poster of Distinction at the American Society of Colon and Rectal Surgeons in Los Angeles, April 30 to May 4, 2016.
REFERENCES
- 1.Cohen ME, Ko CY, Bilimoria KY, et al. Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. Journal of the American College of Surgeons. 2013;217(2):336–346 e331. [DOI] [PubMed] [Google Scholar]
- 2.Bilimoria KY, Liu Y, Paruch JL, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. Journal of the American College of Surgeons. 2013;217(5):833–842 e831–833. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Martocchia A, Frugoni P, Indiano I, et al. Screening of frailty in elderly patients with disability by the means of Marigliano-Cacciafesta polypathology scale (MCPS) and Canadian Study of Health and Aging (CSHA) scales. Archives of gerontology and geriatrics. 2013;56(2):339–342. [DOI] [PubMed] [Google Scholar]
- 4.Rockwood K, Song X, MacKnight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne. 2005;173(5):489–495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gupta PK, Franck C, Miller WJ, Gupta H, Forse RA. Development and validation of a bariatric surgery morbidity risk calculator using the prospective, multicenter NSQIP dataset. Journal of the American College of Surgeons. 2011;212(3):301–309. [DOI] [PubMed] [Google Scholar]
- 6.Gupta PK, Gupta H, Sundaram A, et al. Development and validation of a risk calculator for prediction of cardiac risk after surgery. Circulation. 2011;124(4):381–387. [DOI] [PubMed] [Google Scholar]
- 7.Anderson JE C D. Using Electronic Health Records for Surgical Quality Improvement in the Era of Big Data. JAMA Surgery. 2015;150:24–29. [DOI] [PubMed] [Google Scholar]
- 8.Sharabiani MT, Aylin P, Bottle A. Systematic review of comorbidity indices for administrative data. Medical care. 2012;50(12):1109–1118. [DOI] [PubMed] [Google Scholar]
- 9.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. Journal of chronic diseases. 1987;40(5):373–383. [DOI] [PubMed] [Google Scholar]
- 10.Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. Journal of clinical epidemiology. 1992;45(6):613–619. [DOI] [PubMed] [Google Scholar]
- 11.Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. Journal of clinical epidemiology. 1993;46(10):1075–1079; discussion 1081–1090. [DOI] [PubMed] [Google Scholar]
- 12.Yurkovich M, Avina-Zubieta JA, Thomas J, Gorenchtein M, Lacaille D. A systematic review identifies valid comorbidity indices derived from administrative health data. Journal of clinical epidemiology. 2015;68(1):3–14. [DOI] [PubMed] [Google Scholar]
- 13.Holman CD, Preen DB, Baynham NJ, Finn JC, Semmens JB. A multipurpose comorbidity scoring system performed better than the Charlson index. Journal of clinical epidemiology. 2005;58(10):1006–1014. [DOI] [PubMed] [Google Scholar]
- 14.Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Medical care. 1998;36(1):8–27. [DOI] [PubMed] [Google Scholar]
- 15.van Walraven C, Austin PC, Jennings A, Quan H, Forster AJ. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Medical care. 2009;47(6):626–633. [DOI] [PubMed] [Google Scholar]
- 16.Schneider EB, Hyder O, Wolfgang CL, et al. Patient readmission and mortality after surgery for hepato-pancreato-biliary malignancies. Journal of the American College of Surgeons. 2012;215(5):607–615. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Stukenborg GJ, Wagner DP, Connors AF Jr. Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations. Medical care. 2001;39(7):727–739. [DOI] [PubMed] [Google Scholar]
- 18.Tan WP, Talbott VA, Leong QQ, Isenberg GA, Goldstein SD. American Society of Anesthesiologists class and Charlson’s comorbidity index as predictors of postoperative colorectal anastomotic leak: a single-institution experience. The Journal of surgical research. 2013;184(1):115–119. [DOI] [PubMed] [Google Scholar]
- 19.Lieffers JR, Baracos VE, Winget M, Fassbender K. A comparison of Charlson and Elixhauser comorbidity measures to predict colorectal cancer survival using administrative health data. Cancer. 2011;117(9):1957–1965. [DOI] [PubMed] [Google Scholar]
- 20.Mayo SC, Gilson MM, Herman JM, et al. Management of patients with pancreatic adenocarcinoma: national trends in patient selection, operative management, and use of adjuvant therapy. Journal of the American College of Surgeons. 2012;214(1):33–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Menendez ME, Neuhaus V, van Dijk CN, Ring D. The Elixhauser comorbidity method outperforms the Charlson index in predicting inpatient death after orthopaedic surgery. Clinical orthopaedics and related research. 2014;472(9):2878–2886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Shin JH, Worni M, Castleberry AW, et al. The application of comorbidity indices to predict early postoperative outcomes after laparoscopic Roux-en-Y gastric bypass: a nationwide comparative analysis of over 70,000 cases. Obesity surgery. 2013;23(5):638–649. [DOI] [PubMed] [Google Scholar]
- 23.Tan KY, Kawamura Y, Mizokami K, et al. Colorectal surgery in octogenarian patients--outcomes and predictors of morbidity. International journal of colorectal disease. 2009;24(2):185–189. [DOI] [PubMed] [Google Scholar]
- 24.Sheetz KH, Waits SA, Krell RW, et al. Complication rates of ostomy surgery are high and vary significantly between hospitals. Diseases of the colon and rectum. 2014;57(5):632–637. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Dobbins TA, Creighton N, Currow DC, Young JM. Look back for the Charlson Index did not improve risk adjustment of cancer surgical outcomes. Journal of clinical epidemiology. 2015;68(4):379–386. [DOI] [PubMed] [Google Scholar]
- 26.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845. [PubMed] [Google Scholar]
- 27.Dekker JW, Gooiker GA, van der Geest LG, et al. Use of different comorbidity scores for risk-adjustment in the evaluation of quality of colorectal cancer surgery: does it matter? European journal of surgical oncology : the journal of the European Society of Surgical Oncology and the British Association of Surgical Oncology. 2012;38(11):1071–1078. [DOI] [PubMed] [Google Scholar]
- 28.Thombs BD, Singh VA, Halonen J, Diallo A, Milner SM. The effects of preexisting medical comorbidities on mortality and length of hospital stay in acute burn injury: evidence from a national sample of 31,338 adult patients. Annals of surgery. 2007;245(4):629–634. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Kirchhoff P, Clavien PA, Hahnloser D. Complications in colorectal surgery: risk factors and preventive strategies. Patient safety in surgery. 2010;4(1):5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Sharma A, Deeb AP, Iannuzzi JC, Rickles AS, Monson JR, Fleming FJ. Tobacco smoking and postoperative outcomes after colorectal surgery. Annals of surgery. 2013;258(2):296–300. [DOI] [PubMed] [Google Scholar]
- 31.Nordlinger B, Sorbye H, Glimelius B, et al. Perioperative chemotherapy with FOLFOX4 and surgery versus surgery alone for resectable liver metastases from colorectal cancer (EORTC Intergroup trial 40983): a randomised controlled trial. Lancet. 2008;371(9617):1007–1016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Gandaglia G, Ghani KR, Sood A, et al. Effect of minimally invasive surgery on the risk for surgical site infections: results from the National Surgical Quality Improvement Program (NSQIP) Database. JAMA Surg. 2014;149(10):1039–1044. [DOI] [PubMed] [Google Scholar]
- 33.Quan H, Parsons GA, Ghali WA. Validity of information on comorbidity derived rom ICD-9-CCM administrative data. Medical care. 2002;40(8):675–685. [DOI] [PubMed] [Google Scholar]
- 34.Edge SB, Compton CC. The American Joint Committee on Cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM. Annals of surgical oncology. 2010;17(6):1471–1474. [DOI] [PubMed] [Google Scholar]
- 35.Bailey CE, Hu CY, You YN, et al. Increasing disparities in the age-related incidences of colon and rectal cancers in the United States, 1975–2010. JAMA Surg. 2015;150(1):17–22. [DOI] [PMC free article] [PubMed] [Google Scholar]