Abstract
Background:
Physiology-based severity of illness scores are often used for risk adjustment in observational studies of intensive care unit (ICU) outcome. However, the complexity and time constraints of these scoring systems may limit their use in administrative databases. Comorbidity is a main determinant of ICU outcome, and comorbidity scores can be computed based on data from most administrative databases. However, limited data exist on the performance of comorbidity scores in predicting mortality of ICU patients.
Objectives:
To examine the performance of the Charlson comorbidity index (CCI) alone and in combination with other readily available administrative data and three physiology-based scores (acute physiology and chronic health evaluations [APACHE] II, simplified acute physiology score [SAPS] II, and SAPS III) in predicting short- and long-term mortality following intensive care.
Methods:
For all adult patients (n = 469) admitted to a tertiary university–affiliated ICU in 2007, we computed APACHE II, SAPS II, and SAPS III scores based on data from medical records. Data on CCI score age and gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy, primary diagnosis, and complete follow-up for 1-year mortality was obtained from administrative databases. We computed goodness-of-fit statistics and c-statistics (area under ROC [receiver operating characteristic] curve) as measures of model calibration (ability to predict mortality proportions over classes of risk) and discrimination (ability to discriminate among the patients who will die or survive), respectively.
Results:
Goodness-of-fit statistics supported model fit for in-hospital, 30-day, and 1-year mortality of all combinations of the CCI score. Combining the CCI score with other administrative data revealed c-statistics of 0.75 (95% confidence interval [CI] 0.69–0.81) for in-hospital mortality, 0.75 (95% CI 0.70–0.80) for 30-day mortality, and 0.72 (95% CI 0.68–0.77) for 1-year mortality. There were no major differences in c-statistics between physiology-based systems and the CCI combined with other administrative data.
Conclusion:
The CCI combined with administrative data predict short- and long-term mortality for ICU patients as well as physiology-based scores.
Keywords: epidemiology, CCI, physiology-based scores, ICU
Introduction
Within intensive care medicine, limited evidence is available from randomized controlled trials (RCTs).1 The heterogeneity and complex clinical course of intensive care unit (ICU) patients makes it difficult to standardize treatment as required in RCTs and to maintain randomized assignments. Therefore, large health care databases are increasingly used to study outcomes of ICU interventions.1–6
The validity of observational studies on ICU outcomes depends, among other factors, on appropriate risk adjustment for case-mix differences. A number of physiology-based severity of illness scoring systems have been developed to adjust for case mix in observational studies of ICU outcomes, such as the simplified acute physiology scores (SAPS),7 the acute physiology and chronic health evaluations (APACHE),8 and the mortality probability models (MPM).9 These systems were developed to estimate the probability of in-hospital mortality based on a summary score consisting of measures of physiological derangement in combination with demographics, comorbidity, and reason for ICU admission. In contrast to older models (SAPS II and APACHE II), newer models (SAPS III) achieve a substantially larger part of the predictive power, with data available before ICU admission, including comorbidity and reason for ICU admission. However, the complexity and time constraints of the physiology-based scoring systems limit their use in large population-based databases.
Comorbidity has been shown to be an important determinant of ICU outcome.10–12 This is underlined by the increasing weight ascribed to comorbidity in the most recent developed severity-of-illness models, such as the SAPS III model. The Charlson comorbidity index (CCI),13,14 developed to predict 1-year mortality among medical patients, is one of the most frequently used measures of comorbidity.15 However, only four studies examined the performance of comorbidity scores in predicting in-hospital mortality in ICU patients.11,16–18 The studies either did not combine the comorbidity score with other readily available administrative data11,18 or included special study populations, such as patients from Veterans Affairs hospitals.11 A recent Australian study examined the performance of the CCI in combination with APACHE II data to predict long-term mortality of ICU patients.12 No data for the CCI exclusively was presented.
We therefore examined the performance of the CCI alone and in combination with other readily available administrative data and three physiology-based scoring systems (APACHE II, SAPS II, and SAPS III) in predicting short-term mortality (in-hospital and 30-day) and long-term mortality (1-year) in a cohort of Danish ICU patients.
Methods
The study population included all patients older than 15 years admitted to the ICU at Aarhus University Hospital, Skejby, Denmark, between January 1, 2007 and December 31, 2007. We included only each patient’s first ICU admission during the study period. We did not include in the cohort coronary care patients, or cardiac surgery patients and other patients admitted for planned postoperative observation of less than 24 hours as defined in the original physiology-based models.7,9,19 The 14-bed facility is a highly specialized university-affiliated surgical/medical tertiary unit serving as both a primary and referral ICU. The nurse to patient ratio is 1:1. Patients are admitted from departments of thoracic surgery or cardiology, as well as from departments of infectious diseases, gynecology and obstetrics, nephrology, and urology. Its patients include those with severe respiratory insufficiency requiring extracorporeal membrane oxygenation and patients undergoing organ transplantation.
The Danish national health care system provides the entire Danish population with unrestricted access to tax-supported public health services. In Denmark, all critically ill patients receive care in public hospitals.
Since 1968, every Danish citizen has received at birth a unique civil registration number that encodes age, gender, and date of birth. This number is included in all Danish registries and permits accurate linkage among registries.
Comorbidity
We obtained data on comorbidity from the Danish National Registry of Patients (DNRP). For all hospital admissions to Danish acute care hospitals since 1977 and, since 1995, for all hospital outpatient and emergency room visits (more than 99% complete), the DNRP has recorded the patients’ civil registration numbers, dates of admission and discharge, up to 20 surgical procedures, and up to 20 discharge diagnoses, classified according to the International Classification of Diseases, eighth revision (ICD-8), until the end of 1993 and tenth revision (ICD-10) thereafter.20 All hospital diagnoses were coded by the physician treating the patient at the time of hospital discharge. Since 2005, treatments such as mechanical ventilation and renal replacement therapy have also been registered.
We used the CCI to estimate comorbidity levels among study patients.13 In calculating the CCI, a weight (1–6) is assigned to each of 19 comorbid disease categories, and the score is the sum of these weights. We used a version of the Deyo ICD-9 adaptation of the CCI,15 modified for use with ICD-8 and ICD-10 discharge codes (see Appendix 1). Using the DNRP, we identified all study patients’ post-1977 hospital diagnoses registered before the date of ICU admission. In a sensitivity analysis, we restricted comorbidity information to the 5 years prior to ICU admission. The score was included as a continuous covariate in the analysis. The comorbid diseases and predefined weights in the CCI may not necessarily be the optimal choice for an ICU population. We therefore also conducted analyses, including ten separate clinically relevant comorbidity categories based on the original 19 comorbid disease categories in the CCI (see Appendix).
Other administrative data
We used the DNRP to identify the primary hospital diagnosis for all hospital stays that included ICU care and grouped patients into eight diagnostic categories: (1) infectious diseases; (2) endocrinology, including diabetes; (3) cardiovascular diseases; (4) respiratory diseases; (5) gastrointestinal and liver diseases; (6) cancer; (7) trauma and poisoning; and (8) others (details on ICD codes are provided in the Appendix). Using the DNRP, we also obtained information on surgical procedures performed on all study patients. We defined surgical patients as patients who underwent surgery on the day of ICU admission or within 7 days before that date and medical patients as those who had no surgery within 7 days before ICU admission.7 Data on mechanical ventilation and renal replacement therapy in the ICU were obtained from the DNRP. Through the Civil Registration System, we obtained data on marital status (married, divorced, widowed, never married, or unknown) and urbanization (city, town, or provincial town) as measures of social status.21
Physiology-based scores
To obtain data on physiology-based scores (SAPS II, SAPS III, APACHE II), two reviewers (SC, CFC) reviewed all medical records for study patients (for details on the scores, see Appendix Table C). Data on physiological variables were obtained from a computerized patient-data management system (Picis Critical Care Manager, Picis Inc, Wakefield, MA) that prospectively collects a wide range of clinical information, including detailed data on mechanical ventilation, body temperature, blood pressure, and heart rate. From a computerized laboratory database, we obtained the laboratory data included in the physiology-based scores (hemoglobin, white blood cell count, creatinine, and urea). All clinical data were reviewed; this allowed us to avoid including invalid data from the computerized databases, for example, incorrect blood pressure measured during sampling blood from an arterial line. Data on reason for ICU admission were obtained from medical records. We used the original definitions of all variables in the APACHE II, SAPS II, and SAPS III systems. The Glasgow Coma Scale (GCS) score is difficult to assess based on review of medical records.22 We assumed a normal GCS score (=15) if the GCS score was not described in the medical record or if the patient was sedated upon ICU admission. While the study design called for the reviewers to be blinded to the study endpoint, in practice it was not possible to obtain perfect blinding for in-hospital mortality.
Record linkage and mortality
Since 1968, every Danish citizen has received at birth a unique personal identifier, a civil registration number encoding gender and date of birth. This number is included in all Danish registries and permits accurate linkage among registries.21 The Danish Civil Registration System provides information on vital status and residence for the entire Danish population, updated daily since 1968. Using this database, we were able to track the study outcomes – that is, in-hospital mortality, 30-day mortality, and 1-year mortality – following the date of first ICU admission during the study period.
Statistical analysis
We used medians and interquartile ranges (IQRs) to describe the distribution of scores within the cohort of ICU patients. We computed Spearman’s rank correlation coefficient for correlation among scores.
For each endpoint (in-hospital, 30-day, and 1-year mortality) logistic regression models were constructed for the following eleven combinations of variables: (1) CCI score; (2) CCI, age, gender; (3) CCI, age, gender, surgical/medical status; (4) CCI, age, gender, surgical/medical status, social factors; (5) CCI, age, gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy; (6) CCI, age, gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy, primary diagnosis; (7) ten separate comorbidity groups; (8) ten comorbidity groups in combination with age, gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy, primary diagnosis; (9) SAPS II; (10) SAPS III; and (11) APACHE II.
We used Hosmer–Lemeshow goodness-of-fit statistics to assess model calibration except for the model including CCI only and the model including the ten separate comorbidity groups only. In these two models we used Pearson’s chi-square statistics because of the limited number of observed and expected deaths in the higher deciles of risk.
For all models, c-statistics (area under ROC [receiver operating characteristic] curve) were calculated as a measure of a model’s ability to discriminate between survivors and nonsurvivors. c-statistics range from 0 to 1, with 1 indicating perfect discrimination and 0.5 indicating a chance discrimination; a c-statistic between 0.7 and 0.8 is considered a good discrimination, and a c-statistic >0.8 an excellent discrimination.23
To assess the impact of length of follow-up for comorbidity included in the CCI, we reran the analysis including comorbidity information from the 5 years prior to ICU admission only.
To assess whether colinearity may have influenced our analysis, we reran our models using multiple linear regression with a binary dependent variable. While this model is clearly not a correct one and violates key assumptions, it allows us to run diagnostic tests for colinearity among the independent variables. These measures (VIF [variance inflation factor] and its reciprocal, tolerance) were examined and showed no indication that colinearity is a factor in our multivariable analyses (data not shown).
Results
We identified 469 adult ICU patients during the study period. The majority of patients (66.9%) were older than 60 years, and 298 (63.5%) patients had surgery within 7 days before ICU admission (Table 1). Cardiovascular diagnoses such as ischemic heart diseases were the primary diagnoses for the majority of patients (58.9%). Median CCI score was 2 (IQR 1–3), and median score was 16 (IQR 11–21) for APACHE II, 36 (IQR 26–47) for SAPS II, and 57 (IQR 45–66) for SAPS III.
Table 1.
Characteristics of 469 adult ICU patients, Aarhus University Hospital, Skejby, Denmark, 2007
| Characteristic | Number | % |
|---|---|---|
| Gender | ||
| Women | 158 | 33.7% |
| Men | 311 | 66.3% |
| Age group | ||
| 15–45 | 47 | 10.0% |
| 46–60 | 108 | 23.0% |
| 61–75 | 192 | 40.9% |
| 76+ | 122 | 26.0% |
| Surgical/Medical statusa | ||
| Medical | 171 | 36.5% |
| Surgical | 298 | 63.5% |
| Primary diagnosis | ||
| Infectious disease | 18 | 3.8% |
| Cancer | 40 | 8.5% |
| Diabetes | 2 | 0.4% |
| Cardiovascular disease | 276 | 58.9% |
| Respiratory disease | 40 | 8.5% |
| Gastrointestinal disease | 11 | 2.4% |
| Trauma/poisoning | 5 | 1.1% |
| Other | 77 | 16.4% |
| Treatment | ||
| Mechanical ventilation | 320 | 68.2% |
| Renal replacement therapy | 126 | 26.9% |
| Marital status | ||
| Married | 275 | 58.6% |
| Never married | 59 | 12.6% |
| Divorced | 49 | 10.5% |
| Widow(er) | 84 | 17.9% |
| Unknown | 2 | 0.4% |
| Urbanization | ||
| Provincial town | 186 | 39.7% |
| Town | 153 | 32.6% |
| City | 130 | 27.7% |
Note:
Surgery within 7 days before ICU admission.
Abbreviation: ICU, intensive care unit.
Correlation
The correlation was poor between CCI score and SAPS scores and only slightly better for the CCI and APACHE II scores (Table 2). The correlation among the physiology-based scores was moderate to high.
Table 2.
Correlation among scoring systems (Spearman’s rank coefficient)
| SAPS II | SAPS III | APACHE II | |
|---|---|---|---|
| Charlson score | 0.124 (P = 0.0074) | 0.082 (P = 0.0750) | 0.228 (P < 0.0001) |
| SAPS II | – | 0.691 (P < 0.0001) | 0.770 (P < 0.0001) |
| SAPS III | – | 0.659 (P < 0.0001) |
Abbreviations: APACHE, acute physiology and chronic health evaluations; SAPS, simplified acute physiology score.
Calibration
The Hosmer–Lemeshow goodness-of-fit statistics supported model fit of all models for in-hospital, 30-day, and 1-year mortality, as indicated by P-values being above 0.05 (Table 3).
Table 3.
Model calibration assessed by Hosmer–Lemeshow goodness-of-fit statistics and Pearson’s chi-square
| Score/measure |
In-hospital mortality |
30-day mortality |
1-year mortality |
||||||
|---|---|---|---|---|---|---|---|---|---|
| H-L goodness-of-fit* | DF | P-value | H-L goodness-of-fit* | DF | P-value | H-L goodness-of-fit* | DF | P-value | |
| Charlson score* | 14.65* | 9 | 0.10 | 8.97* | 9 | 0.44 | 15.61* | 9 | 0.08 |
| Charlson score, age and gender | 8.92 | 8 | 0.35 | 6.91 | 8 | 0.55 | 8.06 | 8 | 0.43 |
| Charlson score, age and gender, surgical/medical status | 7.40 | 8 | 0.49 | 11.66 | 8 | 0.17 | 9.13 | 8 | 0.33 |
| Charlson score, age and gender, surgical/medical status, social factors | 6.26 | 8 | 0.62 | 2.69 | 8 | 0.95 | 4.10 | 8 | 0.85 |
| Charlson score, age and gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy | 13.80 | 8 | 0.09 | 10.23 | 8 | 0.25 | 12.12 | 8 | 0.15 |
| Charlson score, age and gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy, primary diagnosis | 12.26 | 8 | 0.14 | 9.07 | 8 | 0.34 | 10.48 | 8 | 0.23 |
| Ten separate comorbidity groups* | 85.30* | 92 | 0.68 | 85.49* | 92 | 0.67 | 106.12* | 92 | 0.15 |
| Ten comorbidity groups, with age, gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy, primary diagnosis | 7.79 | 8 | 0.45 | 8.89 | 8 | 0.35 | 7.38 | 8 | 0.50 |
| SAPS II | 4.37 | 10 | 0.93 | 3.49 | 10 | 0.97 | 1.58 | 10 | 1.00 |
| SAPS III | 9.23 | 10 | 0.51 | 10.89 | 10 | 0.37 | 11.55 | 10 | 0.32 |
| APACHE II | 13.66 | 10 | 0.19 | 12.65 | 10 | 0.24 | 8.12 | 10 | 0.62 |
Note:
Pearson chi-square (see text for details).
Abbreviations: APACHE, acute physiology and chronic health evaluations; DF, degrees of freedom; H-L, Hosmer–Lemeshow; SAPS, simplified acute physiology score.
Discrimination
For the model including only the CCI, the ability to discriminate between survivors and nonsurvivors of the current hospitalization was poor (c-statistic = 0.52; 95% confidence interval [CI] 0.46–0.59), as well as within 30 days (c-statistic = 0.52; 95% CI 0.46–0.57) and 1 year of ICU admission (c-statistic = 0.58; 95% CI 0.53–0.63) (Table 4). Adding age, gender, social factors, surgical/medical status, mechanical ventilation/renal replacement therapy, and primary diagnosis to the CCI score in a multivariable model improved the discrimination substantially, with c-statistics of 0.75 (95% CI 0.69–0.81) for in-hospital mortality, 0.75 (95% CI 0.70–0.80) for 30-day mortality, and 0.72 (95% CI 0.68–0.77) for 1-year mortality. Physiology-based scores discriminated as well as the CCI in combination with administrative data between short-term survivors and nonsurvivors. C-statistics for 1-year mortality was 0.70 (95% CI 0.66–0.75) for SAPS II, 0.69 (95% CI 0.64–0.73) for SAPS III, and 0.69 (95% CI 0.64–0.73) for APACHE II. Including ten comorbidity groups as separate covariates in the regression model only slightly improved discrimination compared with the CCI (c-statistic for 30-day mortality = 0.76; 95% CI 0.71–0.81). Of note, c-statistics were almost nearly the same for in-hospital, 30-day, and 1-year mortality within the same categories of comorbidity measures.
Table 4.
c-statistics (area under ROC curve) as measure of discrimination between survivors and nonsurvivors (in-hospital, 30-day, and 1-year mortality)
| Score/measure |
Outcome c-statistic (95% CI) |
||
|---|---|---|---|
| In-hospital mortality | 30-day mortality | 1-year mortality | |
| Mortality | 83/469 (17.7%) | 117/469 (24.9%) | 178/469 (40.0%) |
| Charlson score | 0.52 (0.46–0.59) | 0.52 (0.46–0.57) | 0.58 (0.53–0.63) |
| Charlson score, age and gender | 0.64 (0.57–0.71) | 0.65 (0.59–0.71) | 0.65 (0.60–0.70) |
| Charlson score, age and gender, surgical/medical status | 0.67 (0.60–0.7) | 0.67 (0.61–0.73) | 0.66 (0.61–0.71) |
| Charlson score, age and gender, surgical/medical status, social factors | 0.68 (0.62–0.75) | 0.70 (0.64–0.75) | 0.68 (0.63–0.73) |
| Charlson score, age and gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy | 0.72 (0.65–0.78) | 0.73 (0.68–0.78) | 0.71 (0.67–0.76) |
| Charlson score, age and gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy, primary diagnosis | 0.75 (0.69–0.81) | 0.75 (0.70–0.80) | 0.72 (0.68–0.77) |
| Ten separate comorbidity groups* | 0.59 (0.52–0.65) | 0.58 (0.52–0.64) | 0.63 (0.58–0.68) |
| Ten comorbidity groups,* with age, gender, surgical/medical status, social factors, mechanical ventilation and renal replacement therapy, primary diagnosis | 0.75 (0.69–0.82) | 0.76 (0.71–0.81) | 0.75 (0.70–0.79) |
| SAPS II | 0.74 (0.69–0.80) | 0.72 (0.67–0.75) | 0.70 (0.66–0.75) |
| SAPS III | 0.69 (0.63–0.75) | 0.68 (0.63–0.74) | 0.69 (0.64–0.73) |
| APACHE II | 0.73 (0.67–0.78) | 0.71 (0.66–0.76) | 0.71 (0.66–0.76) |
Note:
See text for details.
Abbreviations: APACHE, acute physiology and chronic health evaluations; CI, confidence interval; ROC, receiver operating characteristic; SAPS, simplified acute physiology score.
Including data on comorbidity from within 5 years prior to ICU admission had virtually no influence on c-statistics for the CCI models (data not shown).
Discussion
In this study of 469 critically ill ICU patients, we found that the CCI combined with other readily available administrative data performed as well as physiology-based scoring systems in predicting in-hospital, 30-day, and 1-year mortality.
Our findings for in-hospital mortality extend data from four previous studies that examined the performance of comorbidity scores in predicting in-hospital mortality among ICU patients.11,16–18 In a US study including more than 17,000 ICU patients from Veterans Affairs Medical Centers, Johnston et al found that the 30 comorbidity variables included in the Elixhauser index generated from administrative databases discriminated better between in-hospital survivors and nonsurvivors (c-statistic = 0.700), compared with the chronic health evaluation component of the APACHE score (c-statistic = 0.568).11 The Elixhauser index uses 30 comorbidity variables separately and allows each variable to influence the outcome independently. When combined with other clinical data (laboratory, principal diagnosis, age, and admission source) the discrimination of the Elixhauser index was excellent (c-statistic = 0.874); however, some of the data used are not readily available in other settings – particularly laboratory, which accounted for 67.7% of the model’s unique attributable chi square. Model calibration was not reported. In a 1996 US study of 201 general ICU patients, the CCI showed less than optimal discriminating ability for in-hospital mortality (c-statistic = 0.67).18 Data on comorbidity were collected by chart review, and the CCI was not combined with other administrative data. Ho et al found that the CCI had poor predictive performance for short-term mortality among 24,303 ICU patients in Western Australia (c-statistic ≤ 0.610).16 This study also did not combine the CCI with other administrative data. A 2006 Canadian study among 1603 ICU patients found that APACHE II predicted in-hospital survival better than the CCI (c-statistics = 0.77 versus 0.69).17
Very limited data exist on how well comorbidity scores and physiology-based scores predict long-term mortality of ICU patients. Recently, an Australian study including more than 11,000 ICU patients found that in a prediction model (PREDICT) of 1-, 5-, and 15-year mortality, age, and comorbidity, as measured by the CCI, were the most important determinants of prognosis.12 The c-index of the full PREDICT model was 0.757 (95% CI 0.745–0.769); however, no data on the performance of the CCI without APACHE II score in the model was presented. In the SUPPORT prognostic model study, Knaus et al developed a prediction model of 180-day mortality for ICU patients (c-statistics = 0.79) primarily based on diagnosis and physiological variables; however, they did not include a comorbid-ity score in their model.24
In our study, the physiology-based scoring systems performed less well in predicting in-hospital mortality than previously reported.25 In an assessment of the performance of APACHE, SAPS, and MPM in 22 general ICUs in Scotland, Livingstone et al found good to excellent discrimination for all three scoring systems, with c-statistics ranging between 0.785 and 0.854.26 We obtained data on physiological variables from computerized databases containing prospectively collected data, which reduced the risk of information bias. However, some clinical variables, in particular the GCS score, were missing in a number of medical records. and by assuming that patients with missing values had normal GCS scores we may have underestimated the SAPS and APACHE scores.22 The majority of patients in our study were admitted from emergency departments or following surgery and, therefore, were stabilized before arrival at the ICU. By underestimating the physiological derangement we may, thus, have slightly underestimated the SAPS and APACHE scores. We did not include patients admitted for planned postoperative observation for less than 24 hours but may still have included some cardiac patients who, by definition, were not included in the original physiology-based scores. This may have led to an underestimation of the predictive performance of the physiology-based scores. The SAPS II and APACHE II were developed more than 20 years ago and improvements in treatment strategies, such as implementation of continuous renal replacement therapy, may explain at least a part of the poorer performance of the physiology-based scores. However, we found virtually similar predictive performances among SAPS II and APACHE II scores compared with the newer SAPS III score, suggesting that this had only minor influence on our results.
Based on our results, the CCI, combined with other routinely collected data from administrative medical databases, seem to perform at least as well as physiology-based scores in predicting mortality in intensive care patients. Our data suggest that the combination of ten separate comorbidity groups with other routinely collected data may perform slightly better than combining the CCI with other routinely collected data. However, the performance of administrative data largely depends on the completeness and validity of the data. Our access to high-quality population-based medical databases, linked using a unique personal identifier, may explain the good overall performance of the administrative data in our study. Of note, restricting comorbidity information to 5 years prior to ICU admission had virtually no impact on the performance of the CCI model. The current study was based on data from a general ICU. Still, since Skejby Hospital is the largest center for treatment of cardiovascular diseases in western Denmark, including heart transplant and extra-corporeal membrane oxygenation treatment, the majority of patients were admitted with cardiovascular diseases as the primary diagnosis. APACHE II has previously been reported to perform less well in patients with cardiovascular diseases compared with general ICU patients. Thus, whether the CCI in combination with administrative data perform as well as physiology-based scores in predicting mortality in other settings remains to be clarified.
In conclusion, our study demonstrates that the CCI combined with other readily available administrative data performed as well as physiology-based scoring systems in predicting in-hospital, 30-day, and 1-year mortality.
Appendix.
Table A.
Charlson comorbidity index and comorbidity groups
| Charlson comorbidity category | Number of patients (%) | 30-day mortality | ICD-8 | ICD-10 | Charlson comorbidity index score | Comorbidity groups |
|---|---|---|---|---|---|---|
| Myocardial infarction | 111 (23.7%) | 22.2% | 410 | I21; I22; I23 | 1 | Myocardial infarction |
| Congestive heart failure | 86 (18.3%) | 20.5% | 427.09; 427.10; 427.11; 427.19; 428.99; 782.49 | I50; I11.0; I13.0; I13.2 | 1 | Congestive heart failure |
| Peripheral vascular disease | 92 (19.6%) | 17.1% | 440; 441; 442; 443; 444; 445 | I70; I71; I72; I73; I74; I77 | 1 | Peripheral vascular disease |
| Cerebrovascular disease | 70 (14.9%) | 14.5% | 430–438 | I60–I69; G45; G46 | 1 | Cerebrovascular disease |
| Dementia | 5 (1.1%) | 2.6% | 290.09–290.19; 293.09 | F00–F03; F05.1; G30 | 1 | – |
| Chronic pulmonary disease | 94 (20.0%) | 23.1% | 490–493; 515–518 | J40–J47; J60–J67; J68.4; J70.1; J70.3; J84.1; J92.0; J96.1; J98.2; J98.3 | 1 | Chronic pulmonary disease |
| Connective tissue disease | 27 (5.8%) | 5.1% | 712; 716; 734; 446; 135.99 | M05; M06; M08; M09; M30; M31; M32; M33; M34; M35; M36; D86 | 1 | – |
| Ulcer disease | 48 (10.2%) | 10.6% | 530.91; 530.98; 531–534 | K22.1; K25–K28 | 1 | Peptic ulcer disease |
| Mild liver disease | 15 (3.2%) | 3.4% | 571; 573.01; 573.04 | B18; K70.0–K70.3; K70.9; K71; K73; K74; K76.0 | 1 | Liver disease |
| Diabetes type 1 | 52 (11.1%) | 7.7% | 249.00; 249.06; 249.07; 249.09 | E10.0, E10.1; E10.9 | 1 | Diabetes |
| Diabetes type 2 | 250.00; 250.06; 250.07; 250.09 | E11.0; E11.1; E11.9 | ||||
| Hemiplegia | 1 (0.2%) | – | 344 | G81; G82 | 2 | – |
| Moderate to severe renal disease | 60 (12.8%) | 11.1% | 403; 404; 580–583; 584; 590.09; 593.19; 753.10–753.19; 792 | I12; I13; N00–N05; N07; N11; N14; N17–N19; Q61 | 2 | Renal disease |
| Diabetes with end organ damage type 1 | 38 (8.1%) | 6.8% | 249.01–249.05; 249.08 | E10.2–E10.8 | 2 | Diabetes |
| type 2 | 250.01–250.05; 250.08 | E11.2–E11.8 | ||||
| Any tumor | 74 (15.8%) | 17.1% | 140–194 | C00–C75 | 2 | Cancer |
| Leukemia | 3 (0.6%) | 1.7% | 204–207 | C91–C95 | 2 | Cancer |
| Lymphoma | 5 (1.1%) | 0.8% | 200–203; 275.59 | C81–C85; C88; C90; C96 | 2 | Cancer |
| Moderate to severe liver disease | 5 (1.1%) | 0.9% | 070.00; 070.02; 070.04; 070.06; 070.08; 573.00; 456.00–456.09 | B15.0; B16.0; B16.2; B19.0; K70.4; K72; K76.6; I85 | 3 | Liver disease |
| Metastatic solid tumor | 2 (0.4%) | 1.7% | 195–198; 199 | C76–C80 | 6 | Cancer |
| AIDS | 1 (0.2%) | – | 079.83 | B21–B24 | 6 | – |
Table B.
Diagnostic categories
| Infectious diseases (ICD-10: A00–B99) |
| Endocrinology including diabetes (ICD-10: E00–E90) |
| Cardiovascular diseases (ICD-10: I00–I99) |
| Respiratory diseases (ICD-10: J00–J99) |
| Gastrointestinal and liver disease (ICD-10: K00–K99) |
| Cancer (ICD-10: C00–D89) |
| Trauma and poisoning (ICD-10: S00–T98) |
| And others (ICD-10: all codes not included in other categories) |
Table C.
Overview of variables included in APACHE II, SAPS II, and SAPS III
| Variables | APACHE II | SAPS II | SAPS III |
|---|---|---|---|
| Temperature | x | x | x |
| Mean arterial pressure | x | ||
| Heart rate | x | x | x |
| Respiratory rate | x | ||
| Oxygenation | x | x | x |
| Arterial pH | x | x | |
| Serum sodium | x | x | |
| Serum potassium | x | x | |
| Serum creatinine/urea | x | x | x |
| Hematocrit | x | ||
| White blood count | x | x | x |
| Glasgow coma scale | x | x | x |
| Type of admission | x | ||
| Chronic diseases | x | x | x |
| Age | x | x | x |
| Systolic blood pressure | x | x | |
| Urine output | x | ||
| Bilirubin | x | x | |
| HCO3 | x | ||
| Length of stay before ICU admission | x | ||
| Intra-hospital location before ICU admission | x | ||
| Reason for ICU admission | x | ||
| Surgical status | x | ||
| Acute infection at ICU admission | x | ||
| Platelets | x |
Abbreviations: AIDS, acquired immunodeficiency syndrome; APACHE, acute physiology and chronic health evaluations; ICD-8, International Classification of Diseases (Revision 8); ICD-10, International Classification of Diseases (Revision 10); ICU, intensive care unit; SAPS, simplified acute physiology score.
Footnotes
Disclosure
The authors report no conflicts of interest in this work.
References
- 1.Vincent JL. Evidence-based medicine in the ICU: important advances and limitations. Chest. 2004;126(2):592–600. doi: 10.1378/chest.126.2.592. [DOI] [PubMed] [Google Scholar]
- 2.Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310. doi: 10.1097/00003246-200107000-00002. [DOI] [PubMed] [Google Scholar]
- 3.Barnato AE, Alexander SL, Linde-Zwirble WT, Angus DC. Racial variation in the incidence, care, and outcomes of severe sepsis: analysis of population, patient, and hospital characteristics. Am J Respir Crit Care Med. 2008;177(3):279–284. doi: 10.1164/rccm.200703-480OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Vincent JL, Sakr Y, Sprung C, Harboe S, Damas P. Are blood transfusions associated with greater mortality rates? Results of the Sepsis Occurrence in Acutely Ill Patients study. Anesthesiology. 2008;108(1):31–39. doi: 10.1097/01.anes.0000296070.75956.40. [DOI] [PubMed] [Google Scholar]
- 5.Vincent JL, Baron JF, Reinhart K, et al. Anemia and blood transfusion in critically ill patients. JAMA. 2002;288(12):1499–1507. doi: 10.1001/jama.288.12.1499. [DOI] [PubMed] [Google Scholar]
- 6.Padkin A, Rowan K, Black N. Using high quality clinical databases to complement the results of randomised controlled trials: the case of recombinant human activated protein C. BMJ. 2001;323(7318):923–926. doi: 10.1136/bmj.323.7318.923. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Le G, Jr, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–2963. doi: 10.1001/jama.270.24.2957. [DOI] [PubMed] [Google Scholar]
- 8.Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–829. [PubMed] [Google Scholar]
- 9.Lemeshow S, Teres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J. Mortality probability models (MPM II) based on an international cohort of intensive care unit patients. JAMA. 1993;270(20):2478–2486. [PubMed] [Google Scholar]
- 10.Pittet D, Thievent B, Wenzel RP, Li N, Gurman G, Suter PM. Importance of pre-existing co-morbidities for prognosis of septicemia in critically ill patients. Intensive Care Med. 1993;19(5):265–272. doi: 10.1007/BF01690546. [DOI] [PubMed] [Google Scholar]
- 11.Johnston JA, Wagner DP, Timmons S, Welsh D, Tsevat J, Render ML. Impact of different measures of comorbid disease on predicted mortality of intensive care unit patients. Med Care. 2002;40(10):929–940. doi: 10.1097/00005650-200210000-00010. [DOI] [PubMed] [Google Scholar]
- 12.Ho KM, Knuiman M, Finn J, Webb SA. Estimating long-term survival of critically ill patients: the PREDICT model. PLoS ONE. 2008;3(9):e3226. doi: 10.1371/journal.pone.0003226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 14.Needham DM, Scales DC, Laupacis A, Pronovost PJ. A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research. J Crit Care. 2005;20(1):12–19. doi: 10.1016/j.jcrc.2004.09.007. [DOI] [PubMed] [Google Scholar]
- 15.Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613–619. doi: 10.1016/0895-4356(92)90133-8. [DOI] [PubMed] [Google Scholar]
- 16.Ho KM, Finn J, Knuiman M, Webb SA. Combining multiple comorbidities with Acute Physiology Score to predict hospital mortality of critically ill patients: a linked data cohort study. Anaesthesia. 2007;62(11):1095–1100. doi: 10.1111/j.1365-2044.2007.05231.x. [DOI] [PubMed] [Google Scholar]
- 17.Norena M, Wong H, Thompson WD, Keenan SP, Dodek PM. Adjustment of intensive care unit outcomes for severity of illness and comorbidity scores. J Crit Care. 2006;21(2):142–150. doi: 10.1016/j.jcrc.2005.11.011. [DOI] [PubMed] [Google Scholar]
- 18.Poses RM, McClish DK, Smith WR, Bekes C, Scott WE. Prediction of survival of critically ill patients by admission comorbidity. J Clin Epidemiol. 1996;49(7):743–747. doi: 10.1016/0895-4356(96)00021-2. [DOI] [PubMed] [Google Scholar]
- 19.Knaus W, Wagner D, Draper E. APACHE III study design: analytic plan for evaluation of severity and outcome in intensive care unit patients. Implications. Crit Care Med. 1989;17(12 Pt 2):S219–S221. [PubMed] [Google Scholar]
- 20.Andersen TF, Madsen M, Jorgensen J, Mellemkjoer L, Olsen JH. The Danish National Hospital Register. A valuable source of data for modern health sciences. Dan Med Bull. 1999;46(3):263–268. [PubMed] [Google Scholar]
- 21.Frank L. Epidemiology. When an entire country is a cohort. Science. 2000;287(5462):2398–2399. doi: 10.1126/science.287.5462.2398. [DOI] [PubMed] [Google Scholar]
- 22.Afessa B, Keegan MT, Gajic O, Hubmayr RD, Peters SG. The influence of missing components of the Acute Physiology Score of APACHE III on the measurement of ICU performance. Intensive Care Med. 2005;31(11):1537–1543. doi: 10.1007/s00134-005-2751-9. [DOI] [PubMed] [Google Scholar]
- 23.Schneeweiss S, Seeger JD, Maclure M, Wang PS, Avorn J, Glynn RJ. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol. 2001;154(9):854–864. doi: 10.1093/aje/154.9.854. [DOI] [PubMed] [Google Scholar]
- 24.Knaus WA, Harrell FE, Jr, Lynn J, et al. The SUPPORT prognostic model. Objective estimates of survival for seriously ill hospitalized adults. Study to understand prognoses and preferences for outcomes and risks of treatments. Ann Intern Med. 1995;122:191–203. doi: 10.7326/0003-4819-122-3-199502010-00007. [DOI] [PubMed] [Google Scholar]
- 25.Strand K, Flaatten H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand. 2008;52:467–478. doi: 10.1111/j.1399-6576.2008.01586.x. [DOI] [PubMed] [Google Scholar]
- 26.Livingston BM, MacKirdy FN, Howie JC, Jones R, Norrie JD. Assessment of the performance of five intensive care scoring models within a large Scottish database. Crit Care Med. 2000;28:1820–1827. doi: 10.1097/00003246-200006000-00023. [DOI] [PubMed] [Google Scholar]