Abstract
Background
Understanding comorbidity prevalence and the effects of comorbidities in older veterans with lower extremity amputations may aid in assessing patient outcomes, resource use, and facility-level quality of care.
Objectives
To determine the degree to which adding outpatient to inpatient administrative data sources yields higher comorbidity prevalence estimates and improved explanatory power of models predicting 1-year mortality and to compare the Charlson/Deyo and Elixhauser comorbidity measures.
Methods
A retrospective cohort study applying frequencies, cross-tabulations, and logistic regression models was conducted, including data from 2,375 veterans with lower extremity amputations. Comorbidity prevalence according to the Charlson/Deyo and Elixhauser measures, 1-year mortality rates, and standardized mortality ratios (SMRs) were analyzed.
Results
Comorbidity prevalence estimates increased sharply for both the Charlson/Deyo and Elixhauser measures with the addition of data from multiple settings. The Elixhauser compared to the Charlson/Deyo generally yielded higher estimates but did not improve explanatory power for mortality. Modeling expected versus actual deaths produced varying SMRs across geographic regions but was not dependent on which measure or data sources were used.
Conclusions
Merging outpatient with inpatient data may reduce the under coding of comorbidities but does not enhance mortality prediction. Compared to the Charlson/ Deyo, the Elixhauser has a more complete coding scheme for comorbid conditions, such as diabetes mellitus and peripheral vascular disease, important to addressing lower extremity amputation etiology.
Keywords: Elderly, Quality measurement, Administrative data, Amputation, Comorbidity prevalence, Mortality
Introduction
It is known that comorbidity in the aging population is associated with higher healthcare utilization and spending [1]. To evaluate these expenditures, it is essential to understand chronic disease. The inclusion of data from a variety of inpatient and outpatient settings could yield more complete prevalence estimates by capturing comorbidities important at different care settings.
The Veterans Health Administration (VHA) linkable administrative databases allow us to study the full continuum of care of an entire population of lower extremity amputees in a way that is not possible in the private sector. We hypothesized that records added from outpatient visits before surgery and from separate bed sections to those of the inpatient stay would generate more complete comorbidity prevalence estimates. We also speculated that there would be a difference in performance between the Charlson/Deyo and the Elixhauser comorbidity measures, based on previous studies [2, 3]. By evaluating these two measures across multiple data sources, we hoped to identify the best methodology for case-mix adjustment in the use of standardized mortality ratios (SMRs) as a measure of quality [4]. To our knowledge, this is the first comparison of alternative comorbidity methods in lower extremity amputees. Applying aspects of the burden of illness concept presented by Elixhauser et al. [5], this study defines comorbidity as any condition present prior to surgery other than the surgical amputation. Amputation is considered the primary diagnosis.
Methods
This study was approved by the University of Pennsylvania, Philadelphia, Pa., the Samuel S. Stratton Veterans Affairs Medical Center (VAMC), Albany, N.Y., and the Kansas City VAMC, Kansas City, Mo., Institutional Review Boards.
Database Description
Data were obtained from the VHA administrative databases that track the case-mix and healthcare utilization of primarily veterans. Inpatient datasets include four files referred to as the patient treatment files (PTF) [6]. The main file is a discharge file for the entire hospitalization episode. Multiple bed section files include diagnoses associated with specific specialty treating services within the episode. The procedures file includes each day’s procedures, and the surgery file describes each day’s surgeries during the inpatient episode. There are two outpatient care files [7] that describe visit and event. The visit file includes information about all of one day’s care encounters or stops. The event file includes procedures and International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnoses associated with that stop. The Beneficiary Identification Record Locator System (BIRLS) death file contains records of all beneficiaries, including veterans whose survivors applied for death benefits [8].
Study Population
There were 2,375 records with discharge dates between October 1, 2002, and September 30, 2003, for transtibial, transfemoral, or hip disarticulation amputations that were captured using the following surgical procedure codes: 84.10, 84.13–84.19, and 84.91 [9]. Patients who had amputations that involved toes only or who had a record of a previous lower extremity amputation within 12 months preceding the index surgical stay were excluded. The hospitalization at the time of the surgical amputation represented the ‘index surgical stay’.
Comorbidity Measurements
Both the Charlson/Deyo and the Elixhauser are based on ICD-9-CM codes. The Charlson Comorbidity Index, a weighted index that accounts for the quantity and severity of comorbidities, was created in 1987 to predict mortality due to comorbidities in breast cancer patients [10]. Deyo et al. [11] adapted the Charlson to lumbar spine surgery, identifying selected ICD-9-CM codes as indicators for defining 17 different comorbidities. Deyo’s adaptation has been widely used in medical literature. The Elixhauser measure, created in 1997, applies ICD-9-CM codes to identify 30 different comorbidities as separate variables [5] and was selected because of its capacity to surpass the Charlson/Deyo in its ability to explain mortality risk differences in patients with acute myocardial infarction (MI) [2, 3]. The 2003 version of the Elixhauser that distinguishes between hypertension with and without complication was used [12]. Thus, our study uses 31 conditions. Since the Elixhauser comorbidities are not weighted [5], we expressed each Charlson/Deyo diagnosis as a separate set of binary variables, 1 = present and 0 = absent, to compare predictive values of the two measures.
Database Development
We developed three datasets to demonstrate the effect of increasing data sources on the operation of the two measures. The ‘inpatient only’ set (dataset 1) consisted of diagnoses from the main hospitalization. In the ‘inpatient + bed section’ set (dataset 2), diagnoses from each PTF bed section files were added. The ‘inpatient + bed section + outpatient’ set (dataset 3) added outpatient data falling within 90 days preceding the hospital admission date. Two sets of parallel analytic subfiles for each comorbidity measure were created, yielding six models.
The inpatient and the BIRLS databases were used to acquire mortality information. The inpatient database was accessed first, since it identifies patients who died in a VA hospital. If a death date was found on discharge, that date was used. If there was no date or the death date followed the discharge date, the BIRLS was examined. If no death date was found in either database, it was assumed that the veteran was still alive 1 year following the amputation.
Statistical Analyses
Comorbidity prevalence estimates were obtained by calculating the frequency of each Charlson/Deyo and Elixhauser comorbidity separately. χ2 analyses compared the estimated comorbidity and mortality prevalence. One-year mortality risk was estimated through six logistic regression models. Model performance was addressed by the C-statistic measuring model discrimination [13] and the Hosmer-Lemeshow goodness-of-fit statistic, which determines if the model provides a good fit for the data [14]. Predictive probabilities for each were calculated to compare expected to actual 1-year deaths. Actual deaths divided by expected deaths yielded a SMR that was aggregated to the Veterans Integrated Service Network (VISN) level for the entire sample. Standard errors of the expected death probabilities were used to calculate Z scores for each SMR at the VISN level. Statistical significance at p < 0.05 was used to identify VISNs with significantly higher or lower death rates compared to the expected death rates, comparing the two measures across the increasing data sources (datasets 1 and 3) [4]. All analyses were performed using SAS Version 9.1.
Results
Out of the 2,375 amputees, 889 (37.4%) were ≤65 years of age, and 51 (2.1%) were >85 years of age. Sixteen Charlson/Deyo and 30 Elixhauser comorbidities were present in our data.
Table 1 shows that prevalence estimates increased across all Elixhauser conditions from dataset 1 to dataset 3. Similar results were seen for the Charlson/Deyo conditions (results not shown). The Elixhauser often yielded higher estimated prevalence. In dataset 3 and in the Elixhauser, 74.5% of patients had codes indicating peripheral vascular disease (PVD), but only 66.9% according to the Charlson/Deyo. Increasing data sources showed little effect on mortality rates in those Elixhauser conditions with high prevalence. Certain lower prevalence conditions, such as hypertension with complications, lymphoma, and pulmonary circulation disease, showed higher rates of mortality (>20% increase), while coagulopathy, metastatic cancer, and other neurological disorders showed lower rates (>15% decrease). Pattern changes in mortality rates were similar for the Charlson/Deyo (results not shown).
Table 1.
Increasing comorbidity prevalence and estimated one-year mortality rate defined by Elixhauser measure across treatment setting
| Comorbidity1 | Dataset 12
|
Dataset 2
|
Dataset 3
|
|||
|---|---|---|---|---|---|---|
| prevalence | mortality | prevalence | mortality | prevalence | mortality | |
| Acquired immune deficiency syndrome | 7 (0.3) | 1 (14.3) | 9 (0.4) | 3 (33.3) | 17 (0.7) | 4 (23.5) |
| Alcohol abuse | 72 (3.0) | 11 (15.3) | 95 (4.0) | 16 (16.8) | 130 (5.5) | 25 (19.2) |
| Blood loss anemia | 33 (1.4) | 13 (39.4) | 43 (1.8) | 16 (37.2) | 45 (1.9) | 17 (37.8) |
| Cardiac arrhythmias | 278 (11.7) | 114 (41.0) | 330 (13.9) | 139 (42.1) | 382 (16.1) | 156 (40.8) |
| Chronic pulmonary disease | 359 (15.1) | 137 (38.2) | 413 (17.4) | 157 (38.0) | 477 (20.1) | 179 (37.5) |
| Coagulopathy | 52 (2.2) | 25 (48.1) | 65 (2.7) | 32 (49.2) | 104 (4.4) | 39 (37.5) |
| Congestive heart failure | 401 (16.9) | 157 (39.2) | 483 (20.3) | 187 (38.7) | 539 (22.7) | 209 (38.8) |
| Deficiency anemias | 306 (12.9) | 77 (25.2) | 369 (15.5) | 101 (27.4) | 451 (19.0) | 138 (30.6) |
| Depression | 95 (4.0) | 17 (17.9) | 124 (5.2) | 26 (21.0) | 211 (8.9) | 49 (23.2) |
| Diabetes mellitus | 457 (19.2) | 114 (24.9) | 519 (21.9) | 136 (26.2) | 671 (28.3) | 170 (25.3) |
| Diabetes mellitus with complication | 808 (34.0) | 206 (25.5) | 913 (38.4) | 237 (26.0) | 1,069 (45.0) | 273 (25.5) |
| Drug abuse | 21 (0.9) | 4 (19.0) | 30 (1.3) | 6 (20.0) | 54 (2.3) | 8 (14.8) |
| Fluid and electrolyte disorders | 257 (10.8) | 93 (36.2) | 393 (16.5) | 150 (38.2) | 447 (18.8) | 160 (35.8) |
| Hypertension | 932 (39.2) | 202 (21.7) | 1,091 (45.9) | 251 (23.0) | 1,390 (58.5) | 343 (24.7) |
| Hypertension with complication | 7 (0.3) | 1 (14.3) | 8 (0.3) | 2 (25.0) | 13 (0.5) | 4 (30.8) |
| Hypothyroidism | 54 (2.3) | 16 (29.6) | 66 (2.8) | 21 (31.8) | 89 (3.7) | 32 (36.0) |
| Liver disease | 56 (2.4) | 19 (33.9) | 71 (3.0) | 24 (33.8) | 81 (3.4) | 27 (33.3) |
| Lymphoma | 6 (0.3) | 2 (33.3) | 7 (0.3) | 3 (42.9) | 9 (0.4) | 4 (44.4) |
| Metastatic cancer | 23 (1.0) | 16 (69.6) | 28 (1.2) | 18 (64.3) | 32 (1.3) | 18 (56.3) |
| Other neurological disorders | 42 (1.8) | 18 (42.9) | 50 (2.1) | 20 (40.0) | 72 (3.0) | 26 (36.1) |
| Paralysis | 76 (3.2) | 12 (15.8) | 81 (3.4) | 15 (18.5) | 93 (3.9) | 16 (17.2) |
| Peptic ulcer disease with bleeding | 15 (0.6) | 2 (13.3) | 18 (0.8) | 2 (11.1) | 35 (1.5) | 7 (20.0) |
| Peripheral vascular disease | 1,489 (62.7) | 404 (27.1) | 1,594 (67.1) | 439 (27.5) | 1,770 (74.5) | 482 (27.2) |
| Psychoses | 83 (3.5) | 19 (22.9) | 99 (4.2) | 23 (23.2) | 164 (6.9) | 40 (24.4) |
| Pulmonary circulation disease | 11 (0.5) | 5 (45.5) | 14 (0.6) | 7 (50.0) | 17 (0.7) | 10 (58.8) |
| Renal failure | 312 (13.1) | 137 (43.9) | 347 (14.6) | 152 (43.8) | 410 (17.3) | 168 (41.0) |
| Rheumatoid arthritis | 18 (0.8) | 5 (27.8) | 22 (0.9) | 6 (27.3) | 32 (1.3) | 8 (25.0) |
| Solid tumor without metastasis | 104 (4.4) | 44 (42.3) | 111 (4.7) | 49 (44.1) | 166 (7.0) | 68 (41.0) |
| Valvular disease | 69 (2.9) | 28 (40.6) | 92 (3.9) | 38 (41.3) | 111 (4.7) | 50 (45.0) |
| Weight loss | 82 (3.5) | 26 (31.7) | 101 (4.3) | 33 (32.7) | 109 (4.6) | 35 (32.1) |
Percentage values are shown in parentheses.
There were no cases with the ICD-9-CM code for obesity.
Dataset 1 = inpatient only; dataset 2 = inpatient + bed section; dataset 3 = inpatient + bed section + outpatient.
In the adjusted multivariate models using the Elixhauser, the C-statistic showed little change (0.69 in data-set 1 to 0.70 in dataset 3) (results not shown). Dataset 3 improved model fit of the data (Hosmer-Lemeshow p = 0.02 in dataset 1 to p = 0.08 in dataset 3). For the Charlson/Deyo, the C-statistic increased from 0.65 to 0.68 (results not shown), and all three models adequately fit the data (all p > 0.05). The Elixhauser expected mortality estimates ranged from 5.6 to 88.7% and the Charlson/Deyo from 11.1 to 89.5%.
Figure 1 shows the SMRs for each VISN. All four methods appear to track in a highly correlated fashion, with only one visible deviation in VISN ‘L’, which likely relates to a small denominator (n = 51) compared to the other VISNs (mean = 116). There was little variation in the statistical significance of expected versus actual deaths in the four methods (results not shown). Only two VISNs demonstrated variation that was statistically significant as ‘outlier’ performers.
Fig. 1.
Standard mortality ratios in each VISN using Charlson/Deyo and Elixhauser comorbidity measures with different sources to obtain ICD-9-CM diagnoses, sorted by the Charlson/Deyo obtained from dataset 1. Charlson/Deyo 1 is the Charlson/Deyo method using the inpatient only dataset. Charlson/Deyo 3 is the Charlson/Deyo method using inpatient + bed section + outpatient dataset. Elixhauser 1 is the Elixhauser method using the inpatient only dataset. Elixhauser 3 is the Elixhauser method using inpatient + bed section + outpatient dataset.
Discussion
The presence of comorbidities is an important issue in the elderly [1], especially for older amputees. We found that in both the Charlson/Deyo and the Elixhauser, additional sources led to higher estimated comorbidity prevalence. Other studies have also shown that adding additional data sources to inpatient data increases comorbidity detection [15].
While prevalence estimates increased sharply, there was little effect on predicted mortality, with a few exceptions. Results suggest that some clinically important cases of valvular disease had not been coded in the PTF main file, whereas only more advanced cancers had been included. Systematic reporting bias appeared largely random and not associated with mortality. Moreover, certain conditions including DM and hypertension appeared protective even with increased data sources. Patients with these conditions may be receiving better quality of care, since their conditions require close monitoring.
Our results illustrate that the estimated prevalence of particular conditions depends on both the data source and the measure used. The Elixhauser tends to include more exhaustive ICD-9-CM definitions. For example, DM with peripheral circulatory disorders was counted as DM with complication in the Elixhauser but only as DM in the Charlson/Deyo. Also, clinicians may have selected codes to document renal, ophthalmic, or neurological complications of DM not included in either measure. Including these diagnoses would have increased estimated prevalence of DM with complication from 0 to 11.2% and from 45.0 to 56.3% for the Charlson/Deyo and Elixhauser, respectively. Moreover, there are prominent differences in the ICD-9-CM codes selected for PVD. The Elixhauser includes 28 codes, while the Charlson/Deyo incorporates only 17. This emphasizes the need to be aware of differences in the sensitivity and specificity of particular ICD-9-CM based comorbidity measures and the importance of selection, depending on the primary conditions being studied and the questions being asked. Since DM and PVD are such essential risk factors for limb loss [9], the Elixhauser maybe more appropriate.
In contrast to others [2, 3] who found clear evidence that the Elixhauser surpassed the Charlson/Deyo in predicting mortality, we found little differences between the two. Mortality models among patients with acute MI, CHF, COPD, and hypertension had C-statistic values ranging from 0.608 to 0.715 based on the Charlson/ Deyo and from 0.657 to 0.793 based on the Elixhauser [2, 3]. Our models with the Charlson/Deyo (0.654–0.681) or Elixhauser (0.685–0.697) performed within the low range. Comparisons need to be drawn carefully, since timeframes (in-hospital versus 1-year) differ.
We found that the SMRs were consistent across both measures. Others have shown that source and measure affect outcomes. Iezzoni et al. [16] found that ICD-9-CM codes outperformed clinical severity data from medical records when predicting in-hospital deaths following coronary artery bypass graft (CABG) surgery. In contrast, Parker et al. [17] found that clinical data was slightly better than diagnostic codes in predicting mortality following CABG. Conversely, Tseng et al. [18] showed that adding claims data to demographic data contributed significantly to predicting mortality among diabetic veteran amputees. Conclusions about quality could relate to methodological decisions rather than regional differences in quality of care.
Advantages of this study include the VA’s reputation for innovation and leadership in electronic medical records and administrative databases as well as the study’s homogeneous population. Limitations include the small female sample, the use of administrative data which may be less accurate than chart review, and the inclusion of only VA hospitals.
We show that limiting analyses to inpatient data may lead to compressed comorbidity prevalence estimates for elderly amputees. The inclusion of broader data sources may add precision to prevalence estimates; however, future studies will need to confirm the accuracy of outpatient ICD-9-CM coding. The addition of outpatient data appeared to have minimum value in mortality risk adjustment and may have resulted in higher false positive rates. We do not necessarily endorse the addition of pre-hospitalization outpatient data when estimating postsurgical mortality. Data sources and comorbidity measures applied did not have a substantial effect on identifying VISNs with outlier mortality rates after adjustment based on SMRs. Caution in the interpretation of these types of estimates is both prudent and recommended.
Acknowledgments
This material is based upon work supported in part by the National Institutes of Health (RO1-HD042588). It is also supported by resources and the use of facilities at the Samuel S. Stratton Department of Veterans Affairs Medical Center, Albany, N.Y., and the Kansas City Department of Veterans Affairs Medical Center, Kansas City, Mo. The opinions and conclusions of the authors are not necessarily those of the sponsoring agencies.
References
- 1.Cesari M, Onder G, Russo A, Zamboni V, Barillaro C, Ferrucci L, Pahor M, Bernabei R, Landi F. Comorbidity and physical function: results from the aging and longevity study in the Sirente geographic area (ilSIRENTE study) Gerontology. 2006;52:24–32. doi: 10.1159/000089822. [DOI] [PubMed] [Google Scholar]
- 2.Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care. 2004;42:355–360. doi: 10.1097/01.mlr.0000118861.56848.ee. [DOI] [PubMed] [Google Scholar]
- 3.Stukenborg GJ, Wagner DP, Connors AF., Jr Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations. Med Care. 2001;39:727–739. doi: 10.1097/00005650-200107000-00009. [DOI] [PubMed] [Google Scholar]
- 4.Iezzoni LI. Risk Adjustment for Measuring HealthCare Outcomes. Portland/OR: Health Administration Press; 2003. [Google Scholar]
- 5.Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36:8–27. doi: 10.1097/00005650-199801000-00004. [DOI] [PubMed] [Google Scholar]
- 6.Hines Edward J, Jr, VA Hospital . FY2000 VHA Medical SAS Inpatient Datasets. Hines/IL: Veterans Affairs Information Resource Center; Apr, 2003. VIReC Research User Guide. [Google Scholar]
- 7.Hines Edward J, Jr, VA Hospital . FY2000 VHA Medical SAS Outpatient Datasets. Hines/IL: Veterans Affairs Information Resource Center; Jan, 2003. VIReC Research User Guide. [Google Scholar]
- 8.Kubal JD, Webber S, Cooper DC, Waight S, Hynes DM. VI-ReC Insights. 5. Hines/IL: Veterans Affairs Information Resource Center; 2000. A primer on US mortality databases used in health services research. [Google Scholar]
- 9.Mayfield JA, Reiber GE, Maynard C, Czerniecki JM, Caps MT, Sangeorzan BJ. Survival following lower-limb amputation in a veteran population. J Rehabil Res Dev. 2001;38:341–345. [PubMed] [Google Scholar]
- 10.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 11.Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–619. doi: 10.1016/0895-4356(92)90133-8. [DOI] [PubMed] [Google Scholar]
- 12.Agency for Healthcare Research and Quality. Comorbidity Software 3.1. 2005. [Google Scholar]
- 13.Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic curve. Radiology. 1982;143:29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
- 14.Hosmer DL. Applied Logistic Regression. New York: Wiley; 1989. [Google Scholar]
- 15.Klabunde CN, Potosky AL, Legler JM, Warren JL. Development of a comorbidity index using physician claims data. J Clin Epidemiol. 2000;53:1258–1267. doi: 10.1016/s0895-4356(00)00256-0. [DOI] [PubMed] [Google Scholar]
- 16.Iezzoni LI, Ash AS, Shwartz M, Landon BE, Mackiernan YD. Predicting in-hospital deaths from coronary artery bypass graft surgery. Do different severity measures give different predictions? Med Care. 1998;36:28–39. doi: 10.1097/00005650-199801000-00005. [DOI] [PubMed] [Google Scholar]
- 17.Parker JP, Li Z, Damberg CL, Danielsen B, Carlisle DM. Administrative versus clinical data for coronary artery bypass graft surgery report cards: the view from California. Med Care. 2006;44:687–695. doi: 10.1097/01.mlr.0000215815.70506.b6. [DOI] [PubMed] [Google Scholar]
- 18.Tseng CL, Rajan M, Miller DR, Hawley G, Crystal S, Xie M, Tiwari A, Safford M, Pogach L. Use of administrative data to risk adjust amputation rates in a national cohort of Medicare-enrolled veterans with diabetes. Med Care. 2005;43:88–92. [PubMed] [Google Scholar]