Developing and Validating a Risk Score for Lower-Extremity Amputation in Patients Hospitalized for a Diabetic Foot Infection

Abstract

OBJECTIVE

Diabetic foot infection is the predominant predisposing factor to nontraumatic lower-extremity amputation (LEA), but few studies have investigated which specific risk factors are most associated with LEA. We sought to develop and validate a risk score to aid in the early identification of patients hospitalized for diabetic foot infection who are at highest risk of LEA.

RESEARCH DESIGN AND METHODS

Using a large, clinical research database (CareFusion), we identified patients hospitalized at 97 hospitals in the U.S. between 2003 and 2007 for culture-documented diabetic foot infection. Candidate risk factors for LEA included demographic data, clinical presentation, chronic diseases, and recent previous hospitalization. We fit a logistic regression model using 75% of the population and converted the model coefficients to a numeric risk score. We then validated the score using the remaining 25% of patients.

RESULTS

Among 3,018 eligible patients, 21.4% underwent an LEA. The risk factors most highly associated with LEA (P < 0.0001) were surgical site infection, vasculopathy, previous LEA, and a white blood cell count >11,000 per mm³. The model showed good discrimination (c-statistic 0.76) and excellent calibration (Hosmer-Lemeshow, P = 0.63). The risk score stratified patients into five groups, demonstrating a graded relation to LEA risk (P < 0.0001). The LEA rates (derivation and validation cohorts) were 0% for patients with a score of 0 and ~50% for those with a score of ≥21.

CONCLUSIONS

Using a large, hospitalized population, we developed and validated a risk score that seems to accurately stratify the risk of LEA among patients hospitalized for a diabetic foot infection. This score may help to identify high-risk patients upon admission.

Lower-extremity amputation (LEA) is one of the complications of diabetes that is perhaps most feared by patients with this disease (1) and rightfully so. These LEAs are generally the end point of a characteristic sequence of events: a foot wound, usually a consequence of peripheral neuropathy, becomes infected and does not respond to treatment (2). More than 60% of nontraumatic LEAs in the U.S. occur among people with diabetes, in whom the rate is 6 to 10 times higher than for people without diabetes (3). After a first LEA, up to 50% of patients require another amputation within 3–5 years. Furthermore, the 5-year mortality after LEA is ~50% (4), with the risk considerably higher for diabetic compared with nondiabetic patients (5).

Considering the substantial morbidity and mortality associated with LEA in people with diabetes, the ability to identify which patients hospitalized for a diabetic foot infection are at highest risk for this complication could help clinicians direct special prevention efforts to these individuals. This information also could help identify the baseline risk for LEA among patients admitted to a medical center, allowing fairer comparisons of amputation rates at different centers. Although the factors associated with diabetic people developing a foot ulcer are well defined (1), risk factors for amputation are less clear. Previous studies have identified independent risk factors that include (in approximate order of odds ratio) a history of a foot ulcer (6), limb ischemia, underlying bone involvement, the presence of gangrene (e.g., a higher Wagner grade), deep wounds, older age, elevated inflammatory markers (7), poor glycemic control (8), a specific ethnicity or geographical region (9,10), nephropathy (8), and retinopathy (6). To determine whether we could develop and validate a scoring system to predict the risk of LEA, we examined data from a large group of patients hospitalized with a diabetic foot infection.

RESEARCH DESIGN AND METHODS

We used data from a clinical research database of patients hospitalized at 97 acute-care hospitals in the U.S. that was compiled by CareFusion (Department of Clinical Research, CareFusion, Marlborough, MA). The database includes extensive data in the following categories: clinical (including diagnoses and vital signs); laboratory (e.g., chemistry, hematology, and microbiology); and administrative (e.g., demographics, admission source, length of hospitalization, and discharge status). Eligible patients were those discharged from one of the designated hospitals between 1 January 2003 and 30 June 2007 with a principal diagnosis (ICD-9-CM) of diabetes and a secondary diagnosis indicating skin or soft-tissue infection (including cellulitis, infected ulcer, or surgical-site infection [SSI]) that was culture documented within 48 h of admission. This study was approved by the New England Institutional Review Board Human Subjects Research Committee (Wellesley, MA) and was conducted in compliance with the Health Insurance Portability and Accountability Act.

Study design and statistical analysis

We randomly split the study population into two groups: one for model derivation (75% of the population) and the other for model validation (the remaining 25%). Candidate predictor variables used in the model were demographic data, a history of previous health care–associated infection, clinical presentation, concomitant chronic disease(s), previous LEA, and type of skin or soft-tissue infection (cellulitis, ulcer or other infection, or SSI). Our outcome measure of interest was LEA during the index hospitalization, which was identified by ICD-9 procedure codes (841.1 amputation, toe; 841.2 amputation, foot; 841.3 disarticulation, ankle; 841.4 amputation, Malleoli; 841.5 amputation, knee, below NEC [not elsewhere classified]; 841.6 disarticulation, knee; and 841.7 amputation, knee, above).

Using the derivation cohort, we conducted univariate analyses to determine the proportion of patients with each candidate predictor who underwent an LEA. We then fit a stepwise multivariate logistic regression model to identify independent predictors of LEA and to estimate their relative predictive weights (coefficients). Using previously published methods, we converted the coefficients for the independent predictors into a simplified risk score system (11). Specifically, we calculated the number of points assigned to each variable by dividing its regression coefficient by the smallest coefficient in the model then rounded this quotient to the nearest whole number. We then calculated each subject’s LEA risk score by summing up the points of all variables present on admission. We then validated the risk score system using the remaining 25% of the population.

We assessed model discrimination using the c-statistic, which defines how well a model or prediction rule can discriminate between patients who do and do not have an event and measures how well a clinical prediction rule correctly ranks patients in order by risk. We assessed model calibration using the Hosmer-Lemeshow goodness-of-fit test, which assesses whether the observed and expected event rates match in subgroups of the model population. The test specifically identifies subgroups as the deciles of fitted risk values, and models with similar expected and observed event rates (i.e., a large P value) are considered to be well calibrated. We then used the Cochran-Armitage trending statistic, which modifies the χ² test to incorporate a suspected ordering, to assess the ability of the risk score system to differentiate low-risk from high-risk patients in a graded response. All analyses were conducted using Statistical Analysis System (version 9.01; SAS Institute, Cary, NC).

RESULTS

Patient characteristics

Among hospitalized patients, 3,018 met our inclusion criteria for a diabetic foot infection, with cellulitis (in 80%) and an infected ulcer (16%) being the most common diagnoses. We used 2,230 patients for the derivation cohort and 788 for the validation cohort; all baseline characteristics were similar between the two cohorts (Table 1). A total of 646 (21.4%) patients underwent an LEA during their index hospitalization; the number (and rate) in the derivation cohort was 463 (20.8%) compared with 183 (23.2%) for the validation cohort. For those patients undergoing an LEA, the median time from admission to amputation was 4 days, with an interquartile range of 2–7 days. A previous LEA of some type was noted in ~27% of patients in the derivation cohort and 26% in the validation cohort.

Table 1.

Characteristics for patients in the derivation and the validation cohorts

Variable	Derivation cohort	Validation cohort
n	2,230	788
Mortality (death during hospitalization)	30 (1.3)	10 (1.3)
Amputation during index hospitalization	463 (20.8)	184 (23.3)
Median age (years [first through third quartiles])	60 (50–71)	60 (50–71)
Male sex	1,359 (60.9)	493 (62.6)
Previous admission within ≤30 days	214 (9.6)	68 (8.6)
Transferred from an acute-care hospital	19 (0.9)	8 (1.0)
Transferred from a skilled nursing facility	72 (3.2)	32 (4.1)
Race/ethnicity
White	1,574 (70.6)	593 (75.3)
Black	420 (18.8)	115 (14.6)
Other	236 (10.6)	80 (10.2)
Comorbidities
Congestive heart failure	522 (23.4)	171 (21.7)
History of coronary disease	532 (23.9)	193 (24.5)
Immunosuppressive medication	73 (3.3)	33 (4.2)
Cancer	46 (2.1)	12 (1.5)
Peripheral vascular disease	807 (36.2)	287 (36.4)
Chronic liver disease	31 (1.4)	9 (1.1)
Chronic lung disease	232 (10.4)	97 (12.3)
Previous stroke	234 (10.5)	74 (9.4)
Chronic renal disease	445 (20.0)	153 (19.4)
History of amputation	611 (27.4)	203 (25.8)
Renal dialysis treatment	53 (2.4)	23 (2.9)
Type of skin and soft-tissue infection
Cellulitis	1,788 (80.2)	629 (79.8)
Infected ulcer	360 (16.1)	129 (16.4)
Surgical site	82 (3.7)	30 (3.8)
Severe infection clinical presentation
Systolic blood pressure <100 mmHg	293 (13.1)	110 (14.0)
Temperature <96°F or >100.5°F	681 (30.5)	238 (30.2)
Pulse <49 or >125 bpm	128 (5.7)	40 (5.1)
Respiration <10 or >29 breaths per minute	86 (3.9)	35 (4.4)
Altered mental status	173 (7.8)	66 (8.4)
Laboratory results
Albumin <2.8 g/dL	237 (10.6)	95 (12.1)
Blood urea nitrogen >40 mg/dL	399 (17.9)	121 (15.4)
Creatinine >3 mg/dL	176 (7.9)	65 (8.2)
Sodium >145 mEq/dL	24 (1.1)	11 (1.4)
Total bilirubin >0.8 mg/dL	206 (9.2)	89 (11.3)
pO2 <55 or >140 or O2 sat <90%	37 (1.7)	10 (1.3)
Prothrombin time international normalized ratio >1.2 or prothrombin time >14 s	209 (9.4)	68 (8.6)
Bands on leukocyte differential >13%	80 (3.6)	29 (3.7)
White blood cell count >11,000 per mm3	1,037 (46.5)	397 (50.4)
Glucose on admission (mg/dL)
≤70	100 (4.5)	35 (4.4)
71–135	331 (14.8)	124 (15.7)
136–240	717 (32.2)	236 (29.9)
>240	1,082 (48.5)	393 (49.9)

Data are n (%), unless otherwise indicated. The P values for each variable is >0.05, indicating that the derivation and validation cohorts are similar.

For the entire study cohort, the patients who underwent an LEA were significantly older (median age in years [interquartile range] 62 [53–72] vs. 60 [50–71]; P < 0.0001), and their in-hospital mortality rate was significantly higher (2.3 vs. 1.1%; P < 0.05) compared with patients who did not require amputation. The most common finding on culture was a polymicrobial (two or more different microorganisms) infection, which accounted for ~57% of all patients. A detailed accounting of pathogen distribution is shown in Supplementary Appendix A.

Univariate analysis of risk factors associated with LEA

As shown in Table 2, in the derivation cohort the univariate analysis revealed that the following factors were significantly associated with LEA (P < 0.05): older age; male sex; transfer from another hospital or nursing home; previous LEA; coronary, renal, or peripheral vascular disease; low serum albumin; elevated values for white blood cell count, prothrombin time or international normalized ratio, or creatinine; elevated body temperature; the presence of a foot ulcer; and the presence of an SSI.

Table 2.

Univariate analysis of risk factors associated with LEA in the derivation cohort

Variable	Derivation cohort (% [n LEA/n evaluable]) (n = 2,230)	P*
n cases	20.8 (463/2,230)
Mortality (death during hospitalization)	33.3 (10/30)	0.1094
Age ≥50 years	23.1 (394/1,708)	<0.0001
Male sex	22.3 (303/1,359)	0.0282
Previous admission ≤30 days	24.8 (53/214)	0.1322
Transferred from an acute-care hospital	63.2 (12/19)	0.0001
Transferred from a skilled nursing facility	40.3 (29/72)	0.0002
Comorbidities
Congestive heart failure	24.3 (127/522)	0.0227
History of coronary disease	26.1 (139/532)	0.0006
Immunosuppressive medication	26.0 (19/73)	0.3032
Cancer	23.9 (11/46)	0.5829
Peripheral vascular disease	32.2 (260/807)	<0.0001
Chronic liver disease	25.8 (8/31)	0.5033
Chronic lung disease	24.1 (56/232)	0.1993
Previous stroke	23.9 (56/234)	0.2025
Chronic renal disease	28.8 (128/445)	<0.0001
History of amputation	31.3 (191/611)	<0.0001
Renal dialysis treatment	41.5 (22/53)	0.0005
Type of skin and soft tissue infection		<0.0001
Cellulitis	16.9 (302/1,788)
Infected ulcer	32.8 (118/360)
Surgical site	52.4 (43/82)
Acute clinical presentation
Systolic blood pressure <100 mmHg	24.6 (72/293)	0.0892
Temperature <96°F or >100.5°F	27.0 (184/681)	<0.0001
Pulse <49 or >125 bpm	24.2 (31/128)	0.3138
Respiration <10 or >29 breaths per minute	25.6 (22/86)	0.2776
Altered mental status	23.7 (41/173)	0.3293
Laboratory results
Albumin <2.8 g/dL	36.7 (87/237)	<0.0001
Blood urea nitrogen >40 mg/dL	23.6 (94/399)	0.1341
Creatinine >3 mg/dL	35.8 (63/176)	<0.0001
Sodium >145 mEq/dL	25.0 (6/24)	0.6133
Total bilirubin >0.8 mg/dL	23.8 (49/206)	0.2791
pO2 <55 or >140 or O2 sat <90%	21.6 (8/37)	0.8398
Prothrombin time international normalized ratio >1.2 or prothrombin time >14 s	37.8 (79/209)	<0.0001
Bands on leukocyte differential >13%	26.3 (21/80)	0.2092
White blood cell count >11,000 per mm3	30.3 (314/1,037)	<0.0001
Glucose on admission (mg/dL)		0.0603
≤70	12.0 (12/100)
71–135	18.1 (60/331)
136–240	22.0 (158/717)
>240	21.5 (233/1,082)

*Fisher exact test.

Multivariable LEA predictive model

Using stepwise regression analysis, we found 11 independent predictors of LEA (Table 3). The most highly significant (P < 0.0001) were SSI, vasculopathy, previous LEA, and white blood cell count >11,000 per mm³. The predictive model developed using these predictors had very good discrimination (c-statistic 0.76) and excellent calibration between predicted and observed LEA rates (Hosmer-Lemeshow test showing that they did not significantly differ across risk deciles; P = 0.63) (Fig. 1). Patients in the highest risk decile had a predicted probability of LEA of 59.4% and an observed LEA rate of 58.7%, whereas those in the lowest decile had a predicted probability of LEA of 4% and an observed LEA rate of 5%. The predictive model yielded a good calibration when applied to the validation cohort (Hosmer-Lemeshow test; P = 0.33).

Table 3.

Multivariable model and risk score for LEA

Risk factor	Coefficient	Odds ratio (95% CI)	P	Risk score weight*
Chronic renal disease or creatinine >3 mg/dL†	0.1372	1.15 (0.89–1.49)	0.2998	1
Male sex†	0.1988	1.22 (0.97–1.54)	0.0963	1
Temperature <96°F or >100.5°F	0.2830	1.33 (1.05–1.68)	0.0187	2
Age ≥50 years	0.5477	1.73 (1.28–2.34)	0.0004	4
Infected ulcer versus cellulitis	0.5168	1.68 (1.27–2.21)	0.0002	4
History of amputation	0.5020	1.65 (1.29–2.11)	<0.0001	4
Albumin <2.8 g/dL	0.6203	1.86 (1.35–2.56)	0.0001	5
History of peripheral vascular disease	0.7485	2.11 (1.66–2.69)	<0.0001	5
White blood cell count ≥11 (1,000 per mm3)	0.9596	2.61 (2.07–3.30)	<0.0001	7
Surgical site vs. cellulitis	1.3845	3.99 (2.44–6.55)	<0.0001	10
Transferred from other acute-care facility	1.6418	5.16 (1.78–15.02)	0.0026	12

*We used the method described by Sullivan et al. (11) to calculate the risk score weight: Step 1: divide each regression coefficient by the smallest coefficient in the model (in our model, this is chronic renal disease or creatinine >3 mg/dL). Step 2: round this quotient to the nearest whole number. For example, to calculate the score weight of male sex, we divided its coefficient of 0.1988 by 0.1371, resulting in a quotient of 1.44. Rounding this quotient to its nearest integer resulted in 1 for the score weight of this variable. We then calculated each subject’s overall LEA risk score by summing the points of all variables present on admission.

†We retained these two variables for clinical plausibility despite the fact that they are not statistically significant at the 0.05 level in the model.

Figure 1.

Comparison of the predicted probability of LEA against the observed amputation rate for both derivation and validation cohorts, by decile. The diagonal line represents perfect correlation of predicted and observed LEA rates. Model Hosmer-Lemeshow goodness-of-fit test χ² = 6.2, P = 0.63 vs. χ² = 9.2, P = 0.33 for the derivation vs. the validation cohorts, indicating excellent fit of the model.

Simplified risk score strata

For each patient, we summed all the variables present on admission to create an LEA risk score. We then grouped the risk scores into five risk strata. The use of a five-level risk strata allows easy application of risk stratification for comparisons of outcomes. LEA rates in the derivation and validation cohorts increased significantly by risk score strata (P < 0.0001 by the Cochran-Armitage trending test for both derivation and validation cohorts) (Fig. 2). For the derivation cohort, the risk of LEA for patients aged <50 years and without any of the other 10 factors in the risk score system was essentially zero. In contrast, for those whose score was ≥21, LEA risk was ~50%. The findings in the validation cohort were similar to those in the derivation cohort.

Figure 2.

Observed LEA rates by risk score strata for both derivation and validation cohorts (Cochran-Armitage trending test, P < 0.0001).

CONCLUSIONS

In this large cohort of patients hospitalized for a diabetic foot infection, more than one-fifth required an LEA. In reviewing numerous clinical and laboratory variables present at hospital admission in our derivation cohort, we identified 11 significant independent risk factors for LEA. By rounding the logistic regression coefficients into integers, we developed a simple LEA risk score system with five strata that we demonstrated was highly predictive of the risk for LEA. Using the patients in the validation cohort, we were then able to demonstrate that this risk score was indeed valid and well calibrated.

Most of the factors included in our risk score have been reported as risks for LEA in smaller, previously published studies (7,12,13). The presence of infection and peripheral vascular disease are the most powerful predictors. Most patients with diabetic foot ulcers do not have a fever or leukocytosis (14), which define a severe infection according to the Infectious Diseases Society of America and the International Working Group on the Diabetic Foot criteria. These severe diabetic foot infections are associated with a greater risk of LEA than those of mild or moderate severity. Previous studies (15,17) also have identified renal insufficiency as being associated with an increased risk of amputations. Other studies have identified increasing age (18), male sex (16), and hypoalbuminemia (19) as risks for LEA. Likewise, a previous LEA is a strong risk factor predicting the need for another amputation (12). In none of these previous studies, however, did the authors attempt to construct a scoring system to predict amputation risk for both men and women.

In a previous study (20), we showed that patients with SSIs who were transferred from another acute-care facility had worse clinical and economic outcomes, perhaps because patients with infections of greater severity are more likely to be transferred to hospitals with more intensive resources or greater expertise. A recent meta-analysis (21) demonstrated a direct association between hyperglycemia (as measured by hemoglobin A_1c) and LEA. Unfortunately, we did not have hemoglobin A_1c values on most of our patients, but we did note a nonsignificant trend toward higher amputation rates in those with increased blood glucose levels on admission. All of our patients had an infection; therefore, that variable was not among those included in our scoring system. It is noteworthy that the type of infection was associated with amputation risk, with SSIs at the highest risk, followed by infected ulcers, when compared with cellulitis. Our assumption is that these SSIs may be associated with failed lower-extremity bypass procedures. We found no other studies that investigated the risk of adverse outcomes in patients with an infected ulcer compared with cellulitis or SSIs. We did find other studies (7,14,22,23) that reported that deep infections (especially those involving bone) and necrotizing infections more often resulted in amputations.

The simplified five risk strata that we devised correlated strongly with LEA rates. This may have important clinical implications on how to allocate resources. In particular, a patient with a low score may need fewer medical resources than a patient with a high LEA risk score. At the other extreme, to try to avoid the tragedy of amputation, health care providers should concentrate efforts on a patient with a risk score of >21, who has a 50% chance of an LEA. Our finding that patients transferred from another acute-care hospital had the highest odds ratio for LEA highlights the need of risk adjustment to appropriately evaluate outcomes for hospitals treating the most severe patients. Because LEA rates are sometimes used to compare quality of care for patients with diabetic foot complications, our risk adjustment score could be used to ensure that centers treating higher-risk patients are not unfairly penalized. Furthermore, although this has not been tested, the score might be helpful to clinicians in deciding which patients with diabetic foot infections may need to be hospitalized. The LEA risk score system has the benefit of being simple to use; each of the risk factors is readily available, usually at the time of admission or soon afterward.

Our study is limited by the fact that our analysis was retrospective, and, although fairly inclusive, we could have missed potentially significant factors. For example, the individual reasons for amputation and whether amputation was elective or urgent were not captured in the database. One major risk factor that we did not capture that could have an effect on our risk score is a history of a previous lower-extremity revascularization procedure, which was a significant factor in other reports (24,25). Selection bias is another potential limitation when using administrative data to identify patients with skin or soft-tissue infections. To minimize potential bias related to the use of ICD-9 coding, we limited our study to culture-confirmed infections.

In conclusion, we used a large clinical database to develop and validate a risk score that seems to accurately stratify LEA risk among patients hospitalized for a diabetic foot infection. This score may help clinicians identify patients at highest risk of LEA upon admission. Once patient identification is achieved, methods to reduce the risk can be investigated. We would like to see our risk score validated prospectively, including in patients treated on an outpatient basis.