ABSTRACT
Background
Non-traumatic lower extremity amputation (LEA) is a severe complication during dialysis. To inform decision-making for physicians, we developed a multivariable prediction model for LEA after starting dialysis.
Methods
Data from the Swedish Renal Registry (SNR) between 2010 and 2020 were geographically split into a development and validation cohort. Data from Netherlands Cooperative Study on the Adequacy of Dialysis (NECOSAD) between 1997 and 2009 were used for validation targeted at Dutch patients. Inclusion criteria were no previous LEA and kidney transplant and age ≥40 years at baseline. A Fine–Gray model was developed with LEA within 3 years after starting dialysis as the outcome of interest. Death and kidney transplant were treated as competing events. One coefficient, ordered by expected relevance, per 20 events was estimated. Performance was assessed with calibration and discrimination.
Results
SNR was split into an urban development cohort with 4771 individuals experiencing 201 (4.8%) events and a rural validation cohort with 4.876 individuals experiencing 155 (3.2%) events. NECOSAD contained 1658 individuals experiencing 61 (3.7%) events. Ten predictors were included: female sex, age, diabetes mellitus, peripheral artery disease, cardiovascular disease, congestive heart failure, obesity, albumin, haemoglobin and diabetic retinopathy. In SNR, calibration intercept and slope were –0.003 and 0.912, respectively. The C-index was estimated as 0.813 (0.783–0.843). In NECOSAD, calibration intercept and slope were 0.001 and 1.142 respectively. The C-index was estimated as 0.760 (0.697–0.824). Calibration plots showed good calibration.
Conclusion
A newly developed model to predict LEA after starting dialysis showed good discriminatory performance and calibration. By identifying high-risk individuals this model could help select patients for preventive measures.
Keywords: amputation, chronic kidney disease, dialysis, external validation, prediction
Graphical Abstract
Graphical Abstract.
KEY LEARNING POINTS.
What was known:
Patients on dialysis are at significant risk of lower extremity amputations (LEA), associated with significant morbidity and mortality.
Measures such as proactive foot care have shown to be effective in preventing LEA.
To our knowledge, no prediction for LEA for patients starting dialysis has been developed.
This study adds:
We developed and externally validated a model to predict LEA in patients starting dialysis.
Our model appears to be well-suited to identify patients at low and high risk.
By using both a Swedish and Dutch cohort, we show our model works in different populations.
Potential impact:
Our model could help to identify high-risk patients in order to initiate preventive measures.
Our model could be a first step for future research on preventative measures by allowing researchers to select those patients most at risk.
INTRODUCTION
Chronic kidney disease (CKD) affects more than 800 million individuals worldwide and is projected to be the fifth leading cause of death by 2040 [1]. Patients with end-stage kidney disease (ESKD) that require dialysis experience a very high burden of morbidity and mortality [2]. In addition to their high mortality and risk of cardiovascular disease, patients on dialysis are also at high risk of foot ulceration and non-traumatic lower extremity amputation (LEA). Studies show a prevalence of LEA ranging from 1.7% to 13.4% [3–5]. In patients with diabetes mellitus, dialysis is an even greater risk factor for LEA than previous ulceration [6]. Dialysis is also an independent risk factor for foot ulceration and recent commencement of dialysis increases the incidence of ulceration [7–9].
The consequences of an LEA for patients are significant. A recent study found that an LEA in dialysis patients was associated with a 4-fold increase of mortality risk [10]. Another study found a median survival of 16 months after amputations below knee and 6 months after amputations above knee [11]. Amputations have also been associated with significant morbidity and lead to decreased functional independence and quality of life [12, 13].
Treatment of foot ulceration includes relieving of high-pressure areas, revascularization and aggressive control of infection. If this is not effective, an LEA will have to be performed. Some studies have focused on the prevention of LEA. Marn Pernat et al. found that in diabetes patients on dialysis, monthly foot checks resulted in a 17% decrease in amputations [14]. Education programs have been shown to improve foot self-care knowledge and behavior [15]. Another study found that aggressive treatment of peripheral artery disease decreased the incidence of LEA [16].
To our knowledge, no prediction model has been developed to predict LEA in patients starting dialysis. Being able to identify patients at risk would help the physician to select patients for preventive measures. Additionally, despite the high incidence of LEA in patients on dialysis, no larger trials focused on prevention have been performed. Our model could help make these future trials more cost-effective. Therefore, we aimed to develop and validate a multivariable prognostic model to predict LEA in patients on chronic dialysis.
MATERIALS AND METHODS
We followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement for reporting (Supplementary data) [17].
Data sources
Data from the Swedish Renal Registry (SNR) were used for model development and geographic validation. The SNR is an ongoing registry of patients with CKD since 2008. Registration is mandatory for the nephrologist once a patient has an estimated glomerular filtration rate <30 mL/min/1.73 m2. The registry contains information on aetiology of kidney disease, demographics, type of kidney replacement therapy, vascular access, blood pressure measurements and laboratory values. Yearly coverage of Swedish dialysis clinics ranges from 98% to 100% [18]. By linking the SNR to national registries, medication use, comorbidities and procedures were identified through International Classification of Diseases, Tenth Revision, KKÅ and ATC codes. A list of definitions and how predictors were exactly derived can be found in Supplementary data, Table S1. The national registries automatically include every Swedish resident. Patients are informed of these registries and are allowed to opt out.
The Netherlands Cooperative Study on the Adequacy of Dialysis (NECOSAD) was used for a targeted (external) model validation in Dutch dialysis patients. NECOSAD is a multicentre cohort study in 38 dialysis centres in the Netherlands from 1 January 1997 until 1 January 2007. Patient follow-up lasted until the 26 May 2009. Patients with incident ESKD were included at the time of initiation of dialysis. Study visits took place at the start of dialysis, at 3 and 6 months, and subsequently at 6-month intervals until death, kidney transplant or loss to follow-up. Data on demographics, comorbidities, medication use and laboratory measurements were collected on the first visit. Each subsequent visit patients were asked if they had been admitted to the hospital and/or if they had received surgery. Reasons for admission and types of surgery were registered. All patients in the study gave informed consent for inclusion [19].
Participants
All patients starting haemo- or peritoneal dialysis were extracted from the SNR. Baseline was set at start of dialysis. The minimum age of inclusion into our study was ≥40 years at baseline. Patients with prior amputation and/or kidney transplantation were excluded from the analysis. Patients who started dialysis before 1 January 2010 were excluded because the medication registry does not contain accurate data for these patients. Similarly, to exclude interference of the COVID-19 pandemic administrative censoring was set at 1 March 2020. The dataset was geographically split between urban counties functioning as development cohort and rural counties functioning as geographical validation cohort targeted at rural areas.
In NECOSAD, all patients with an age of ≥40 years at baseline and without prior amputation and kidney transplantation were extracted. Baseline was set at 3 months after starting dialysis. We consider these patients to be chronic and on their ‘definitive’ dialysis mode.
Outcome
The outcome of interest for our model was non-traumatic lower extremity amputation within 3 years after baseline.
Covariates
In both cohorts, data on demographics, dialysis modality and vascular access were extracted at baseline. All relevant comorbidities registered before baseline were extracted. Relevant medications were extracted if dispensed within 180 days before baseline. Laboratory values and other clinical parameters were extracted based on how many days before or after baseline they were measured, as detailed in the Supplementary Methods.
Statistical analysis
Continuous values were described as means with standard deviations (SD) or as medians with interquartile ranges (IQR). Categorical values were summarized as counts with percentages. The incidence of amputation, kidney transplantation and death were visualized with cumulative incidence functions using the sub-distribution function, which accounts for competing events [20].
Missing data were imputed with multiple imputations (five times) using the MICE package in R before splitting of data into development and validation cohorts. We prespecified that predictors would not be used if they had more than 80% missing values. Number of values missing per covariate and all variables used in the imputation process are detailed in Supplementary data, Table S2.
Candidate predictors were identified based on available literature, clinical expertise, transportability between different settings, reliable registration and availability in daily clinical practice [4, 21–26]. We preselected a list of predictors and ordered them based on relevance (and therefore order of inclusion into the model). A list can be found in Supplementary data, Table S3. In NECOSAD, two slightly different predictor definitions had to be used. First, we used any peripheral artery disease as a proxy for symptomatic peripheral artery disease. Second, we used retinopathy with laser therapy as a proxy for diabetic retinopathy. We limited the number of coefficients based on available events, aiming to have 20 or more events available per estimated coefficient. Because the data were not primarily collected to develop our model, assessment of predictors was blind for the outcome and vice versa. A time to event Fine–Gray model was used to account for the presence of competing risks [20]. Death and kidney transplantation were treated as competing events. The coefficients were combined with the baseline cumulative incidence function (CIF) to create the final model.
The model assumptions were assessed through clinical reasoning and additional analyses where needed. Collinearity of predictors was examined through a correlation matrix using Spearman's rank correlation coefficients. Linearity of continuous predictors was checked by plotting them as a restricted cubic spline to the log relative hazard. The proportionality assumption was tested by visual inspection of the residuals plot.
Model performance was assessed through calibration and discrimination. Calibration plots were created by plotting the predicted to the observed event rate with censoring handled based on jack-knife pseudo-values [27]. A LOESS curve was fitted with a span of 0.33. Calibration intercepts and calibration slopes were also estimated, which in a perfectly calibrated model approach the values of 0 and 1, respectively. Because of the presence of competing events, the observed risk was estimated using the Aalen–Johansen estimator [28, 29].
Because of the presence of competing risk, discrimination was determined using an adaptation of the Harrell's C-index by Wolbers et al. Patients experiencing a competing event are not censored but retained in the risk set whilst setting their follow-up time to infinity, thus indicating that they will never experience the event of interest [28–30]. The cumulative/dynamic AUCt, proposed as a better measure of discrimination in single time-point prediction models, was estimated as well [31]. Confidence intervals for the C-index and the AUCt were calculated with 300 bootstraps.
We performed a number of sensitivity analyses in SNR. During these analyses we used the original model to perform additional validation analyses. First, because patients can still suffer an amputation after receiving a kidney transplant, an analysis which did not treat kidney transplantation as a competing or censoring event was done. Second, to assess performance in patients with previous amputations, an analysis was done which did not exclude these patients. Third, because patients that recover from dialysis might not be representative for the general dialysis population, an analysis was done which excluded these patients.
In NECOSAD, to assess performance if the model is used at dialysis initiation, we performed a sensitivity analysis with baseline set at start of dialysis instead of 3 months after start.
We used R version 4.2.1 (R Foundation for Statistical Computing, Vienna, Austria) for all statistical analyses.
Ethical approval
For SNR, patients were informed and had the possibility to opt-out. No additional individual consent is required for specific this research project. The study was approved by the ethical review board in Stockholm (Dnr 2018/1591-31/2). For NECOSAD, the institutional review board of the Academic Medical Hospital, Amsterdam, the Netherlands approved the study (MEC95/226a), and the institutional review boards of all participating hospitals provided additional local approval. All patients gave written informed consent. Furthermore, the protocol was approved by the scientific review board of the department of Clinical Epidemiology.
RESULTS
Baseline characteristics
We extracted data on 27 158 patients from the SNR, of which 9647 were identified to meet our inclusion criteria. SNR was then split into a development cohort with 4771 patients and a validation cohort with 4876 patients. We extracted 1955 patients from NECOSAD, of which 1658 were identified to meet our inclusion criteria (Fig. 1).
Figure 1:
Flow of patients through the study. Flowchart describing the number of patients being excluded in each step of the selection process. (A) Out of 27258 patients starting dialysis in the SNR, 9647 patients were included into the final analysis. (B) Out of 1995 patients in NECOSAD, 1658 were included in the final analysis.
Baseline characteristics of both SNR cohorts and the NECOSAD cohort are summarized in Table 1. The three most common comorbidities in SNR were hypertension (88.1%), diabetes mellitus (44.3%) and obesity (35.1%). Characteristics of the two SNR cohorts are similar. However, small differences are seen in the causes of CKD and the modality of dialysis.
Table 1:
Baseline characteristics of development and validation cohorts, before imputation.
| SNR overall | SNR development | SNR validation | NECOSAD | |
|---|---|---|---|---|
| Number of patients | 9647 | 4771 | 4876 | 1658 |
| Age, median (IQR) | 70 (60, 77) | 69 (59, 76) | 70 (61, 78) | 65 (55, 73) |
| Female (%) | 3207 (33.2) | 1563 (32.8) | 1644 (33.7) | 640 (38.6) |
| BMI (kg/m2), mean (SD) | 27.4 (5.7) | 27.3 (5.8) | 27.4 (5.6) | 24.9 (4.2) |
| Blood pressure, mean (SD) | ||||
| Systolic blood pressure (mmHg) | 144.1 (22.4) | 143.5 (22.5) | 144.7 (22.4) | 144.1 (22.4) |
| Diastolic blood pressure (mmHg) | 77.0 (12.8) | 76.9 (12.9) | 77.1 (12.7) | 77.0 (12.8) |
| Cause of CKD, n (%) | ||||
| Diabetic nephropathy | 2446 (25.4) | 1283 (26.9) | 1163 (23.9) | 246 (16.1) |
| Glomerulonephritis | 1138 (11.8) | 579 (12.1) | 559 (11.5) | 190 (12.5) |
| Hypertension/renovascular | 1895 (19.6) | 768 (16.1) | 1127 (23.1) | 317 (20.8) |
| Polycystic kidney disease | 696 (7.2) | 358 (7.5) | 338 (6.9) | 181 (11.9) |
| Pyelonephritis | 262 (2.7) | 149 (3.1) | 113 (2.3) | 113 (7.4) |
| Other | 1077 (11.2) | 557 (11.7) | 520 (10.7) | 200 (13.1) |
| Unspecified | 3188 (33.0) | 1737 (36.4) | 1451 (29.8) | 278 (18.2) |
| PD (%) | 1077 (11.2) | 557 (11.7) | 520 (10.7) | 526 (31.7) |
| HD vascular access, n (%) | ||||
| AV fistula | 2087 (39.3) | 923 (37.0) | 1164 (41.3) | – |
| Central venous catheter | 2946 (55.5) | 1402 (56.2) | 1544 (54.8) | – |
| Other | 14 (0.3) | 3 (0.1) | 11 (0.4) | – |
| Synthetic graft | 265 (5.0) | 166 (6.7) | 99 (3.5) | – |
| Laboratory values | ||||
| eGFR (mL/min/1.73 m2), median (IQR) | 6.3 (4.9, 8.3) | 6.3 (4.8, 8.5) | 6.3 (4.9, 8.2) | 6.4 (5.0,8.3) |
| HbA1C (mmol/mol), median (IQR) | 45.0 (37.8, 57.0) | 44.0 (36.0, 55.0) | 48.0 (39.0, 58.5) | – |
| Albumin (g/L), mean (SD) | 34.1 (5.7) | 32.8 (5.4) | 35.3 (5.7) | 35.9 (7.1) |
| Haemoglobin (mmol/L), mean (SD) | 6.7 (0.9) | 6.7 (0.8) | 6.8 (0.9) | 6.9 (1.0) |
| Calcium albumin corrected (mmol/L), mean (SD) | 2.4 (0.2) | 2.4 (0.2) | 2.4 (0.2) | 2.4 (0.3) |
| Phosphate (mmol/L), mean (SD) | 1.7 (0.5) | 1.7 (0.5) | 1.7 (0.5) | 1.8 (0.4) |
| PTH (pg/mL), median (IQR) | 23.2 (13.7, 38.0) | 23.0 (13.2, 38.0) | 24.0 (14.0, 38.0) | – |
| CRP (mg/L), median (IQR) | 5.3 (3.0, 15.0) | 5.0 (2.2, 13.0) | 6.0 (3.8, 15.0) | 6.0 (3.0,15.0) |
| Cholesterol total (mmol/L), mean (SD) | 4.4 (2.3) | 4.4 (1.9) | 4.4 (2.7) | 5.0 (1.3) |
| Cholesterol HDL (mmol/L), median (IQR) | 1.1 (0.9, 1.4) | 1.1 (0.9, 1.4) | 1.1 (0.9, 1.4) | – |
| Triglycerides (mmol/L), median (IQR) | 1.6 (1.2, 2.4) | 1.6 (1.2, 2.4) | 1.7 (1.2, 2.3) | – |
| Medication, n (%) | ||||
| Antiplatelets | 3831 (39.7) | 1839 (38.5) | 1992 (40.9) | – |
| Acetylsalicylic acid | 3548 (36.8) | 1706 (35.8) | 1842 (37.8) | 464 (31.1) |
| Clopidogrel | 532 (5.5) | 237 (5.0) | 295 (6.1) | – |
| Dipyridamole | 82 (0.9) | 42 (0.9) | 40 (0.8) | – |
| Antidiabetics | 3086 (32.0) | 1565 (32.8) | 1521 (31.2) | – |
| Non-insulin | 695 (7.2) | 372 (7.8) | 323 (6.6) | – |
| Insulin | 2719 (28.2) | 1371 (28.7) | 1348 (27.6) | 221 (14.9) |
| Statins | 5109 (53.0) | 2524 (52.9) | 2585 (53.0) | 454 (30.4) |
| Antihypertensives | 8740 (90.6) | 4352 (91.2) | 4388 (90.0) | 1390 (91.6) |
| ACE-I | 2322 (24.1) | 1215 (25.5) | 1107 (22.7) | 564 (34.0) |
| ARBs | 3179 (33.0) | 1567 (32.8) | 1612 (33.1) | 191 (11.5) |
| Calcium antagonists | 6707 (69.5) | 3330 (69.8) | 3377 (69.3) | 788 (47.5) |
| Beta blocker | 6748 (69.9) | 3338 (70.0) | 3410 (69.9) | 675 (40.7) |
| Thiazide diuretics | 603 (6.3) | 255 (5.3) | 348 (7.1) | 588 (35.5) |
| Comorbidities, n (%) | ||||
| Peripheral artery disease | 868 (9.0) | 448 (9.4) | 420 (8.6) | 216 (14.3) |
| Diabetes mellitus | 4270 (44.3) | 2171 (45.5) | 2099 (43.0) | 353 (23.3) |
| Cardiovascular disease | 3734 (38.7) | 1824 (38.2) | 1910 (39.2) | 449 (29.7) |
| Coronary artery disease | 2868 (29.7) | 1401 (29.4) | 1467 (30.1) | 344 (22.8) |
| Cerebral vascular disease | 1511 (15.7) | 744 (15.6) | 767 (15.7) | 148 (9.8) |
| Congestive heart failure | 2876 (29.8) | 1451 (30.4) | 1425 (29.2) | 191 (12.6) |
| Atrial fibrillation | 1924 (19.9) | 939 (19.7) | 985 (20.2) | – |
| Valvular disease | 790 (8.2) | 398 (8.3) | 392 (8.0) | – |
| Obesity | 1950 (35.1) | 952 (36.6) | 998 (33.9) | 161 (10.3) |
| Hypertension | 8499 (88.1) | 4247 (89.0) | 4252 (87.2) | 1390 (91.6) |
| Dyslipidemia | 2849 (29.5) | 1440 (30.2) | 1409 (28.9) | – |
| Bone fracture in the past year | 449 (4.7) | 241 (5.1) | 208 (4.3) | – |
| Malignancy in the past 10 years | 2291 (23.7) | 1181 (24.8) | 1110 (22.8) | 165 (10.9) |
Continuous values are described as means with standard deviations if normally distributed, or as medians with IQRs if skewed. Categorical values are summarizes as counts with percentages.
BMI, body mass index; PD, peritoneal dialysis; HD, haemodialysis; AV fistula, arteriovenous fistula; eGFR, estimated glomerular filtration rate; PTH, parathyroid hormone; CRP, C-reactive protein; HDL, high-density lipoprotein; ACE-I, ACE inhibitors; ARBs, angiotensin II receptor blockers; –, no data available.
Compared with SNR, patients in the NECOSAD cohort on average were younger (65 vs 70 years) and their cause of CKD was less often classified as diabetic nephropathy (16.1% vs 25.4%). Regarding comorbidities, they less often suffered from diabetes mellitus (23.3% vs 44.3%), cardiovascular disease (29.7% vs 38.7%), congestive heart failure (12.6% vs 29.8%) and obesity (10.3% vs 35.1%), but more often had peripheral artery disease (14.3% vs 9.0%). Regarding medication use, patients in NECOSAD also less often used statins (30.4% vs 53.0%), acetylsalicylic acid (31.1% vs 36.8%) and insulin (14.9% vs 28.2%).
Outcome incidence
The cumulative incidence functions showed a high incidence of competing events in both SNR and NECOSAD (Fig. 2).
Figure 2:
Cumulative incidence function of amputation, death or transplantation. (A) The cumulative incidence function accounts for competing events and censoring. It shows the cumulative probability of the three possible outcomes within 10 years: amputation as first event, kidney transplant as first event or death in the overall SNR cohort. (B) Cumulative incidence function for the NECOSAD cohort.
In SNR, we identified 356 non-traumatic LEA (3.7%). A single traumatic amputation was identified and excluded from the analysis. Fewer amputations occurred in the validation cohort (3.2%) compared with the development cohort (4.1%). Almost 70% of amputations were above the ankle. A total of 201 non-traumatic amputations were identified in the development cohort (Table 2).
Table 2:
Number and site of non-traumatic LEA within 3 years after starting dialysis.
| SNR overall | SNR development | SNR validation | NECOSAD validation | |
|---|---|---|---|---|
| Number of patients | 9647 | 4876 | 4771 | 1658 |
| Months to incident amputation, median (IQR) | 11.5 (6.1, 21.0) | 12.2 (6.8, 21.1) | 9.6 (5.6, 20.6) | 14.0 (8.2, 25.7) |
| Amputation, n (%) | 356 (3.7) | 201 (4.1) | 155 (3.2) | 61 (3.7) |
| Above knee, n (%) | 71 (20.0) | 39 (19.4) | 32 (20.6) | 8 (13.0) |
| Below knee, n (%) | 166 (46.8) | 89 (44.3) | 77 (49.7) | 23 (37.7) |
| Forefoot, n (%) | 33 (9.3) | 24 (11.9) | 9 (5.8) | 5 (8.2) |
| Toe, n (%) | 86 (24.2) | 49 (24.4) | 37 (23.9) | 20 (32.8) |
| Unspecified, n (%) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 5 (8.2) |
In NECOSAD, we identified 61 non-traumatic LEA (3.7%). Approximately half of amputations were above the ankle. Overall, 8.2% of amputations were classified as amputations of the ‘leg’ without specifying the exact site. Median time to amputation was longer in NECOSAD compared with SNR (14.0 vs 11.5 months) (Table 2).
Model development
After taking the number of events into account, we included the following 10 variables: female sex, age when starting dialysis, diabetes mellitus, symptomatic peripheral artery disease, cardiovascular disease (cerebral vascular disease or coronary artery disease), congestive heart failure, obesity, albumin, haemoglobin and diabetic retinopathy. The model assumptions were all met (Supplementary data, Table S4, and Figs S1 and S2). No variable transformations were deemed necessary.
The coefficients and the cumulative incidence at 3 years were combined into the following formula:
 |
If a comorbidity is present it is coded as 1, otherwise it is coded as 0. For example, a 56-year-old woman with an albumin of 25 g/L, a haemoglobin of 5 mmol/L and a history of diabetes and coronary artery disease would have a predicted risk of:
 |
Therefore, her predicted probability of suffering an LEA within 3 years after starting dialysis is 2.2%.
A 60-year-old obese man with an albumin of 20 g/L, a haemoglobin of 6 mmol/L, a history of diabetes, peripheral artery disease and congestive heart failure would have a predicted risk of:
Therefore, his predicted probability of suffering an LEA within 3 years after starting dialysis is 9.5%.
 |
Therefore, his predicted probability is 16.3%.
A summary of the cumulative incidence at 3 years and coefficient values including standard errors can be found in Supplementary data, Table S5.
Model validation
In the SNR validation cohort predicted probabilities ranged from 0.5% to 45.9%. For patients with diabetes mellitus type 2 the median (IQR) predicted probability was 6.6% (3.7%, 10.5%), for patients without diabetes mellitus type 2 this was 1.4% (1.1%, 1.9%). The distribution of all predicted probabilities can be found in Supplementary data, Fig. S3. The vast majority of patients (97.5%) had a predicted probability of less than 20%. These predictions are well calibrated. For the small number of patients at higher risk than 20% the model overpredicts (Fig. 3A). The calibration intercept and slope were –0.003 and 0.912, respectively. The C-index was estimated at 0.813 (0.783–0.843). AUCt was estimated at 0.818 (0.787–0.849) (Table 3).
Figure 3:
Calibration plot. (A) The predicted probability in the SNR validation cohort is plotted to the observed probability. The shaded area indicates the 95% confidence interval. The grey 45 degree line indicates perfect calibration. Histograms show the relative density of either amputation or other type event compared with the predicted probability. The cohort is also divided into 10 percentiles according to their predicted probability. The grey dots show the average predicted probability of each group plotted against the average observed frequency. (B) The same figure for the NECOSAD cohort.
Table 3:
Discrimination and calibration during main and sensitivity analyses.
| Cohort | C-index | AUCt | Calibration intercept | Calibration slope |
|---|---|---|---|---|
| Main analyses | ||||
| SNR validation (rural) | 0.813 (0.783–0.843) | 0.818 (0.787–0.849) | –0.003 | 0.912 |
| NECOSAD validation | 0.760 (0.697–0.824) | 0.764 (0.697–0.831) | 0.001 | 1.142 |
| Sensitivity analyses | ||||
| NECOSAD: baseline 0 months | 0.769 (0.697–0.841) | 0.770 (0.698–0.843) | 0.001 | 1.328 |
| SNR: recovered patients excluded | 0.811 (0.782–0.839) | 0.818 (0.786–0.849) | –0.003 | 0.911 |
| SNR: TX not censored/competing | 0.812 (0.782–0.842) | 0.818 (0.787–0.849) | –0.004 | 0.915 |
| SNR: previous TX not excluded | 0.799 (0.767–0.832) | 0.800 (0.766–0.834) | –0.002 | 0.934 |
TX, kidney transplantation.
In the NECOSAD cohort predicted probabilities ranged from 0.5% to 33.2%. The model was well calibrated in all ranges (Fig. 3B). The calibration intercept and slope were 0.001 and 1.142, respectively. The C-index was estimated at 0.760 (0.697–0.824). AUCt was estimated at 0.764 (0.697–0.831) in the NECOSAD cohort (Table 3).
The sensitivity analyses in SNR showed a slightly lower C-index in patients starting dialysis after failing a kidney transplant 0.799 (0.767–0.832). The sensitivity analysis in NECOSAD with baseline set at start of dialysis found a slightly higher C-index of 0.769 (0.697–0.841) but the model seemed to underpredict somewhat between 7% to 17% predicted probability. Other sensitivity analyses did not find any differences in performance (Table 3). Calibration plots can be found in Supplementary data, Fig. S4.
DISCUSSION
To our knowledge, we have developed the first model to predict the probability of an LEA in patients starting dialysis. In our cohorts, 3.7% of patients starting dialysis suffered an LEA within 3 years. The high incidence of LEA and the relative severity of the amputations, with 50%–70% of amputations performed above the ankle, underline the importance of this subject. As expected, median predicted probability was higher for patients with diabetes mellitus type 2 compared with those without (6.6% vs 1.4%). In the SNR validation (rural) cohort, our model was well calibrated for almost all patients and we found a C-index of 0.813 (0.783–0.843). While patients starting dialysis in the NECOSAD cohort were on average 5 years younger and had far fewer comorbidities, our model still performed relatively well with a C-index of 0.760 (0.697–0.824) and a calibration plot showing that the model was well calibrated in all ranges.
Models to predict LEA have traditionally focussed on patients suffering from diabetes mellitus or presenting with critical ischemia. A recent meta-analysis found 34 models to predict foot ulceration or LEA; however, although dialysis significantly increases the risk of LEA, none of these models included dialysis as predictor or inclusion criterion [23]. None of the mentioned models was applicable to our target population.
Due to the associated morbidity and mortality, prevention of LEA is important. The Dutch association of internal medicine (NIV) and International Working Group on the Diabetic Foot (IWGDF) guidelines recommend all diabetic patients on dialysis to visit the podiatrist every 1–3 months [32, 33]. Similarly, in the UK, the National Institute for Health and Care Excellence (NICE) guideline recommends a visit every 1 week to 2 months [34]. We feel our model could help physicians and diabetic nurses identify for which patients this frequency should be increased. Importantly, these guidelines have proposed frequent foot care for all dialysis patients since 2015. As a result, the predictions of our model reflect a setting in which half of the patients in SNR—who were included between 2010 and 2020–would already fall under this new guideline. The physician or diabetic nurse should account for this when deciding if a patient should receive a lower than recommended frequency of foot care based on low predicted risk [35]. The validation in the Dutch cohort is not affected by this since no patients were included after 2007 and our follow-up was limited to 3 years.
Further research is needed to explore the applicability of the model in different clinical settings of interest, and additional targeted validation studies are encouraged to assess its performance [36, 37]. Additionally, more research on prevention of amputations is necessary. Only one large study has examined the effect of routine foot checks on the incidence of LEA in patients on dialysis. While the authors found it decreased amputation rate by 17%, it did not have a control group and only examined patients suffering from diabetes [14]. Our model could provide the first step for future research, by allowing researchers to select those patients most at risk. In the meantime, the model can be a valuable tool to identify individual patients at high risk who are in need for more care.
Besides foot care, other possible interventions that could prevent LEA can be started. Stricter glycaemic targets in patients with diabetes should be considered [38]. More aggressive treatment of peripheral artery disease with statins is also a promising intervention [39]. Even a healthier lifestyle could be promoted by showing the patient insight into their risk of LEA.
Our study has a number of strengths. First, the high incidence of competing events shows that developing a Fine–Gray model instead of a Cox proportional hazards model was essential. The high mortality in our cohort would have led to overprediction if a Cox proportional hazards model had been used [28]. Second, by preselecting our predictors, using a large sample size and limiting the model to a minimum of 20 events per coefficient we minimized the risk of overfitting, supported by robust discrimination and calibration in two different validation cohorts. Third, performance stayed high even though patients in NECOSAD were younger and had far fewer comorbidities than those in SNR. Fourth, sensitivity analyses showed model performance is robust to our choices made during development.
There are some limitations. First, data on several potentially important predictors such as smoking, ethnicity, neuropathy, mineral bone disease and ulceration were not available. However, a part of the predictive effect of these predictors is likely included into our model through other predictors (e.g. someone who smokes is also more likely to have cardiovascular disease). Our results during validation show we do not need these predictors included into the model to generate accurate predictions. Future studies could try to add these predictors. Second, because the model is developed and validated in Northern/Western European countries it might be less applicable in regions with differing income levels, ethnic makeup, dialysis preferences or healthcare systems. Therefore, it requires further targeted validation in these settings. Third, because the NECOSAD cohort included patients from 1997 to 2007 it might be less representative of current care.
In conclusion, LEA are a severe complication during dialysis and occur frequently enough to warrant preventive measures. To our knowledge, we have developed the first model to predict the risk of amputations after starting dialysis. Our model was well suited to identify patients at low and high risk in Dutch and Swedish cohorts. Using our prediction model to identify high-risk individuals could help in the stratification of patients for preventive measures.
Supplementary Material
ACKNOWLEDGEMENTS
For NECOSAD, the nursing staff of the 38 different dialysis units, who collected most of the data, are gratefully acknowledged for their assistance. Moreover, the authors thank the staff of the NECOSAD trial office for assistance in the logistics of this study. The NECOSAD Study Group consisted of A.J. Apperloo, J.A. Bijlsma, M. Boekhout, W.H. Boer, P.J.M. van der Boog, H.R. Büller, M. van Buren, F.Th. de Charro, C.J. Doorenbos, M.A. van den Dorpel, A. van Es, W.J. Fagel, G.W. Feith, C.W.H. de Fijter, L.A.M. Frenken, J.A.C.A. van Geelen, P.G.G. Gerlag, W. Grave, J.P.M.C. Gorgels, R.M. Huisman, K.J. Jager, K. Jie, W.A.H. Koning-Mulder, M.I. Koolen, T.K. Kremer Hovinga, A.T.J. Lavrijssen, A.J. Luik, J. van der Meulen, K.J. Parlevliet, M.H.M. Raasveld, F.M. van der Sande, M.J.M. Schonck, M.M.J. Schuurmans, C.E.H. Siegert, C.A. Stegeman, P. Stevens, J.G.P. Thijssen, R.M. Valentijn, G. H. Vastenburg, C.A. Verburgh, H.H. Vincent and P.F. Vos.
Contributor Information
Bram Akerboom, Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands.
Roemer J Janse, Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands.
Aurora Caldinelli, Department of Clinical Science, Intervention and Technology, Karolinska Institutet, Stockholm, Sweden.
Bengt Lindholm, Department of Clinical Science, Intervention and Technology, Karolinska Institutet, Stockholm, Sweden.
Joris I Rotmans, Department of Internal Medicine, Division of Nephrology, Leiden University Medical Center, Leiden, The Netherlands.
Marie Evans, Department of Clinical Science, Intervention and Technology, Karolinska Institutet, Stockholm, Sweden.
Merel van Diepen, Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands.
FUNDING
B.A. was supported by a grant from the Leiden University Fund (LUF) International Study Fund (LISF). R.J.J. and M.v.D. were supported by a grant from the Dutch Kidney Foundation (20OK016). M.E. was supported by a grant from Stockholm City Council (ALF Medicine) and Center for Innovative Medicine (CIMED). B.L. was supported by a grant from Baxter Healthcare Corporation to Karolinska Institutet.
DATA AVAILABILITY STATEMENT
SNR and NECOSAD data in this article cannot be shared on a public repository due to privacy reasons. SNR data can be made available upon reasonable request to the main author after ethical review permission or through application to the SRR (www.snronline.se). NECOSAD data will be available (conditional on agreement on privacy matters and appropriate usage of the data) upon request. Contact information: Merel van Diepen, M.van_Diepen@lumc.nl or Friedo W. Dekker, F.W.Dekker@lumc.nl.
CONFLICT OF INTEREST STATEMENT
M.E. has received payment for lectures from Astellas Pharma, AstraZeneca, Vifor Pharma, Baxter Healthcare and Fresenius Medical Care, and receives advisory board payments from Astellas Pharma and AstraZeneca.
REFERENCES
- 1. Kovesdy CP. Epidemiology of chronic kidney disease: an update 2022. Kidney Int Suppl (2011) 2022;12:7–11. 10.1016/j.kisu.2021.11.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Johansen KL, Chertow GM, Foley RN et al. US Renal Data System 2020 Annual Data Report: epidemiology of kidney disease in the United States. Am J Kidney Dis 2021;77:A7–8. 10.1053/j.ajkd.2021.01.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Gilhotra RA, Rodrigues BT, Vangaveti VN et al. Non-traumatic lower limb amputation in patients with end-stage renal failure on dialysis: an Australian perspective. Ren Fail 2016;38:1036–43. 10.1080/0886022X.2016.1193872 [DOI] [PubMed] [Google Scholar]
- 4. Gilhotra RA, Rodrigues BT, Vangaveti VN et al. Prevalence and risk factors of lower limb amputation in patients with end-stage renal failure on dialysis: a systematic review. Int J Nephrol 2016;2016:4870749. 10.1155/2016/4870749 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Combe C, Albert JM, Bragg-Gresham JL et al. The burden of amputation among hemodialysis patients in the Dialysis Outcomes and Practice Patterns Study (DOPPS). Am J Kidney Dis 2009;54:680–92. 10.1053/j.ajkd.2009.04.035 [DOI] [PubMed] [Google Scholar]
- 6. Lavery LA, Lavery DC, Hunt NA et al. Amputations and foot-related hospitalisations disproportionately affect dialysis patients. Int Wound J 2015;12:523–6. 10.1111/iwj.12146 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Game FL, Chipchase SY, Hubbard R et al. Temporal association between the incidence of foot ulceration and the start of dialysis in diabetes mellitus. Nephrol Dial Transplant 2006;21:3207–10. 10.1093/ndt/gfl427 [DOI] [PubMed] [Google Scholar]
- 8. Mcgrath NM, Curran BA. Recent commencement of dialysis is a risk factor for lower-extremity amputation in a high-risk diabetic population. Diabetes Care 2000;23:432–3. 10.2337/diacare.23.3.432 [DOI] [PubMed] [Google Scholar]
- 9. Ndip A, Rutter MK, Vileikyte L et al. Dialysis treatment is an independent risk factor for foot ulceration in patients with diabetes and stage 4 or 5 chronic kidney disease. Diabetes Care 2010;33:1811–6. 10.2337/dc10-0255 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Schroijen MA, Van Diepen M, Hamming JF et al. Mortality after amputation in dialysis patients is high but not modified by diabetes status. Clin Kidney J 2020;13:1077–82. 10.1093/ckj/sfz116 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Arhuidese I, Nejim B, Aji EA et al. Survival after major lower extremity amputation in patients with end-stage renal disease. J Vasc Surg 2019;70:1291–8. 10.1016/j.jvs.2018.12.055 [DOI] [PubMed] [Google Scholar]
- 12. Arneja AS, Tamiji J, Hiebert BM et al. Functional outcomes of patients with amputation receiving chronic dialysis for end-stage renal disease. Am J Phys Med Rehabil 2015;94:257–68. 10.1097/PHM.0000000000000259 [DOI] [PubMed] [Google Scholar]
- 13. Wukich DK, Ahn J, Raspovic KM et al. Comparison of transtibial amputations in diabetic patients with and without end-stage renal disease. Foot Ankle Int 2017;38:388–96. 10.1177/1071100716688073 [DOI] [PubMed] [Google Scholar]
- 14. Marn Pernat A, Peršič V, Usvyat L et al. Implementation of routine foot check in patients with diabetes on hemodialysis: associations with outcomes. BMJ Open Diabetes Res Care 2016;4:e000158. 10.1136/bmjdrc-2015-000158 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Alshammari L, O'Halloran P, McSorley O et al. Health education programmes to improve foot self-care knowledge and behaviour among older people with end-stage kidney disease (ESKD) receiving haemodialysis (a systematic review). Healthcare (Basel) 2022;10:1143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Kiesz RS, Wiernek SL, Wiernek BK et al. Long-term results of plaque excision combined with aggressive pharmacotherapy in high-risk patients with advanced peripheral artery disease (SAVE a LEG registry). Catheter Cardiovasc Interv 2013;82:E244–50. 10.1002/ccd.24756 [DOI] [PubMed] [Google Scholar]
- 17. Moons KGM, Altman DG, Reitsma JB et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162:W1–73. 10.7326/M14-0698 [DOI] [PubMed] [Google Scholar]
- 18. Swedish Renal Registry . Annual report 2022. [Google Scholar]
- 19. Korevaar JC, Jansen MA, Dekker FW et al. When to initiate dialysis: effect of proposed US guidelines on survival. Lancet 2001;358:1046–50. 10.1016/S0140-6736(01)06180-3 [DOI] [PubMed] [Google Scholar]
- 20. Austin PC, Lee DS, Fine JP. Introduction to the analysis of survival data in the presence of competing risks. Circulation 2016;133:601–9. 10.1161/CIRCULATIONAHA.115.017719 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Bonnet J-B, Sultan A. Narrative review of the relationship between CKD and diabetic foot ulcer. Kidney Int Rep 2022;7:381–8. 10.1016/j.ekir.2021.12.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Abualhin M, Gargiulo M, Bianchini Massoni C et al. A prognostic score for clinical success after revascularization of critical limb ischemia in hemodialysis patients. J Vasc Surg 2019;70:901–12. 10.1016/j.jvs.2018.11.034 [DOI] [PubMed] [Google Scholar]
- 23. Beulens JWJ, Yauw JS, Elders PJM et al. Prognostic models for predicting the risk of foot ulcer or amputation in people with type 2 diabetes: a systematic review and external validation study. Diabetologia 2021;64:1550–62. 10.1007/s00125-021-05448-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. O'Hare AM, Bacchetti P, Segal M et al. Factors associated with future amputation among patients undergoing hemodialysis: results from the Dialysis Morbidity and Mortality Study Waves 3 and 4. Am J Kidney Dis 2003;41:162–70. 10.1053/ajkd.2003.50000 [DOI] [PubMed] [Google Scholar]
- 25. Farchioni L, Gennai S, Giuliani E et al. A prognostic risk score for major amputation in dialysis patients with chronic limb-threatening ischemia after endovascular revascularization. Int Angiol 2021;40:206–12. 10.23736/S0392-9590.21.04523-5 [DOI] [PubMed] [Google Scholar]
- 26. Kaminski MR, Lambert KA, Raspovic A et al. Risk factors for foot ulceration in adults with end-stage renal disease on dialysis: a prospective observational cohort study. BMC Nephrol 2019;20:423. 10.1186/s12882-019-1594-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Andersen PK, Pohar Perme M. Pseudo-observations in survival analysis. Stat Methods Med Res 2010;19:71–99. 10.1177/0962280209105020 [DOI] [PubMed] [Google Scholar]
- 28. Ramspek CL, Teece L, Snell KIE et al. Lessons learnt when accounting for competing events in the external validation of time-to-event prognostic models. Int J Epidemiol 2022;51:615–25. 10.1093/ije/dyab256 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Van Geloven N, Giardiello D, Bonneville EF et al. Validation of prediction models in the presence of competing risks: a guide through modern methods. BMJ 2022;377:e069249. 10.1136/bmj-2021-069249 [DOI] [PubMed] [Google Scholar]
- 30. Wolbers M, Blanche P, Koller MT et al. Concordance for prognostic models with competing risks. Biostatistics 2014;15:526–39. 10.1093/biostatistics/kxt059 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Blanche P, Kattan MW, Gerds TA. The c-index is not proper for the evaluation of $t$-year predicted risks. Biostatistics 2018;20:347–57. 10.1093/biostatistics/kxy006 [DOI] [PubMed] [Google Scholar]
- 32. Schaper NC, Van Netten JJ, Apelqvist J et al. Practical guidelines on the prevention and management of diabetic foot disease (IWGDF 2019 update). Diabetes Metab Res Rev 2020;36:e3266. 10.1002/dmrr.3266 [DOI] [PubMed] [Google Scholar]
- 33. Richtlijn diabetische voet 2017, Nederlands Internisten Vereniging (NIV).
- 34. Coppini D. New NICE guidelines on diabetic foot disease prevention and management. Pract Diabetes 2015;32:286. 10.1002/pdi.1974 [DOI] [Google Scholar]
- 35. Van Geloven N, Swanson SA, Ramspek CL et al. Prediction meets causal inference: the role of treatment in clinical prediction models. Eur J Epidemiol 2020;35:619–30. 10.1007/s10654-020-00636-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Van Calster B, Steyerberg EW, Wynants L et al. There is no such thing as a validated prediction model. BMC Med 2023;21:70. 10.1186/s12916-023-02779-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Wessler BS, Nelson J, Park JG et al. External validations of cardiovascular clinical prediction models: a large-scale review of the literature. Circ Cardiovasc Qual Outcomes 2021;14:e007858. 10.1161/CIRCOUTCOMES.121.007858 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Caruso P, Scappaticcio L, Maiorino MI et al. Up and down waves of glycemic control and lower-extremity amputation in diabetes. Cardiovasc Diabetol 2021;20:135. 10.1186/s12933-021-01325-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Sagris M, Katsaros I, Giannopoulos S et al. Statins and statin intensity in peripheral artery disease. Vasa 2022;51:198–211. 10.1024/0301-1526/a001012 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
SNR and NECOSAD data in this article cannot be shared on a public repository due to privacy reasons. SNR data can be made available upon reasonable request to the main author after ethical review permission or through application to the SRR (www.snronline.se). NECOSAD data will be available (conditional on agreement on privacy matters and appropriate usage of the data) upon request. Contact information: Merel van Diepen, M.van_Diepen@lumc.nl or Friedo W. Dekker, F.W.Dekker@lumc.nl.