Development of a prediction model for foot ulcer recurrence in people with diabetes using easy-to-obtain clinical variables

Abstract

Introduction

We aimed to develop a prediction model for foot ulcer recurrence in people with diabetes using easy-to-obtain clinical variables and to validate its predictive performance in order to help risk assessment in this high-risk group.

Research design and methods

We used data from a prospective analysis of 304 people with foot ulcer history who had 18-month follow-up for ulcer outcome. Demographic, disease-related and organization-of-care variables were included as potential predictors. Two logistic regression prediction models were created: model 1 for all recurrent foot ulcers (n=126 events) and model 2 for recurrent plantar foot ulcers (n=70 events). We used 10-fold cross-validation, each including five multiple imputation sets for internal validation. Performance was assessed in terms of discrimination using area under the receiver operating characteristic curve (AUC) (0–1, 1=perfect discrimination), and calibration with the Brier Score (0–1, 0=complete concordance predicted vs observed values) and calibration graphs.

Results

Predictors in model 1 were: a younger age, more severe peripheral sensory neuropathy, fewer months since healing of previous ulcer, presence of a minor lesion, use of a walking aid and not monitoring foot temperatures at home. Mean AUC for model 1 was 0.69 (2SD 0.040) and mean Brier Score was 0.22 (2SD 0.011). Predictors in model 2 were: a younger age, plantar location of previous ulcer, fewer months since healing of previous ulcer, presence of a minor lesion, consumption of alcohol, use of a walking aid, and foot care received in a university medical center. Mean AUC for model 2 was 0.66 (2SD 0.023) and mean Brier Score was 0.16 (2SD 0.0048).

Conclusions

These internally validated prediction models predict with reasonable to good calibration and fair discrimination who is at highest risk of ulcer recurrence. The people at highest risk should be monitored more carefully and treated more intensively than others.

Trial registration number

NTR5403.

Significance of this study.

What is already known about this subject?

• People with diabetes, neuropathy and foot ulcer history are in all ulcer risk classification systems stratified as those at highest risk; however, disease severity and ulcer risk vary substantially within this stratum. To provide appropriate preventative treatment and adequately allocate limited recourses, further differentiation for ulcer risk using easy-to-obtain clinical variables in this stratum is necessary.

What are the new findings?

• Predictors for foot ulcer recurrence were: a younger age, more severe peripheral sensory neuropathy, fewer months since healing of the previous ulcer, presence of a minor lesion, use of a walking aid and not monitoring foot temperature at home.

• Predictors for plantar foot ulcer recurrence were: a younger age, plantar location of the previous ulcer, fewer months since healing of the previous ulcer, presence of a minor lesion, consumption of alcohol, use of a walking aid, and foot care received in a university medical center.

• All predictors found can be easily obtained by healthcare professionals when screening people with diabetes and foot ulcer history.

How might these results change the focus of research or clinical practice?

• Clinical practice may use these prediction models to better determine which high-risk people with diabetes should be monitored more carefully and treated more intensively.

Introduction

A foot ulcer is a feared and common complication in people with diabetes mellitus. The presence of a foot ulcer has a major impact on an individual’s quality of life and places a large burden on both healthcare systems and society.^1–3 The annual incidence of a foot ulcer in people with diabetes is approximately 2%,⁴ while recently the global prevalence was estimated at 18.6 million (4.8% of all people with diabetes).⁵ Approximately 40% of the patients who heal from an ulcer develop another one within the first 12 months, and 60% within 3 years.⁶ Because of the high incidence of recurrence and subsequent risk of infection, hospital admission and amputation,⁷ prevention of ulcer recurrence is paramount.

In (inter)national guidelines and in current clinical practice, treatment to help prevent a foot ulcer starts with the determination of someone’s risk of developing one.^8–10 Various systems have been developed and validated to stratify people with diabetes according to their risk for ulceration.^11–13 Guidelines, such as those from the American Diabetes Association,⁹ or the International Working Group on the Diabetic Foot (IWGDF),¹⁰ recommend that people should be screened more frequently when their risk for ulceration increases. In all systems, people with neuropathy and a foot ulcer history are stratified as those at highest risk. However, within this risk group, disease severity and ulcer risk vary substantially.^14–19 To provide appropriate preventative treatment strategies and to adequately allocate limited recourses, it is important to further differentiate for foot ulcer risk within those people in the highest risk stratum.

Risk factor models can be used for this purpose. Armstrong and colleagues reviewed studies reporting risk factor models for ulcer recurrence,⁶ and found a vibration perception threshold >25 V,²⁰ a minor lesion,¹⁵ a previous ulcer at the plantar foot surface,^{14 21} and peripheral artery disease¹⁴ to be the most important independent risk factors. However, the risk models used in these studies are incomplete and inconsistent in description and interpretation and often lack validation. Also, these studies aimed to explain whether foot ulcer recurrence can reliably be attributed to a risk factor, after adjusting for confounders, in a multivariate analysis, that is, a search for causality. In contrast, prediction models use multiple variables to predict, as accurately as possible, the risk of a future outcome, regardless of causality.²² All causal factors are predictors, but not every predictor is a cause. Therefore, more and often easier to obtain variables can be considered in a prediction model than an etiological (causal) model, making prediction models more suitable in daily practice to timely identify people at high risk of ulcer recurrence.

Crawford and colleagues were the first to develop and externally validated a prediction model for people with diabetes at low to high risk of ulceration, and identified as predictors a history of ulceration, inability to feel a 10 g monofilament and absence of at least one pedal pulse.²³ These are easy-to-obtain variables in every clinical setting, which facilitates implementation. However, this model does not distinguish between the ulcer risk of people who are stratified as high risk (IWGDF grade 3). The only prediction model developed for high-risk people is our own, on data from 171 people with a recently healed plantar foot ulcer who participated in a trial on efficacy of custom-made footwear.²⁴ This model included quantitative biomechanical parameters that are not available in every clinical setting. Furthermore, this model focused on plantar foot ulcer recurrence only, while at least half of all foot ulcers develop at other anatomical locations.⁷ It also used a selected group of high-risk people who all wore custom-made footwear and had their previous foot ulcer healed within 18 months of study entry, limiting its generalizability and implementation. We therefore aimed to develop a prediction model for foot ulcer recurrence using a variety of more easy-to-obtain clinical variables in a diverse group of high-risk people with diabetes (all IWGDF risk 3), and to validate its predictive performance, so to help risk assessment and preventative treatment in this high-risk group.

Methods

Population

We used data from a multicenter randomized controlled trial (RCT) on the effectiveness of at-home monitoring of foot temperatures to prevent foot ulcer recurrence in diabetes.²⁵ In this trial, we recruited participants between 2015 and 2018 from multidisciplinary outpatient diabetic foot clinics of three university medical centers and four community hospitals, and several affiliated professional podiatry practices, all from various regions across the Netherlands. The participants received their primary foot care in the centers where they were recruited. From a total of 1411 people screened for eligibility, 304 participants with diabetes, loss of protective sensation (LOPS) and a history of foot ulceration (<48 months prior to enrollment) or a history of Charcot neuro-osteoarthropathy were included in this study. LOPS was assessed using a 10 g Semmes-Weinstein monofilament and a 128 Hz tuning fork.¹⁰ We defined a Charcot neuro-osteoarthropathy as a non-infectious destruction of bone and joint(s) associated with neuropathy, in the acute (active) phase associated with signs of inflammation such as edema, erythema and skin temperature changes.^{26 27} After consolidation and in the absence of clinical signs the episode of active Charcot neuro-osteoarthropathy was considered to be resolved.^{26 27} We excluded individuals if they had a foot ulcer, a foot infection, an active Charcot neuro-osteoarthropathy, chronic limb-threatening ischemia (ie, grade 3 Perfusion Extent Depth Infection Sensation (PEDIS) classification),²⁸ bilateral amputation proximal to the tarsometatarsal (Lisfranc) joint, an estimated survival less than 18 months, or if they already used at-home foot temperature monitoring. Participants were randomly assigned to usual care or enhanced care that in addition to usual care included at-home daily measurement of foot temperature at six to eight predefined locations on the foot using infrared thermometry.²⁵ Follow-up time was 18 months. This study was registered on the Netherlands Trial Register.

Primary outcome

The primary outcome was foot ulcer recurrence during the 18 months of follow-up. A foot ulcer was defined as a full-thickness lesion of the skin of the foot, irrespective of duration.²⁷ If the participant or treating healthcare professional identified an ulcer during follow-up, the professional was instructed to complete a foot ulcer form, take photographs of the ulcer, and send all materials to the study team. During three monthly follow-up calls, participants were asked about any lesion that had occurred, and we checked the electronic patient files for any unreported ulcer. Three independent diabetic foot experts assessed ulcer forms and photographs to determine the ulcer outcome. If not unanimous, two other experts were consulted and a majority vote determined the outcome.

Potential predictors

We included demographic, disease-related, and organization-of-care variables as potential predictors of foot ulcer recurrence. These potential predictors were collected at baseline through anamnesis, physical examination or questionnaires, and are listed in table 1.

Table 1.

Characteristics of the 304 study participants

Potential predictor	Outcome	Missing values n (%)
Age (years)	64.6±10.5
Males	220 (72.4)
Body mass index (kg/m2)	29.8±5.3
Caucasian	283 (93.1)
Type of diabetes		4 (1.3)
Type 1	66 (21.7)
Type 2	234 (77.0)
Years of diabetes	20±14	3 (1.0)
HbA1c (%)	7.7±3.6	65 (21.4)
Retinopathy	151 (49.7)	2 (0.7)
Nephropathy	60 (19.7)	1 (0.3)
Dialysis	4 (1.3)
Smoking or history of smoking	169 (55.6)
Consumption of alcohol	199 (65.5)
Walking aid	89 (29.3)
Living alone	105 (34.5)
Level of education		2 (0.7)
Low	117 (38.5)
Medium	96 (31.6)
High	89 (29.3)
Employed	75 (24.7)
Custom-made footwear	205 (67.4)	2 (0.7)
Walking barefoot at home	113 (37.2)
Adherence to self-care	4.7±1.4
At-home foot temperature monitoring	151 (49.7)
Care center
University medical center	88 (28.9)
Community hospital	134 (44.1)
Podiatry practice	82 (27.0)
Peripheral neuropathy
Mild	28 (9.2)
Severe	276 (90.8)
Peripheral artery disease
Grade 1	197 (64.8)
Grade 2	107 (35.2)
Foot deformity
Absent	17 (5.6)
Mild	58 (19.1)
Moderate	202 (66.4)
Severe	27 (8.9)
History of amputation
Absent	223 (73.4)
Lesser toe(s)	29 (9.5)
Hallux or more proximal*	52 (17.1)
Minor lesions at entry	121 (39.8)	31 (10.2)
Plantar location of previously healed ulcer		1 (0.3)
Non-plantar†	185 (60.9)
Plantar	118 (38.8)
Months since healing of previous ulcer‡	7(2–15)	1 (0.3)
Months duration of last two ulcers	4(2–9)	1 (0.3)
36-Item Short Form Health Survey		13 (4.3)
Physical functioning	59±22
Role functioning/physical	50(0–100)
Role functioning/emotional	100(33–100)
Energy/fatigue	60±22
Emotional well-being	78±18
Social functioning	75(63–100)
Pain	67±27
General health	49±20
EuroQol visual analog scale	69±15	23 (7.6)
Social economic score	−0.24±1.17	3 (1.0)

Data are expressed as number (%), mean±SD or median [IQR].

*Seven participants had a unilateral transtibial or transfemoral amputation.

†Including nine participants who were included based on having a history of Charcot neuro-osteoarthropathy.

‡For participants included based only on having a history of Charcot neuro-osteoarthropathy, 48 months was used.

We classified the consumption of alcohol as none versus ≥1 unit/week and employment as none versus any (retirement was considered as unemployed). If participants were treated by a nephrologist or ophthalmologist, they were considered to have diabetic nephropathy or diabetic retinopathy, respectively. Peripheral neuropathy was assessed by measuring the LOPS by using the 10 g (5.07) Semmes-Weinstein monofilament at the plantar surface of the hallux and the first and fifth metatarsal heads of both feet.¹⁰ LOPS was present when the monofilament was not felt on two or more locations. If the monofilament was felt on two or more locations, a 128 Hz tuning fork was used to assess loss of vibratory sensation. The tuning fork was held on the apex of the great toe and loss of vibratory sensation was present when the participant indicated not to feel the vibration.¹⁰ We defined severity of peripheral neuropathy as mild when participants were able to sense the 10 g monofilament, but not the 128 Hz tuning fork, and as severe when they were unable to sense both. Peripheral artery disease was defined as grade 1 or 2.²⁸ We classified foot deformity as absent, mild, moderate or severe.^{15 29} A minor lesion was defined as a non-ulcerative lesion of the skin on the foot, that is, abundant callus, hemorrhage, or blister.¹⁵

We asked participants seven questions regarding their self-care behavior in ulcer prevention (eg, ‘do you walk barefoot at home?’, see online supplemental table 1), and scored their adherence to self-care from 0 (worst) to 7 (best). We obtained health-related quality of life on eight domains of the RAND 36-Item Short Form Health Survey (SF-36) (version 1.0)³⁰ and the EuroQol visual analog scale (EQ VAS).³¹ SF-36 scores were recoded and combined and ranged from 0 (worst) to 100 (best). The EQ VAS score also ranged from 0 (worst imaginable health) to 100 (best imaginable health). We assessed the socioeconomic status (SES) per participant as per postal code, provided by the Netherlands Institute for Social Research (outcome ranges from −5 for lowest possible SES to +5 for highest possible SES).

There were no potential predictors that showed to be strongly correlated with each other (ie, correlation coefficient >0.5) and therefore we considered all the above-mentioned potential predictors as variables in the model.

No variable had more than 25% of missing data (table 1). We used five multivariate imputations for all variables with missing values by applying the chained equations approach.³² This provided multiple imputations for multivariate missing data for any variable type, where each incomplete variable was imputed by a separate model (fully conditional specification method).

Model development

We developed two logistic regression models to predict foot ulcer recurrence: one for any ulcer recurrence (model 1), and one for plantar foot ulcer recurrence (model 2). We adhered to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis statement.³³

We considered all potential predictors as variables for the models based on clinical reasoning, knowledge from peer-reviewed literature, and availability in everyday clinical practice.

Model fitting and validation

Model development was conformed to previous strategies,²⁴ and went through four stages: (1) the creation of five data sets without missing values using multiple imputations; (2) selection of potential predictors selected in the majority of the imputed data sets; (3) fitting a logistic regression model on each of the five imputed data sets to predict ulcer outcome based on these variables; and (4) pooling the coefficients of the separate five models to obtain the final prediction model. The final logistic regression model was represented by its linear predictor. Predicted probabilities were calculated using the linear predictor in the formula: 1/(1+e^{-linear predictor}).

In each imputation data set we used backward variable selection based on the Akaike information criterion (AIC) to find the optimal set of predictors.³⁴ By giving a penalty for model complexity (in terms of the number of included variables), the AIC strikes a good balance between the likelihood of the model (which always increases with the number of included variables) and its complexity (the more complex the model, the more likely it overfits the data).

We used 10-fold cross-validation to internally validate our prediction model. This means that the entire model development strategy (including the five multiple imputation data sets and the variable selection process) was repeated in each of the 10-folds on the training set (90% of the data) and tested on the 10% held-out data set of that fold.

Because our RCT may show that at-home foot temperature monitoring reduces risk of ulcer recurrence, this intervention may predict outcome in one or both models. We therefore developed the model both including and excluding this intervention, as foot temperature monitoring is not yet standard of practice, and other predictors may enter the model if the intervention is not considered.

Model performance

We assessed model performance in terms of discrimination and calibration.³⁵ Discrimination refers to the ability of a model to give a higher probability of the event (ie, ulcer recurrence) to those participants with the event than those without. We measured discrimination for the final prediction model by the area under the receiver operating characteristic curve (AUC) using the mean and two times the SD (2SD).³⁶ The AUC ranges from 0 to 1, with 1 representing perfect discrimination. Furthermore, we presented the AUC plots of the final models. Calibration refers to the closeness of the predicted values to the observed ones. We assessed the calibration using calibration graphs. The Brier Score, which combines both discrimination and calibration, was also assessed for all five pooled models and the final prediction model (mean, 2SD). The Brier Score is the mean squared error of a prediction and ranges from 0 to 1, with 0 representing perfect concordance between predicted and observed values.³⁷

We performed descriptive statistics using SPSS V.26.0 (SPSS) and all model analyses in the R statistical environment (R Foundation for Statistical Computing V4.0.2 for Windows (http://www.R-project.org)),³⁸ with the mice V.3.11.0 and bootStepAIC V.1.2-0 packages (both available on https://cran.r-project.org/ at the time of publication).

Results

Baseline participant characteristics are presented in table 1. Of a total 304 participants, 220 were male and the mean age was 64.6 years. Demographic data corresponded with previous studies.^{7 29} Foot ulcer recurrence in 18 months occurred in 126 participants (41.4%), with a mean time to ulceration of 212 days (SD: 154, range 5–532). Plantar foot ulcer recurrence occurred in 70 participants (23.0%), with a mean time to plantar ulceration of 206 days (SD: 159, range 15–532). During 18 months of follow-up there were no participants lost to follow-up, while eight participants died during follow-up. Table 2 provides the univariate analyses of all potential predictors with both outcomes.

Table 2.

Univariate analyses for potential predictors associated with outcome 1 (all recurrent foot ulcers) and outcome 2 (all recurrent plantar foot ulcers)

Potential predictor	Ulcer recurrence at any foot site		Plantar foot ulcer recurrence
	OR (95% CI)	P value	OR (95% CI)	P value
Age (years)	0.977 (0.956 to 0.999)	0.042	0.974 (0.950 to 0.998)	0.038
Males	1.299 (0.775 to 2.180)	0.321	1.032 (0.567 to 1.880)	0.917
Body mass index (kg/m2)	0.982 (0.941 to 1.026)	0.418	0.997 (0.948 to 1.049)	0.920
Caucasian	1.064 (0.434 to 2.607)	0.892	0.774 (0.252 to 2.379)	0.654
Type of diabetes
Type 1	Reference		Reference
Type 2	0.938 (0.544 to 1.620)	0.880	0.869 (0.464 to 1.629)	0.661
Years of diabetes	1.001 (0.985 to 1.017)	0.923	0.999 (0.980 to 1.017)	0.881
HbA1c (%)	1.003 (0.989 to 1.017)	0.690	1.008 (0.992 to 1.025)	0.329
Retinopathy	1.242 (0.787 to 1.962)	0.352	1.250 (0.732 to 2.136)	0.414
Nephropathy	1.024 (0.579 to 1.812)	0.934	1.244 (0.652 to 2.374)	0.507
Dialysis	1.419 (0.197 to 10.212)	0.728	1.116 (0.114 to 10.900)	0.925
Smoking or history of smoking	0.997 (0.630 to 1.579)	0.991	1.365 (0.791 to 2.356)	0.264
Consumption of alcohol	1.315 (0.809 to 2.135)	0.269	1.879 (1.024 to 3.448)	0.042
Walking aid	1.487 (0.903 to 2.447)	0.119	1.604 (0.912 to 2.820)	0.101
Living alone	1.231 (0.763 to 1.986)	0.394	0.907 (0.515 to 1.598)	0.736
Level of education
Low	Reference	0.163	Reference	0.516
Medium	1.367 (0.787 to 2.374)	0.268	1.177 (0.612 to 2.262)	0.625
High	1.714 (0.978 to 3.002)	0.060	1.462 (0.765 to 2.794)	0.251
Employed	0.994 (0.585 to 1.687)	0.982	0.709 (0.369 to 1.364)	0.303
Custom-made footwear	1.582 (0.958 to 2.612)	0.073	1.472 (0.808 to 2.684)	0.207
Walking barefoot at home	1.134 (0.708 to 1.816)	0.602	0.922 (0.529 to 1.607)	0.774
Adherence to self-care	1.089 (0.924 to 1.282)	0.309	1.141 (0.939 to 1.388)	0.185
At-home foot temperature monitoring	0.626 (0.395 to 0.992)	0.046	0.814 (0.476 to 1.390)	0.451
Care center
University medical center	Reference	0.152	Reference	0.045
Community hospital	0.729 (0.424 to 1.252)	0.252	0.592 (0.322 to 1.088)	0.091
Podiatry practice	0.543 (0.292 to 1.008)	0.053	0.404 (0.192 to 0.849)	0.017
Peripheral neuropathy
Mild	Reference		Reference
Severe	6.699 (1.976 to 22.711)	0.002	4.250 (0.983 to 18.375)	0.053
Peripheral artery disease
Grade 1	Reference		Reference
Grade 2	1.103 (0.685 to 1.777)	0.687	0.803 (0.454 to 1.422)	0.452
Foot deformity
Absent	Reference	0.060	Reference	0.113
Mild	0.698 (0.221 to 2.204)	0.540	0.747 (0.175 to 3.193)	0.694
Moderate	1.444 (0.514 to 4.056)	0.486	1.495 (0.412 to 5.417)	0.541
Severe	2.292 (0.656 to 8.009)	0.194	2.745 (0.630 to 11.956)	0.179
History of amputation
Absent	Reference	0.005	Reference	0.044
Lesser toe(s)	1.424 (0.652 to 3.111)	0.375	1.032 (0.397 to 2.685)	0.949
Hallux or more proximal	2.805 (1.506 to 5.223)	0.001	2.277 (1.186 to 4.373)	0.013
Minor lesions at entry	3.066 (1.908 to 4.925)	0.001	2.659 (1.532 to 4.616)	0.001
Plantar location of previously healed ulcer
Non-plantar	Reference		Reference
Plantar	1.497 (0.938 to 2.389)	0.091	2.667 (1.545 to 4.603)	0.001
Months since healing of previous ulcer	0.949 (0.926 to 0.973)	0.001	0.951 (0.921 to 0.983)	0.003
Months duration of last two ulcers	1.025 (0.999 to 1.051)	0.059	1.024 (0.998 to 1.052)	0.072
SF–36 Short Form Health Survey
Physical functioning	0.997 (0.989 to 1.005)	0.464	0.997 (0.988 to 1.007)	0.595
Role functioning/physical	1.000 (0.995 to 1.005)	0.982	1.001 (0.995 to 1.007)	0.720
Role functioning/emotional	0.998 (0.992 to 1.003)	0.406	1.003 (0.996 to 1.010)	0.415
Energy/fatigue	0.997 (0.986 to 1.007)	0.549	0.998 (0.986 to 1.011)	0.806
Emotional well-being	0.993 (0.980 to 1.006)	0.284	1.001 (0.986 to 1.016)	0.874
Social functioning	0.995 (0.985 to 1.004)	0.270	1.000 (0.989 to 1.011)	0.958
Pain	1.000 (0.991 to 1.008)	0.991	0.998 (0.989 to 1.008)	0.766
General health	0.994 (0.983 to 1.005)	0.294	1.004 (0.991 to 1.018)	0.531
EuroQol visual analog scale	0.999 (0.984 to 1.014)	0.898	1.007 (0.990 to 1.025)	0.417
Social economic score	1.044 (0.857 to 1.272)	0.670	0.986 (0.784 to 1.241)	0.903

Model 1: ulcer recurrence at any foot site

This model contained six predictors (table 3): a younger age, more severe peripheral sensory neuropathy, fewer months since healing of the previous ulcer, presence of a minor lesion, and the use of a walking aid were positive predictors for ulcer recurrence and at-home foot temperature monitoring was a negative predictor for ulcer recurrence. The linear predictor was: 0.284–0.0299*age in years+1.57*more severe peripheral sensory neuropathy–0.0486*months since last healed ulcer+0.704*minor lesion present+0.800*use of walking aid–0.503*use of at-home foot temperature monitoring. When model 1 was rerun excluding foot temperature monitoring as intervention, the model contained the same predictors (results not shown).

Table 3.

Predictors for model 1 (all recurrent foot ulcers) and model 2 (all recurrent plantar foot ulcers)

Predictor	Coefficient	95% CI
Model 1
Intercept	0.284	−0.163 to 2.20
Age	−0.0299	−0.0541 to −0.00570
Severity of peripheral sensory neuropathy	1.57	0.327 to 2.82
Months since healing of previous ulcer	−0.0486	−0.0757 to −0.0215
Minor lesions	0.704	0.170 to 1.24
Walking aid	0.800	0.225 to 1.37
At-home foot temperature monitoring	−0.503	−1.01 to 0.000222
Model 2
Intercept	−0.129	−2.07 to 1.81
Age	−0.0313	−0.0608 to −0.00191
Plantar location of previous ulcer	0.921	0.313 to 1.53
Months since healing of previous ulcer	−0.0379	−0.0721 to −0.00370
Minor lesions	0.777	0.140 to 1.41
Walking aid	0.828	0.179 to 1.48
Alcohol consumption	0.966	0.274 to 1.66
Care center
University medical center	Reference
Community hospital	−0.564	−1.23 to 0104
Podiatry practice	−0.917	−1.74 to −0.0941

Based on the average predictions per participant of the final model in the five imputed data sets, the predicted probability of foot ulcer recurrence and the observed number of recurrent foot ulcers agreed over almost the whole range of probabilities (figure 1). When the predicted probability was >0.70 the model slightly underestimated the proportion of observed ulcers. The mean AUC of the model was 0.69 (2SD 0.040) (figure 2). The mean Brier Score was 0.22 (2SD 0.011).

Figure 1.

Calibration graphs for model 1 (all recurrent foot ulcers) and model 2 (all recurrent plantar foot ulcers). In each graph, the black lines show the observed proportion of the event versus the probability of the event as predicted by the model. Ideally, all the points fall on the diagonal red line.

Figure 2.

Area under the receiver operating characteristic curve (AUC) for model 1 (all recurrent foot ulcers) and model 2 (all recurrent plantar foot ulcers).

Model 2: plantar foot ulcer recurrence

This model contained seven predictors (table 3): a younger age, plantar location of the previous ulcer, fewer months since healing of the previous ulcer, presence of a minor lesion, use of a walking aid, consumption of alcohol and foot care received in a university medical center. The linear predictor was: –0.129–0.0313*age in years+0.921*plantar location of the previous ulcer–0.0379*months since last healed ulcer+0.777*minor lesion present+0.828*use of walking aid+0.966*consumption of alcohol–(0.554*foot care received in a community hospital|–0.917*foot care received in a podiatry practice). For this equation, foot care received in a university medical center was the reference category.

Based on the average predictions per participant of the final model in the five imputed data sets, the predicted probability of foot ulcer recurrence and the observed number of recurrent foot ulcers agreed over almost the whole range of probabilities (figure 1). The predicted probability underestimated the observed ulcer recurrences when around 0.30 and slightly overestimated when >0.50 (figure 1). The mean AUC of the model was 0.66 (2SD 0.023) (figure 2). The mean Brier Score was 0.16 (2SD 0.0048).

Table 4 provides the predicted probabilities including the 95% CIs for two characteristic persons with diabetes using synthesized data.

Table 4.

The predicted probability of ulcer recurrence within 18 months using model 1 (all recurrent foot ulcers) and model 2 (all recurrent plantar foot ulcers) for two characteristic persons with diabetes using synthesized data

	Person A	Person B
Model 1
Age (years)	50	80
Severity of peripheral sensory neuropathy	Severe	Mild
Months since healing of previous ulcer	3	24
Minor lesions	Yes	No
Walking aid	No	Yes
At-home foot temperature monitoring	Yes	No
Probability of ulcer recurrence (95% CI)	0.60 (0.54 to 0.66)	0.08 (0.04 to 0.13)
Model 2
Age (years)	50	80
Plantar location of previous ulcer	Yes	No
Months since healing of previous ulcer	3	24
Minor lesions	Yes	No
Walking aid	No	Yes
Alcohol consumption	No	Yes
Care center
University medical center	University medical center
Community hospital
Podiatry practice		Podiatry practice
Probability of ulcer recurrence (95% CI)	0.47 (0.38 to 0.56)	0.06 (0.04 to 0.10)

Discussion

We used data from the largest two-arm trial on foot ulcer recurrence in diabetes to date, including a representative and demographically and disease-related diverse group of people all at high risk of diabetic foot ulceration (IWGDF risk 3).¹⁰ In this group we found six predictors of foot ulcer recurrence. These predictors are all easy to obtain in clinical practice, and together capable of predicting ulcer recurrence with good calibration and fair discrimination. We found seven predictors of plantar foot ulcer recurrence, also all easy-to-obtain variables, together predicting with reasonable calibration and fair discrimination. These prediction models can help in risk assessment and in reallocating resources for ulcer prevention treatment in this high-risk group of people with diabetes.

Four predictors were identified in both models: younger age, use of a walking aid, presence of a minor lesion, and fewer months since last healed ulcer. A younger age has previously also been associated with a higher risk of ulcer development.^{2 39–41} Use of a walking aid was not previously considered as potential predictor in studies and subsequently never associated with ulcer recurrence. Use of a walking aid may represent more disease severity (eg, neuropathy, peripheral artery disease, foot deformity) or more frailty, affecting one’s ability to walk without additional support, and thus increasing risk of trauma and recurrence. Both of these variables are easy to obtain in clinical practice.

A minor lesion is a well-known risk factor of plantar foot ulcer recurrence, as previous studies have shown.^{15 24} Our findings confirm these results and extend them to any foot ulcer recurrence. The presence of a minor lesion should therefore warn healthcare professionals that immediate treatment is needed, such as through callus removal or offloading the minor lesion, and that patients should be seen more frequently until the minor lesion has been resolved.^{13 42} Fewer months since the last ulcer healed is likely a predictor because skin and underlying tissue are still regaining strength and remain vulnerable for breakdown in the first months after epithelization. Our models confirm earlier reports of a higher risk when time since last ulcer healed is shorter.⁶ This suggests that extra attention to offloading and more frequent foot care should be given in the first months after healing.

Peripheral sensory neuropathy and at-home monitoring foot temperature were specific predictors for ulcer recurrence at any foot site. Peripheral sensory neuropathy has been studied extensively in ulcer risk estimation in populations with and without ulcer history.^{23 43 44} Few studies found it to be associated with ulcer recurrence,^{20 40} presumably because most high-risk people have neuropathy, limiting its differential effect in risk analyses. However, peripheral sensory neuropathy advances progressively,^{45 46} where inability to perceive vibration from a tuning fork is a sign of an earlier stage of neuropathy, while inability to perceive pressure of a monofilament is a later stage sign.^{45 46} This may explain our finding that more severe peripheral sensory neuropathy (ie, lack of perception of both the vibration from the tuning fork and pressure from the monofilament) predicted ulcer recurrence (in comparison to only not sensing the vibration from the tuning fork).

These prediction models were based on data from an RCT explaining that the intervention of at-home monitoring of foot temperature was one of the potential predictors and turned out to be a predictor of ulcer recurrence in the study. Despite evidence from two meta-analyses,^{47 48} and recommendations for its use included in international guidelines,¹³ at-home monitoring of foot temperatures is not standard in clinical practice. We therefore ran the prediction model with and without this intervention and found the same predictors of ulcer recurrence regardless of its inclusion, indicating that these predictors are important, independent of this intervention, in foot ulcer recurrence.

Specific predictors for ulcer recurrence on the plantar foot were a plantar location of the previous ulcer, consumption of alcohol and foot care received in a university medical center. The first may be understandable from the more biomechanical etiology of plantar versus non-plantar foot ulcers, with plantar ulcers more likely to recur at the same site.^{6 15} Increased attention to offloading these high-risk plantar areas is needed.^{29 42} Only one previous study associated more alcohol consumption with ulcer recurrence.¹⁹ While moderate use of alcohol is acceptable in people with diabetes, a potential explanation could be the effect of alcohol on a person’s health in general and specifically on blood glucose regulation, which may in turn increase ulcer risk.⁴⁹ Unfortunately, detailed information on the use of alcohol was not available in our study, and more research on this association is needed.

Foot care in a university medical center represents tertiary foot care as proposed in the IWGDF guidelines, and is therefore probably a marker for more disease severity.¹⁰ Furthermore, since it is only a predictor for plantar foot ulcer recurrence, it most likely reflects the more advanced biomechanical burden in these people treated in tertiary care. This larger burden might be explained by the wider availability of biomechanical assessment tools in university medical centers and subsequent referral from secondary to tertiary care of patients requiring such assessment.

Because data were obtained from a large multicenter trial in diabetic foot disease, our prediction models are limited to high-risk people with diabetes that match the inclusion criteria for the trial. While external validation is needed to determine generalizability of the models, the only trial-specific criteria concerned excluding participants with bilateral amputation proximal to the Lisfranc joint and expected survival <18 months. Future external validation may show differences in the performance due to potential selection bias in our trial compared with the situation were a prospective observational cohort analysis would have been conducted. Furthermore, with 126 and 70 ulcer events in the first and second models, respectively, we were limited by the number of predictors to select for the final model to avoid overfitting. Despite considering multiple potential predictors, our models had only fair discriminating performance. To increase performance, one likely needs to include more complex behavioral (eg, therapy adherence,²⁹ stride count measures^{15 24 50}), social-economic,⁵¹ or biomechanical (eg, barefoot or in-shoe plantar pressures^{15 24}) variables. However, that would jeopardize clinical utility as these variables are hard to obtain in everyday clinical practice. Another limitation is the interobserver variability in assessing some foot-related parameters by different investigators, such as foot deformity, peripheral sensory neuropathy, peripheral artery disease and minor lesions. To improve on the latter, two independent observers assessed the photographs of the feet for presence of foot deformity and minor lesions and they reached consensus on the outcome.

All predictors found can be easily obtained by healthcare professionals when screening people at high risk, and thus the prediction models can be readily applied in everyday clinical practice. Using the linear predictor from both models, a healthcare provider can determine someone’s risk of ulcer recurrence. When doing so, it is important to know that predictors should not be interpreted individually, but only as a combination of variables that together may determine the risk of ulceration. To illustrate the potential use of the prediction models in clinical practice, we used synthesized data of two characteristic persons with diabetes and estimated the probability for ulcer recurrence (table 4). Person A has a high probability of developing a recurrent ulcer within 18 months, while person B has a low probability of developing a recurrent ulcer within 18 months. The combination of non-modifiable and modifiable predictors may help healthcare providers to better determine the frequency of foot screening and care, while the modifiable predictors (ie, minor lesions, use of at-home foot temperature monitoring, alcohol use) may suggest potential preventative treatment that aims to mitigate the risk for recurrence. Because of the high risk of ulcer recurrence, person A might, for example, be monitored carefully and treated for minor lesions. However, we emphasize that to understand the effect on ulcer recurrence risk of interventions targeting these modifiable predictors, adequately powered trials or etiological analyses are needed. On a more macroscopic level, our prediction models may help national health authorities and healthcare insurance companies in health policy so to better allocate the limited resources for foot care for people with diabetes who are at risk of foot ulceration.

Conclusion

These internally validated prediction models contain easy-to-obtain modifiable and non-modifiable variables and are built from a representative and diverse group of people with diabetes, neuropathy and foot ulcer history. These models better stratify people at high risk of foot ulceration and help determine who should be monitored more carefully and treated more intensively with the aim to improve on the prevention of foot ulcer recurrence in people with diabetes.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study was approved by the research ethics committees of all seven participating centers (approval ID 2015_105 from the ethics committee of the main study site, Amsterdam UMC–Location AMC). We obtained written informed consent prior to inclusion from all participants.