Assessment of Simple Bedside Wound Characteristics for a Prediction Model for Diabetic Foot Ulcer Outcomes

Abstract

Background:

Evidence-based learning systems built on prediction models can support wound care community nurses (WCCNs) during diabetic foot ulcer care sessions. Several prediction models in the area of diabetic foot ulcer healing have been developed, most built on cardiovascular measurement data. Two other data types are patient information (i.e. sex and hemoglobin A1c) and wound characteristics (i.e. wound area and wound duration); these data relate to the status of the diabetic foot ulcer and are easily accessible for WCCNs. The aim of the study was to assess simple bedside wound characteristics for a prediction model for diabetic foot ulcer outcomes.

Method:

Twenty predictor variables were tested. A pattern prediction model was used to forecast whether a given diabetic foot ulcer would (i) increase in size (or not) or (ii) decrease in size. Sensitivity, specificity, and area under the curve (AUC) in a receiver-operating characteristics curve were calculated.

Results:

A total of 162 diabetic foot ulcers were included. In combination, the predictor variables necrosis, wound size, granulation, fibrin, dry skin, and age were most informative, in total an AUC of 0.77.

Conclusions:

Wound characteristics have potential to predict wound outcome. Future research should investigate implementation of the prediction model in an evidence-based learning system.

Introduction

Diabetic foot ulcers are a frequent late diabetic complication affecting more than 15% of all people with diabetes¹ and often result in lower limb amputations.^2-6 Treating diabetic foot ulcers is a complex task, and the complexity associated with treating chronic ulcers commands the use of evidence-based treatment and consistency in practice, and due to the complexity of treating chronic ulcers, it is crucial to have evidence-based treatments and consistency in practice. Several systematic reviews from the Cochrane Library indicate a lack of evidence within diabetic foot ulcer treatment.^7-15 Moreover, the literature shows that wound care is inconsistent as it varies from professional to professional.¹⁶ In an attempt to increase the evidence base, a large number of randomized controlled trials testing aspects of foot ulcers have been conducted. However, these randomized controlled trials are challenged by the fact that the group of people with diabetes is very heterogeneous, leading to low reliability, as results from one randomized controlled trial cannot be transferred to the whole population with diabetes. Perhaps reflecting the relatively low level of evidence, the conventional way wound care community nurses (WCCNs) acquire diabetic foot ulcer knowledge is through experience, knowledge, and qualifications from their bachelor degree in nursing, knowledge sharing among colleagues, and experiences as wound-care specialists.¹⁶

A recent systematic literature review investigating clinical decision support systems that support healthcare professionals such as WCCNs treating chronic ulcers showed that none of the systems specifically targeted diabetic foot ulcers.¹⁷ Some systems were based on prediction models; others were based on expert knowledge.

Prediction models can help detecting or screening undiagnosed high-risk subjects, predict future events of disease or death, and assist in medical decision-making.¹⁸ For WCCNs, prediction models may be developed to show the likelihood of, e.g., an increase or decrease in the wound size of a diabetic foot ulcer, given specific treatments. Even though no clinical decision support systems supporting healthcare professionals target diabetic foot ulcers specifically, several types of models for predicting wound outcomes in diabetic foot ulcers exist.^1,2,19-33 Overall, these prediction models apply three categories of data: “cardiovascular measurements,” “patient information,” and “wound characteristics.”

Predictor Variables

A systematic review and meta-analysis from 2016 compared evidence from different tests from the cardiovascular measurements category that predict wound healing in diabetic foot ulcers²⁶: ankle-brachial index (ABI), ankle peak systolic velocity, transcutaneous oxygen measurement (TcPO₂), toe-brachial index, toe systolic blood pressure, microvascular oxygen saturation, skin perfusion pressure, and hyperspectral imaging. It was concluded that all eight types of tests most likely predict wound healing; however, most of the evidence only evaluates TcPO₂ and ABI. Studies find that for people with diabetes having cardiovascular problems, cardiovascular measurement category tests are better at predicting wound outcomes than the tests in the two other categories.^{1,26,31,32,34} It should be noted that most people with diabetic complications have cardiovascular problems. A challenge related to the use of cardiovascular measurements is, however, that cardiovascular tests can be time-consuming for some WCCNs as well as expensive to conduct if the WCCNs are not familiar with the procedure. Another challenge related to their use is that predictor variables are very seldom documented in electronic health records.

The second category of predictor variables, patient information, covers personal information related to the patient. In longitudinal data analyses, retrospective cohort studies, prospective cohort studies, and systematic reviews, selected predictor variables such as age, sex, comorbidity, and hemoglobin A1c have proven useful in predicting wound healing.^{2,21,22,28,35} However, a challenge in using patient information predictor variables is that WCCNs are often not in the vicinity of the electronic health records when they make decisions regarding wound care, and therefore do not always have access to this type of information.

The third category of predictor variables, wound characteristics, contains data types that relate to the status of the diabetic foot ulcer. Predictor variables within this category include wound area, percentage of wound area reduction, wound duration, wound grade, and epithelial migration.^{20,23,28,30,36-38} Such simple bedside predictor variables in the third category are available to WCCNs since they are part of WCCNs’ wound anamneses when consulting patients with diabetic foot ulcers. According to a systematic review and meta-analysis conducted by Crawford et al, more evidence is needed to assess the value of wound characteristics when predicting wound healing.³⁹

The three types of data categories, cardiovascular measurements, patient information, and wound characteristics, are relevant to predict diabetic foot ulcer outcomes. In the present study, we assessed simple bedside wound characteristics for a prediction model for diabetic foot ulcer outcomes.

Methods and Materials

Dataset

Data were obtained from 27 of 29 municipalities in the Capital Region in Denmark from August 2005 to April 2018. In total, 616 patients with one or more diabetic foot ulcers participated. From that, 938 diabetic foot ulcers were registered in a nearly complete data set. In total, the data set consisted of 20 potential predictor variables from the categories patient information and wound characteristics. The predictor variables were epithelialization, callosity, atrophie blanche, pigmentation, edema, dry skin, eczema, rubor, maceration, smell, exudate, visible tendon, visible bone, hypergranulation, fibrin, granulation, necrosis, wound size, gender, and age.

Data were obtained from the Danish wound database and web-based communication tool pleje.net. The pleje.net system is a liaison between WCCNs and wound specialists from Danish hospitals who collaborate in the area of diabetic foot ulcers. The database structure was a simple two-dimensional table consisting of rows and columns, where each row or record corresponds to an entity including the unique patient ID and each column represents patient information details (e.g. age, sex, and weight) and various wound characteristics and measurements.

The Danish Patient Safety Authority gave their permission to this study (Journal No. 3-3013-1867/1/). Furthermore, the study was registered at Danish Data Protection Agency Journal No. 2015-57-0001, and the patients gave their consent to letting the researchers work with their personal healthcare information.

As a first step, after receiving written consent from each patient, the WCCN created a patient profile where information on the patient was registered. After each of the consultations the patients subsequently attended, the WCCN documented in detail the treatment and the development of the status of the diabetic foot ulcer in the database. The inspection includes the appearance of the diabetic foot ulcer, and rate it according to the Wagner classification system.⁴⁰ Internationally, other classification systems may be used.⁴¹ They are, however, not used in Denmark. The size of the wound was measured according to national standards used in Denmark where the WCCNs place a customized ruler next to the diabetic foot ulcer before taking a photo of the diabetic foot ulcer. The customized ruler contains color gamut, millimeter, a box to write a patient ID, and a box to write the date. As WCCNs have tight schedules and, therefore, have limited time to do wound reports and exhaustive documentation, the documentation is often far from complete. In particular, cardiovascular measurements and patient information are very often missing in pleje.net. The Danish WCCNs conduct the measurements, i.e. measure the wound size, register the characteristics, e.g. if there is callus, fibrin, dry skin, and so on, and take a photo of the diabetic foot ulcer. The next step in the process is that WCCNs upload the photo and register the identified measurements into the database pleje.net. Experts (i.e. specialized doctors and specialized nurses in diabetic foot ulcers from hospital departments; all located at University Hospitals in Denmark) will then verify the information and give the WCCNs advice, if necessary. It should be noted that all the WCCNs, as part of a WCCN training program, have passed a telemedicine-wound course conducted by experts from the Danish University Hospitals. In order to include ulcers featuring noticeable progression and that could be assumed to have undergone a relatively homogeneous development in the observation period, the following inclusion criteria were applied: observation period between 14 and 30 days, minimum 0.5cm² wound size, and at least two registrations of the wound size.

Prediction Model Development

We developed a pattern prediction model to forecast individualized development of diabetic foot ulcers. The model predicts whether a given ulcer will (i) increase in size (or not) or (ii) decrease in size. Sensitivity was defined by the ratio between the number of ulcers increasing in size and where an increase was predicted (the true positives), and the total number of ulcers increasing in size (true positives plus false negatives). Sensitivity and corresponding specificity were used to calculate the area under the curve (AUC) in a receiver operating characteristics (ROC) curve, which was optimized to identify wound characteristics yielding the best wound prediction outcome.

Since data included both nominal and ordinal data types, a binary logistic regression classification was chosen. An ROC curve illustrates the relation between the false positive rate on the x-axis and the true positive rate on the y-axis using various detection cutoff thresholds, and is a plot of sensitivity against 1−specificity for different choices of cutoff.

We applied forward selection to build the model. We chose forward selection since it starts with a null model (a model with no predictor variables) and then adds extra predictor variables randomly, one by one, while keeping the predictor variables that give the best improvement in model performance and ignoring predictor variables that give less improvement.⁴² The selection method allowed us to follow the creation of the model. Forward selection included a ranking algorithm calculating a mean separability measure for each predictor variable, which refers to the ability to separate, based on the AUC, above the random classifier slope of the ROC.

A fivefold cross-validation was applied to reduce overfitting of the model and ensure the transferability of the results.^43,44 This approach means that the model was trained on four parts of the data (four out of five parts) and tested on the rest of the data (one out of five parts). This was repeated five times (one for each of the five parts), and the five results were then averaged. All data analyses were done in MATLAB R2016b (MathWorks, Natrick, MA, USA).

Results

From the total dataset (N = 938 diabetic foot ulcers, N = 616 patient with diabetes, wound size index: 0.1-205mm) obtained between 2005 and 2018, 162 diabetic foot ulcers in 138 patients were eligible (26 female and 112 male, 119 patients with 1 ulcer, and 19 patients with 2 or more ulcers). One hundred eleven out of the 162 wounds healed. One hundred eleven wounds healed and 51 wounds did not heal. The mean age of this group of patients was 71 (±12) years. The distribution of males and females within the group was 83% males and 17% females. Within the given observation period, 69% of the 162 diabetic foot ulcers decreased in size. Table 1 shows the completeness of the WCCNs’ data registration.

Table 1.

Completeness of the WCCNs’ Data Registration.

No.	Name of predictor variable	Completeness (numbers)	Completeness (%)
1	Age (yes/no)	162 (162)	100
2	Gender (M/F)	134/28	100
3	Wound size (yes/no)	162 (162)	100
4	Necrosis (yes/no)	99 (162)	61.1
5	Granulation	65 (162)	40.12
6	Fibrin (yes/no)	58 (162)	35.8
7	Hypergranulation (yes/no)	157 (162)	96.91
8	Bone contact (yes/no)	151 (162)	93.21
9	Bare joint/tendon (yes/no)	154 (162)	95.06
10	Exudate (yes/no)	10 (162)	6.17
11	Smell (yes/no)	140 (162)	86.42
12	Maceration (yes/no)	102 (162)	62.96
13	Rubor (yes/no)	97 (162)	59.88
14	Eczema (yes/no)	156 (162)	96.3
15	Dry skin (yes/no)	107 (162)	66.05
16	Edema (yes/no)	83 (162)	51.23
17	Pigmentation (yes/no)	159 (162)	98.15
18	Atrophie blanche (yes/no)	161 (162)	99.38
19	Callosity (yes/no)	116 (162)	71.6
20	Epithelialization (yes/no)	145 (162)	89.51

F, female; M, male; WCCN, wound care community nurse.

Figure 1 shows the separability measure defined as the AUC of the ROC above the random 0.5 for each of the 20 predictor variables. An AUC of the ROC curve of 0.5 indicates that the test cannot discriminate between diabetic foot ulcers increasing and decreasing in size (i.e. it is a random performance), whereas values between 0.5 and the maximum value of 1 indicate that the test can discriminate between diabetic foot ulcers increasing and decreasing in size; an AUC of 1 indicates 100% sensitivity and 100% specificity. From visual inspection, it appears from Figure 1 that both necrosis and age are very informative predictor variables compared with, for example, the callosity predictor variable. It should be noted that the separability measure does not distinguish between a positive and negative correlation between predictor variable and outcome.

Figure 1.

The separability measure in terms of ROC AUC above 0.5 for each predictor variable.

AUC, area under the curve; ROC, receiver operating characteristics.

The final model consisted of the six most informative predictor variables (Table 2), and data yielded a cross-validated ROC AUC of 0.77. In the forward selection process, necrosis was the first predictor variable, as can be seen in Table 2. The next predictor variables were wound size, granulation, fibrin, dry skin, and age. It should be noted that, due to potentially significant correlations between individual predictor variables, the six best predictor variables do not correspond to the six predictor variables that, assessed individually, gave the highest separability measure shown in Table 1. The ROC performance of the final classifier is seen in Figure 2.

Table 2.

Predictor Variables Included in the Model (Predictor Variable No., Predictor Variable Name, Log Odds, Odds Ratio, and Model AUC Increase) and Sequence of Inclusion (Top to Bottom of the Table).

No.	Predictor variable	Log odds (β)	Odds ratio (eβ)	Model AUC
1	Necrosis (yes/no)	−1.4286	0.2396	0.66
2	Wound size (per cm)	0.0294	1.0298	0.70
3	Granulation (yes/no)	0.1789	1.1959	0.72
4	Fibrin (yes/no)	−0.7640	0.4657	0.73
5	Dry skin (yes/no)	−0.7041	0.4945	0.74
6	Age (per years)	−0.0464	0.9546	0.77

AUC, area under the curve.

Figure 2.

The ROC curve for the final model.

ROC, receiver-operating characteristic.

Discussion

In the present study, a model that predicts wound outcome was developed. Twenty different predictor variables from the wound characteristics category and patient information category were tested and validated by forward selection. The outcome of this process was a weighted list of six informative predictor variables: necrosis, wound size, granulation, fibrin, dry skin, and age. In a cohort study by Margolis et al, comparable to the present study, a prediction model was developed to determine which patients with a diabetic neuropathic foot ulcer would respond successfully to standard therapy within a reasonable period of time. Their prediction model included age, male, number of wounds, duration, wound size, and wound grades¹—the same number of predictor variables as the model included in the present study. Both prediction models contain a limited number of unique predictor variables, and the two models had a number of predictor variables in common. The predictor variables that overlapped in the category of common predictor variables were wound size and age. These two variables in particular are often used in other prediction models too.^2,26,37 The AUC of Margolis et al’s prediction model was 0.70 and the AUC of the prediction model in the present study was 0.77. Even though an AUC of 0.77 is relatively good, false predictions may have important consequences, which should be addressed in future works. Since the dependent variable was dichotomous (binary), a linear regression model would not have been appropriate and, thus, multiple logistic regression analysis was used. Other classification methods, such as neural network or various clustering methods, might have been considered. The results from using such models would, however, be harder to explain: both to experts in the field and to clinicians.

The idea behind using readily available information has been advocated in other studies. For instance, in a prospective study by Boyko et al, readily available clinical information was used to predict the occurrence of diabetic foot ulcers. Boyko et al reached an AUC of 0.81 and included seven predictor variables: Amici, vision poorer than 20/40, history of foot ulcer, history of amputation, monofilament insensitivity, tinea pedis, and onychomycosis. None of these variables were the same as the predictor variables in our prediction model.⁴⁵ As can be seen in Figure 1, the predictor variable with the highest ROC AUC is necrosis. According to the figure, the next predictor variable to be added to the model should be age, since it has the second highest ROC AUC when used on its own. However, using forward selection as our model-building approach, wound size was the next predictor variable that should be added to give the best prediction with two variables. This perhaps surprising result can be explained by necrosis and age being more correlated than necrosis and wound size; i.e. wound size, when added to necrosis, provides more additional information than age. The same forward selection process when the next four variables were added gives the results shown in Table 2. It might be noted that even though the present study focuses on using readily available data, i.e. simple bedside wound characteristics, to build a model, another approach could have been to focus on how to give WCCNs access to less available data, e.g. cardiovascular measurements, and to predictions based on such data.

Although the results showed that age should not be included as the second variable in the prediction model, it should be included as the last and sixth most informative predictor variable. In total, the AUC peaks at a value of 0.77, going from 0.74 with five variables, after age is included as the sixth variable. Other authors who have not used forward selection have recommended caution in using age as a predictor variable, since it correlates with several predictor variables.⁴⁶ A study by Oyibo et al monitored 194 diabetic foot ulcer patients over 12 months. Their results showed that age as a predictor variable had no effect on health-related outcome.³⁰ The number of parameters in the model should be in accordance with the classical rule of thumb. The “one in ten rule,” for the maximum number of model parameters that can be estimated from data when doing regression analysis, states that without overfitting the model, one parameter can be included in the model for every 10 events.⁴⁷ In the current study, data from 162 diabetic foot ulcers were used to estimate 6 predictor parameters, which is in good accordance with rule of thumb.

Limitations

The present study has a number of limitations. The data set was not complete regarding WCCNs’ registrations of wound care practice. Incomplete data is a typical problem in health care since healthcare professionals are often too busy and get frustrated if they have to document in more than one system. In the present study, the WCCNs had to use a separate login to log on to the wound database pleje.net in addition to logging on to their normal healthcare record. Consistency in documentation where data can be structured as in an inquiry form or unstructured as in a free text box is also a typical issue for clinical data.⁴⁸ Due to problems with incompleteness and lack of consistency, some potentially relevant predictor variables could not be included in the analysis.⁴⁹ Since we in our study have paid attention to unique wounds, and not subjects, we did not consider subject exclusion. This, however, means that for those subjects with 2 or more ulcers (19 subjects out of the 138 subjects), ulcers from the same subject may be in both train and test dataset, which may be regarded a potential limitation of the results.

Conclusion

On-site wound characteristics seem to have potential for predicting wound outcome. Our results showed that the combination of the predictor variables necrosis, wound size, granulation, fibrin, dry skin, and age is the best combination, and that, used in combination, these six predictors lead to an AUC of 0.77. Future work on the model should include more data and also investigate the potential of implementing the prediction model in an evidence-based learning system. Future work on the clinical use of the model in such a system should address the question of how much the clinicians are to be involved in the clinical decision process—she or he should probably still play a key role in making the final decision for each wound.