Abstract
Introduction
In Egypt, diabetic foot ulcers markedly contribute to the morbidity and mortality of diabetic patients. Accurately predicting the risk of diabetic foot ulcers could dramatically reduce the enormous burden of amputation.
Objective
The aim of this study is to design an artificial intelligence-based artificial neural network and decision tree algorithms for the prediction of diabetic foot ulcers.
Methods
A case–control study design was utilized to fulfill the aim of this study. The study was conducted at the National Institute of Diabetes and Endocrine Glands, Cairo University Hospital, Egypt. A purposive sample of 200 patients was included. The tool developed and used by the researchers was a structured interview questionnaire including three parts: Part I: demographic characteristics; Part II: medical data; and Part III: in vivo measurements. Artificial intelligence methods were used to achieve the aim of this study.
Results
The researchers used 19 significant attributes based on medical history and foot images that affect diabetic foot ulcers and then proposed two classifiers to predict the foot ulcer: a feedforward neural network and a decision tree. Finally, the researchers compared the results between the two classifiers, and the experimental results showed that the proposed artificial neural network outperformed a decision tree, achieving an accuracy of 97% in the automated prediction of diabetic foot ulcers.
Conclusion
Artificial intelligence methods can be used to predict diabetic foot ulcers with high accuracy. The proposed technique utilizes two methods to predict the foot ulcer; after evaluating the two methods, the artificial neural network showed a higher improvement in performance than the decision tree algorithm. It is recommended that diabetic outpatient clinics develop health education and follow-up programs to prevent complications from diabetes.
Keywords: foot ulcer, artificial neural network, decision tree, diabetic patients
Introduction
Diabetes mellitus (DM) is a serious issue for both patients and healthcare professionals because it is a condition that will affect 700 million people worldwide by the year 2045. Egypt is one of the top 10 countries in the world with the greatest incidence of diabetes, with around 9 million people aged 20 to 79 living with the disease. The number of diabetic patients in Egypt has risen quickly, from approximately 4.5 million in 2007 to 7.5 million in 2013, and is predicted to reach 13.1 million by 2035. Diabetic foot ulcers (DFUs) are more common in Africa (7.2%) than in Asia (5.5%) or Europe (3%) (International Diabetes Federation (IDF), 2019).
Predicting which patients are most susceptible to getting a DFU is thus extremely valuable in decreasing the terrible effects of lower extremity amputations. Machine learning techniques may be used in the healthcare field to enhance patient outcomes by allowing for early identification and diagnosis of disease processes (Stefanopoulos et al., 2022).
Review of Literature
Patients with type 2 DM (T2DM) are at risk for and frequently develop DFUs. The quality of life of a person is significantly impacted by foot ulcers, which also place a huge financial and social burden on healthcare systems. Ulcer recurrence prevention is essential due to the high occurrence and subsequent risk of infection, hospitalization, and amputation (Aan de Stegge et al., 2021). Numerous factors, including sociodemographic ones like age, gender, place of residence, and level of education, contribute to the convoluted etiology of DFUs (Yazdanpanah et al., 2018).
Clinical variables include the type and duration of diabetes, a lack of glycemic control, an elevated body mass index, and foot abnormalities. An increased risk of acquiring diabetes-related foot ulcers is also linked to comorbidities such as peripheral vascular disease, retinopathy, nephropathy, and neuropathy (Zhang et al., 2017). Diabetes-related foot ulcers are also impacted by lifestyle variables like alcohol intake, smoking, exercising, and having bad foot hygiene habits (Naemi et al., 2019).
Proper glycemic management, routine foot examinations, wearable footwear, patient education, and early referral for preulcerative lesions can all help prevent DFUs. The diabetic foot must be examined for any lesions and screened for peripheral neuropathy and peripheral vascular disease, both of which can cause wounds or ulcers. Additionally, prompt referral of patients with foot ulcers and signs of infection, foot and systemic sepsis, or ischemia to a clinic specifically for diabetic feet will lower morbidity and mortality (Howard et al., 2021).
Artificial intelligence (AI) is the machine manifestation of human intelligence. It is currently classified as a branch of computer science that can analyze complex medical data and aid in bettering patient outcomes (Shin et al., 2020). Most frequently, it is considered to be the study of algorithms that let computers think and carry out cognitive tasks (Sheikhtaheri et al., 2014).
A variety of new AI technologies have recently become accessible. AI is broadly classified as machine learning, natural language processing, artificial neural networks (ANNs), and computer vision (Sardar et al., 2019). In line with its mission of promoting “health for all, everywhere,” WHO (2019) published its global strategy report on digital health. In this report, AI is defined as “the adoption of clinical information systems, mobile-health applications, wearable health technologies, and AI applications to improve health.”
The use of AI technology in healthcare is becoming more common in clinical settings throughout the world, with global spending on these technologies expected to exceed US$36 billion by 2025 (Robert, 2019). The usage of AI in our daily lives and in the field of health is growing by the day. Health professionals should be able to employ AI-enabled technologies in healthcare effectively, as well as understand the influence of these technologies on patient care, the healthcare system, and society (Grzeska et al., 2021).
The nurse plays an important role in identifying risk factors that contribute to complications and developing educational programs, interventions, and close monitoring. Nursing consultation is a private action performed by nurses to accurately identify people with diabetes who are at risk of ulceration. As a result, a comprehensive examination of the feet is an important step in determining the risk of problems in these limbs (Boulton et al., 2016).
Community health nurses are crucial members of the diabetes treatment team, having an important role in the prevention of diabetic foot problems as well as the care and education of patients at risk of diabetic foot problems. Nurses can advise patients to follow a few basic guidelines to help prevent foot ulcers or their recurrence, such as checking shoes before putting them on, keeping feet clean, and maintaining skin and nail care. They can also teach patients how to perform physical examinations and care for their feet on a daily basis. Training in choosing the right shoes is also required (Kaya & Karaca, 2018).
Community health nurses also assess patient needs and develop a unique educational program for each patient and their families, as well as facilitate the active participation of patients and family members in care. They can also teach patients about the importance of regular clinic visits, blood tests at regular times, and the primary principle of prevention (Aalaa et al., 2012). So, the aim of this study is to design an AI-based ANN and decision tree (DT) algorithms for the prediction of DFUs.
Methods
Research Design
The current study used a case–control study design. Case–control research begins with a set of cases, or individuals, who have the desired outcome. The researcher then attempts to create a second set of individuals known as “controls,” who are similar to the case persons but do not have the desired outcome. The researcher next examines historical circumstances to see whether any exposures are more prevalent in the cases than in the controls. If the exposure is discovered more frequently in cases than in controls, the researcher might hypothesize that the exposure is related to the result (Tenny et al., 2022).
Research Hypotheses
To achieve the aim of this study, the following research hypotheses are formulated:
H1:An artificial intelligence-based artificial neural network will reliably predict the occurrence of diabetic foot ulcers.
H2: An artificial intelligence-based decision tree will reliably predict the occurrence of diabetic foot ulcers.
Sample
The researchers used power analysis to calculate the sample size; the calculation takes into account several factors, including the expected prevalence of DFUs, the desired level of precision, the level of significance, and the desired statistical power. We expect the prevalence of DFUs to be around 20%. We set the desired level of precision at 5%, the level of significance at 0.05 (alpha = 0.05), and the desired power at 0.8 (1 − beta = 0.8). So, the power analysis formula is as follows:
where Z_Alpha/2 is the critical value of the standard normal distribution at alpha/2 = 1.96, p is the expected prevalence of the outcome = 0.2, d is the desired level of precision = 0.05, and then n = (1.96)2 * 0.2 * (1 − 0.2)/0.052 = 245.
Therefore, we would need a sample size of approximately 245 patients to predict the occurrence of DFUs with a maximum acceptable margin of error of 5%, a level of significance of 0.05, and a power of 0.8.
Inclusion/Exclusion Criteria
A purposive sample of 200 patients at the National Institute of Diabetes and Endocrine Glands at Cairo University Hospital in Egypt who were willing to participate in this study was included according to the following inclusion criteria: both sexes of adult diabetic patients with diabetes duration greater than 1 year were included. Diabetic children and those with diabetes for less than 1 year were excluded from the current study.
Tool for Data Collection
After reviewing related literature, the following tool was used for data collection for this study: a structured interview questionnaire consists of three parts:
Part I: demographic characteristics that covered the following variables: age, gender, level of education, place of residence, and income.
Part II: medical data, which covers the following variables: duration of diabetes, family history, type of treatment, history of exercise, pain in the feet, and complications from diabetes.
Part III: in vivo measurements, such as weight, height, body mass index, and blood sugar level.
Tool Validity and Reliability
A panel of three experts in the fields of computers and AI, community health nursing, and medical surgical nursing reviewed the developed tool in order to determine the validity of its content. To measure the questionnaire's reliability, Cronbach's alpha was used using the Statistical Package for the Social Sciences (SPSS), and it was 0.95. This indicates that the questionnaire is measuring the intended construct with a high level of precision and consistency.
Ethical Considerations
This study was approved by an institutional review board from the Faculty of Nursing at Cairo University (approval number 63-2022) and conducted in accordance with accepted national and international standards. Permission to conduct the study was obtained from the director of the National Institute of Diabetes and Endocrine Glands at Cairo University Hospital. In addition, before the phase of data collection, the researchers obtained written informed consent from each patient who agreed to participate in this study after informing them about the aim of the study and emphasizing that participation in the study is entirely voluntary and that they have the right to withdraw at any time without giving any reason. Anonymity and confidentiality were assured through coding the data.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the committee responsible for human experimentation and with the Helsinki Declaration of 1975, as revised in 2013 (http://ethics.iit.edu/ecodes/node/3931).
Procedure
Upon receiving formal approval from the Faculty of Nursing, Cairo University, to conduct the study, official permission was obtained from the director of the National Institute of Diabetes and Endocrine Glands, Cairo University Hospital, to approve the fieldwork and collect data. Before beginning the study, the researchers explained the aim of the study to each patient to gain their cooperation and participation in the study. Interviewing the patients was carried out in the waiting area beside the clinic. Before the distribution of the sheet, the researchers informed each patient about the confidentiality of the collected data and their right to withdraw from the study at any time. The researchers were present with the patients during the filling of the questionnaire to clarify the sheet and ensure an individualized response.
The interview questionnaire was filled out by the patients, except for those who cannot read or write, whose questionnaire was filled out by the researchers. The time spent filling out the questionnaire ranged between 10 and 15 min, and the researchers met the patients twice per week from 9 a.m. to 1 p.m. at the outpatient clinic. A questionnaire from this study was distributed to the patients in the waiting area to fill out. Measurements were taken by the researchers in the outpatient clinic at the National Institute of Diabetes and Endocrine Glands, such as weight, height, and body mass index, using a balance scale and tape; the patients’ blood sugar level was measured using an electronic blood glucose meter and test strip. Data were collected from November 2022 to January 2023.
The prediction of foot ulcers using AI for diabetic patients was done through the following steps: first, the demographic characteristics, medical history, and in vivo measurements were obtained; second, the participants were divided into two groups: one group had a foot ulcer, and the other did not; third, AI applications were used by the MATLAB program to select the strongest predictors from all factors to determine the predictors affecting the occurrence of ulcers for the participants; fourth, the data collected from the participants were divided into two parts (training samples and test samples) so that about 75% of the data were entered for the two methods used (ANN and DT) for training and 25% for testing the method after its design so that the number of correct cases collected from the data used in the test was divided by all cases and multiplied by 100 to find out the success rate of the method so that it reaches a success rate greater than or equal to 90%.
After data collection from 200 diabetic patients, 82 of them had foot ulcers, while 118 did not. An ANN and a DT were designed using images and medical data from patients for training the model. After completing the training of the AI methods, diabetic patients who did not have foot ulcers were followed for 3 months to ensure the occurrence of foot ulcers or none, and after that, the reliability of the AI methods was tested, and the accuracy was 97% using an ANN.
Proposed Model
This study develops a novel method to predict the foot ulcer using some selected parameters and input images using the correlation method for foot ulcers and non-foot ulcers. The suggested approach makes use of two well-known classifiers as well as certain calculated characteristics in the input's frequency and spatial domains. These classifiers use multilayer neural networks and DTs. Figure 1 displays the suggested method's general block diagram. The strongest feature vector is chosen after preprocessing the medical history data input and the image. The preprocessed image's Fourier Transform (FT) is then computed. Following that, six features are chosen from the medical history data, and 13 characteristics are computed from the input image. These features are what give the class designation its meaning. The two classifiers are supplied with these characteristics. Analysis of the classifiers’ performance is conducted. The sections below go into greater detail about the proposed algorithm.
Figure 1.
The general block diagram of the proposed method.
Statistical Analysis
SPSS version 26 was used to score, tabulate, compute, and analyze the obtained data to get the strongest features. To present the data gathered, descriptive and inferential statistics were employed. Numbers and percentages were used to represent the qualitative characteristics. Means and standard deviations (SDs) were used to present quantitative variables. Pearson's chi-square test was used to analyze relationships between the variables and determine how well DFUs correlated with the study's independent variables. MATLAB version 2021 was used to forecast the foot ulcer.
Results
Table 1 illustrates that 63.4% of foot ulcer patients were male, 84.7% of diabetic patients were from urban areas, and 38.9% of them were aged 50 to less than 60 years, with a mean age of 53.47 ± 12.17 years old. Regarding the educational level, 30.5% of diabetic patients could not read and write. In terms of income, 84.7% of diabetic patients reported that their income was insufficient to cover their regular expenses; however, no significant association with income was found between cases. A highly significant association was found with place of residence (p = .000), and a significant association was found with age (p = .050).
Table 1.
Demographic Data of Diabetic Patients (n = 200).
| Variable | No. (%) | Chi-square | df | p-value | |
|---|---|---|---|---|---|
| Foot ulcer n = 82 |
None n = 118 |
||||
| Gender | 2.333 | 1 | .127 | ||
| Male | 52 (63.4%) | 62 (52.5%) | |||
| Female | 30 (36.6%) | 56 (47.5%) | |||
| Place of residence | 14.579 | 1 | .000** | ||
| Urban | 50 (61%) | 100 (84.7%) | |||
| Rural | 32 (39%) | 18 (15.3%) | |||
| Age (years) | 9.494 | 4 | .050* | ||
| Less than 40 | 24 (29.3%) | 20 (16.9%) | |||
| 40 to less than 50 | 16 (19.5%) | 18 (15.3%) | |||
| 50 to less than 60 | 18 (21.9%) | 46 (38.9%) | |||
| 60 to less than 70 | 20 (24.4%) | 24 (20.4%) | |||
| 70 and more | 4 (4.9%) | 10 (8.5%) | |||
| Mean ± SD | 50.98 ± 13.38 | 53.47 ± 12.17 | |||
| Educational level | 8.415 | 5 | .135 | ||
| Cannot read and write | 24 (29.3%) | 36 (30.5%) | |||
| Read and write | 18 (21.9%) | 22 (18.6%) | |||
| Primary education | 8 (9.8%) | 2 (1.8%) | |||
| Preparatory education | 10 (12.2%) | 20 (16.9%) | |||
| Secondary education | 14 (17.0%) | 28 (23.7%) | |||
| University education | 8 (9.8%) | 10 (8.5%) | |||
| Income | 4.316 | 2 | .116 | ||
| Sufficient | 22 (26.8%) | 18 (15.3%) | |||
| Insufficient | 60 (73.2%) | 100 (84.7%) | |||
*Significant at .05.
**Significant at .01.
According to Table 2, 72.9% of diabetic patients had diabetes for 5 years or more, and 74.6% had a family history of diabetes; 89.8% were treated with insulin injections. Regarding exercise, 62.7% of the patients did not practice it, and 96.6% of patients had pain, while 38.9% of them complained of hypertension related to diabetes. Exercise history and foot pain were found to have a highly significant relationship (p = .008 and .001, respectively).
Table 2.
Medical Data and In Vivo Measurement of Diabetic Patients (n = 200).
| Variable | No. (%) | Chi-square | df | p-value | |
|---|---|---|---|---|---|
| Foot ulcer n = 82 |
None n = 118 |
||||
| Duration of diabetes | 2.249 | 3 | .522 | ||
| 1 year to less than 5 years | 28 (34.2%) | 32 (27.1%) | |||
| 5 years and over | 54 (65.8%) | 86 (72.9%) | |||
| Family history of diabetes | 11.665 | 6 | .070 | ||
| Yes | 60 (73.2%) | 88 (74.6%) | |||
| No | 22 (26.8%) | 30 (25.4%) | |||
| Type of treatment for diabetes | 0.203 | 1 | .652 | ||
| Insulin | 72 (87.8%) | 106 (89.8%) | |||
| Tablets | 10 (12.2%) | 12 (10.2%) | |||
| History of exercise | 9.603 | 2 | .008** | ||
| Yes | 42 (51.2%) | 44 (37.3%) | |||
| No | 40 (48.8%) | 74 (62.7%) | |||
| Pain in feet | 16.108 | 3 | .001** | ||
| Yes | 76 (92.7%) | 114 (96.6%) | |||
| No | 6 (7.3%) | 4 (3.4%) | |||
| Complications from diabetes | 9.419 | 6 | .151 | ||
| Hypertension | 20 (24.4%) | 46 (38.9%) | |||
| Visual disturbances | 10 (12.2%) | 14 (11.9%) | |||
| Nerve problems | 6 (7.3%) | 6 (5.1%) | |||
| Kidney problems | 6 (7.3%) | 4 (3.4%) | |||
| Blood vessel problems | 10 (12.2%) | 10 (8.5%) | |||
| Nothing | 30 (36.6%) | 38 (32.2%) | |||
| Body mass index | 15.846 | 4 | .003** | ||
| Less than 25 | 8 (9.8%) | 8 (6.8%) | |||
| 25 to less than 30 | 16 (19.5%) | 32 (27.1%) | |||
| 30 to less than 35 | 32 (39.0%) | 28 (23.7%) | |||
| 35 to less than 40 | 8 (9.8%) | 34 (28.8%) | |||
| 40 and over | 18 (21.9%) | 16 (13.6%) | |||
| Mean ± SD | 33.02 ± 7.41 | 32.81 ± 5.83 | |||
| Blood sugar level | 7.797 | 3 | .050* | ||
| Less than 200 | 14 (17%) | 12 (10.2%) | |||
| 200 to less than 400 | 54 (66%) | 78 (66.1%) | |||
| 400 and over | 14 (17%) | 28 (23.7%) | |||
| Mean ± SD | 312.61 ± 100.36 | 315.85 ± 101.73 | |||
*Significant at .05.
**Significant at .01.
Table 2 also shows that 39% of foot ulcer patients had a body mass index of 30 to less than 35 kg/m² with a mean of 33.02 ± 7.41 kg/m2. Regarding the blood sugar level, 66.1% of diabetic patients had a blood sugar level of 200 to less than 400, while 10.2% of them had a blood sugar level of less than 200, with a mean of 315.85 ± 101.7. A highly significant association was found with body mass index (p = .003), and a significant association was found with blood sugar level (p = .050).
Finally, as shown in the above tables, place of residence, age, history of exercise, pain in feet, body mass index, and blood sugar level features give highly significant associations to the class label, so these six attributes will be selected and added to the attributes from the input foot images.
Feature Extraction and Selection of Image Data
Input foot images and participant data are provided to the suggested algorithm. These photos have various sizes. In order to use the suggested algorithm, preprocessing is required. The initial preprocessing step is to zoom (resize) photos to the same size (Zohuri & Moghaddam, 2020). Subsequent processing is facilitated by this procedure. The zooming process makes use of a bilinear interpolation method. The bilinear method is used to translate one point's position on the screen to another. Using the four adjacent or surrounding pixels, the weighted average is calculated (Hashemi, 2019).
The pixel is then assigned to this average. The unknown pixel's interpolated value is determined by taking into account its 2-by-2 known neighboring pixels and taking their weighted average (Cao et al., 2017). Shrinking an image is analogous to undersampling. It might be accomplished by eliminating portions of the image's columns and rows.
The goal of feature extraction is to take the characteristics from the input foot photos that are necessary for categorization. The researchers take advantage of some frequency and spatial properties. The FT of the input image might be used to extract frequency domain characteristics, as shown in Figure 2, which contains an example of a foot ulcer image and contains an example of a non-foot ulcer image. The FT examines the input image's sine and cosine components; FT might be calculated using the Fast FT (FFT) method. A matrix of frequency coefficients emerges from the FT analysis.
Figure 2.
Example of foot ulcer and non-foot ulcer images.
Thirteen calculated characteristics from the input image were employed by the researchers in this investigation. Mean absolute value (MAV), variance, mean of energy, average amplitude change (AAC), minimum value amplitude, maximum value amplitude, average frequency, maximum frequency, minimum frequency, half point of the energy (HaPo), power spectral density (PSD), image entropy, and root mean square (RMS) are some of these characteristics (Boonyakitanont et al., 2020; Hindarto et al., 2019).
Equation (1) provides the MAV m of the FT coefficients as follows:
| (1) |
The original intensity values of the input image are represented by Xi, where n is the total number of frequency coefficients. Equation (2) gives the variance of the FT as follows:
| (2) |
The researchers chose the mean of energy as their third characteristic. It is also regarded as a form of the average frequency component value. Equation (3) provides it as follows:
| (3) |
Xi stands for the frequency coefficients calculated from the FT, where the researchers use AAC to show the difference between the averages of the difference between two subsequent pixels. It can be expressed as Equation (4):
| (4) |
The minimum and maximum amplitude properties of the FT, respectively, were utilized by the researchers. The maximum and minimum energy values from the frequency components of the input image are represented by the largest and least energies, correspondingly. The following is the result of Equation (5) typical frequency:
| (5) |
where p is the PSD and f is the frequency vector. The highest and lowest energy frequencies on the spectrum are known as the maximum and minimum frequencies, respectively. The frequency that divides the electromagnetic spectrum into two sections is known as the HaPo. Random signals have a PSD, which defines them. It represents the power against the frequency of the signal. Equation (6) gives the PSD as follows:
| (6) |
The researchers also employed RMS and entropy as two additional spatial domain characteristics. Information theory objects’ complexity may be calculated using entropy. In order to describe the texture of the input image, it is a statistical measure of randomness. Thus, the intrinsic properties of the signal might be extracted using entropy. Equation (7) gives the entropy as follows:
| (7) |
where P is the probability that currently surrounds each level of gray. The intensity of the images is expressed by the RMS. Equation (8) provides it as follows:
| (8) |
where xi represents the different image intensities.
Artificial Neural Network
A multilayer perceptron is an ANN feedforward model that transforms input datasets into a set of useful outputs. It contains input, output, and hidden layers. The input layer is the one that receives the processing signal. The output layer, which includes prediction and classification, performs the relevant duties. The multilayer perceptron's real processing engine is made up of an endless number of hidden layers between the input and output sides. The feedforward neural network, like a transmission network in a multilayer perceptron, flows data from the input layer in a forward manner to the output layer (Badr et al., 2022).
An ANN is composed of nodes, an input layer (symbolized by ), a hidden layer that is optional, and y as an output layer. The ANN goal is to find weights between the input, output, and hidden layers that minimize the sum of the total squared errors. These weights are modified during training according to λ, a learning parameter that belongs to [0, 1] until the results are compatible with each other (Bhoi, 2021). In the current study, the two instance classes are foot ulcers and non-foot ulcers. The neural network was trained using 19 features to get better weights.
Figure 3 indicates the training of the ANN; the ANN consists of a set of layers and nodes in each layer; as shown in this figure, the ANN used in the proposed solution consists of an input layer, two hidden layers, and an output layer, with 19 nodes in the first hidden layer containing the input parameters for each sample, connecting to 7 nodes in the second layer and connecting to 7 nodes in the third layer; finally, the output layer contains one node that displays if the sample has a foot ulcer or not.
Figure 3.
The proposed artificial neural network for classifying foot ulcers.
Decision Tree
A random forest (RF) is an ensemble of multiple DTs that functions as a meta-estimator. While generating each individual tree, it uses baggage and randomness to try to create a forest of negatively connected trees whose committee prediction is more reliable than any one tree. Each tree in the RF spreads a class prediction, and the class with the most votes becomes the model's predictor (Muaad et al., 2022).
The first step in utilizing DTs to find a solution is to prepare a set of solved cases. After that, the full set is divided into two groups: (1) a training set for creating a DT and (2) a testing set for assessing the accuracy of an answer that is formed. All qualities characterizing each scenario are initially offered (as input data) before selecting a feature that will act as a choice for the given challenge (output data). For each input attribute, distinct value classes have been established. Each discrete value of an attribute takes its own class if an attribute may only accept one of those discrete values; if an attribute has a range of possible numerical values, then a set of characteristic intervals representing several classes must be created.
As shown in Figure 4, each attribute, also known as an attribute node or a test node, can represent one internal node in a DT that has been created. An attribute node of this type has precisely as many branches as there are distinct value classes. Decisions are the leaves of a DT, and decision classes are the value classes of the decision attribute (Figure 4). When a decision must be made for an unsolved case, we begin with the DT's root node and work our way along the attribute nodes, choosing branches where the values of the relevant attributes in the unsolved case match those in the DT. We continue this process until we reach the leaf node, which represents the decision.
Figure 4.
The proposed decision tree for classifying foot ulcers.
Experimental Results
The suggested model findings are described in this section. All calculations are made with MATLAB 2021 to compare performance. Every experiment employs k-fold cross-validation, with k equal to 10. To predict whether a foot ulcer would develop or not, the researchers examined two classifiers: DTs and neural networks. The researchers used RFs for DTs, and then we trained the data for neural networks using a variety of hidden layers and layer node structures to find the optimum structure. We are mostly concerned with each classifier's accuracy. But it is also important to be sensitive and specific. Sensitivity may be more important than specificity in medical data, though. That is, as was already shown, increased sensitivity predicts a lower false negative.
The researchers used 26 attributes, but only 19 were significant based on medical history and foot images that affect DFUs. Small datasets may be due to the privacy concerns of the patients; the medical data contain sensitive information about patients, and it is critical to protect their privacy. As a result, it can be challenging to obtain access to large datasets due to strict regulations, ethical considerations, and concerns about patient privacy. The authors solve this point in the current study by using k-fold cross-validation through the distribution of the data into training and testing. k-fold cross-validation is a powerful technique that can be used to maximize the use of limited data when building machine learning models and prevent overfitting in machine learning models.
Table 3 shows that the ANN outperforms the DT in terms of evaluation metrics, with 0.97 accuracy rather than 0.93 using the DT. The sensitivity measure in the ANN was better than the DT; the specificity and positive predictive value (PPV) were 1 and 0.99, respectively, which are higher than those in the DT; the ANN outperforms the DT in terms of the negative predictive value (NPV) measure; finally, there is an increase in F-score by 0.05 in the ANN rather than the DT. This table supported the first and second research hypotheses.
Table 3.
Artificial Neural Network and Decision Tree Results.
| Results | Accuracy | Sensitivity | Specificity | PPV | NPV | F-score |
|---|---|---|---|---|---|---|
| ANN | 0.97 | 0.95 | 1 | 0.99 | 0.93 | 0.97 |
| DT | 0.93 | 0.93 | 0.93 | 0.95 | 0.90 | 0.92 |
ANN = artificial neural network; DT = decision tree; NPV = negative predictive value; PPV = positive predictive value.
Figure 5 shows the receiver operating characteristic (ROC) curves for a DT and an ANN, respectively, and it can be seen that the ROC of the ANN is close to the y-axis and has a small deviation from the DT with area under the ROC curve (AUC) 1.0 rather than 0.96 in the DT. The ROC curve plots the true positive rate against the false positive rate at different classification thresholds; this means that an ANN gives the smallest error in foot ulcer cases.
Figure 5.
Receiver operating characteristic (ROC) curve for the decision tree and artificial neural network.
Discussion
Technologies that enhance ulcer diagnosis and treatment methods have the potential to revolutionize diabetic foot care. Prediction models can help identify or screen undiagnosed high-risk individuals, forecast the occurrence of diseases or fatalities, and support medical decision-making (Lee et al., 2016).
Regarding diabetic patients’ demographic data, the current study indicates that more than half of foot ulcer patients were male, and more than one-third of them were aged from 50 to less than 60 years, while nearly one-third of diabetic patients could not read and write. More than three-quarters of diabetic patients reported having insufficient income. A highly significant association was found with place of residence, and a significant association was found with age.
A previous study done by Fawzy et al. (2019) on the factors associated with diabetic feet among 200 T2DM patients in northern Saudi Arabia reported that sociodemographic characteristics such as older age and male gender were associated with an increase in DFUs.
Along the same line, Rossaneis et al. (2016) conducted a study to investigate the differences in foot self-care and lifestyle among a sample of 1,515 men and women with DM living in the urban area of a large city in the south of Brazil and found that the foot self-care deficit was significantly higher among men, though men presented a lower prevalence of foot scaling and the use of inappropriate shoes when compared to women. With regard to lifestyle, men presented less healthy habits, such as not adhering to a proper diet and not taking laboratory exams to check for a lipid profile at the frequency recommended. From the researchers’ point of view, one possible explanation is that women are more likely to stick to diabetes-related lifestyle adjustments and health-seeking behaviors.
Another study supported the current finding done by Galal et al. (2021), which aimed to investigate the predictors of foot ulcers among 488 Egyptian diabetic patients, finding a substantial link between illiteracy and a greater risk of DFUs, which may be related to the difficulty in accessing information regarding diabetes and its consequences. The same results were reported in agreement by Cardoso et al. (2019), who carried out a study aimed at identifying the factors related to the development of DFUs in 600 individuals treated by the primary healthcare network in a public unit in Brazil. The similarities between studies may be due to socioeconomic and cultural factors, as the extremely poor may have less opportunities for health services.
Also, a study done by Sriyani et al. (2013), which aimed to identify the sociodemographic, lifestyle, and foot examination-related predictors of diabetic foot and leg ulcers with a view to developing a screening tool appropriate for use among 168 patients in Sri Lanka, reported that male sex, an education level of 6 and below, and a monthly household with less income were significant risk factors for foot ulcers in patients with DM.
The current study shows that a highly significant association was found with history of exercise. This result is in line with a study done by Naemi et al. (2019) that aimed to identify the biomechanical, neurological, and clinical parameters along with other demographics and lifestyle risk factors that could explain the presence of foot ulcers in 1,270 patients with diabetes in Africa and reported that lifestyle factors, including exercise, have an impact on DFUs. Regular exercise can help prevent foot ulcers in diabetic patients; exercise can improve blood circulation, which can reduce the risk of foot ulcers by promoting wound healing and reducing tissue damage. Patients with a history of regular exercise may have a lower risk of developing foot ulcers compared to patients who do not exercise regularly.
The current study indicates that nearly three-quarters of diabetic patients had the disease for 5 years or more, and the majority of them use insulin injections for treatment, while more than one-third of them complain of hypertension related to diabetes. No significant association was found with duration, type of treatment, or complications from diabetes.
These results are supported by a study done by Atosona and Larbie (2019), which examined the prevalence and determinants of DFUs and lower extremity amputations in 100 diabetic patients in Ghana and reported that diet, insulin, and oral hypoglycemic agents were not linked to DFUs.
On the other hand, a study done by Al-Rubeaan et al. (2015) about diabetic foot complications and their risk factors in 62,681 patients in the Saudi National Diabetes Registry stated that the length of diabetes has also been linked to the development of complications, including neuropathy. The use of insulin increased the incidence of foot ulcers and amputations significantly. In contrast, a study done by Galal et al. (2021) in Egypt indicated that a protective predictor of DFUs was treatment with diet, oral hypoglycemic agents, and insulin, while a significant positive predictor of DFUs was the occurrence of two or more diabetes problems. One likely explanation is that insulin was added to oral hypoglycemic agents in order to enhance blood glucose control and avoid issues like diabetic feet.
In the experiments, the patient's condition was predicted using the medical dataset (foot ulcer or none). Each incident in the dataset was either labeled with the desired field or reported as a foot ulcer or no case. By employing our models to train and test the dataset, the neural network approach was deployed. The models’ acquired results have been isolated and examined separately for training and testing outcomes. The multiple neural network approach is superior to the DT technique, as discussed.
In the experiment carried out by Goyal et al. (2018), they collected an extensive dataset of foot images that contained DFUs from different patients. The goal of this DFU classification problem, which assessed the two classes as normal skin (healthy skin) and abnormal skin (DFU), was to assess the skin conditions of both groups, which are highly susceptible to misclassification by computer vision algorithms. Additionally, they employed convolutional neural networks for the first time in this binary categorization. To detect feature differences between healthy skin and DFUs, they suggested DFUNet, a new convolutional neural network architecture with improved feature extraction. DFUNet attained an AUC value of 0.961 using 10-fold cross-validation.
Among the ANN techniques used, the work reported by Singh et al. (2013) applied a multilayer perceptron to evaluate the risk of DFUs in patients with T2DM, whereas others have applied the convolutional neural network, which is more often recommended for image processing. Of the AI-based works, Wang et al. (2016) used images to analyze a diabetic foot and showed that the cascaded two-stage approach provides high global performance rates (average sensitivity = 73.3%, specificity = 94.6%) and is sufficiently efficient for smartphone-based image analysis.
The neural network performs better when we have numerical features and/or a small amount of data, like the dataset used in this experiment (200 samples), which are some of the reasons for employing ANNs to achieve the best results. Because it can learn from both linear and non-linear relationships between the features in the current dataset, the neural network is a fairly complicated algorithm that is better suited for working with huge numbers of features than the DT, which performs better in the case of linear datasets only (Hameed et al., 2020).
A previous study done by Poradzka and Czupryniak (2023) about the use of the ANN for a 3-month reliable prognosis in 175 patients with diabetic foot syndrome found that the ANN was created with nine input neurons, six hidden nodes, and two output neurons. The overall accuracy was 82.21%, the sensitivity was 91.6%, and the specificity was 66.18%. So, the ANN, as a new prognosis method in diabetic foot syndrome ulcers, can be reliably used in the prediction, helping physicians and patients predict the course and outcome of the treatment. The algorithm can be particularly useful in identifying individuals who fail to heal.
Also, Stefanopoulos et al. (2022) carried out a study about the machine learning prediction of DFUs in the nationwide inpatient population and indicated that the six-variable model performance of the training data was 79.6% (80.9% sensitivity and 78.3% specificity). The performance of the testing data for the six-variable model was 79.5% (80.6% sensitivity and 78.3% specificity).
In addition, Wang et al. (2013) conducted a study on 8,640 rural adults in Henan Province to develop and evaluate an effective classification approach without biochemical parameters to identify those at high risk of T2DM and reported that the ANN model's sensitivity, specificity, and positive and negative predictive values for identifying T2DM were 86.93%, 79.14%, 31.86%, and 98.18%, respectively, indicating more accurate predictive performance for identifying those at high risk of T2DM based on demographic, lifestyle, and anthropometric data.
In relation to the prediction of foot ulcers using the DT, Sudha et al. (2019) conducted a study about diabetic foot risk classification using the DT and bioinspired evolutionary algorithms for 112 diabetic patients in India and indicated that the DT for risk classification is learned and built from the data using the classification and regression trees (CART) algorithm. The dataset is split into train and test data using five-fold cross-validation and leave-one-out cross-validation (LOOCV) techniques. The tree was huge with a lot of nodes, which is very confusing and not easily interpretable by healthcare professionals. Also, the accuracy was only 57% with DT-LOOCV and 55% with random forest (RF) and cross-validation (CV) with 5 folds (RF-CV5).
DFUs are a prevalent consequence for diabetics. Foot ulcers have a significant influence on an individual's quality of life and place a significant burden on both healthcare systems and society (Aan de Stegge et al., 2021). DFU management is complex, requiring a multidisciplinary team of doctors, nurses, and allied health professionals to be successful. As a result, the community health nurse plays an important role in early detection, education, and effective treatment, including referral to a multidisciplinary diabetic foot clinic, which can slow its progression.
Strengths and Limitations
Despite the fact that the current study provides valuable information for implementing preventive measures for DFUs, it has some limitations as follows: it did not take into account patients’ compliance with foot care practices; also, the study may not have been validated externally, which means that the results may not apply to other populations or settings. The data are only from Cairo University.
Implications for Practice
The findings of this study have implications for practice. Accurate prediction of DFUs is valuable for early diagnosis. The ability to predict DFUs early plays a vital role in the patient's appropriate treatment and also helps to decrease the costs spent on healthcare, as well as the biopsychosocial burden on those affected. This study helps with the early detection of the problem in the evaluation of feet and the need to implement a referral service for early screening and prevention of lower limb complications in people diagnosed with DM. The community health nurse provides care for diabetic patients in different healthcare settings, especially in the community. The study findings revealed the importance of the nurse's role in reducing DFUs among diabetic patients through in-service education regarding basic guidelines to prevent foot ulcers.
Conclusions
The study's results indicated that using the model may reduce the number of needed diagnostic tests, resulting in better DFU treatment. Accurate prediction is essential since diabetic foot ulcers are one of the main complications of diabetes. If it is detected early enough, prompt treatment might prevent the diabetic patient from losing any leg tissue. The proposed technique utilizes two methods to predict the foot ulcer: an ANN and a DT. After evaluating the two methods, the ANN showed a higher improvement in performance with 0.04 accuracy than the DT algorithm.
The following recommendations are made in light of the study's findings:
Health education programs on DFU risk factors and proper foot care techniques should be offered to diabetic patients.
Effective prevention efforts should be included in the duties of healthcare organizations and policymakers in addition to the treatment of DFUs.
Follow-up plans should be created by diabetic outpatient clinics to stop complications from diabetes, including DFUs.
For the purpose of generalizing the findings, this study should be repeated with a larger sample and in various contexts.
Artificial intelligence technology should be employed in different health care settings to improve patients' outcomes.
Acknowledgements
The authors gratefully acknowledge all diabetic patients at the National Institute of Diabetes and Endocrine Glands at Cairo University Hospital in Egypt who volunteered to participate in the study.
Footnotes
Author Contributions: All author(s) have read the final manuscript, have approved the submission to the journal, and have accepted full responsibilities pertaining to the manuscript's delivery and contents.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
ORCID iD: Khadraa Mohamed Mousa https://orcid.org/0000-0003-4529-4009
References
- Aalaa M., Malazy O. T., Sanjari M., Peimani M., Mohajeri-Tehrani M. R. (2012). Nurses’ role in diabetic foot prevention and care; a review. Journal of Diabetes & Metabolic Disorders, 11(1), 1–6. 10.1186/2251-6581-11-24 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Aan de Stegge W. B., Schut M. C., Abu-Hanna A., van Baal J. G., van Netten J., Bus S. A. (2021). Development of a prediction model for foot ulcer recurrence in people with diabetes using easy to-obtain clinical variables. BMJ Open Diab Res Care, 9, e002257. 10.1136/bmjdrc-2021-00225 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Al-Rubeaan K., Al Derwish M., Ouizi S. (2015). Diabetic foot complications and their risk factors from a large retrospective cohort study. PLoS One, 10(5), e0124446. 10.1371/journal.pone.012444633 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Atosona A., Larbie C. (2019). Prevalence and determinants of diabetic foot ulcers and lower extremity amputations in three selected tertiary hospitals in Ghana. Journal of Diabetes Research, 7132861. 10.1155/2019/713286131 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Badr M., Al-Otaibi S., Alturki N., Abir T. (2022). Detection of heart arrhythmia on electrocardiogram using artificial neural networks. Computational Intelligence and Neuroscience.Special Issue-Volume 2022, 1–10. https://doi.org/10.1155/2022/1094830 [DOI] [PMC free article] [PubMed] [Google Scholar] [Retracted]
- Bhoi S. K. (2021). Prediction of diabetes in females of Pima Indian heritage: A complete supervised learning approach. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(10), 3074–3084. https://doi.org/10.17762/turcomat.v12i10.4958 [Google Scholar]
- Boonyakitanont P., Lek-Uthai A., Chomtho K., Songsiri J. (2020). A review of feature extraction and performance evaluation in epileptic seizure detection using EEG. Biomedical Signal Processing and Control, 57, 101702. 10.1016/j.bspc.2019.101702 [DOI] [Google Scholar]
- Boulton A. J. M., Armstrong D. G., Albert S. F., Frykberg R. G., Hellman R., Kirkman M. S. (2016). Comprehensive foot examination and risk assessment. Diabetes Care, 31(8), 1679–1685. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2494620/ 10.2337/dc08-9021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cao L., Jiao L., Li Z., Liu T., Zhong Y. (2017). Grayscale image colorization using an adaptive weighted average method. Journal of Imaging Science and Technology, 61(6), 60502–1. https://doi.org/10.2352/J.ImagingSci.Technol.2017.61.6.060502 [Google Scholar]
- Cardoso H. C., Zara A. L., Rosa S. d. S. R. F. (2019). Risk factors and diagnosis of diabetic foot ulceration in users of the Brazilian Public Health System. Journal of Diabetes Research, 5319892. 10.1155/2019/531989232 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fawzy M. S., Alshammari M. A., Alruwaili A. A. (2019). Factors associated with diabetic foot among type 2 diabetes in northern area of Saudi Arabia: A descriptive study. BMC Research Notes, 12(1), 51. 10.1186/s13104-019-4088-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Galal Y. S., Khairy W. A., Taha A. A., Amin T. T. (2021). Predictors of foot ulcers among diabetic patients at a tertiary care center, Egypt. Risk Management and Healthcare Policy, 14, 3817–3827. 10.2147/RMHP.S325065 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goyal M., Reeves N. D., Davison A. K., Rajbhandari S., Spragg J., Yap M. H. (2018). DFUNet: Convolutional neural networks for diabetic foot ulcer classification. IEEE Transactions on Emerging Topics in Computational Intelligence, 4(5), 728–739. 10.1109/TETCI.2018.2866254 [DOI] [Google Scholar]
- Grzeska A., Ali S., Szmuda T., Słoniewski P. (2021). Objective outcomes evaluation of innovative digital health curricula. Comment on “Undergraduate medical competencies in digital health and curricular module development: Mixed methods study”. Journal of Medical Internet Research, 23(5), e26034. 10.2196/26034 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hameed K., Bajwa I. S., Ramzan S., Anwar W., Khan A. (2020). An intelligent IoT based healthcare system using fuzzy neural networks. Scientific Programming. [Google Scholar]
- Hashemi M. (2019). Enlarging smaller images before inputting into convolutional neural network: Zero-padding vs. interpolation. Journal of Big Data, 6(1), 1–13. 10.1186/s40537-019-0263-7 [DOI] [Google Scholar]
- Hindarto H., Muntasa A., Sumarno S. (2019). Fourier transform for feature extraction of electro encephalo graph (EEG) signals. Journal of Physics: Conference Series, 1402(6), 066027. https://doi.org/10.1088/1742-6596/1402/6/066027 [Google Scholar]
- Howard T., Ahluwalia R., Papanas N. (2021). The advent of artificial intelligence in diabetic foot medicine: A new horizon, a new order, or a false dawn? The International Journal of Lower Extremity Wounds. Special Issue, 2021, 1–6. https://doi.org/10.1177/15347346211041866 [DOI] [PubMed] [Google Scholar]
- International Diabetes Federation (IDF) (2019). https://www.idf.org/aboutdiabetes/what-is-diabetes/facts-figures.html (accessed 5 February 2021)
- Kaya Z., Karaca A. (2018). Evaluation of nurses’ knowledge levels of diabetic foot care management. Nursing Research and Practice. Special Issue, 2018, 1–12. https://doi.org/10.1155/2018/8549567 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lee Y., Bang H., Kim D. J. (2016). How to establish clinical prediction models. Endocrinology & Metabolism, 31(1), 38–44.https://doi.org/10.3803/EnM.2016.31.1.38 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Muaad A. Y., Jayappa Davanagere H., Benifa J. V., Alabrah A., Naji Saif M. A., Pushpa D., Al-antari M. A., Alfakih T. M. (2022). Artificial intelligence-based approach for misogyny and sarcasm detection from Arabic texts. Computational Intelligence and Neuroscience, 2022, 1–9. https://doi.org/10.1155/2022/7937667 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Naemi R., Chockalingam N., Lutale J. K., Abbas Z. G. (2019). Can a combination of lifestyle and clinical characteristics explain the presence of foot ulcer in patients with diabetes? Journal of Diabetes and Its Complications, 33(6), 437–444. 10.1016/j.jdiacomp [DOI] [PubMed] [Google Scholar]
- Poradzka A. A., Czupryniak L. (2023). The use of the artificial neural network for three-month prognosis in diabetic foot syndrome. Journal of Diabetes and its Complications, 108392. 10.1016/j.jdiacomp.2022.108392 [DOI] [PubMed] [Google Scholar]
- Robert N. (2019). How artificial intelligence is changing nursing. Nursing Management, 50(9), 30–39. 10.1097/01.NUMA.0000578988.56622.21 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rossaneis M. A., MdoCFL H., TadeF M., Marcon S. S. (2016). Differences in foot self-care and lifestyle between men and women with diabetes mellitus. Revista Latino-americana de Enfermagem, 24, e2761. 10.1590/1518-8345.1203.2761 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sardar P., Abbott J. D., Kundu A., Aronow H. D., Granada J. F., Giri J. (2019). Impact of artificial intelligence on interventional cardiology: From decision-making aid to advanced interventional procedurassistance. JACC. Cardiovascular Interventions, 12(14), 1293–1303. 10.1016/j.jcin.2019.04.048 [DOI] [PubMed] [Google Scholar]
- Sheikhtaheri A., Sadoughi F., Hashemi Dehaghi Z. (2014). Developing and using expert systems and neural networks in medicine: A review on benefits and challenges. Journal of Medical Systems, 38(9), 110. 10.1007/s10916-014-0110-5 [DOI] [PubMed] [Google Scholar]
- Shin L., Bowling F. L., Armstrong D. G., Boulton A. J. M. (2020). Saving the diabetic foot during the COVID-19 pandemic: A tale of two cities. Diabetes Care, 43(8), 1704–1709. 10.2337/dc20-1176 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Singh K., Singh V. K., Agrawal N. K., Gupta S. K., Singh K. (2013). Association of toll-like receptor 4 polymorphisms with diabetic foot ulcers and application of artificial neural network in DFU risk assessment in type 2 diabetes patients. BioMed Research International, 2013, 1–9. https://doi.org/10.1155/2013/318686 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sriyani K. A., Wasalathanthri S., Hettiarachchi P., Prathapan S. (2013). Predictors of diabetic foot and leg ulcers in a developing country with a rapid increase in the prevalence of diabetes mellitus. PloS One, 8(11), e80856. 10.1371/journal.pone.0080856 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stefanopoulos S., Ayoub S., Qiu Q., Ren G., Osman M., Nazzal M., Ahmed A. (2022). Machine learning prediction of diabetic foot ulcers in the inpatient population. Vascular, 30(6), 1115–1123. 10.1177/17085381211040984 [DOI] [PubMed] [Google Scholar]
- Sudha B. G, Umadevi V., Shivaram J. M., Sikkandar M. Y., Pavan B., Al Amoudi A. (2019). Diabetic foot risk classification using decision tree and bio-inspired evolutionary algorithms. International Journal of Innovative Technology and Exploring Engineering, 9(2S). 10.35940/ijitee.B1081.1292S19 [DOI] [Google Scholar]
- Tenny S., Kerndt C. C., Hoffman M. R. (2022). Case control studies. StatPearls Publishing. PMID: 28846237. [PubMed] [Google Scholar]
- Wang C., Li L., Wang L., Ping Z., Flory M. T., Wang G., … & Li W. (2013). Evaluating the risk of type 2 diabetes mellitus using artificial neural network: An effective classification approach. Diabetes Research and Clinical Practice, 100(1), 111–118. 10.1016/j.diabres.2013.01.023 [DOI] [PubMed] [Google Scholar]
- Wang L., Pedersen P. C., Agu E., Strong D. M., Tulu B. (2016). Area determination of diabetic foot ulcer images using a cascaded two-stage SVM-based classification. IEEE Transactions on Biomedical Engineering, 64(9), 2098–2109. 10.1109/TBME.2016.2632522 [DOI] [PubMed] [Google Scholar]
- WHO (2019). WHO guideline: Recommendations on digital interventions for health system strengthening. Retrieved 25 February 2022 fromhttps://www.who.int/reproductivehealth/publications/digitalinterventions-health-system-strengthening/en/
- Yazdanpanah L., Shahbazian H., Nazari I. (2018). Incidence and risk factors of diabetic foot ulcer: A population-based diabetic foot cohort (ADFC study)-two-year follow-up study. International Journal of Endocrinology, 10.1155/2018/7631659 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhang P., Lu J., Jing Y., Tang S., Zhu D., Bi Y. (2017). Global epidemiology of diabetic foot ulceration: A systematic review and meta-analysis. Annals of Medicine, 49(2), 106–116. 10.1080/07853890.2016.1231932 [DOI] [PubMed] [Google Scholar]
- Zohuri B., Moghaddam M. (2020). Deep learning limitations and flaws. Modern Approaches on Material Science, 2(3), 241–250. https://doi.org/10.32474/MAMS.2020.02.000138 [Google Scholar]





