Abstract
AIM
To develop a useful diabetic retinopathy (DR) screening tool for patients with type 2 diabetes mellitus (T2DM).
METHODS
A DR prediction model based on the Logistic regression algorithm was established on the development dataset containing 778 samples (randomly assigned to the training dataset and the internal validation dataset at a ratio of 7:3). The generalization capability of the model was assessed using an external validation dataset containing 128 samples. The DR risk calculator was developed through WeChat Developer Tools using JavaScript, which was embedded in the WeChat Mini Program.
RESULTS
The model revealed risk factors (duration of diabetes, diabetic nephropathy, and creatinine level) and protective factors (annual DR screening and hyperlipidemia) for DR. In the internal and external validation, the recall ratios of the model were 0.92 and 0.89, respectively, and the area under the curve values were 0.82 and 0.70, respectively.
CONCLUSION
The DR screening tool integrates education, risk prediction, and medical advice function, which could help clinicians in conducting DR risk assessments and providing recommendations for ophthalmic referral to increase the DR screening rate among patients with T2DM.
Keywords: diabetic retinopathy, type 2 diabetes mellitus, prediction model, screening
INTRODUCTION
Diabetic retinopathy (DR) is the leading cause of permanent and irreversible blindness in working-age adults globally[1]. The preventive effects of therapy and the fact that patients with proliferative DR or macular edema may be asymptomatic provide strong support for screening to detect DR[2]. The American Diabetes Association (ADA) recommends dilated and comprehensive eye examinations every 1-2y for patients with diabetes without evidence of retinopathy, and more frequent examinations are necessary if any level of DR is present or if sight is threatened[2]. However, not every patient undergoes annual dilated and comprehensive eye examinations because of lack of knowledge of DR, economic burden, and unevenly distributed resources for ophthalmic care[3]. As an article has reported, only approximately 60% of patients with diabetes undergo an annual dilated eye examination in the United States[4].
A large number of studies on the automated identification of DR have been conducted using fundus images[5]–[7]. However, these screening systems cannot meet the needs of a significant proportion of patients with diabetes. In some medically underserved areas, patients with diabetes are examined for their health in primary medical institutions, such as community hospitals. Most primary medical institutions are not equipped with fundus cameras and do not have ophthalmologists. Doctors not belonging to the ophthalmology and endocrinology departments often do not have the awareness of preventing DR when receiving a patient with diabetes and may not provide recommendations for DR screening.
Hence, we aimed to establish a DR risk prediction model and develop a DR screening tool which is easy to promote and operate, and integrates disease education, risk prediction, medical advice function to help patients with type 2 diabetes mellitus (T2DM) to self-manage and assist primary medical institutions in making referral recommendations.
SUBJECTS AND METHODS
Ethical Approval
This retrospective study was approved by the Chinese PLA General Hospital Clinical Research Ethics Committee (No.S2019-326-02, February 25, 2020) and adhered to the tenets of the Declaration of Helsinki.
Development Dataset
The prediction model was developed from the clinical data of inpatients diagnosed with T2DM extracted from the Chinese PLA General Hospital electronic medical record system from January 1, 2019 to December 31, 2019. The development dataset was randomly divided into training and internal validation sets. The training set, containing 70% of the samples, was used to train the prediction model. The internal validation set, obtaining the rest of the samples, was used to assess the prediction performance of the model to optimize the parameters.
Data Extraction Criteria
The first record of the measurements upon admission for each variable was extracted. Diagnostic information on the disease was extracted from the discharge diagnosis records. The retrieved variables included demographic characteristics (sex and age), duration of diabetes, living habits (smoking, drinking, and diet control), DR screening history [absence of DR screening, annual DR screening, and number of years until the first fundus examination after diabetes diagnosis (first DR screening)], diabetes complications (diabetic nephropathy, neuropathy, and retinopathy), other chronic diseases (hypertension, duration of hypertension, hyperlipidemia, atherosclerosis, and coronary heart disease), physical indicators (weight, height, and systolic and diastolic blood pressure), and laboratory parameters [levels of fasting blood glucose, postprandial blood glucose, glycosylated hemoglobin (HbA1c), triglycerides, total cholesterol, high-density lipoproteins, low-density lipoproteins, urea, and serum creatinine]. Body mass index was calculated as weight (kg) divided by height in meters squared (m2).
External Validation Dataset
The clinical data of inpatients diagnosed with T2DM extracted from the Strategic Support Force Medical Centre electronic medical record system in the same period were used for external validation. To better verify the generalization ability of the model, 64 patients with T2DM with DR and 64 patients with T2DM without DR were randomly selected, with a ratio of 1:1. The variables retained in the prediction model were extracted using the same extraction criteria.
Inclusion and Exclusion Criteria
The patients admitted in the Ophthalmology Department or had received an ophthalmic consultation were included in this study. The patients with missing data or those with cataracts, keratitis, corneal speckles, and other eye diseases that affect fundus examination were excluded.
Diagnostic Criteria
The diagnostic criteria for T2DM followed those established by the 2013 ADA[8]. DR was diagnosed according to the 2017 ADA Position Statement of DR[9] using color fundus photography and indirect ophthalmoscopy when the pupils were dilated. Fundus examination, image reading, and diagnosis were performed by at least two experienced ophthalmologists for each patient, whether directly admitted to the ophthalmology department or for ophthalmic consultation in other departments. All patients with diabetic fundus lesions, including mild non-proliferative DR, were included in the DR group.
Statistical Analysis
In this study, the R programming language (version 4.0.2) was used for statistical analyses. A baseline analysis was conducted on the development dataset. Continuous variables were expressed as mean±SD or median (interquartile range) after the normality distribution test. Categorical variables are expressed as numbers and percentages. The Student's t-test was performed on continuous variables that followed a normal distribution, while the Mann-Whitney U test was performed on those with skewed distributions. The Chi-squared test was used to compare categorical variables. Statistical significance was set at P<0.05.
Prediction Model Training and Internal Validation
The Lasso regression method was used to determine the optimal variables for feature selection. Feature selection was aimed at eliminating redundant factors without losing key information, and to obtain a factor set of lower dimensions, improve the accuracy, and reduce the complexity of the model.
Multiple Logistic regression analysis was applied to the training dataset to generate the DR prediction model. The variables included in the Logistic regression analysis were determined based on the feature selection results and clinical utility. The Hosmer-Lemeshow test was used to assess the goodness of fit of the model. Since the model was developed to screen patients requiring further fundus examination, we focused on the recall ratio [true positive/(true positive+false negative)] of the internal validation dataset when optimizing the parameters, and evaluated the overall performance of the model by accuracy, F-measure, and the areas under the receiver operator characteristic curves (ROC-AUC).
External Validation
External validation was performed to test the generalizability of the model. First, we conducted a statistical analysis of the differences between the model inclusion variables in the external validation and model development sets. The method was the same as the baseline analysis described above. Then, the trained model was applied to the external validation set to predict the occurrence of DR. The performance assessment indices were the same as those for internal validation.
WeChat Mini Program Development
To facilitate the predictive function in clinical practice, we designed and developed a DR risk prediction calculator for the above model. This calculator, developed through WeChat Developer Tools using JavaScript, was embedded in the WeChat Mini Program. The general schema for the prediction model building and the WeChat Mini Program development is shown in Figure 1.
Figure 1. The general schema for the prediction model building and the WeChat Mini Program development DR: Diabetic retinopathy.
RESULTS
Baseline Analysis of the Development Dataset
The data of 778 inpatients with T2DM, including 413 patients with DR and 365 patients without DR, and 28 variables were extracted. The baseline analysis of the development datasets is presented in Table 1. The patients with and without DR were similar in terms of sex, smoking history, drinking history, diet control, atherosclerosis, coronary heart disease, and levels of blood glucose, total cholesterol, and triglycerides. Diabetes duration was shorter in the non-DR group than that in the DR group [8.00 (3.00, 14.00) vs 15.00 (10.00, 20.00), P<0.01]. The DR group had a higher prevalence of hypertension, diabetic nephropathy, and diabetic nephropathy than the non-DR group.
Table 1. Baseline analysis results of 28 variables of 778 patients with T2DM.
| Variables | Non-DR (n=365) | DR (n=413) | P |
| Sex (female), % | 119 (32.6) | 147 (35.6) | 0.405 |
| Age, y | 54.55±14.48 | 58.67±10.99 | <0.01b |
| Duration of diabetes, y | 8.00 (3.00, 14.00) | 15.00 (10.00, 20.00) | <0.01b |
| Smoking, n (%) | 147 (40.3) | 154 (37.3) | 0.417 |
| Drinking, n (%) | 102 (27.9) | 118 (28.6) | 0.873 |
| Diet control, n (%) | 197 (54.0) | 216 (52.3) | 0.666 |
| Hypertension, n (%) | 207 (56.7) | 278 (67.3) | <0.01b |
| Duration of hypertension, y | 3.00 (1.00, 7.00) | 3.00 (1.00, 13.00) | 0.427 |
| Diabetic nephropathy, n (%) | 24 (6.6) | 169 (40.9) | <0.01b |
| Diabetic neuropathy, n (%) | 140 (38.4) | 203 (49.2) | <0.01b |
| Hyperlipidemia, n (%) | 272 (74.5) | 140 (66.1) | 0.012a |
| Coronary heart disease, n (%) | 73 (20) | 98 (23.7) | 0.225 |
| Atherosclerosis, n (%) | 197 (54) | 252 (61) | 0.05 |
| Annual DR screening, n (%) | 98 (26.8) | 38 (9.2) | <0.01b |
| First DR screening, y | 4.00 (1.00, 10.00) | 11.00 (5.00, 16.50) | <0.01b |
| Absence of DR screening, n (%) | 194 (53.2) | 189 (45.8) | 0.044a |
| BMI, kg/m2 | 26.12±3.65 | 25.77±3.43 | 0.171 |
| Systolic blood pressure, mm Hg | 133.60±18.98 | 141.51±22.92 | <0.01b |
| Diastolic blood pressure, mm Hg | 78.17±12.63 | 78.92±13.05 | 0.418 |
| Fasting blood glucose, mmol/L | 8.30±2.67 | 8.49±3.08 | 0.384 |
| Postprandial blood glucose, mmol/L | 12.45±3.84 | 13.00±4.34 | 0.06 |
| HbA1c, % | 8.25±2.02 | 8.25±2.13 | 0.974 |
| Urea, mmol/L | 5.40 (4.27, 6.65) | 5.96 (4.70, 8.29) | <0.01b |
| Serum creatinine, µmol/L | 72.00 (60.15, 86.90) | 80.90 (64.7, 122.75) | <0.01b |
| Total cholesterol, mmol/L | 4.18±1.21 | 4.22±1.33 | 0.632 |
| Triglyceride, mmol/L | 1.55 (1.08, 2.46) | 1.56 (1.06, 2.26) | 0.592 |
| High-density lipoproteins, mmol/L | 1.04 (0.85, 1.26) | 1.06 (0.88, 1.26) | 0.642 |
| Low-density lipoproteins, mmol/L | 2.60 (1.86, 3.17) | 1.06 (0.88, 1.26) | 0.887 |
BMI: Body mass index; HbA1c: Glycosylated hemoglobin; Non-DR: Diabetics without diabetic retinopathy; DR: Diabetic retinopathy; First DR screening: Number of years until the first fundus examination after diabetes diagnosis. aP<0.05; bP<0.01.
mean±SD
Only 9.2% of the DR group underwent DR screening annually; this proportion was significantly lower than that in the non-DR group (26.8%, P<0.01). The first DR screening was performed earlier in the non-DR group than that in the DR group [4.00 (1.00, 10.00) vs 11.00 (5.00, 16.50), P<0.01]. Almost half of the patients with diabetes did not undergo a fundus examination after diagnosis, indicating that 45.8% of those in the DR group underwent their first DR screening during this hospitalization and were found to have retinal lesions.
Baseline Comparison of the Development Dataset and External Dataset
According to the results of the Lasso regression, the following ten variables were selected: duration of diabetes, diabetic retinopathy, diabetic nephropathy, hyperlipidemia, annual DR screening, first DR screening, systolic blood pressure, and levels of postprandial blood glucose, serum creatinine, and triglycerides. Considering that few primary medical institutions will check for diabetic neuropathy during routine physical examinations, most users of this DR screening tool may not know whether they have diabetic neuropathy; therefore, we deleted this item. Meanwhile, we added two basic information: sex and age. Finally, 11 variables were selected to build the prediction model. The results of the baseline comparison of the development and external datasets are presented in Table 2.
Table 2. Baseline comparison of development dataset and external dataset.
| Variables | Development dataset (n=778) | External dataset (n=128) | P |
| Age, y | 56.74±12.90 | 61.66±7.84 | <0.01b |
| Sex (male), % | 512 (65.8) | 50 (39.1) | <0.01b |
| Duration of diabetes, y | 12.00 (5.00, 18.00) | 12.00 (8.25, 17.00) | 0.065 |
| Diabetic retinopathy, n (%) | 413 (53.1) | 64 (50) | 0.567 |
| Diabetic nephropathy, n (%) | 193 (24.8) | 11 (8.6) | <0.01b |
| Hyperlipidemia, n (%) | 545 (70.1) | 81 (63.3) | 0.077 |
| Annual DR screening, n (%) | 136 (17.5) | 18 (14.1) | 0.206 |
| First DR screening, y | 8.00 (1.00, 14.00) | 4.00 (1.00, 9.75) | <0.01b |
| Systolic blood pressure, mm Hg | 137.89±21.52 | 125.70±13.40 | <0.01b |
| HbA1c, % | 8.25±2.08 | 7.43±1.25 | <0.01b |
| Serum creatinine, µmol/L | 42.20 (38.98, 45.1) | 62.95 (53.43, 72.00) | <0.01b |
| Triglyceride, mmol/L | 1.56 (1.07, 2.36) | 1.49 (1.13, 2.12) | 0.602 |
HbA1c: Glycosylated hemoglobin; DR: Diabetic retinopathy; First DR screening: Number of years until the first fundus examination after diabetes diagnosis. aP<0.05; bP<0.01.
mean±SD
Model Training
A Logistic regression model was built on the training dataset, which consisted of 270 patients with DR and 247 patients without DR. The Hosmer-Lemeshow test showed good model fitness (P=0.157). The odds ratios (OR) for each variable are presented in Table 3. Diabetes duration (OR=1.05, 95%CI: 1.01-1.10, P<0.01), diabetic nephropathy (OR=5.83, 95%CI: 3.24-10.97, P<0.01), and serum creatinine level (OR=1.00, 95%CI: 1.00-1.01, P=0.02) were independent risk factors for DR, and annual DR screening (OR=0.34, 95%CI: 0.17-0.65, P<0.01) and hyperlipidemia (OR=0.62, 95%CI: 0.39-0.97, P=0.04) were independent protective factors.
Table 3. Results of multivariable Logistic regression analysis.
| Variables | OR (95%CI) | P |
| Age, y | 1.01 (0.99, 1.03) | 0.59 |
| Sex (female) | 1.21 (0.78, 1.87) | 0.395 |
| Duration of diabetes, y | 1.05 (1.01, 1.10) | <0.01b |
| Diabetic nephropathy, n (%) | 5.83 (3.24, 10.97) | <0.01b |
| Hyperlipidemia, n (%) | 0.62 (0.39, 0.97) | 0.04a |
| Annual DR screening, n (%) | 0.34 (0.17, 0.65) | <0.01b |
| First DR screening, y | 1.02 (0.97, 1.07) | 0.358 |
| Systolic blood pressure, mm Hg | 1.00 (1.00, 1.01) | 0.335 |
| HbA1c, % | 1.07 (0.95, 1.20) | 0.266 |
| Serum creatinine, µmol/L | 1.00 (1.00, 1.01) | 0.020a |
| Triglyceride, mmol/L | 0.94 (0.82, 1.06) | 0.338 |
Hosmer-Lemeshow test: P=0.157. HbA1c: Glycosylated hemoglobin; DR: Diabetic retinopathy; First DR screening: Number of years until the first fundus examination after diabetes diagnosis; OR: Odd ratio; CI: Confidence interval. aP<0.05; bP<0.01.
Internal and External Validation
For internal validation, the AUC reached a value of 0.82 (Figure 2). Since this is a screening model, we hoped to identify as many patients with DR as possible without significantly affecting the overall prediction accuracy, that is, to have a high recall ratio. When the probability of DR was >0.3, the model judged the sample as a DR sample. External validation was done to assess the generalizability of the model which attained the AUC value of 0.70 (Figure 2). The results of the performance assessment accuracy, recall ratio, F-measure, and AUC are detailed in Table 4.
Figure 2. ROC curve of validation set.
Table 4. Performance of prediction model in the internal and external validation dataset.
| Parameters | Accuracy | Recall ratio | F-measure | ROC-AUC |
| Internal dataset | 0.72 | 0.92 | 0.78 | 0.82 |
| External dataset | 0.61 | 0.89 | 0.69 | 0.70 |
ROC-AUC: Areas under receiver operator characteristic curves.
WeChat Mini Program
In this study, we developed a WeChat Mini Program named “Risk calculator for diabetic retinopathy”. The WeChat Mini Program for clinical practice can be widely used by patients with T2DM and medical staff (Figure 3). Users can visit the program on WeChat and use the prediction model by entering 11 clinical indicators. If the calculator determines that the user has DR, it will respond with a message saying, “High risk! Please visit the Ophthalmology Department as early as possible for a fundus examination.” If otherwise, it will feedback with a message that says, “Please follow your doctor's instructions for regular DR screening.” Meanwhile, we have designed a health education page on DR to improve awareness of DR screening and prevention for patients with diabetes.
Figure 3. The WeChat Mini Program of “Risk calculator for diabetic retinopathy”.
A: The blank calculator page; B: An example for a 75-year-old female user who suffered from type 2 diabetes for 16y. When she fills in the blanks and clicks Submit, a red message appears at the bottom of the page.
DISCUSSION
We developed an innovative and practical WeChat Mini Program for DR screening. This tool required 11 items of easily accessible case information, independent of fundus images, that could be used by patients with diabetes and primary medical institutions without ophthalmologists to determine whether a patient with diabetes should be referred to the ophthalmology department.
Several risk factors were identified in the development of the Logistic regression prediction model: duration of diabetes, diabetic nephropathy, and serum creatinine level, which are consistent with the findings of other studies[10]–[12]. Here, we included variables related to DR screening. Our results suggest that performing screening regularly has a positive effect on preventing DR occurrence (OR=0.34, P<0.01). This may be due to more background knowledge of the disease and better awareness of health management in these patients, as well as greater access to medical advice during the screening. According to the baseline analysis of the development dataset, only 17.48% of the 778 patients with T2DM underwent regular DR screening annually. Before hospitalization, almost half of the patients with diabetes did not undergo a fundus examination after having been diagnosed with T2DM [194 (53.2%) patients without DR and 189 (45.8%) patients with DR], thereby failing to meet the ADA guidelines for DR screening[2]. Given the low DR screening rate, doctors have much work to do. Regardless of whether in central hospitals or primary medical institutions, when non-ophthalmologist doctors receive patients with diabetes, it may be a simple and practical way to apply our DR screening tool to conduct a DR risk assessment and provide recommendations for referral in combination with the popular science page of DR.
One of the main intended use scenarios of this DR prediction tool is prior to a patient's visit to the ophthalmology department, with the goal of maximizing the screening rate for DR. We assumed that the loss caused by misdiagnosis of DR by this model is less than that by a missed diagnosis when patients do not receive an ophthalmologic evaluation after being diagnosed with diabetes. Therefore, when setting the prediction parameters of the model, we attempted to increase the recall ratio without significantly affecting the overall prediction effect; that is, we attempted to screen as many patients with DR as possible (recall ratio=0.92, AUC=0.82). Nevertheless, our model will still miss some patients with DR, so we have added a disease science page to enhance patients' awareness of preventing and treating DR. For patients who are predicted to be non-DR by the model, the reminder is “Please follow your doctor's instructions for regular DR screening.” to increase the DR screening rate further.
Remarkably, in the external validation dataset where baselines varied significantly, the prediction performance of the model achieved a recall rate of 0.89 and an AUC of 0.70. We believe that the generalization ability is acceptable. However, we advocate that every medical institution should be able to develop its own predictive model. The development of disease prediction models is not a one-time task. Even when applied only to the medical institution where the model is being developed, parameters need to be tuned regularly. Over time, the model should be assessed by updating it[13]. Since changes in Medicare policy, therapy, the comprehensive strength of these hospitals, follow-up schedule, the COVID-19 pandemic, among others, could affect the patient population and further differently affect the weight of each risk factor. Thus, the validation of a prediction model does not provide an absolute result, and it should be a dynamic process evolving over time[14].
A number of previous studies have combined DR prediction models with practical clinical tools. Histograms and scoring scales were common (Table 5)[15]–[19]. Users need to manually calculate the risk of disease, which may not be easy for patients to master using the method, with a probability of error and low efficiency. The offline or paper scale and histogram have a low promotion efficiency and are not easy to update in sync with the model. Online clinical tools, such as a study on the prediction of progression of chronic kidney disease (CKD)[20], can address these questions. Researchers designed and developed a CKD prediction system embedded in a web tool for the model they trained. Clinicians can visit the system website and enter the features from patients with CKD to obtain feedback of “mild” or “moderate/severe”[20]. The online tool can be accessed by anyone with an Internet connection and is user-friendly with its simple operation patterns. The computer calculates the results directly, with low error rates. Similarly, we developed a DR prediction program using the WeChat as the carrier. Patients and clinicians can access and bookmark this mini program while using this mainstream social software, which makes it more convenient to use and promote.
Table 5. Comparison with previous clinical tools.
| Parameters | Zhuang et al[16] | Kang et al[17] | Mendoza-Herrera et al[18] | Xiao et al[20] | The present study |
| Published time | 2019 | 2018 | 2017 | 2019 | 2021 |
| Disease | DR | NPDR | DR | CKD | DR |
| Training dataset | 413 | 413 | 11468 | 440 | 517 |
| Internal validation dataset | NA | NA | Ten-fold cross validation | 111 | 261 |
| External validation dataset | NA | NA | NA | NA | 128 |
| Prediction model | Logistic regression model | Cox proportional hazard model | Predictive probit model | Logistic regression model | Logistic regression model |
| Clinical tool | Nomograms | Nomograms | Scoring scale | Online tool | Online tool |
| ROC-AUC | 0.781 (training dataset) | NA | 0.78 | 0.87 (internal validation) | 0.82 (internal validation) 0.70 (external validation) |
DR: Diabetic retinopathy; NPDR: Non-proliferative diabetic retinopathy; CKD: Chronic kidney disease; ROC-AUC: Areas under receiver operator characteristic curves.
To our knowledge, this is the first study on the development of DR screening tool using social software. The main strength of this study is its practical way of having developed a DR prediction system to facilitate patient self-health management and assist non-ophthalmologists in referral-making decisions. Additionally, the factors used in the prediction model are all routine items in the health examination, which are easy to popularize in primary medical institutions. Most importantly, we collected data from another hospital and performed external validation to evaluate the tool's generalization ability.
Admittedly, this study has some limitations. The sample size of the model development dataset was not sufficiently large for a better prediction performance, and the parameters of the prediction model need to be further optimized. Moreover, it is difficult to collect external verification data, so multiple external validations were not performed.
In Conclusion, annual DR screening has a positive effect on preventing DR occurrence. The online DR prediction clinical tool, which integrates disease education, risk prediction, and medical advice functions, could assist in making medical decisions and provide a reference for studies on patient management.
Acknowledgments
Conflicts of Interest: Li WY, None; Yang M, None; Song YN, None; Luo L, None; Nie C, None; Zhang MN, None.
REFERENCES
- 1.Congdon NG, Friedman DS, Lietman T. Important causes of visual impairment in the world today. JAMA. 2003;290(15):2057–2060. doi: 10.1001/jama.290.15.2057. [DOI] [PubMed] [Google Scholar]
- 2.American Diabetes Association. 11. Microvascular Complications and Foot Care: Standards of Medical Care in Diabetes-2019. Diabetes Care. 2019;42(Suppl 1):S124–S138. doi: 10.2337/dc19-S011. [DOI] [PubMed] [Google Scholar]
- 3.Gibson DM. Eye care availability and access among individuals with diabetes, diabetic retinopathy, or age-related macular degeneration. JAMA Ophthalmol. 2014;132(4):471–477. doi: 10.1001/jamaophthalmol.2013.7682. [DOI] [PubMed] [Google Scholar]
- 4.Jones HL, Walker EA, Schechter CB, Blanco E. Vision is precious: a successful behavioral intervention to increase the rate of screening for diabetic retinopathy for inner-city adults. Diabetes Educ. 2010;36(1):118–126. doi: 10.1177/0145721709356116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–2410. doi: 10.1001/jama.2016.17216. [DOI] [PubMed] [Google Scholar]
- 6.Sayres R, Taly A, Rahimy E, Blumer K, Coz D, Hammel N, Krause J, Narayanaswamy A, Rastegar Z, Wu D, Xu S, Barb S, Joseph A, Shumski M, Smith J, Sood AB, Corrado GS, Peng L, Webster DR. Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy. Ophthalmology. 2019;126(4):552–564. doi: 10.1016/j.ophtha.2018.11.016. [DOI] [PubMed] [Google Scholar]
- 7.Gargeya R, Leng T. Automated Identification of diabetic retinopathy using deep learning. Ophthalmology. 2017;124(7):962–969. doi: 10.1016/j.ophtha.2017.02.008. [DOI] [PubMed] [Google Scholar]
- 8.American Diabetes Association. Diagnosis and classification of diabetes mellitus. Diabetes Care. 2013;36(Suppl 1):S67–S74. doi: 10.2337/dc13-S067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Solomon SD, Chew E, Duh EJ, Sobrin L, Sun JK, VanderBeek BL, Wykoff CC, Gardner TW. Diabetic retinopathy: a position statement by the American Diabetes Association. Diabetes Care. 2017;40(3):412–418. doi: 10.2337/dc16-2641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wang J, Xin X, Luo W, Wang R, Wang X, Si S, Mo M, Shao B, Wang S, Shen Y, Chen X, Yu Y. Anemia and diabetic kidney disease had joint effect on diabetic retinopathy among patients with type 2 diabetes. Invest Ophthalmol Vis Sci. 2020;61(14):25. doi: 10.1167/iovs.61.14.25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Zhang X, Kumari N, Low S, Ang K, Yeo D, Yeoh LY, Liu A, Kwan PY, Tang WE, Tavintharan S, Sum CF, Lim SC. The association of serum creatinine and estimated glomerular filtration rate variability with diabetic retinopathy in Asians with type 2 diabetes: a nested case-control study. Diab Vasc Dis Res. 2018;15(6):548–558. doi: 10.1177/1479164118786969. [DOI] [PubMed] [Google Scholar]
- 12.Alabdulwahhab KM, Sami W, Mehmood T, Meo SA, Alasbali TA, Alwadani FA. Automated detection of diabetic retinopathy using machine learning classifiers. Eur Rev Med Pharmacol Sci. 2021;25(2):583–590. doi: 10.26355/eurrev_202101_24615. [DOI] [PubMed] [Google Scholar]
- 13.Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009;338:b606. doi: 10.1136/bmj.b606. [DOI] [PubMed] [Google Scholar]
- 14.Toll DB, Janssen KJ, Vergouwe Y, Moons KG. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol. 2008;61(11):1085–1094. doi: 10.1016/j.jclinepi.2008.04.008. [DOI] [PubMed] [Google Scholar]
- 15.Semeraro F, Parrinello G, Cancarini A, Pasquini L, Zarra E, Cimino A, Cancarini G, Valentini U, Costagliola C. Predicting the risk of diabetic retinopathy in type 2 diabetic patients. J Diabetes Complications. 2011;25(5):292–297. doi: 10.1016/j.jdiacomp.2010.12.002. [DOI] [PubMed] [Google Scholar]
- 16.Zhuang X, Cao D, Yang D, Zeng Y, Yu H, Wang J, Kuang J, Xie J, Zhang S, Zhang L. Association of diabetic retinopathy and diabetic macular oedema with renal function in southern Chinese patients with type 2 diabetes mellitus: a single-centre observational study. BMJ Open. 2019;9(9):e031194. doi: 10.1136/bmjopen-2019-031194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kang EY, Lo FS, Wang JP, Yeh LK, Wu AL, Tseng YJ, Yeh CT, Liu L, Chen KJ, Wu WC, Lai CC, Wang NK, Chang Gung Juvenile Diabetes Eye Study Group Nomogram for prediction of non-proliferative diabetic retinopathy in juvenile-onset type 1 diabetes: a cohort study in an Asian population. Sci Rep. 2018;8(1):12164. doi: 10.1038/s41598-018-30521-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Mendoza-Herrera K, Quezada AD, Pedroza-Tobías A, Hernández-Alcaraz C, Fromow-Guerra J, Barquera S. A diabetic retinopathy screening tool for low-income adults in Mexico. Prev Chronic Dis. 2017;14:E95. doi: 10.5888/pcd14.170157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Azrak C, Palazón-Bru A, Baeza-Díaz MV, Folgado-De la Rosa DM, Hernández-Martínez C, Martínez-Toldos JJ, Gil-Guillén VF. A predictive screening tool to detect diabetic retinopathy or macular edema in primary health care: construction, validation and implementation on a mobile application. PeerJ. 2015;3:e1404. doi: 10.7717/peerj.1404. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Xiao J, Ding R, Xu X, Guan H, Feng X, Sun T, Zhu S, Ye Z. Comparison and development of machine learning tools in the prediction of chronic kidney disease progression. J Transl Med. 2019;17(1):119. doi: 10.1186/s12967-019-1860-0. [DOI] [PMC free article] [PubMed] [Google Scholar]