Prediction of Incident Diabetic Retinopathy in Adults With Type 1 Diabetes Using Machine Learning Approach: An Exploratory Study

Abstract

Background:

Early detection and intervention are crucial for preventing vision-threatening diabetic retinopathy (DR) in adults with type 1 diabetes (T1D). This exploratory study uses machine learning on continuous glucose monitoring (CGM) data to identify factors influencing DR and predict high-risk individuals for timely intervention.

Methods:

Between June 2018 and March 2022, adults with T1D with incident DR or no retinopathy (control) were identified. The CGM data were collected retrospectively for up to seven years before the date of defining incident DR or no retinopathy. A mixture of three machine learning algorithms was trained and evaluated in two different scenarios, using different glycemic features extracted from CGM traces (scenario 1), and the two principal components (two PCs; exposure to hyperglycemia and hypoglycemia risk) of those features (scenario 2). Classifiers were evaluated through 10-fold cross-validation using the receiver operating characteristic area under the curve (AUC-ROC) to select the best classification model.

Results:

The CGM data of 30 adults with incident DR (mean±SD age of 21.2±9.4 years, glycated hemoglobin [HbA_1c] of 8.6%±1.0%, and body mass index [BMI] of 24.5±4.8 kg/m²) and 30 adults without DR (age of 41.8±14.7 years, HbA_1c of 7.0%±0.9%, and BMI of 26.2±3.6 kg/m²) were included in this analysis. In scenario 2, classifiers outperformed scenario 1, resulting in an average AUC-ROC increase to 0.92 for two of three models, indicating that the two PCs captured vital classification data, representing the most discriminative aspects and enhancing model performance.

Conclusion:

Machine learning approaches using CGM data may have potential to aid in identifying adults with T1D at risk of DR.

Introduction

Diabetic retinopathy (DR) remains one of the most prevalent complications of diabetes and stands as the foremost cause of irreversible blindness among individuals of working age worldwide.^{1 -3} The susceptibility of adults with type 1 diabetes (T1D) to DR complications is heightened due to the earlier onset and inherent glycemic variability characteristic of T1D. ⁴ In addition to its impact on vision, DR indicates an increased risk of severe systemic vascular complications, posing significant threats to overall health. ⁵ Early detection and new treatments could prevent vision loss, blindness, and significantly reduce disease burden.

With increasing recognition of limitations of glycated hemoglobin (HbA_1c) in diabetes management, continuous glucose monitoring (CGM) metrics and its goal have been proposed. ⁶ To validate CGM-based time in range (TIR; 70-180 mg/dL) and its association with DR, Dr Beck and colleagues estimated TIR using seven-point finger stick glucose data obtained every three months in the Diabetes Control and Complications Trial (DCCT) and found a significant association between TIR derived from blood glucose measurements and the advancement of microvascular complications. ⁷ Many cross-sectional studies have also indicated a link between TIR derived from CGM and diabetes complications.^{8 -11} In addition, a recent first longitudinal study using up to seven years of CGM data for evaluating the association of CGM metrics with incident DR in adults with TID has reported that TIR, time in tight target range 70-140 mg/dL, time above 180 mg/dL, and mean glucose are highly correlated and similarly strongly associated with DR risk. ¹²

Lately, it has been demonstrated that the incorporation of CGM data into machine learning models can aid in the creation of predictive models, facilitating clinicians in enhancing diabetes screening and treatment. For instance, a logistic regression model utilized glycemic variability features extracted from CGM data to classify individuals according to their diabetes type (ie, with or without diabetes). ¹³ Besides that, we recently used a one-week CGM home test with a linear support vector machine (SVM) model to classify individuals’ autoantibody status (ie, antibody positive vs antibody negative) and used the same model to classify healthy individuals’ risk status (low-risk vs high-risk) of developing T1D.^14,15 Nevertheless, the challenge of visualizing CGM data in a particular format and automatically predicting the occurrence of DR from CGM data remains unexplored. Here, we want to explore the feasibility of using CGM profiles to predict DR risk in adults with T1D. The objective of this work is to classify adults with T1D with incident DR versus without DR by using CGM data and machine-learning approaches.

Methods

Study Design and Data Overview

We used the data from a previously reported study aimed to evaluate the association between CGM metrics and incident DR in adults with T1D. The detailed methods of this study were published previously. ¹² In brief, we used CGM data spanning up to seven years on a subset of individuals with incident DR (DR group; n=30) and those without DR (control group; n=30), chosen randomly from the original study. Incident DR was defined as the presence of DR in at least one retinal examination during the study period, with the two preceding retinal examinations showing no DR. Retinal examination reports, whether quantitative or qualitative, were manually checked from the electronic medical record (EMR) system, to ensure the accuracy of DR classification. Only patients with CGM use for at least one year and at least one clinic visit in the year before the date of eye examination were included in these analyses.

In addition, we applied strict criteria for the inclusion of daily CGM profiles, accepting only those with less than 60 minutes of missing data (out of 288 glucose readings per 24-hour period). Profiles with more than 60 minutes of missing data or those affected by calibration errors were excluded from the analysis. Missing data within acceptable profiles were interpolated to maintain the integrity of the data set.

The CGM-Based Glycemia Metrics and DR Group Comparison

The CGM-based metrics and assessment of glycemia by two groups (DR group vs control group) were conducted using daily CGM profiles. The CGM traces from the participants were collected and 16 glycemic metrics were extracted and computed, including (1) mean glucose (MG), (2) standard deviation (SD), (3) coefficient of variation (CV), (4) percent time of glucose >180 mg/dL (T>180), (5) >250 mg/dL (T>250), (6) >300 mg/dL (T>300), (7) <70 mg/dL (T<70), (8) <54 mg/dL (T<54), (9) TIR (70-180 mg/dL), (10) time in tight range (TITR; 70-140 mg/dL), (11) low blood glucose index (LBGI, measures the frequency and magnitude of hypoglycemia), (12) high blood glucose index (HBGI, measures the frequency and magnitude of hyperglycemia), (13) range (glucose range, difference between the highest and lowest CGM values), (14) the AUC above the baseline value at t=0 (IAUCN; incremental area under the curve for the night CGM traces from 12:00 am to 06:00 am), (15) glucose management indicator (GMI), and (16) the interquartile range (IQR). We used these 16 glycemic metrics as most of these metrics are clinically used for diabetes management and captures the dynamic characteristics of CGM profiles for each participant.

The DR Classification/Grouping Procedure

The extracted glycemic metrics from daily CGM profiles were used to define different classifier models based on the DR class. Then, these metrics were aggregated per participant and each metric was mean-centered and scaled before entering the classification procedure (categorization/grouping process). Principal component analysis (PCA) ¹⁶ was used as a dimensionality reduction technique for this analysis to address the collinearity among glycemic features and this technique has been used widely in machine learning to condense a vast data set into a more compact form, retaining essential patterns and trends. The PCA serves multiple functions within machine learning, encompassing tasks such as data visualization, feature extraction, noise mitigation, and enhancing algorithm efficiency through diminished computational intricacy. By reducing the dimensionality of the data, PCA can help uncover hidden patterns and relationships, making it a valuable tool in exploratory data analysis and model building. The number of PCs accounting for most of the total variance in the data was selected using the Kaiser criterion; ¹⁷ the percentage of variance explained by the selected PCs was then calculated. A mixture of three different classification models (linear, complex linear, and complex nonlinear models) was used to develop a DR classifier and define the best classifier model: linear discriminant analysis (LDA), SVM with linear kernel, and random forest (RF).^18,19 For all methods, the Caret function used to build the classifiers was used with default tuning parameters.

A 10-fold cross-validation technique was implemented in this analysis. ²⁰ The entire data set of 16 glycemic features from all participants is aggregated per participant and is randomly shuffled. Then, it was subdivided into 10 approximately equal-sized folds with stratified sampling. Stratified sampling plays a crucial role in ensuring that the proportions of different DR classes are maintained in each fold of the cross-validation process. One of the 10 folds was used as a test set to evaluate classification performance, while the remaining nine folds were used to train the classifier models with the optimal features. The process was iterated 10 times to gauge the average performance of diverse classifier models. Ensuring that data from each participant is either included in the training or test set aims to prevent overfitting and enhance the overall applicability of the findings.

Classifier Model Evaluation

To evaluate the performance of three models, a confusion matrix was employed to depict the four possible outcomes resulting from comparing the true and predicted classes: true negatives (TN), false negatives (FN), true positives (TP), and false positives (FP). The selection of the best performing classifier models was based on the receiver operating characteristic area under the curve (AUC-ROC), which provides a numerical representation of the trade-off between sensitivity (true-positive rate) and (one-specificity; false-positive rate) across various cutoff points. As the AUC-ROC value approaches 1, the model demonstrates increasingly effectiveness in discriminating between participants with and without DR.

Statistical Procedures

All statistical analyses were conducted using R Statistical Software version 4.0.5 (R Foundation for Statistical Computing). Continuous variables were described as mean±SD, and categorical variables were summarized as frequency (percentage). The Shapiro-Wilk test assessed the normality of glycemic feature distributions. For normally distributed continuous variables, a t test was used to compare the means between DR and without DR groups. Non-normally distributed variables were analyzed using the Wilcoxon signed-rank test to detect differences between glycemic features across DR groups. Significance was set at P < .05. Pearson correlation matrix was calculated to assess the collinearity between glycemic metrics.

Results

Demographic Characteristics

Adults without DR were older (41.8±14.7 years vs 21.2±9.4 years, P = .001), had a longer duration of diabetes (21.6±9.9 years vs 11.7±4.5 years, P = .001), and had a lower HbA_1c compared with adults with DR (7.0%±0.9% vs 8.6%±1.0%, P = .001), as shown in Table 1.

Table 1.

Clinical and Demographic Characteristics of 60 Subjects in the Two Different Classes of Incident Diabetic Retinopathy (DR) Used for Analysis.

Characteristic	Control group (without DR)	DR group (with DR)	P value
No. of subjects (N)	30	30	—
Age (y)	41.8 (14.7)	21.2 (9.4)	.001
Sex, % female	50	60	.604
Race, % non-Hispanic white/other	93.3	83.3	.195
HbA1C (%)	7.0 (0.9)	8.6 (1.0)	.001
BMI (kg/m2)	26.2 (3.6)	24.5 (4.8)	.234
Diabetes duration (y)	21.6 (9.9)	11.7 (4.5)	.001

Statistics are presented as N, mean (SD), or (%).

Abbreviations: BMI, body mass index; HbA_1c, glycated hemoglobin.

Glycemic Profile Between Two Groups

The supplementary figure shows the single ambulatory glucose profile (AGP) display between two groups. A total of 6414 complete daily CGM profiles were obtained from 30 adults with DR, whereas 8983 complete daily CGM profiles were collected from 30 adults without DR. The daily CGM profiles were different between two groups as shown in the supplementary figure.

Of 16 glycemic metrics, 13 were significantly different between adults without DR and with DR, and three metrics, namely, CV (P = .506), IAUCN (P = .101), and T<54 (P = .197) were statistically not different between two groups as shown in Figure 1. Compared with the adults without DR, mean TIR (66.1% vs 44.4%, P = .001) and TITR (43.9% vs 25.8%, P = .001) were significantly lower in adults with DR. The most distinguishable metrics between the two groups were the hyperglycemia-related metrics, such as T> 250 (8.5% vs 25.9%, P = .001), and T> 300 (3.0% vs 13.1%, P = .001).

Figure 1.

Characterization of daily CGM profiles through various glycemic features. Boxplots for 16 features were extracted from daily CGM profiles of 60 participants across two different groups of diabetic retinopathy (DR).

Abbreviations: CV, coefficient of variation; CGM, continuous glucose monitoring; GMI, glucose management indicator; HBGI, high blood glucose index; IAUCN, incremental area under the curve for the night CGM traces (mg/min/dL); IQR, interquartile range (mg/dL); LBGI, low blood glucose index; MG, mean glucose; SD, standard deviation; T>180, percent time >180 mg/dL; T>250, percent time >250 mg/dL; T>300, percent time >300 mg/dL; T<54, percent time <54 mg/dL; T<70, percent time <70 mg/dL; TIR, percent time in target range 70-180 mg/dL; TITR, percent time in tight target range 70-140 mg/dL.

A significance level of 5% (P value <.05) was considered to be significant to distinguish between the different groups of DR.

Defining Classifier Models Based on the DR Groups

Scenario 1

The three binary classifier models with a 10-fold cross-validation technique were implemented with all glycemic features (ie, 16 glycemic features) to classify participants in terms of Without DR versus With DR. A linear SVM classifier model outperformed the other classifier models with a mean AUC-ROC of 0.90. The LDA model and linear SVM model performed better than RF with a mean AUC-ROC of 0.86, 0.90, and 0.81 respectively, as shown in Figure 2.

Figure 2.

Comparison of classification performance of three models (LDA, SVM with a linear kernel, and RF) in terms of AUC-ROC based on different classes of DR (ie, without DR vs with DR) in two different scenarios (all features vs two PCs). (a) Represents the ROC curves for the 10-fold cross-validation, with the average AUC values annotated in red, by using 16 glycemic features in the three models (scenario 1). (b) Represents the ROC curves for the 10-fold cross-validation, by using only the two PCs in the three models (scenario 2). The color gradient represents the AUC values of individual folds, ranging from 0.4 (blue) to 1.0 (red). The dashed diagonal line represents a random classifier with an AUC of 0.5.

Abbreviations: LDA, linear discriminant analysis; SVM, support vector machine; RF, random forest; AUC-ROC, receiver operating characteristic area under the curve; PCs, principal components.

Scenario 2

As MG shows a strong correlation with hyperglycemia-related features, such as T>180, T>250, T>300, and HBGI (correlation range: 0.83-1.0), and GMI is a linear function of MG, indicating a correlation of 1.0, we applied PCA in this scenario to eliminate collinearity among glycemic features. The PCA was applied to the 16 CGM metrics separately for both DR groups and for the entire data set. In the overall data, two principal components (PCs) were selected based on the Kaiser criterion, explaining 87.09% of the original variance. The first PC accounted for 69.96% of the variability, and the second PC accounted for 17.13%. In the control group, the two PCs explained 87.53% of the variance, with the first PC accounting for 69.07% and the second PC accounting for 18.46% (Figure 3a). In the DR group, the two PCs explained 88.03% of the variance, with the first PC accounting for 62.58% and the second PC accounting for 25.45% (Figure 3b). The loadings for the two selected PCs in both DR groups are shown in Figure 3. As illustrated, the first PC (dimension 1 [dim 1]) is characterized by high loadings of MG and hyperglycemia-related metrics, such as T>180 and T>250, indicating exposure to hyperglycemia or therapy efficacy. The second PC (dimension 2 [Dim 2]) has high positive loadings of T<54, T<70, LBGI, and CV, reflecting the risk for hypoglycemia or therapy safety.

Figure 3.

The PCA for the 16 CGM metrics in the two different DR groups (control group (a); vs DR group (b)). The black vectors represent the coordinates of each metric (ie, loadings multiplied by the component standard deviations). On dimension 1 (dim 1), metrics have large positive loadings, such as MG, T>180, and T>250. On dimension 2 (dim 2), metrics have large positive loadings such as low blood glucose index (LBGI), T<54, and T<70. Positively correlated metrics point to the same side of the graph.

Abbreviations: PCA, principal component analysis CGM, continuous glucose monitoring; DR, diabetic retinopathy.

The three binary classifier models with a 10-fold cross-validation technique were implemented with the two PCs to classify participants in terms of without DR versus with DR. The LDA model and linear SVM model performed better than RF with a mean AUC-ROC of 0.92, 0.92, and 0.85 respectively, as shown in Figure 2.

Discussion

In this work, we used data from the first longitudinal study (retrospective study) that used up to seven years of CGM data prior to eye examination for predicting incident DR in adults with T1D. Although DR typically affects about 20% of people with diabetes, our study employed a case-control design with a balanced sample of 30 individuals with DR and 30 without DR. This approach ensures robust comparisons between groups while focusing on identifying predictive factors for DR. We found that lower TIR, lower TITR, higher T>180, higher T>250 (hyperglycemia-related glycemic features), and higher MG were all strongly associated with an increased risk of incident DR in adults with T1D (DR group). These glycemic features showed statistically significant differences between the DR groups. These results are consistent with prior studies that cross-sectionally, in both T1D and type 2 diabetes (T2D), assessed the association of CGM metrics with DR.^{7 -10,21}

In this study, we used these glycemic features in two different scenarios to predict incident DR and classify participants with incident DR versus without DR by using CGM data and three machine-learning models (LDA, SVM with linear kernel, and RF). In scenario 1, through using 16 glycemic features extracted from daily CGM profiles, a linear SVM outperformed LDA, and RF with an average AUC-ROC of 0.90. In scenario 2, the use of two PCs instead of all 16 glycemic features resulted in an increase in the average AUC for all three models. This indicates that the two PCs captured important information from the data that are relevant for classification. These PCs likely represent the most discriminative aspects of the data, leading to improved model performance. Among the models tested, SVM with linear kernel consistently achieved the highest average AUC, regardless of the feature set used. We highlight that our study uniquely concentrates on glycemic features derived from daily CGM profiles, whereas previous studies incorporated tabular features, such as risk factors, patient demographics, and comorbidity status into their machine-learning models.^22,23 In addition, demographic and clinical data, such as gender, diabetes type and duration, glycated hemoglobin (HbA_1c) or average blood glucose levels, blood pressure, and current retinopathy grade, were incorporated into a mathematical algorithm to estimate personalized screening intervals. ²⁴ Our study is distinctive in its longitudinal design, analyzing CGM data collected over several years prior to the onset of DR, thus avoiding the potential biases inherent in cross-sectional studies where CGM data is gathered post-DR development. This is also the first longitudinal study to employ CGM features and machine-learning approaches for predicting DR in adults with T1D. Additional strengths of our study include a well-characterized cohort of participants in both DR groups and DR determinations based on retinal eye examination records rather than self-reports.

In scenario 2, PCA identified two key metrics: quantifying hyperglycemia exposure (treatment efficacy) and assessing hypoglycemia risk (treatment safety). These metrics explained approximately 90% of the variance in both DR groups, consistent with previous studies.^25,26 The high percentage of variance accounted for by the two PCs in both groups indicates that the PCA model effectively captures the essential variability in the CGM data for both groups. In both groups, the first PC accounts for the majority of the variance, with a higher percentage in the control group (69.07%) compared with the DR group (62.58%). This suggests that the CGM data’s variability is predominantly captured by one component in the control group, indicating more uniform glucose control and more homogeneous glucose variability patterns among individuals without DR. In contrast, the second PC explains more of the variance in the DR group (25.45%) compared with the control group (18.46%). This higher contribution in the DR group indicates more complex and varied glucose control patterns, likely linked to the presence and progression of DR. These additional significant patterns in the CGM data of the DR group reflect the increased complexity and variability of glucose regulation in individuals with DR.

Limitations of the current exploratory study include a relatively small sample size although it did not hinder the identification of significant differences between groups. Retinopathy data were collected retrospectively from the EMR system (qualitative or quantitative reports) rather than through standardized eye examinations, such as those in the DCCT trial. Most of our cohort participants were non-Hispanic Whites, which may limit the generalizability of our findings to more diverse populations. We also acknowledge that the LDA model assumes equal variance and normality, assumptions that might not hold in our data set. However, the inclusion of LDA was exploratory and aimed at comparing its performance with more flexible models, such as SVM and RF, which do not require such assumptions. The LDA’s performance was evaluated within this broader context of model comparison, where the primary focus remained on more robust models. Furthermore, although the high AUC-ROC values observed are encouraging, we recognize that they do not necessarily translate into immediate clinical usefulness. Our models were evaluated for their predictive power within the current data set, with steps taken to avoid overfitting through techniques, such as cross-validation and PCA. However, further validation of these models in larger, independent data sets is necessary to establish their generalizability. Future studies should incorporate larger and more diverse cohorts to validate the performance of our models and the predictive power of the selected features. In addition, integrating demographic and clinical variables, such as age and diabetes duration, alongside CGM metrics within the PCA framework, would allow for a more nuanced investigation of their interactions. This could provide a more comprehensive understanding of DR risk and how these factors contribute to its development.

Conclusion

In this exploratory study, we demonstrate that integrating CGM into diabetes management not only enhances glycemic control but also significantly lowers the risk of DR and other microvascular complications. By leveraging machine learning, we developed a method to differentiate CGM patterns between individuals with and without DR, independent of eye examinations. Broad application of this approach could prevent DR progression before clinical signs emerge, thereby reducing the risk of vision loss, blindness, and overall disease burden significantly.