Predicting acute and chronic diabetes complications among adults newly diagnosed with diabetes: development and validation of dynamic risk prediction models

Abstract

Diabetes affects one in ten adults worldwide, yet clinicians lack tools to simultaneously predict which complications a given patient is most likely to develop in the near term. Existing risk models typically address single complications, were developed in specialized cohorts, and use heterogeneous time horizons, limiting their utility for individualized clinical decision-making. Here, we show that machine learning models trained on nationwide U.S. administrative claims data for 400,400 adults with newly diagnosed type 1 or type 2 diabetes can accurately predict the monthly risk of nine acute and chronic complications (cardiovascular disease, cerebrovascular disease, peripheral vascular disease, nephropathy, neuropathy, retinopathy, and hypoglycemic and hyperglycemic crises) with predictions that update dynamically as new clinical data become available. Using Regularized Logistic Regression (GLMnet) and Extreme Gradient Boosting (XGBoost) with walk-forward temporal validation, we demonstrate strong discrimination in external validation against an independent electronic health record cohort (areas under the receiver operating characteristic curve 0.77–0.85 across complications for the best-performing model). The Diabetes Complications Risk Calculator therefore provides encounter-specific near-term risk estimates for multiple diabetes complications using routinely available clinical data, offering a potential framework for personalized risk stratification that could support shared decision-making and population health management across healthcare settings. The generalizability of the Diabetes Complications Risk Calculator to other cohorts will need to be tested.

Here the researchers developed novel machine learning models that use routine clinical data to predict the risks of the nine common acute and chronic diabetes complications with predictions that update dynamically as new clinical data become available, with validation across two independent U.S. healthcare populations.

Introduction

Nearly one in seven American adults¹ and 11% adults worldwide² are living with diabetes. While the predominant therapeutic approach to managing diabetes is to reduce blood glucose levels while targeting therapy to existing comorbidities and complications – most notably cardiovascular and kidney disease³ - the ultimate goal of treatment is to prevent diabetes complications, ideally before they develop or incur harm. This includes acute severe dysglycemic events (i.e., hypoglycemic and hyperglycemic crises), chronic macrovascular complications (i.e., cardiovascular disease [CVD], cerebrovascular disease [CeVD], and peripheral vascular disease [PVD]), and chronic microvascular complications (i.e., retinopathy, nephropathy, and neuropathy). Personalized diabetes management is predicated on the ability to understand, anticipate, and contextualize each person’s risk for experiencing each of these complications.

Multiple models have been developed to predict atherosclerotic cardiovascular events (typically as a composite of fatal and non-fatal CVD and CeVD events)^4–12, chronic kidney disease^13–16, retinopathy^17–19, and severe hypoglycemia^20–23. However, these models were developed in different populations, often using specialized cohorts from prospective observational or interventional studies, and using heterogeneous methods and time horizons, resulting in uncertainty about how these models would perform if implemented in routine practice conditions to examine an individual patient’s risk of experiencing each of the acute and chronic complications at a specific point in time. Such real-time individualized data are necessary to inform point-of-care shared decision-making about goals of diabetes management, optimal therapeutic targets, and preferred pharmacologic agents that are most likely to reduce the totality of the patient’s complications’ risk rather than each complication alone.

However, there are challenges to building and implementing such models in the healthcare space. These include lack of data that would be sufficiently large, heterogeneous, longitudinal, and granular to enable the development and validation of models for many diabetes complications concurrently; use of single-center data that often lacks racial and ethnic diversity; reliance on simple statistical modeling approaches that are often unable to learn, represent, and interpret complex underlying relationships in the data; and the scarcity of adequate computing power to implement complex models on very large data inputs.

In this work, we followed a systematic and structured approach to develop and internally validate a suite of longitudinal machine learning (ML) risk prediction models for acute and chronic diabetes complications—the Diabetes Complications Risk Calculator (DCRC)—in a large, nationwide claims dataset. To establish trustworthiness, reproducibility, and generalizability, we subsequently conducted external validation of the models in the electronic health record (EHR) of an integrated healthcare delivery system spanning four states. We ultimately demonstrate how—by relying on variables routinely available in claims data—it is possible to develop a generalizable and scalable approach to predicting diabetes complications that could be implemented across diverse settings and populations.

Results

Study cohorts

In the Optum Labs Data Warehouse (OLDW; used for the development and internal validation cohorts), we identified 400,400 adults with newly diagnosed diabetes meeting study eligibility criteria, including 5603 (1.39%) with type 1 diabetes. Their mean follow-up was 34.6 (SD, 29.3) months. The mean age of the study population was 57.36 years (SD, 13.53), 50.5% were women, and 61.8% were non-Hispanic white (Table 1). The Mayo Clinic external validation cohort included 10,201 adults with diabetes, of whom 566 (5.5%) had type 1 diabetes, with mean age 62.18 (SD, 13.80) years, 44.6% female, and 92.3% non-Hispanic white.

Table 1.

Patient characteristics

	Development/Internal Validation, N (%)	External Validation, N (%)
Number of Patients	400,400	10,201
Sociodemographic Variables
Age, years, mean (SD)	57.36 (13.53)	62.18 (13.80)
Race/ethnicity
Asian	19,528 (4.9)	251 (2.5)
Black	64,338 (16.1)	342 (3.4)
Hispanic	58,634 (14.6)	89 (0.9)
White	247,471 (61.8)	9418 (92.3)
Other/Unknown	10,429 (2.6)	101 (1.0)
Sex
Women	202,147 (50.5)	4546 (44.6)
Men	198,253 (49.5)	5655 (55.4)
U.S. census region
Midwest	58,028 (14.5)	6981 (68.4)
Missing/unknown	271 (0.1)	0 (0.0)
Northeast	48,162 (12.0)	0 (0.0)
South	247,058 (61.7)	661 (6.5)
West	46,881 (11.7)	2559 (25.1)
Clinical Variables
Total number of health conditions	7.98 (10.86)	8.98 (6.94)
Total number of medications	5.03 (3.01)	7.31 (5.57)
Diabetes Type
Type 1 diabetes	5603 (1.4)	566 (5.5)
Type 2 diabetes	394,797 (98.6)	9635 (94.5)
Comorbidities/complications
Composite (any complication)	68,403 (17.1)	3664 (35.9)
Neuropathy	21,114 (5.3)	775 (7.6)
Peripheral vascular disease	9275 (2.3)	400 (3.9)
Nephropathy	13,702 (3.4)	1146 (11.2)
Cardiovascular disease	37,371 (9.3)	2032 (19.9)
Retinopathy	5041 (1.3)	195 (1.9)
Hyperglycemic crisis	733 (0.2)	53 (0.5)
Severe hypoglycemia	3059 (0.8)	71 (0.7)
Cerebrovascular disease	8644 (2.2)	297 (2.9)
Lab results, mean (SD)
Hemoglobin A1c (%)	7.17 (1.67)	7.13 (1.75)
Creatinine (mg/dL)	0.93 (0.38)	1.07 (1.01)
ALT (IU/L)	30.70 (26.51)	41.20 (93.03)
AST (IU/L)	25.51 (19.82)	37.71 (133.48)
Alkaline phosphatase (IU/L)	81.28 (29.56)	88.28 (52.20)
Hemoglobin (g/dL)	13.94 (1.51)	13.72 (1.88)
Total Cholesterol (mg/dL)	190.19 (41.14)	180.26 (47.30)
LDL Cholesterol (mg/dL)	108.00 (31.18)	97.49 (34.47)
HDL Cholesterol (mg/dL)	46.92 (12.29)	46.38 (14.35)
Triglycerides (mg/dL)	183.65 (195.03)	189.65 (169.45)
TSH (mIU/mL)	2.40 (3.26)	2.81 (4.33)

Baseline characteristics of the Optum Labs Data Warehouse (development and internal validation) and Mayo Clinic (external validation) cohorts.

HDL high-density lipoprotein, LDL low-density lipoprotein, TSH thyroid-stimulating hormone, ALT alanine aminotransferase, AST aspartate aminotransferase.

Development and Internal Validation

The DCRC models included Regularized Logistic Regression (GLMnet) and Extreme Gradient Boosting (XGBoost), with Cox proportional hazards (Cox PH) regression evaluated as a traditional statistical comparator. Supplemental Table s1 presents the optimal hyperparameters selected for the GLMnet and XGBoost models during model development; Cox PH regression does not require hyperparameter selection. The composite outcome of any diabetes complication occurred in 89,229 (22.20%) patients at 3 months, 111,203 (29.32%) patients at 6 months, 129,288 (32.29%) patients at 9 months, 145,228 (34.96%) patients at 1 year, and 193,290 (44.24%) patients at 2 years in the follow-up period (Fig. 1). By the end of follow-up (172 months), 261,032 (65.04%) patients experienced at least one diabetes complication, increasing from 17.08% at baseline.

Fig. 1. Prevalence of having any diabetes complication over time.

Prevalence of the composite outcome (any acute or chronic diabetes complication) over time in the Optum Labs Data Warehouse cohort (including both the development and internal validation cohorts).

Supplemental Table s2 presents the accuracy, area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the GLMnet and XGBoost models for predicting the risk of each of the diabetes complications on the development cohort based on the 13-fold walk-forward validation (WFV). The XGBoost model achieved excellent performance (AUC ≥ 85) in predicting each of the diabetes complications, with best performance for predicting CeVD (AUC 0.91 [95% CI 0.80–0.95]), nephropathy (AUC 0.90 [95% CI 0.78–0.93]), and hypoglycemic crises (AUC 0.90 [95% CI 0.85–0.93]), and lowest for neuropathy (AUC 0.85 [95% CI 0.77–0.90]) and the complications composite (AUC 0.85 [95% CI 0.76–0.89]). The GLMnet model on the other hand mostly exhibited good performance for all complications (AUC between 0.69–0.80).

Internal validation (Table 2) showed very strong performance for both models (XGBoost AUC between 0.66 and 0.83, GLMnet AUC between 0.71 and 0.84). However, the performance of the XGBoost model deteriorated on the internal validation cohort, while the GLMnet model slightly improved. Similar performance trends were observed for accuracy, sensitivity, specificity, F1-Score, PPV, and NPV. The models showed good tradeoff between the sensitivity and specificity.

Table 2.

Internal validation of the Diabetes Complications Risk Calculator models (primary analysis)

Complication	Model	Accuracy	AUC	Sensitivity	Specificity	F1 score	PPV	NPV
Cardiovascular disease	XGBoost	0.72	0.83	0.82	0.67	0.66	0.56	0.88
	GLMnet	0.77	0.84	0.74	0.79	0.69	0.64	0.86
	Cox PH	0.74	0.74	0.64	0.54	0.56	0.57	0.79
Neuropathy	XGBoost	0.57	0.71	0.78	0.50	0.45	0.32	0.89
	GLMnet	0.72	0.72	0.54	0.77	0.46	0.41	0.85
	Cox PH	0.67	0.66	0.40	0.65	0.31	0.53	0.76
Cerebrovascular disease	XGBoost	0.74	0.80	0.71	0.74	0.36	0.24	0.96
	GLMnet	0.77	0.83	0.72	0.78	0.39	0.27	0.96
	Cox PH	0.33	0.75	0.97	0.10	0.36	0.25	0.96
Nephropathy	XGBoost	0.73	0.82	0.77	0.72	0.50	0.37	0.94
	GLMnet	0.77	0.83	0.72	0.78	0.52	0.41	0.93
	Cox PH	0.71	0.76	0.65	0.60	0.50	0.46	0.85
Peripheral vascular disease	XGBoost	0.68	0.75	0.68	0.68	0.35	0.24	0.94
	GLMnet	0.74	0.77	0.63	0.76	0.38	0.27	0.93
	Cox PH	0.62	0.74	0.76	0.48	0.44	0.35	0.90
Retinopathy	XGBoost	0.83	0.70	0.42	0.85	0.21	0.14	0.96
	GLMnet	0.76	0.74	0.59	0.77	0.21	0.13	0.97
	Cox PH	0.68	0.69	0.40	0.71	0.19	0.31	0.89
Hypoglycemic crisis	XGBoost	0.84	0.66	0.38	0.86	0.13	0.08	0.98
	GLMnet	0.66	0.71	0.64	0.66	0.10	0.06	0.98
	Cox PH	0.67	0.64	0.43	0.63	0.17	0.18	0.95
Hyperglycemic crisis	XGBoost	0.97	0.66	0.18	0.98	0.11	0.08	0.99
	GLMnet	0.90	0.75	0.43	0.90	0.08	0.04	0.99
	Cox PH	0.85	0.72	0.29	0.86	0.11	0.19	0.97
Composite (any diabetes complication)	XGBoost	0.69	0.81	0.93	0.37	0.78	0.67	0.80
	GLMnet	0.73	0.81	0.70	0.77	0.75	0.80	0.65
	Cox PH	0.60	0.71	0.45	0.80	0.48	0.67	0.55

Performance of models predicting new (incident) diabetes complications during the follow-up period in the internal validation cohort. This primary analysis excludes patients with baseline-prevalent complications to ensure that model performance reflects true prospective predictive capability. Bolded AUC values indicate superior performance.

The sensitivity analysis examining both incident and prevalent diabetes complications (Table 3) yielded similar results to those of the incident-only complications.

Table 3.

Sensitivity analysis of the Diabetes Complications Risk Calculator models in the internal validation cohort (incident and prevalent cases)

Complication	Model	Accuracy	AUC	Sensitivity	Specificity	F1-Score	PPV	NPV
Cardiovascular disease	XGBoost	0.73	0.84	0.84	0.64	0.72	0.63	0.85
	GLMnet	0.75	0.83	0.76	0.75	0.72	0.68	0.82
Neuropathy	XGBoost	0.56	0.73	0.83	0.44	0.54	0.4	0.86
	GLMnet	0.66	0.72	0.66	0.66	0.54	0.46	0.81
Cerebrovascular disease	XGBoost	0.73	0.82	0.75	0.73	0.44	0.31	0.95
	GLMnet	0.73	0.81	0.75	0.73	0.44	0.31	0.95
Nephropathy	XGBoost	0.71	0.83	0.80	0.69	0.56	0.43	0.92
	GLMnet	0.73	0.81	0.76	0.72	0.56	0.44	0.91
Peripheral vascular disease	XGBoost	0.67	0.77	0.73	0.66	0.42	0.29	0.93
	GLMnet	0.69	0.76	0.69	0.69	0.42	0.30	0.92
Retinopathy	XGBoost	0.79	0.74	0.56	0.81	0.31	0.21	0.95
	GLMnet	0.65	0.73	0.70	0.65	0.25	0.15	0.96
Hypoglycemic crisis	XGBoost	0.81	0.75	0.57	0.82	0.24	0.15	0.97
	GLMnet	0.51	0.71	0.78	0.5	0.14	0.08	0.98
Hyperglycemic crisis	XGBoost	0.97	0.75	0.37	0.97	0.25	0.19	0.99
	GLMnet	0.80	0.75	0.56	0.81	0.08	0.04	0.99
Composite of any complication	XGBoost	0.74	0.81	0.94	0.31	0.83	0.75	0.72
	GLMnet	0.73	0.79	0.76	0.68	0.80	0.84	0.57

Performance of the XGBoost and GLMnet models in predicting diabetes complications when both incident and baseline-prevalent cases were included in the Optum Labs Data Warehouse internal validation cohort. This sensitivity analysis complements the primary incident-only analysis by assessing model robustness under a more inclusive outcome definition.

External validation

Compared to the internal validation results, the GLMnet model demonstrated substantially better results in predicting each of the diabetes complications in the external validation data, outperforming the XGBoost model (Table 4). Specifically, GLMnet achieved excellent performance in predicting 8 out of the 10 outcomes: CVD (AUC, 0.84), CeVD (AUC, 0.84), nephropathy (AUC, 0.84), PVD (AUC, 0.80), retinopathy (AUC, 0.80), hypoglycemic crises (AUC, 0.85), and the composite (AUC, 0.83); and good performance in predicting neuropathy (AUC, 0.79), and hyperglycemic crises (AUC, 0.77). XGBoost attained excellent performance for only three of the outcomes: CVD (AUC, 0.82), nephropathy (AUC, 0.82), and the composite (AUC, 0.81). The XGBoost model showed no discriminatory ability in predicting hyperglycemic crises (AUC, 0.51).

Table 4.

External validation of the Diabetes Complications Risk Calculator models (primary analysis)

Complication	Model	Accuracy	AUC	Sensitivity	Specificity	F1 Score	PPV	NPV
Cardiovascular disease	XGBoost	0.69	0.82	0.83	0.64	0.46	0.59	0.91
	GLMnet	0.77	0.84	0.75	0.78	0.64	0.56	0.89
	Cox PH	0.79	0.72	0.64	0.52	0.62	0.66	0.61
Neuropathy	XGBoost	0.61	0.75	0.80	0.56	0.34	0.48	0.91
	GLMnet	0.76	0.79	0.60	0.80	0.52	0.46	0.88
	Cox PH	0.68	0.64	0.38	0.67	0.34	0.60	0.58
Cerebrovascular disease	XGBoost	0.76	0.79	0.67	0.77	0.22	0.33	0.96
	GLMnet	0.78	0.84	0.71	0.79	0.36	0.24	0.97
	Cox PH	0.43	0.73	0.98	0.08	0.45	0.37	0.97
Nephropathy	XGBoost	0.74	0.82	0.77	0.73	0.44	0.56	0.92
	GLMnet	0.77	0.84	0.75	0.78	0.59	0.49	0.92
	Cox PH	0.72	0.69	0.69	0.53	0.58	0.57	0.66
Peripheral vascular disease	XGBoost	0.70	0.74	0.67	0.70	0.22	0.33	0.95
	GLMnet	0.76	0.80	0.68	0.77	0.38	0.26	0.95
	Cox PH	0.70	0.71	0.73	0.48	0.50	0.45	0.74
Retinopathy	XGBoost	0.71	0.69	0.57	0.72	0.15	0.24	0.95
	GLMnet	0.62	0.80	0.84	0.60	0.26	0.16	0.98
	Cox PH	0.62	0.69	0.48	0.62	0.30	0.43	0.75
Hypoglycemic crisis	XGBoost	0.84	0.74	0.57	0.84	0.04	0.07	0.99
	GLMnet	0.58	0.85	0.89	0.58	0.04	0.02	1.00
	Cox PH	0.75	0.81	0.54	0.65	0.29	0.28	0.95
Hyperglycemic crisis	XGBoost	0.94	0.51	0.10	0.97	0.12	0.10	0.97
	GLMnet	0.85	0.77	0.47	0.87	0.18	0.11	0.98
	Cox PH	0.80	0.69	0.23	0.90	0.24	0.40	0.78
Composite (any diabetes complication)	XGBoost	0.67	0.81	0.93	0.43	0.61	0.74	0.74
	GLMnet	0.75	0.83	0.69	0.80	0.73	0.78	0.73
	Cox PH	0.57	0.71	0.48	0.79	0.52	0.73	0.46

Performance of models predicting incident diabetes complications during the follow-up period in the Mayo Clinic external validation cohort. This primary analysis excluded patients with baseline-prevalent complications to ensure that performance metrics reflect true prospective predictive capability across institutions. Bolded AUC values indicate superior performance.

Supplemental Figs. s1–s9 show the ROC, cumulative gain, sensitivity, PPV, and calibration plots of the DCRC for predicting diabetes complications in the external validation cohort. The GLMnet and XGBoost models showed similar performance in predicting the composite outcome, with a slight advantage for GLMnet (Supplemental Fig. s1).

Results from the Cox PH models are provided in Supplemental Table s3, Table 2, and Table 4. Across both the OLDW internal validation and the Mayo Clinic external validation cohorts, Cox models demonstrated reasonable performance for some complications but were consistently outperformed by the machine learning models, which achieved substantially higher discrimination overall.

Subgroup analysis

In post-hoc analyses of the Mayo Clinic external validation cohort, model discrimination remained broadly consistent across age, sex, and race/ethnicity subgroups, with only modest changes in performance relative to the overall cohort (average ΔAUC ranging from 0.00 to −0.03 across models and subgroup categories). However, threshold-based performance metrics revealed more pronounced heterogeneity. In particular, among younger adults (age 18–44), the GLMnet model exhibited high specificity but reduced sensitivity for most complications, while XGBoost showed higher sensitivity for the composite outcome but limited sensitivity for several individual complications. In the older age group ( ≥75 years), the opposite pattern was observed for some outcomes, with higher sensitivity but lower specificity, potentially leading to over-identification. Performance across sex and race/ethnicity subgroups showed smaller deviations in threshold-based metrics. Across subgroup analyses, GLMnet demonstrated slightly greater discrimination stability than XGBoost, consistent with its superior performance in the primary external validation (Supplemental Tables s4-5).

Laboratory ablation analysis

To assess robustness in laboratory-sparse settings, we performed post-hoc data-level ablation analyses in the external validation cohort using a carry-forward approach that replaced current laboratory values with the most recent prior values, and a permutation-based robustness analysis. Carry-forward ablation resulted in only minor reductions in performance (average ΔAUC of −0.02 for GLMnet and −0.01 for XGBoost), indicating that model predictions remain stable even when up-to-date laboratory data are unavailable. Permutation-based ablation yielded similarly modest changes overall (average ΔAUC of −0.03 for GLMnet and −0.02 for XGBoost), suggesting that while laboratory features contribute predictive signal, the models do not rely exclusively on these inputs. Together, these findings demonstrate that model performance is robust to variations in laboratory data availability (Supplemental Table s6).

Explainability: drivers of high diabetes complications risk

Figure 2 presents Shapley Additive Explanations (SHAP) plots for the GLMnet model predicting the composite outcome complication models for a random sample (10%) of patients in internal validation cohort.

Fig. 2. SHAP summary plot for the top 20 variables of the GLMnet model contributing to predict the composite outcome of any diabetes complication.

Positive SHAP values (x-axis) indicate the variable contribute to push the model to make high risk prediction, while negative values indicate the variable contribute to push the model to make low risk prediction. Variables on the y-axis are sorted by the sum of the absolute SHAP values over all patients. Each plotted point represents a patient, and the color represents the variable value: red indicates high and green indicates low.

The top five drivers of high-risk were advanced age, hypertension with complications, longer duration of diabetes, heart valve disorders, high creatinine, and other connective tissue disease. However, the coloring depicted by some of the variables indicates that the above relationships do not necessarily apply to every patient. For example, while on average longer diabetes duration indicates an overall greater risk for diabetes complications (more points are skewed towards the red spectrum), for some patients (the green points to the right, with positive SHAP values), short diabetes duration may also predict heighted risk of any diabetes complication. Supplemental Figs. s10–s17 present SHAP plots for the top 20 variables from each GLMnet diabetes complication models for a random sample (10%) of patients in the internal validation cohort.

Explainability: temporal dynamics of predicted risk

Having identified key drivers of risk with the GLMnet model, we next evaluated how predicted risk for the composite diabetes complication endpoint changes over time within individuals. Using encounter-level predictions recalculated monthly, Fig. 3 presents trajectories for six representative patients in the OLDW validation cohort. The trajectories display diverse temporal patterns—rising, declining, stable, or fluctuating—consistent with updates in comorbidities, clinical measurements, and healthcare utilization recorded between encounters in response to changes in individual features.

Fig. 3. Predicted risk trajectories for six representative patients in the OLDW internal validation cohort (primary incident-only analysis).

Predicted probabilities of the composite incident diabetes complication endpoint recalculated monthly for six randomly selected patients from the OLDW internal validation cohort based on the GLMnet model. The trajectories illustrate diverse temporal patterns, including steadily rising, declining, fluctuating, or stable risk profiles, reflecting changes in underlying clinical status, comorbidities, and care utilization over time.

To assess sensitivity to mutable factors, we performed a perturbation analysis by varying age ( ±15 years) for the same six patients while holding other covariates fixed and recomputed predictions with the GLMnet model. As shown in Supplemental Fig. s18, increasing age uniformly shifts the predicted risk of the composite endpoint upward (and decreasing age shifts it downward), with effects that accumulate over time. Together, these results demonstrate that the GLMnet model yields encounter-specific, dynamic risk estimates for the composite endpoint that are responsive to both the clinical course and clinically relevant changes in risk factors.

Discussion

We used administrative claims data for a U.S. nationwide cohort of adults with newly diagnosed diabetes to develop and validate the DCRC, a series of dynamic risk prediction models for acute and chronic diabetes complications, which—as demonstrated by our successful external validation using EHR data for a distinct patient population—can be readily deployed in clinical practice. Indeed, the DCRC models were accurate and reliable in both the internal and external validation datasets, demonstrating that ML models can be effectively deployed both in different populations (single integrated healthcare delivery system as opposed to national data) and different data types (EHR as opposed to claims). Thus, the DCRC can be implemented in a wide range of settings to predict short-term risks of both acute and chronic complications of diabetes, inform shared decision-making at the point-of-care, and ultimately support precision approaches in diabetes care both at individual and population levels.

Precision medicine in diabetes is predicated on tailoring diagnostics and therapeutics to individuals most likely to benefit from them through accurate, reliable, and efficient diagnosis, prevention, and treatment of diabetes and its complications by integrating multidimensional data and accounting for individual differences²⁴. The 2020 consensus report from the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD) on precision medicine in diabetes called for precision medicine approaches to treating diabetes that focus on outcomes of highest relevance to patients (i.e., hard outcomes), are relevant to broader populations and ethnic groups, and can be implemented in clinical practice as decision support tools²⁴. Major challenges to meeting these goals include the interconnectedness of diabetes complications, which makes it difficult to derive models for isolated complications that do not consider all complications in parallel and to use models that were developed in cohorts without diabetes; as well as the need for diverse, heterogeneous, and representative model development and validation cohorts that are often infeasible in single center EHRs or prospective cohort studies²⁵. Our work, and the resulting DCRC, seeks to address these challenges.

Several diabetes complications prediction models have been developed in prior research²⁵. Most risk prediction models for cardiovascular disease had pooled C statistics ranging from 0.54 and 0.81; models for retinopathy, 0.83 to 0.89; models for nephropathy, 0.87 to 0.96; and models for neuropathy, 0.53 to 0.84²⁵. The C statistics observed during external validation of the DCRC were, cardiovascular disease, 0.84; neuropathy, 0.79; cerebrovascular disease, 0.84; nephropathy, 0.84; retinopathy, 0.80; peripheral vascular disease, 0.80; hypoglycemic crisis, 0.85; hyperglycemic crisis, 0.77; and composite (any incidence of the diabetes complications including hypoglycemic and hyperglycemic crises), 0.83. Few studies have simultaneously predicted multiple diabetes complications^26–30, as we have done here.

Building on this foundation, we developed dynamic ML models to predict near-term risk of acute and chronic diabetes complications in parallel, with predictions that can be updated in real-time as new clinical data become available. Unlike existing models that typically address a single complication over a fixed time horizon, the DCRC provides a unified framework for simultaneously assessing risk across nine complication endpoints at each clinical encounter, and is both trained on and validated for use in real-world data and under routine care conditions. To achieve this, we employed two ML algorithms: GLMnet and XGBoost. A robust model development framework was implemented using k-fold walk forward validation (WFV) strategy to mitigate bias and overfitting. The final model was trained on the complete development cohort using the best tuning parameters identified during WFV. We assessed model generalizability through internal validation on national claims data (which temporally succeeded the development cohort) and external validation on the EHR of a multi-state integrated healthcare delivery system. Our results demonstrated that the GLMnet model consistently outperformed XGBoost, showing superior accuracy and reliability across validation cohorts. Furthermore, our comparison with Cox proportional hazards (PH) models highlights that while traditional regression provides reasonable baseline performance, our ML approaches substantially improve predictive accuracy for diabetes complications. Nevertheless, advanced survival-deep learning models capable of handling complex time-varying covariates^31–33 represent a valuable methodological frontier, and may offer complementary avenues for future extension of the DCRC.

Leveraging SHAP values, GLMnet provided transparent and interpretable predictions at both the population and patient levels, enhancing its practical utility for clinical decision-making. We also demonstrated that our ML models provide mutable, encounter-specific risk estimates that not only track changes in patient status (as apparent in Supplemental Fig. s24, which illustrates how the DCRC captures dynamic risk that is mutable within a patient even with changes to a single parameter) but also have clear clinical interpretability, enabling clinicians to monitor evolving risk trajectories and identify windows of opportunity for timely intervention.

In the development of our ML models for predicting diabetes complications, we included patients with one or more diabetes complications in the baseline period for training. This approach was necessary to capture the complex interplay between baseline health conditions and the onset and progression of diabetes. By incorporating these patients, our models were exposed to a more comprehensive dataset, leading to improved robustness and predictive accuracy. Patients with pre-existing complications provided valuable insights into disease progression and factors influencing the development of diabetes-related conditions. This inclusion further enabled our models to account for patient heterogeneity and to identify and understand risk factors associated with diabetes complications. Moreover, this approach ensured that our models were not biased towards patients without any baseline complications, which could have led to an underestimation of the true predictive power of the model.

Strengths of the DCRC include the high accuracy of the models both in internal and external validation datasets. Indeed, the models were more accurate in the external data than in the internal data; this is likely due to the fact the models were trained on a nationally representative longitudinal data for 400,400 patients with diabetes. OLDW includes patients with various health conditions from different racial and ethnic communities, age groups, rural areas, and across health systems and geographic regions. We acknowledge that formal fairness analyses were not performed, and future work will evaluate performance across key sociodemographic subgroups to identify and address any potential sources of systematic bias. Another strength of the DCRC is that the development cohort was derived from routine care conditions and included covariates collected as part of usual clinical care, rather than a prospective research study. A model trained on such diverse and heterogeneous data is often more robust, adaptable, and better at generalizing to new datasets, including those with less variation, due to its exposure to a wide range of scenarios and its ability to learn the true underlying patterns. Supplemental Discussion 1-2 include additional benefits of training ML models on broadly representative national datasets. The DCRC also innovatively predicts diabetes complications in the short-term with the goal of supporting real-time clinical decision-making and monitoring decisions at the point-of-care; although some risk features may capture early diagnostic activity, the potential impact is expected to be small and is inherent to any real-world prediction setting that uses routinely collected clinical data. Future work will need to examine these models for prediction of complications in longer time horizons.

Another strength of the DCRC is its demonstrated generalizability, stability, and suitability for real-world deployment across heterogeneous clinical settings. Subgroup analyses in the Mayo Clinic external validation cohort showed that model performance was broadly consistent across sex, age, and race/ethnicity groups, with average ΔAUC approximately 0.00 to −0.03 across subgroup categories. Nevertheless, threshold-dependent metrics (i.e., sensitivity and specificity) varied more substantially, particularly across age strata, with higher specificity but lower sensitivity in younger adults, and higher sensitivity but lower specificity in older adults. These findings underscore the importance of age-specific threshold recalibration before clinical deployment and suggest that a single classification threshold derived from the overall development cohort may not be optimal for all patient subgroups.

In addition, laboratory ablation analyses demonstrated that replacing current laboratory values with prior measurements (carry-forward) resulted in only small reductions in performance, while permutation-based ablation produced similarly modest changes. These findings indicate that, although laboratory data contribute meaningful predictive signal, model performance is not heavily dependent on the availability of recent laboratory values. Together, these results support the robustness and generalizability of the DCRC and reinforce its potential applicability in real-world clinical settings where data completeness may vary.

However, these models also have limitations. They were developed and validated among insured patients with access to and utilization of healthcare services. Specifically, the development and internal validation cohorts were comprised of enrollees in commercial and Medicare Advantage health plans—not those with public health plans or without insurance—and relied on claims, or healthcare utilization, for both prediction and outcome ascertainment. Similarly, the external validation cohort was comprised of individuals—though with any insurance type, public or private—who accessed medical care at Mayo Clinic. Individuals without insurance, and those with barriers to accessing care, were either underrepresented or not included. How our models generalize to this broader patient population will need to be examined in future research. Moreover, while we were able to successfully implement our models in the Mayo Clinic EHR, the process of mapping the covariates between OLDW and Mayo Clinic EHR was challenging and computationally expensive, requiring several iterations between model training and feature mapping, and therefore may not be feasible to implement in all healthcare institutions. Because SHAP analyses were conducted only in the OLDW development cohort (the Mayo Clinic cohort could not be transferred into the secure SHAP computational environment), interpretability findings may partially reflect characteristics specific to that data source, such as coding patterns, laboratory ordering frequency, or measurement density, rather than universally generalizable clinical signals. Accordingly, external SHAP analyses will be an important direction for future work to strengthen interpretability across healthcare settings. Our models focused on data routinely available in structured EHR data, such that some clinically important covariates could not be captured, resulting in underestimation of certain complications like nephropathy (because the urine albumin-to-creatinine ratio was unavailable), neuropathy (without unstructured physical exam data), and peripheral vascular disease (without an ankle brachial index or lower extremity ultrasound studies). Moreover, because diagnoses, medications, and laboratory values were aggregated within fixed temporal windows to balance computational scalability, feature stability, and consistency across large cohorts drawn from heterogeneous claims and EHR sources, some short-term clinical fluctuations may have been smoothed over, potentially obscuring rapid changes in risk for certain complications.

Furthermore, our study cohorts were comprised of individuals with both type 1 and type 2 diabetes (though people with type 1 diabetes comprised ~5% of the cohort, as expected given its epidemiology) in order to best support population health management using claims and EHR data where misclassification of diabetes type is common and patients may have both diagnoses included in their records. Our analyses used an algorithmic approach to classify diabetes³⁴, but such an approach may be infeasible in routine care. Another limitation of our approach is that we excluded patients and variables with high levels of missing laboratory data and applied imputation under a Missing at Random assumption for the remaining features. While this strategy helped preserve model stability and generalizability, we acknowledge that laboratory testing in real-world practice is often Missing Not at Random (MNAR), which could introduce selection bias and attenuate performance in certain clinical contexts. It will be important to continue examining DCRC’s accuracy in other settings, including when implemented in laboratory data scarce datasets (i.e., most claims) and in non-integrated healthcare delivery systems were missingness is a greater concern.

Importantly, these prediction models should not be used to guide care delivery in isolation, particularly as the AUC’s for some of the complications fall into the 0.77 (hyperglycemic crisis) and 0.79 (neuropathy) range. Although discrimination was consistently strong across outcomes, several complications (most notably hyperglycemic crisis) demonstrated imperfect calibration, indicating that recalibration may be required prior to clinical deployment, particularly in healthcare systems with risk profiles different from those used for model development. The DCRC can therefore serve as part of the evidence available to clinicians and patients at the point-of-care to inform shared decision-making and to health systems as they engage in proactive population health management. For example, these models can act as risk stratification aids to help identify patients who may benefit from enhanced surveillance or preventive interventions. They can also support more nuanced discussions with patients when seeking to contextualize their comparative risks of different diabetes complications in the near-term, so that efforts can be focused on preventing complications for which the patient is at highest risk. Use of the DCRC in these contexts will need to be studied in prospective trials.

Early and individualized prediction of acute and chronic complications of diabetes using routinely available clinical data can help support precision evidence-based care at the point-of-care. The models comprising the DCRC have high accuracy and reliability, use covariates that are available in routine clinical practice, and were developed and validated in diverse patient cohorts, paving the way for their use in clinical care settings. It will be essential to test the implementation of these models in clinical care to examine their impact on shared decision-making, therapeutic decisions, and patient health outcomes.

Methods

Study design and setting

We used administrative claims data from the OLDW for model training and internal validation, and EHR data from Mayo Clinic for external validation. OLDW includes deidentified claims data for enrollees in commercial and Medicare Advantage plans across the U.S. representing a diverse mixture of ages, racial and ethnic groups, income levels, and geographic regions across the U.S^35,36. The data include longitudinal enrollment information (i.e., demographics, geographic location, income level), ambulatory and hospital medical claims, and ambulatory pharmacy claims. Laboratory results data were also available for a subset of individuals based on data sharing agreements with commercial laboratory companies.

Mayo Clinic is an integrated healthcare delivery system serving local, national, and international patients in four practice sites: Rochester, Minnesota; Phoenix/Scottsdale, Arizona; Jacksonville, Florida; and the Mayo Clinic Health System (MCHS), a network of 54 community clinics and hospitals in rural and small urban areas across Minnesota and Wisconsin. Laboratory results data were available for most patients in the Mayo Clinic EHR.

Ethics approval

Model development and internal validation were exempt from review by the Mayo Clinic Institutional Review Board because these were retrospective analyses of de-identified data; OLDW data are de-identified consistent with HIPAA expert de-identification determination³⁷. External model validation using Mayo Clinic EHR was approved by the Mayo Clinic Institutional Review Board (21-006326) with a granted waiver of informed consent because the study involved only retrospective analysis of existing EHR, posed no more than minimal risk to participants, and could not be practically conducted if individual consent were required. The study is reported according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) for Artificial Intelligence (AI) reporting guideline³⁸.

Study cohorts

Model development and internal validation cohorts

In OLDW, we identified adults ≥18 years who were newly diagnosed with diabetes between 01/01/2005 and 03/31/2018 (Supplemental Fig. s19). Diagnosis of diabetes was established using Healthcare Effectiveness Data and Information Set (HEDIS) claims-based criteria³⁹. We required patients to have 12 months of continuous enrollment prior to the first instance that HEDIS criteria were met (“baseline period”) to classify them as being newly diagnosed with diabetes; if this clean period was not available, patients were classified as prevalent diabetes and excluded from the cohort. We then required an additional 12 months of enrollment after the HEDIS date, during which diagnosis codes and pharmacy fills were used to classify patients as having type 1 or type 2 diabetes as previously described³⁴. Patients were followed-up until disenrollment or end of the study period (03/31/2019), whichever came first.

We divided the OLDW study sample into a model development cohort and an internal validation cohort, according to whether the follow-up time was before (development) or after (internal validation) 03/31/2015. As some patients may have had follow-up in both the development and internal validation cohorts, we further restricted the internal validation cohort to patients with no follow-up in the development cohort.

Model external validation cohort

Mayo Clinic EHR was used to identify adults ≥18 years who were newly diagnosed with diabetes and paneled to a Mayo Clinic primary care clinician (to exclude patients receiving only specialty care by referral, as medical records for such patients are incomplete in the Mayo Clinic EHR) between 01/01/2012 and 06/01/2021 (Supplemental Fig. s20). The diagnosis of diabetes, its incident status, and diabetes type were established using the same procedures as for the OLDW cohort.

Outcomes

Primary outcomes were the diagnoses of acute and chronic diabetes complications during the follow-up period. Chronic complications were defined using diagnosis codes from the Diabetes Complications Severity Index (DCSI)^40,41: CVD, CeVD, PVD, retinopathy, neuropathy, and nephropathy. Acute complications included emergency department (ED) visits and hospitalizations for hypoglycemia and hyperglycemia (i.e., diabetic ketoacidosis and hyperglycemic hyperosmolar state)^34,42. We secondarily examined a composite measure, i.e., diagnosis of any acute or chronic diabetes complication(s).

Because some patients may have one or more diabetes-related complications during the baseline period (i.e., prior to or concurrent with their diabetes diagnosis), these patients were intentionally included in the development cohort for training the ML models. This approach was adopted because leveraging historical clinical data, including baseline complications, provides critical contextual information about patients’ underlying health status and disease trajectory, enabling the models to learn complex clinical patterns associated with subsequent complication risks. By incorporating a comprehensive spectrum of patient profiles, from uncomplicated diabetes diagnoses to cases already accompanied by baseline complications, we can improve the robustness and generalizability of our predictive models. Nevertheless, to rigorously evaluate predictive accuracy in a realistic clinical scenario, we restricted our primary validation analyses exclusively to new incidences of acute and chronic diabetes complications occurring during the follow-up period. This design ensures that performance metrics reflect true predictive capabilities and avoid potential inflation by excluding baseline-prevalent conditions from validation outcomes. However, in a sensitivity analysis, we included both incident and prevalent cases of diabetes complications.

Covariates

For model development and internal validation, patient demographic information was ascertained on the HEDIS date from enrollment files and included age, sex, race/ethnicity, annual household income, U.S. Census region of residence, and health plan type (commercial or Medicare Advantage). We then captured all of the patients’ diagnoses, medications, and sites of care (ambulatory, ED, or hospital) during both the 12-months baseline and the follow-up periods as previously described⁴³. Specifically, we captured all ambulatory, ED, and hospital (inpatient and observation stay) Evaluation & Management (E&M) encounters using Current Procedural Terminology (CPT) and revenue codes in the medical claims. International Classification of Diseases, Ninth/Tenth Revision (ICD-9/10-CM) diagnosis codes from these encounters were categorized according to the Healthcare Cost and Utilization Project (HCUP) Clinical Classification Software (CCS)^44,45 to represent actively managed health conditions affecting the patient during the baseline and follow-up periods. Health conditions were further labeled as “acute” or “chronic” using the HCUP Chronic Condition Indicator^46,47. Diabetes complications were further subcategorized using the DCSI^40,41.

Medications were identified from pharmacy claims and were hierarchically categorized as belonging to a specific class of glucose-lowering, lipid-lowering, cardiovascular, glucocorticoid, or hormonal medications (Supplemental Table s7). All remaining medications were attributed to a broader generic therapeutic class as classified in OLDW.

Acute diagnoses were labeled as present only for the month in which they were documented, while chronic diagnoses were labeled as present in the month they were diagnosed and remained with the patient until the end of follow-up. Medications were counted as present for a given month if any medication from that class was taken within that month.

Ambulatory laboratory test results were identified using Logical Observation Identifiers Names and Codes (LOINC) and included (as available) hemoglobin, hemoglobin A1C, fructosamine, creatinine, fasting glucose, low-density lipoprotein (LDL) cholesterol, high-density lipoprotein (HDL) cholesterol, total cholesterol, triglycerides, uric acid, amylase, lipase, thyroid-stimulating hormone (TSH), free thyroxine, total thyroxine, total triiodothyronine, free triiodothyronine, c-peptide, glutamic acid decarboxylase-65 (GAD-65) antibody, insulin antibody, islet antigen-2 (IA-2) antibody, 21-hydroxylase antibody, zinc transporter 8 antibody, 25-hydroxyvitamin D2 or D3, aspartate aminotransferase (AST), alanine aminotransferase (ALT), alkaline phosphatase, and urine microalbumin.

Missing values

There were no missing diagnosis or medications data per se because variables were defined by the presence of a claim (in OLDW) or encounter (in EHR) with corresponding diagnosis codes, procedure codes, or medications. In contrast, laboratory data had high rates of missingness in OLDW, as availability of ambulatory laboratory test results is conditional on contractual agreements between Optum Labs and commercial laboratory companies. Approximately 40% of the cohort had no laboratory data and were excluded. For the remaining cohort, we excluded laboratory variables with more than 40% missing observations and applied the Random Forest imputation method, “missForest”⁴⁸, to impute any missing values in the remaining variables.

Feature engineering

The development, internal validation, and external validation cohorts were structured longitudinally where each month represented one time step and corresponded to an aggregated clinical “encounter”. The ML task was then to predict the next months’ risk of diabetes complications. The baseline or starting point (time = 0) represented aggregated diagnosis data from 1-year prior to and including the HEDIS (index) date (corresponding to date of first diagnosis of diabetes), aggregated medications from the past 120 days, and aggregated laboratory results data from the past 90 days. Additional detail is available in Supplemental Methods 1.

Continuous variables were normalized using the Min-Max technique, which scaled each value to a range between 0 and 1 based on the minimum and maximum values in the dataset, while categorical variables were converted into binary format using the one-hot encoding. In one-hot encoding, a categorical predictor with k possible values is transformed into k binary predictors, with only one active predictor. Rare binary/categorical variables ( <0.01%) were dropped, and rare levels of categorical variables were combined into a single category.

Machine leaning risk stratification algorithms

We trained two ML algorithms, which have been previously applied for clinical risk prediction: elastic-net logistic regression (GLMnet) and extreme gradient boosting (XGBoost)^49,50. GLMnet is an extension of the least absolute shrinkage and selection operator (LASSO) and the ridge (Ridge) regularization methods for the classical logistic regression algorithm (Supplemental Methods 1). This regularization strategy penalizes models with many non-zero coefficients, thereby reducing the likelihood of overfitting, especially for datasets where the number of covariates may exceed the number of observations. The penalty term in the algorithm is controlled by a hyperparameter called the mixing parameter (alpha), with a value of 1 making GLMnet equivalent to Lasso and a value of 0 making it equivalent to Ridge. Values between 0 and 1 allow for a blend of both penalties. GLMnet is particularly useful when dealing with datasets containing many correlated features, providing a balance between feature selection and handling multicollinearity.

XGBoost is a powerful machine learning algorithm that belongs to the ensemble learning category, specifically the gradient boosting machine (GBM) frameworks (Supplemental Methods 1)^51,52. GBM builds strong predictive models by combining the predictions of multiple weak learners, such as decision stumps (trees with only one split), sequentially, with each new tree attempting to correct the errors of the combined model from the previous iterations. XGBoost is a generalized implementation of the GBM technique with several algorithmic enhancements to improve scalability, speed, and prediction accuracy. Importantly, XGBoost enhances the GBM by implementing a LASSO and Ridge regularization strategy, which penalizes more complex models to prevent overfitting. XGBoost is widely used for both regression and classification tasks.

In addition to the two ML models, we also developed Cox proportional hazards (Cox PH) regression models for baseline comparison with the ML methods. Because full Cox model objects could not be exported from Optum Labs, we implemented a horizon-based approximation strategy in which baseline survival probabilities at prespecified horizons were combined with Cox regression coefficients to generate absolute risk predictions for external validation (see Supplemental Methods 2).

Training and validation

To ensure patients’ current and historical characteristics were adequately represented, we required each patient in the development cohort to have data in at least two months. To accommodate the longitudinal nature of the data, we implemented a k-fold walk-forward validation (WFV) strategy, which iteratively splits the development cohort into training and validation data, guaranteeing that all the samples in the training dataset temporally precede the ones in the validation dataset (Supplemental Methods 3)⁵³. This contrasts with conventional cross-validation, which randomly splits data into training and testing sets. Instead, WFV preserves the temporal order of the data, allowing for temporal trends to be captured and better reflecting real-world scenarios where a model is trained on past data and tested on (i.e., implemented in) future data. The training data was used to develop the GLMnet, XGBoost, and Cox PH models, while the validation data was used to optimize the parameters. Because the GLMnet and XGBoost algorithms require choosing one or more tuning parameters for optimal performance (e.g., the LASSO and Ridge in GLMnet and XGBoost), we set up a grid search to select the best (based on the area under the ROC curve [AUC]) combination of hyper-parameters within the k-fold WFV procedure. Specifically, with each portion of the training data starting from 01/01/2005, we set up a grid for each combination of parameters, applied the algorithm with those parameters to the training fold, and assessed its performance on the validation fold. This procedure was repeated for all k-folds, which consist of shifting the training and validation folds a fixed number of days forward k times. In our experiments, we aimed at predicting diabetes complications in the next month, so we set the validation folds to be the next 30 days. Starting with an initial training fold period of 306 days (approximately 10 months) anchored at 01/01/2005, the WFV process shifts the training window forward by 306 days with each iteration. Consequently, both the training and validation folds extend by approximately 336 days in each step, spanning from 01/01/2005 to 03/31/2015. This process generates a total of 13 ( = k) folds (Supplemental Fig. s21). Finally, for each ML algorithm, a final model was obtained by retraining the algorithm on the entire development cohort using the optimal parameters selected. The performance of this final model was then assessed on the internal validation cohort, comprised of the 01/01/2015-03/31/2019 subset of the study period.

We then conducted an external validation of the final models on adult patients newly diagnosed with diabetes at Mayo Clinic. The workflow of the WFV procedure, internal validation, and external validation of the final model is illustrated in Fig. 4. See Supplemental Methods 3 for a detailed description of the WFV procedure. This workflow was also implemented for the sensitivity analysis that examined both incident and prevalent diabetes complications.

Fig. 4. Workflow of the machine learning training and validation procedure and selection of final model.

We applied a k-fold walk forward cross-validation procedure to split the development cohort (follow-up ≤3/31/2015) into 13 folds. For each iteration, we train the ML algorithms on the training folds, which consist of shifting the previous training folds by approximately 306 days. The best tuning hyperparameters of the models are then selected through grid search by evaluating the performance of the model on the validation folds, which consist of the immediate next 30 days. The final model is obtained by re-training the model using the optimal tuning parameters on the complete development cohort and evaluated on the internal validation cohort (follow-up >3/31/2015) and external validation on Mayo Clinic data.

Performance measures

Based on the predicted probabilities for each diabetes complication derived by each ML model, we computed the sensitivities and specificities across all possible classification thresholds and constructed ROC curves. The classification threshold is a boundary or cutoff point based on the predicted probability, above which the model assigns a patient to a given diabetes complication status. We selected the optimal classification threshold based on the development cohort, defined as the value at which the distance between the ROC curve and the upper left corner point (0,1) of the unit square is minimum (Supplemental Methods 4, Fig. s22). Using the optimal classification threshold, we computed several scalar binary classification performance measures including accuracy, AUC, sensitivity, specificity, F1-score, PPV, and NPV. We also computed graphical performance measures including the ROC, cumulative gain, sensitivity and PPV curves.

Subgroup and laboratory ablation analyses

To further evaluate model generalizability and robustness, we conducted post-hoc subgroup and laboratory ablation analyses in the Mayo Clinic external validation cohort (Supplemental Methods 5). Subgroup analyses were performed across predefined categories of age, sex, and race/ethnicity. Model performance within each subgroup was assessed using the same trained (frozen) models and thresholding framework as in the primary external validation. To examine robustness to reduced laboratory availability, we performed laboratory ablation without re-training the models by replacing current-month laboratory values with the most recent prior values (carry-forward ablation). As a complementary robustness analysis, we also evaluated model performance after randomly permuting laboratory values to disrupt their predictive signal.

Changes in discrimination were summarized using ΔAUC, defined as the difference between the AUC under each subgroup or ablation condition and the original AUC from the primary external validation, that is ΔAUC = AUC (subgroup/ablation)—AUC (primary). Negative values indicate reduced performance relative to the original model.

Interpretability

A post-hoc analysis of Shapley Additive Explanations (SHAP) values was conducted to derive feature importance and determine the positive or negative influence of each feature on model predictions, providing transparency to each model (Supplemental Methods 6). We report the average SHAP values for the top 20 variables for the final ML models.

To illustrate temporal changes in predicted risk, patient-specific probabilities were recalculated at each monthly interval and plotted longitudinally for a random subset of representative patients, showing how predicted risk of the composite diabetes-complication endpoint evolved over time.

Statistical analysis

Cohort characteristics were computed using mean and standard deviation (SD) for continuous variables and counts and percentages for categorical variables. For each ML model, we report the mean and 95% confidence intervals (CI) of the binary performance measures across the 13-fold WFV estimates and the point estimates on the internal and external validation cohorts. We also report the graphical performance metrics (ROC, cumulative gain, sensitivity, and PPV plots) on the development cohort. Models’ discriminatory ability was categorized based on AUC as outstanding (0.9–1.0), excellent (0.80–0.89), good (0.70–0.79), fair (0.60–0.69), poor (0.56–0.59), or fail/no discriminatory capacity (0.50–0.55).

For model interpretation, we reported the average SHAP values of the top 20 variables from the best performing model based on a random sample (10%) of patients in the internal validation cohort. To illustrate temporal dynamics in predicted risk, six representative patients were randomly selected from the internal validation cohort using a fixed random seed to ensure reproducibility. The small sample size was chosen to maintain visual clarity while capturing diverse longitudinal risk trajectories observed across the broader population.

Data management was performed using SAS 9.4 (SAS Institute, Inc, Cary, NC), and all statistical analyses were performed using R 4.1.0. R packages used for the analysis include glmnet (v4.1-8), xgboost (v0.81.0.1), missForest (v1.5), and survival (v3.8-3)^48,54–56. No sex-based analyses were performed.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Author contributions

Study conception (R.G.M.) and design (R.G.M., C.N.). Data acquisition and management (H.C.H., Zhang). Data analysis (S.P., L.F., N.Z., C.N.). Drafting of the manuscript (R.G.M., C.N.). Critical review and revision of the manuscript (S.P., L.F., H.C.H., N.Z., B.C.). Supervision (R.G.M., C.N.). Funding (R.G.M., C.N.).

Peer review

Peer review information

Nature Communications thanks Nouman Ahmed and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. [A peer review file is available.]

Data availability

Optum Labs Data Warehouse (OLDW) data are third party data owned by Optum Labs and contain sensitive patient information; these data cannot be shared as a condition for data access. Interested researchers engaged in HIPAA compliant research may contact connected@optum.com for data access requests. The OLDW data use agreement requires researchers to pay for rights to use and access the data. De-identified Mayo Clinic electronic health record data used for external validation may be made available to qualified researchers for purposes of replication and verification of the published findings, subject to Mayo Clinic institutional review, data use agreements, and applicable ethics approvals. Requests should be directed to the corresponding author. An initial determination regarding data access requests will typically be provided within 4-6 weeks of receipt of a complete request. Access may require funding to cover the costs associated with data preparation, de-identification, and secure data transfer. Source data are provided with this paper.

Code availability

Code is detailed in Supplemental Methods 1–2. The glmnet (v4.1-8) R package was used to train the GLMnet model. The xgboost (v0.81.0.1) R package was used to train the XGBoost model. The coxphHorizon R package implementing the horizon-based approximation of the Cox PH models for external validation is publicly available at https://github.com/nguforche/coxphHorizon.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41467-026-76673-3.