Abstract
Context
Hemoglobin A1c (A1c) and fasting plasma glucose (FPG) miss diabetes diagnosis in many individuals, but the alternative, oral glucose tolerance testing (OGTT), is difficult to obtain.
Objective
This work aimed to develop and validate a model to identify diabetes missed by A1c and FPG but detected by OGTT and associate its outcomes with mortality.
Methods
We trained machine-learning models using National Health and Nutrition Examination Survey (NHANES) data, applied them to corresponding National Center for Health Statistics mortality data, and externally validated the models with Diabetes Prevention Program Outcomes Study (DPPOS) data. We tested models including the A1c, vital signs, complete blood count, and comprehensive metabolic panel with and without the FPG. This retrospective analysis included NHANES study participants from 1999 to 2016 and DPPOS study participants from 1996 to 2020. Main outcome measures included the area under the receiver operating curve, sensitivity, and specificity of models to detect diabetes.
Results
The A1c/FPG+ model had better sensitivity for DPPOS individuals with A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL compared to a logistic regression of A1c and FPG and recalibrated FPG for individuals both on (0.69 vs 0.53 vs 0.57, respectively) and off (0.75 vs 0.65 vs 0.68, respectively) medication. NHANES participants without diabetes but with A1c/FPG+-predicted diabetes had elevated risk of mortality compared to those without A1c/FPG+-predicted diabetes (HR = 1.4; P < 2 × 10−16).
Conclusion
Machine-learning models can improve the detection of individuals who would need OGTT assessment to diagnose diabetes using standard nonfasting laboratory values and vital signs with and without FPG.
Keywords: machine learning, diagnosis, oral glucose tolerance test, diabetes
The type 2 diabetes epidemic continues to grow in the United States, affecting an estimated 38.4 million people (8.7 million likely undiagnosed) and generating health care costs of $412.9 billion per year (1, 2). The American Diabetes Association recommends the glycated hemoglobin A1c (A1c), fasting plasma glucose (FPG), or oral glucose tolerance test (OGTT) for the diagnosis of diabetes (3). The A1c has become the predominant method clinically due to its ease of use (4-6). However, the A1c is inaccurate in the setting of chronic diseases such as kidney disease, heart failure, cirrhosis, and after solid-organ transplant (7, 8), which are all increasing in prevalence (1, 9-11). In the presence of these chronic diseases, the FPG is more sensitive but requires fasting (3). OGTTs are recommended in patient populations with these chronic diseases as well (12), but application at the population level is hindered by high rates of nonadherence (13, 14). Though A1c-FPG-OGTT discordance has been established (15-17), this has not been examined recently at the nationally representative level. We use the US National Health and Nutrition Examination Survey (NHANES) data to examine the discordance across the A1c, FPG, and OGTT diabetes testing modalities using nationally representative general population data and determine an updated proportion of individuals in whom type 2 diabetes is diagnosed only with the OGTT.
Given the burden of OGTT-only diabetes, identifying patients with A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL who need OGTT would be challenging and would need to be performed by already-taxed general practitioners (18). Clinical decision support tools could aid in this detection (19, 20). This used to be a laborious process, but with the widespread adoption of the electronic medical record (EMR) significant amounts of patient data are readily accessible within these systems, allowing for machine-learning solutions to this diagnostic dilemma. A powerful machine-learning approach, gradient-boosted decision tree analysis (XGBoost (21)), overcomes the problems of collinearity and missing data that affect large dataset analysis and has been used to predict outcomes such as sepsis, heart failure mortality, and decompensated cirrhosis (22-24). Prior machine-learning approaches have improved the detection of diabetes, but these have focused on patient-reported history (25, 26) as their predictive label, as opposed to the clinical difficult-to-obtain 2-hour glucose from an OGTT. Liu et al and Sun et al use the 2-hour glucose in a Chinese population, but focus on patient history questions more than routine laboratory values, and do not focus on comorbid chronic diseases (27, 28). Here, we develop NHANES-trained XGBoost-based machine-learning models to predict diabetes using the A1c and nonfasting features normally obtained during routine care, with and without stepwise inclusion of the FPG. We externally validated these models using data from the Diabetes Prevention Program Outcomes Study (DPPOS), which also has A1c, FPG, and OGTT data.
The diagnostic thresholds for A1c, FPG, and OGTT were chosen to be 6.5%, 126 mg/dL, and 200 mg/dL, respectively, due to their association with the development of diabetic retinopathy (6). However, diabetes is additionally a main driver of cardiovascular disease and mortality. In addition to establishing our primary objective of developing and validating an accurate model to predict diabetes, we established a secondary objective to understand the association of diagnosis subtype (A1c vs FPG vs OGTT) and our model predictions with overall mortality.
In this work, we develop and validate a machine-learning model to identify individuals who would benefit from OGTT assessment for diabetes and establish the relevance of these models in predicting mortality.
Materials and methods
This study used data from the NHANES (29), a national survey of US residents. Many of these individuals have corresponding mortality data available from the National Center for Health Statistics (NCHS). A subset of individuals in NHANES from 1999 to 2016 received A1c and FPG measurements and a subset from 2005 to 2016 received A1c, FPG, and OGTT measurements. Data were divided into training (2005-2014 cohorts, 11 708 individuals) and evaluation (2015-2016 cohort, 2082 individuals) datasets, in accordance with the TRIPOD-AI guidelines (30). We split the data in this way to ensure that our model was trained on earlier data (2005-2014) and tested to predict future data (2015-2016). Our models were validated using data from the DPPOS (31), a longitudinal study of diabetes prevalence, prevention, and progression. We included all DPPOS individuals for whom an A1c, FPG, and OGTT were present (2379 participants). We further divided the cohort into those receiving nonmedication treatment (lifestyle, placebo) vs medication treatment (metformin, troglitazone). When duplicate measurements were present, we selected the measurement with the fewest missing features, and then selected the earliest measurement if duplicate measurements were still present. This resulted in 1375 individuals in the nonmedication cohort and 1022 medication cohort.
Primary outcome: diagnosis of diabetes
The primary outcome was the diagnosis of diabetes based on A1c greater than or equal to 6.5% (48 mmol/mol), FPG greater than or equal to 126 mg/dL, and/or OGTT 2-hour glucose greater than or equal to 200 mg/dL. For national estimates we resampled the NHANES data using fasting or OGTT-specific weights.
Secondary outcome: survival
The secondary outcome was survival based on linked mortality data from the NCHS.
Predictor variable extraction
We extracted a set of standard clinical features, including vital signs, anthropometrics, and baseline additional labs (supplementary methods (32)) from NHANES. We focused on features that would be already collected during routine standard of care for US adults. We included age, sex, race/ethnicity, vital signs (systolic blood pressure [BP], diastolic BP, height, weight, waist circumference, pulse), complete blood count (hemoglobin, white blood cell count, platelets, mean cellular volume, red blood cell count, red blood cell distribution width), comprehensive metabolic panel (sodium, potassium, chloride, bicarbonate, blood urea nitrogen, creatinine, calcium, total protein, total bilirubin, albumin, aspartate transaminase, alanine transaminase, alkaline phosphatase), A1c, and FPG. A complete blood count and comprehensive metabolic panel are required for screening for metabolic dysfunction–associated steatotic liver disease (MASLD, previously NAFLD), which should be conducted for patients at risk of prediabetes or diabetes (33, 34). We used OGTT-specific NHANES sample weights in our model training.
Model development
We used gradient-boosted decision tree analysis (XGBoost) to develop 2 versions of our model. The A1c+ model included vital signs, routine nonfasting laboratory values described earlier, and the A1c. The A1c/FPG+ model additionally included the FPG. XGBoost has the desirable characteristics of being able to self-determine the importance of relationships between features (eg, assessing height and weight instead of body mass index [BMI]), being able to handle missing data without imputation, and not being negatively affected by correlated variables (eg, weight and waist circumference). See the supplemental methods for further information (32).
Model evaluation
For comparisons, the model test characteristic diagnostic thresholds were determined using the Youden index. In the evaluation dataset, we compared the A1c+ model to the standard approach of A1c greater than or equal to 6.5% (48 mmol/mol) and the A1c/FPG+ model to the standard approach of A1c greater than or equal to 6.5% (48 mmol/mol) or FPG greater than or equal to 126 mg/dL. We compared models based on input characteristics, area under the receiver operating curve (AUROC), sensitivity, specificity, and DeLong's test to compare AUROC values across models.
To establish the utility of our models with a complementary method, we employed decision curve analysis. Decision curve analysis measures the net benefit difference between approaches based on what risk of diabetes would be needed to convince the patient or provider to order additional testing (in this case, an OGTT).
Survival analysis
We generated Kaplan-Meier curves to estimate and visualize survival probabilities according to mode of diagnosis. We tested differences in survival based on diagnosis and prediction of diabetes using survey-weighted Cox proportional hazard regression models to estimate hazard ratios (HRs) and 95% CIs for all-cause mortality. Due to the strong role these play in diabetes outcomes (35), we adjusted our mortality analysis for age, sex, race/ethnicity, and BMI when comparing the association of the mode of diabetes diagnosis with mortality and when comparing the association of the model-based predictions of diabetes with mortality. To account for the weighted sample, we used masked variance pseudo-strata (PSU) and masked variable pseudo-PSU in addition to mobile examination center sample weights, fasting sample weights, and OGTT sample rates where appropriate. See the supplemental methods for further information (32).
Sensitivity analyses
One potential limitation of the A1c and FPG thresholds is that they may not be calibrated to a modern 2005 to 2016 population. To account for this and to further validate the strength of our models, we performed sensitivity analyses where we allowed the A1c and FPG to vary, thereby generating an AUROC for the A1c and FPG. To account for the clinical scenario in which both the A1c and FPG are available, we generated a logistic model using the A1c and FPG (A1c + FPG) alone for comparison. To understand the ability of our models to identify diabetes in those with an A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL, we performed a subanalysis solely with those individuals.
Method and data availability
R Markdown files are available at the GitHub repository alanlhutchison/NHANES-DMDx (36). Data for this study were obtained from the 1999 to 2016 NHANES, conducted by the NCHS, Centers for Disease Control and Prevention (CDC) (29). Data from the DPPOS reported here are available for request at the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) Central Repository (NIDDK-CR) website, Resources for Research (R4R) (31).
Results
Patient characteristics
There were 92 062 participants from 1999 to 2016 in NHANES, but only 13 800 individuals received A1c, FPG, and OGTT assessment and had appropriate statistical weights assigned (Supplementary Fig. S1 (32), Table 1). Of those 13 800 individuals, the mean age was 41.2 years (±20.5) and 49.8% were female. A total of 42% of the individuals were non-Hispanic White, 20% were non-Hispanic Black, 28.4% were Hispanic, and 9.9% were other/multiracial. The average height was 167.2 cm (±10.1 cm), the average weight was 77.9 kg (±21.5 kg), the average waist circumference was 94.6 cm (±16.9 cm), the average BMI was 27.7 kg/m2 (±6.8 kg/m2), and the average systolic and diastolic BPs were 120 mm Hg (±17) and 67 mm Hg (±13), respectively. The average FPG was 101.9 mg/dL (±18.2 mg/dL), the average A1c was 5.4% (±0.6%) (36 ± 4 mmol/mol), and the average 2-hour glucose was 119 mg/dL (±50 mg/dL). Table 1 shows the features for the 2005 to 2014 training cohort participants, 2015 to 2016 evaluation cohort participants, the DPPOS nonmedication and medication cohorts, the 1999 to 2016 NHANES nontraining nonevaluation cohort individuals without OGTT assessment with mortality results, and the 2005 to 2016 NHANES individuals used in training and evaluation who had mortality results.
Table 1.
Features included in the A1c/FPG+ model to predict diabetes for the NHANES OGTT 2005 to 2014 training cohort, the NHANES OGTT 2015 to 2016 validation cohort, the DPPOS nonmedication cohort, the DPPOS medication cohort, the NHANES non-OGTT 1999 to 2016 cohort that had mortality data available from the NCHS, and the 2005 to 2016 NHANES cohort that had mortality data available from the NCHS
| Variable | 2005-2014 Training cohort (n = 11 718) |
2015-2016 Testing cohort (n = 2082) |
DPPOS Nonmedication cohort (n = 1375) |
DPPOS Medication cohort (n = 1022) |
1999-2016 Mortality cohort (n = 41 670) |
2005-2016 Mortality cohort (n = 11 550) |
|---|---|---|---|---|---|---|
| Age, y | 41.1 (±20.6) | 41.8 (±20.3) | 56.4 (±9.2) | 56.2 (±9.0) | 47.8 (±19.9) | 46.4 (±18.4) |
| Sex (% female) | 5823 (49.7%) | 1046 (50.2%) | 924 (67.2%) | 667 (65.3%) | 21 782 (52.3%) | 5786 (50.1%) |
| Race/Ethnicity | ||||||
| Hispanic | 3276 (28.0%) | 643 (30.9%) | 228 (16.6%) | 165 (16.1%) | 11 151 (26.8%) | 3095 (26.8%) |
| Non-Hispanic Black | 2350 (20.1%) | 411 (19.7%) | 276 (20.1%) | 204 (20.0%) | 9118 (21.9%) | 2171 (18.8%) |
| Non-Hispanic White | 5039 (43.0%) | 720 (34.6%) | 789 (57.4%) | 615 (60.2%) | 18 205 (43.7%) | 5141 (44.5%) |
| Other | 1053 (9.0%) | 308 (14.8%) | 82 (6.0%) | 38 (3.7%) | 3196 (7.7%) | 1143 (9.9%) |
| Waist circumference, cm | ||||||
| Mean (SD) | 94.4 (±16.7) | 95.9 (±17.5) | 101.9 (±15.0) | 102.3 (±13.7) | 97.7 (±16.3) | 97.2 (±15.9) |
| Missing | 189 (1.6%) | 41 (2.0%) | 26 (1.9%) | 58 (5.7%) | 5128 (12.3%) | 211 (1.8%) |
| Systolic BP, mm Hg | ||||||
| Mean (SD) | 119.2 (±16.8) | 121.7 (±17.5) | 122.2 (±14.3) | 123.6 (±14.5) | 124.6 (±20.0) | 121.6 (±17.3) |
| Missing | 258 (2.2%) | 26 (1.2%) | 22 (1.6%) | 38 (3.7%) | 4672 (11.2%) | 241 (2.1%) |
| Diastolic BP, mm Hg | ||||||
| Mean (SD) | 66.6 (±13.3) | 66.8 (±12.6) | 74.7 (±9.0) | 75.6 (±8.8) | 69.7 (±13.7) | 68.3 (±12.7) |
| Missing | 258 (2.2%) | 26 (1.2%) | 22 (1.6%) | 38 (3.7%) | 4672 (11.2%) | 241 (2.1%) |
| Height (cm) | ||||||
| Mean (SD) | 167.4 (±10.1) | 166.3 (±10.1) | 165.2 (±9.0) | 165.9 (±9.4) | 167.0 (±10.2) | 167.8 (±10.0) |
| Missing | 73 (0.6%) | 7 (0.3%) | 70 (5.1%) | 53 (5.2%) | 3229 (7.7%) | 77 (0.7%) |
| Weight, kg | ||||||
| Mean (SD) | 77.8 (±21.5) | 78.5 (±22.0) | 90.8 (±20.3) | 91.1 (±18.5) | 80.1 (±21.0) | 80.4 (±21.0) |
| Missing | 67 (0.6%) | 8 (0.4%) | 22 (1.6%) | 39 (3.8%) | 3296 (7.9%) | 71 (0.6%) |
| Fasting glucose, mg/dL | ||||||
| Mean (SD) | 101.8 (±18.0) | 102.6 (±18.9) | 105.3 (±12.4) | 104.2 (±11.9) | 112.8 (±47.2) | 103.0 (±19.2) |
| Missing | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) | 29 924 (71.8%) | 0 (0%) |
| 2-h glucose, mg/dL | ||||||
| Mean (SD) | 118.6 (±50.5) | 119.5 (±49.6) | 151.3 (±41.5) | 151.8 (±40.8) | (±) | 122.1 (±53.3) |
| Missing | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) | 41 670 (100.0%) | 0 (0%) |
| HbA1c, % | ||||||
| Mean (SD) | 5.4 (±0.6) | 5.5 (±0.6) | 5.8 (±0.5) | 5.8 (±0.4) | 5.7 (±1.2) | 5.5 (±0.6) |
| Missing | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) | 5144 (12.3%) | 0 (0%) |
| Creatinine, mg/dL | ||||||
| Mean (SD) | 0.8 (±0.3) | 0.8 (±0.3) | 0.8 (±0.2) | 0.8 (±0.2) | 0.9 (±0.5) | 0.9 (±0.3) |
| Missing | 81 (0.7%) | 6 (0.3%) | 156 (11.3%) | 64 (6.3%) | 5771 (13.8%) | 68 (0.6%) |
| Year | ||||||
| Mean (SD) | 2009.0 (±2.8) | 2015.0 (±0.0) | (±) | (±) | 2006.4 (±5.2) | 2010.0 (±3.3) |
| Missing | 0 (0%) | 0 (0%) | 1375 (100.0%) | 1022 (100.0%) | 0 (0%) | 0 (0%) |
| Pulse, bpm | ||||||
| Mean (SD) | 72.1 (±11.8) | 72.3 (±11.4) | (±) | (±) | 73.2 (±12.5) | 71.5 (±11.6) |
| Missing | 220 (1.9%) | 25 (1.2%) | 1375 (100.0%) | 1022 (100.0%) | 4330 (10.4%) | 211 (1.8%) |
| White blood cells, mm−3) | ||||||
| Mean (SD) | 6.7 (±2.2) | 6.8 (±3.2) | (±) | (±) | 7.5 (±2.4) | 6.7 (±2.5) |
| Missing | 21 (0.2%) | 9 (0.4%) | 1375 (100.0%) | 1022 (100.0%) | 5115 (12.3%) | 24 (0.2%) |
| Red blood cells, mm−3 | ||||||
| Mean (SD) | 4.7 (±0.5) | 4.8 (±0.5) | (±) | (±) | 4.7 (±0.5) | 4.7 (±0.5) |
| Missing | 21 (0.2%) | 9 (0.4%) | 1375 (100.0%) | 1022 (100.0%) | 5114 (12.3%) | 24 (0.2%) |
| Hemoglobin, g/L | ||||||
| Mean (SD) | 14.3 (±1.5) | 14.1 (±1.5) | (±) | (±) | 14.1 (±1.6) | 14.3 (±1.5) |
| Missing | 21 (0.2%) | 9 (0.4%) | 1375 (100.0%) | 1022 (100.0%) | 5114 (12.3%) | 24 (0.2%) |
| MCV, fL | ||||||
| Mean (SD) | 88.9 (±5.8) | 88.3 (±6.1) | (±) | (±) | 89.4 (±5.8) | 89.3 (±5.8) |
| Missing | 21 (0.2%) | 9 (0.4%) | 1375 (100.0%) | 1022 (100.0%) | 5114 (12.3%) | 24 (0.2%) |
| Platelets, K/mm−3 | ||||||
| Mean (SD) | 251.4 (±66.4) | 237.5 (±58.8) | (±) | (±) | 256.3 (±68.2) | 246.6 (±66.0) |
| Missing | 22 (0.2%) | 9 (0.4%) | 1375 (100.0%) | 1022 (100.0%) | 5114 (12.3%) | 25 (0.2%) |
| Sodium, mEq/L | ||||||
| Mean (SD) | 139.3 (±2.1) | 138.8 (±1.9) | (±) | (±) | 139.1 (±2.4) | 139.3 (±2.1) |
| Missing | 81 (0.7%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5773 (13.9%) | 68 (0.6%) |
| Potassium, mEq/L | ||||||
| Mean (SD) | 4.0 (±0.3) | 4.0 (±0.3) | (±) | (±) | 4.0 (±0.4) | 4.0 (±0.3) |
| Missing | 83 (0.7%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5774 (13.9%) | 70 (0.6%) |
| Chloride, mEq/L | ||||||
| Mean (SD) | 104.2 (±2.6) | 103.6 (±2.7) | (±) | (±) | 103.4 (±3.0) | 104.1 (±2.7) |
| Missing | 82 (0.7%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5776 (13.9%) | 68 (0.6%) |
| Bicarbonate, mEq/L | ||||||
| Mean (SD) | 25.0 (±2.1) | 24.4 (±2.1) | (±) | (±) | 24.6 (±2.4) | 25.0 (±2.2) |
| Missing | 96 (0.8%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5834 (14.0%) | 83 (0.7%) |
| BUN, mg/dL | ||||||
| Mean (SD) | 12.1 (±4.8) | 13.9 (±5.1) | (±) | (±) | 13.5 (±6.3) | 12.8 (±5.1) |
| Missing | 81 (0.7%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5773 (13.9%) | 68 (0.6%) |
| Calcium, mg/dL | ||||||
| Mean (SD) | 9.4 (±0.4) | 9.3 (±0.3) | (±) | (±) | 9.5 (±0.4) | 9.4 (±0.3) |
| Missing | 99 (0.8%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5789 (13.9%) | 81 (0.7%) |
| Total protein, g/dL | ||||||
| Mean (SD) | 7.2 (±0.5) | 7.2 (±0.4) | (±) | (±) | 7.2 (±0.5) | 7.2 (±0.5) |
| Missing | 89 (0.8%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5818 (14.0%) | 76 (0.7%) |
| Albumin, g/dL | ||||||
| Mean (SD) | 4.3 (±0.3) | 4.4 (±0.3) | (±) | (±) | 4.3 (±0.4) | 4.3 (±0.3) |
| Missing | 81 (0.7%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5769 (13.8%) | 68 (0.6%) |
| Total bilirubin, mg/dL | ||||||
| Mean (SD) | 0.8 (±0.3) | 0.6 (±0.3) | (±) | (±) | 0.7 (±0.3) | 0.7 (±0.3) |
| Missing | 91 (0.8%) | 7 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5786 (13.9%) | 79 (0.7%) |
| AST, IU/L | ||||||
| Mean (SD) | 25.7 (±20.0) | 24.7 (±10.8) | (±) | (±) | 25.5 (±18.2) | 25.8 (±20.3) |
| Missing | 102 (0.9%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5844 (14.0%) | 89 (0.8%) |
| ALT, IU/L | ||||||
| Mean (SD) | 24.5 (±19.8) | 23.6 (±14.0) | (±) | (±) | 25.2 (±25.9) | 25.4 (±19.8) |
| Missing | 99 (0.8%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 5842 (14.0%) | 86 (0.7%) |
| ALP, IU/L | ||||||
| Mean (SD) | 83.2 (±56.5) | 81.5 (±52.8) | (±) | (±) | 72.8 (±28.8) | 68.5 (±22.7) |
| Missing | 82 (0.7%) | 6 (0.3%) | 1375 (100.0%) | 1022 (100.0%) | 10 481 (25.2%) | 69 (0.6%) |
| Antidiabetes medications | ||||||
| Yes | 1 (0.009%) | 1 (0.05%) | 0 (0%) | 1022 (100%) | 3296 (7.9%) | 2 (0.02%) |
| Missing | 10 977 (93.7%) | 1832 (88.0%) | 0 (0%) | 0 (0%) | 35 982 (86.3%) | 10 595 (91.7%) |
Features as (mean) (±SD).
Abbreviations: ALP, alkaline phosphatase; ALT, alanine transaminase; AST, aspartate transaminase; BP, blood pressure; BUN, blood urea nitrogen; DPPOS, Diabetes Prevention Program Outcomes Study; FPG, fasting plasma glucose; HbA1c, glycated hemoglobin A1c; MCV, mean corpuscular volume; NHANES, National Health and Nutrition Examination Survey; OCTT, oral glucose tolerance test.
National prevalence of diabetes by diagnosis type
Resampling the 13 800 participants based on NHANES-assigned sampling weights, the prevalence of diabetes diagnosed by any method was 5.46%, while the prevalence of diabetes diagnosed by OGTT alone was 1.34%. Supplementary Table S1 shows the confusion matrix for the diagnosis of diabetes and prediabetes (32). To better understand this discordance, we compared the A1c to the 2-hour glucose in individuals with an A1c less than 6.5% (48 mmol/mol) in the setting of different comorbidities: chronic kidney disease, anemia, obesity, liver fibrosis, age, and hypertension. The relationship between the A1c and the 2-hour glucose varied widely across these (Supplementary Fig. S2 (32)).
Comparison of the glycated hemoglobin A1c (A1c) ± model and A1c/fasting plasma glucose (FPG) ± model with A1c and FPG for diagnosing diabetes
We compared our nonfasting routine clinical feature model (A1c+) and that model including the FPG (A1c/FPG+) as described in “Materials and methods” to the current diagnostic thresholds of the A1c of 6.5% (48 mmol/mol) and FPG of 126 mg/dL, as well as sensitivity analyses against models in which the A1c and FPG were allowed to fluctuate to different thresholds (Fig. 1).
Figure 1.
(A and C) Receiver operating characteristics and (B and D) decision curve analysis net benefit plots for the A1c+ and A1c/FPG+ models on NHANES internal validation for both the full internal validation cohort (A and B) as well as individuals with A1c less than 6.5% (48 mmol/mol) and fasting plasma glucose (FPG) less than 126 mg/dL (C and D). A, On internal validation, among nonfasting approaches, the nonfasting A1c+ model has higher area under the receiver operating curve (AUROC) and sensitivity than the A1c with and without fixing the diagnostic threshold at 6.5% (48 mmol/mol), while among the fasting models the A1c/FPG+ has higher AUROC and equivalent sensitivity than the FPG and a logistic regression of the A1c and the FPG (A1c + FPG), but higher sensitivity than the FPG at a diagnostic threshold of 126 mg/dL. Points indicate sensitivity and specificity for A1c greater than or equal to 6.5% (48 mmol/mol) (circle) and FPG greater than or equal to 126 mg/dL (square). B, On internal validation, the decision curve analysis shows that the A1c+ model performs similarly to the recalibrated A1c but superior to the A1c greater than or equal to 6.5% (48 mmol/mol) and the A1c/FPG+ model outperforms the A1c + FPG, recalibrated FPG, and FPG greater than or equal to 126 mg/dL. C, For individuals with A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL, among nonfasting approaches, the routine clinical feature A1c+ model has superior AUROC and sensitivity than the recalibrated A1c, but worse specificity. Among the fasting models the A1c/FPG+ has higher AUROC and higher sensitivity than the recalibrated FPG and the logistic regression of the A1c and the FPG (A1c + FPG), but lower specificity than the A1c + FPG. Points indicate sensitivity and specificity for A1c greater than or equal to 6.5% (circle) and FPG greater than or equal to 126 mg/dL (square). D, For individuals with A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL, the decision curve analysis shows that the A1c/FPG+ model performs superiorly to the other models. Diagonal line in A and C indicates test characteristics of a random test. Solid gray lines without points in B and D indicate the net benefit of testing all candidates or testing no candidates with oral glucose tolerance testing for diagnosis of diabetes.
On internal validation (see Fig. 1A and 1B), among the nonfasting approaches (A1c ≥ 6.5% [48 mmol/mol], A1c, and A1c+), the A1c+ model had better AUROC than the recalibrated A1c (0.87 vs 0.83) and better sensitivity than the A1c both at Youden's point (A1c ≥ 5.85% [40 mmol/mol]) and the diagnostic threshold of 6.5% (48 mmol/mol) (sensitivity: 0.72 vs 0.66 vs 0.3, respectively). The net benefit was equivalent between the A1c+ model classification and recalibrated A1c classification, though both were better than the A1c greater than or equal to 6.5% (48 mmol/mol) threshold (see Fig. 1B). When including FPG in the model, we fit a logistic regression of A1c and FPG using the training data, with the Youden index at the equation Probability of Diabetes = 1/(1 + e−(−29.5+1.7(A1c)+0.2(FPG))), which corresponds to points such as (FPG = 120 mg/dL, A1c = 6.4% [46 mmol/mol], probability = 61%) or (FPG = 125 mg/dL, A1c = 6.0% [42 mmol/mol], probability = 64%) (Supplementary Fig. S3 (32)). The A1c/FPG+ model had better AUROC than the logistic regression model (A1c + FPG) and the FPG (0.96 vs 0.93 vs 0.92, respectively; see Fig. 1A) and had an overall improved net benefit (see Fig. 1B). The A1c/FPG+ had equivalent sensitivity to the A1c + FPG logistic regression and FPG at Youden's point at FPG greater than or equal to 110 mg/dL, but better sensitivity than the FPG greater than or equal to 126 mg/dL threshold (0.85 vs 0.86 vs 0.84 vs 0.6, respectively) but superior specificity to the A1c + FPG logistic regression and FPG at Youden's point (0.94 vs 0.89 vs 0.88, respectively) (see Fig. 1A). All comparisons had P values less than .005 by DeLong's test (Supplementary Table S2 (32)).
We conducted a subanalysis involving only internal validation NHANES participants with A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL (Fig. 1C and 1D), for which the A1c greater than or equal to 6.5% (48 mmol/mol) and FPG greater than or equal to 126 mg/dL would have sensitivity of 0 and specificity of 1. We found that among the nonfasting approaches (recalibrated A1c and A1c+), the A1c+ model had better AUROC than the recalibrated A1c (0.81 vs 0.75) and better sensitivity than the A1c both at Youden's point (A1c ≥ 5.85% [40 mmol/mol]) (sensitivity: 0.95 vs 0.75, respectively) (see Fig. 1C). The net benefit was equivalent between the A1c+ model classification and recalibrated A1c classification (see Fig. 1D). The A1c/FPG+ model had better AUROC than the logistic regression model (A1c + FPG) and the FPG (0.86 vs 0.79 vs 0.76, respectively; see Fig. 1C) and had an overall improved net benefit (see Fig. 1D). The A1c/FPG+ had superior sensitivity to the A1c + FPG logistic regression and FPG at Youden's point at FPG greater than or equal to 110 mg/dL (0.84 vs 0.64 vs 0.75, respectively) but worse specificity than the A1c + FPG logistic regression and equivalent specificity to the FPG at Youden's point (0.71 vs 0.86 vs 0.72, respectively) (see Fig. 1C). All comparisons of nonfasting methods to nonfasting methods (A1c+ vs A1c) had P less than .02 and comparisons of fasting methods to fasting methods (A1c/FPG+ vs A1c + FPG vs FPG) had P values less than or equal to .01 by DeLong's test (see Supplementary Table S2 (32)).
We also externally validated our results using DPPOS, comparing the nonmedication and medication cohorts stratified into both full (all A1c and FPG values) and subdiagnostic (A1c < 6.5% [48 mmol/mol] and FPG < 126 mg/dL) cohorts (Fig. 2). We input overlapping features from the DPPOS cohort into the model, leaving features as NA where absent. For the full cohorts (all A1c and FPG values), the A1c+ model performed equivalently to the recalibrated A1c both for the nonmedication and medication cohorts for AUROC, sensitivity, and specificity (see Figs. 2A-2D). For the nonmedication cohort, the A1c/FPG+ model had equivalent AUROC and sensitivity compared to the logistic A1c + FPG model (0.87 vs 0.85 and 0.74 vs 0.74), though both were better than the recalibrated FPG (AUROC: 0.81, sensitivity 0.67) (Supplementary Table S3 (32)), with slightly better specificity for the A1c/FPG+ model than the A1c + FPG or FPG (0.86 vs 0.81 vs 0.82, respectively) (see Fig. 2A). The A1c/FPG+ model had an improved net benefit over the other models for threshold probability greater than 12% for the nonmedication cohort (see Fig. 2B). For the medication cohort, the A1c/FPG+ model had slightly better AUROC than the A1c + FPG and FPG (0.84 vs 0.80 vs 0.75), but worse sensitivity than the A1c + FPG model at Youden's point (equivalent to the recalibrated FPG), though this came with an improved specificity (sensitivity: 0.6 vs 0.67 vs 0.6, specificity: 0.91 vs 0.79 vs 0.78, respectively) (see Fig. 2C) (Supplementary Table S4 (32)). The A1c/FPG+ model had an improved net benefit over the other models for threshold probability greater than 15% for the medication cohort (see Fig. 2D).
Figure 2.
Receiver operating characteristics (A, C, E, F) and decision curve analysis net benefit plots (B and D) for the A1c+ and A1c/FPG+ models on Diabetes Prevention Program Outcomes Study (DPPOS) external validation for the full external validation cohort of individuals receiving lifestyle or placebo (A and B) or receiving metformin or troglitazone (C and D), as well as individuals with A1c less than 6.5% (48 mmol/mol) and fasting plasma glucose (FPG) less than 126 mg/dL for the E, nonmedication and F, medication cohorts. A, On external validation of individuals not receiving medication, the A1c+ model performed equivalently to the recalibrated A1c and had superior sensitivity to the A1c at diagnostic threshold of 6.5% (48 mmol/mol). The A1c/FPG+ model performed equivalently to the A1c + FPG logistic regression model, with superior performance compared to the recalibrated FPG and FPG at a fixed diagnostic threshold of 126 mg/dL. B, On external validation of subjects not receiving medication, the A1c/FPG+ model had superior net benefit in comparison to the model models for threshold probability greater than 12%. C, On external validation of individuals receiving medication, the A1c+ model performed equivalently to the recalibrated A1c and had superior sensitivity to the A1c at a diagnostic threshold of 6.5% (48 mmol/mol). The A1c/FPG+ model had better specificity but worse sensitivity compared to the A1c + FPG logistic regression model, with superior performance compared to the recalibrated FPG and FPG at a fixed diagnostic threshold of 126 mg/dL. D, On external validation of individuals receiving medication, the A1c/FPG+ model had a superior net benefit in comparison to the other models for threshold probability greater than 15%. E, On external validation of individuals not receiving medication who had nondiagnostic (non-diag) A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL, the A1c+ model had better specificity but worse sensitivity than the recalibrated A1c. The A1c/FPG+ model had superior sensitivity and worse specificity compared to the A1c + FPG logistic regression model. F, On external validation of individuals receiving medication who had nondiagnostic (non-diag) A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL, the A1c+ model had better specificity but worse sensitivity than the recalibrated A1c. The A1c/FPG+ model had superior sensitivity and worse specificity compared to the A1c + FPG logistic regression model and recalibrated FPG.
We additionally looked at individuals with nondiagnostic A1c and FPG both for the nonmedication and medication cohorts. For these cohorts the A1c+ model had equivalent AUROC, worse sensitivity, and improved specificity compared to the recalibrated A1c (Fig. 2E and 2F, Supplementary Tables S3 and S4 (32)). However, the A1c/FPG+ model displayed the opposite pattern, with equivalent AUROC, improved sensitivity, and worse specificity in both cohorts.
To ensure that testing on absent data from DPPOS that was present in the training NHANES data was not negatively affecting model performance, we trained a version of the A1c+ and A1c/FPG+ models only on the aforementioned features included in DPPOS: This did not meaningfully change AUCs of the models, so the full models were used for further analysis (Supplementary Table S5). Calibration of the A1c/FPG+ model was confirmed by quantile calibration (Supplementary Fig. S6 (32)).
Top features of interest in diabetes prediction
To understand the relative weighting of features in the A1c+ and A1c/FPG+ models, we plotted the top 20 important features of each for the NHANES internal validation (Fig. 3A and 3C) and DPPOS external validation (using the full cohort and pooling the medication and nonmedication cohorts) (Fig. 3B and 3D). For the A1c+ model (see Fig. 3A and 3B), the top 5 vital sign features were age, systolic BP, waist circumference, height, and heart rate. The top 5 nonfasting laboratory values after A1c were hemoglobin, alanine transaminase (ALT), total protein, aspartate transaminase (AST), and creatinine. For the A1c/FPG+ model (see Fig. 3C and 3D), the inclusion of FPG increased the importance of creatinine to the next important laboratory value after A1c and FPG, followed by ALT, AST, white blood cell count, and red blood cell mean cellular volume.
Figure 3.
Plot of Shapley (SHAP) values for top 20 features of the A1c+ (A and B) and A1c/FPG+ (C and D) models. Plots shown for NHANES internal validation (A and C) and Diabetes Prevention Program Outcomes Study (DPPOS) external validation (B and D). Each point represents a subject, with the SHAP value representing the contribution of a feature value to the difference between the actual prediction and the mean prediction without that feature. Solid black points represent missing (NA) data. Notably, the DPPOS data lacked laboratory values other than the A1c, fasting plasma glucose (FPG), and creatinine. Positive values indicate a positive effect on the probability of diabetes, and negative values indicate a negative effect. ALT, alanine transaminase; AST, aspartate transaminase; RBC, red blood cell; Circ, circumference; BP, blood pressure; prot., protein; bili., bilirubin; MCV, mean cellular volume; alk. phos., alkaline phosphatase; WBC, white blood cell.
Association of diabetes diagnosis with mortality
To assess the importance of OGTT-only diabetes relative to diabetes diagnosable by the A1c and/or FPG, we examined the 2005 to 2016 cohort, in which 11 500 of 138 000 individuals with A1c, FPG, OGTT, and appropriate sample weights had paired NCHS mortality data. We separated individuals by no diagnosis of diabetes, diagnosis of diabetes by A1c or FPG, and diagnosis of diabetes by OGTT only. We found that when controlling for age, sex, race/ethnicity, and BMI, participants with OGTT-only diabetes had an increased risk of mortality compared to those without diabetes (HR = 1.4; P = .002) and individuals with A1c/FPG-diabetes had an increased risk of mortality compared to those without diabetes (HR = 1.6; P = 3 × 10−5). There was no statistical difference between the two modes of diagnosing diabetes on mortality risk (HR = 0.9; P = .45) (Fig. 4A).
Figure 4.
A, Individuals with diabetes (DM) diagnosed via oral glucose tolerance test (OGTT) alone had equivalent mortality to those with diabetes diagnosed with hemoglobin A1c (A1c) and/or fasting plasma glucose (FPG). Data are from the National Center for Health Statistics (NCHS) data and National Health and Nutrition Examination Survey (NHANES) 2005 to 2016 cohorts that had OGTT measurement stratified by individuals with a diagnosis of DM by A1c greater than or equal to 6.5% (48 mmol/mol) or FPG greater than or equal to 126 mg/dL (dark gray), individuals with a diagnosis of DM by OGTT alone (2-hour glucose >200 mg/dL but A1c < 6.5% [48 mmol/mol] and FPG < 126 mg/dL, black), or individuals without a diagnosis of DM (light gray). A, Kaplan-Meier curves of mortality stratified by presence and diagnostic method of DM. Comparisons made adjusting for age, sex, race/ethnicity, and BMI. For the * and + comparisons, the reference is to individuals without DM. For the Δ comparison, the reference is to individuals with DM diagnosed by OGTT alone. B, Individuals without a diagnosis of DM at time of examination that were predicted to have DM via the A1c/FPG+ model had higher associated hazard ratio with mortality than those not predicted to have DM, though this risk was lower than those with a diagnosis of DM. Comparison of mortality from NCHS data for NHANES 1999 to 2016 cohorts that did not have OGTT measurement stratified by individuals with a diagnosis of DM by A1c greater than or equal to 6.5% (48 mmol/mol) or FPG greater than or equal to 126 mg/dL (dark gray) and individuals without a diagnosis of DM (A1c < 6.5% (48 mmol/mol) and FPG < 126 mg/dL) based on whether the A1c/FPG+ model predicts (black) or does not predict (light gray) a diagnosis of DM. For the * and + comparisons, the reference is to individuals with A1c less than 6.5% (48 mmol/mol), FPG less than 126 mg/dL, and no prediction of DM. For the Δ comparison, the reference is to individuals with A1c less than 6.5% (48 mmol/mol), FPG < 126 mg/d, and predicted DM.
Association of glycated hemoglobin A1c/fasting plasma glucose ± model predictions with mortality
To assess the importance of our model predictions, we applied the A1c/FPG+ model to the 41 670 participants in NHANES from 1999 to 2016 who did not have OGTT measurements (and therefore were not used for model training or evaluation) and had NCHS mortality data. We stratified individuals into those with A1c- or FPG-diagnosed diabetes, those with subdiagnostic A1c and FPG but with A1c/FPG+ model-predicted diabetes, and those with subdiagnostic A1c and FPG but without A1c/FPG+ model-predicted diabetes (see Fig. 3B). After adjusting for age, sex, race/ethnicity, and BMI, we found that among individuals without A1c/FPG-diagnosed diabetes, model-predicted diabetes had a 1.7-fold increased risk of mortality compared to those without model-predicted diabetes (P = 1 × 10−24). This was in comparison to individuals with A1c/FPG-diagnosed diabetes, who had a 2.4-fold increased risk of mortality compared to participants without A1c- or FPG-diagnosed diabetes and without model-predicted diabetes (P = 1 × 10−56). A1c/FPG-diagnosed diabetes had a 1.5-fold increased risk of mortality compared to individuals without A1c- or FPG-diagnosed diabetes and with model-predicted diabetes (P = 3 × 10−27) (see Fig. 4B).
Discussion
In this study, we developed, validated, and justified a model to use routine clinical features to improve the identification of individuals that need an OGTT to diagnose their diabetes. By combining the A1c and FPG with routine clinical features (vital signs, complete blood count, comprehensive metabolic panel) that are already obtained during standard adult care into the A1c/FPG+ model, we outperformed the A1c and FPG at their diagnostic thresholds for diabetes and performed equivalently or better than when we recalibrated the A1c and FPG based on the NHANES training data. We then justified the relevance of OGTT-only diabetes and our diabetes predictions by showing that they were associated with an increased risk of mortality.
The A1c was originally chosen as a diagnostic method for diabetes with a threshold of 6.5% (48 mmol/mol) due to its clinical convenience. However, our work confirms that this misses diabetes in many individuals and explores two alternatives. The first is implementation of a diagnostic cascade using an A1c threshold at which a follow-up FPG or OGTT is warranted (Youden's index for the A1c was 5.85% [40 mmol/mol] for the NHANES data and 6.25% [44.5 mmol/mol] for the DPPOS data) and a FPG threshold at which an OGTT is warranted (Youden's index for the FPG was 110 mg/dL for the NHANES data and 124 mg/dL for the DPPOS data). These can even be combined as we did in our A1c + FPG logistic regression (see Supplementary Fig. S3 (32)). The second approach implements a diagnostic cascade that leverages the readily-available nonfasting data available in the EMR, such as the complete blood count, comprehensive metabolic panel, and vital signs, as we do in generation of the A1c+ and A1c/FPG+ models, which outperformed the recalibrated A1c, FPG, and A1c + FPG regression. For individuals with A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL, the A1c/FPG+ had much better sensitivity than the other approaches in the internal validation NHANES cohort and both the nonmedication and medication versions of the external validation DPPOS cohort.
A limitation of our study is that the DPPOS data miss all but one of the non-A1c non-FPG routine laboratory features used in the training and internal validation of the A1c+ and A1c/FPG+ models. On internal validation the A1c/FPG+ model outperformed the FPG and A1c + FPG logistic regression equivalently to how the A1c+ model outperformed the A1c. Despite the lack of laboratory features, on external validation the A1c/FPG+ model outperformed the recalibrated FPG and A1c + FPG while the A1c+ model performed equivalently to the recalibrated A1c. Importantly, when looking specifically at DPPOS individuals with A1c less than 6.5% (48 mmol/mol) and FPG less than 126 mg/dL, we found that the A1c/FPG+ model had a higher sensitivity than the alternative approaches (including a sensitivity of 0 for A1c ≥ 6.5% (48 mmol/mol) and FPG ≥ 126 mg/dL) for identifying individuals who would have diabetes if tested with an OGTT. This was true regardless of whether they were in the placebo and lifestyle arms or the metformin and troglitazone arms of the DPPOS. We expect that when paired with more routine laboratory data, the A1c+ model would perform superiorly to the recalibrated A1c.
We have reaffirmed the importance of OGTT-only diabetes as having an increased HR for mortality compared to no diabetes (HR = 1.4; P = .002), statistically insignificantly lower than (HR = 0.9; P = .45) the HR for mortality of A1c- or FPG-diagnosed diabetes compared to no diabetes (HR = 1.6; P = .00003). Studies in earlier populations had established the association of OGTT results with mortality (37), but this had not been examined in a modern US 21st-century cohort. While the underlying pathophysiology of postprandial hyperglycemia may differ from the elevated hepatic insulin resistance that drives increases in FPG, the risk of mortality appears to remain similar. From 2005 to 2016 more than 1 in 100 individuals had OGTT-only diabetes. These individuals are missed by conventionally used A1c and FPG screening approaches, justifying the need for a means to identify these individuals who would otherwise be missed despite having equivalently bad outcomes. Individuals without diabetes whom our model predicted had diabetes had higher mortality than those without diabetes whom our model predicted did not have diabetes: This is analogous to the increased risk of mortality with OGTT-only diabetes when the A1c is less than 6.5% (48 mmol/mol) and FPG is less than 126 mg/dL. However, we do not recommend our model predictions to be used to risk stratify for mortality—instead we provide this analysis to highlight that the model predictions of OGTT-only diabetes are meaningful and warrant further study to understand their predictive potential.
We have developed a machine-learning model that can easily integrate into clinical care as it does not require additional vitals, demographics, medication information, or laboratory tests other than what are routine clinical features for a US adult. Given this is information available in the EMR, future integration of A1c/FPG+-type models into the EMR should guide screening. Additional models could be developed and validated that would tailor included clinical features to specific patient populations. This will be especially of use to general practitioners, who may not have the same familiarity with conditions that would require more extensive testing for diabetes with OGTT or bandwidth to pursue the work-up and diagnosis. In the setting of a subdiagnostic A1c, an EMR model that calculated a high risk of diabetes could trigger an alert recommending fasting laboratory values, and if the FPG was subdiagnostic again an EMR model with a high risk could trigger a recommendation to obtain an OGTT, or at the very least repeat follow-up laboratory values quickly (Fig. 5). While earlier detection and therefore earlier glucose control should lead to improved outcomes (38), further work and implementation will be needed to validate this.
Figure 5.
Schematic of conceptual framework of clinical application of A1c+ and A1c/FPG+ models. A1c, glycated hemoglobin A1c; CKD, chronic kidney disease; OGTT, oral glucose tolerance test.
There is an opportunity for future exploration of this topic. This study externally validates the A1c/FPG+ model using mortality data from the NCHS—these machine-learning models should be validated using other general population datasets as well as ideally chronic disease–specific datasets. Focusing on chronic diseases for which the A1c is known to be inaccurate, such as pregnancy (39), cystic fibrosis (40), pancreatitis (41), chronic liver disease (42, 43), and chronic kidney disease (44), would fine-tune the models where there is a higher pretest probability of OGTT-only diabetes. This would not necessarily require separate models: The underlying methodological framework of the decision-tree analysis should be able to fine-tune the branches of the tree in which creatinine, lipase, or ALT is elevated, for example. In addition to further attention to etiology, additional models could incorporate the effects of newer medications that lower blood sugar, even when used for nondiabetes indications (such as weight loss or cardiovascular risk reduction), though we showed that metformin and troglitazone had little effect on model characteristics. Further work could also focus on the continuous aspect of prediction: Here, we threshold on predicting diabetes vs no diabetes, despite losing statistical power doing so (45). Instead, a continuous risk score may better guide when to repeat standard testing as opposed to obtaining an OGTT, and the timeline in which to pursue these options.
In this work, we generated NHANES-trained models that can improve the detection of individuals in the general population who would need OGTT assessment to diagnose diabetes using nonfasting laboratory values, vital signs, and the stepwise inclusion of the FPG. We externally validated the results and showed the predictions were meaningfully associated with mortality. These models lay the framework for future implementation of a clinical decision support tool that could improve the diagnosis of diabetes and lead to earlier interventions. Future work will need to validate, refine, and integrate these findings to improve our ability to personalize diabetes risk-stratification for patients.
Acknowledgments
We would like to thank Dr Esra Tasali and Molly White for their feedback in preparation of this manuscript. No patients were involved in the study design or preparation of this manuscript. The DPPOS was conducted by the study investigators and supported by the NIDDK. The resources from the DPPOS (31) reported here were supplied by the NIDDK Central Repository (NIDDK-CR) and are available on request at https://repository.niddk.nih.gov. This manuscript was not prepared under the auspices of the DPPOS and does not necessarily reflect the opinions or views of the DPPOS, NIDDK-CR, or NIDDK. The authors acknowledge the NCHS at the CDC for the design, collection, and administration of the NHANES data used in this study (29). This study was evaluated by the University of Chicago Institutional Review Board (study identification Nos. 25-1210 and 25-0943) and deemed to be exempt from further review.
Abbreviations
- ALT
alanine transaminase
- AST
aspartate transaminase
- AUROC
area under the receiver operating curve
- BMI
body mass index
- BP
blood pressure
- CDC
Centers for Disease Control and Prevention
- DPPOS
Diabetes Prevention Program Outcomes Study
- EMR
electronic medical record
- FPG
fasting plasma glucose
- HbA1c
glycated hemoglobin A1c
- NCHS
National Center for Health Statistics
- NHANES
National Health and Nutrition Examination Survey
- NIDDK
National Institute of Diabetes and Digestive and Kidney Diseases
- OCTT
oral glucose tolerance test
Contributor Information
Alan L Hutchison, Email: AlanLHutchison@uchicago.edu, Department of Medicine, Section of Gastroenterology, Hepatology and Nutrition, University of Chicago Medicine, Chicago, IL 60637, USA.
Simar Narula, Department of Medicine, Section of Gastroenterology, Hepatology and Nutrition, University of Chicago Medicine, Chicago, IL 60637, USA.
Brandon Faubert, Department of Medicine, Section of Hematology and Oncology, University of Chicago Medicine, Chicago, IL 60637, USA.
Celeste C Thomas, Department of Medicine, Section of Endocrinology, Diabetes and Metabolism, University of Chicago Medicine, Chicago, IL 60637, USA.
Mary E Rinella, Department of Medicine, Section of Gastroenterology, Hepatology and Nutrition, University of Chicago Medicine, Chicago, IL 60637, USA.
Raghavendra G Mirmira, Department of Medicine, Section of Endocrinology, Diabetes and Metabolism, University of Chicago Medicine, Chicago, IL 60637, USA.
William F Parker, Department of Medicine, Section of Pulmonary and Critical Care, University of Chicago Medicine, Chicago, IL 60637, USA.
Funding
A.L.H. has received support from the National Institutes of Health (NIH) K23 DK146188, NIH P30 DK020595-8056, NIH P30 DK092949-5104, NIH T32 DK007074, the American Association for the Study of Liver Diseases, and the University of Chicago Physician Scientist Development Program. R.G.M. has received support from NIH U01 DK127786, R01 DK060581, and R01 DK105588. W.F.P. has received support by NIH K08 HL150291, R01 LM014263, R01 HL176972, and the Greenwall Foundation.
Author contributions
A.L.H., S.N., C.C.T., R.G.M., B.F., M.E.R., and W.F.P. were involved in the conception, design, and conduct of the study and the analysis and interpretation of the results. A.L.H. wrote the first draft of the manuscript, and all authors edited, reviewed, and approved the final version of the manuscript. A.L.H. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Disclosures
A.L.H., W.F.P., R.G.M., B.F, C.C.T., M.E.R., and S.N. have no relevant financial disclosures.
Data availability
Restrictions apply to the availability of the DPPOS data analyzed during this study because they were used via data use agreement with the NIDDK Central Repository (31). The corresponding author will on request detail the restrictions and any conditions under which access to some data may be provided. The NHANES and NCHS data are freely available for download from the CDC (29).
References
- 1. National Institute of Diabetes and Digestive and Kidney Diseases . Diabetes statistics. Reviewed January 2024. Accessed October 13, 2026. https://www.niddk.nih.gov/health-information/health-statistics/diabetes-statistics
- 2. Parker ED, Lin J, Mahoney T, et al. Economic costs of diabetes in the U.S. in 2022. Diabetes Care. 2024;47(1):26‐43. [DOI] [PubMed] [Google Scholar]
- 3. American Diabetes Association Professional Practice Committee . 2. Diagnosis and classification of diabetes: standards of care in diabetes-2025. Diabetes Care. 2025;48(Supplement_1):S27‐S49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Hanas R, John G; International HBA1c Consensus Committee . 2010 consensus statement on the worldwide standardization of the hemoglobin A1C measurement. Diabetes Care. 2010;33(8):1903‐1904. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. ElSayed NA, Aleppo G, Aroda VR, et al. 6. glycemic targets: standards of care in diabetes-2023. Diabetes Care. 2023;46(Supplement_1):S97‐S110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. International Expert Committee . International expert committee report on the role of the A1C assay in the diagnosis of diabetes. Diabetes Care. 2009;32(7):1327‐1334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Klein KR, Lingvay I, Tuttle KR, Flythe JE. Glycemic management and individualized diabetes care in dialysis-dependent kidney failure. Diabetes Care. 2025;48(2):164‐176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Malkani S, Mordes JP. Implications of using hemoglobin A1C for diagnosing diabetes mellitus. Am J Med. 2011;124(5):395‐401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Centers for Disease Control and Prevention . Chronic kidney disease in the United States. Updated March 31, 2026. Accessed October 13, 2026. https://www.cdc.gov/kidney-disease/php/data-research/index.html
- 10. National Institute of Diabetes and Digestive and Kidney Diseases . Overweight & obesity statistics. Reviewed September 2021. Accessed October 13, 2026. https://www.niddk.nih.gov/health-information/health-statistics/overweight-obesity
- 11. Acquavella J, Mehra R, Bron M, Suomi JM, Hess GP. Prevalence of narcolepsy and other sleep disorders and frequency of diagnostic tests from 2013-2016 in insured patients actively seeking care. J Clin Sleep Med. 2020;16(8):1255‐1263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Sharif A, Chakkera H, de Vries APJ, et al. International consensus on post-transplantation diabetes mellitus. Nephrol Dial Transplant. 2024;39(3):531‐549. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Paul JC, Fitzpatrick JJ. Postpartum glucose screening among women with gestational diabetes. Appl Nurs Res. 2020;56:151341. [DOI] [PubMed] [Google Scholar]
- 14. Brown SD, Hedderson MM, Zhu Y, et al. Uptake of guideline-recommended postpartum diabetes screening among diverse women with gestational diabetes: associations with patient factors in an integrated health system in USA. BMJ Open Diabetes Res Care. 2022;10:e002726. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Camacho JE, Shah VO, Schrader R, Wong CS, Burge MR. Performance of A1C versus OGTT for the diagnosis of prediabetes in a community-based screening. Endocr Pract. 2016;22(11):1288‐1295. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Meijnikman AS, De Block CEM, Dirinck E, et al. Not performing an OGTT results in significant underdiagnosis of (pre)diabetes in a high risk adult Caucasian population. Int J Obes (Lond). 2017;41(11):1615‐1620. [DOI] [PubMed] [Google Scholar]
- 17. Cowie CC, Rust KF, Byrd-Holt DD, et al. Prevalence of diabetes and impaired fasting glucose in adults in the U.S. population: national health and nutrition examination survey 1999-2002. Diabetes Care. 2006;29(6):1263‐1268. [DOI] [PubMed] [Google Scholar]
- 18. Porter J, Boyd C, Skandari MR, Laiteerapong N. Revisiting the time needed to provide adult primary care. J Gen Intern Med. 2023;38(1):147‐155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Griffin SJ, Little PS, Hales CN, Kinmonth AL, Wareham NJ. Diabetes risk score: towards earlier detection of type 2 diabetes in general practice. Diabetes Metab Res Rev. 2000;16(3):164‐171. [DOI] [PubMed] [Google Scholar]
- 20. Bang H, Edwards AM, Bomback AS, et al. Development and validation of a patient self-assessment score for diabetes risk. Ann Intern Med. 2009;151(11):775‐783. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery; 2016:785-794. 10.1145/2939672.2939785 [DOI]
- 22. Hu C, Li L, Huang W, et al. Interpretable machine learning for early prediction of prognosis in sepsis: a discovery and validation study. Infect Dis Ther. 2022;11(3):1117‐1132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Zhang Q, Xu L, He W, Lai X, Huang X. Survival prediction for heart failure complicated by sepsis: based on machine learning methods. Front Med (Lausanne). 2024;11:1410702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Zheng J, Li J, Zhang Z, et al. Clinical data based XGBoost algorithm for infection risk prediction of patients with decompensated cirrhosis: a 10-year (2012-2021) multicenter retrospective case-control study. BMC Gastroenterol. 2023;23(1):310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. De Silva K, Lim S, Mousa A, et al. Nutritional markers of undiagnosed type 2 diabetes in adults: findings of a machine learning analysis with external validation and benchmarking. PLoS One. 2021;16(5):e0250832. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Esser K, Duong M, Kain K, et al. Predicting diabetes in Canadian adults using machine learning. Annu Int Conf IEEE Eng Med Biol Soc. 2024;2024:1‐4. [DOI] [PubMed] [Google Scholar]
- 27. Liu X, Zhang W, Zhang Q, et al. Development and validation of a machine learning-augmented algorithm for diabetes screening in community and primary care settings: a population-based study. Front Endocrinol (Lausanne). 2022;13:1043919. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Sun K, Xiao X, You L, et al. Development and validation of a nomogram for assessing risk of isolated high 2-hour plasma glucose. Front Endocrinol (Lausanne). 2022;13:943750. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. National Center for Health Statistics . National Health and Nutrition Examination Survey data, 1999-2016. US Department of Health and Human Services, Centers for Disease Control and Prevention; 2016. Accessed October 13, 2026. https://www.cdc.gov/nchs/nhanes/index.html
- 30. Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+ AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Fowler S. (2026). Diabetes Prevention Program Outcomes Study (DPPOS) (Version 7) [Dataset] NIDDK Central Repository. 10.58020/66x5-8y21 [DOI]
- 32. Hutchison AL, Narula S, Faubert B, et al. Supplementary material for: Development and validation of a model using routine clinical features to improve diabetes diagnosis. Zenodo; 2026. https://zenodo.org/records/21533817
- 33. Rinella ME, Neuschwander-Tetri BA, Siddiqui MS, et al. AASLD practice guidance on the clinical assessment and management of nonalcoholic fatty liver disease. Hepatology. 2023;77(5):1797‐1835. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Rinella ME, Lazarus JV, Ratziu V, et al. A multisociety delphi consensus statement on new fatty liver disease nomenclature. Hepatology. 2023;78(6):1966‐1986. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Clements JM, West BT, Yaker Z, et al. Disparities in diabetes-related multiple chronic conditions and mortality: the influence of race. Diabetes Res Clin Pract. 2020;159:107984. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Hutchison A. NHANES-DMDx. In: alanlhutchison, editor. Github; 2026. https://github.com/alanlhutchison/NHANES-DMDx
- 37. DECODE Study Group, the European Diabetes Epidemiology Group . Glucose tolerance and cardiovascular mortality: comparison of fasting and 2-hour diagnostic criteria. Arch Intern Med. 2001;161(3):397‐405. [DOI] [PubMed] [Google Scholar]
- 38. Winn AN, Newman Z, Deckard A, et al. Development and internal validation of the multiethnic type 2 diabetes outcomes model for the U.S. (DOMUS). Diabetes Care. 2025;48(11):1942‐1950. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Radin MS. Pitfalls in hemoglobin A1c measurement: when results may be misleading. J Gen Intern Med. 2014;29(2):388‐394. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Moran A, Brunzell C, Cohen RC, et al. Clinical care guidelines for cystic fibrosis-related diabetes: a position statement of the American diabetes association and a clinical practice guideline of the cystic fibrosis foundation, endorsed by the pediatric endocrine society. Diabetes Care. 2010;33(12):2697‐2708. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Chen G, Zhang R, Tan C, Liu X, Yu L, Chen Y. Optimal glycated hemoglobin A1c value for prediabetes and diabetes in patients with pancreatic diseases. Front Endocrinol (Lausanne). 2023;14:1208187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Bragança AC, Alvares-da-Silva MR. Prevalence of diabetes mellitus and impaired glucose tolerance in patients with decompensated cirrhosis being evaluated for liver transplantation: the utility of oral glucose tolerance test. Arq Gastroenterol. 2010;47(1):22‐27. [DOI] [PubMed] [Google Scholar]
- 43. Barron CVM, Heenan HF, Thompson H, Chan H, Ngu J, Lunt H. Detecting dysglycaemia in compensated liver cirrhosis: comparison of oral glucose tolerance test and glycated haemoglobin, with continuous glucose monitoring. Diabet Med. 2022;39(3):e14778. [DOI] [PubMed] [Google Scholar]
- 44. Chadban SJ, Ahn C, Axelrod DA, et al. KDIGO clinical practice guideline on the evaluation and management of candidates for kidney transplantation. Transplantation. 2020;104(4S1):S11‐S103. [DOI] [PubMed] [Google Scholar]
- 45. Hutchison AL, Cifu AS, Pillai AA. Discarding dichotomization: retrieving data in the service of patient care. Clin Liver Dis (Hoboken). 2022;19(1):12‐16. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Citations
- Hutchison AL, Narula S, Faubert B, et al. Supplementary material for: Development and validation of a model using routine clinical features to improve diabetes diagnosis. Zenodo; 2026. https://zenodo.org/records/21533817
Data Availability Statement
Restrictions apply to the availability of the DPPOS data analyzed during this study because they were used via data use agreement with the NIDDK Central Repository (31). The corresponding author will on request detail the restrictions and any conditions under which access to some data may be provided. The NHANES and NCHS data are freely available for download from the CDC (29).