Abstract
Background
This study compares a previously developed Diabetes Risk Score to commonly used clinical tools for type 2 diabetes risk evaluation in the Insulin Resistance Atherosclerosis Study (IRAS) cohort, a multi-ethnic US cohort. Available as a clinical test, the PreDx® Diabetes Risk Score uses fasting concentrations of adiponectin, C-reactive protein, ferritin, interleukin-2 receptor alpha, HbA1c, glucose and insulin, plus age and gender to predict 5-year risk of diabetes. It was developed in a Northern European population.
Methods
The Diabetes Risk Score was measured using archived fasting plasma specimens from 722 non-diabetic IRAS participants, 17.6% of whom developed diabetes during 5.2 years median follow-up (inter-quartile range: 5.1–5.4 years). The study included non-Hispanic whites (41.8%), Hispanics (34.5%) and African Americans (23.7%). Performance of the algorithm was evaluated by area under the receiver operating characteristic curve (AROC) and risk reclassification against other tools.
Results
The Diabetes Risk Score discriminates participants who developed diabetes from those who did not significantly better than fasting glucose (AROC=0.763 versus 0.710, p=0.003). The Diabetes Risk Score performed equally well in subpopulations defined by race/ethnicity or gender. The Diabetes Risk Score provided a significant net reclassification improvement of 0.24 (p=0.01) when comparing predefined low/moderate/high Diabetes Risk Score categories to metabolic syndrome risk factor counting. The Diabetes Risk Score complemented the use of the oral glucose tolerance test by identifying high risk patients with impaired fasting glucose but normal glucose tolerance, 33% of whom converted.
Conclusions
Measuring the Diabetes Risk Score of elevated-risk US patients could help physicians decide which patients warrant more intensive intervention. The Diabetes Risk Score performed equally well across the ethnic subpopulations present in this cohort
Keywords: type 2 diabetes, risk scores, biomarkers, multi-ethnic cohort, net reclassification improvement, diabetes prevention
Introduction
Clinicians providing medical care to adults need effective tools for identifying patients at high risk of developing type 2 diabetes that can be easily incorporated into routine clinical practice. Controlled clinical trials have provided evidence that intensive lifestyle modifications or pharmacological treatment can delay or prevent the onset of type 2 diabetes [1–4]. Physicians commonly screen patients for impaired fasting glucose (IFG), elevated HbA1c or metabolic syndrome to identify patients at moderately elevated risk, but these methods suffer from low specificity. Because interventions can be costly and resources limited, effective prevention strategies must focus on individuals at greatest risk of developing diabetes.
The PreDx DRS is a multi-marker test for evaluating a patient's absolute 5-year risk of incident type 2 diabetes. The DRS algorithm uses biomarker concentrations measured in fasting blood samples, plus age and gender. It was developed to provide physicians with a means of estimating risk that could be ordered as a simple laboratory test that does not require input of medical history, demographic or anthropometric information. The DRS is intended for use among patients already known to be at elevated risk of developing diabetes to further risk stratify patients for diabetes prevention efforts. In a general population, it has been shown to provide a better assessment of diabetes risk than fasting plasma glucose alone [5]. Because the DRS was originally developed and independently validated using samples from adults of Northern European origin [5–7], it is important to assess the DRS in multi-ethnic cohorts more representative of its intended-use population.
The Insulin Resistance Atherosclerosis Study (IRAS) is a multicenter, epidemiological study that has examined insulin resistance and cardiovascular risk factors and disease across different ethnic groups and varying states of glucose tolerance [8]. This well-characterized, ethnically diverse US study included Hispanics, non-Hispanic whites (NHW) and African-American participants.
The aim of the present study was to evaluate the performance of the DRS in assessing the 5-year risk of incident type 2 diabetes in IRAS. The performance of the DRS in discriminating participants with diabetes (‘converters’) from participants without diabetes (‘non-converters’) at the 5-year follow-up visit was compared with that of other risk assessment tools, including fasting glucose, body mass index (BMI), fasting insulin, the homeostasis model assessment of insulin resistance (HOMA-IR) [9] and the oral glucose tolerance test (OGTT). The performance of the DRS in subpopulations of the study defined by race/ethnicity gender, glucose tolerance status, fasting glucose and metabolic syndrome was also examined.
Research design and methods
Participants
The selection of participants and the study design for the IRAS cohort have been previously described [8,10]. The IRAS protocol was approved by local institutional review committees, and all subjects gave informed consent. Men and women aged 40–70 years who were predisposed to have impaired glucose tolerance (IGT) were recruited at four clinical centres in the United States. NHW and African-American participants were recruited at centres in Oakland, California and Los Angeles, California. NHW and Hispanic participants were recruited at centres in San Luis Valley, Colorado and San Antonio, Texas. For this current study, baseline plasma specimens from 722 participants without diabetes were available for testing, including 127 from participants who had converted to diabetes after median follow-up time of 5.2 years (inter-quartile range: 5.1–5.4) on the basis of the 1999 World Health Organization criteria [11].
Clinical measurements and procedures
The initial clinical evaluation consisted of two visits about 1–2 weeks apart. Anthropometric measurements (BMI, blood pressure, waist, etc.), routine laboratory measures (fasting glucose, insulin, triglycerides, cholesterol, high-density lipoprotein (HDL) and low-density lipoprotein (LDL)) and the OGTT were performed as described [10].
Laboratory methods
Plasma glucose and insulin were measured at the time of the IRAS clinical examination. Plasma glucose was measured on an autoanalyzer (Yellow Springs Instruments, Yellow Springs, OH, USA). Plasma insulin was measured using a dextran-charcoal radioimmunoassay [12]. The remaining biomarkers were assayed in fasting plasma specimens from the second examination that had been stored at −80 °C. Ferritin and interleukin 2 receptor α (IL2Ra) were measured using solid-phase, two-site chemiluminescent immunometric assays. C-reactive protein (CRP) was measured using an immuno-turbidometric assay, and adiponectin was measured using a sandwich enzyme-linked immunosorbent assay. The coefficients of variation for ferritin, IL2Ra, CRP and adiponectin were 4.6%, 6.5%, 6.8% and 12.5%, respectively. Missing values were imputed using Harrell's additive-regression imputation function (Hmisc R package, version 3.4-3) [13].
Statistical analysis
All statistical analyses were performed with the R statistical programming language [14], version 2.11.1.
DRS calculation
Development and validation of the DRS have been described in detail [5–7]. Briefly, the DRS uses concentrations of adiponectin, ferritin, CRP, IL2Ra, glucose, insulin and HbA1c measured in fasting blood to estimate 5-year risk of incident type 2 diabetes [5]. DRS was initially developed in a nested case–control study drawn from the Inter99 cohort, a population-based primary prevention study of cardiovascular disease in participants aged 30 to 60 years from Copenhagen County, Denmark [5,15]. The performance of DRS, with HbA1c added, was validated in a clinical laboratory setting by using samples from Inter99 [6]. The robustness of this DRS model was subsequently demonstrated using the Botnia cohort, a family-based prospective study of diabetes [7].
In the present study, DRS was calculated in the IRAS cohort by using the pre-specified model developed in the Inter99 cohort [6], without refit. HbA1c was not measured at baseline in the IRAS, and whole blood was not available for assay. To estimate a value for use in the DRS calculations appropriate for this elevated-risk cohort, a linear model of HbA1c was developed using data from the 2001–2008 National Health and Nutrition Examination Surveys [16], which attempts to survey a nationally representative sample of approximately 5000 Americans each year. The model selected, consisting of age, fasting glucose and race/ethnicity terms, was used to estimate HbA1c values for all IRAS participants. The mean predicted HbA1c value of 5.4% was used in the DRS calculation. Because the same value was used for all participants, HbA1c has no effect on metrics of discrimination in this study. Additional details may be found in the Supporting information.
Baseline characteristics
Analysis of variance (ANOVA) was used to examine whether baseline characteristics of IRAS participants differed by conversion status or race/ethnicity. The Shapiro–Wilks test of normality was used to select a transform for each continuous variable. All variables were log-transformed except for fasting glucose and LDL cholesterol, where the square root was applied, systolic blood pressure, where the reciprocal was used, and age, which was not transformed. A linear model of each transformed variable was fit, adjusted for BMI, age, clinic, gender and either race (when comparing differences by conversion status) or conversion status (when assessing differences by race). Age and gender models were not adjusted for themselves. Significance was assessed at the 0.05 level based on the F-statistic. To determine if racial differences in the DRS represent a miscalibration of true risk, differences in DRS among race and conversion status were tested with an ANOVA with an interaction term between race and conversion status. Although this analysis has substantially less power than the main effects, it is the only direct test of the non-linearity of the DRS between race and conversion status.
Model performance
The DRS was compared with fasting glucose and other measures for ability to discriminate between converters and non-converters by calculating the area under the receiver operating characteristic curve (AROC). Significance of the differences in AROC was calculated using the method of DeLong et al. [17]. A bootstrap resampling technique [18] was used to estimate the significance of differences between the AROC of the cohort as a whole and the AROCs of subpopulations defined by race/ethnicity or gender.
Clinical reclassification
The DRS classified participants as low (DRS <4.5), moderate (4.5–7.9) or high (≥8.0) risk, using established thresholds [6]. To assess how the DRS might impact clinical practice in the United States, we compared risk classification by the DRS with other classification methods that may be commonly used by physicians to evaluate diabetes risk.
In one analysis, we evaluated the ability of DRS to further risk stratify elevated-risk patients identified by IFG (≥100 mg/dL) or metabolic syndrome. Metabolic syndrome as defined by the American Heart Association criteria [19] incorporates information on waist circumference, gender, hypertension, and fasting glucose, triglyceride and HDL cholesterol levels. In each case, the sensitivity and specificity of the two tests to 5-year conversion status were calculated and compared by McNemar's test. Positive and negative predictive values of the two tests were also compared, and the significance of the difference was estimated using a bootstrap resampling technique.
In another analysis, the classification by DRS was compared with classification based on the count of metabolic syndrome risk factors present: 1–2, 3 or 4–5. We defined these three levels in order to calculate the net reclassification improvement (NRI) [20] of the three DRS levels. The metabolic syndrome thresholds were selected so that the number of participants in each level would be similar to the number of participants in the corresponding DRS level. The number of participants and the fraction who converted in each combination of classifications were used to calculate NRI with significance estimated by permutation test.
Finally, we assessed the ability of the DRS to further risk stratify participants following the OGTT. Conversion status and DRS classification were cross-tabulated among participants with normal glucose tolerance (NGT), IGT (≥140 mg/dL), and isolated IFG. Equivalence of conversion rates between DRS strata was assessed by chi-squared test.
Results
Table 1 presents numbers of participants and conversion rates in the IRAS cohort stratified by race/ethnicity and gender, decade of age, fasting glucose status or 2-h glucose status. Overall, 17.6% of IRAS participants in this study had developed diabetes by the 5-year follow-up visit, approximately 4.5 times higher than the rate of 4.0% among adults 20 years and older in the US population as a whole [16,21,22].
Table 1. Numbers of Insulin Resistance Atherosclerosis Study participants, fraction of cohort and 5-year conversion rates stratified by race/ethnicity and gender, decade of age at baseline, fasting glucose status, or 2-h glucose status.
| Numbers of participants in strata (%) by race/ethnicity | Number in stratum who had converted at 5 years (%) | |||||||
|---|---|---|---|---|---|---|---|---|
|
|
|
|||||||
| Stratum | Overall | NHW | Hispanic | African American | Overall | NHW | Hispanic | African American |
| Whole study | 722 | 302 | 249 | 171 | 127 (17.6%) | 54 (17.9%) | 44 (17.7%) | 29 (17.0%) |
| Gender | ||||||||
| Female | 394 (54.6%) | 151 (50.0%) | 146 (58.6%) | 97 (56.7%) | 73 (18.5%) | 29 (19.2%) | 27 (18.5%) | 17 (17.5%) |
| Male | 328 (45.4%) | 151 (50.0%) | 103 (41.4%) | 74 (43.3%) | 54 (16.5%) | 25 (16.6%) | 17 (16.5%) | 12 (16.2%) |
| Age at baseline, years | ||||||||
| 40–49 | 234 (32.4%) | 88 (29.1%) | 86 (34.5%) | 60 (35.1%) | 33 (14.1%) | 13 (14.8%) | 10 (11.6%) | 10 (16.7%) |
| 50–59 | 253 (35.0%) | 105 (34.8%) | 92 (36.9%) | 56 (32.7%) | 43 (17.0%) | 17 (16.2%) | 18 (19.6%) | 8 (14.3%) |
| 60–70 | 235 (32.5%) | 109 (36.1%) | 71 (28.5%) | 55 (32.2%) | 51 (21.7%) | 24 (22%) | 16 (22.5%) | 11 (20.0%) |
| Fasting glucose | ||||||||
| NFG | 525 (72.7%) | 208 (68.9%) | 207 (83.1%) | 110 (64.3%) | 60 (11.4%) | 23 (11.1%) | 28 (13.5%) | 9 (8.2%) |
| IFG | 197 (27.3%) | 94 (31.1%) | 42 (16.9%) | 61 (35.7%) | 67 (34.0%) | 31 (33.0%) | 16 (38.1%) | 20 (32.8%) |
| 2-h glucose | ||||||||
| NGT | 481 (66.6%) | 203 (67.2%) | 165 (66.3%) | 113 (66.1%) | 43 (8.9%) | 18 (8.9%) | 19 (11.5%) | 6 (5.3%) |
| IGT | 241 (33.4%) | 99 (32.8%) | 84 (33.7%) | 58 (33.9%) | 84 (34.9%) | 36 (36.4%) | 25 (29.8%) | 23 (39.7%) |
NHW, non-Hispanic white; NFG, normal fasting glucose; IFG, impaired fasting glucose; NGT, normal glucose tolerance; IGT, impaired glucose tolerance.
Tables 2 and 3 compare baseline characteristics of the IRAS cohort using ANOVA. Table 2 presents unadjusted means and standard deviations by conversion status in order to show the ranges of the biomarkers in this study. Differences between converters and non-converters were assessed after adjusting for BMI, age, clinic, gender and race. All biomarkers in the DRS except CRP and IL2Ra differed significantly between converters and non-converters on a univariate basis; CRP differed significantly when not adjusted for BMI (p < 0.001). Table 3 presents unadjusted means and standard deviations for each ethnic group. The p-values indicate whether the differences were significant after adjusting for BMI, age, clinic, gender and conversion status. All DRS biomarkers except CRP and ferritin differed significantly between racial/ethnic groups, but the differences in DRS were only marginally significant (p = 0.064). An interaction between race and conversion status was tested for each predictor; it was only significant for LDL cholesterol (p = 0.05) and therefore was not included in the models.
Table 2. Baseline characteristics of Insulin Resistance Atherosclerosis Study participants stratified by conversion status.
| Diabetes conversion status | |||
|---|---|---|---|
|
|
|||
| Baseline variable | NC | C | p-value |
| Number of participants (N) | 595 | 127 | |
| Age (years) | 55 (46–63) | 57 (49–65) | 0.0132 |
| Fasting glucose (mg/dL) | 92 (83–102) | 100 (90–112) | <0.0001 |
| Fasting insulin (pmol/L) | 70 (38–129) | 108 (55–211) | <0.0001 |
| Adiponectin (mg/mL) | 7.6 (4.9–11.7) | 6.3 (4.1–9.6) | <0.0001 |
| Ferritin (ng/mL) | 95 (33–275) | 134 (56–323) | 0.0078 |
| Interleukin receptor 2 alpha (u/mL) | 321 (215–479) | 338 (223–512) | 0.7543 |
| C-reactive protein (mg/L) | 1.8 (0.6–5.6) | 2.7 (1.0–7.3) | 0.4674 |
| Diabetes Risk Score | 4.5 (1.7–7.7) | 7.7 (4.8–9.3) | <0.0001 |
| BMI (kg/m2) | 27 (23–33) | 31 (25–38) | <0.0001 |
| Waist circumference (cm) | 88 (77–101) | 95 (83–109) | 0.3099 |
| Systolic blood pressure (mm Hg) | 119 (105–136) | 124 (108–145) | 0.1365 |
| Diastolic blood pressure (mm Hg) | 77 (69–87) | 78 (69–89) | 0.5325 |
| Total cholesterol (mmol/L) | 5.4 (4.4–6.5) | 5.4 (4.5–6.5) | 0.905 |
| HDL cholesterol (mmol/L) | 1.2 (0.9–1.6) | 1.1 (0.8–1.4) | 0.0094 |
| LDL cholesterol (mmol/L) | 3.6 (2.8–4.5) | 3.6 (2.7–4.7) | 0.9728 |
| Triglycerides (mmol/L) | 1.2 (0.7–2.2) | 1.5 (0.9–2.5) | 0.0085 |
| 2-h glucose (mg/dL) | 115 (88–152) | 149 (118–188) | <0.0001 |
Unadjusted means (±1 standard deviation) were calculated on the transformed variablesa and back-transformed. The p-value tests the null hypothesis that there is no significant difference between converters (C) and non-converters (NC) after adjusting for body mass index (BMI), age, clinic, gender and race.
All variables were log-transformed except for fasting glucose and LDL cholesterol, where the square root was applied, systolic blood pressure, where the reciprocal was used, the DRS, which was logit-transformed, and age, which was not transformed.
Table 3. Differences in baseline characteristics of Insulin Resistance Atherosclerosis Study participants stratified by race/ethnicity.
| Racial/ethnic group | ||||
|---|---|---|---|---|
|
|
||||
| Baseline variable | NHW | Hispanic | African American | p-value |
| Number of participants (N) | 302 | 249 | 171 | |
| Age (years) | 56 (47–64) | 54 (46–63) | 55 (46–63) | 0.0607 |
| Fasting glucose (mg/dL) | 94 (83–106) | 91 (82–101) | 96 (86–107) | 0.0452 |
| Fasting insulin (pmol/L) | 67 (35–128) | 81 (42–159) | 81 (45–147) | 0.0078 |
| Adiponectin (mg/mL) | 7.9 (5.2–12.2) | 7.2 (4.7–11.2) | 6.5 (4.3–10) | <0.0001 |
| Ferritin (ng/mL) | 99 (36–272) | 99 (33–298) | 107 (39–293) | 0.1199 |
| Interleukin receptor 2 alpha (u/mL) | 350 (235–520) | 335 (227–493) | 270 (184–395) | <0.0001 |
| C-reactive protein (mg/L) | 1.7 (0.5–5.3) | 2.4 (0.9–6.5) | 1.9 (0.6–5.8) | 0.1779 |
| Diabetes Risk Score | 5.0 (1.8–8.2) | 4.9 (1.7–8.1) | 5.8 (2.6–8.5) | 0.0642 |
| BMI (kg/m2) | 27 (23–33) | 28 (23–34) | 29 (24–34) | 0.0363 |
| Waist circumference (cm) | 90 (78–103) | 89 (78–103) | 90 (78–103) | 0.0242 |
| Systolic blood pressure (mm Hg) | 119 (105–137) | 118 (103–136) | 123 (109–142) | 0.0198 |
| Diastolic blood pressure (mm Hg) | 76 (68–86) | 78 (69–88) | 78 (69–89) | 0.0242 |
| Total cholesterol (mmol/L) | 5.3 (4.5–6.4) | 5.3 (4.3–6.6) | 5.5 (4.7–6.6) | 0.2188 |
| HDL cholesterol (mmol/L) | 1.1 (0.8–1.6) | 1.1 (0.8–1.5) | 1.3 (1.0–1.7) | 0.0038 |
| LDL cholesterol (mmol/L) | 3.6 (2.8–4.5) | 3.5 (2.6–4.5) | 3.8 (3.0–4.7) | 0.0148 |
| Triglycerides (mmol/L) | 1.3 (0.8–2.2) | 1.5 (0.9–2.6) | 1.0 (0.6–1.7) | <0.0001 |
| 2-h glucose (mg/dL) | 120 (91–158) | 120 (89–162) | 122 (93–161) | 0.9693 |
Unadjusted means (±1 standard deviation) for each racial group on the transformed variablesa. The p-value indicates the probability that no significant difference exists between groups after adjusting for age, gender, clinic, body mass index (BMI) and conversion status. NHW, non-Hispanic white.
All variables were log-transformed except for fasting glucose and LDL cholesterol, where the square root was applied, systolic blood pressure, where the reciprocal was used, the DRS, which was logit-transformed, and age, which was not transformed.
Performance of the DRS in various subpopulations of the IRAS cohort is shown in Figure 1. No significant difference in discrimination was observed between any of the individual racial groups or between genders. In the whole cohort (n=722), the DRS had a significantly higher AROC than fasting glucose (0.763 versus 0.711; p=0.003), fasting insulin (0.690; p= 0.003), HOMA-IR (0.716; p=0.03) and BMI (0.671; p< 0.001), and was statistically equivalent to 2-h glucose (0.770; p=0.8). Among elevated-risk participants who had IFG or the metabolic syndrome at baseline (n = 283), the DRS had a significantly higher AROC than fasting glucose (0.739 versus 0.662; p=0.01). This was also the case among the 197 participants with IFG independent of the metabolic syndrome (0.738 versus 0.658; p = 0.03).
Figure 1.
Discrimination of the Diabetes Risk Score between converters and non-converters in the subpopulations of the Insulin Resistance Atherosclerosis Study cohort defined by race and gender, compared with discrimination in the cohort as a whole, by AROC. The 95% CIs on each AROC value, and p-values were estimated by bootstrap resampling. The vertical lines indicate the estimated AROC and its 95% CIs for the entire study. The p-values indicate the likelihood that the differences between each subgroup and the study as a whole would be observed by chance. NHW, non-Hispanic white
Classification of elevated-risk subpopulations of the IRAS cohort by the DRS was also compared with other classification methods potentially used by physicians to identify patients at the highest risk of developing type 2 diabetes.
The DRS was applied to 283 participants who had IFG and/or metabolic syndrome. In this subpopulation, we compared the sensitivity, specificity, positive predictive value, and negative predictive value for ‘high risk’ DRS versus the combination of IFG and metabolic syndrome. Results showed that the DRS classification had a higher sensitivity (0.71 versus 0.53; p=0.003), positive predictive value (0.48 vs 0.40; p = 0.02) and negative predictive value (0.84 vs 0.77; p=0.002), and equivalent specificity (0.68 versus 0.66; p= 0.6).
Table 4 further explores the potential value of the DRS in clinical practice by comparing how participants with IFG or metabolic syndrome are classified by the DRS versus counting of metabolic syndrome risk factors. In this analysis, previously developed cut points were used to define low, moderate and high risk DRS [6]. An NRI of 0.24 indicates that the DRS provided a significant improvement in classification of participants relative to risk factor counting (p=0.01). Among participants without metabolic syndrome (fewer than three risk factors), the DRS reclassified 85.4% of participants; of the 31 reclassified as high risk, 48.4% converted to type 2 diabetes. Conversely, the DRS reclassified five participants as low risk who had four or five metabolic syndrome risk factors present at baseline; none of whom had converted after 5 years.
Table 4. Risk reclassification of 283 Insulin Resistance Atherosclerosis Study participants with impaired fasting glucose and/or the metabolic syndrome by Diabetes Risk Score (DRS) compared with classification by number of metabolic syndrome risk factors present.
| Risk group by number of metabolic syndrome risk factors present | Risk group by DRS | No. reclassified [%] | |||
|---|---|---|---|---|---|
|
| |||||
| Low (<4.5) | Moderate (4.5–7.9) | High (≥8.0) | Any DRS | ||
| 1–2 risk factors | |||||
| Number of observations | 12 | 39 | 31 | 82 | 70 |
| Fraction of population | 4.2% | 13.8% | 11.0% | 29.0% | 85.4% |
| Number converted at 5 years | 1 | 6 | 15 | 22 | |
| Fraction converted at 5 years | 8.3% | 15.4% | 48.4% | 26.8% | |
| 3 risk factors | |||||
| Number of observations | 27 | 59 | 48 | 134 | 75 |
| Fraction of population | 9.5% | 20.8% | 17.0% | 47.3% | 56.0% |
| Number converted at 5 years | 2 | 13 | 19 | 34 | |
| Fraction converted at 5 years | 7.4% | 22.0% | 39.6% | 25.4% | |
| 4–5 risk factors | |||||
| Number of observations | 5 | 17 | 45 | 67 | 22 |
| Fraction of population | 1.8% | 6.0% | 15.9% | 23.7% | 32.8% |
| Number converted at 5 years | 0 | 3 | 26 | 29 | |
| Fraction converted at 5 years | 0% | 19.1% | 57.8% | 43.3% | |
| Any number of risk factors | |||||
| Number of observations | 44 | 115 | 124 | 283 | 167 |
| Fraction of population | 15.5% | 40.6% | 43.8% | 100% | 59.0% |
| Number converted at 5 years | 3 | 22 | 60 | 85 | |
| Fraction converted at 5 years | 6.8% | 19.1% | 48.4% | 30.0% | |
A significant net reclassification improvement of 0.24 was observed (p =0.01).
Table 5 assesses the ability of the DRS to further risk stratify patients who have already been classified by the results of an OGTT. Both in the subpopulation whose glucose tolerance was normal and the IGT subpopulation, the DRS identified substantial fractions as high risk that did in fact have a significantly higher 5-year incidence of diabetes than the rest of each subpopulation. This was also true among participants who had isolated IFG.
Table 5. Further risk stratification of patients by Diabetes Risk Score (DRS) after an oral glucose tolerance test (OGTT).
| DRS | Participants | Non-converters | Converters | 5-year risk | RR (95% CI) | p-value |
|---|---|---|---|---|---|---|
| Participants with NGT (N=481) | <0.0001 | |||||
| Low-to-moderate risk | 436 | 406 | 30 | 6.9% | 4.2 | |
| High risk | 45 | 32 | 13 | 28.9% | (2.4–7.5) | |
| Participants with isolated IFG (N=97) | 0.002 | |||||
| Low-to-moderate risk | 61 | 56 | 5 | 8.2% | 4.1 | |
| High risk | 36 | 24 | 12 | 33.3% | (1.6–10.6) | |
| Participants with IGT (N=241) | <0.0001 | |||||
| Low-to-moderate risk | 140 | 109 | 31 | 22.1% | 2.4 | |
| High risk | 101 | 48 | 53 | 52.5% | (1.7–3.4) | |
Risk of diabetes conversion is compared between participants classified as high or low-to-moderate risk by the DRS in subpopulations defined by results of the OGTT. The relative risk (RR: DRS high versus the rest of the subpopulation), its 95% CI, and the p-value (χ2 test) are also reported.
NGT, normal glucose tolerance; IFG, impaired fasting glucose; IGT, impaired glucose tolerance; RR, relative risk.
Conclusions
This study demonstrated that the DRS may be used to effectively evaluate a patient's risk of developing diabetes in an elevated-risk, multi-ethnic US cohort. The DRS provided better discrimination of converters from non-converters than other commonly used risk assessment methods, such as fasting glucose, fasting insulin, BMI and HOMA-IR. The AROC of DRS was lower in IRAS (0.763) than was observed in the Northern European Inter99 study population (0.837) in which it was developed [6]. However, the 5-year conversion rate in IRAS (17.6%) was about fivefold higher than in Inter99 (3.4%). The greater conversion rate in IRAS is due to the sampling strategy, which emphasized recruitment of persons with IGT. The skewed distribution of glucose values in the IRAS population thus made discrimination more difficult. However, the DRS had a significantly higher AROC than fasting glucose in both IRAS (by 0.05) and Inter99 (by 0.07). The performance of the DRS relative to fasting glucose was also superior among elevated-risk participants as defined by metabolic syndrome and/or IFG. These results provide confidence that the earlier work translates to higher risk populations beyond those in which the DRS was developed.
In the United States, age-adjusted and gender-adjusted prevalence of type 2 diabetes is higher among African Americans (18.7%) and Hispanics (20.1%) than among Caucasians (11.0%) [23], emphasizing the need for accurate risk assessment tools in these subpopulations. However, in IRAS, rates of conversion were strikingly similar between these subgroups across various strata (Table 1). This likely reflects an enrollment strategy that resulted in recruitment of individuals with similar risk profiles independent of race. Conversely, ANOVA identified significant differences in concentrations of individual biomarkers between ethnic groups (Table 3). Despite this, all the DRS markers, with the exception of IL2Ra, were predictive of risk on a univariate basis when adjusted for the racial differences. No significant interaction was found between racial/ethnic group and conversion status for the DRS markers.
As shown in Figure 1, no significant differences in AROC were observed between any of the racial/ethnic subpopulations of IRAS and the cohort as a whole. This is an important result because the DRS was developed in an entirely Caucasian population. This also demonstrates an advantage of using a multivariate approach to risk estimation. Although the individual biomarkers were predictive of diabetes conversion, concentrations differed significantly by race even after adjusting for age, gender, BMI and conversion status, complicating their interpretation. This study shows that besides providing better discrimination than any of the markers individually, DRS integrates the information from multiple markers in such a way that the differences between racial groups are less significant. One shortcoming of the present study is that it does not include Asian Americans or Native Americans.
The lack of HbA1c measurements in IRAS precludes assessment of this biomarker's contribution. The method used to impute a HbA1c value is conservative and has no impact on discrimination. HbA1c improved risk assessment in the Kansai Healthcare Study [24] compared with glucose alone. We expect this biomarker will also improve the performance of DRS, but this must be confirmed in other studies. Additionally, recent reports have cited differences in HbA1c on the basis of ethnicity [25], which could impact the AROC when HbA1c is included. Because HbA1c was not measured, we were also unable to measure the performance of the DRS for assessing risk of diabetes as defined by the 2010 American Diabetes Association diagnostic criteria [26].
We have also demonstrated that the DRS is a more accurate tool for classifying patients by diabetes risk. As much as one third of the US population has pre-diabetes, depending on the definition used [23,27]. Physicians need better tools to identify those at highest risk so that limited diabetes prevention resources may be allocated efficiently. It is envisioned that the DRS would be used with patients considered at risk by common tools, such as fasting glucose, HbA1c or metabolic syndrome. Although cost–benefit analysis of such a strategy is beyond the scope of this article, it is explored in another recent publication [28], which showed that use of DRS to further risk stratify ‘at risk’ patients was cost-effective.
We report that the DRS provides better risk stratification among participants with metabolic syndrome or IFG than high risk classification on the basis of a combination of the two. Moreover, significant NRI was observed using pre-specified criteria for DRS low, moderate and high risk classifications versus classification based on counting of metabolic syndrome risk factors present, such as recommended in the Endocrine Society guidelines [29]. The choice of cut points for DRS can affect the NRI. There are different ways in which cut points could be selected. In the development of the DRS, a cut point to define ‘high’ risk was chosen, which put approximately 10% of the population in a ‘high’ risk category with a risk 3–4 times higher than the overall risk of the population. The cut point for ‘low’ risk was chosen such that the risk was 3–4 times lower than the population risk. In a recent publication [30], cut points for DRS were chosen to match the population fraction defined by the number of metabolic syndrome risk factors. Again, DRS showed a significant NRI compared with metabolic risk factor counting in a population with either metabolic syndrome or IFG. Thus, the DRS is complementary to routine screening methods that use either standard lab tests or clinical risk factors commonly used by physicians when making treatment decisions. The IRAS cohort provided an excellent opportunity to study these subpopulations because its enrollment criteria enriched for participants at higher risk than the US population as a whole.
The ability of the OGTT to discriminate between participants by 5-year conversion status was equivalent to the DRS. However, the shortcomings of the OGTT are well known [31]. In non-diabetic patients, OGTT is most commonly used to diagnose IGT, using a single threshold as defined by American Diabetes Association guidelines [11,26]. We have shown that when used in this manner, the OGTT is complemented by the DRS. The DRS identifies high risk participants among both NGT and IGT subgroups who are significantly more likely to develop diabetes. Most importantly, the DRS identified as high risk nearly 40% of participants who had IFG but NGT; one third of these participants converted within 5 years. These are patients who might not receive the level of intervention warranted by their risk in the absence of this additional information.
In summary, we have demonstrated in a multi-ethnic US cohort that the DRS meets an important clinical need for improved diabetes risk assessment tools. It is convenient for physicians, as it may currently be ordered as a laboratory test on a fasting blood sample, and does not require them to enter anthropometric, demographic or patient history information. It complements routine diabetes risk screening tools, including the OGTT, by further stratifying elevated-risk subpopulations where the need for such tools is greatest, and is effective in non-Caucasian subpopulations.
Supplementary Material
Acknowledgments
Parts of this study were presented as an abstract and oral presentation at the 6th World Congress on Prevention of Diabetes and its Complications, Dresden, Germany, 8–11 April 2010.
The authors gratefully acknowledge the excellent statistical review provided by Uyen Thao Nguyen and the skillful technical work of Amalia Ocampo, Marianne Weinell and Claudia Elkin, Tethys Bioscience. The authors acknowledge the work of Andrea Anderson, Wake Forest University, in assisting in the development of IRAS databases. The authors would also like to thank Linda Wuestehube for editorial assistance.
Footnotes
Conflict of interest: Authors Michael W. Rowe and Janice A. Kolberg are employees and shareholders of Tethys Bioscience, Inc., a biomedical company dedicated to the discovery, development and commercialization of novel biological markers that provide practical tools to address the growing global challenge of chronic metabolic diseases such as diabetes. Tethys Bioscience developed and markets the DRS, upon which this article is based. Richard N. Bergman is Chair of the Scientific Advisory Board of Tethys Bioscience and has a financial interest in the Company. Lynne Wagenknecht is a consultant to Tethys Bioscience.
Supporting information: Supporting information may be found in the online version of this article.
References
- 1.Gerstein HC, Yusuf S, Bosch J, et al. Effect of rosiglitazone on the frequency of diabetes in patients with impaired glucose tolerance or impaired fasting glucose: a randomised controlled trial. Lancet. 2006;368:1096–1105. doi: 10.1016/S0140-6736(06)69420-8. [DOI] [PubMed] [Google Scholar]
- 2.Knowler WC, Barrett-Connor E, Fowler SE, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002;346:393–403. doi: 10.1056/NEJMoa012512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Lindstrom J, Peltonen M, Eriksson JG, et al. Determinants for the effectiveness of lifestyle intervention in the Finnish Diabetes Prevention Study. Diabetes Care. 2008;31:857–862. doi: 10.2337/dc07-2162. [DOI] [PubMed] [Google Scholar]
- 4.Tuomilehto J, Lindstrom J, Eriksson JG, et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med. 2001;344:1343–1350. doi: 10.1056/NEJM200105033441801. [DOI] [PubMed] [Google Scholar]
- 5.Kolberg JA, Jorgensen T, Gerwien RW, et al. Development of a type 2 diabetes risk model from a panel of serum biomarkers from the Inter99 cohort. Diabetes Care. 2009;32:1207–1212. doi: 10.2337/dc08-1935. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Urdea M, Kolberg J, Wilber J, et al. Validation of a multimarker model for assessing risk of type 2 diabetes from a five-year prospective study of 6784 Danish people (Inter99) J Diabetes Sci Technol. 2009;3:748–755. doi: 10.1177/193229680900300422. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lyssenko V, Jorgensen T, Gerwein RW, et al. Validation of a multi-marker model for the prediction of incident type 2 diabetes mellitus: combined results of the Inter99 and Botnia studies. Diab Vasc Dis Res. 2012;9:59–67. doi: 10.1177/1479164111424762. [DOI] [PubMed] [Google Scholar]
- 8.Wagenknecht LE, Mayer EJ, Rewers M, et al. The insulin resistance atherosclerosis study (IRAS) objectives, design, and recruitment results. Ann Epidemiol. 1995;5:464–472. doi: 10.1016/1047-2797(95)00062-3. [DOI] [PubMed] [Google Scholar]
- 9.Wallace TM, Levy JC, Matthews DR. Use and abuse of HOMA modeling. Diabetes Care. 2004;27:1487–1495. doi: 10.2337/diacare.27.6.1487. [DOI] [PubMed] [Google Scholar]
- 10.Haffner SM, D'Agostino R, Saad MF, et al. Increased insulin resistance and insulin secretion in nondiabetic African-Americans and Hispanics compared with non-Hispanic whites. The Insulin Resistance Atherosclerosis Study. Diabetes. 1996;45:742–748. doi: 10.2337/diab.45.6.742. [DOI] [PubMed] [Google Scholar]
- 11.World Health Organization. Definition, Diagnosis and Classification of Diabetes Mellitus and Its Complications: Part 1: Report of a WHO Consultation: Diagnosis and Classification of Diabetes Mellitus. World Health Organization; Geneva: 1999. [Google Scholar]
- 12.Herbert V, Lau KS, Gottlieb CW, Bleicher SJ. Coated charcoal immunoassay of insulin. J Clin Endocrinol Metab. 1965;25:1375–1384. doi: 10.1210/jcem-25-10-1375. [DOI] [PubMed] [Google Scholar]
- 13.Harrell FE. Hmisc S function library. [Accessed 1 November 2010];2004 Programs available from http://biostat.mc.vanderbilt.edu/s/Hmisc.
- 14.R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; Vienna, Austria: 2010. [Google Scholar]
- 15.Jorgensen T, Borch-Johnsen K, Thomsen TF, Ibsen H, Glumer C, Pisinger C. A randomized non-pharmacological intervention study for prevention of ischaemic heart disease: baseline results Inter99. Eur J Cardiovasc Prev Rehabil. 2003;10:377–386. doi: 10.1097/01.hjr.0000096541.30533.82. [DOI] [PubMed] [Google Scholar]
- 16.Centers for Disease Control and Prevention (CDC) National Center for Health Statistics (NCHS) National Health and Nutrition Examination Survey Data. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention; Hyattsville, MD: 2001–2008. [Accessed 9 March 2010]. http://www.cdc.gov/nchs/nhanes/ [Google Scholar]
- 17.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–845. [PubMed] [Google Scholar]
- 18.Efron B, Tibshirani RJ. An Introduction to the Bootstrap. Chapman and Hall/CRC; Boca Raton, FL: 1993. [Google Scholar]
- 19.Alberti KG, Eckel RH, Grundy SM, et al. Harmonizing the metabolic syndrome: a joint interim statement of the International Diabetes Federation Task Force on Epidemiology and Prevention; National Heart, Lung, and Blood Institute; American Heart Association; World Heart Federation; International Atherosclerosis Society; and International Association for the Study of Obesity. Circulation. 2009;120:1640–1645. doi: 10.1161/CIRCULATIONAHA.109.192644. [DOI] [PubMed] [Google Scholar]
- 20.Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150:795–802. doi: 10.7326/0003-4819-150-11-200906020-00007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Centers for Disease Control and Prevention. National Diabetes Fact Sheet: National Estimates and General Information on Diabetes and Prediabetes in the United States 2011. U.S. Dept. of Health and Human Services, Centers for Disease Control and Prevention; Atlanta, GA: 2011. [Google Scholar]
- 22.U.S. Census Bureau. Statistical Abstract of the United States: 2012. U.S. Dept. of Commerce, U.S. Census Bureau; Washington, DC: 2012. [Google Scholar]
- 23.Cowie CC, Rust KF, Ford ES, et al. Full accounting of diabetes and pre-diabetes in the U.S. population in 1988–1994 and 2005–2006. Diabetes Care. 2009;32:287–294. doi: 10.2337/dc08-1296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Sato KK, Hayashi T, Harita N, et al. Combined measurement of fasting plasma glucose and A1C is effective for the prediction of type 2 diabetes. Diabetes Care. 2009;32:644–646. doi: 10.2337/dc08-1631. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bloomgarden ZT. A1C: recommendations, debates, and questions. Diabetes Care. 2009;32:e141–e147. doi: 10.2337/dc09-zb12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.American Diabetes Association. Diagnosis and classification of diabetes mellitus. Diabetes Care. 2010;33(Suppl 1):S62–S69. doi: 10.2337/dc10-S062. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.James C, Bullard KM, Rolka DB, et al. Implications of alternative definitions of prediabetes for prevalence in U.S. adults. Diabetes Care. 2011;34:387–391. doi: 10.2337/dc10-1314. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Sullivan SD, Garrison PG, Jr, Rinde H, Kolberg J, Moler EJ. Cost-effectiveness of risk stratification for preventing type 2 diabetes using a multi-marker score. J Med Econ. 2011;14:609–616. doi: 10.3111/13696998.2011.602160. [DOI] [PubMed] [Google Scholar]
- 29.Rosenzweig JL, Ferrannini E, Grundy SM, et al. Primary prevention of cardiovascular disease and type 2 diabetes in patients at metabolic risk: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2008;93:3671–3689. doi: 10.1210/jc.2008-0222. [DOI] [PubMed] [Google Scholar]
- 30.Shafizadeh TB, Moler EJ, Kolberg JA, et al. Comparison of accuracy of diabetes risk score and components of the metabolic syndrome in assessing risk of incident type 2 diabetes in inter99 cohort. PLoS One. 2011;6:e22863. doi: 10.1371/journal.pone.0022863. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Davidson MB. Counterpoint: the oral glucose tolerance test is superfluous. Diabetes Care. 2002;25:1883–1885. doi: 10.2337/diacare.25.10.1883. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.