Prognostic Performance of Metabolic Indexes in Predicting Onset of Type 1 Diabetes

Abstract

OBJECTIVE

In this investigation we evaluated nine metabolic indexes from intravenous glucose tolerance tests (IVGTTs) and oral glucose tolerance tests (OGTTs) in an effort to determine their prognostic performance in predicting the development of type 1 diabetes in those with moderate risk, as defined by familial relation to a type 1 diabetic individual, a positive test for islet cell antibodies and insulin autoantibody, but normal glucose tolerance.

RESEARCH DESIGN AND METHODS

Subjects (n = 186) who had a projected risk of 25–50% for developing type 1 diabetes within 5 years were followed until clinical diabetes onset or the end of the study as part of the Diabetes Prevention Trial–Type 1. Prognostic performance of the metabolic indexes was determined using receiver operating characteristic (ROC) curve and survival analyses.

RESULTS

Two-hour glucose from an OGTT most accurately predicted progression to disease compared with all other metabolic indicators with an area under the ROC curve of 0.67 (95% CI 0.59–0.76), closely followed by the ratio of first-phase insulin response (FPIR) to homeostasis model assessment of insulin resistance (HOMA-IR) with an area under the curve value of 0.66. The optimal cutoff value for 2-h glucose (114 mg/dl) maintained sensitivity and specificity values >0.60. The hazard ratio for those with 2-h glucose ≥114 mg/dl compared with those with 2-h glucose <114 mg/dl was 2.96 (1.67–5.22).

CONCLUSIONS

The ratio of FPIR to HOMA-IR from an IVGTT provided accuracy in predicting the development of type 1 diabetes similar to that of 2-h glucose from an OGTT, which, because of its lower cost, is preferred. The optimal cutoff value determined for 2-h glucose provides additional guidance for clinicians to identify subjects for potential prevention treatments before the onset of impaired glucose tolerance.

Early disease prediction and prevention are some of the most important strategies in health care. Preventative care can substantially decrease mortality and morbidity and significantly reduce public health costs (1,2). As genetic/familial factors and autoimmune factors have become available to screen subjects for the risk of developing type 1 diabetes, early intervention trials for this disease have become a reality (3–7). The characterization of this risk may be refined by additional factors to more precisely target individuals who would benefit from preventative treatment. To most accurately select individuals who are at risk for developing disease, beyond screening for antibodies and genetic factors, metabolic risk indicators are being investigated for the development of a more effective clinical prognostic index (8–12).

The principal metabolic indexes currently being evaluated as prognostic indicators for type 1 diabetes have been focused on measurements from oral glucose tolerance tests (OGTTs) and intravenous glucose tolerance tests (IVGTTs). Previous research from the Diabetes Prevention Trial–Type 1 (DPT-1) has indicated that some metabolic indexes derived from an OGTT provide substantial predictive value in receiver operating characteristic (ROC) area under the curve (AUC) analysis (13). IVGTT-derived indexes, such as first-phase insulin response (FPIR), homeostasis model assessment of insulin resistance (HOMA-IR), and FPIR-to–HOMA-IR ratio have also demonstrated prognostic value (14–16). However, indexes from both methods have not been compared for predictive accuracy in moderate-risk subjects who are antibody-positive and have genetic risk factors but do not have impaired glucose tolerance. Because subjects in this population who would develop disease are in an early stage of disease progression, they are an important subgroup to target for preventative intervention. If OGTT or IVGTT measurements produce superior predictive indexes compared with each other, costs in future trials can be reduced by relying on a single method of measurement that produces the greatest predictive accuracy.

In addition to determining the superior testing method for producing predictive indexes (OGTT vs. IVGTT), there also remains a need to produce effective prognostic thresholds to select between individuals who will progress to disease and who will not, because screening tests for familial, genetic, and immunoglobin risk factors are not precise enough to accurately select subjects, particularly those at an early stage of disease progression who do not exhibit impaired glucose tolerance. Future intervention trials will depend on a refined selection tool to choose subjects for early intervention to ensure an accurate characterization of treatment effects. Optimal cutoff values derived from ROC AUC analysis from metabolic indexes would provide valuable guidance for clinicians and researchers in evaluating patient risk for progressing to type 1 diabetes by providing a threshold, above which the risk is characterized with greater precision than is provided by their underlying risk factors.

In this investigation, we assessed the prognostic accuracy of nine metabolic indexes for predicting the progression to clinical onset of type 1 diabetes over a 5-year period using the data from DPT-1. The optimal cutoff values of metabolic indexes were determined to provide previously unavailable guidance to clinicians and researchers in selecting patients likely to progress to disease, who are therefore candidates for early preventative intervention.

RESEARCH DESIGN AND METHODS

DPT-1 was a longitudinal study in North America, which was designed to determine whether type 1 diabetes can be prevented or delayed by preclinical intervention of an oral insulin supplement. DPT-1 screened for islet cell antibody (ICA) positivity in 103,390 first- and second-degree nondiabetic relatives of individuals in whom type 1 diabetes had been diagnosed before the age of 45 years. The 3,483 relatives positive for ICAs were staged to quantify the projected 5-year risk of diabetes (17). Staging consisted of ICA confirmation, HLA-DQ typing, determination of insulin autoantibody (IAA), an IVGTT, and an OGTT.

A total of 372 subjects whose 5-year risk was considered to be 25–50%, without metabolic abnormality or loss of the FPIR (defined as moderate risk), were entered into the DPT-1 oral insulin trial. To study the natural history of the disease, the current investigation evaluated 186 subjects who were randomly assigned to the placebo arm of the study. All subjects were examined every 6 months from enrollment until diabetes onset or study end after randomization. All subjects (and/or their parent) signed a written consent form approved by the participating study center's human subjects committee.

Laboratory measurements

An IVGTT was performed after an overnight fast. The IVGTT was done as recommended by the ICARUS Study Group. This recommendation includes instructions for a diet containing at least 150 g of carbohydrate/day for the 3 days before the test. Blood samples for determination of glucose and insulin levels were drawn at −10 and −4 min. A solution of 25% dextrose (0.5 g/kg body wt up to a maximum of 35 g) was then infused over 3 min (±15 s). Repeat blood samples for determination of glucose and insulin levels were drawn at 1, 3, 5, 7, and 10 min after the glucose infusion. FPIR was calculated as the sum of the serum insulin concentrations at 1 and 3 min after intravenous injection of glucose. FPIR was above threshold if ≥10th percentile for siblings, offspring, and second-degree relatives (≥100 μU/ml if aged ≥8 years and ≥60 μU/ml if aged <8 years) and ≥1st percentile for parents (≥60 μU/ml). These thresholds were determined from the Gainesville, Florida, Family Study and a general school population study (18). FPIR above threshold was required for eligibility. HOMA-IR was calculated as the fasting insulin (milliunits per liter) × fasting glucose (millimoles per liter)/22.5 from the mean of fasting insulin at 4 and 10 min and fasting glucose at 4 min before each IVGTT was performed.

OGTTs were administered to assess glycemic status. The dose of oral glucose was 1.75 g/kg (maximum, 75 g of carbohydrate). Blood samples were obtained for C-peptide measurements in the fasting state and at then 30, 60, 90, and 120 min after oral glucose. Peak C-peptide was the maximum point of all measurements. The AUC C-peptide was calculated using the trapezoid rule. A normal OGTT during staging was required for eligibility.

Diabetes was diagnosed according to the American Diabetes Association criteria: fasting glucose ≥126 mg/dl; 2-h glucose ≥200 mg/dl with confirmation by either an elevated fasting or 2-h glucose level at a follow-up visit; or random plasma glucose ≥200 mg/dl accompanied by symptoms of polyuria, polydipsia, and/or weight loss (19).

Statistical methods

For baseline characteristics, categorical variables were compared by a Pearson χ² test, and continuous variables were evaluated by t test for the differences in means or by the Wilcoxon rank-sum test for differences in order between progressors and nonprogressors. ROC curves were used to compare the discriminative power of nine different metabolic indexes to predict progression to type 1 diabetes. The global performance of each measurement in predicting progression to type 1 diabetes was summarized by the AUC, and results are presented as the mean (95% CI). CIs that exclude 0.5 were considered to indicate significant results (20). ROC AUCs were compared using the algorithm suggested by Delong et al. (21), which is a nonparametric approach to the analysis of areas under correlated ROC curves by using the theory on generalized U-statistics to generate an estimated covariance matrix. The ROC curve is constructed by varying the cut point used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false-positive rates (1 − specificity). The optimal cutoff points are the values yielding maximum sums of sensitivity and specificity from the ROC curves. The Cox proportional hazards model was used to calculate the hazard ratio. The log-rank test was used for survival curves comparison. Analyses were performed by using SAS (version 9.2; SAS Institute, Cary, NC) software.

RESULTS

A summary of clinical and metabolic characteristics comparing those who progressed to clinical disease onset and those who did not is shown in Table 1. By design, subjects were ICA⁺ and IAA⁺ with a normal FPIR and normal glucose tolerance (n = 186), giving them a projected risk of 25−50% for progression to clinical diabetes over 5 years. The actual risk was 35% over 5 years. The subjects were followed for a median of 1,213 days (3.3 years; interquartile range 726–1,718). Annual rate of loss to follow-up was 0.2%. The subjects who were lost to follow-up before the end of the study were considered to be nonprogressors. The progressors and nonprogressors did not significantly differ in age, sex, race, or relationship to the proband.

Table 1.

Clinical and metabolic characteristics of the study subjects

Characteristic	Progressor	Nonprogressor	P value
n	53 (28)	133 (72)
Age (years)	9.78 ± 6.35	12.79 ± 9.23	0.67
BMI z score*	−0.67 (−1.56 to 0.53)	−1.10 (−2.61 to 0.33)	0.07
Race			0.64
White	47 (88.68)	116 (89.23)
African American	0 (0.00)	2 (1.54)
Hispanic	4 (7.55)	10 (7.69)
Other	2 (3.77)	2 (1.54)
Sex			0.53
Male	28 (52.83)	77 (57.89)
Female	25 (47.17)	56 (42.11)
Relationship to patient with diabetes			0.09
Sibling	36 (67.92)	72 (54.14)
Offspring	15 (28.30)	38 (28.57)
Parent	1 (1.89)	6 (4.51)
Second-degree	1 (1.89)	17 (12.78)
Immunological factors
ICA titer (JDF units)	160.00 (80.00–320.00)	80.00 (20.00–160.00)	0.001
IAA titer (nU/ml)	385.10 (125.40–672.00)	156.70 (73.70–343.00)	0.001
ICA512 antibodies			0.01
Positive	32 (64.00)	50 (42.37)
Negative	18 (36.00)	68 (57.63)
GAD antibodies			0.674
Positive	38 (76.00)	86 (72.88)
Negative	12 (24.00)	32 (27.12)
Metabolic factors
IVGTT
Fasting glucose (mmol/l)	87.66 ± 9.54	87.12 ± 9.18	0.67
Fasting insulin (mU/l)	17.02 ± 10.07	14.84 ± 9.50	0.17
FPIR (μl/ml)	145.22 ± 80.49	163.90 ± 105.84	0.2
HOMA-R	3.79 ± 2.69	3.25 ± 2.20	0.16
FPIR-to–HOMA-IR ratio	44.28 ± 19.20	59.93 ± 36.58	0.001
OGTT
Fasting glucose (mg/dl)	86.26 ± 7.69	86.18 ± 7.85	0.95
2-h glucose (mg/dl)	113.25 ± 18.71	102.75 ± 19.17	0.001
Peak C-peptide (nmol/l)	5.09 ± 1.98	5.59 ± 2.26	0.16
AUC C-peptide (nmol/l)	472.13 ± 172.51	523.96 ± 21.698	0.12

Data are n (%), mean ± SD, or median (interquartile range).

*BMI z score from 2000 Centers for Disease Control and Prevention growth chart. JDF, Juvenile Diabetes Foundation.

The ROC AUCs of various metabolic indexes for the prediction of progression to type 1 diabetes are summarized in Table 2. Fasting glucose from both IVGTTs and OGTTs performed poorly and did not demonstrate prognostic ability with the same AUC value of 0.49 (95% CI 0.40–0.59). Analysis of IVGTT fasting insulin demonstrated some prognostic value with an AUC value of 0.59 (0.5 – 0.68) although the estimate had borderline significance. The AUC estimate of HOMA-IR and FPIR exceeded 0.5, although the lower confidence limit for both variables fell slightly below 0.5 at 0.49 and 0.48, respectively, rendering them nonsignificant predictors. However, when the ratio of FPIR to HOMA-IR was analyzed, it resulted in an AUC value of 0.66 (0.57–0.74), representing the best index among the indexes derived from IVGTTs. The only statistically significant AUC among the standard indexes derived from OGTT testing was 2-h glucose, which yielded the greatest AUC value at 0.67 (0.59–0.76) of all metabolic indexes examined (Fig. 1). A composite index that included AUC glucose and peak C-peptide was developed, using the proportional hazard model [index = 3.54 × 10^–4 (AUC glucose) − 0.15 × (peak C-peptide)]. The ROC AUC result for the OGTT composite index was 0.71 (0.63–0.79). Although higher than the FPIR-to–HOMA-IR ratio and the 2-h glucose, the differences were not significant. The prediction performance of antibody titers was evaluated for comparison. AUCs for ICA titer and IAA titer were 0.69 (0.61–0.77) and 0.67(0.58–0.76), respectively. They did not provide better prediction than 2-h glucose or the FPIR-to–HOMA-IR ratio (P > 0.05) in this population.

Table 2.

AUC, specificity, sensitivity, PPV, and NPV at the optimal cutoff value

Index/testing	AUC (95% CI)	Optimal cutoff	Sensitivity	Specificity	PPV	NPV
IVGTT
Fasting glucose	0.49 (0.40–0.59)	96	0.23	0.88	0.46	0.74
Fasting insulin	0.59 (0.50–0.68)	10.23	0.82	0.29	0.32	0.8
FPIR	0.57 (0.48–0.66)	156	0.76	0.4	0.34	0.81
HOMA-IR	0.58 (0.49–0.67)	2.64	0.66	0.48	0.33	0.77
FPIR-to-HOMA-IR ratio*	0.66 (0.57–0.74)	49.22	0.68	0.61	0.41	0.83
OGTT
Fasting glucose	0.49 (0.40–0.59)	88	0.47	0.56	0.27	0.7
2-h glucose	0.67 (0.59–0.76)	114	0.62	0.71	0.46	0.83
Peak C-peptide	0.56 (0.47–0.66)	5.3	0.7	0.46	0.34	0.79
AUC C-peptide	0.56 (0.47–0.65)	595	0.81	0.31	0.33	0.8

*AUCs are significantly better than AUC fasting glucose derived from an IVGTT or OGTT (P < 0.01).

Figure 1.

ROC AUC for various metabolic indexes. ●, 2-h glucose–OGTT; ○, FPIR; ▵, FPIR-to–HOMA-IR ratio; □, HOMA-IR.

The cutoff values for optimal prediction of progression to type 1 diabetes using ROC analysis for all metabolic indexes are summarized in Table 2 Although FPIR demonstrated high sensitivity (0.76) at the optimal cutoff value, the low specificity (0.40) depreciates FPIR as a useful prognostic index. Optimal cutoff values for fasting insulin and AUC C-peptide also had high positive predictive ability with sensitivities >0.80 but were lacking in specificity, similar to FPIR, which restricts them from being useful prognostic indexes in this moderate-risk group. The FPIR-to–HOMA-IR ratio and 2-h glucose OGTT produced cutoff values with both sensitivity and specificity levels >0.6, demonstrating the greatest potential as prognostic indexes compared with all other metabolic indexes. The positive predictive value (PPV) was 0.46 and 0.41 for 2-h glucose and FPIR-to–HOMA-IR ratio, respectively. The negative predictive value (NPV) was 0.83 for both indexes.

The optimal cutoff value of 114 mg/dl for OGTT 2-h glucose was notably less than the value currently used to define the lower range of impaired glucose tolerance (140 mg/dl). The hazard ratio for those with 2-h glucose equal to or in excess of the optimal cutoff value (114) compared with those less than the optimal cutoff was 2.96 (95% CI 1.67–5.22) after adjustment for age, sex, and BMI. The 5-year risk for those with baseline values equal to or in excess of the optimal 2-h glucose cutoff level was 46% compared with 17% for those with baseline values less than the optimal cutoff. The hazard ratio for those under the optimal cutoff level of the FPIR-to–HOMA-IR ratio (<49.22) compared with those equal to or in excess of the optimal cutoff level of the FPIR-to–HOMA-IR ratio (≥49.22) was 2.94 (1.62–5.35) after adjustment for age, sex, and BMI. The 5-year risk for those with baseline values equal to or in excess of the optimal FPIR-to–HOMA-IR ratio cutoff level was 41 with to 17% for those with baseline values less than the optimal cutoff. Kaplan-Meier curves (supplementary Figs. A1 and A2, available in an online appendix at http://care.diabetesjournals.org/cgi/content/full/dc10-0802/DC1) illustrate the risk of diabetes over the study period by the level of OGTT 2-h glucose (P < 0.001) and the level of IVGTT FPIR-to–HOMA-IR ratio (P < 0.001).

CONCLUSIONS

The comprehensive baseline data from DPT-1 has provided the opportunity to assess and validate the accuracy of metabolic risk indicators in predicting the future development of type 1 diabetes. Elevated fasting glucose and substantially impaired glucose tolerance are the primary metabolic indicators currently used to identify those with significant risk for progressing to type 1 diabetes; however, these indicators cannot be used to effectively predict disease progression among individuals who had not demonstrated gross metabolic abnormality. To generate accurate prognostic indexes for those with such moderate risk factors, in the current investigation we evaluated nine metabolic indicators for their prediction accuracy associated with the following underlying risk factors: being a relative without impaired glucose tolerance of an individual with type 1 diabetes and having ICA/IAA positivity. The results of this investigation identified two metabolic indicators as having a significant predictive accuracy for identifying individuals likely to progress to clinical onset of type 1 diabetes within 5 years: the FPIR-to–HOMA-IR ratio and 2-h postprandial glucose.

Investigations into the progression of type 2 diabetes have shown that insulin resistance is demonstrated long before overt diabetes is diagnosed and that this can be a powerful predictor of disease progression (20–23). Abdul-Ghani et al. (22) investigated the prognostic performance of insulin secretion/insulin resistance indexes using ROC analysis to determine their predictive accuracy for progression to type 2 diabetes and reported that insulin secretion/insulin resistance is the best predictor of type 2 diabetes demonstrated by substantial sensitivity and specificity. In the current investigation, the role of insulin resistance is also implied from the analysis of the FPIR-to–HOMA-IR ratio, which demonstrated significant predictive accuracy in the study population, with an AUC cutoff point that maintained 0.68 sensitivity and 0.61 specificity for detecting progression to clinical onset. The etiological significance of this observation is the indication that the early stages of type 1 diabetes may be demarcated by a disturbance in the balance between insulin response and insulin activity, although the effects of an insulin secretion/insulin resistance disturbance may be more subtle in type 1 diabetes progression (24). The counterregulatory hormones associated with puberty may play a role in the modulation of insulin secretion and insulin resistance during this developmental period (24,25).

The clinical utility of the FPIR-to–HOMA-IR ratio must be considered in the context of the predictive accuracy found in a standard index derived from an OGTT, a test that is both less burdensome to the patient and, in some instances, may be administered at lower cost. The FPIR-to–HOMA-IR ratio is actually a composite of three measures: two from an IVGTT to determine FPIR and a fasting status measurement to determine HOMA-IR. In contrast, the relatively simplistic 2-h postprandial glucose test from an OGTT maintained the largest ROC AUC and the greatest degree of both sensitivity and specificity at the optimal cutoff value compared with all other metabolic indicators evaluated in this investigation. In addition, when compared at the same NPV, 2-h glucose retained a higher estimated PPV than the FPIR-to–HOMA-IR, although this difference was not statistically significant. However, because HOMA-IR does not require an IVGTT, future research may indicate that insulin resistance can be incorporated as a part of a composite to increase predictive value.

The OGTT is the gold standard method to diagnose type 1 diabetes and impaired glucose tolerance. A standard 2-h glucose tolerance test contributes substantial predictive accuracy beyond screening for the underlying risk factors. This result is consistent with the findings by Sosenko et al. (13), who determined that the accuracy of 2-h glucose was 0.66 for the combined study population of subjects with and without impaired glucose tolerance at baseline (13). The prognostic accuracy of this index is probably due to 2-h glucose being influenced by insulin production and insulin resistance, two important factors that modulate the progression of this disease. This finding is evidenced by an observed level of prognostic accuracy similar to the FPIR-to–HOMA-IR ratio. Because a principal objective of this investigation was to examine metabolic indexes to provide clinical guidance for selecting individuals at high risk for progression to disease despite the absence of clinical metabolic abnormality, we have determined the optimal cutoff point for the metabolic index with the largest ROC AUC, 2-h glucose at 114 mg/dl. This threshold derived from standard OGTT diagnostics may provide cost-effective guidance for clinicians and researchers for selecting patients likely to progress to disease if not provided preventative care, even among those presenting with normal glucose tolerance, in future prevention trials.

This investigation contributes to a growing body of evidence that metabolic indexes derived from OGTT testing are the most efficient and effective analytical method of determining the risk for progression to type 1 diabetes in those with known genetic and autoimmune risk factors (9,13). Measurements from an OGTT are the clinical standard for diagnosing impaired glucose tolerance and clinical diabetes and are therefore a necessary component in evaluating subjects for type 1 diabetes. In the absence of superior predictive value from indexes produced by IVGTT measurements, IVGTT essentially replicates the results from OGTT measurements. Future intervention trials may consider elimination of IVGTT measurements as an effective cost-reduction strategy.

The success of preventative medicine is dependent on accurate identification of patients with risk for disease development at an early stage of disease progression. In this investigation, we analyzed the prediction accuracy of nine common and novel metabolic indicators for identifying patients with moderate risk factors, who display no clinical metabolic abnormality as yet and who progress to type 1 diabetes within 5 years. Both the FPIR-to–HOMA-IR ratio from the IVGTT and the 2-h glucose from the OGTT provided significant prognostic value. The standard OGTT index of 2-h glucose is preferred because it achieved the largest prognostic accuracy in predicting disease onset, making the FPIR-to–HOMA-IR ratio a redundant and unnecessary predictive index. In addition, our findings confirm prior DPT-1 findings that accuracy for selecting at-risk patients could be improved by using 2-h glucose values below the current cutoff for impaired glucose tolerance. Future analysis should focus on the combination of metabolic markers, immunological markers, and/or genetic markers to improve the modest PPVs derived from solitary predictors. Our results indicated that autoantibody titers may not provide a cost-benefit improvement over OGTTs as a sole predictor, but they may be an important component of a composite modeled score. This is a secondary analysis of DPT-1 data with the sample size limited to the placebo arm of the oral insulin protocol. Nonetheless, it provides much needed guidance for clinicians and researchers for selecting subjects in future prevention trials in populations with underlying risk factors and clinically normal glucose tolerance by providing both the evidence for a preferred index and a threshold for selection.