The Lack of Utility of Circulating Biomarkers of Inflammation and Endothelial Dysfunction for Type 2 Diabetes Risk Prediction Among Postmenopausal Women

Abstract

Background

Recent studies have linked plasma markers of inflammation and endothelial dysfunction to type 2 diabetes mellitus (DM) development. However, the utility of these novel biomarkers for type 2 DM risk prediction remains uncertain.

Methods

The Women’s Health Initiative Observational Study (WHIOS), a prospective cohort, and a nested case-control study within the WHIOS of 1584 incident type 2 DM cases and 2198 matched controls were used to evaluate the utility of plasma markers of inflammation and endothelial dysfunction for type 2 DM risk prediction. Between September 1994 and December 1998, 93 676 women aged 50 to 79 years were enrolled in the WHIOS. Fasting plasma levels of glucose, insulin, white blood cells, tumor necrosis factor receptor 2, interleukin 6, high-sensitivity C-reactive protein, E-selectin, soluble intercellular adhesion molecule 1, and vascular cell adhesion molecule 1 were measured using blood samples collected at baseline. A series of prediction models including traditional risk factors and novel plasma markers were evaluated on the basis of global model fit, model discrimination, net reclassification improvement, and positive and negative predictive values.

Results

Although white blood cell count and levels of interleukin 6, high-sensitivity C-reactive protein, and soluble intercellular adhesion molecule 1 significantly enhanced model fit, none of the inflammatory and endothelial dysfunction markers improved the ability of model discrimination (area under the receiver operating characteristic curve, 0.93 vs 0.93), net reclassification, or predictive values (positive, 0.22 vs 0.24; negative, 0.99 vs 0.99 [using 15% 6-year type 2 DM risk as the cutoff]) compared with traditional risk factors. Similar results were obtained in ethnic-specific analyses.

Conclusion

Beyond traditional risk factors, measurement of plasma markers of systemic inflammation and endothelial dysfunction contribute relatively little additional value in clinical type 2 DM risk prediction in a multiethnic cohort of postmenopausal women.

Type 2 diabetes mellitus (DM) is a leading cause of morbidity and mortality in the United States.¹ According to the Centers for Disease Control and Prevention, more than 24 million Americans were living with diabetes in 2007,² and this number is forecasted to increase to 48 million by 2050.³

The alarming surge of diabetes incidence highlights the importance of developing effective screening and preventive measures to control this devastating disease. Among high-risk individuals, lifestyle modification and pharmacological interventions have been demonstrated to be effective.^4,5 It is therefore of clinical and public health importance to identify high-risk individuals for aggressive treatment.

The recent discovery of novel biomarkers for type 2 DM risk not only improves molecular understanding of diabetes etiology but also brings new opportunity for clinical risk stratification and management. Within the Women’s Health Initiative Observational Study (WHIOS) and several other large cohort studies, plasma markers of systemic inflammation^6–19 and endothelial dysfunction^20–22 appeared to be prospectively associated with risk of type 2 DM (Table 1). These findings support the notion that chronic inflammation and endothelial dysfunction are antecedents of clinical diabetes. However, it remains unknown whether incorporating such markers may improve the performance of risk prediction in apparently healthy individuals. In this study, we examined the added predictive value of plasma inflammation markers (ie, white blood cell count, tumor necrosis factor receptor 2, interleukin 6 [IL-6], and high-sensitivity C-reactive protein [hsCRP]) and endothelial dysfunction markers (E-selectin, soluble intercellular adhesion molecule 1 [ICAM-1], and vascular cell adhesion molecule 1 [VCAM-1]) beyond the traditional diabetes risk factors in postmenopausal women of diverse ethnicities.

Table 1.

Adjusted Relative Risk of Selected Inflammation and Endothelial Dysfunction Plasma Markers for Women in the Highest vs Lowest Quartiles in the Women’s Health Initiative^a

Marker	RR (95% CI)
Inflammation plasma markersb
TNF-R2 level	1.47 (1.10–1.97)
IL-6 level	3.08 (2.25–4.23)
hsCRP level	3.46 (2.50–4.80)
Endothelial dysfunction plasma markersc
E-selectin level	3.46 (2.56–4.68)
ICAM-1 level	2.34 (1.75–3.13)
VCAM-1 level	1.48 (1.07–2.04)

Abbreviations: CI, confidence interval; hsCRP, high-sensitivity C-reactive protein; ICAM-1, soluble intercellular adhesion molecule 1; IL-6, interleukin 6; RR, relative risk; TNF-R2, tumor necrosis factor receptor 2; VCAM-1, vascular cell adhesion molecule 1.

^aResults from the same nested case-control study used in the present risk prediction study. The model was adjusted for matching factors (age, race/ethnicity, clinical center, time of blood draw, and length of follow-up), body mass index, alcohol intake, level of physical activity, cigarette smoking status, use or nonuse of postmenopausal hormone therapy, and presence or absence of family history of diabetes. Plasma markers were not mutually adjusted in the model.

^bFrom Liu et al.¹⁴

^cFrom Song et al.²¹

METHODS

STUDY POPULATION

The WHIOS is an ongoing longitudinal study designed to examine associations between clinical, socioeconomic, behavioral, and dietary risk factors and subsequent health outcomes, including cardiovascular disease (CVD) and diabetes in multiethnic postmenopausal women (see the eAppendix for the list of Women’s Health Initiative Investigators; http://www.archinternmed.com). Details of the rationale, eligibility, and other design aspects have been published elsewhere.²³ In brief, between September 1994 and December 1998, the WHIOS enrolled a total of 93 676 women aged 50 to 79 years at 40 clinical centers throughout the United States. Women completed study questionnaires and provided fasting blood samples at study baseline, and they were followed up by means of annually mailed self-administered questionnaires and an additional clinical center visit at 3 years after enrollment for obtaining data on disease outcomes and possible exposures, collecting physical and anthropometric measures, and obtaining blood samples. Methods of data collection and measurement characteristics have been reported previously.^23–25 Of the 93 676 postmenopausal women enrolled into the WHIOS cohort, 82 069 had no history of diabetes or CVD. The WHIOS has been approved by human subjects review committees at each participating institution.

A case-control study nested in the WHIOS was conducted to investigate the associations between novel plasma markers and risk of clinical type 2 DM.^14,21,26 Incident type 2 DM was defined as first-time use of oral hypoglycemic agents or insulin assessed by medication use or self-report when medication use data were unavailable. A total of 1584 cases and 2198 controls who were free of reported diabetes and CVD at baseline and who provided adequate blood specimens were selected during a median follow-up of 5.9 years (mean, 5.5 years; range, September 1994 to February 2004). Controls were selected following the principles of risk-set sampling and matched to the cases by age (±2.5 years), race/ethnicity, clinical center, time of blood drawn (±0.10 hours), and length of follow-up.

MEASUREMENTS OF BIOMARKERS

White blood cell count was measured for all women in the WHIOS. Plasma levels of fasting glucose, insulin, tumor necrosis factor receptor 2, IL-6, hsCRP, E-selectin, ICAM-1, and VCAM-1 were measured only for women selected in the nested case-control study. Details of laboratory procedures and specific assay methods have been published previously.^14,21,26

STATISTICAL ANALYSIS

We first determined the distributions of plasma markers and other risk factors in women enrolled in the WHIOS cohort and those in the case-control study. Correlations between plasma markers were evaluated by calculating Pearson partial correlation coefficients. Continuous variables with skewed distribution were transformed with natural logarithm. We then determined the best functional forms for continuous variables by examining smoothed plots and fractional polynomial models.²⁷ Linear fit appeared to be reasonable for all continuous variables considered. Conditional logistic regression²⁸ was used for building the prediction models. Because of the need for numerous model comparisons in this study, we performed multiple imputations to address the missing data issue²⁹ (14% of the women had missing information on ≥1 of the covariates). In a sensitivity analysis, we repeated all the subsequent analyses, excluding women whose baseline glucose level was 126 mg/dL or greater (737 cases and 27 controls) (to convert glucose levels to millimoles per liter, multiply by 0.0555).

Reference Model

Clinical risk factors included age, race/ethnicity, waist circumference, hypertension (≥130/85 mm Hg³⁰ or treated hypertension), history of high cholesterol levels requiring medication, physical activity, cigarette smoking, alcohol use, and family history of diabetes in 1 or more first-degree relatives. These clinical risk factors coupled with fasting glucose levels (the traditional risk factors) constituted our reference model. Age and race/ethnicity were matching factors and were adjusted in the conditional logistic regression. We chose to include waist circumference as opposed to body mass index or waist to hip ratio because models containing waist circumference resulted in a better model fit measured by the model likelihood ratio χ² statistic.

We also evaluated a secondary reference model including the homeostasis model assessment of insulin resistance (HOMA-IR) and beta cell function (HOMA-B) indices. The HOMA-IR was computed as (fasting insulin level × fasting glucose level)/22.5, with insulin level reported in microinternational units per milliliter and glucose level in millimoles per milliliter. The HOMA-B was computed as (20 × fasting insulin level)/(fasting glucose level-3.5), with insulin level reported in microinternational units per milliliter and glucose level in millimoles per milliliter. This secondary reference model did not perform better than the model with fasting glucose level, as assessed by global model fit statistics (described in the “Added Predictive Value of Novel Plasma Markers” subsection) and area under the receiver operating characteristic curve (AUC). Therefore, the reference model with fasting glucose level was used for the subsequent analyses.

Risk Prediction

Model-predicted type 2 DM risk for each woman in the case-control study was calculated using methods described previously.^31–33 Briefly, we first estimated the overall 6-year survival probability from the entire WHIOS cohort by using the Kaplan-Meier estimator. This overall survival probability was then used as the basis for calculating the individual model-predicted risk with a scaling factor determined by the linear function of model coefficients and the specific covariate values of each woman, corrected by the mean covariate values in the WHIOS. The mean values of plasma markers in the cohort were approximated by the values of these markers among controls.

Added Predictive Value of Novel Plasma Markers

The predictive value for each of the inflammation and endothelial dysfunction markers beyond traditional risk factors was assessed by comparing the reference model with the model that included the markers added one at a time (as separate models). The predictive value was evaluated on the basis of (1) improvement in global model fit and (2) improvement in model discrimination.^34–36 For assessing improvement in global model fit, we computed −2 log likelihood, the variable likelihood ratio χ² statistic (the improvement of fit for inclusion of each biomarker separately), the model likelihood ratio χ² statistic (fit for the entire model), and Bayesian information criteria (BIC; fit for the entire model taking into account the number of variables). To compare model discrimination adding specific markers, we calculated the AUC and the integrated discrimination improvement (IDI) recently proposed by Pencina et al³⁶ (a measure that quantifies the improvement of predicted risk for cases and controls). Because of the case-control sampling and the matching design, we calculated AUC and IDI weighted using the stabilized weight of inversed case and control sampling fraction. The inversed sampling fraction (weight) was stabilized to reflect the population distribution of age and ethnicity in cases and controls in the full cohort for unbiased variance.

To further explore the utility of novel plasma markers in type 2 DM risk prediction in different ethnicities, we repeated all analyses for each ethnicity separately.

Building a Best Parsimonious Prediction Model

To evaluate whether novel plasma markers could replace the role of certain traditional risk factors in the prediction equation, we performed the following analyses for model selection: (1) a model that considered traditional risk factors only; (2) a model that considered novel plasma markers and traditional risk factors; and (3) a model that considered novel markers and a subset of traditional risk factors that can be objectively measured (ie, excluding history of smoking, alcohol use, and exercise). Model 3 was fitted because novel plasma markers might replace the need to collect self-reported complex behaviors that were susceptible to differential measurement error. We used backward and stepwise model-building strategies for model selection with a significance criterion level of α =.05. If the two model-selection approaches suggested different models, the one with the smaller BIC was selected. For comparison purposes, a previously proposed clinical model^37,38 based on the metabolic syndrome–related covariates, including waist circumference, hypertension, history of high cholesterol levels, family history of type 2 DM, and fasting glucose level, was also fitted.

Global model fit statistics (ie, −2 log likelihood and BIC) and model discrimination ability (AUC and IDI) were calculated for these models. In addition, net reclassification improvement, a measure that quantifies the improvement of prediction of clinically meaningful risk categories,³⁶ was also calculated. Clinically meaningful risk categories were adopted and modified from Adult Treatment Panel III guidelines for primary CVD prevention (low-, medium-, and high-risk categories defined as <6%, 6%–20%, and >20% 10-year risk of developing CVD, respectively). We interpolated these cutoffs for 6-year type 2 DM risk (ie, 3.6% and 12.5%) and used less than 5%, 5% to 15%, and greater than 15% as the low-, medium-, and high-risk categories, respectively, for the ease of interpretation and patient communication. Positive and negative predictive values (PPV and NPV) were estimated for each model following the Bayes theorem, which used sensitivity and specificity as well as the background risk in the population. Because a 15% 6-year type 2 DM risk is considered a high risk in this study, we used 15% as the cutoff to calculate PPV and NPV for our models. For sensitivity analysis, 10% and 20% cutoff were also used for the calculation of PPV and NPV. Again, the weighted calculation of AUC, IDI, net reclassification improvement, and predictive values was performed. All analyses were performed with SAS statistical software (version 9.1; SAS Institute Inc, Cary, North Carolina).

RESULTS

Baseline characteristics of the WHIOS cohort participants and the 3782 women in the case-control study are presented in Table 2. Overall, the newly developed type 2 DM case patients had higher prevalence of overweight/obesity, hypertension, hypercholesterolemia, cigarette smoking, and family history of diabetes than controls at baseline. These incident type 2 DM cases also had higher levels of fasting glucose (70.2% of cases had fasting glucose levels ≥110 mg/dL), insulin, HOMA-IR, and inflammation and endothelial dysfunction markers at baseline. The 6-year estimated cumulative risk of developing type 2 DM risk for each age and ethnicity group is based on the entire WHIOS cohort (Table 3). At all ages, minority women had higher diabetes incidence than white women (eg, 8% in blacks vs approximately3% in whites).

Table 2.

Baseline Characteristics for Women Enrolled in the WHIOS Cohort and for Women Selected in the Nested Case-Control Study of Type 2 Diabetes^a

Variable	WHIOS Cohortb (N=80 509)	Cases (n=1584)	Controls (n=2198)	P Valuec
Age, mean (SD), y	63.3 (7.3)	62.7 (7.0)	62.3 (7.0)
Age, No. (%), y
<60	26 885 (33.4)	549 (34.7)	822 (37.4)
60–69	35 304 (43.9)	715 (45.1)	972 (44.2)
≥70	18 320 (22.8)	320 (20.2)	404 (18.4)
BMI, mean (SD)	26.9 (5.6)	32.3 (7.0)	27.6 (5.9)	<.001
Body weight, mean (SD), kg	70.8 (16.3)	84.3 (20.2)	72.2 (17.5)	<.001
Waist, mean (SD), cm	83.1 (2.0)	97.0 (15.2)	83.3 (2.0)	<.001
Waist to hip ratio, mean (SD)	0.8 (0.1)	0.9 (0.1)	0.8 (0.1)	<.001
Race/ethnicity, No. (%)
White	69 367 (86.2)	968 (61.1)	968 (44.0)
Black	5736 (7.1)	366 (23.1)	732 (33.3)
Hispanic/Latina	3035 (3.8)	152 (9.6)	303 (13.8)
Asian/Pacific Islander	2371 (2.9)	98 (6.2)	195 (8.9)
Hypertension, No. (%)	23 115 (28.7)	788 (49.7)	706 (32.1)	<.001
Treated high cholesterol levels, No. (%)	9872 (12.3)	294 (18.6)	262 (11.9)	<.001
Physical activity, mean (SD), METS/wk	14.1 (14.5)	9.8 (12.4)	12.8 (14.4)	<.001
Alcohol consumption, No. (%)
Nondrinkerd	8394 (10.4)	271 (17.1)	352 (16.0)
Past drinker	13 384 (16.6)	427 (27.0)	470 (21.4)	.32
<1 Drink/mo	9219 (11.5)	249 (15.7)	286 (13.0)	.74
<1 Drink/wk	16 474 (20.5)	325 (20.5)	444 (20.2)	.02
1–6 Drinks/wk	21 741 (27.0)	221 (14.0)	449 (20.4)	<.001
≥7 Drinks/wk	10 761 (13.4)	76 (4.8)	182 (8.3)	<.001
Cigarette smoking, No. (%)
Neverd	40 686 (50.5)	786 (49.6)	1211 (55.1)
Past	33 912 (42.1)	657 (41.5)	811 (36.9)	.05
Current	4828 (6.0)	121 (7.6)	148 (6.7)	.02
Family history of diabetes, No. (%)	23 625 (29.3)	849 (53.6)	777 (35.4)	<.001
WBC count, median (IQR), ×1000/mL	5.6 (4.7–6.6)	6.3 (5.3–7.7)	5.4 (4.5–6.5)	<.001
Fasting glucose level, mg/dL		122 (106–147)	92 (87–98)	<.001
Impaired fasting glucose level, No. (%)
≥110 mg/dL		1112 (70.2)	139 (6.3)	<.001
≥100 mg/dL		1329 (83.9)	448 (20.4)	<.001
Fasting insulin level, median (IQR), μIU/mL		12.6 (8.1–18.6)	6.4 (4.4–9.6)	<.001
HOMA-IR, median (IQR)		4.0 (2.5–6.3)	1.4 (1.0–2.3)	<.001
HOMA-B, median (IQR)		75.1 (44.6–118.1)	81.7 (56.1–120.2)	.06
TNF-R2 level, median (IQR), pg/mL		2632.7 (2190.6–3296.6)	2361.4 (1927.5–2883.6)	<.001
IL-6 level, median (IQR), pg/mL		2.6 (1.6–4.6)	1.5 (1.0–2.8)	<.001
hsCRP level, median (IQR), mg/L		4.0 (2.0–7.6)	2.1 (0.8–4.4)	<.001
E-selectin level, median (IQR), ng/mL		49.3 (33.6–71.0)	36.9 (26.0–50.9)	<.001
ICAM-1 level, median (IQR), ng/mL		323.6 (268.2–383.9)	280.4 (234.0–330.4)	<.001
VCAM-1 level, median (IQR), ng/mL		765.2 (595.2–972.1)	696.3 (543.0–861.8)	<.001

Abbreviations: BMI, body mass index (calculated as weight in kilograms divided by height in meters squared); HOMA-B, homeostasis model assessment of beta cell function; HOMA-IR, homeostasis model assessment of insulin resistance; IQR, interquartile range; METS, metabolic equivalents; WBC, white blood cell; WHIOS, Women’s Health Initiative Observational Study. Other abbreviations: See Table 1.

SI conversion factors: To convert glucose to millimoles per liter, multiply by 0.0555; hsCRP to nanomoles per liter, multiply by 9.524; insulin to picomoles per liter, multiply by 6.945; and WBC to 10⁹ per liter, multiply by 0.001.

^aNumbers might not total all women in the WHIOS cohort/cases/controls because of missing values. Percentages have been rounded and might not total 100 because of missing values.

^bExcludes those with a history of cardiovascular disease or diabetes at baseline, Native American subjects, and those with unknown race/ethnicity. Biomarkers were not measured (except WBC) for women in the WHIOS cohort except those in the nested case-control study. Hence, the values were left blank for the biomarkers.

^cP value from comparing cases and controls in the nested case-control study. P value is not presented for the matching factors.

^dIndicates reference category.

Table 3.

Estimated 6-Year Risk of Developing Type 2 Diabetes in the Women’s Health Initiative Observational Study Cohort by Age Group and Ethnicity

Race	Age Group, y
	<60	60–69	≥70
White
Total No. (No. of cases) in cohort	22 160 (690)	30 765 (1115)	16 442 (573)
6-y riska	0.02	0.03	0.03
Black
Total No. (No. of cases) in cohort	2401 (215)	2401 (209)	934 (71)
6-y riska	0.08	0.08	0.08
Hispanic/Latina
Total No. (No. of cases) in cohort	1528 (98)	1157 (71)	350 (26)
6-y riska	0.06	0.06	0.08
Asian/Pacific Islander
Total No. (No. of cases) in cohort	796 (44)	981 (49)	594 (36)
6-y riska	0.05	0.05	0.06

^aRisk was determined for those who did not have diabetes or cardiovascular disease at baseline.

CONTRIBUTION TO MODEL FIT AND RISK DISCRIMINATION BY NOVEL PLASMA MARKERS

When we assessed the contribution of individual novel plasma markers, levels of white blood cells, IL-6, hsCRP, and ICAM-1 appeared to improve global model fit, with ICAM-1 level improving the model fit the most as suggested by the variable likelihood ratio χ² statistic (Table 4). However, none of the novel plasma markers increased the AUC or the IDI in any significant manner. Although the groups of inflammatory and endothelial dysfunction markers resulted in similar improvement in model fit overall (Table 4), the added contribution by these plasma markers varied across ethnic groups (Table 5). While hsCRP level appeared to enhance model fit in white women, IL-6 and ICAM-1 levels were the only 2 markers that enhanced model fit in black women. However, none of the markers resulted in more favorable model BIC or AUC in all race/ethnic groups. Similar results were obtained when women with baseline glucose levels of 126 mg/dL or more were excluded from the analyses (Table 4, bottom).

Table 4.

Relative Contribution of Plasma Inflammation and Endothelial Dysfunction Markers for Model Fit and Discrimination

Model	−2 LnLa	LR χ2 Statistic	Variable LR χ2 Statistic	BICa	AUC	IDIb
Clinical risk factorsc+fasting glucose level (reference model)	823.82	1845.47		897.96	0.93
+WBC count	817.84	1851.45	5.98d	900.22	0.93	0.001
+TNF-R2 level	823.60	1845.69	0.22	905.98	0.93	<−0.001
+IL-6 level	815.13	1854.17	8.69d	897.51	0.93	0.002
+hsCRP level	814.33	1854.96	9.49d	896.71	0.93	0.005
+WBC count and TNF-R2, IL-6, and hsCRP levels	807.50	1861.79	16.32d	914.60	0.93	0.005
+E-selectin level	820.76	1848.53	3.05	903.14	0.93	0.002
+ICAM-1 level	810.73	1858.56	13.08d	893.11	0.93	0.007
+VCAM-1 level	820.03	1849.26	3.79	902.41	0.93	0.001
+E-selectin, ICAM-1, and VCAM-1 levels	809.42	1859.88	14.40d	908.27	0.93	0.007
Excluding Women With Baseline Fasting Glucose Level ≥126 mg/dL
Clinical risk factorsc+fasting glucose level (reference model)	719.81	733.13		793.95	0.87
+WBC count	715.74	737.20	4.07d	798.12	0.87	−0.002
+TNF-R2 level	718.93	734.01	0.88	801.31	0.87	−0.000
+IL-6 level	712.04	740.91	7.78d	794.42	0.88	0.004
+hsCRP level	712.36	740.58	7.45d	794.74	0.88	0.009
+WBC count and TNF-R2, IL-6, and hsCRP levels	705.88	747.06	13.93d	812.97	0.88	0.009
+E-selectin level	717.04	735.91	2.78	799.42	0.87	0.003
+ICAM-1 level	707.30	745.65	12.52d	789.68	0.87	0.015
+VCAM-1 level	717.78	735.16	2.03	800.16	0.87	0.002
+E-selectin, ICAM-1, and VCAM-1 levels	706.84	746.10	12.97d	805.69	0.87	0.015

Abbreviations: AUC, area under the receiver operating characteristic curve; BIC, Bayesian information criteria; IDI, integrated discrimination improvement; LnL, log likelihood; LR, likelihood ratio; WBC, white blood cell. Other abbreviations: See Table 1.

SI conversion factor: To convert glucose to millimoles per liter, multiply by 0.0555.

^aThe smaller the model −2 LnL and BIC, the better the model fit (considering the number of covariates in the BIC).

^bNone of the IDI was statistically significantly greater than 0 (P>.05, 1-sided test). The 1-sided test was used because the key question of interest was whether the novel plasma markers improved model prediction.

^cClinical risk factors included the matching factors (age and race/ethnicity) and waist circumference, history of hypertension, history of high cholesterol levels requiring medication, physical activity, smoking (never, current, or past), current alcohol drinking (0 indicates <1 drink/mo; 1, <1 drink/wk; 2, 1–6 drinks/wk; and 3, ≥7 drinks/wk), and family history of diabetes (yes or no).

^dP<.05 in deviance test (likelihood ratio test) compared with the reference model.

Table 5.

Relative Contribution of Plasma Inflammation and Endothelial Dysfunction Markers in Model Fit and Discrimination by Race/Ethnicity

Model	White			Black			Hispanic/Latina			Asian/Pacific Islander
	−2 LnL	BIC	AUC	−2 LnL	BIC	AUC	−2 LnL	BIC	AUC	−2 LnL	BIC	AUC
Clinical risk factorsa+fasting glucose level (reference model)	367.59	375.16	0.93	363.69	370.69	0.88	148.47	154.59	0.90	61.96	67.64	0.92
	Variable LR χ2 Statistic	BIC	AUC	Variable LR χ2 Statistic	BIC	AUC	Variable LR χ2 Statistic	BIC	AUC	Variable LR χ2 Statistic	BIC	AUC
+WBC count	2.61	375.01	0.93	0.75	397.29	0.88	3.54	173.22	0.90	1.82	104.90	0.92
+TNF-R2 level	1.44	376.17	0.93	0.01	398.02	0.88	4.15b	172.61	0.91	1.19	105.53	0.92
+IL-6 level	1.27	376.35	0.93	6.21b	391.82	0.88	0.74	176.02	0.90	0.00	106.72	0.92
+hsCRP level	6.62b	371.00	0.93	1.66	396.37	0.88	1.65	175.11	0.90	1.86	104.86	0.92
+WBC count and TNF-R2, IL-6, and hsCRP levels	9.88b	390.44	0.93	6.49	412.55	0.88	9.25	185.87	0.91	4.99	118.77	0.92
+E-selectin level	2.48	376.77	0.93	0.18	397.85	0.88	1.89	174.87	0.90	0.51	106.21	0.91
+ICAM-1 level	1.51	377.82	0.93	5.46b	392.57	0.88	7.47b	169.29	0.90	0.40	106.32	0.92
+VCAM-1 level	0.56	378.72	0.93	0.20	397.83	0.88	5.87b	170.89	0.90	0.61	106.11	0.92
+E-selectin, ICAM-1, and VCAM-1 levels	3.13	391.28	0.93	5.56	406.47	0.88	10.25b	178.75	0.90	0.91	117.17	0.92

Abbreviations; AUC, area under the receiver operating characteristic curve; BIC, Bayesian information criteria; LnL, log likelihood; LR, likelihood ratio; WBC, white blood cell. Other abbreviations: See Table 1.

^aDescribed in Table 4.

^bP<.05 in deviance test (likelihood ratio test) compared with the reference model.

BEST PARSIMONIOUS PREDICTION MODELS

Based on the evaluation of BIC, the best prediction model of novel plasma markers (BIC=891.09; Table 6) had global model fit performance that was similar to the model based on the metabolic syndrome–related covariates and family history of type 2 DM (BIC=892.77). All 4 types of best prediction models had similar discrimination/reclassification performance, as indicated by the minimal change in AUC (all models achieved an AUC of 0.93), IDI, and net reclassification improvement (Table 6). Based on cutoff values of 10%, 15%, and 20% 6-year predicted risk, PPV and NPV were similarly comparable for all 4 models (Table 6). In the sensitivity analysis excluding women with baseline glucose levels of 126 mg/dL or more, similar results were obtained.

Table 6.

Comparison of Best Parsimonious Prediction Models

	Model 1a		Model 2b		Model 3c		Model 4d
	HR	P Value	HR	P Value	HR	P Value	HR	P Value
Waist circumference per cm	1.09	<.001	1.06	<.001	1.07	<.001	1.10	<.001
History of hypertension	1.58	<.01	1.51	<.01	1.49	<.001	1.52	<.001
History of treated high cholesterol levels	1.78	<.01	1.88	<.01	1.93	<.001	1.88	<.001
Current smoker	1.74	.03
Past smoker	1.49	<.01	1.51	<.01
Alcohol intake	0.80	<.01	0.82	<.01
Family history of diabetes	1.56	<.001	1.55	<.001	1.55	<.001	1.55	<.001
Log fasting blood glucose level per SD	3.24	<.001	3.17	<.001	3.14	<.001	3.20	<.001
Log WBC count			1.68	.05	1.73	.03
Log hsCRP level			1.19	<.01	1.18	.01
ICAM-1 level			1.003	<.01	1.002	<.001
	Model 1a		Model 2b		Model 3c		Model 4d
−2 LnL	834.87		808.71		822.55		851.58
BIC	900.78		891.09		888.45		892.77
AUC 0.93	0.93		0.93		0.93		0.93
IDIe	1.000 [Reference]		0.012		0.007		−0.006
NRIe	1.000 [Reference]		−0.005		−0.008		−0.008
SE/SPf	0.83/0.91		0.84/0.89		0.83/0.90		0.82/0.91
PPV/NPVf	0.24/0.99		0.22/0.99		0.23/0.99		0.24/0.99
PPV/NPVg	0.18/0.99		0.17/0.99		0.17/0.99		0.18/0.99
PPV/NPVh	0.28/0.99		0.26/0.99		0.27/0.99		0.30/0.99
Excluding Women With Baseline Fasting Glucose Level ≥126 mg/dLi
AUC	0.87		0.88		0.87		0.87
IDIe	1.000 [Reference]		0.026		0.012		−0.013
NRIe	1.000 [Reference]		−0.011		−0.007		−0.023
SE/SPf	0.66/0.91		0.70/0.90		0.68/0.90		0.66/0.91
PPV/NPVf	0.21/0.99		0.20/0.99		0.21/0.99		0.21/0.99
PPV/NPVg	0.16/0.99		0.16/0.99		0.16/0.99		0.16/0.99
PPV/NPVh	0.24/0.99		0.23/0.99		0.23/0.99		0.25/0.98

Abbreviations: AUC, area under the receiver operating characteristic curve; BIC, Bayesian information criteria; HR, hazard ratio; hsCRP, high-sensitivity C-reactive protein; ICAM-1, soluble intercellular adhesion molecule 1; IDI, integrated discrimination improvement; LnL, log likelihood; NPV, negative predictive value; NRI, net reclassification improvement; PPV, positive predictive value; SE, sensitivity; SP, specificity; WBC, white blood cell.

SI conversion factor: To convert glucose to millimoles per liter, multiply by 0.0555.

^aIncludes clinical risk factors described in Table 4 and fasting glucose level. This model is different from the reference model specified in Table 4 because this is the best parsimonious model and had only the variables retained from the model selection procedure.

^bIncludes clinical risk factors described in Table 4, fasting glucose level, and novel plasma markers retained from the model selection procedure.

^cRisk factors that can be objectively measured included all plasma markers and clinical risk factors described in Table 4 except physical activity, cigarette smoking, and alcohol use.

^dIncludes risk factors based on the metabolic syndrome and family history of type 2 diabetes mellitus.

^eNone of the IDI or NRI was significantly greater than 0 (P>.05, 1-sided test). The 1-sided test was used because the key question of interest was whether the novel plasma markers improved model prediction. Risk categories for NRI were low (<5%), medium (5%–15%), and high (>15%) for 6-year type 2 diabetes mellitus risk.

^fCutoff was 15%.

^gCutoff was 10%.

^hCutoff was 20%.

ⁱModel 2 selected for this group included the following covariates: waist circumference, history of hypertension, history of high cholesterol levels requiring medication, past smoking, current alcohol drinking, family history of diabetes, and fasting blood glucose, hsCRP, tumor necrosis factor receptor 2, and ICAM-1 levels. Model 4 selected for this group included the following covariates: waist circumference, history of hypertension, history of high cholesterol levels requiring medication, family history of diabetes, and fasting blood glucose, hsCRP, and ICAM-1 levels. Models 1 and 4 selected for this group included the same covariates as those in the primary analysis for all women (as shown in this table).

COMMENT

In this study of multiethnic postmenopausal women, plasma markers of systemic inflammation (white blood cells, tumor necrosis factor receptor 2, IL-6, and hsCRP levels) and endothelial dysfunction (E-selectin, ICAM-1, and VCAM-1 levels) did not appear to provide additional values for type 2 DM risk stratification and prediction, despite the fact that they were independent risk factors for this disease. Although the addition of some novel biomarkers significantly improved global model fit, none of these biomarkers led to a meaningful improvement in model discrimination as assessed by AUC and IDI. In particular, the final best model that included novel plasma markers yielded model fit, discrimination, reclassification ability, and predictive values similar to those of the model incorporating only well-established traditional risk factors. The latter model also performed similarly to a simpler model using only the metabolic syndrome–related covariates and family history of diabetes. These results suggest that neither detailed lifestyle factors (eg, history of smoking and alcohol use) nor novel plasma markers are critical for type 2 DM risk stratification in this multiethnic cohort of postmenopausal women.

Few studies have comprehensively and prospectively examined the added predictive value of novel plasma markers for type 2 DM risk stratification in men and women, especially in ethnic minorities. In a cohort of 822 middle-aged men and women followed up for 5 years, Hanley and colleagues³⁹ found no improvement for type 2 DM risk discrimination by adding the hsCRP level (AUC=0.71) to a model incorporating metabolic syndrome (AUC=0.69).

Recent studies have clearly associated biomarkers of systemic inflammation and endothelial dysfunction with type 2 DM risk, independent of fasting glucose levels and other clinical risk factors.^{6–13,15,17–20,22} However, risk factors that are statistically significant in etiology research are not necessarily useful predictors in screening or predictive tests. In fact, even risk factors with strong effect sizes may fail to perform in risk stratification because of substantial overlapping in the distribution of the specific risk factor in individuals who will and will not ultimately develop the disease.⁴⁰ Conventional approaches for evaluating the performance of model prediction emphasized the use of AUC, which can be directly interpreted as the probability that the model will assign a higher predicted risk to cases than to noncases. However, the implications of this approach are limited because of its lack of consideration of the actual predicted risk in relation to event and clinically meaningfully classification. Recently, Pencina and colleagues³⁶ proposed the use of IDI and net reclassification improvement to capture correct vs incorrect movements in model-predicted risk/risk category among cases and noncases compared with the reference model. Cook^41,42 and Greenland⁴³ also argued for the application of measures for predictive gain (eg, model calibration and PPV and NPV) because this information is more relevant for clinical decision making. In the present study, we found that inclusion of inflammatory and endothelial dysfunction markers did not improve most of these measures in our model.

Several type 2 DM risk prediction algorithms have been proposed in the literature. Algorithms that use anthropometrics and clinical and lifestyle risk factors were reported to have an AUC of approximately 0.80 in various populations. Examples include the Cambridge risk score⁴⁴ (age, sex, body mass index, use of corticosteroids, treatment for hypertension, and parental history of diabetes), the National Cholesterol Education Program metabolic syndrome criteria,⁴⁵ the San Antonio Heart Study model⁴⁶ (age, sex, ethnicity, fasting systolic blood pressure, plasma glucose and high-density lipoprotein cholesterol levels, body mass index, and parental history of diabetes), and the Finnish Diabetes Risk Score³⁰ (age, body mass index, waist circumference, treatment of hypertension, fasting plasma glucose level, physical activity, and diet). Although these models appear to have reasonable discrimination ability, their calibration and predictive values remain unknown. We reported PPVs of 0.22 to 0.28 for our models when a 6-year risk of 15% was chosen as the cutoff. However, the predictive values are a function of population background risk, and they may not be generalizable to women in different age or racial/ethnic groups or geographic areas.

Several potential limitations need to be considered when interpreting our findings. First, model calibration (accuracy of the predicted risk compared with actual event rate) could not be directly assessed because novel plasma markers were measured only in the nested case-control study and not in the entire WHIOS. Second, because the same study samples were used for model fitting and AUC calculation, the AUC values might have been overestimated.⁴⁷ However, given our sample size, this potential bias is expected to be minimal.⁴⁷ Third, the AUC and other measures of model prediction may be less generalizable to ethnic minorities because of the small sample sizes of minorities in our study (eg, Asian and Hispanic/Latina subjects). Finally, our analyses did not account for differential competing risk (ie, death), although any bias due to differential competing risk should be minimal because death occurred in less than 5% of the noncase women in the cohort during the case-control sampling period.

In conclusion, in this US multiethnic cohort of post-menopausal women prospectively followed up for 6 years, our modeling indicates that novel plasma markers of systemic inflammation and endothelial dysfunction did not have additional value for clinical type 2 DM risk prediction. Further evaluation by other prospective, population-based cohort studies is needed to confirm and assess the generalizability of these findings.