Cardiovascular Risk Prediction in Diabetic Men and Women using Hemoglobin A1c versus Diabetes as a High Risk Equivalent

Abstract

Background

It is unclear whether models which include hemoglobin A1c (HbA1c) levels only for diabetic patients improve ability to predict cardiovascular disease (CVD) risk when compared to the currently recommended classification of diabetes as a cardiovascular risk equivalent.

Methods

24,674 women (including 685 diabetic participants at baseline) and 11,280 men (including 563 diabetic participants at baseline) were followed prospectively for CVD. 125 CVD events occurred in diabetic women (666 in non-diabetic women) and 170 events occurred in diabetic men (1382 in non-diabetic men). Models for CVD risk were generated separately for men and women using the traditional CVD risk factors with the addition of a term for HbA1c levels only for diabetic individuals. In diabetic participants, the resulting predicted risks were compared to classification of diabetes as a cardiovascular risk equivalent (10-year CVD-risk of at least 20%).

Results

In women, the models including HbA1c levels in diabetic participants improved the c-statistic by 0.177 (p <0.001) over the risk equivalence model and showed improved reclassification (NRI of 26.7%, p = 0.001). In men, the improvements were more modest but still statistically significant (c-statistic change of 0.039, p=0.015; NRI of 9.2%, p= 0.042). Including HbA1c levels also improved prediction over a dichotomous term for diabetes in women (NRI of 11.8%, p = 0.033) but not in men.

Conclusions

In both women and men with baseline diabetes, we observed significant improvements in predictive ability of CVD risk using models incorporating HbA1c levels compared to classification of diabetes as a cardiovascular risk equivalent.

Diabetes is a well-established risk factor for future cardiovascular disease (CVD) and current treatment guidelines for assessment of CVD risk recommend designation of diabetes as a risk equivalent to prevalent CVD, thus placing all diabetic patients in a high-risk classification.^{1, 2} However, more recent studies suggest that diabetes alone has a consistently lower relative risk of future CVD than a prior myocardial infarction.^{3, 4} Additionally, substantial variability in CVD risk among diabetic patients with higher risk has been demonstrated in those with additional CVD risk factors such as high blood pressure and high cholesterol.^{5, 6} Simulated cost-benefit analyses have suggested that this variability in CVD risk could provide an opportunity for tailored preventive therapy in diabetic patients.⁷

In numerous prospective epidemiologic studies, hemoglobin A1c (HbA1c) has been shown to predict CVD risk in addition to traditional CVD risk factors among individuals with diabetes.⁸ However, whether or not allowing CVD risk associated with diabetes to vary based on HbA1c levels and other CVD risk factors compared with classification of all diabetic patients as high risk would improve CVD risk prediction is uncertain, especially in populations with differing levels of overall CVD risk. This is of clinical relevance since, if superior, a single CVD risk model combining traditional factors with an additional HbA1c term for diabetic patients, for whom this information is already clinically available, could be used irrespective of diabetes status.

To address this question, we used large prospective cohorts of men and women to generate CVD risk models which included HbA1c levels for the diabetic participants. We then compared the predictive ability of the resulting models to classification based on current guidelines for the diabetic participants. In secondary analyses, we also examined the effect of a simple dichotomous term for diabetes in place of HbA1c levels.

Research Design and Methods

Women’s Health Study

Female study participants were members of the Women’s Health Study (WHS). The WHS was designed as a trial of vitamin E and aspirin for the prevention of cardiovascular disease and cancer.⁹ Participants were 39,896 U.S. female health professionals, older than 45 years, and free of major chronic disease at the time of enrollment. All participants of the WHS provided written informed consent and the study was approved by the review board of the Brigham and Women’s Hospital (Boston, Massachusetts). After excluding women age 80 or older and those without complete data or a baseline blood sample, 685 women who were diabetic at baseline were available for model comparisons. For the model generation, 24,674 women were available (including the 685 diabetic women), with median follow-up of 10.2 years (inter quartile range of 9.7 – 10.6).

Physician’s Health Study II

Male study participants were members of the Physician’s Health Study II (PHS II), a randomized trial of beta-carotene, vitamins C and E, and multivitamins in 14,641 male U.S. physicians for the prevention of cardiovascular disease and cancer.¹⁰ All participants of PHS II provided written informed consent and the study was approved by the review board of the Brigham and Women’s Hospital (Boston, Massachusetts). After excluding men age 80 or older and those without complete data, 563 men who were diabetic at baseline were available for the model comparisons. For the model generation, 11,280 men (including the 563 diabetic men) were included, with median follow-up of 11.8 years (inter quartile range 8.7 – 12.2).

Covariate and Outcome Ascertainment

In both WHS and PHS II, information on race, age, parental history of premature myocardial infarction (MI), smoking, blood pressure, and medication use was collected at study baseline by questionnaire. High density lipoprotein and total cholesterol, high sensitivity C-reactive protein and HbA1c were measured using the baseline blood sample, which had been stored in liquid nitrogen until laboratory analysis. HbA1c was measured using the Tina-Quant turbidimetric inhibition immunoassay (Roche Diagnostics, Indianapolis Indiana) standardized to the Diabetes Control and Complications Trial. The coefficient of variation from blinded, simultaneously analyzed, quality controls was 7.2%. Participants were classified as diabetic at baseline if they reported having ever been diagnosed with diabetes. Participants were followed for incident cardiovascular disease (comprised of myocardial infarction, ischemic stroke, coronary revascularization, or cardiovascular death) and events were adjudicated by medical record review using standardized criteria.^{9, 10}

Statistical Methods

All analysis was done separately in the WHS and PHS II cohorts using the structure outlined below. Baseline characteristics were compared using the Kruskal-Wallis test for continuous measures and the chi-squared test for categorical measures.

The cardiovascular risk equivalent model was generated to simulate application of current risk scores and to allow for variation above the 20% 10-year risk cut-off. In order to achieve that, a model using a base set of known CVD risk factors was fit in the non-diabetic participants and applied to the diabetic participants. The non-diabetics were used both to generate stable estimates of risk factor effects and to ensure that risk factor effect estimates were not affected by correlations between traditional risk factors and diabetes status. Then the diabetic participants were assigned the higher of 20% 10-year risk or their risk-equivalent model predicted 10-year risk. The comparison model was fit in the whole population and included the same base set of known CVD risk factors as well as a linear term for HbA1c only in the diabetic participants. HbA1c was set to zero in the non-diabetic participants and linearity was assessed using penalized spline models. Predicted risks were generated in Cox proportional hazards models and compared at 8 years, a time point at which follow-up was >90% complete in both cohorts, for increased stability and all results were extrapolated to 10-year predicted risks for presentation by assuming an exponential survival distribution.

Two base sets of cardiovascular risk factors were used. For the primary base model, we refit the variables (natural log of age, natural log of systolic blood pressure, natural log of total and high density lipoprotein cholesterol, and smoking) from the Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (ATP III) risk score, which is based on the Framingham risk score, to our data to obtain new coefficients.¹ A second base model, including the additional Reynolds risk score (RRS) variables natural log of C-reactive protein and parental history of premature myocardial infarction, was examined using the same structure.¹¹ Calibration, or match between absolute predicted and observed rates, was assessed on the total study population, using the Hosmer-Lemeshow goodness-of-fit test, which divides the predicted risk into deciles and compares the average predicted risk in each decile to the observed risk for the decile.¹²

The estimated predicted risks from the risk equivalent model and the HbA1c model were then compared for discrimination, or ability to rank cases higher then non-cases, using the c-statistic for participants with complete 8 year follow-up.¹³ Reclassification was assessed by comparing the predicted risk for each pair of models across four categories (less than 5% 10-year risk, 5% to less than 10%, 10% to less than 20%, and 20% or higher risk.¹⁴ From the resulting table, we computed the Net Reclassification Improvement (NRI) and the Integrated Discrimination Improvement (IDI) for the participants with complete 8-year follow-up.¹⁵ We also calculated the reclassification calibration statistic for the table, which assesses the match between predicted and observed event rates for each model in each division of the reclassification table, with lower values and higher p-values suggesting better fit.¹⁶

In order to compare the two cohorts as directly as possible, the model comparisons were repeated in a subgroup of diabetic WHS and PHS III participants aged 55 to 70 for whom the distribution of ages was matched by 5 year age category. We also assessed an intermediate model for diabetes with a dichotomous term for diagnosed diabetes at baseline added to the ATP III or RRS covariates, to see if the linear term for HbA1c added predictive value beyond the simpler dichotomous term. We also repeated the entire analysis using 10-fold cross-validation.

All statistical analysis was done using R, version 2.10.0.

Results

Women’s Health Study

Over the follow-up period, 125 cardiovascular events occurred in the 685 diabetic women (103 by 8 years), with an additional 666 events in the non-diabetic women. The diabetic subjects were older with a less favorable risk factor profile compared to the non-diabetic subjects, with the exception of similar rates of smoking and parental history of premature MI (Table 1).

Table 1.

Baseline Characteristics for Women’s Health Study and Physicians Health Study II Participants^*

	Diabetic Participants	Non-Diabetic Participants	P-value†
Women’s Health Study, N	685	23,989
Age, years	55 (50, 62)	52 (48, 58)	<0.001
Systolic blood pressure, mmHg	135 (125, 145)	125 (115, 135)	<0.001
High density lipoprotein cholesterol, mg/dL	42.2 (35.0, 51.5)	52.2 (43.5, 62.7)	<0.001
Total cholesterol, mg/dL	214 (190, 242)	208 (184, 235)	<0.001
C-reactive protein, mg/L	5.0 (2.4, 8.4)	2.0 (0.8, 4.2)	<0.001
Current smoking, %	12.0%	11.5%	0.78
Parental history of premature MI, %	14.9%	12.8%	0.12
HbA1c, %	7.0 (6.0, 8.3)	-	-
Physician’s Health Study II, N	563	11,280
Age, years	67.8 (62.5, 73.0)	63.1 (57.0, 69.8)	<0.001
Systolic blood pressure, mmHg	130 (120, 140)	128 (120, 135)	<0.001
High density lipoprotein cholesterol, mg/dL	38.4 (30.9, 47.9)	42.5 (34.4, 52.3)	<0.001
Total cholesterol, mg/dL	195.0 (173.0, 222.0)	203.1 (180.3, 227.0)	<0.001
C-reactive protein, mg/L	1.2 (0.6, 2.6)	0.9 (0.4, 1.7)	<0.001
Current smoking, %	4.6%	3.2%	0.09
Parental history of premature MI, %	10.8%	10.8%	0.97
HbA1c, %	6.5 (6, 7.4)	-	-

^*Median (25^th percentile, 75^th percentile) unless otherwise noted

^†Kruskal-Wallis used for comparison of continuous characteristics and chi-squared for comparison of dichotomous characteristics

Adding a linear term for HbA1c in the diabetic women to the ATP III or RRS covariates was independently predictive (HR 1.17, p<0.001 for ATP III, HR 1.16, p<0.001 for RRS) and the coefficients for all models are shown in eTable1. There was no evidence of non-linearity. All of the models, including the risk equivalent model, were calibrated in the total participants. The distributions of the 10-year predicted risks in the diabetic women from the ATP III model including HbA1c are shown in Figure 1 part A, with similar results from the RRS model (not shown). As shown, 71.9% of the diabetic WHS participants had a predicted 10-year risk less than 20%.

Figure 1.

Distribution of Predicted 10-Year Cardiovascular Risk in Diabetic Participants from ATP III Models including HbA1c

When compared to classification of baseline diabetes as a cardiovascular risk equivalent in which all diabetic participants are placed in the category of 20% 10-year risk of CVD or higher, 19–20% of diabetic women moved into the lowest predicted 10-year risk category using the HbA1c models (Table 2). The lowest predicted risk categories for both models (<5% 10 year risk) had observed Kaplan-Meier 10-year incident rates of 2.8 and 1.9 % for the ATP III and RRS models which was within the predicted <5% range. The intermediate risk categories had slightly higher than expected risks (15.2% and 17.8% for ATP III and 14.0% and 20.5% for RRS), while the observed rate in the high risk category was over 30% for both groups. Overall, the models with an HbA1c term showed substantial improvement in the diabetic participants (Table 3). The models including HbA1c improved the c-statistic (p <0.001 for both the ATP III and RRS), had a significant NRI of 26.7% for ATP III (p=0.001) and 23.6% for RRS (p=0.003), and had an IDI of 0.072 with p<0.001 for both.

Table 2.

Predicted 10-year Cardiovascular Risk Category in Diabetic Participants for Models Including HbA1c Compared to Classification of Diabetes as a Cardiovascular Risk Equivalent

	Predicted 10-Year Risk Category from Model with HbA1c				Total
	<5%	5 to <10%	10 to <20 %	20+ %
WHS, ATP III
N (%)	135 (19.7%)	155 (22.6%)	203 (29.6%)	192 (28.0%)	685
Kaplan-Meier	2.8 %	15.2 %	17.8 %	32.9 %	18.6%
Event Rate (95% CI)	(0%, 5.9%)	(8.6%, 21.2%)	(11.7%, 23.5%)	(25.2%, 39.9%)	(15.3%, 21.8%)
Cases/Non-Cases at 8 years	3/129	19/134	29/166	52/134	103/563
WHS, RRS
N (%)	132 (19.3%)	151 (22.0%)	211 (30.8%)	191 (27.9%)	685
Kaplan-Meier	1.9 %	14.0 %	20.5 %	31.4 %	18.6%
Event Rate (95% CI)	(0.0%, 4.5%)	(7.6%, 20%)	(14.2%, 26.3%)	(23.7%, 38.3%)	(15.3%, 21.8%)
Cases/Non-Cases at 8 years	2/128	17/130	35/170	49/135	103/563
PHS II, ATP III
N (%)	2 (0.3%)	20 (3.6%)	116 (20.6%)	425 (75.5%)	563
Kaplan-Meier	58% (0.0%,	12.3% (0.0%,	26.4%	34.9%	32.4%
Event Rate (95% CI)	92.6%)	27.0%)	(17.0%, 34.8%)	(29.6%, 39.8%)	(28.0%, 36.6%)
Cases/Non-Cases at 8 years	1/1	2/17	25/88	119/269	147/375
PHS II, RRS
N (%)	3 (0.5%)	23 (4.1%)	116 (20.6%)	421 (74.8%)	563
Kaplan-Meier	39.8%	10.7%	24.2%	35.9%	32.4%
Event Rate (95% CI)	(0.0%, 77.8%)	(0.0%, 23.8%)	(15.1%, 32.4%)	(30.5%, 40.8%)	(28.0%, 36.6%)
Cases/Non-Cases at 8 years	1/2	2/20	23/91	121/262	147/375

WHS = Women’s Health Study, PHS II = Physician’s Health Study II, ATP III = base covariates from Adult Treatment Panel III, RRS = base covariates from Reynolds risk score

Table 3.

Discrimination and Reclassification for the Comparison of Models including HbA1c to Classification of Diabetes as a Cardiovascular Risk Equivalent in Diabetic Participants

	ATP III	RRS
Women’s Health Study
C-statistic for Risk Equivalent	0.515	0.519
C-statistic for model with HbA1c	0.692	0.697
P value for C-statistic comparison	<0.001	<0.001
NRI (p value)	26.7% (0.001)	23.6% (0.003)
IDI (p value)	0.072 (<0.001)	0.072 (<0.001)
Reclassification χ2 for Risk Equivalent (p value)	37.6 (<0.001)	35.9 (<0.001)
Reclassification χ2 for model with HbA1c (p value)	14.4 (0.002)	15.9 (0.001)
Physician’s Health Study II
C-statistic for Risk Equivalent	0.563	0.578
C-statistic for model with HbA1c	0.602	0.605
P value for C-statistic comparison	0.015	0.081
NRI (p value)	9.2 % (0.042)	12.4 % (0.005)
IDI (p value)	0.039 (<0.001)	0.041 (<0.001)
Reclassification χ2 for Risk Equivalent (p value)	26.5 (<0.001)	27.1 (<0.001)
Reclassification χ2 for model with HbA1c (p value)	9.8 (0.002)	6.5 (0.011)

ATP III = base covariates from Adult Treatment Panel III, RRS = base covariates from Reynolds risk score

When a dichotomous diabetes term was used instead of a linear HbA1c term, the pattern of improvement over classification of diabetes as a cardiovascular risk equivalent was similar. However, the HbA1c term showed significant improvement in prediction over models with the diabetes term, with an NRI of 11.8% (p=0.033) in ATP III and an NRI of 10.5% (p=0.044) for the RRS.

Physician’s Health Study II

In the 563 diabetic men, 170 cardiovascular events occurred over follow-up (147 by 8 years), with an additional 1382 events in the non-diabetic men. As in the women, the diabetic participants were older with a less favorable risk factor profile than the non-diabetic participants, with the exception smoking and parental history of premature MI (Table 1). However, the PHS II participants were older overall than the WHS participants (median age 67.8 vs 55.0 for diabetic participants).

Similarly to the women, adding a linear term for HbA1c in the diabetic participants to the ATP III or RRS covariates was independently predictive of CVD risk for men (HR 1.10, p<0.001 for ATP III, HR 1.10, p<0.001 for RRS, additional model information in eTable1). As with the women, all models were calibrated in the total participants and there was no evidence of non-linearity. The distributions of the 10-year predicted risks in the diabetic men from the ATP III model including HbA1c are shown in Figure 1 part B with similar results from the RRS models (not shown). In contrast to the WHS, only 24.5% of the diabetic PHS II participants had a predicted 10-year CVD risk less than 20%.

As seen in the figure, few men were predicted to have low CVD risk, and consequently only 0.4% of the men were reclassified to the lowest risk categories for the ATP III model, with 0.5% reclassified for the RRS model (Table 2). All of the risk categories had higher observed event rates than expected. The models with an HbA1c term showed modest improvement in prediction overall for the diabetic participants compared to the cardiovascular risk equivalent classification (Table 3). There was a significant improvement in the c-statistic only in the ATP III model (p=0.015), an NRI of 9.2% (p=0.042) and 12.4% (p=0.004) for the ATP III and RRS respectively, and an IDI of 0.04 (p<0.001) for both.

When a dichotomous diabetes term was used instead of a linear HbA1c term, a similar pattern of improvement over classification of diabetes as a cardiovascular risk equivalent was seen. There was no significant improvement in the c-statistic for either model, NRI of 11.1% (p=0.009) and 9.4% (p=0.034) for the ATP III and RRS respectively, and an IDI of 0.03 for ATP III and 0.04 for RRS (p<0.001 for both). Unlike in women, there was not consistent improvement in prediction for models using HbA1c compared to models using a dichotomous diabetes term in men (NRI −2.3%, p=0.37, for ATP III and 2.8%, p=0.04 for RRS).

Additional analysis

Limiting to a subset of 255 of the diabetic PHS II and WHS participants matched by 5-year age groups (55 to 59, 60 to 64, and 65 to 70) yielded similar results to the overall analysis. The distribution of predicted risk for the age-matched subset is shown in the eFigure1. While the distributions are more similar than those shown in Figure 1, the percent of PHS II participants with predicted 10-year CVD risk at least 20% remains higher than that in WHS. 10-fold cross-validations also produced similar results to the primary analysis (e.g. NRI of 8.2% for ATP III with HbA1c compared to diabetes as a high-risk equivalent for men, 26.2% for women). Using direct estimates of 10-year risk also produced similar results (data not shown).

Comment

We found that in these large population-based cohorts of both men and women, presence of diabetes alone did not confer a 10 year risk of CVD over 20% and measurement of HbA1c in diabetic subjects improved risk prediction compared to classification as cardiovascular risk equivalent. We found that the improvement in prediction was stronger in the WHS, where the use of HbA1c levels improved prediction both beyond classification as a cardiovascular risk equivalent and, moreover, even when compared to models which included a dichotomous term for diabetes.

These results are consistent with previously published studies suggesting that not all diabetic patients are at high risk of future vascular events.^{5, 6} One response to the range of risk in diabetic patients has been the development of specialized risk models, such as the U.K. Prospective Diabetes Study (UKPDS) model¹⁷, or the use of models with traditional cardiovascular risk factors and a dichotomous term for diabetes diagnosis such as the Atherosclerosis Risk In Communities (ARIC) model.¹⁸ We chose to use a hybrid approach which incorporates an HbA1c term only in the diabetic individuals to generate a single model which can be used in both diabetic and non-diabetic populations. We were also able to show that in the WHS, the model with the HbA1c term improved prediction over the model with the dichotomous term. However, further research in populations with both a larger range of HbA1c values and additional clinical variables such as duration of diabetes and age at onset will be necessary to derive and test an optimal model for CVD prediction in individuals with and without diabetes. The other limitation of a single model as opposed to separate models for diabetic and non-diabetic populations is calibration. We did see lack of calibration in the middle risk categories in both the WHS and PHS II. This is consistent with previous reports of underestimation of absolute risk using non-specialized models.¹⁹ However, as might be expected, the lowest risk group of women appears to be reasonably calibrated.

The differences in results between the two cohorts are worthy of note, and are likely attributable at least in part to the increase in CVD risk with age as well as the delayed risk in women. In the cohorts included in this study, the men were older at baseline with higher rates of CVD, so we were unable to compare populations of men and women with similar overall CVD risk. Even in the age-matched subset, the differences in prediction persisted. There were also differences in the estimated effects of HbA1c between WHS and PHS II, with a larger effect observed in the WHS. This is consistent with prior studies showing a stronger relative risk of CVD for diabetes in women than men.^{20, 21} Given these differences, further replication is needed before the results can be generalized. However, our findings do suggest that the improvement in CVD risk prediction, and possibly calibration, obtained with adding HbA1c levels is highest in lower risk populations. The current high-risk equivalent classification confines the potential for reclassification to movement into lower-risk categories. Thus, the improved performance in low risk populations is due to the reclassification of people to those lower risk categories, while retaining the high risk classification for the subset at high risk. Our analysis demonstrates the potential for improvement in prediction by incorporating the full range of predicted risks.

This study draws upon a large, prospective sample of both men and women for generation of the models with extensive follow-up. While the group of diabetic participants may be viewed as small, there were a substantial number of events in the both the diabetic men and women during follow-up. Diabetes information was collected by self-report, but has been shown to be highly valid in similar populations of health professionals.^{22, 23} Treating diabetic individuals earlier in life may improve later-life risk through reducing cumulative exposure to risk factors, resulting in effects which may not be evident in studies of 10-year risk. Other timescales of risk may also be important in evaluating treatment decisions and were not explored in this analysis, including lifetime risk of CVD which has been shown to be higher for the diabetic population.²⁴ Lifetime risk might also provide additional comparability across populations.

Conclusion

We found improvements in prediction in both men and women with the use of HbA1c level in diabetic subjects compared to classification of diabetes as a cardiovascular risk equivalent. These results may be particularly helpful in light of current discussion around treatment choices for diabetic patients for prevention of cardiovascular disease, including use of statins²⁵ and aspirin²⁶. Our results suggest that the use of HbA1c levels as part of overall CVD risk scores may improve predictive ability in diabetic patients, whose HbA1c levels are routinely measured in clinical practice.