Abstract
The authors studied the incremental value of adding serum cystatin C or creatinine to the Framingham risk score variables (FRSVs) for the prediction of incident cardiovascular disease (CVD) among 6,653 adults without clinical CVD utilizing the Multi-Ethnic Study of Atherosclerosis (2000–2008). CVD events included coronary heart disease, heart failure, stroke, and peripheral arterial disease. Variables were transformed to yield optimal prediction of 6-year CVD events in sex-stratified models with FRSVs alone, FRSVs + cystatin C, and FRSVs + creatinine. Risk prediction in the 3 models was assessed by using the C statistic, and net reclassification improvement was calculated. The mean ages were 61.9 and 64.6 years for individuals with and without diabetes, respectively. After 6 years of follow-up, 447 (7.2%) CVD events occurred. In the total cohort, no significant change in the C statistic was noted with FRSVs + cystatin C and FRSVs + creatinine compared with FRSVs alone, and net reclassification improvement for CVD risk was extremely small and not significant with the addition of cystatin C or creatinine to FRSVs. Similar findings were noted after stratifying by baseline presence of diabetes. In conclusion, the addition of cystatin C or serum creatinine to FRSVs does not improve CVD risk prediction among adults without clinical CVD.
Keywords: cardiovascular diseases, creatinine, cystatin C, risk model
Clinical cardiovascular disease (CVD) prevention relies on the modification of traditional risk factors (hypertension, dyslipidemia, diabetes mellitus, smoking, and so on). The Framingham risk score remains the most frequently utilized prediction model to estimate the risk of developing clinical coronary heart disease or CVD (1–3). The Framingham risk score is based upon the assessment of multiple risk factors in a population free of clinical cardiovascular disease at baseline (1–3). It is increasingly recognized that the reduced glomerular filtration rate (GFR), as determined by serum creatinine, is a strong risk factor for the development of CVD (4, 5). However, serum creatinine levels are affected by factors such as age, sex, race, and muscle mass, and this confounds the association between GFR and cardiovascular risk (6, 7). Cystatin C, a cysteine protease inhibitor with a small molecular unit (13.3 kDa), is freely filtered by glomeruli and subsequently metabolized by the proximal tubules (8). Because cystatin C levels are independent of age, sex, and muscle mass, serum cystatin C may be more sensitive for the detection of mild to moderate decrements in GFR compared with serum creatinine (8). Cystatin C is associated with presence of subclinical CVD and appears to be an independent predictor of death, cardiovascular events, and heart failure among elderly individuals (9–13).
Despite this body of information, the association between measures of GFR and incident CVD among individuals without established CVD at baseline has not been fully investigated (14). Furthermore, because kidney disease cosegregates with multiple cardiovascular risk factors including hypertension, diabetes, and older age, it remains unclear whether measures of kidney function, such as serum cystatin C or creatinine, improve the prediction of CVD events when added to the measures of traditional risk factors, especially in adults without established CVD. In this study, the incremental value of adding either serum cystatin C or creatinine to the Framingham risk score variables (FRSVs) for the prediction of cardiovascular events in a population without baseline clinical CVD was evaluated by using reclassification tables in addition to traditional measures of risk prediction and model fit.
MATERIALS AND METHODS
Population
The Multi-Ethnic Study of Atherosclerosis (MESA) is a population-based study of 6,814 men and women aged 45–84 years, without clinical CVD at baseline, recruited from 6 US communities (Baltimore, Maryland; Chicago, Illinois; Forsyth County, North Carolina; Los Angeles County, California; northern Manhattan, New York; and St. Paul, Minnesota). The main objective of MESA is to determine the characteristics of subclinical CVD and its progression. Adults with symptoms or history of medical or surgical treatment for CVD were excluded. Information on the sampling frame and study design has been reported previously (15). Participants who self-reported their race/ethnicity group as Caucasian or white, African American or black, Chinese, or Spanish/Hispanic/Latino were asked to participate and were enrolled between July 2000 and August 2002. Institutional review board approval was obtained at all MESA sites. After excluding 161 (2.3%) participants with missing data on at least 1 FRSV, cystatin C, or serum creatinine at the baseline examination, we included a total of 6,653 individuals (3,136 men and 3,517 women) in the analysis.
Cardiovascular disease events
Information on new CVD events was obtained by trained personnel who contacted the participants or family members. Self-reported diagnoses were confirmed by reviewing medical records. All CVD events were adjudicated and classified by 2 members of the mortality and morbidity review committee. For a mortality event, both hospital records and family interviews were used to determine whether death was directly due to a CVD event. A CVD endpoint was defined as one of the following clinical events: cardiovascular death, resuscitated cardiac arrest, definite or probable myocardial infarction, definite or probable angina, stroke, definite or probable peripheral arterial disease, or congestive heart failure, similar to the definition of CVD used in the Framingham Heart Study (3). Cardiovascular death was defined as the underlying cause if the death was consistent with a fatal cardiovascular cause in the absence of a known nonatherosclerotic or noncardiac cause of death. Resuscitated cardiac arrest was defined as successful recovery from a full cardiac arrest through cardiopulmonary resuscitation. Definite or probable myocardial infarction was based on a combination of symptoms, abnormal cardiac biomarkers (2 times upper limits of normal), and electrocardiographic findings. Probable angina required, in addition to symptoms, a physician's diagnosis of angina and medical treatment. Definite angina required both symptoms and objective evidence of obstructive coronary artery disease, such as the performance of coronary artery bypass graft surgery or other revascularization procedure, 70% or greater obstruction on coronary angiography, or evidence of ischemia by stress tests. Stroke was defined as present given a rapid onset of a documented focal neurologic deficit lasting 24 hours or until death. Probable peripheral arterial disease required a documented physician's diagnosis and symptoms. Definite peripheral arterial disease required 1 or more other criteria, such as ultrasound evidence of obstruction, an exercise test positive for claudication, revascularization of peripheral arterial disease, amputation for ischemia, an ankle/arm ratio of 0.8 or less, evidence of aortic aneurysm on imaging tests, or a vascular procedure for abdominal aortic aneurysm. Definite or probable congestive heart failure necessitated heart failure symptoms, such as shortness of breath or edema. In addition to symptoms, probable congestive heart failure required a congestive heart failure diagnosis by a physician and the patient's receiving medical treatment for congestive heart failure. Definite congestive heart failure mandated 1 or more other criteria, such as pulmonary edema/congestion by chest radiograph, dilated ventricle or poor left ventricular function by echocardiography or ventriculography, or evidence of left ventricular diastolic dysfunction. Events that occurred within the first 6 years of follow-up were included in the analysis.
Framingham risk score variables
The FRSVs were age, low- and high-density lipoprotein cholesterol, blood pressure, diabetes mellitus, and smoking (1–3). Although the Adult Treatment Panel III risk score excluded diabetes from the equations, we utilized the variables included in the original and the most recent versions of the Framingham risk score including diabetes mellitus (1–3). All MESA participants completed self-administered questionnaires, provided fasting blood samples, and were interviewed and examined by trained research staff. Blood pressure was measured 3 times at 1-minute intervals by using a Dinamap PRO 100 automated oscillometric device (GE Medical Systems Information Technologies, Inc., Milwaukee, Wisconsin). The average of the second and third measurements was used for this analysis. Diabetes mellitus was defined as a self-reported diagnosis, use of insulin or oral hypoglycemic agents, or fasting glucose of 126 mg/dL or higher. Hypertension was defined as self-reported treatment or systolic blood pressure of 140 mm Hg or higher or diastolic blood pressure of 90 mm Hg or higher. All biochemistry assays were performed on plasma or serum drawn in the morning after an overnight fast.
Serum cystatin C and creatinine
Cystatin C measurements were completed by using a BNII nephelometer on plasma specimens (N Latex Cystatin C; Dade Behring, Inc., Deerfield, Illinois) (16). The assay range was 0.195–7.330 mg/L. The reported reference range for cystatin C in young, healthy individuals is 0.53–0.96 mg/L. The ranges of intraassay and interassay coefficients of variation were 2.0%–2.8% and 2.3%–3.1%, respectively. Serum creatinine was measured by using colorimetry with a Vitros 950 analyzer (Johnson & Johnson Clinical Diagnostics, Inc., Rochester, New York). The coefficients of variation for serum creatinine were less than 2%.
Statistical analysis
Models were fitted by using Cox proportional hazards models, restricting predictors to components of the Framingham risk score and adding serum cystatin C levels or creatinine levels (17). To determine the functional form used for each predictor, we used the Box-Cox transformation technique to determine the best fit for each covariate (SAS, version 9.1, software; SAS Institute, Inc., Cary, North Carolina). The optimal transformation within the Box-Cox power transformation includes the logarithm, inverse, square root, and other transformations. Likelihood ratio tests with 95% confidence intervals were calculated to determine the optimal transformation for each continuous variable. When fit was similar for transformations, we chose the simplest form, usually a linear term or log transformation. The inverse of the square root of systolic blood pressure, log base 10 of high-density lipoprotein cholesterol, and the inverse of low-density lipoprotein cholesterol were included as continuous variables. Serum cystatin C and creatinine values were transformed as follows:
Cystatin C: cystatin_c(−0.25) in men, cystatin_c(−0.75) in women
Creatinine: creatinine(−0.5) in men, creatinine(−0.75) in women
These transformed values of cystatin C and creatinine were included in the model as continuous variables. The presence of diabetes and current smoking status were fitted as dichotomous variables.
To compare risk discriminatory values of the prediction models, we calculated the area under the receiver operating characteristic curve (C statistic) and likelihood-based measures for each model. Likelihood-based measures include the −2 log likelihood, the Akaike Information Criterion, and the Bayes Information Criterion, which assess the goodness of fit of the model with lower values indicating better fit (17, 18). Areas under the receiver operating characteristic curves were compared by using an algorithm suggested by DeLong et al. (19). Model calibration was assessed by using the Hosmer-Lemeshow test with the default setting of 10 bins. Potential interactions of diabetes, age greater than 65 years, and race/ethnicity with serum cystatin C and serum creatinine on CVD risk were assessed by fitting interaction terms (e.g., diabetes × serum creatinine and diabetes × serum cystatin C) in Cox proportional hazard models that included FRSVs and either serum cystatin C or serum creatinine. Because the interaction term between diabetes and creatinine on CVD risk was significant among men, we performed subgroup analyses stratified by baseline diabetes and by sex.
MESA participants were classified according to their 6-year CVD risk: low risk (≤5%), intermediate risk (>5%–<10%), and high risk (≥10%). Using reclassification tables, we calculated calibration as the fraction of individuals with and without events who were classified in high- and low-risk groups, respectively. Reclassification table-based measures, such as risk stratification capacity, classification accuracy, and net reclassification improvement, were calculated to evaluate the incremental predictive ability of adding a new marker to the existing risk prediction model (20–23). Risk stratification capacity was defined as the proportion of participants classified as low or high risk (20–22). Classification accuracy was defined as the proportion of participants without events who are assigned to the low predicted risk group plus the proportion of participants with events who are assigned to the high predicted risk group (20–22). Net reclassification improvement was defined as a sum of differences in proportions of individuals moving to a higher risk group minus the proportion of participants moving to a lower risk group among individuals who developed events and the proportion of individuals moving to a lower risk group minus the proportion of individuals moving to a higher risk group among those who did not develop events (23).
RESULTS
The baseline characteristics of the study participants (3,136 men and 3,517 women) stratified by the baseline presence of diabetes are shown in Table 1. A total of 820 participants had baseline diabetes including 7 individuals with type 1 diabetes (use of insulin at the time of initial diagnosis of diabetes and age at diabetes diagnosis <40 years). The MESA cohort demonstrated substantial variability in the distributions of major CVD risk factors. In the total MESA cohort, serum values of cystatin C and creatinine were 0.92 (standard deviation (SD), 0.27) mg/dL and 1.07 (SD, 0.32) mg/dL in men, respectively, and 0.87 (SD, 0.21) mg/dL and 0.85 (SD, 0.20) mg/dL in women, respectively. Serum cystatin C and creatinine were higher in men and women with diabetes compared with men and women without diabetes (Table 1). During 6 years of follow-up, 297 (9.5%) events occurred in men and 180 (5.1%) in women, with 122 events occurring in participants with baseline diabetes. The cardiovascular risk prediction models with FRSVs alone, FRSVs + cystatin C, and FRSVs + creatinine levels are shown in Table 2. When added to the FRSVs, cystatin C was independently associated with occurrence of incident events in both men and women, while creatinine was a significant predictor only in men. The interaction term between cystatin C and diabetes on CVD risk was not significant for men (P = 0.27) or women (P = 0.46). However, the interaction term between diabetes and serum creatinine on CVD risk was significant among men (P = 0.02) but not among women (P = 0.45). Interaction terms between age greater than 65 years and cystatin C and serum creatinine and between race/ethnicity and cystatin C and serum creatinine on CVD risk did not reach statistical significance. Among participants with baseline diabetes, cystatin C (P < 0.0001) and serum creatinine (P = 0.005) were significantly associated with incident CVD among men when added to the FRSVs but were not significant among women (P = 0.17 for cystatin C and P = 0.39 for creatinine).
Table 1.
Baseline Characteristics of Participants by Baseline Presence of Diabetes Mellitus and Gender, The Multi-Ethnic Study of Atherosclerosis, 2000–2002a
| Variables | Diabetes Not Present at Baseline |
Diabetes Present at Baseline |
|||||||||||
| Men (n = 2,709) |
Women (n = 3,124) |
Men (n = 427) |
Women (n = 393) |
||||||||||
| Mean (SD) | No. | % | Mean (SD) | No. | % | Mean (SD) | No. | % | Mean (SD) | No. | % | ||
| Age, years | 61.9 (10.3) | 61.8 (10.2) | 64.6 (9.3) | 64.7 (9.5) | |||||||||
| Body mass index, kg/m2 | 27.6 (4.3) | 28.3 (6.0) | 29.5 (4.8) | 32.0 (6.6) | |||||||||
| Current smoking | 382 | 14.1 | 366 | 11.7 | 69 | 16.2 | 38 | 9.7 | |||||
| Systolic blood pressure, mm Hg | 125.6 (18.8) | 126.6 (22.9) | 131.5 (20.4) | 135.9 (23.1) | |||||||||
| Diastolic blood pressure, mm Hg | 75.3 (9.4) | 69.4 (10.3) | 75.0 (9.5) | 69.3 (9.8) | |||||||||
| Total cholesterol, mg/dL | 188.6 (33.5) | 199.4 (34.0) | 181.2 (36.8) | 193.8 (37.1) | |||||||||
| LDL cholesterol, mg/dL | 117.8 (30.5) | 118.1 (31.6) | 109.2 (33.3) | 114.3 (33.3) | |||||||||
| HDL cholesterol, mg/dL | 45.7 (11.8) | 57.2 (15.3) | 42.6 (10.7) | 50.9 (13.3) | |||||||||
| Race/ethnicity | |||||||||||||
| White | 1,135 | 41.9 | 1,268 | 40.6 | 92 | 21.6 | 64 | 16.3 | |||||
| Chinese | 330 | 12.2 | 366 | 11.7 | 49 | 11.5 | 41 | 10.4 | |||||
| Black | 675 | 24.9 | 862 | 27.6 | 153 | 35.8 | 163 | 41.5 | |||||
| Hispanic | 569 | 21.0 | 628 | 20.1 | 133 | 31.2 | 125 | 31.8 | |||||
| Cystatin C, mg/L | 0.91 (0.20) | 0.87 (0.19) | 0.97 (0.52) | 0.93 (0.36) | |||||||||
| Creatinine, mg/dL | 1.07 (0.22) | 0.85 (0.16) | 1.09 (0.68) | 0.86 (0.36) | |||||||||
Abbreviations: HDL, high density lipoprotein; LDL, low density lipoprotein; SD, standard deviation.
Data in continuous variables are shown as mean (SD), and data in categorical values are shown as number and percent.
Table 2.
Transformed Variables and Beta Coefficients in the 3 Risk Models, The Multi-Ethnic Study of Atherosclerosis, 2000–2008
| Variablesa | Risk Model With FRSVs Alone |
Risk Model With FRSVs + Cystatin C |
Risk Model With FRSVs + Creatinine |
|||||||||
| Men |
Women |
Men |
Women |
Men |
Women |
|||||||
| Beta Coefficient | P Value | Beta Coefficient | P Value | Beta Coefficient | P Value | Beta Coefficient | P Value | Beta Coefficient | P Value | Beta Coefficient | P Value | |
| Age | 0.049 | <0.0001 | 0.064 | <0.0001 | 0.039 | <0.0001 | 0.051 | <0.0001 | 0.047 | <0.0001 | 0.064 | <0.0001 |
| SBP | −57.808 | <0.0001 | −52.339 | 0.0024 | −55.939 | <0.0001 | −49.824 | <0.0001 | −55.998 | <0.0001 | −52.312 | <0.0001 |
| Diabetes mellitus | 0.630 | <0.0001 | 0.856 | 0.0001 | 0.645 | <0.0001 | 0.851 | <0.0001 | 0.656 | <0.0001 | 0.858 | <0.0001 |
| Current smoking | 0.601 | 0.0001 | 0.804 | 0.0002 | 0.576 | 0.0002 | 0.748 | 0.0005 | 0.629 | <0.0001 | 0.804 | 0.0002 |
| HDL cholesterol | −0.804 | 0.0011 | −0.389 | 0.678 | −0.532 | 0.0342 | −0.188 | 0.5309 | −0.771 | 0.0018 | −0.389 | 0.1931 |
| LDL cholesterol | −57.203 | 0.0088 | −20.814 | <0.0001 | −61.614 | 0.0048 | −19.890 | 0.4547 | −56.267 | 0.0097 | −20.723 | 0.4398 |
| Cystatin C | NA | NA | NA | NA | −4.856 | <0.0001 | −1.551 | 0.0007 | NA | NA | NA | NA |
| Creatinine | NA | NA | NA | NA | NA | NA | NA | NA | −1.409 | 0.0111 | −0.028 | 0.949 |
Abbreviations: FRSV, Framingham risk score variable; HDL, high density lipoprotein; LDL, low density lipoprotein; NA, not available; SBP, systolic blood pressure.
In the models, age was a continuous variable, and diabetes mellitus and current smoking were dichotomous variables. Other variables were transformed as follows: SBP: 1/√SBP; HDL cholesterol: log10 HDL cholesterol; LDL cholesterol: 1/LDL cholesterol; cystatin C: cystatin C(−0.25) in men and cystatin C(−0.75) in women; and creatinine: creatinine(−0.5) in men and creatinine(−0.75) in women.
Comparisons of overall model fitness for the 3 models stratified by sex and by baseline diabetes are shown in Tables 3–5. In the total cohort, the addition of cystatin C modestly improved all measures of risk prediction (C statistic, −2 log likelihood ratios, the Akaike Information Criterion, and the Bayes Information Criterion) in men, and modest improvement in the C statistic was noted among women with the addition of cystatin C to FRSVs (Table 3). In contrast, the addition of serum creatinine to FRSVs did not change the overall prediction of incident CVD risk in men or women in the total cohort (Table 3). Among male participants with baseline diabetes, the C statistic improved substantially with the addition of cystatin C to FRSVs compared with the FRSVs-alone model (Table 4). Among women, no substantial change in the C statistic was noted with the addition of cystatin C to the FRSVs compared with the model with FRSVs alone regardless of diabetes status. The addition of serum creatinine to FRSVs actually decreased the C statistic by 0.16 among women without baseline diabetes (Table 5).
Table 3.
Measures of Model Fit in the Total Cohort, The Multi-Ethnic Study of Atherosclerosis, 2000–2008
| P Valuea | C Statistic | −2 Log Likelihood | AIC | BIC | |
| Male (n = 3,136) | |||||
| FRSVs alone | <0.0001 | 0.722 | 1,823.68 | 1,827.68 | 1,839.78 |
| FRSVs + cystatin C | <0.0001 | 0.733 | 1,806.0** | 1,810.0 | 1,822.1 |
| FRSVs + creatinine | <0.0001 | 0.724 | 1,814.98* | 1,818.98 | 1,831.08 |
| Female (n = 3,517) | |||||
| FRSVs alone | <0.0001 | 0.758 | 1,306.59 | 1,310.59 | 1,322.92 |
| FRSVs + cystatin C | <0.0001 | 0.770 | 1,307.91 | 1,311.91 | 1,324.24 |
| FRSVs + creatinine | <0.0001 | 0.758 | 1,306.64 | 1,310.64 | 1,322.97 |
Abbreviations: AIC, Akaike Information Criterion, BIC, Bayes Information Criterion; FRSV, Framingham risk score variable.
* P < 0.01; **P < 0.001.
Hosmer-Lemeshow test.
Table 4.
Measures of Model Fit Among Participants With Diabetes, The Multi-Ethnic Study of Atherosclerosis, 2000–2008
| P Valuea | C Statistic | -2 Log Likelihood | AIC | BIC | |
| Male (n = 427) | |||||
| FRSVs alone | 0.5827 | 0.647 | 396.874 | 400.874 | 408.988 |
| FRSVs + cystatin C | 0.5036 | 0.733 | 396.874 | 400.874 | 408.988 |
| FRSVs + creatinine | 0.3390 | 0.675 | 396.874 | 400.874 | 408.988 |
| Female (n = 393) | |||||
| FRSVs alone | 0.3408 | 0.690 | 287.765 | 291.765 | 299.713 |
| FRSVs + cystatin C | 0.2746 | 0.699 | 287.765 | 291.765 | 291.713 |
| FRSV + creatinine | 0.7901 | 0.697 | 287.765 | 291.761 | 299.713 |
Abbreviations: AIC, Akaike Information Criterion, BIC, Bayes Information Criterion; FRSV, Framingham risk score variable.
Hosmer-Lemeshow test.
Table 5.
Measures of Model Fit Among Participants Without Diabetes, The Multi-Ethnic Study of Atherosclerosis, 2000–2008
| P Valuea | C Statistic | −2 Log Likelihood | AIC | BIC | |
| Male (n = 2,709) | |||||
| FRSVs alone | 0.0263 | 0.716 | 1,184.062 | 1,188.062 | 1,199.871 |
| FRSVs + cystatin C | 0.0005 | 0.716 | 1,183.635 | 1,187.635 | 1,199.444 |
| FRSVs + creatinine | 0.0492 | 0.716 | 1,184.414 | 1,188.414 | 1,200.223 |
| Female (n = 3,124) | |||||
| FRSVs alone | 0.0557 | 0.758 | 848.740 | 852.740 | 864.834 |
| FRSVs + cystatin C | 0.0169 | 0.751 | 845.012 | 849.012 | 861.106 |
| FRSVs + creatinine | 0.0892 | 0.742 | 849.003 | 853.003 | 865.097 |
Abbreviations: AIC, Akaike Information Criterion, BIC, Bayes Information Criterion; FRSV, Framingham risk score variable.
Hosmer-Lemeshow test.
Tables 6 and 7 compare risk stratification between the FRSVs and FRSVs + cystatin C models, as well as the FRSVs and FRSVs + creatinine models, respectively. Calibration of the risk prediction models can be assessed by comparing the proportions of events in the margins of Tables 6 and 7 with the corresponding row and column labels. Six-year event rates in low-, intermediate-, and high-risk groups were 1.6%, 5.9%, and 13.9%, respectively, in the model with the FRSVs alone and 1.8%, 5.1%, and 14.5%, respectively, in the model with FRSVs + cystatin C. In the model with FRSVs + creatinine, the 6-year event rates in low-, intermediate-, and high-risk groups were 1.6%, 5.9%, and 13.9%, respectively. Thus, the calibrations of the 3 models were very similar. Comparisons of the models with FRSVs alone, FRSVs + cystatin C, and FRSVs + creatinine led to 870 (13.1%) and 245 (3.7%) individuals stratified to other risk groups, respectively. Risk stratification capacity was similar across the models with FRSVs (72.6%), FRSVs + cystatin C (72.0%), and FRSVs + creatinine (72.1%) when judged by risk or when judged by classification accuracy (41.1%, 41.3%, and 40.4% in the FRSVs, FRSVs + cystatin C, and FRSVs + creatinine models, respectively) (Table 8). Net reclassification improvement was very small with the addition of cystatin C to FRSVs (0.016; P = 0.48) and slightly negative with the addition of serum creatinine to FRSVs (−0.009; P = 0.49) (Table 8). The addition of cystatin C or serum creatinine to FRSVs did not substantially impact risk stratification in MESA participants with or without baseline diabetes. The net reclassification improvement was less than 1% for FRSVs + cystatin C and for FRSVs + creatinine in the cohort with baseline diabetes. No meaningful impact on CVD risk prediction was noted after the addition of cystatin C or creatinine to FRSVs in racial/ethnic subgroups (Table 8).
Table 6.
Risk Stratification Table of Framingham Risk Score Variables Alone Versus Those With Cystatin C, The Multi-Ethnic Study of Atherosclerosis, 2000–2008
| 6-Year Risk in Model With Cystatin C | 6-Year Risk in Model Without Cystatin C |
|||||||||||
| 0–5% |
>5–<10% |
≥10% |
Total |
|||||||||
| No. of Participants | No. of Events | Observed 6-Year Event Risk, % | No. of Participants | No. of Events | Observed 6-Year Event Risk, % | No. of Participants | No. of Events | Observed 6-Year Event Risk, % | No. of Participants | No. of Events | Observed 6-Year Event Risk, % | |
| 0–5% | 2,229 | 32 | 1.4 | 216 | 11 | 5.1 | 4 | 0 | 0 | 2,449 | 43 | 1.8 |
| >5–<10% | 213 | 7 | 3.3 | 1,409 | 70 | 5 | 240 | 18 | 7.5 | 1,862 | 95 | 5.1 |
| ≥10% | 2 | 0 | 0 | 201 | 26 | 12.9 | 2,139 | 313 | 14.6 | 2,342 | 339 | 14.5 |
| Total | 2,444 | 39 | 1.6 | 1,826 | 107 | 5.9 | 2,383 | 331 | 13.9 | 6,653 | 477 | 7.2 |
Table 7.
Risk Stratification Table of Framingham Risk Score Variables Alone Versus Those Without Serum Creatinine, The Multi-Ethnic Study of Atherosclerosis, 2000–2008
| 6-Year Risk in Model With Cystatin C | 6-Year Risk in Model Without Cystatin C |
|||||||||||
| 0–5% |
>5–<10% |
≥10% |
Total |
|||||||||
| No. of Participants | No. of Events | Observed 6-Year Event Risk, % | No. of Participants | No. of Events | Observed 6-Year Event Risk, % | No. of Participants | No. of Events | Observed 6-Year Event Risk, % | No. of Participants | No. of Events | Observed 6-Year Event Risk, % | |
| 0–5% | 2,388 | 38 | 1.6 | 50 | 2 | 4 | 0 | 0 | 2,438 | 40 | 1.6 | |
| >5–<10% | 56 | 1 | 1.8 | 1,717 | 100 | 5.8 | 80 | 9 | 11.3 | 1,853 | 110 | 5.9 |
| ≥10% | 0 | 0 | 59 | 5 | 8.5 | 2,303 | 322 | 14 | 2,362 | 327 | 13.8 | |
| Total | 2,444 | 39 | 1.6 | 1,826 | 107 | 5.9 | 2,383 | 331 | 13.9 | 6,653 | 477 | 7.2 |
Table 8.
Reclassification Measures in the Total Cohort, Participants With Diabetes, and Participants in Racial/Ethnic Subgroups, The Multi-Ethnic Study of Atherosclerosis, 2000–2008
| Risk Stratification Capacity, % | Classification Accuracy, % | Net Reclassification Improvement, % | P Value | |
| Total MESA cohort | ||||
| FRSVs | 72.6 | 41.1 | NA | |
| FRSVs + cystatin C | 72.0 | 41.3 | 0.016 | 0.48 |
| FRSVs + creatinine | 72.1 | 40.4 | −0.009 | 0.49 |
| Diabetes | ||||
| FRSVs | 82.3 | 18.7 | NA | |
| FRSVs + cystatin C | 81.2 | 19.3 | 0.0069 | 0.49 |
| FRSVs + creatinine | 81.6 | 18.8 | −0.029 | 0.47 |
| Whites | ||||
| FRSVs | 71.1 | 37.5 | NA | |
| FRSVs + cystatin C | 71.2 | 37.6 | −0.0084 | 0.49 |
| FRSVs + creatinine | 72.0 | 38.1 | 0.0025 | 0.49 |
| African Americans | ||||
| FRSVs | 72.8 | 38.7 | NA | |
| FRSVs + cystatin C | 72.5 | 39.3 | 0.0022 | 0.50 |
| FRSVs + creatinine | 73.2 | 39.4 | −0.29531 | 0.32 |
| Hispanics | ||||
| FRSVs | 81.1 | 46.7 | NA | |
| FRSVs + cystatin C | 78.4 | 46.1 | −0.0017 | 0.50 |
| FRSVs + creatinine | 78.8 | 45.7 | −0.0067 | 0.50 |
| Chinese | ||||
| FRSVs | 71.9 | 54.1 | NA | |
| FRSVs + cystatin C | 72.1 | 53.9 | −0.00531 | 0.48 |
| FRSVs + creatinine | 72.3 | 53.7 | −0.01194 | 0.50 |
Abbreviations: FRSV, Framingham risk score variable; MESA, Multi-Ethnic Study of Atherosclerosis; NA, not available.
DISCUSSION
In this cohort of adults without clinical CVD, cystatin C was significantly associated with CVD events after adjustment for FRSVs, while serum creatinine was not. This may be a reflection of cystatin C being a more accurate biomarker of kidney function compared with serum creatinine. It is also possible that variability in cystatin C levels reflects other factors, such as inflammation, that may influence CVD risk. However, the addition of cystatin C or serum creatinine to FRSVs did not substantially affect CVD risk prediction in this cohort. This may be explained by the high prevalence of traditional CVD risk factors among individuals with chronic kidney disease. Moreover, the majority of kidney disease cases are individuals over the age of 65 years, and this age group will have a higher CVD risk compared with younger individuals. Thus, traditional risk factors likely mediate much of the association between kidney disease and CVD risk.
Use of the glomerular filtration rate (GRF) to improve CVD risk prediction has been examined in several previous studies. Weiner et al. (24) tested the impact of adding an indicator of chronic kidney disease (defined as an estimated GFR of <60 mL/minute/1.73 m2) to the Framingham equations (sex- and race-specific models) to predict 5-year cardiac events and mortality among individuals aged 45–74 years without preexisting cardiovascular disease (24). Although the presence of chronic kidney disease was an important predictor of the composite outcome, particularly among African Americans, it did not improve discrimination of cardiac events or mortality regardless of gender and race. Others have investigated the use of estimated GFR and spot urine albumin/creatinine ratios to improve prediction of cardiovascular mortality in a Norwegian community-based cohort, and modest improvement was noted after these terms were added to traditional CVD risk factors including age, diabetes, and blood pressure (25).
Risk prediction models are commonly evaluated with the Hosmer-Lemeshow test and the receiver operating characteristic curve or C statistic, standard tools for evaluating the calibration and discrimination value of screening markers, respectively (24, 26, 27). The significant P values for the Hosmer-Lemeshow test in this study may reflect the sensitivity of this test to large sample sizes. Because of this limitation, the Hosmer-Lemeshow test should be supplemented with additional measures of model calibration when the sample size is large (28, 29). We further assessed model performance with reclassification tables (20–22, 28, 29). In this study, the reclassification tables demonstrated that all 3 risk prediction models (FRSVs, FRSVs + cystatin C, and FRSVs + creatinine) appropriately stratified the MESA participants into low-, intermediate-, or high-risk categories.
The strengths of our study include the racial/ethnic diversity of the MESA participants. In addition, CVD events were defined by using previously established criteria and were adjudicated (3). The potential study limitations include a short follow-up period (6 years), especially given that the most recently described Framingham risk score predicted CVD events during a 12-year follow-up period (3). However, the objective of this study was to compare the predictive value of the FRSVs with and without cystatin C or serum creatinine in a population without clinical CVD.
In conclusion, we found that cystatin C remained significantly associated with incident CVD events after adjustment for FRSVs in adults without baseline clinical CVD while creatinine did not. However, the addition of cystatin C or serum creatinine to FRSVs did not substantially change the overall CVD risk prediction or improve the classification of individuals to low or high CVD risk categories. It is possible that these findings may not be applicable to high risk groups, such as populations with older age or chronic kidney disease. Future studies should examine whether the addition of serum creatinine and cystatin C to traditional cardiovascular risk factors improves CVD risk prediction in the high risk groups.
Acknowledgments
Author affiliations: Division of Cardiology, Department of Medicine, Loyola University Medical Center, Maywood, Illinois (Hiroki Ito, Ivan V. Pacold); Department of Preventive Medicine, Loyola University Medical Center, Maywood, Illinois (Ramon Durazo-Arvizu, Holly Kramer); Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois (Kiang Liu); Department of Epidemiology and Biostatistics, San Francisco Veterans Affairs Medical Center, San Francisco, California (Michael G. Shilipak); Department of Epidemiology and Prevention, Wake Forest University School of Medicine, Winston-Salem, North Carolina (David C. Goff, Jr.); and Department of Pathology and Laboratory Medicine, University of Vermont College of Medicine, Burlington, Vermont (Russell P. Tracy).
This research was supported by contracts N01-HC-95159 through N01-HC-95165 and N01-HC-95167 from the National Heart, Lung, and Blood Institute, Bethesda, Maryland.
The authors thank Guichan Cao for her contributions to the statistical analysis.
The study results were presented at American Heart Association Annual Scientific Sessions on November 16, 2010 (Chicago, Illinois).
Conflict of interest: none declared.
Glossary
Abbreviations
- CVD
cardiovascular disease
- FRSF
Framingham risk score variable
- GFR
glomerular filtration rate
- MESA
Multi-Ethnic Study of Atherosclerosis
- SD
standard deviation
References
- 1.Wilson PW, D'Agostino RB, Levy D, et al. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97(18):1837–1847. doi: 10.1161/01.cir.97.18.1837. [DOI] [PubMed] [Google Scholar]
- 2.Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults. Executive summary of the Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) JAMA. 2001;285(19):2486–2497. doi: 10.1001/jama.285.19.2486. [DOI] [PubMed] [Google Scholar]
- 3.D'Agostino RB, Sr, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;117(6):743–753. doi: 10.1161/CIRCULATIONAHA.107.699579. [DOI] [PubMed] [Google Scholar]
- 4.Sarnak MJ, Levey AS, Schoolwerth AC, et al. Kidney disease as a risk factor for development of cardiovascular disease: a statement from the American Heart Association Councils on Kidney in Cardiovascular Disease, High Blood Pressure Research, Clinical Cardiology, and Epidemiology and Prevention. Circulation. 2003;108(17):2154–2169. doi: 10.1161/01.CIR.0000095676.90936.80. [DOI] [PubMed] [Google Scholar]
- 5.Weiner DE, Tighiouart H, Amin MG, et al. Chronic kidney disease as a risk factor for cardiovascular disease and all-cause mortality: a pooled analysis of community-based studies. J Am Soc Nephrol. 2004;15(5):1307–1315. doi: 10.1097/01.asn.0000123691.46138.e2. [DOI] [PubMed] [Google Scholar]
- 6.Levey AS, Coresh J, Balk E, et al. National Kidney Foundation practice guidelines for chronic kidney disease: evaluation, classification, and stratification. National Kidney Foundation. Ann Intern Med. 2003;139(2):137–147. doi: 10.7326/0003-4819-139-2-200307150-00013. [DOI] [PubMed] [Google Scholar]
- 7.Levey AS, Eckardt KU, Tsukamoto Y, et al. Definition and classification of chronic kidney disease: a position statement from Kidney Disease: Improving Global Outcomes (KDIGO) Kidney Int. 2005;67(6):2089–2100. doi: 10.1111/j.1523-1755.2005.00365.x. [DOI] [PubMed] [Google Scholar]
- 8.Coll E, Botey A, Alvarez L, et al. Serum cystatin C as a new marker for noninvasive estimation of glomerular filtration rate and as a marker for early renal impairment. Am J Kidney Dis. 2000;36(1):29–34. doi: 10.1053/ajkd.2000.8237. [DOI] [PubMed] [Google Scholar]
- 9.Seliger SL, Longstreth WT, Jr, Katz R, et al. Cystatin C and subclinical brain infarction. J Am Soc Nephrol. 2005;16(12):3721–3727. doi: 10.1681/ASN.2005010006. [DOI] [PubMed] [Google Scholar]
- 10.Ix JH, Katz R, De Boer IH, et al. Association of chronic kidney disease with the spectrum of ankle brachial index: The CHS (Cardiovascular Health Study) J Am Coll Cardiol. 2009;54(13):1176–1184. doi: 10.1016/j.jacc.2009.06.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Shlipak MG, Sarnak MJ, Katz R, et al. Cystatin C and the risk of death and cardiovascular events among elderly persons. N Engl J Med. 2005;352(20):2049–2060. doi: 10.1056/NEJMoa043161. [DOI] [PubMed] [Google Scholar]
- 12.Ix JH, Shlipak MG, Chertow GM, et al. Association of cystatin C with mortality, cardiovascular events, and incident heart failure among persons with coronary heart disease: data from the Heart and Soul Study. Circulation. 2007;115(2):173–179. doi: 10.1161/CIRCULATIONAHA.106.644286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Sarnak MJ, Katz R, Stehman-Breen CO, et al. Cystatin C concentration as a risk factor for heart failure in older adults. Cardiovascular Health Study. Ann Intern Med. 2005;142(7):497–505. doi: 10.7326/0003-4819-142-7-200504050-00008. [DOI] [PubMed] [Google Scholar]
- 14.Schiffrin EL, Lipman ML, Mann JF. Chronic kidney disease: effects on the cardiovascular system. Circulation. 2007;116(1):85–97. doi: 10.1161/CIRCULATIONAHA.106.678342. [DOI] [PubMed] [Google Scholar]
- 15.Bild DE, Bluemke DA, Burke GL, et al. Multi-Ethnic Study of Atherosclerosis: objectives and design. Am J Epidemiol. 2002;156(9):871–881. doi: 10.1093/aje/kwf113. [DOI] [PubMed] [Google Scholar]
- 16.Erlandsen EJ, Randers E, Kristensen JH. Evaluation of the Dade Behring N Latex Cystatin C assay on the Dade Behring Nephelometer II System. Scand J Clin Lab Invest. 1999;59(1):1–8. doi: 10.1080/00365519950185940. [DOI] [PubMed] [Google Scholar]
- 17.Burnham KP, Anderson DR. Model Selection and Inference: A Practical Information-Theoretic Approach. New York, NY: Springer-Verlag; 1998. [Google Scholar]
- 18.Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York, NY: Springer-Verlag; 2001. [Google Scholar]
- 19.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845. [PubMed] [Google Scholar]
- 20.Cook NR, Buring JE, Ridker PM. The effect of including C-reactive protein in cardiovascular risk prediction models for women. Ann Intern Med. 2006;145(1):21–29. doi: 10.7326/0003-4819-145-1-200607040-00128. [DOI] [PubMed] [Google Scholar]
- 21.Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928–935. doi: 10.1161/CIRCULATIONAHA.106.672402. [DOI] [PubMed] [Google Scholar]
- 22.Janes H, Pepe MS, Gu W. Assessing the value of risk predictions by using risk stratification tables. Ann Intern Med. 2008;149(10):751–760. doi: 10.7326/0003-4819-149-10-200811180-00009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Pencina MJ, D'Agostino RB, Sr, D'Agostino RB, Jr, et al. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–172. doi: 10.1002/sim.2929. [DOI] [PubMed] [Google Scholar]
- 24.Weiner DE, Tighiouart H, Griffith JL, et al. Kidney disease, Framingham risk scores, and cardiac and mortality outcomes. Am J Med. 2007;120(6):552.e1–552.e8. doi: 10.1016/j.amjmed.2006.05.054. [DOI] [PubMed] [Google Scholar]
- 25.Hallan S, Astor B, Romundstad S, et al. Association of kidney function and albuminuria with cardiovascular mortality in older vs. younger individuals: the HUNT II Study. Arch Intern Med. 2007;167(22):2490–2496. doi: 10.1001/archinte.167.22.2490. [DOI] [PubMed] [Google Scholar]
- 26.Folsom AR, Chambless LE, Ballantyne CM, et al. An assessment of incremental coronary risk prediction using C-reactive protein and other novel risk markers: the Atherosclerosis Risk in Communities Study. Arch Intern Med. 2006;166(13):1368–1373. doi: 10.1001/archinte.166.13.1368. [DOI] [PubMed] [Google Scholar]
- 27.de Ruijter W, Westendorp RG, Assendelft WJ, et al. Use of Framingham risk score and new biomarkers to predict cardiovascular mortality in older people: population based observational cohort study [electronic article] BMJ. 2009;338 doi: 10.1136/bmj.a3083. a3083. (doi:10.1136/bmj.a3083) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit Care Med. 2007;35(9):2052–2056. doi: 10.1097/01.CCM.0000275267.64078.B0. [DOI] [PubMed] [Google Scholar]
- 29.Marcin JP, Romano PS. Size matters to a model's fit. Crit Care Med. 2007;35(9):2212–2213. doi: 10.1097/01.CCM.0000281522.70992.EF. [DOI] [PubMed] [Google Scholar]