Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Nov 1.
Published in final edited form as: Am J Kidney Dis. 2012 Jul 25;60(5):795–803. doi: 10.1053/j.ajkd.2012.06.015

Estimated GFR and Incident Cardiovascular Disease Events in American Indians: The Strong Heart Study

Nawar M Shara 1,2,3, Hong Wang 1,2, Mihriye Mete 1,2, Yaman Rai Al-Balha 1,4, Nameer Azalddin 1,5, Elisa T Lee 6, Nora Franceschini 7, Stacey E Jolly 8, Barbara V Howard 1,2,3, Jason G Umans 1,2,3
PMCID: PMC3473098  NIHMSID: NIHMS396691  PMID: 22841159

Abstract

Background

In populations with high prevalence of diabetes and obesity, estimating glomerular filtration rate (GFR) by using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation may predict cardiovascular disease risk better than by using the Modification of Diet in Renal Disease (MDRD) Study equation.

Study design

Longitudinal cohort study comparing the association of GFR estimated using either the CKD-EPI or MDRD Study equations with incident cardiovascular disease outcomes.

Setting and participants

American Indians participating in the Strong Heart Study, a longitudinal population-based cohort with high prevalences of diabetes, cardiovascular disease, and CKD.

Predictor or factor

eGFR predicted using the CKD-EPI and MDRD Study equations.

Outcomes

Fatal and nonfatal cardiovascular events, consisting of coronary heart disease, stroke, and heart failure.

Measurements

The association between eGFR and outcomes was explored in Cox proportional hazards models, adjusted for traditional risk factors and albuminuria; the net reclassification index and integrated discrimination improvement were determined for the CKD-EPI versus MDRD Study equations.

Results

Among 4549 participants, diabetes was present in 45%, cardiovascular disease in 7%, and stage 3–5 CKD in 10%. Over a median of 15 years, there were 1280 cases of incident CVD, 929 of incident coronary heart disease, 305 of incident stroke, and 381 of incident heart failure. Reduced eGFR (<90 mL/min/1.73 m2) was associated with adverse events in most models. Compared with the MDRD Study equation, the CKD-EPI equation correctly reclassified 17.0% of 2,151 participants without incident CVD to a lower risk (higher eGFR) category and 1.3% (n=28) were incorrectly reclassified to a higher risk (lower eGFR) category.

Limitations

Single measurements of eGFR and albuminuria at study visits.

Conclusions

Although eGFR based on either equation had similar associations with incident cardiovascular disease, coronary heart disease, stroke, and heart failure events, among those not having events, reclassification of participants to eGFR categories was superior using the CKD-EPI equation compared with the MDRD Study equation.

Index words: cardiovascular disease risk, chronic kidney disease, estimated glomerular filtration rate, Strong Heart Study


Early identification of chronic kidney disease (CKD) may lead to better targeting of prevention efforts, allocation of health-care resources, and patient outcomes. Decreased estimated glomerular filtration rate (eGFR), creatinine clearance, or the presence of micro- or macroalbuminuria each predict cardiovascular disease (CVD).1,2 Each of the above measures of CKD also has been shown to predict incident CVD in American Indians, a population with high prevalence of obesity, diabetes mellitus, and incident CVD events.1 In the absence of measured GFR, we often depend on serum creatinine (SCr)-based eGFR to assess kidney function, particularly in population-based studies. Our previous work, however, showed that adding albuminuria to SCr-based eGFR may help identify individuals at higher risk for coronary heart disease (CHD).3 Since then, the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) developed an equation, also based on SCr measures, to improve GFR estimation.4 Several studies have suggested that the CKD-EPI equation serves as a better risk predictor than the Modification of Diet in Renal Disease (MDRD) Study equation in some populations and improves overall risk reclassification compared with the MDRD Study equation.5,6,7,8, A recent study showed similar results with all-cause mortality, CVD, and end-stage renal disease mortality as outcomes.9 Thus, the CKD-EPI equation may improve CVD risk prediction, compared with estimates based on the MDRD Study equation. In this article, we use SCr values from the Strong Heart Study (SHS) Phase 1 examination (1989–1991) that were recalibrated to an isotope dilution mass spectrometry (IDMS)-traceable SCr assay to estimate GFR using the IDMS-traceable 4-variable MDRD Study and CKD-EPI equations. Our aim was to examine the performance of the two equations in predicting CVD, CHD, heart failure (HF), and stroke in this American Indian population with high baseline prevalence of obesity, diabetes, and CKD.

METHODS

Study Population and Variables

Data from the SHS, a population-based longitudinal study of CVD risk in American Indians, ages 45–74 years, were used for the current analysis. The SHS was initiated in 1988 to investigate CVD and its risk factors in geographically diverse groups of American Indians. The Indian Health Service, institutional review boards, participating tribes, and the MedStar Health Research Institute approved the study. All participants provided informed consent. The SHS design, survey methods, and laboratory techniques have been published.10 The SHS cohort of 4549 includes men and women who were seen at the first (1989–1991), second (1993–1995), and third (1998–2000) examinations.

Of the 4549 SHS participants, those missing SCr values (n=173) and/or having prevalent CVD (n=331) at the baseline examination were excluded. Those with SHS IDMS-calibrated SCr <0 (n=1) and extreme eGFR values calculated by using the IDMS-traceable 4-variable MDRD Study (>2000 mL/min/1.73m2) (n=3) also were excluded, leaving 4081 participants (40% male) for the current analysis. Criteria used to define definite fatal myocardial infarction (MI), stroke, CHD, and nonfatal CVD have been published, as have methods for ascertaining incident CVD events.10,11,12 Of the 4081 participants, 1280 (31%) developed incident CVD during a median 15-yr follow-up.

Baseline Examination and Laboratory Measures

Baseline and follow-up examinations consisted of a personal interview and physical examination. Baseline demographic information, medical history, and smoking status were collected during the interview. Height, weight, waist and hip circumferences, systolic blood pressure (SBP), and diastolic blood pressure (DBP) were measured using standardized protocols by trained personnel. Fasting blood samples were obtained for measures of lipids, insulin, SCr, fibrinogen, and glycated hemoglobin. Urine samples were collected to measure albumin and creatinine. Detailed clinical and laboratory measures have been published.10,13

Hypertension was defined by use of antihypertensive medication, SBP ≥140 mmHg, or DBP ≥90 mmHg. Micro- and macroalbuminuria were defined as urinary albumin-creatinine ratios of 30–299 mg/g and ≥300 mg/g, respectively. Diabetes was identified by use of hypoglycemic agents, fasting glucose ≥126 mg/dL, or self-report of diagnosis by a physician.14

Creatinine Recalibration

During SHS Phase 1, SCr was measured using an alkaline picrate rate method (Roche Diagnostics, Indianapolis) on the Hitachi 717 platform.10,11 For the current analyses, we randomly selected 300 stored serum specimens from SHS Phase 1, including 60 from each quintile of the SCr distribution. We then added all 39 available samples with baseline values of 2.2–4.8 mg/dL, because these were under-represented in the original sample of 300. All samples were thawed, centrifuged, and assayed in triplicate using an IDMS-traceable slide-based enzymatic creatinine assay on the Vitros Fusion 5,1 platform (Ortho Clinical Diagnostics, Rochester). Original and IDMS-traceable SCr values and their paired differences (IDMS-traceable SCr minus original SCr) were compared using scatterplots (Figure S1, available as online supplementary material), Bland-Altman plots (Figure S2) and Deming regression.15 Finally, the derived calibration equation (SHS IDMS-calibrated SCr = −0.159 + (0.980 × original SHS SCr)) was assessed by residual analysis. This calibration equation was similar to that reported previously for NHANES samples collected and assayed in 1988–1994 (−0.184 + (0.96 × original NHANES SCr)). 16

Estimation of GFR

As described in the previous section, we recalibrated SCr to IDMS-traceable values. Then, we estimated GFR by the IDMS-traceable 4-variable MDRD Study equation16a, as eGFRMDRD = 175 × (standardized SCr)−1.154 × Age−0.203 × (0.742 if female) × (1.210 if African-American), where SCr is measured in mg/dL and age in years. In addition, we estimated GFR by the CKD-EPI equation4, as eGFRCKD-EPI = 141 × (minimum of standardized SCr [mg/dL]/κ or 1)α × (maximum of standardized SCr [mg/dL]/κ or 1)−1.209 × 0.993age × (1.018 if female) × (1.159 if black), where κ is 0.7 if female and 0.9 if male and α is −0.329 if female and −0.411 if male. Because the SHS includes only American Indians, the equation factors for race were dropped for all participants for both equations. Previous researchers have handled American Indian data similarly or used a constant midway between those for whites and blacks.17

In a supplemental analysis, eGFR was calculated using the original 4-variable MDRD Study equation16b (eGFR = 186 × (uncalibrated SCr)−1.154 × Age−0.203 × (0.742 if female) × (1.210 if African-American), where SCr is measured in mg/dL and age in years) and the original SHS SCr values. We also compared eGFR values calculated with this original 4-variable MDRD Study equation and the CKD-EPI equation (Table S1).

Estimated GFR was originally categorized using the cut-off values suggested by NKF-KDOQI (the National Kidney Foundation’s Kidney Disease Outcomes Quality Initiative), >90, 60–89, 30–59, 15–29, and <15 mL/min/1.73 m2.18 Because of the small number of participants with eGFR < 15 (n=17), participants in this category were combined with those having an eGFR of 15–29. Further, we separated high eGFR (≥120 mL/min/1.73 m2) values into a separate category because of the possibility that these may reflect loss of muscle mass, diabetic hyperfiltration, or inaccurately estimated GFR values.19 Therefore, our results are presented in the following categories: ≥120, 90 – <120, 60 – < 90, 30 – < 60, and <30 mL/min/1.73 m2.

Follow-Up and Clinical Outcomes

Fatal and nonfatal CVD events have been defined.11,12,20 Incident CHD consisted of nonfatal definite MI; definite CHD; ECG-evident definite MI; cardiac procedures, including percutaneous transluminal coronary angioplasty or coronary artery bypass graft; fatal definite MI; and sudden, definite, and possible CHD death. Incident stroke consisted of both nonfatal and fatal definite and possible events. Total incident CVD was defined as all events noted above; in addition, it included fatal and nonfatal HF. Surveillance for nonfatal and fatal clinical CVD events occurred throughout the follow up and is complete through December 31, 2007.

Statistical Analyses

Participants were classified according to category of eGFRCKD-EPI by using the categories ≥ 120, 90–<120, 60–<90, 30–<60, and <30 mL/min/1.73 m2. The baseline characteristics of the study population were calculated according to category of eGFRCKD-EPI. A Poisson regression model with eGFR as a restricted cubic spline term was used to evaluate the continuous association between eGFR, as calculated by the CKD-EPI and MDRD Study equations, and CVD incidence rate, with and without adjustment for age and sex. Five knots were placed at eGFR = 45, 60, 75, 90, and 105 mL/min/1.73 m2. The plot was truncated at 35 and 126 mL/min/1.73 m2 of eGFR (the 1.25th and 98.75th percentiles of eGFRCKD-EPI, respectively, and the 1.25th and 86th percentiles of eGFRMDRD, respectively). Cox proportional hazards models were used to evaluate risk of incident CVD and CVD subcategories (CHD, stroke, and HF) according to eGFR categories as calculated with the MDRD Study and CKD-EPI equations. Estimated GFR of 90–119 ml/min/1.73 m2 was used as the reference group in separate models. Model 1 was adjusted for age, sex, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol, hypertension (yes/no), diabetes (yes/no), and smoking (never, past, or current smoker). Model 2 was adjusted for the variables in Model 1 plus albuminuria. The models were assessed by Akaike information criterion (AIC), with discrimination assessed by Harrell’s C statistics and calibration assessed by Hosmer-Lemeshow statistics. Reclassification tables for subjects with and without CVD events were generated using the MDRD Study and CKD-EPI equations by category of eGFR. Net reclassification index (NRI) was then calculated to assess overall change in reclassification of risk. We also assessed the NRI and integrated discrimination improvement (IDI) using risk categories (eGFR <10%, 10%–<20%, and ≥20%) for CVD and for each CVD subcategory from Cox proportional hazard models. All P values were 2-tailed, and P<0.05 was considered statistically significant. All analyses were conducted using SAS statistical software, version 9.1 (SAS Institute, Cary, NC).

RESULTS

Table 1 shows the baseline characteristics of the study population according to category of eGFRCKD-EPI. As expected, the mean eGFR using the CKD-EPI equation (97.33± 17.43) was lower than the mean eGFR using the MDRD Study equation (102.42± 30.39); this was driven by very high eGFR levels in the group with highest MDRD Study equation-based eGFR. Estimated GFR CKD-EPI and eGFRMDRD were highly correlated (Pearson correlation coefficient =0.857 [p<0.001]). Compared to those with eGFRCKD-EPI of 90–119 mL/min/1.73m2, participants with eGFRCKD-EPI of 30–59 mL/min/1.73m2 or < 30 mL/min/1.73m2 were more likely to be older; female; and have diabetes, hypertension, macroalbuminuria, an adverse lipid profile, and higher SBP and DBP. These individuals were less likely to be current smokers.

Table 1.

Baseline Characteristics by eGFRCKD-EPI Category

Characteristic Category of eGFRCKD-EPI (mL/min/1.73 m2) P
≥ 120 (n=135) 90 – <120 (n=3022) 60 – <90 (n=761) 30 – <60 (n=119) <30 (n=44)
eGFRCKD-EPI, mL/min/1.73 m2 125.7±6.5 103.6±7.2 79.6±8.1 49.4±7.4 17.8±8.2 <.001
eGFRMDRD, mL/min/1.73 m2 181.6±48.8 109.3±19.2 74.5±8.5 46.8±6.8 17.5±7.7 <.001
Age, y 49.2±4.2 54.5±7.1 62.1±8.3 62.0±7.2 58.7±8.0 <.001
BMI, kg/m2 32.7±7.7 31.0±6.3 29.9±5.8 29.7±5.8 30.3±5.2 0.001
Waist circumference, cm 108.6±16.3 105.3±14.7 103.4±14.0 104.5±13.9 107.0±12.1 0.07
LDL-C, mg/dL 107.9±34.7 115.5±32.3 122.9±35.2 116.0±41.5 117.6±40.4 0.1
HDL-C, mg/dL 44.7±11.7 45.9±13.3 46.4±13.8 48.5±17.1 44.0±12.8 0.02
SBP, mmHg 124.7±16.2 125.5±17.7 131.4±21.2 138.3±26.3 153.1±29.1 <.001
DBP, mmHg 77.1±9.4 76.9±9.9 76.0±10.4 77.4±11.9 81.1±13.0 0.9
Fasting glucose, mg/dL 178.1±83.3 148.0±74.2 144.8±72.5 163.6±87.0 165.2±97.8 0.5
2-h glucose, mg/dL 221.2±119.4 177.0±106.9 166.0±97.5 198.0±137.1 235.5±165.0 0.9
Triglycerides, mmol/L 121.0(80.0–196.0) 117.0(82.0–167.0) 123.0(89.0–174.0) 146.0(91.0–237.0) 175.5(97.0–211.5) 0.003
Female sex 49.6 58.4 65.8 79.0 70.5 <.001
Diabetes 63.7 41.3 42.3 64.7 72.7 <.001
Hypertension 26.7 32.8 48.4 67.2 88.6 <.001
Microalbuminuria 32.8 18.7 17.1 19.5 11.9 0.5
Macroalbuminuria 11.9 6.2 13.9 51.7 88.1 <.001
Current smoker 43.3 36.3 27.0 26.9 27.3 <.001
Past smoker 35.1 32.2 37.2 34.5 31.8 0.03

Values for continuous variables given as mean ± SD; values for categorical variables given as percentage.

Conversion factors for units: eGFR in mL/min/1.73 m2 to mL/s/1.73 m2, x0.01667; cholesterol in mg/dL to mmol/L, x0.02586; glucose in mg/dL to mmol/L, x0.05551; triglycerides in mmol/L to mg/dL, x88.6

Abbreviations: eGFRCKD-EPI = estimated glomerular filtration rate as calculated with the Chronic Kidney Disease Epidemiology Collaboration equation; eGFRMDRD = estimated glomerular filtration rate as calculated with the Modification of Diet in Renal Disease Study equation; LDL-C, low-density lipoprotein cholesterol; HDL-C, high-density lipoprotein cholesterol; SBP, systolic blood pressure; DBP, diastolic blood pressure; BMI, body mass index.

During a median 15-year follow-up (maximum, 18.6 years), 1280 cases of incident CVD, 929 cases of incident CHD, 305 cases of incident stroke, and 381 cases of incident HF were observed. Incidence rates (per 1,000 person-years) for total CVD, CHD, stroke, and HF (Figure 1), according to each eGFR equation and calculated by Poisson regression with a restricted cubic spline function of eGFR, showed similar relations for both equations. However, for eGFR 30–89 mL/min/1.73m 2, estimates using the MDRD Study equation showed slightly lower relative risk for CVD, CHD, stroke, and HF compared with estimates using the CKD-EPI equation. For eGFR ≥90 mL/min/1.73m2, the slope was steeper for eGFR estimates using the CKD-EPI equation than those using the MDRD Study equation. A similar association was observed after adjustment for age and sex.

Figure 1.

Figure 1

Incidence rates (per 1,000 person-years) for cardiovascular disease, coronary heart disease, stroke, and heart failure, according to eGFRCKD-EPI (solid and dashed black lines) and eGFRMDRD (solid and dashed gray lines) equations. Solid lines indicate no adjustment and dashed lines indicate adjustment for baseline age and sex. Results were obtained by a Poisson regression model with restricted cubic splines function of eGFR (knots at 45, 60, 75, 90, and 105 mL/min/1.73m2).

Adjusted hazard ratios and corresponding 95% confidence intervals for incident total CVD, CHD, stroke, and HF are shown by eGFR categories for each equation (Table 2). Compared with participants who had well-preserved kidney function, those with lower eGFR estimated by either equation were significantly and similarly more likely to progress to incident CVD after adjustment for covariates (Model 1). The significant associations between eGFR ≥30 to 89 mL/min/1.73m2 and incident CVD were attenuated for eGFRCKD-EPI and eGFRMDRD after further adjustment for albuminuria (Model 2), as were associations for eGFR <30 mL/min/1.73m2. Similar results were observed for incident CHD, but eGFR <60 mL/min/1.73m2 calculated using either equation had significantly higher risk of incident CHD in Models 1 and 2. Decreased eGFR was associated with greater risks of incident stroke and HF, but this association was attenuated after adjustment for albuminuria. Harrell’s C statistic was similar for eGFRCKD-EPI and eGFRMDRD. The AIC and Hosmer-Lemeshow statistics also showed no significant difference for eGFRCKD-EPI and eGFRMDRD (Table S2). Similar results were observed across eGFR categories with the non-IDMS-traceable 4-variable MDRD Study equation (Table S1).

Table 2.

Association of Incident CVD, CHD, Stroke, and HF with eGFR Category

Category of eGFR (mL/min/1.73 m2) C statistic (95% CI)
≥ 120 90–<120 60 – <90 30 – <60 <30
No. of participants
 CKD-EPI 135 3022 761 119 44 --
 MDRD 934 1789 1167 145 46 --
CVD
 CKD-EPI
  Model 1 1.38 (0.99–1.92) 1 (ref) 1.15 (0.99–1.34) 1.74 (1.30–2.34) 2.63 (1.71–4.04) 0.701 (0.638–0.760)
  Model 2 1.29 (0.92–1.80) 1 (ref) 1.10 (0.94–1.28) 1.27 (0.93–1.73) 1.70 (1.08–2.69) 0.711 (0.648–0.770)
 MDRD
  Model 1 1.19 (1.02–1.39) 1 (ref) 1.22 (1.06–1.39) 1.79 (1.36–2.36) 2.84 (1.86–4.34) 0.702 (0.639–0.761)
  Model 2 1.14 (0.98–1.33) 1 (ref) 1.18 (1.03–1.35) 1.34 (1.01–1.78) 1.83 (1.17–2.87) 0.711 (0.649–0.771)
CHD
 CKD-EPI
  Model 1 1.25 (0.84–1.84) 1 (ref) 1.16 (0.97–1.39) 2.17 (1.55–3.04) 2.87 (1.76–4.69) 0.713 (0.640–0.781)
  Model 2 1.17 (0.79–1.75) 1 (ref) 1.12 (0.93–1.34) 1.60 (1.13–2.28) 1.92 (1.13–3.25) 0.721 (0.647–0.788)
 MDRD
  Model 1 1.14 (0.95–1.36) 1 (ref) 1.24 (1.05–1.45) 2.20 (1.61–3.01) 3.14 (1.94–5.08) 0.714 (0.642–0.782)
  Model 2 1.08 (0.90–1.30) 1 (ref) 1.21 (1.03–1.42) 1.66 (1.20–2.30) 2.10 (1.25–3.52) 0.722 (0.649–0.789)
Stroke
 CKD-EPI
  Model 1 2.24 (1.26–3.99) 1 (ref) 1.30 (0.97–1.75) 1.87 (1.06–3.29) 1.95 (0.71–5.34) 0.717 (0.591–0.828)
  Model 2 2.07 (1.16–3.69) 1 (ref) 1.23 (0.92–1.66) 1.39 (0.78–2.49) 1.29 (0.46–3.61) 0.733 (0.609–0.842)
 MDRD
  Model 1 1.40 (1.03–1.89) 1 (ref) 1.19 (0.90–1.57) 1.66 (0.95–2.88) 1.89 (0.69–5.20) 0.715 (0.589–0.826)
  Model 2 1.36 (1.00–1.85) 1 (ref) 1.15 (0.87–1.52) 1.25 (0.71–2.21) 1.20 (0.43–3.38) 0.733 (0.608–0.842)
HF
 CKD-EPI
  Model 1 1.09 (0.55–2.13) 1 (ref) 1.51 (1.16–1.97) 1.83 (1.10–3.05) 3.16 (1.54–6.49) 0.741 (0.631–0.838)
  Model 2 1.02 (0.52–2.00) 1 (ref) 1.37 (1.05–1.79) 1.14 (0.66–1.96) 1.78 (0.85–3.73) 0.758 (0.648–0.853)
 MDRD
  Model 1 1.35 (1.01–1.80) 1 (ref) 1.54 (1.20–1.98) 1.78 (1.08–2.92) 3.81 (1.91–7.60) 0.740 (0.630–0.837)
  Model 2 1.30 (0.97–1.73) 1 (ref) 1.41 (1.09–1.82) 1.16 (0.69–1.95) 2.12 (1.04–4.32) 0.757 (0.647–0.852)

Unless otherwise indicated, data are hazard ratio (95% CI).

Model 1: Adjusted for age, sex, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol, hypertension (yes vs. no), diabetes (yes vs. no), and smoking (never, past, or current). Model 2: Adjusted for all variables in Model 1, plus albuminuria (normal, microalbuminuria, or macroalbuminuria).

Abbreviations: CHD = coronary heart disease; CKD-EPI = Chronic Kidney Disease Epidemiology Collaboration equation; CVD = cardiovascular disease; eGFR = estimated glomerular filtration rate; MDRD = Modification of Diet in Renal Disease Study equation; HF, heart failure; CI, confidence interval; ref, reference.

Participants were reclassified into eGFRCKD-EPI categories for each category of eGFRMDRD, according to each equation and stratified according to incident CVD, CHD, stroke, and HF during follow-up (Table 3). Individuals with eGFR ≥120 mL/min/1.73m2 were excluded from this analysis because reclassification of such individuals to lower values may not predict differing CVD outcomes. Analyses of the reclassification of risk were measured by NRI, using eGFRCKD-EPI and eGFRMDRD and stratified according to incident CVD, CHD, stroke, and HF (Table 4).

Table 3.

Reclassification of eGFR Categories by the CKD-EPI and the MDRD Study Equations

eGFRMDRD eGFRCKD-EPI Total no.
90 – <120 60 – <90 30 – <60 <30
Without incident CVD
 90 – <120 1254 (58.3) 28 (1.3) 0 (0.0) 0 (0.0) 1282
 60 – <90 349 (16.2) 423 (19.7) 0 (0.0) 0 (0.0) 772
 30 – <60 0 (0.0) 16 (0.7) 60 (2.8) 0 (0.0) 76
 <30 0 (0.0) 0 (0.0) 1 (0.05) 20 (0.9) 21
 Total no. 1603 467 61 20 2151
With incident CVD
 90 – <120 485 (48.7) 22 (2.2) 0 (0.0) 0 (0.0) 507
 60 – <90 135 (13.6) 260 (26.1) 0 (0.0) 0 (0.0) 395
 30 – <60 0 (0.0) 12 (1.2) 57 (5.7) 0 (0.0) 69
 <30 0 (0.0) 0 (0.0) 1 (0.1) 24 (2.4) 25
 Total no. 620 294 58 24 996
Without incident CHD
 90 – <120 1389 (57.4) 34 (1.4) 0 (0.0) 0 (0.0) 1423
 60 – <90 378 (15.6) 502 (20.8) 0 (0.0) 0 (0.0) 880
 30 – <60 0 (0.0) 19 (0.8) 70 (2.9) 0 (0.0) 89
 <30 0 (0.0) 0 (0.0) 1 (0.04) 26 (1.1) 27
 Total no. 1767 555 71 26 2419
With incident CHD
 90 – <120 350 (48.1) 16 (2.2) 0 (0.0) 0 (0.0) 366
 60 – <90 106 (14.6) 181 (24.9) 0 (0.0) 0 (0.0) 287
 30 – <60 0 (0.0) 9 (1.2) 47 (6.5) 0 (0.0) 56
 <30 0 (0.0) 0 (0.0) 1 (0.1) 18 (2.5) 19
 Total no. 456 206 48 18 728
Without incident stroke
 90 – <120 1633 (56.0) 41 (1.4) 0 (0.0) 0 (0.0) 1674
 60 – <90 461 (15.8) 610 (20.9) 0 (0.0) 0 (0.0) 1071
 30 – <60 0 (0.0) 26 (0.9) 102 (3.5) 0 (0.0) 128
 <30 0 (0.0) 0 (0.0) 2 (0.1) 40 (1.4) 42
 Total no. 2094 677 104 40 2915
With incident stroke
 90 – <120 106 (45.7) 9 (3.9) 0 (0.0) 0 (0.0) 115
 60 – <90 23 (9.9) 73 (31.5) 0 (0.0) 0 (0.0) 96
 30 – <60 0 (0.0) 2 (0.9) 15 (6.5) 0 (0.0) 17
 <30 0 (0.0) 0 (0.0) 0 (0.0) 4 (1.7) 4
 Total no. 129 84 15 4 232
Without incident HF
 90 – <120 1614 (56.6) 47 (1.6) 0 (0.0) 0 (0.0) 1661
 60 – <90 451 (15.8) 582 (20.4) 0 (0.0) 0 (0.0) 1033
 30 – <60 0 (0.0) 25 (0.9) 97 (3.4) 0 (0.0) 122
 <30 0 (0.0) 0 (0.0) 1 (0.04) 35 (1.2) 36
 Total no. 2065 654 98 35 2852
With incident HF
 90 – <120 125 (42.4) 3 (1.0) 0 (0.0) 0 (0.0) 128
 60 – <90 33 (11.2) 101 (34.2) 0 (0.0) 0 (0.0) 134
 30 – <60 0 (0.0) 3 (1.0) 20 (6.8) 0 (0.0) 23
 <30 0 (0.0) 0 (0.0) 1 (0.3) 9 (3.1) 10
 Total no. 158 107 21 9 295

eGFR given in mL/min.1.73 m2. Unless otherwise indicates, values given are number (percentage).

Abbreviations: eGFR, estimated glomerular filtration rate; eGFRCKD-EPI = estimated glomerular filtration rate as calculated with the Chronic Kidney Disease Epidemiology Collaboration equation; eGFRMDRD = estimated glomerular filtration rate as calculated with the Modification of Diet in Renal Disease Study equation; HF, heart failure; CHD = coronary heart disease; CVD = cardiovascular disease;

Table 4.

NRI for Incident CVD, CHD, Stroke, and Heart Failure Using eGFRCKD-EPI

NRI (95% CI) P
Incident CVD
 Without 0.157 (0.139, 0.175) <0.001
 With −0.127 (−0.152, −0.101) <0.001
 Overall 0.031 (−0.001, 0.062) 0.06
Incident CHD
 Without 0.150 (0.134, 0.167) <0.001
 With −0.137 (−0.168, −0.106) <0.001
 Overall 0.013 (−0.022, 0.048) 0.5
Incident stroke
 Without 0.154 (0.138, 0.169) <0.001
 With −0.069 (−0.118, −0.020) <0.001
 Overall 0.085 (0.033, 0.136) 0.001
Incident HF
 Without 0.151 (0.135, 0.167) <0.001
 With −0.115 (−0.157, −0.073) <0.001
 Overall 0.036 (−0.009, 0.080) 0.1

Abbreviations: CHD = coronary heart disease; CVD = cardiovascular disease; HF = heart failure, NRI = net reclassification index; eGFRCKD-EPI = estimated glomerular filtration rate as calculated with the Chronic Kidney Disease Epidemiology Collaboration equation

The CKD-EPI equation incorrectly reclassified 1.3% (n=28) of 2151 participants without incident CVD to a higher risk (lower eGFR) category and correctly reclassified 17.0% of participants without incident CVD to a lower risk (higher eGFR) category. In 996 participants with incident CVD, the CKD-EPI equation incorrectly reclassified 14.9% to a lower risk category and correctly reclassified 2.2 % to a higher risk category. Overall, the NRI for CVD risk with the CKD-EPI equation was 0.031 (P=0.06). The NRI with the use of the CKD-EPI equation for CHD, stroke, and HF were all positive and significant for those who were free of CVD events during follow up. However, the overall NRI, including those with and without incident CVD events, was not significant. No significant overall NRI or IDI were observed for any outcomes predicted from Cox proportional hazard models, including participants with eGFR≥120 mL/min/1.73m2 (Table S3).

DISCUSSION

Accurate early identification of CKD may lead to better outcomes and help target clinical interventions to the patients at higher risk for co-morbid CVD. In the absence of measured GFR, clinicians often depend on eGFR, most commonly determined using the MDRD Study equation. It is well recognized that the MDRD Study equation underestimates GFR in healthy subjects. The current results also showed lower estimates using the MDRD Study equation compared with the CKD-EPI in this population characterized by high baseline prevalence of diabetes and CKD. Both GFR estimating equations categorized participants similarly in the lower eGFR categories of <30 and 30–60 mL/min/1.73 m2, with greater discrepancy at the higher eGFRs, whether in the 60–89 or ≥90 mL/min/1.73 m2 range. The two equations performed similarly with respect to association with incident CVD, CHD, HF, and stroke, with or without adjustment for albuminuria. While the Cox proportional hazards models showed similar performance for both equations, differences were detected in the NRI results, which showed a small but significant benefit of using the CKD-EPI equation to estimate risk in participants without CVD events. In the absence of measured GFR values, this large population with high rates of diabetes, CVD, and CKD provides valuable information on the performance of these two equations in assessing the relationship between CKD and CVD.

Some studies have shown that the CKD-EPI equation provides more accurate GFR estimates compared with the MDRD Study equation overall and in groups with eGFR > 60 mL/min/1.73 m2.21 Similar findings of better accuracy with the CKD-EPI equation have been reported in studies from Japan, the Netherlands, and the United States.22,23, 24 In a Spanish study, the prevalence of decreased kidney function was observed to be substantially reduced in participants with diabetes, hypertension, and CVD.25 Recent studies have shown that the CKD-EPI equation correctly reclassifies people at lower risk of CKD and death into higher eGFR categories.26,27 Compared with the MDRD Study equation, the CKD-EPI equation suggested lower prevalence of CKD in the KEEP (Kidney Early Evaluation Program) study, a predominantly white population with relatively low prevalence of obesity or diabetes, but similar diabetes prevalence rates associated with CKD across all stages.28

Similar controversies have been raised concerning the use of cystatin C and SCr in eGFR equations and in predicting CVD events. We did not include cystatin C in the current analyses, but in a previous analysis in this population, we found that continuous cystatin C-based eGFR equations were consistently associated with CVD risk and that the addition of SCr to cystatin C to estimate GFR did not provide significant benefit in CVD prediction.29 In the current study, we found that either MDRD Study- or CKD-EPI-based eGFRs were similarly associated with incident CVD, CHD, HF, and stroke, with and without adjustment for albuminuria.

Published work has identified albuminuria as an independent risk factor for CHD in American Indians.30 We previously showed that including either eGFRMDRD or albuminuria in a CVD risk model using established risk factors helped identify participants at risk for CHD, as might be expected in our population with a high prevalence of diabetes mellitus and with primarily diabetes-associated CKD.3 Other studies have investigated the combined contribution of albuminuria and eGFR in predicting CVD risk and mortality, largely in randomized clinical trials.31, 32 However, few population-based studies have explored the benefit of using these two measures together to predict CVD risk. In the current study, the association between either eGFR measure was attenuated after adjusting for albuminuria. As we have previously discussed,3 the Gubbio Population Study,31 which involved middle-aged white participants with low prevalence of diabetes, found that the lack of association between albuminuria and eGFR gives complementary information, identifying distinct groups at risk and providing additional, independent information for predicting CVD risk. The HUNT II study33 demonstrated that adding albuminuria and eGFR to traditional risk models improved CVD mortality prediction. A separate report of a representative sample of U.S. adults concluded that albuminuria and reduced eGFR had an independent non-synergistic impact on CVD and all-cause mortality.32 Using a large prospective database in Hong Kong, investigators observed that adjusting for eGFR and albuminuria attenuated associations between lipids, A1c, BP, duration of diabetes, and CHD.34 Furthermore, a Dutch population-based study found that there was an association between including eGFR and albuminuria in risk prediction and better prediction of vascular events.35 Finally, in the López-Suárez et al. study, the CKD-EPI equation may have overestimated the degree of decrease in kidney function in normoalbuminuric women.25 In sum, as we have previously alluded to,3 this literature suggests that the additional contribution of albuminuria and/or eGFR measures may vary based on the population, its diabetes prevalence, and on the outcomes of interest.

This study was limited by the availability of only a single measurement of SCr and urine albumin at each examination. The NKF-KDOQI guidelines recommend two readings >90 days apart for clinical diagnosis. Also, the GFR estimating equations were not developed in an American Indian population; nevertheless, this analysis in a single ethnic group is valuable, because this population’s high rates of diabetes and diabetic-CVD have rendered it a model for other ethnic groups experiencing the epidemic of obesity and diabetes.

In conclusion, the current analysis showed that the CKD-EPI equation provided better risk reclassification, while the CKD-EPI and MDRD Study equations showed similar associations with incident CVD, in this population with a high prevalence of obesity and kidney disease.

Supplementary Material

01
02
03
04
05

Acknowledgments

We thank Rachel Schaperow, MedStar Health Research Institute, for editing the manuscript.

Support: Funding for the Strong Heart Study was provided by the National Heart, Lung, and Blood Institute, cooperative agreements U01-HL41642, U01-HL41652, UL01-HL41654, U01-HL65520, and U01-HL65521. This project has been funded in whole or in part with Federal funds (Grant # UL1RR031975) from the National Center for Research Resources (NCRR) and the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health (NIH), through the Clinical and Translational Science Awards Program (CTSA), part of the Roadmap Initiative, “Re-Engineering the Clinical Research Enterprise.” The study sponsors had no role in the study design; collection, analysis, and interpretation of data; writing the report; or the decision to submit the report for publication. The opinions expressed herein do not necessarily reflect the views of the Indian Health Service or the NIH.

Footnotes

Supplementary Material

Table S1: Association of incident CVD, CHD, stroke, and HF with category of GFR estimated by the non–IDMS-traceable 4-variable MDRD Study equation.

Table S2: Comparison of models using AIC, C-statistics, and Hosmer-Lemeshow statistics.

Table S3: NRI by eGFRCKD-EPI risk categories and integrated discrimination improvement.

Figure S1: Scatterplot of uncalibrated and IDMS-calibrated creatinine.

Figure S2: Bland-Altman plot of uncalibrated and IDMS-calibrated creatinine.

Note: The supplementary material accompanying this article (doi:_______) is available at www.ajkd.org

Financial Disclosure: Dr Howard has served on the advisory boards for Merck/Schering Plough and has received research support in the form of drug donations from Merck/Schering Plough. The other authors declare that they have no other relevant financial interests.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Shara NM, Resnick HE, Lu L, et al. Decreased GFR estimated by MDRD or Cockcroft-Gault equation predicts incident CVD: The Strong Heart Study. J Nephrol. 2009;22(3):373–80. [PMC free article] [PubMed] [Google Scholar]
  • 2.Wang F, Ye P, Luo L, Xiao W, Wu H. Association of risk factors for cardiovascular disease and glomerular filtration rate: a community-based study of 4925 adults in Beijing. Nephrol Dial Transplant. 2010;25(12):3924–3931. doi: 10.1093/ndt/gfq327. [DOI] [PubMed] [Google Scholar]
  • 3.Shara NM, Wang H, Valaitis E, et al. Comparison of estimated glomerular filtration rates and albuminuria in predicting risk of coronary heart disease in a population with a high prevalence of diabetes mellitus and renal disease: The Strong Heart Study. Am J Cardiol. 2011;107(3):399–405. doi: 10.1016/j.amjcard.2010.09.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Levey AS, Stevens LA, Schmid CH, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. 2009;150(9):604–12. doi: 10.7326/0003-4819-150-9-200905050-00006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Matsushita K, Selvin E, Bash LD, et al. Risk implications of the new CKD Epidemiology Collaboration (CKD-EPI) equation compared with the MDRD Study equation for estimated GFR. Am J Kidney Dis. 2010;55(4):648–659. doi: 10.1053/j.ajkd.2009.12.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Stevens LA, Li S, Kurella Tamura M, et al. Comparison of the CKD Epidemiology Collaboration (CKDEPI) and Modification of Diet in Renal Disease (MDRD) study equations. Am J Kidney Dis. 2011;57(3 suppl 2):S9–S16. doi: 10.1053/j.ajkd.2010.11.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Skali H, Uno H, Levey AS, et al. Prognos tic assessment of estimated glomerular filtration rate by the new Chronic Kidney Disease Epidemiology Collaboration equation in comparison with the Modification of Diet in Renal Disease Study equation. Am Heart J. 2011;162(3):548–554. doi: 10.1016/j.ahj.2011.06.006. [DOI] [PubMed] [Google Scholar]
  • 8.White SL, Polkinghorne KR, Atkins RC, Chadban SJ. Comparison of the prevalence and mortality risk of CKD in Australia using the CKD Epidemiology Collaboration (CKD-EPI) and Modification of Diet in Renal Disease (MDRD) Study GFR estimating equations. Am J Kidney Dis. 2010;55(4):660–670. doi: 10.1053/j.ajkd.2009.12.011. [DOI] [PubMed] [Google Scholar]
  • 9.Matsushita K, Mahmoodi BK, Woodward M, et al. for the Chronic Kidney Disease Prognosis Consortium. Comparison of risk prediction using the CKD-EPI equation and the MDRD study equation for estimated glomerular filtration rate. JAMA. 2012;307(18):1941–1951. doi: 10.1001/jama.2012.3954. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Lee ET, Welty TK, Fabsitz R, et al. The Strong Heart Study. A study of cardiovascular disease in American Indians: design and methods. Am J Epidemiol. 1990;132(6):1141–55. doi: 10.1093/oxfordjournals.aje.a115757. [DOI] [PubMed] [Google Scholar]
  • 11.Howard BV, Lee ET, Cowan LD, et al. Coronary heart disease prevalence and its relation to risk factors in American Indians: the Strong Heart Study. Am J Epidemiol. 1995;142(3):254–268. doi: 10.1093/oxfordjournals.aje.a117632. [DOI] [PubMed] [Google Scholar]
  • 12.Howard BV, Lee ET, Cowan LD, et al. Rising tide of cardiovascular disease in American Indians: the Strong Heart Study. Circulation. 1999;99(18):2389–2395. doi: 10.1161/01.cir.99.18.2389. [DOI] [PubMed] [Google Scholar]
  • 13.Howard BV, Welty TK, Fabsitz RR, et al. Risk factors for coronary heart disease in diabetic and nondiabetic Native Americans. The Strong Heart Study. Diabetes. 1992 Oct;41(Suppl 2):4–11. doi: 10.2337/diab.41.2.s4. [DOI] [PubMed] [Google Scholar]
  • 14.American Diabetes Association. Diagnosis and classification of diabetes mellitus. Diabetes Care. 2004 Jan;27(Suppl 1):S5–S10. doi: 10.2337/diacare.27.2007.s5. [DOI] [PubMed] [Google Scholar]
  • 15.Martin RF. General Deming regression for estimating systematic bias and its confidence interval in methodcomparison studies. Clin Chem. 2000;46(1):100–104. [PubMed] [Google Scholar]
  • 16.Selvin E, Manzi J, Stevens LA, et al. Calibration of serum creatinine in the National Health and Nutrition Examination Surveys (NHANES) 1988–1994, 1999–2004. Am J Kidney Dis. 2007;50(6):918–926. doi: 10.1053/j.ajkd.2007.08.020. [DOI] [PubMed] [Google Scholar]
  • 16a. [Google Scholar]
  • 16b.Levey AS, Greene T, Kusek J, Beck G. A simplified equation to predict glomerular filtration rate from serum creatinine [Abstract] J Am Soc Nephrol. 2000;11:155A. [Google Scholar]
  • 17.Scavini M, Stidley CA, Paine SS, et al. The burden of chronic kidney disease among the Zuni Indians: the Zuni Kidney Project. Clin J Am Soc Nephrol. 2007;2(3):509–516. doi: 10.2215/CJN.02780806. [DOI] [PubMed] [Google Scholar]
  • 18.National Kidney Foundation. K/DOQI clinical practice guidelines for chronic kidney disease: part 5. Evaluation of laboratory measurements for clinical assessment of kidney disease. Guideline 4. Estimation of GFR. Am J Kidney Dis. 2002;39(S1):S76–S110. [PubMed] [Google Scholar]
  • 19.Cox HJ, Bhandari S, Rigby AS, Kilpatrick ES. Mortality at low and high estimated glomerular filtration rate values: A ‘U’ shaped curve. Nephron Clin Pract. 2008;110(2):c67–c72. doi: 10.1159/000151720. [DOI] [PubMed] [Google Scholar]
  • 20.Lee ET, Devereux RB, Yeh JL, Waung W, Go O. Selected mortality rates in the Strong Heart Study population. J Invest Med. 1995;43:510A. [Google Scholar]
  • 21.Stevens LA, Schmid CH, Greene T, et al. Comparative performance of the CKD Epidemiology Collaboration (CKD-EPI) and the Modification of Diet in Renal Disease (MDRD) study equations for estimating GFR levels above 60 mL/min/1.73 m2. Am J Kidney Dis. 2010;56(3):486–95. doi: 10.1053/j.ajkd.2010.03.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Horio M, Imai E, Yasuda Y, Watanabe T, Matsuo S. Modification of the CKD epidemiology collaboration (CKD-EPI) equation for Japanese: accuracy and use for population estimates. Am J Kidney Dis. 2010;56(1):32–8. doi: 10.1053/j.ajkd.2010.02.344. [DOI] [PubMed] [Google Scholar]
  • 23.Michels WM, Grootendorst DC, Verduijn M, Elliott EG, Dekker FW, Krediet RT. Performance of the Cockcroft-Gault, MDRD, and new CKD-EPI formulas in relation to GFR, age, and body size. Clin J Am Soc Nephrol. 2010;5(6):1003–9. doi: 10.2215/CJN.06870909. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Stevens LA, Schmid CH, Zhang YL, et al. Development and validation of GFR-estimating equations using diabetes, transplant and weight. Nephrol Dial Transplant. 2010;25(2):449–57. doi: 10.1093/ndt/gfp510. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.López-Suárez A, Beltrán-Robles M, Elvira-González J, et al. Comparison of the MDRD and the CKD-EPI equations to estimate the glomerular filtration rate in the general population. Med Clin (Barc) 2010;134(14):617–23. doi: 10.1016/j.medcli.2009.07.055. [DOI] [PubMed] [Google Scholar]
  • 26.Matsushita K, Selvin E, Bash LD, Astor BC, Coresh J. Risk implications of the new CKD Epidemiology Collaboration (CKD-EPI) equation compared with the MDRD Study equation for estimated GFR: the Atherosclerosis Risk in Communities (ARIC) Study. Am J Kidney Dis. 2010;55(4):648–59. doi: 10.1053/j.ajkd.2009.12.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Stevens LA, Li S, Tamura MK, et al. Comparison of the CKD Epidemiology Collaboration (CKD-EPI) and Modification of Diet in Renal Disease (MDRD) Study Equations: Risk Factors for and Complications of CKD and Mortality in the Kidney Early Evaluation Program (KEEP) Am J Kidney Dis. 57(3 S2):S9–S16. doi: 10.1053/j.ajkd.2010.11.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.McFarlane SI, McCullough PA, Sowers JR, et al. Comparison of the CKD Epidemiology Collaboration (CKD-EPI) and Modification of Diet in Renal Disease (MDRD) Study Equations: Prevalence of and risk factors for diabetes mellitus in CKD in the Kidney Early Evaluation Program (KEEP) Am J Kidney Dis. 57(3 S2):S24–S31. doi: 10.1053/j.ajkd.2010.11.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Shara NM, Vupputuri S, Jolly S, et al. Estimated GFR using either cystatin C or serum creatinine based equations and CVD risk: The Strong Heart Study. ASN Renal Week. 2009:Abstract Number: 554424. [Google Scholar]
  • 30.Lee ET, Howard BV, Wang W, et al. Prediction of coronary heart disease in a population with high prevalence of diabetes and albuminuria: the Strong Heart Study. Circulation. 2006;113(25):2897–2905. doi: 10.1161/CIRCULATIONAHA.105.593178. [DOI] [PubMed] [Google Scholar]
  • 31.Cirillo M, Lanti MP, Menotti A, et al. Definition of kidney dysfunction as a cardiovascular risk factor. Use of urinary albumin excretion and estimated glomerular filtration rate. Arch Intern Med. 2008;168(6):617–624. doi: 10.1001/archinte.168.6.617. [DOI] [PubMed] [Google Scholar]
  • 32.Astor BC, Hallan SI, Miller ER, III, Yeung E, Coresh J. Glomerular filtration rate, albuminuria, and risk of cardiovascular and all-cause mortality in the US population. Am J Epidemiol. 2008;167(10):1226–1234. doi: 10.1093/aje/kwn033. [DOI] [PubMed] [Google Scholar]
  • 33.Hallan SI, Ritz E, Lydersen S, Romundstad S, Kvenild K, Orth SR. Combining GFR and albuminuria to classify CKD improves prediction of ESRD. J Am Soc Nephrol. 2009;20(5):1069–77. doi: 10.1681/ASN.2008070730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Yang X, Ma RC, So WY, et al. Impacts of chronic kidney disease and albuminuria on associations between coronary heart disease and its traditional risk factors in type 2 diabetic patients – the Hong Kong diabetes registry. Cardiovasc Diabetol. 2007;6:37–50. doi: 10.1186/1475-2840-6-37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Vlek ALM, van der Graaf Y, Spiering W, Algra A, Visseren FL SMART study group. Cardiovascular events and all-cause mortality by albuminuria and decreased glomerular filtration rate in patients with vascular disease. J Internal Medicine. 2008;264(4):351–360. doi: 10.1111/j.1365-2796.2008.01970.x. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

01
02
03
04
05

RESOURCES