Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Feb 7.
Published in final edited form as: Kidney Int. 2011 Feb 2;79(12):1331–1340. doi: 10.1038/ki.2010.550

Association of estimated glomerular filtration rate and albuminuria with mortality and end-stage renal disease: a collaborative meta-analysis of kidney disease cohorts

The Chronic Kidney Disease Prognosis Consortium
PMCID: PMC3917543  NIHMSID: NIHMS544154  PMID: 21289598

Abstract

Limited data are available on the independent associations of estimated glomerular filtration rate (eGFR) and albuminuria with mortality and end stage renal disease (ESRD) among individuals with chronic kidney disease (CKD). We conducted a collaborative meta-analysis of 21,688 participants selected for CKD from 13 cohorts.

After adjustment for potential confounders and albuminuria, a 15 mL/min/1.73 m2 lower eGFR below 45 mL/min/1.73 m2 was significantly associated with mortality (pooled hazard ratio [HR] 1.47 [95% CI: 1.22–1.79]), and ESRD (pooled HR 6.24 [95% CI: 4.84–8.05]). There was significant heterogeneity between studies for both HR estimates. After adjustment for risk factors and eGFR, an eight-fold higher albumin:creatinine ratio (ACR) or protein:creatinine ratio (PCR) was significantly associated with mortality (pooled HR 1.40 [95% CI: 1.27–1.55]), without evidence of significant heterogeneity. An eight-fold higher ACR or PCR was also strongly associated with ESRD (pooled HR 3.04 [95% CI: 2.27–4.08]), with significant heterogeneity between HR estimates.

Lower eGFR and more severe albuminuria independently predict mortality and ESRD among individuals selected for CKD. The associations are stronger for ESRD than for mortality. The observed associations are consistent with CKD classification based on eGFR stages, and suggest that albuminuria provides additional prognostic information among individuals with CKD.

Introduction

This manuscript is the fourth in a series manuscripts to report the results of collaborative meta-analyses of estimated glomerular filtration rate (eGFR) and albuminuria on outcomes of chronic kidney disease (CKD) undertaken by the CKD Prognosis Consortium. These analyses were conducted in conjunction with the 2009 Controversies Conference, sponsored by Kidney Disease Improving Global Outcomes (KDIGO), which sought to evaluate the current definition and classification of CKD and propose alternatives. The report of the Consensus Conference is included in this issue of Kidney International[1].

The first three papers in this series dealt with all-cause and cardiovascular mortality and kidney outcomes in general population cohorts and high-risk cohorts.[2;2;3;3;4;4] This manuscript reports the results of a collaborative meta-analysis of mortality and end stage renal disease (ESRD) in 13 CKD cohorts, including predominantly individuals with CKD of diverse clinical diagnoses accompanied by decreased eGFR and elevated levels of albuminuria, corresponding to microalbuminuria or macroalbuminuria. A priori we hypothesized that both eGFR and albuminuria would be associated with these outcomes, independent of traditional cardiovascular risk factors and independent of each other, and despite inclusion of diverse study populations. Of particular relevance to these cohorts is the question of whether the severity of albuminuria provides additional prognostic information among individuals with CKD, over and above eGFR, as the current classification of CKD does not have separate stages by severity of albuminuria.[1;5] Previous reports of the associations of eGFR and albuminuria with outcomes in CKD studies did not use uniform analytic approaches and most reports were not powered to evaluate the independent associations of eGFR and albuminuria with these outcomes.[68]

Results

Characteristics of studies and participants

A total of 21,688 participants from 14 studies are included in at least one analysis, including 6 randomized controlled trials, 4 observational studies of referred patients, and 4 studies of participants identified by laboratory testing (Table 1).[618] A total of 4,374 deaths occurred in the 10 studies from which information on mortality was captured. Of these studies, 6 had data on ACR, 3 had data on PCR and 1 had data on dipstick proteinuria. The mortality incidence rate varied dramatically from 19.8 to 254.7 / 1,000 person-years. A total of 4,157 ESRD events occurred in the 12 studies from which such information was captured. Of these studies, 6 had data on ACR, 5 had data on PCR and 1 had data on dipstick proteinuria. The ESRD incidence rate varied markedly from 13.6 to 115.3 / 1,000 person-years. The mean eGFR varied from 22.2 to 69.8 mL/min/1.73m2. The median ACR varied from 26.5 to 1245.5 mg/g, and the median PCR varied from 80.8 to 2337.4 mg/g (Table 2).

Table 1.

Participating studies and incidence of mortality and end-stage renal disease

Study design Source / Intervention N Mortality End-stage renal disease
Follow-up, years # Events Incidence rate Follow-up, years # Events Incidence rate
Studies with Albumin:Creatinine Ratio
British Columbia[9] Observational Referred 13,038 2.8 2449 66.1 2.5 2,222 68.9
CRIB[10] Observational Referred 308 6.1 115 61.4 4.2 149 115.3
Grampian-ACR[23] Observational Identified by laboratory results 208 2.7 94 166.0 - - -
MASTERPLAN[12] Clinical trial Nurse practitioner-aided care 620 4.1 50 19.8 4.1 61 24.8
NephroTest[14] Observational Referred 1021 - - - 2.5 142 55.1
RENAAL[8] Clinical trial Losartan 1513 2.8 313 67.2 3.4 341 79.3
Steno[17] Observational Clinic 380 8.9 115 33.9 7.5 54 18.6
Overall 17,088 - 3,136 - - 2969 -

Studies with Protein:Creatinine Ratio
AASK[7] Clinical trial, followed by observational study Antihypertensives and blood pressure goal 1,084 8.7 250 26.4 7.5 311 38.5
Grampian-PCR[23] Observational Identified by laboratory results 159 2.3 94 254.7 - - -
MDRD[6] Clinical trial, followed by observational study Dietary protein restriction 839 9.5 208 26.1 6.2 553 105.8
MMKD[13] Observational Referred 203 - - - 4.0 73 89.8
REIN[15] Clinical trial Ramipril 352 - - - 2.6 81 88.8
REIN 2[16] Clinical trial Ramipril 335 - - - 1.9 72 113.0
Overall 2,972 - 552 - - 1,090 -

Studies with Dipstick Proteinuria
Kaiser Permanente Northwest[11] Observational Identified by laboratory results 1,628 4.5 686 92.6 4.4 98 13.6

Overall 21,688 - 4,374 - - 4,157 -

per 1,000 person-years; AASK: African American Study of Kidney Disease and Hypertension; CRIB: Chronic Renal Impairment in Birmingham; MASTERPLAN: Multifactorial Approach and Superior Treatment Efficacy in Renal Patients with the Aid of a Nurse Practicioner; MDRD: Modification of Diet in Renal Disease; MMKD: Mild to Moderate Kidney Disease Study; REIN: Ramipril Efficacy in Nephropathy; REIN 2: Ramipril Efficacy in Nephropathy 2; RENAAL: Reduction of Endpoints in Non-insulin Dependent Diabetes Mellitus with the Angiotensin II Antagonist Losartan; Steno: Steno Type 1 Diabetes Study

Table 2.

Baseline characteristics of participating study populations

Mean age, years (sd) Female, % Black, % CVD, % DM, % Smoking, % Hypertension, % Hypercholesterolemia, % Median ACR/PCR, mg/g Mean eGFR ml/min/1.73m2 (sd)
Studies with Albumin:Creatinine Ratio
British Columbia 68.5 (13.9) 44.8 0.5 23.5 32.4 5.4 41.1 -- 94.6 34.9 (17.9)
CRIB 61.7 (14.2) 34.1 5.8 45.5 17.2 13.3 90.6 -- 467.2 22.2 (10.6)
Grampian-ACR 73.0 (11.4) 56.4 0 24.2 8.5 61.6 59.5 -- 26.5 34.6 (5.8)
MASTERPLAN 60.5 (12.4) 31.0 2.7 30.2 24.3 20.2 95.4 18 119.1 36.3 (13.2)
NephroTest 59.6 (15.0) 31.7 9.6 16.0 26.0 10.0 88.7 16.6 77.6 41.1 (20.1)
RENAAL 60.2 (7.4) 36.8 15.2 35.0 100 18.1 96.4 -- 1245.5 39.8 (12.3)
Steno 42.6 (10.8) 38.1 0 10.2 100 51.9 82.2 36.9 498.0 69.8 (27.5)

Studies with Protein:Creatinine Ratio
AASK 54.6 (10.7) 38.8 100.0 51.6 0 29.3 100 55.6 80.8 42.5 (13.2)
Grampian-PCR 73.4 (14.8) 61.6 0 17.0 2.5 55.4 52.8 -- 300.9 34.6 (6.2)
MDRD 51.7 (12.4) 39.5 7.9 9.6 5.1 16.6 86.2 23.0 268.5 32.6 (12.3)
MMKD 46.4 (12.3) 34.0 0 12.3 0 21.7 89.2 38.4 1035.0 43.4 (26.7)
REIN 49.5 (13.6) 23.6 0.6 0 7.7 18.2 86.4 -- 2337.4 42.9 (18.5)
REIN 2 54.2 (15.0) 25.1 0 0 5.1 16.1 73.1 -- 1875.3 31.0 (16.7)

Studies with Dipstick Proteinuria
Kaiser Permanente Northwest 71.79(9.73) 56.0 3.1 44.8 38.8 12.8 92.9 26.0 -- 45.7 (10.5)

per 1,000 person-years; AASK: African American Study of Kidney Disease and Hypertension; CRIB: Chronic Renal Impairment in Birmingham; MASTERPLAN: Multifactorial Approach and Superior Treatment Efficacy in Renal Patients with the Aid of a Nurse Practicioner; MDRD: Modification of Diet in Renal Disease; MMKD: Mild to Moderate Kidney Disease Study; REIN: Ramipril Efficacy in Nephropathy; REIN 2: Ramipril Efficacy in Nephropathy 2; RENAAL: Reduction of Endpoints in Non-insulin Dependent Diabetes Mellitus with the Angiotensin II Antagonist Losartan; Steno: Steno Type 1 Diabetes Study

Independent associations of estimated GFR and albuminuria with mortality

The incidence rate of mortality was generally greater in lower eGFR categories, but there was wide variation in the incidence rate across studies at every eGFR category (Figure 1.a.). After adjustment, all 8 of the studies included showed a positive association between lower eGFR category and mortality (Table 3). Four of the eight studies had a significantly higher HR for an eGFR of 30–44 mL/min/1.73 m2 compared to 45–74 mL/min/1.73 m2, and 7 of 8 studies had a significantly elevated HR for an eGFR of 15–29 mL/min/1.73 m2 compared to 45–74 mL/min/1.73 m2. In continuous analyses, additionally adjusted for category of ACR, PCR or dipstick proteinuria, below an eGFR of 45 mL/min/1.73 m2 the association between a 15 mL/min/1.73 m2 lower eGFR and mortality was statistically significant in 5 of 8 studies, with a pooled HR of 1.47 (95% CI: 1.22, 1.79) (Figure 2.a). There was significant heterogeneity between studies in the HR estimates (I2=82.7%; p<0.001).

Figure 1.

Figure 1

Crude incidence rate of mortality (per 1,000 person-years) by (A) category of estimated glomerular filtration rate and (B) category of albuminuria. Solid lines represent studies which assessed albuminuria using albumin:creatinine ratio (categories: <30, 30–299, 300–999, ≥1,000 mg/g). Dashed lines represent studies which assessed albuminuria using protein:creatinine ratio (categories: <50, 50–499, 500–1,499, ≥1,500 mg/g). Dotted lines represent studies which assessed albuminuria using dipstick protein (categories: −/±, +, ++, ≥+++). Points with ≤5 participants are excluded.

Table 3.

Adjusted* hazard ratio (95% confidence interval) for mortality, by estimated GFR category

Estimated glomerular filtration rate (mL/min/1.73 m2)
45–74 30–44 15–29 <15
Studies with Albumin:Creatinine Ratio
British Columbia Reference 1.40 (1.20, 1.62) 3.06 (2.66, 3.52) 4.07 (3.42, 4.84)
CRIB Reference 1.18 (0.14, 9.65) 1.93 (0.24, 15.32) 3.39 (0.43, 26.88)
MASTERPLAN Reference 1.32 (0.54, 3.20) 2.62 (1.11, 6.21) 4.49 (1.08, 18.66)
RENAAL Reference 1.13 (0.84, 1.50) 1.95 (1.43, 2.66) -
Steno Reference 1.46 (0.80, 2.68) 2.78 (1.48, 5.19) 5.90 (2.33, 14.96)

Studies with Protein:Creatinine Ratio
AASK Reference 1.66 (1.25, 2.20) 1.55 (1.10, 2.19) 6.28 (0.85, 46.44)
MDRD Reference 1.75 (1.10, 2.77) 1.79 (1.13, 2.85) 1.68 (0.79, 3.57)

Studies with Dipstick Proteinuria
Kaiser Permanente Northwest Reference 1.24 (1.05, 1.47) 2.41 (1.94, 2.99) -

OVERALL Reference 1.35 (1.23, 1.49) 2.25 (1.81, 2.79) 3.74 (2.69, 5.20)
*

Adjusted for age, sex, race, prior cardiovascular disease, smoking status, diabetes mellitus, systolic blood pressure, and serum total cholesterol concentration.

AASK: African American Study of Kidney Disease and Hypertension; CRIB: Chronic Renal Impairment in Birmingham; MASTERPLAN: Multifactorial Approach and Superior Treatment Efficacy in Renal Patients with the Aid of a Nurse Practicioner; MDRD: Modification of Diet in Renal Disease; RENAAL: Reduction of Endpoints in Non-insulin Dependent Diabetes Mellitus with the Angiotensin II Antagonist Losartan; Steno: Steno Type 1 Diabetes Study

Figure 2.

Figure 2

Forest plot of adjusted hazard ratio for mortality associated with (A) a 15 mL/min/1.73 m2 lower estimated glomerular filtration rate (below an eGFR of 45 mL/min/1.73 m2) and (B) an 8-fold higher albumin:creatinine ratio or protein:creatinine ratio. The models are adjusted for age, sex, race, prior cardiovascular disease, smoking status, diabetes mellitus, systolic blood pressure, serum total cholesterol concentration and albuminuria (A) or eGFR splines (B).

The incidence rate of mortality also varied within albuminuria categories, with a higher incidence with higher albuminuria categories in most studies (Figure 1.b.) After adjustment, a higher albuminuria category also was associated with the risk of mortality (Table 4). The third category of albuminuria, compared to the lowest category, had a statistically significant association with mortality risk in 2 of the 5 studies with ACR, 2 of 3 studies with PCR, and the one study with dipstick proteinuria. After additional adjustment for eGFR category, an eight-fold higher ACR was significantly associated with mortality risk in 4 of 7 studies and an eight-fold higher PCR was significantly associated with mortality in all 3 studies (Figure 2.b). The pooled HR for ACR studies (1.36; 95% CI: 1.16, 1.59) was very similar to the pooled estimate for PCR studies (1.46; 95% CI: 1.28, 1.66), without evidence of significant heterogeneity overall (I2=39.9%; p=0.10).

Table 4.

Adjusted* hazard ratio (95% confidence interval) for mortality, by albuminuria category

Albumin:Creatinine Ratio (mg/g)
<30 30–299 300–999 ≥1,000
British Columbia Reference 1.49 (1.26, 1.77) 2.42 (2.09, 2.81) 3.01 (2.51, 3.62)
CRIB Reference 1.65 (0.83, 3.27) 2.15 (1.05, 4.40) 3.56 (1.80, 7.02)
Grampian-ACR Reference 1.14 (0.05, 27.74) 14.91 (0.60, 369.78) 43.91 (1.90, 1014.98)
MASTERPLAN Reference 1.12 (0.58, 2.14) 0.53 (0.19, 1.46) 1.10 (0.41, 2.90)
Steno Reference 2.39 (0.96, 5.93) 1.95 (0.81, 4.69) 2.32 (0.94, 5.76)

Overall Reference 1.50 (1.28, 1.75) 1.85 (1.08, 3.16) 2.73 (1.74, 4.26)

Protein:Creatinine Ratio (mg/g)
<50 50–499 500–1,499 ≥1,500

AASK Reference 1.82 (1.35, 2.45) 1.93 (1.30, 2.87) 2.60 (1.54, 4.40)
Grampian-PCR Reference 0.53 (0.22, 1.27) 0.99 (0.40, 2.45) 0.73 (0.25, 2.14)
MDRD Reference 0.99 (0.52, 1.88) 2.17 (1.06, 4.46) 1.80 (0.85, 3.80)

Overall Reference 1.08 (0.54, 2.18) 1.81 (1.30, 2.53) 1.72 (0.90, 3.29)

Dipstick Category
−/± + ++ +++

Kaiser Permanente Northwest Reference 1.46 (1.16, 1.82) 1.58 (1.28, 1.95) 1.98 (1.48, 2.64)

OVERALL Reference 1.46 (1.24, 1.71) 1.80 (1.38, 2.35) 2.26 (1.68, 3.04)
*

Adjusted for age, sex, race, prior cardiovascular disease, smoking status, diabetes mellitus, systolic blood pressure, and serum total cholesterol concentration.

AASK: African American Study of Kidney Disease and Hypertension; CRIB: Chronic Renal Impairment in Birmingham; MASTERPLAN: Multifactorial Approach and Superior Treatment Efficacy in Renal Patients with the Aid of a Nurse Practicioner; MDRD: Modification of Diet in Renal Disease; Steno: Steno Type 1 Diabetes Study

Independent associations of estimated GFR and albuminuria with ESRD

The incidence rate of ESRD was markedly greater with lower eGFR categories (Figure 3.a.). After adjustment, 9 of 11 studies had a significantly higher HR for an eGFR of 30–44 mL/min/1.73 m2 compared to 45–74 mL/min/1.73 m2, with a pooled HR of 2.72 (95% CI: 1.29, 3.37) (Table 5). All 11 studies had a significantly elevated HR for an eGFR of 15–29 mL/min/1.73 m2 compared to 45–74 mL/min/1.73 m2. In continuous analyses additionally adjusting for ACR, PCR or dipstick proteinuria category as appropriate, below an eGFR of 45 mL/min/1.73 m2 the association between a 15 mL/min/1.73 m2 lower eGFR and ESRD was statistically significant in all 12 studies (Figure 4.a). Each 15 mL/min/1.73 m2 lower eGFR was associated with a 6.24-fold (95% CI: 4.84, 8.05) higher risk of ESRD after adjustment for albuminuria and the other covariates. There was significant heterogeneity between studies in the HR estimates (I2=87.9%; p<0.001).

Figure 3.

Figure 3

Crude incidence rate of end-stage renal disease (per 1,000 person-years) by (A) category of estimated glomerular filtration rate and (B) category of albuminuria. Solid lines represent studies which assessed albuminuria using albumin:creatinine ratio (categories: <30, 30–299, 300–999, ≥1,000 mg/g). Dashed lines represent studies which assessed albuminuria using protein:creatinine ratio (categories: <50, 50–499, 500–1,499, ≥1,500 mg/g). Dotted lines represent studies which assessed albuminuria using dipstick protein (categories: −/±, +, ++, ≥+++). Points with ≤5 participants are excluded.

Table 5.

Adjusted* hazard ratio (95% confidence interval) for end-stage renal disease, by estimated GFR category

Estimated glomerular filtration rate (mL/min/1.73 m2)
45–74 30–44 15–29 <15
Studies with Albumin:Creatinine Ratio
British Columbia 1.0 (reference) 1.90 (1.54, 2.35) 8.34 (6.90, 10.07) 25.97 (21.24, 31.75)
MASTERPLAN 1.0 (reference) 2.51 (0.28, 22.51) 40.66 (5.57, 296.57) 203.60 (25.68, 1614.08)
NephroTest 1.0 (reference) 3.75 (1.41, 9.96) 14.67 (5.88, 36.61) 75.80 (29.73, 193.26)
RENAAL 1.0 (reference) 2.66 (1.85, 3.82) 9.33 (6.50, 13.40) -
Steno 1.0 (reference) 4.17 (1.89, 9.18) 13.08 (6.05, 28.28) 156.93 (34.44, 715.10)

Studies with Protein:Creatinine Ratio
AASK 1.0 (reference) 3.49 (2.49, 4.88) 12.24 (8.88, 17.02) 118.85 (27.53, 513.09)
MDRD 1.0 (reference) 2.68 (1.92, 3.73) 6.75 (4.87, 9.34) 27.35 (17.85, 41.90)
MMKD 1.0 (reference) 9.34 (2.10, 41.52) 21.25 (5.01, 90.05) 121.44 (28.12, 524.42)
REIN 1.0 (reference) 3.69 (1.67, 8.14) 11.13 (5.27, 23.49) 63.43 (20.61, 195.19)
REIN 2 1.0 (reference) 1.44 (0.30, 6.91) 8.59 (2.06, 35.85) 27.37 (6.22, 120.48)

Studies with Dipstick Proteinuria
Kaiser Permanente Northwest 1.0 (reference) 2.14 (1.29, 3.55) 15.08 (9.24, 14.60) -

OVERALL 1.0 (reference) 2.72 (2.19, 3.37) 10.21 (8.36, 12.46) 51.48 (31.95, 82.97)
*

Adjusted for age, sex, race, prior cardiovascular disease, smoking status, diabetes mellitus, systolic blood pressure, and serum total cholesterol concentration.

AASK: African American Study of Kidney Disease and Hypertension; CRIB: Chronic Renal Impairment in Birmingham; MASTERPLAN: Multifactorial Approach and Superior Treatment Efficacy in Renal Patients with the Aid of a Nurse Practicioner; MDRD: Modification of Diet in Renal Disease; MMKD: Mild to Moderate Kidney Disease Study; REIN: Ramipril Efficacy in Nephropathy; REIN 2: Ramipril Efficacy in Nephropathy 2; RENAAL: Reduction of Endpoints in Non-insulin Dependent Diabetes Mellitus with the Angiotensin II Antagonist Losartan; Steno: Steno Type 1 Diabetes Study

Figure 4.

Figure 4

Forest plot of adjusted hazard ratio for end-stage renal disease associated with (A) a 15 mL/min/1.73 m2 lower estimated glomerular filtration rate (below an eGFR of 45 mL/min/1.73 m2) and (B) an 8-fold higher albumin:creatinine ratio or protein:creatinine ratio. The models are adjusted for age, sex, race, prior cardiovascular disease, smoking status, diabetes mellitus, systolic blood pressure, serum total cholesterol concentration and albuminuria (A) or eGFR splines (B).

The incidence rate of ESRD was also markedly greater with higher albuminuria categories (Figure 3.b). The association of higher albuminuria category and risk of ESRD remained strong after adjustment (Table 6). The third category of albuminuria, compared to the lowest category, had a statistically significant association with ESRD risk in all 4 studies with ACR, 2 of 3 studies with PCR, and the one study with dipstick proteinuria. In continuous analyses, additionally adjusted for eGFR splines, an eight-fold higher ACR or PCR was significantly associated with ESRD risk in all 11 studies (Figure 4.b). The pooled HR for ACR studies (2.92; (95% CI: 1.96, 4.35) was similar to the pooled estimate for PCR studies (3.42; 95% CI: 1.84, 6.37). In analyses combining studies with ACR and PCR, an 8-fold higher ACR or PCR was associated with a 3.04-fold (95% CI: 2.27, 4.08) higher risk of ESRD after adjustment for eGFR and the other covariates. There was evidence of significant heterogeneity between ACR studies and between PCR studies and overall (I2=93.6%; p<0.001).

Table 6.

Adjusted* hazard ratio (95% confidence interval) for end-stage renal disease, by albuminuria category

Albumin:Creatinine Ratio (mg/g)
<30 30–299 300–999 ≥1,000
British Columbia Reference 2.42 (1.80, 3.26) 7.58 (5.83, 9.86) 12.91 (9.82, 16.95)
CRIB Reference 9.78 (2.35, 40.63) 16.39 (3.95, 68.02) 29.72 (7.18, 123.10)
MASTERPLAN Reference 4.14 (1.22, 14.04) 5.28 (1.49, 18.72) 18.87 (5.57, 63.91)
NephroTest Reference 2.55 (1.26, 5.13) 10.40 (5.31, 20.37) 21.38 (10.86, 42.08)

Overall Reference 2. 87 (1. 91, 4. 34) 7. 96 (6. 27, 10. 09) 14. 61 (11. 16, 19. 13)

Protein:Creatinine Ratio (mg/g)
<50 50–499 500–1,499 ≥1,500

AASK Reference 5.63 (3.49, 9.10) 19.26 (11.75, 31.57) 34.60 (19.76, 60.58)
MDRD Reference 2.18 (1.60, 2.99) 2.87 (2.06, 4.00) 4.47 (3.14, 6.37)
MMKD Reference 1.77 (0.23, 13.82) 4.03 (0.54, 30.23) 4.30 (0.58, 31.74)

Overall Reference 3.18 (1.40, 7.18) 6.38 (1.34, 30.34) 9.47 (1.81, 49.60)

Dipstick Category
−/± + ++ +++

Kaiser Permanente Northwest Reference 1.71 (0.76, 3.84) 7.22 (4.21, 12.97) 19.41 (11.06, 34.04)

OVERALL Reference 2. 92(2. 08, 4. 10) 7. 70(4. 52, 13. 10) 15. 01(8. 36, 26. 95)
*

Adjusted for age, sex, race, prior cardiovascular disease, smoking status, diabetes mellitus, systolic blood pressure, and serum total cholesterol concentration.

AASK: African American Study of Kidney Disease and Hypertension; CRIB: Chronic Renal Impairment in Birmingham; MASTERPLAN: Multifactorial Approach and Superior Treatment Efficacy in Renal Patients with the Aid of a Nurse Practicioner; MDRD: Modification of Diet in Renal Disease; MMKD: Mild to Moderate Kidney Disease Study; REIN: Ramipril Efficacy in Nephropathy; REIN 2: Ramipril Efficacy in Nephropathy 2; Steno: Steno Type 1 Diabetes Study

Discussion

In this meta-analysis of 13 cohorts, including 21,688 of individuals selected because of CKD, we found that lower eGFR and higher albuminuria were each independently associated with mortality and ESRD. Both eGFR and albuminuria were more strongly associated with ESRD than with mortality in these cohorts of individuals with CKD. To our knowledge, this represents the largest and most generalizable study of CKD populations in which these independent associations have been explored, and confirms the strong and independent relationship of higher level of albuminuria and lower levels of eGFR with mortality and ESRD across a wide range of clinical settings. The use of a uniform analysis plan also allows for comparisons of the magnitude of the associations across the studies included.

Compared to an eGFR of 45–74 mL/min/1.73 m2, progressively lower eGFR was associated with progressively greater risk of death and ESRD. The range of eGFR in the selected reference group was necessarily broad to enable analyses, and this precludes a precise determination of the level of eGFR below which risk begins increasing or statements about subdividing CKD stage 3 which are better addressed in higher GFR populations. All CKD studies had data below an eGFR of 45 mL/min/1.73 m2, where each 15 mL/min/1.73 m2 lower eGFR was associated with a 47% higher risk of death and a 6-fold higher risk of ESRD after adjustment for albuminuria. These results are consistent with the use eGFR stages in classification of CKD. Similarly, the relatively low number of participants and few events among individuals with negative dipstick results precluded separating negative dipstick from trace proteinuria. Including trace proteinuria in the reference group would result in lower hazard ratios for +, ++ and +++ categories than would result from using only negative proteinuria as the reference group. Future studies should explore the clinical implications of trace dipstick proteinuria among individuals with known CKD.

We also found that higher albuminuria was associated with greater risk of death and strongly associated with greater risk of ESRD. The risk increased progressively with every higher level of albuminuria, and the associations were similar for studies using ACR, PCR or dipstick proteinuria. An eight-fold higher ACR or PCR was associated with an estimated 40% higher risk of death and an estimated 3-fold higher risk of ESRD after adjustment for eGFR. The current CKD classification system does not discriminate by severity of albuminuria among individuals with CKD.[19] These results suggest that adding data on the presence and severity of albuminuria to eGFR stages provides more prognostic information than the current classification system. This may be especially true for predicting the risk of ESRD. There were no substantial differences in the associations of ACR or PCR with either death or ESRD, suggesting that each measure provides useful prognostic information of broadly equal importance.

The associations of lower eGFR with higher risk of outcomes remained after adjustment for severity of albuminuria and, likewise, the associations of higher albuminuria remained after adjustment for eGFR. These results demonstrate that both lower eGFR and higher albuminuria predict mortality and ESRD, independent of one another.

There was wide variation in the incidence rates for death and for ESRD across the cohorts included in this meta-analysis, presumably reflecting variation in definition of CKD, causes and pathology of kidney disease, range in eGFR and albuminuria, comorbid conditions and other eligibility criteria. We evaluated statistical heterogeneity using the I2 statistic, which reflects the variability among effect sizes due to between-study differences as a percentage of the total variability. The within-study variation is relatively small because the sample sizes of the studies included are relatively large. Thus, the percentage of total variability that is due to between-study variation is relatively large, but does not reflect clinically important inconsistencies in relative risks, as shown in the forest plots.

The incidence rates of death and ESRD were of roughly similar magnitude within most studies. This differs substantially from the general population, in which the incidence rate of death is at least 10 times higher than the incidence rate of ESRD.[20] This difference may be due to a higher incidence of ESRD among individuals selected for CKD than individuals with CKD in the general population, which reduces the influence of the competing risk of mortality when studying incident ESRD. This emphasizes the differences in competing risk for ESRD and mortality, as well as other outcomes, in different study populations.

Lower eGFR and higher albuminuria were much more strongly associated with ESRD than with mortality. Decreased GFR and albuminuria are primary factors in the development of ESRD, whereas numerous comorbid conditions may affect the risk of mortality in these study participants. The much higher incidence of ESRD, relative to the incidence of death in these CKD cohorts, reduces the impact of the competing risk of death and may allow the direct associations between eGFR and albuminuria with ESRD to be observed more accurately than in the general population. These analyses of individuals with CKD also include a much higher proportion of deaths that occurred after ESRD than studies in the general population. Another factor to consider is the selection of participants for the studies included in this meta-analysis. Individuals with relatively high eGFR and/or low albuminuria who are enrolled in these studies likely have more severe comorbid conditions than most individuals with similar eGFR and albuminuria included in general population studies. If these comorbid conditions put the enrolled participants at higher risk of death, the observed associations of eGFR and albuminuria with mortality could be substantially confounded. These comorbid conditions may be less strongly associated with progression to ESRD and, therefore, would have less impact on the observed associations of eGFR and albuminuria with ESRD.

The populations of the participating studies comprise highly selected patients, with the majority enrolled in a randomized clinical trial or referred to a nephrologist.[68;10;1417] The others are comprised of patients selected from clinical populations because of decreased eGFR or albuminuria.[11] There were no obvious differences in the hazard ratios relating either eGFR or albuminuria with the risk of either outcome according to whether the study was a randomized trial or an observational study, although the baseline characteristics and resultant incidence rates of both outcomes varied widely and the number of studies within each category was small, limiting the usefulness of formal meta-regression models. Within observational studies, differences in study populations may also exist between those with protocol-driven study visits (CRIB, NephroTest, MMKD) and those based on administrative data (British Columbia, Grampian ACR and Grampian PCR, Kaiser Permanente Northwest) or passive surveillance of clinical events (Steno). Among the studies included in this meta-analysis, those with protocol-driven examinations had higher incidence of ESRD than those based on administrative data. In addition, the incidence of ESRD was higher than the incidence of mortality among studies with protocol-driven examinations, whereas the incidence rates for these two outcomes were more similar among studies based on administrative data. We hypothesize that these differences are due to the differing selection criteria used to enroll participants in these two types of studies, such that studies with protocol-driven examinations would more likely enroll individuals with more severe disease and/or more comorbidities, whereas studies based on administrative data would be more likely to enroll all individuals eligible based on a limited number of easily-identified diagnostic or laboratory criteria. Despite the differences in ESRD incidence rates, we did not observe any consistent differences in the strength of the associations of eGFR and ACR with both outcomes (i.e., mortality and ESRD) across study types.

We acknowledge that this meta-analysis has limitations. First, we did not perform a systematic literature search to identify all potentially eligible cohorts. However, study selection was unbiased with respect to the associations of interest and most of the included cohorts had not reported or investigated these associations before we performed our pooled analysis. Selection bias is therefore unlikely. Our analyses do not assess changes in eGFR, albuminuria or covariates over time. Any bias from systematic differences in changes in the predictors after baseline, however, would presumably attenuate the associations between baseline predictors and outcomes. No data could be taken into account on effects of treatment that was started during follow-up. Therefore it cannot be excluded that the observed associations are influenced by the start of specific treatments. However, if such treatment is effective in preventing mortality and ESRD, as expected, then it would be expected to lead to an underestimation of the true relative risk of low eGFR and high albuminuria for these outcomes. There were no obvious differences in the associations by the proportion of diabetic individuals in the studies. It is difficult to isolate any effect modification by diabetes in this meta-analysis, however, since diabetic and non-diabetic participants were not analyzed separately, and the participating studies differed with respect to other inclusion and exclusion criteria as well.

Our findings have important implications for clinical practice and research in CKD. Traditionally, prognosis in CKD is based on the clinical diagnosis (cause and pathology) of CKD. Our findings that a higher level of albuminuria and lower level of eGFR increase the risk for mortality and ESRD consistently across cohorts with diverse clinical diagnosis in diverse settings suggests that these measures provide important prognostic information beyond clinical diagnosis, and should be considered in predicting prognosis. We acknowledge that the levels of eGFR and albuminuria among patients with CKD are likely to vary according to the clinical diagnosis and referral pattern. For example, patients with lower eGFR and higher albuminuria are likely to be younger and have primary kidney diseases, such as glomerular, tubulointerstitial, or cystic kidney diseases, while those with higher levels of eGFR or lower levels of albuminuria may be older and have kidney disease due to systemic vascular disease, such as hypertension or diabetes. Similarly, the risk for ESRD or mortality likely varies across clinical diagnosis of CKD and referral pattern, as well as by the level of albuminuria and eGFR. The risk for these outcomes is also influenced by age, sex, race, CVD risk factors and history of CVD. Based on our results, it seems likely that prognosis is determined by all these factors, and development of risk prediction models and scores for these outcomes will need to take all these factors into account. Future studies should also consider the influence of clinical diagnosis and level of eGFR and albuminuria on the risk of concurrent complications of CKD, such as hypertension, anemia, malnutrition, bone and mineral disorders, and electrolyte disorders, as well as for prognosis related to other outcomes associated with CKD, such as infection or cognitive impairment.

In conclusion, we found that lower eGFR and more severe albuminuria independently predict mortality and ESRD among individuals with CKD. The observed associations provide evidence consistent with the use of eGFR stages in classification of CKD, and suggest that addition of albuminuria stages may provide additional prognostic information among individuals with CKD.

Methods

Study selection

Studies were identified by the planning committee and analytic team and discussion between collaborators. This was enhanced by a call for participation at the World Congress of Nephrology in Milan (May, 2009), a published position statement of KDOQI and KDIGO,[5] and an announcement on the KDIGO website (www.kdigo.org). To be eligible for inclusion in this meta-analysis, studies had to include primarily participants selected because of CKD, provide information at baseline on estimated or measured GFR and either urinary albumin or urinary protein excretion, and include at least 50 ESRD events or deaths to ensure sufficient outcomes in the reference cell. The definitions of CKD used in each study are available in the specific references. Individuals with ESRD were excluded from all studies.

Study variables

Estimated GFR was calculated using the Modification of Diet in Renal Disease (MDRD) Study equation using age, sex, race and serum creatinine concentration.[1;21] Each participating study was asked to standardize their serum creatinine measurements to isotope dilution mass spectrometry (IDMS) traceable methods, but calibration was not uniform. Albuminuria was assessed as the urinary albumin-to-creatinine ratio (ACR) or urinary protein-to-creatinine ratio (PCR), preferably measured in a first morning void urine sample. If first morning voids were not available, spot urine samples or samples from 24hr urine collections were used. In studies in which no quantitative albuminuria measurements were available, data on dipstick proteinuria were collected.

History of cardiovascular disease (CVD) was defined as prior myocardial infarction, bypass grafting, percutaneous coronary intervention, heart failure or stroke. Hypertension was defined as systolic blood pressure (SBP) ≥ 140 mmHg or diastolic blood pressure ≥ 90 mmHg or use of antihypertensive medication. Hypercholesterolemia was defined as total cholesterol ≥ 5.0 mmol/L in the case of a positive history of CVD and as ≥ 6.0 mmol/L in the case of a negative history of CVD. Diabetes mellitus was defined as fasting glucose ≥ 7.0 mmol/L or non-fasting glucose ≥ 11.1 mmol/L or use of glucose lowering drugs or, if other data were not available, self-reported diabetes. Smoking status was dichotomized as current versus not current smoking. End-stage renal disease (ESRD) was defined as the start of renal replacement therapy or death due to decreased kidney function not due to acute kidney injury. All deaths occurring before or after ESRD were included in the analyses. The clinical diagnosis (cause and pathology) of CKD was not included because it was not ascertained in all studies and it was not ascertained uniformly across studies in which it was recorded.

Statistical analysis

The primary objective of this study was to evaluate the independent associations of eGFR and albuminuria with the risk of all-cause mortality and ESRD. Investigators from each study newly analyzed their data following an a priori analytic plan using standard statistical programs supplied by the central analysis team. All analyses were conducted using Stata version 10 or 11 (Stata Corp., College Station, Texas, USA), SAS version 9 (SAS Institute, Inc., Cary, North Carolina, USA) or R version 2.9.2 (R Foundation for Statistical Computing, Vienna, Austria).

Categories were created for eGFR (15–29, 30–44, 45–74, 75–89, 90–104 and ≥105 mL/min/1.73m2), ACR (<30, 30–299, 300–999, ≥ 1000 mg/g), PCR (<50, 50–499, 500–1499, ≥1500 mg/g), and dipstick proteinuria (negative/trace, 1+, 2+, ≥3+). As to be expected for CKD cohorts, limited data were available for eGFR categories above 74 mL/min/1.73 m2, so these results are not shown. Cox proportional hazards models were used to obtain adjusted hazard ratios for each category of eGFR relative to the reference group of 45–74 mL/min/1.73m2, and for each category of ACR / PCR / dipstick proteinuria (using the lowest category for each as the reference). A broad eGFR reference group was chosen because several studies had no events or few events in the narrower ranges of eGFR 60–74 or 45–59 mL/min/1.73 m2. These models were adjusted for age, sex, race, history of CVD, smoking status, diabetes mellitus, SBP, and serum total cholesterol concentration, where data were available. Cox proportional hazards models were also constructed with log ACR or log PCR and eGFR modeled as continuous variables adjusted for all the covariates in the categorical analysis. When log ACR or log PCR are modeled as linear terms results are presented for an 8-fold higher ACR or PCR. eGFR was modeled as a linear spline with knots at 45, 60, 75, 90 and 105 mL/min/1.73 m2. Since data were limited at the higher eGFR ranges, results are presented for 15 mL/min/1.73 m2 lower eGFR below 45 mL/min/1.73 m2. Data from specific studies were excluded from specific analyses if the range of values included in the study did not allow hazard ratios to be estimated (e.g., all ACR values > 300 mg/g) or if the study had no events in the reference category.

Pooled estimates of the HR and 95% confidence intervals (CI) were obtained from a random effects meta-analysis. Heterogeneity was estimated using the χ2 test for heterogeneity and the I2 statistic.[22] Meta-analyses were conducted separately for studies with ACR and PCR. The Grampian cohort included ACR data on some participants and PCR data on other participants. These were treated as two separate studies in the analyses.

Supplementary Material

Supplementary Material

Acknowledgments

The CKD Prognosis Consortium is supported by by Kidney Disease: Improving Global Outcomes (KDIGO) and the US National Kidney Foundation. The meta-analyses were conducted jointly at The Johns Hopkins Bloomberg School of Public Health, Baltimore, USA and University Medical Center Groningen, Groningen, The Netherlands, and were supported by the US National Kidney Foundation and the Dutch Kidney Foundation, respectively. KDIGO hosted the 2009 meeting of collaborators.

CKD Prognosis Consortium

Writing Committee: Brad C. Astor, Kunihiro Matsushita, Ron T. Gansevoort, Marije van der Velde, Mark Woodward, Andrew S. Levey, Paul E. de Jong, Josef Coresh

KDIGO Controversies Conference Planning Committee: Andrew S. Levey, Meguid El-Nahas, Paul E. de Jong, Josef Coresh, Kai-Uwe Eckardt, Bertram L. Kasiske.

CKD Prognosis Consortium investigators/collaborators: AASK: Jackson Wright, Larry Appel, Tom Greene; British Columbia CKD: Dr. Adeera Levin, Ognjenka Djurdjev; CRIB: David C Wheeler, Martin J. Landray, John N Townend, Jonathan Emberson; Grampian CKD: Laura E. Clark, Alison Macleod, Angharad Marks, Tariq Ali, Nicholas Fluck, Gordon Prescott; Kaiser Permanente Northwest: David H. Smith, Jessica R. Weinstein, Eric S. Johnson, Micah L. Thorp; MASTERPLAN: Jack F. Wetzels, P.J. Blankestijn, A.D. van Zuilen; MDRD: Vandana Menon, Mark Sarnak, Tom Greene, Gerald Beck; MMKD: Florian Kronenberg, MD; Barbara Kollerits, PhD, MPH; NephroTest: Marc Froissart, Benedicte Stengel, Marie Metzger; REIN 1 and 2: Giuseppe Remuzzi, Piero Ruggenenti, Annalisa Perna; RENAAL: H.J. Lambers Heerspink, Barry Brenner, Dick de Zeeuw; STENO: Peter Rossing, Hans-Henrik Parving.

CKD Prognosis Consortium Analytic Team: Brad C. Astor, Priscilla Auguste, Josef Coresh, Ron T. Gansevoort, Paul E de Jong, Kunihiro Matsushita, Marije van der Velde, Kasper Veldhuis, Yaping Wang, Mark Woodward.

CKD Prognosis Consortium Administration Staff: Laura Camarata, Beverly Thomas.

National Kidney Foundation Staff: Tom Manley.

Footnotes

Contributors

All members of the writing committee contributed to the collection and analysis of the data, and to the preparation of the report. All collaborators were sent the paper as prepared for submission and given the opportunity to comment on the draft manuscript. The writing committee accepts full responsibility for the content of this paper.

Conflict of interest statement

The members of the Writing Committee declare that they have no conflict of interests. A variety of institutions supported the cohorts contributing to the CKD Prognosis Consortium, as described in publications on these cohorts.

Reference List

  • 1.Levey AS. Chronic kidney disease definition, dlassification and prognosis: a KDIGO Controversies Conference reaches a consensus. Kidney Int. 2010 doi: 10.1038/ki.2010.483. (submitted) [DOI] [PubMed] [Google Scholar]
  • 2.The Chronic Kidney Disease Prognosis Consortium. Association of estimated lomerular filtration rate and albuminuria with all-cause and cardiovascular mortality: a collaborative meta-analysis of general population cohorts. Lancet. 2010 doi: 10.1016/S0140-6736(10)60674-5. (submitted) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.The Chronic Kidney Disease Prognosis Consortium. Association of estimated glomerular filtration rate and albuminuria with all-cause and cardiovascular mortality: a collaborative meta-analysis of high-risk cohorts. Kidney Int. 2010 doi: 10.1038/ki.2010.536. (submitted) [DOI] [PubMed] [Google Scholar]
  • 4.The Chronic Kidney Disease Prognosis Consortium. Association of estimated glomerular filtration rate and albuminuria with kidney outcomes: a collaborative meta-analysis of general and high-risk population cohorts. Kidney Int. 2010 doi: 10.1016/S0140-6736(10)60674-5. (submitted) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Eckardt KU, Berns JS, Rocco MV, Kasiske BL. Definition and classification of CKD: the debate should be about patient prognosis--a position statement from KDOQI and KDIGO. Am J Kidney Dis. 2009;53:915–920. doi: 10.1053/j.ajkd.2009.04.001. [DOI] [PubMed] [Google Scholar]
  • 6.Menon V, Wang X, Sarnak MJ, et al. Long-term outcomes in nondiabetic chronic kidney disease. Kidney Int. 2008;73:1310–1315. doi: 10.1038/ki.2008.67. [DOI] [PubMed] [Google Scholar]
  • 7.Wright JT, Jr, Bakris G, Greene T, et al. Effect of blood pressure lowering and antihypertensive drug class on progression of hypertensive kidney disease: results from the AASK trial. JAMA. 2002;288:2421–2431. doi: 10.1001/jama.288.19.2421. [DOI] [PubMed] [Google Scholar]
  • 8.Brenner BM, Cooper ME, de ZD, et al. Effects of losartan on renal and cardiovascular outcomes in patients with type 2 diabetes and nephropathy. N Engl J Med. 2001;345:861–869. doi: 10.1056/NEJMoa011161. [DOI] [PubMed] [Google Scholar]
  • 9.Levin A, Djurdjev O, Beaulieu M, Er L. Variability and risk factors for kidney disease progression and death following attainment of stage 4 CKD in a referred cohort. Am J Kidney Dis. 2008;52:661–671. doi: 10.1053/j.ajkd.2008.06.023. [DOI] [PubMed] [Google Scholar]
  • 10.Landray MJ, Thambyrajah J, McGlynn FJ, et al. Epidemiological evaluation of known and suspected cardiovascular risk factors in chronic renal impairment. Am J Kidney Dis. 2001;38:537–546. doi: 10.1053/ajkd.2001.26850. [DOI] [PubMed] [Google Scholar]
  • 11.Johnson ES, Thorp ML, Yang X, et al. Predicting renal replacement therapy and mortality in CKD. Am J Kidney Dis. 2007;50:559–565. doi: 10.1053/j.ajkd.2007.07.006. [DOI] [PubMed] [Google Scholar]
  • 12.Van Zuilen AD, Wetzels JF, Bots ML, Van Blankestijn PJ. MASTERPLAN: study of the role of nurse practitioners in a multifactorial intervention to reduce cardiovascular risk in chronic kidney disease patients. J Nephrol. 2008;21:261–267. [PubMed] [Google Scholar]
  • 13.Dieplinger B, Mueller T, Kollerits B, et al. Pro-A-type natriuretic peptide and pro-adrenomedullin predict progression of chronic kidney disease: the MMKD Study. Kidney Int. 2009;75:408–414. doi: 10.1038/ki.2008.560. [DOI] [PubMed] [Google Scholar]
  • 14.Moranne O, Froissart M, Rossert J, et al. Timing of onset of CKD-related metabolic complications. J Am Soc Nephrol. 2009;20:164–171. doi: 10.1681/ASN.2008020159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Ruggenenti P, Perna A, Remuzzi G. ACE inhibitors to prevent end-stage renal disease: when to start and why possibly never to stop: a post hoc analysis of the REIN trial results. Ramipril Efficacy in Nephropathy. J Am Soc Nephrol. 2001;12:2832–2837. doi: 10.1681/ASN.V12122832. [DOI] [PubMed] [Google Scholar]
  • 16.Ruggenenti P, Perna A, Loriga G, et al. Blood-pressure control for renoprotection in patients with non-diabetic chronic renal disease (REIN-2): multicentre, randomised controlled trial. Lancet. 2005;365:939–946. doi: 10.1016/S0140-6736(05)71082-5. [DOI] [PubMed] [Google Scholar]
  • 17.Hovind P, Tarnow L, Rossing P, et al. Predictors for the development of microalbuminuria and macroalbuminuria in patients with type 1 diabetes: inception cohort study. BMJ. 2004;328:1105. doi: 10.1136/bmj.38070.450891.FE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Clark LE, Prescott G, Fluck N, et al. Reduced eGFR - does this mean CKD? (abstract) Nephrol Dial Transplant. 2007;22(Suppl 9):ix54–ix62. [Google Scholar]
  • 19.K/DOQI clinical practice guidelines for chronic kidney disease: evaluation, classification, and stratification. Am J Kidney Dis. 2002;39:S1–266. [PubMed] [Google Scholar]
  • 20.Collins AJ, Foley RN, Herzog C, et al. Excerpts from the US Renal Data System 2009 Annual Data Report. Am J Kidney Dis. 2010;55:S1–S7. doi: 10.1053/j.ajkd.2009.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Levey AS, Coresh J, Greene T, et al. Using standardized serum creatinine values in the modification of diet in renal disease study equation for estimating glomerular filtration rate. Ann Intern Med. 2006;145:247–254. doi: 10.7326/0003-4819-145-4-200608150-00004. [DOI] [PubMed] [Google Scholar]
  • 22.Woodward M. Epidemiology: study design and data analysis. 2. Boca Raton, FL: Chapman & Hall/CRC; 2005. [Google Scholar]
  • 23.Clark LE, Prescott G, Fluck N, et al. Reduced eGFR-does this mean CKD? (abstract) Nephrol Dial Transplant. 2007;22(Supple 9):ix54–ix62. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material

RESOURCES