Abstract
Background:
New estimated glomerular filtration rate (eGFR) equations removed race adjustment, but the impact of its removal on prediction of end-stage kidney disease (ESKD) is unknown.
Objective:
To compare the ESKD prediction performance of different eGFR equations.
Design:
Observational, prospective cohort study.
Setting:
Seven US clinical centers.
Participants:
3,873 participants with chronic kidney disease (CKD) from the Chronic Renal Insufficiency Cohort (CRIC) Study contributing 13,902 two-year risk periods.
Measurements:
ESKD was defined as initiation of dialysis or transplantation. eGFR was calculated using five CKD-EPI equations based on serum creatinine and/or cystatin C, with or without race adjustment. The predicted 2-year risk of ESKD was calculated using the 4-variable Kidney Failure Risk Equation (KFRE). We evaluated the prediction performance of eGFR equations and the KFRE score using discrimination and calibration analyses.
Results:
During a maximum 16 years of follow-up, 856 participants developed ESKD. Across all eGFR equations, the KFRE score was superior for predicting 2-year incidence of ESKD compared with eGFR alone (area-under-the-curve ranges, 0.945-0.954 vs. 0.900-0.927). Prediction performance of KFRE scores using different eGFR equations was similar, but the creatinine equation without race adjustment improved calibration among Black participants. Among all participants, compared with an eGFR <20 mL/min/1.73 m2, a KFRE score >20% had similar specificity for predicting 2-year ESKD risk (ranges, 0.94-0.97 vs. 0.95-0.98) but higher sensitivity (ranges, 0.68-0.78 vs. 0.42-0.66).
Limitations:
Data are solely from the US.
Conclusions:
The KFRE score better predicts 2-year risk of ESKD compared with eGFR alone, regardless of race adjustment. The creatinine equation with age and sex may improve calibration among Black patients. A KFRE score >20% showed high specificity and sensitivity for predicting 2-year risk of ESKD.
Introduction
Lower kidney function, as quantified by glomerular filtration rate (GFR), is strongly associated with risks of end-stage kidney disease (ESKD) and all-cause mortality, and is a major public health challenge (1-4). In clinical practice, equations are used to provide estimated GFR (eGFR) based on a combination of endogenous filtration markers (serum creatinine and/or cystatin C) and patient characteristics including age, sex, and race. Race was included in equations using serum creatinine (5,6) because studies show a higher average serum creatinine level in Black vs. non-Black individuals that may not be explained by non-GFR determinants (7). However, race is an ill-defined social construct, and its use in medicine may perpetuate racial inequities in healthcare delivery (8-10). In response to this issue, new eGFR equations without race adjustment were released in 2021 by the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) (11).
Clinical cut-off points based on eGFR are typically used to stage CKD and determine transplant eligibility and referral for dialysis (12,13). However, the equation used to estimate GFR can substantially impact these decisions (11). An alternative is to base decisions on predicted risk using tools that incorporate more information than eGFR alone. The 4-variable Kidney Failure Risk Equation (KFRE), which includes age, sex, eGFR, and urinary albumin-to-creatinine ratio (ACR), has been developed (14) and validated in multinational samples (15). Clinical guidelines mention the KFRE but cite several barriers to its use, and still use eGFR as a primary criterion for decision-making (12,13).
The KFRE may improve clinical decision-making compared with eGFR alone, but it is unknown how the 2021 CKD-EPI equations affect predicted ESKD risk when used within the KFRE in Black and non-Black patients with CKD. The aim of this study was to compare old and new CKD-EPI eGFR equations, alone and used in the KFRE, on their performance for predicting 2-year incidence of ESKD and to evaluate corresponding clinical cut-off points for decision-making in CKD patients.
Methods
Study Design and Participants
The CRIC Study is a prospective cohort study of a racially and ethnically diverse group of 3,939 men and women aged 21 to 74 years with mild-to-moderate CKD, based on an eGFR entry criterion of 20 to 70 mL/min/1.73 m2, enrolled from 7 US clinical centers between 2003 and 2008 with follow-up through May, 2020 (16,17). Patients with cirrhosis, HIV infection, polycystic kidney disease, or renal cell carcinoma; those receiving dialysis or having an organ transplant; or those taking immunosuppressive medications were excluded. Participants attended annual in-person follow-up clinical visits. For the current analysis, we used available follow-up visit data by allowing participants to contribute multiple non-overlapping 2-year periods at risk for ESKD. Periods began at the original study baseline and years 2, 4, 6, 8, 10, 12, and 14. Therefore, we included participants with at least 1 follow-up visit with complete data needed to estimate GFR and calculate the KFRE score. We excluded 66 participants missing urinary protein-to-creatinine ratio (PCR) measurements over follow-up, yielding a final analysis sample of 3,873 participants who contributed a total of 13,902 two-year periods at risk for ESKD (Appendix Table 1). The CRIC Study was approved by the institutional review boards from each clinical center, and all participants provided written informed consent.
Exposure and Covariable Assessment
All CRIC Study data were collected by trained study staff at baseline and annual clinical follow-up visits using data collection procedures standardized across clinical centers. Self-reported sociodemographic characteristics, medical history, and current medications were obtained via questionnaire. Participants self-reporting non-Hispanic Black race were classified as Black race, while those self-reporting non-Hispanic white, Hispanic, or other race/ethnicity were classified as non-Black race (11). Serum creatinine was measured using an enzymatic method with calibration traceable to an IDMS reference measurement procedure and serum cystatin C was measured by a particle-enhanced immunonephelometric assay with internal standardization (18). Urine protein and creatinine were measured in 24-hour or spot urine samples. Urinary PCR measures were converted to urinary ACR values using a validated conversion equation (19). Body weight, height, blood pressure, and cholesterol were measured using standard protocols (16). Diabetes was defined as fasting glucose ≥7.0 mmol/L (126 mg/dL), non-fasting glucose ≥11.1 mmol/L (200 mg/dL), and/or the use of glucose-lowering medications.
We considered 5 CKD-EPI equations, including those updated in 2021, for calculating eGFR based on serum creatinine and/or cystatin C, and with or without adjustment for Black race (Appendix Table 2) (11). In addition, we calculated 2-year predicted risk of ESKD for each eGFR equation by using the respective values with the 4-variable KFRE, which includes age, sex, eGFR, and urinary ACR (14). We used the equation coefficients from the model recalibrated for use in North American populations like the CRIC Study sample (15).
Outcome Assessment
The primary outcome for the current analysis was ESKD, defined as the receipt of dialysis or kidney transplantation. Information on the initiation and maintenance of dialysis and kidney transplant was obtained by annual clinical follow-up visits and interim telephone interviews, and confirmed by dialysis unit. Ascertainment of ESKD was supplemented by information from the US Renal Data System. Given that the current analysis focuses on prediction of 2-year risk of ESKD, for each 2-year period at risk follow-up time was censored at the earliest occurrence of death, loss to follow-up, or 2 years of complete follow-up. Once a participant was lost to follow-up, died, or had an ESKD event, they did not contribute further 2-year periods at risk.
Statistical Analysis
All analyses were stratified by self-reported race (Black vs. non-Black). Baseline characteristics of the study participants were summarized as the mean (SD) or median (IQR) for continuous variables and number (percentage) for categorical variables. We calculated absolute 2-year risk of ESKD using Kaplan-Meier survival analysis.
We evaluated the 2-year ESKD risk prediction performance of eGFR equations alone and predicted probabilities from the published KFRE scores using measures of discrimination and calibration (20). Discrimination refers to the ability of a model to correctly identify those who will or will not experience the outcome and was assessed using the time-dependent (i.e., 2-year) area under the receiver operating characteristic curve (AUC), with higher AUCs indicating better discrimination (21). We compared discrimination between eGFR alone and the KFRE score using the integrated discrimination improvement (IDI) statistic, which indicates the average increase in sensitivity given no changes in specificity (22). Calibration refers to the agreement between observed outcomes and model predictions, and was visualized by plotting the observed vs. predicted risk across deciles of predicted risk. Overall prediction accuracy was quantified using the scaled Brier score (23). A higher scaled Brier score indicates a better performing model, a score of 1.0 indicates a perfect model, and a score ≤0 indicates a useless model.
We evaluated the validity of specific eGFR level and KFRE score cut-off points in predicting 2-year risk of ESKD based on sensitivity and specificity. We used Youden’s index as an overall metric of performance that considers both sensitivity and specificity, which is calculated as [sensitivity] + [specificity] – 1 (20). Although eGFR cut-off points of 15, 20, and 30 mL/min/1.73 m2 are cited in clinical guidelines for referral to transplant or vascular access (12,13,24), there are no established cut-off points for the KFRE score. Therefore, we evaluated and compared 9 eGFR and 13 KFRE score cut-off points for each eGFR equation.
We used bootstrap resampling with 1000 replicates to generate 95% confidence intervals (CIs). We accounted for within-participant correlation over follow-up by resampling a participant’s entire follow-up data in each bootstrap resample. All analyses were conducted using SAS version 9.4 (SAS Institute, Cary, North Carolina) and R version 4.1.1 (R Project for Statistical Computing).
Role of the Funding Source
The funders had no role in the design of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.
Results
Among 3,873 participants included in the analyses, the mean (SD) age was 57.8 (10.9) years, 54.8% were male, and 42.1% self-reported Black race. Compared with non-Black participants, Black participants were more likely to be current smokers, have diabetes, and be taking blood pressure-lowering medications (Table 1). On average, Black participants had higher levels of body mass index, glycated hemoglobin, and systolic and diastolic blood pressure. Regardless of eGFR equation employed, Black participants had lower eGFR, and higher predicted 2-year ESKD risk, on average compared with non-Black participants. Among Black participants, the creatinine equation with age and sex yielded the lowest eGFR on average while the cystatin C equation had the highest eGFR. Consequently, the KFRE scores using the creatinine equation with age and sex, and cystatin C equation with age and sex, produced the highest and lowest estimates of 2-year ESKD risk, respectively.
Table 1.
Baseline Characteristics of Chronic Renal Insufficiency Cohort (CRIC) Study Participants by Self-reported Race
| Variables | Overall (n=3873) | Self-reported Race | |
|---|---|---|---|
| Black (n=1631) | Non-Black (n=2242) | ||
| Age, mean (SD), years | 57.8 (10.9) | 57.8 (10.5) | 57.8 (11.2) |
| Male, No. (%) | 2122 (54.8) | 796 (48.8) | 1326 (59.1) |
| Body mass index, mean (SD), kg/m2 | 32.1 (7.8) | 33.5 (8.3) | 31.1 (7.3) |
| Current smoking, No. (%) | 510 (13.2) | 318 (19.5) | 192 (8.6) |
| Hemoglobin A1C, mean (SD), % | 6.6 (1.6) | 6.9 (1.7) | 6.5 (1.4) |
| Diabetes mellitus, No. (%) | 1868 (48.2) | 842 (51.6) | 1026 (45.8) |
| Use of glucose-lowering medications, No. (%) | 1063 (27.6) | 467 (28.8) | 596 (26.8) |
| Systolic blood pressure, mean (SD), mm Hg | 128.3 (22.0) | 132.6 (22.9) | 125.1 (20.8) |
| Diastolic blood pressure, mean (SD), mm Hg | 71.4 (12.7) | 73.6 (13.7) | 69.9 (11.7) |
| Use of blood pressure-lowering medications, No. (%) | 3536 (91.3) | 1547 (94.8) | 1989 (88.7) |
| Urinary PCR, median [IQR], mg/g | 154.2 [58.1, 785.2] | 199.3 [61.8, 870.3] | 132.7 [55.6, 695.3] |
| Estimated glomerular filtration rate, mean (SD), mL/min/1.73 m2 * | |||
| Creatinine-based with age, sex, and race | 44.2 (15.0) | 43.6 (15.0) | 44.7 (15.0) |
| Creatinine-based with age and sex | 44.4 (15.3) | 40.0 (13.5) | 47.5 (15.7) |
| Cystatin C-based with age and sex | 52.8 (23.8) | 50.9 (23.1) | 54.2 (24.2) |
| Creatinine- and cystatin C-based with age, sex, and race | 47.3 (18.6) | 46.0 (18.3) | 48.2 (18.8) |
| Creatinine- and cystatin C-based with age and sex | 49.2 (19.6) | 46.0 (18.3) | 51.6 (20.2) |
| Predicted 2-year risk of ESKD, mean (SD), % † | |||
| Creatinine-based with age, sex, and race | 5.0 (10.9) | 5.3 (11.0) | 4.8 (10.8) |
| Creatinine-based with age and sex | 5.1 (11.0) | 6.5 (12.5) | 4.1 (9.7) |
| Cystatin C-based with age and sex | 4.8 (11.1) | 5.2 (11.1) | 4.5 (11.0) |
| Creatinine- and cystatin C-based with age, sex, and race | 5.2 (11.3) | 5.6 (11.4) | 4.9 (11.3) |
| Creatinine- and cystatin C-based with age and sex | 4.9 (11.0) | 5.7 (11.7) | 4.3 (10.4) |
| 2-year Kaplan-Meier observed risk of ESKD, % (95% CI) | 5.9 (5.2, 6.7) | 6.5 (5.3, 7.7) | 5.5 (4.6, 6.5) |
ACR = albumin-to-creatinine ratio; CKD-EPI = Chronic Kidney Disease Epidemiology Collaboration; ESKD = end-stage kidney disease; IQR = interquartile range; PCR = protein-to-creatinine ratio; SD = standard deviation
Calculated based on the CKD-EPI equations
Calculated using the 4-variable Kidney Failure Risk Equation (including age, sex, eGFR, and urinary ACR) with glomerular filtration rate estimated using the CKD-EPI equations and urinary ACR converted from urinary PCR
During a mean 9.3-year follow-up (16 years maximum), 856 ESKD events were recorded within the 2-year risk periods (Appendix Table 1). Compared with non-Black participants, Black participants had higher observed 2-year risk of ESKD at baseline (6.5% [5.3-7.7%] vs. 5.5% [4.6-6.5%]).
As shown in Figure 1, across all eGFR equations, the KFRE score was superior for prediction of 2-year risk of ESKD compared with eGFR alone among Black participants (AUC ranges, 0.945-0.953 vs. 0.908-0.927; IDI range, 0.105-0.132) and non-Black participants (AUC ranges, 0.945-0.954 vs. 0.900-0.923; IDI range, 0.105-0.164). In general, among both Black and non-Black participants, discrimination for the KFRE score was similar regardless of eGFR equation employed, characterized by large AUC values with substantially overlapping confidence intervals and ROC curves (Figure 2, panels A and B). The KFRE score underestimated risk for both Black and non-Black participants regardless of which eGFR equation was incorporated, although creatinine-based equations performed better (i.e., had higher scaled Brier scores) than cystatin C-based equations (Figure 2, panels C and D). Accounting for race in addition to age and sex did not improve calibration.
Figure 1.
Discrimination of ESKD Events by eGFR and KFRE Score using Various Equations AUC = area under the receiver operating characteristic curve; CI = confidence interval; CKD-EPI = Chronic Kidney Disease Epidemiology Collaboration; eGFR = estimated glomerular filtration rate; IDI = integrated discrimination improvement; KFRE = Kidney Failure Risk Equation
Higher values for AUC indicate better performing models.
Figure 2. Discrimination and Calibration of ESKD Events by KFRE Score using Various Equations.
AUC = area under the receiver operating characteristic curve; CI = confidence interval; KFRE = kidney failure risk equation.
Higher values for AUC and scaled Brier score indicate better performing models. A. The AUC (95% CI) among Black participants is 0.951 (0.941-0.961) for the creatinine equation with age, sex, and race; 0.951 (0.941-0.961) for the creatinine equation with age and sex; 0.945 (0.936-0.955) for the cystatin C equation with age and sex; 0.953 (0.943-0.962) for the creatinine and cystatin C equation with age, sex, and race; and 0.953 (0.944-0.962) for the creatinine and cystatin C equation with age and sex. B. The AUC (95% CI) among Black participants is 0.954 (0.944-0.962) for the creatinine equation with age, sex, and race; 0.954 (0.944-0.962) for the creatinine equation with age and sex; 0.945 (0.935-0.954) for the cystatin C equation with age and sex; 0.953 (0.943-0.961) for the creatinine and cystatin C equation with age, sex, and race; and 0.953 (0.943-0.961) for the creatinine and cystatin C equation with age and sex. C. The scaled Brier score (95% CI) among Black participants is 0.437 (0.406-0.467) for the creatinine equation with age, sex, and race; 0.451 (0.418-0.480) for the creatinine equation with age and sex; 0.368 (0.337-0.399) for the cystatin C equation with age and sex; 0.427 (0.397-0.457) for the creatinine and cystatin C equation with age, sex, and race; and 0.426 (0.396-0.457) for the creatinine and cystatin C equation with age and sex. D. The scaled Brier score (95% CI) among non-Black participants is 0.419 (0.384-0.449) for the creatinine equation with age, sex, and race; 0.404 (0.369-0.433) for the creatinine equation with age and sex; 0.357 (0.325-0.385) for the cystatin C equation with age and sex; 0.409 (0.375-0.438) for the creatinine and cystatin C equation with age, sex, and race; and 0.396 (0.365-0.426) for the creatinine and cystatin C equation with age and sex.
Table 2 and Appendix Tables 3-6 provide the discrimination ability of eGFR cut-off points for classifying 2-year ESKD risk among Black and non-Black participants. Among Black participants, an eGFR cut-off point of 20 mL/min/1.73 m2 had specificities ranging from 0.95 to 0.96 for all eGFR equations, and sensitivities ranging from 0.49 to 0.66. Conversely, a KFRE score cut-off point of 20% had specificities ranging from 0.94 to 0.95, and sensitivities ranging from 0.69-0.78. According to Youden’s index for these cut-off points, the creatinine equation with age and sex performed best. Among non-Black participants, an eGFR cut-off point of 20 mL/min/1.73 m2 had specificities ranging from 0.96 to 0.98, and sensitivities ranging from 0.42 to 0.51. Conversely, a KFRE score cut-off point of 20% had specificities ranging from 0.95-0.97 and sensitivities ranging from 0.68 to 0.74. According to Youden’s index for these cut-offs, the creatinine and cystatin C equation with age, sex, and race performed best.
Table 2.
Discrimination of ESKD Events by eGFR Levels and Kidney Failure Risk Equation Scores Classified Using Various Equations
| Discrimination of cut-off points |
CKD-EPI Equations to Estimate GFR | ||||
|---|---|---|---|---|---|
| Creatinine with Age, Sex, and Race |
Creatinine with Age and Sex |
Cystatin C with Age and Sex |
Creatinine and Cystatin C with Age, Sex and Race |
Creatinine and Cystatin C with Age and Sex |
|
| Black Participants | |||||
| Estimated glomerular filtration rate <20 mL/min/1.73 m2 | |||||
| Sensitivity (95% CI) | 0.57 (0.52-0.61) | 0.66 (0.61-0.70) | 0.49 (0.44-0.54) | 0.59 (0.53-0.64) | 0.59 (0.54-0.64) |
| Specificity (95% CI) | 0.96 (0.96-0.97) | 0.95 (0.94-0.95) | 0.95 (0.94-0.96) | 0.95 (0.95-0.96) | 0.95 (0.94-0.96) |
| Youden’s index (95% CI) * | 0.53 (0.48-0.57) | 0.60 (0.56-0.65) | 0.44 (0.39-0.49) | 0.54 (0.49-0.59) | 0.54 (0.49-0.59) |
| Kidney Failure Risk Equation predicted 2-year ESKD risk >20% | |||||
| Sensitivity (95% CI) | 0.73 (0.69-0.78) | 0.78 (0.74-0.82) | 0.69 (0.64-0.73) | 0.74 (0.70-0.78) | 0.75 (0.71-0.79) |
| Specificity (95% CI) | 0.95 (0.95-0.96) | 0.94 (0.93-0.95) | 0.95 (0.94-0.96) | 0.95 (0.94-0.95) | 0.94 (0.94-0.95) |
| Youden’s index (95% CI) * | 0.69 (0.64-0.73) | 0.72 (0.68-0.76) | 0.63 (0.59-0.68) | 0.69 (0.64-0.73) | 0.70 (0.65-0.74) |
| Non-Black Participants | |||||
| Estimated glomerular filtration rate <20 mL/min/1.73 m2 | |||||
| Sensitivity (95% CI) | 0.51 (0.46-0.55) | 0.42 (0.38-0.47) | 0.42 (0.37-0.46) | 0.50 (0.46-0.54) | 0.45 (0.40-0.49) |
| Specificity (95% CI) | 0.97 (0.96-0.97) | 0.98 (0.98-0.98) | 0.96 (0.96-0.97) | 0.96 (0.96-0.97) | 0.97 (0.97-0.98) |
| Youden’s index (95% CI) * | 0.48 (0.43-0.52) | 0.40 (0.36-0.45) | 0.38 (0.33-0.42) | 0.46 (0.42-0.51) | 0.42 (0.37-0.46) |
| Kidney Failure Risk Equation predicted 2-year ESKD risk >20% | |||||
| Sensitivity (95% CI) | 0.73 (0.68-0.77) | 0.69 (0.64-0.73) | 0.68 (0.63-0.72) | 0.74 (0.70-0.78) | 0.69 (0.65-0.73) |
| Specificity (95% CI) | 0.96 (0.95-0.96) | 0.97 (0.96-0.97) | 0.96 (0.95-0.96) | 0.95 (0.95-0.96) | 0.96 (0.96-0.97) |
| Youden’s index (95% CI) * | 0.68 (0.64-0.72) | 0.66 (0.61-0.70) | 0.63 (0.59-0.68) | 0.70 (0.65-0.73) | 0.65 (0.61-0.69) |
CI = confidence interval; CKD-EPI = Chronic Kidney Disease Epidemiology Collaboration; eGFR = estimated glomerular filtration rate; ESKD = end-stage kidney disease
Youden’s index is summary metric of validity that considers both sensitivity and specificity, and is calculated as: [sensitivity] + [specificity] – 1
Table 3 provides net reclassification statistics comparing various eGFR equations used within the KFRE score. Among Black participants, the KFRE score using the new creatinine equation with age and sex correctly reclassified 7.0% (95% CI, 4.6% to 9.5%) of ESKD events compared with the equation additionally including race, but incorrectly reclassified 3.8% (95% CI, 3.2% to 4.3%) of non-events; therefore, the net reclassification (95% CI) was 3.2 (0.5 to 5.9). Conversely, among non-Black participants, the KFRE score using the new creatinine equation with age and sex incorrectly reclassified 5.6% (95% CI, 3.6% to 7.8%) of ESKD events compared with the equation additionally including race, but correctly reclassified 2.8% (95% CI, 2.4% to 3.2%) of non-events; therefore, the net reclassification (95% CI) was −2.8 (−5.1 to −0.8). Cystatin C-based equations did not improve net reclassification compared with creatinine-based equations among both Black and non-Black participants.
Table 3.
Reclassification of ESKD Events and Non-events by Kidney Failure Risk Equation Scores Using Various Equations
| Black Participants | Non-Black Participants | |||||
|---|---|---|---|---|---|---|
| Model Comparisons * | Events Reclassified, % (95% CI) |
Non-events Reclassified, % (95% CI) |
Net Reclassification Index (95% CI) |
Events Reclassified, % (95% CI) |
Non-events Reclassified, % (95% CI) |
Net Reclassification Index (95% CI) |
| Comparing with creatinine-based equation with age, sex, and race | ||||||
| Creatinine with age and sex | 7.0 (4.6, 9.5) | −3.8 (−4.3, −3.2) | 3.2 (0.5, 5.9) | −5.6 (−7.8, −3.6) | 2.8 (2.4, 3.2) | −2.8 (−5.1, −0.8) |
| Cystatin C with age and sex | −6.3 (−11.3, −1.6) | 0.2 (−0.7, 0.9) | −6.1 (−11.1, −1.4) | −7.0 (−11.0, −2.9) | 0.4 (−0.2, 0.9) | −6.6 (−10.8, −2.5) |
| Creatinine and cystatin C with age and sex | 3.2 (−0.4, 6.4) | −1.6 (−2.3, −1.0) | 1.6 (−2.1, 4.7) | −5.7 (−9.0, −2.5) | 1.4 (0.9, 1.8) | −4.3 (−7.7, −1.2) |
| Comparing with creatinine-based equation with age and sex | ||||||
| Cystatin C with age and sex | −12.0 (−16.6, −7.5) | 3.4 (2.5, 4.3) | −8.6 (−13.3, −4.0) | −2.7 (−7.0, 1.5) | −2.0 (−2.6, −1.5) | −4.7 (−8.9, −0.3) |
| Creatinine and cystatin C with age and sex | −3.6 (−6.8, −0.5) | 2.0 (1.3, 2.7) | −1.5 (−4.8, 1.7) | −0.5 (−4.1, 3.0) | −1.3 (−1.8, −0.9) | −1.8 (−5.5, 1.7) |
CI = confidence interval; eGFR = estimated glomerular filtration rate; ESKD = end-stage kidney disease
Models are compared for reclassification of ESKD events to a higher predicted risk category and non-events to a lower predicted risk category, using cut-off points of 10% and 20%. Values >0 indicate correct reclassification while values <0 indicate incorrect reclassification.
Discussion
In this analysis of 3,873 participants with mild and moderate CKD, 1,631 of whom self-reported Black race, we found that the choice of eGFR equation did not impact the performance of eGFR and the KFRE score in predicting 2-year risk of ESKD. All five eGFR equations used within the KFRE score had excellent discrimination for ESKD events and each model was well-calibrated in both Black and non-Black participants, although the KFRE scores underestimated risk, particularly among Black participants. Additionally, we observed that the KFRE score was superior to eGFR alone for predicting ESKD, especially at key clinical cut-off points. For all eGFR equations, in both Black and non-Black participants, a KFRE score >20% had similar specificity (approximately 95%) to an eGFR cut-off point of <20 mL/min/1.73 m2 for 2-year ESKD risk and consistently improved sensitivity regardless of eGFR equation used.
The accurate estimation of GFR is essential for the diagnosis and management of CKD and its complications. Therefore, the choice of eGFR equation could have wide-ranging consequences for staging of CKD, medication dosing, dialysis referral, and transplant eligibility. Recent investigations have shown that eGFR values estimated in different ways, whether a removal of the race coefficient altogether (25) or a re-estimation of the equations without including race (11), can impact the prevalence of CKD. Furthermore, eGFR is a major criterion used for advanced kidney disease care referral in clinical guidelines (12,13,24). Because clinical decisions are often made at key clinical cut-off points, and previous eGFR equations have adjusted for race, the potential for disparate access to care has been acknowledged (10). This issue prompted the creation of a task force convened by the National Kidney Foundation and American Society of Nephrology to reassess the inclusion of race in estimating kidney function (26,27). New eGFR equations were developed that remove the race adjustment and recalculate, or refit, the other coefficients (11). Our results show that eGFR alone is an excellent predictor of ESKD, but the KFRE score had substantially better discrimination compared with eGFR alone in our study, regardless of which eGFR equation was used. Therefore, the value of the KFRE score may be an important additional consideration in addressing the current controversy surrounding eGFR race adjustment.
Duggal et al. recently evaluated the prediction of ESKD among participants of the Veterans Affairs (VA) health system and found that removal of the race coefficient from the 2009 CKD-EPI creatinine equation could improve 2-year ESKD risk prediction performance, particularly in terms of calibration (25). The reason for this improvement is a systematic downward shift in eGFR among Black participants, and consequent upward shift in predicted risk. In both the VA and CRIC Study samples, the 2-year KFRE score underestimates risk, particularly among Black participants. Therefore, our findings confirm those of Duggal et al. and extend them to the new 2021 CKD-EPI equations. In the current analysis, the creatinine equation with age and sex increased predicted risk estimates among Black participants, so it shifts this patient population closer to observed risks on average (i.e., improves calibration). Conversely, the equation decreases predicted risk estimates among non-Black participants, slightly worsening calibration. The associated clinically relevant consequences are that the new equations improve net reclassification of ESKD among Black participants (particularly by correctly reclassifying ESKD events to higher predicted risk groups) but worsen net reclassification among non-Black participants (particularly by incorrectly reclassifying ESKD events to lower predicted risk groups).
The current analysis shows that predicting ESKD risk based on clinical cut-points of eGFR alone is inferior to using predicted risk-based cut-off points employing the KFRE score, which additionally includes age, sex, and urinary ACR. The KFRE score has been widely validated in a range of patient populations and geographic regions (15). Grams et al. previously showed that clinical thresholds based on the KFRE score were superior for 1-year ESKD risk estimation compared with eGFR alone among 1,094 participants of the African American Study of Kidney Disease and Hypertension (28). This finding was also replicated in a large primary care sample from the United Kingdom for 2- and 5-year ESKD risk (29). In the current analysis, across all cut-off point comparisons and all participants, the KFRE score was superior to using eGFR alone for ESKD prediction, regardless of eGFR equation used. The results of our IDI analysis suggest that for all eGFR equations, at any given level of specificity, the sensitivity for the KFRE score vs. eGFR alone is >10% higher among both Black and non-Black participants. Because the 4-variable KFRE score simultaneously incorporates more information beyond eGFR, it is plausible to hypothesize it is less impacted by the choice of eGFR equation and whether or not to adjust for race. Our results support this hypothesis. Therefore, we note that a risk-based approach may not only improve prediction of ESKD among CKD patients, but simultaneously allow clinicians and health systems to not rely upon self-reported race, which is a controversial and ill-defined social construct (9,30). While several decision thresholds are possible, the favoring of sensitivity vs. specificity depends on clinical context, particularly the relative importance of screening (i.e., favor sensitivity) vs. recommending potentially invasive interventions (i.e., favor specificity). Compared with a widely-used eGFR cut-off point of <20 mL/min/1.73 m2 for recommending kidney transplant or dialysis referral, a KFRE score >20% had similar specificity for predicting 2-year ESKD risk (ranges, 0.94-0.97 vs. 0.95-0.98) but substantially higher sensitivity (ranges, 0.68-0.78 vs 0.42-0.66).
There are several limitations to our analysis. First, it is possible that Black vs. non-Black differences in the calibration of KFRE scores may be the result of residual confounding of factors not accounted for in the KFRE score. However, it was previously shown that the 6-variable KFRE score (which additionally accounts for diabetes and hypertension) and the 8-variable KFRE score (which additionally accounts for calcium, phosphate, bicarbonate, and albumin) did not improve prediction performance beyond the 4-variable equation in a multinational pooled analysis (15). Second, urinary ACR measurements were not available over follow-up in the CRIC Study. Therefore, we imputed urinary ACR from PCR using a validated conversion equation (19), which may increase variability in estimation of predicted risk. Third, while cystatin C measurements were internally calibrated in the CRIC study over follow-up, they are not calibrated to international standards, and this may be one explanation for why equations using cystatin C did not perform as well as equations using creatinine in the current study. However, this situation is similar to current clinical practice where cystatin C is not traceable to international standards. Finally, the current findings are based solely from data in a US population, so the findings may not be generalizable to global populations.
There are several important areas for future work. Despite our findings that the KFRE score affords excellent prediction of 2-year risk of ESKD, additional investigations are necessary to identify the optimal strategy for clinical implementation, as noted by clinical guidelines (12,13). Clinical trials testing implementation of the KFRE in clinical practice are needed. Additionally, there is a need for new ESKD risk prediction equations, based on new eGFR estimation approaches that do not include race, that are developed and validated in racially, ethnically, and geographically diverse samples. Black vs. non-Black discrepancies in calibration and reclassification in our study may be addressed by development of new equations.
In the current analysis, 2-year ESKD risk prediction performance was similar regardless of which eGFR equation was used. However, the 4-variable KFRE showed better sensitivity and specificity in our study compared with eGFR alone, it is easy to implement in routine clinical settings (31), and does not consider a patient’s race. A KFRE score >20% showed similar specificity (approximately 95%) but higher sensitivity compared with an eGFR <20 ml/min/1.73 m2. Therefore, a KFRE score >20% could be used for preparing kidney replacement therapy. The National Kidney Foundation and American Society of Nephrology task force recommends using the CKD-EPI creatinine equation refit without race, which can be readily implemented in US laboratories (27). In addition, the task force recommends increased measurement of cystatin C to facilitate use of the CKD-EPI creatinine and cystatin C equation refit without race, which may improve GFR estimation. Our results, which show that the choice of eGFR equation does not impact 2-year ESKD prediction, support these recommendations.
Supplementary Material
Acknowledgments:
The authors thank the other investigators, staff, and participants of the CRIC study for their valuable contribution. The CRIC Study Principal Investigators are Lawrence J. Appel, MD, MPH, Jing Chen, MD, MMSc, MSc, Harold I. Feldman, MD, MSCE, Alan S. Go, MD, James P. Lash, MD, Robert G. Nelson, MD, PhD, Mahboob Rahman, MD, Panduranga S. Rao, MD, Vallabh O. Shah, PhD, MS, Raymond R. Townsend, MD, Mark L. Unruh, MD, MS.
Funding Sources:
Funding for the CRIC Study was obtained under a cooperative agreement from National Institute of Diabetes and Digestive and Kidney Diseases (U01DK060990, U01DK060984, U01DK061022, U01DK061021, U01DK061028, U01DK060980, U01DK060963, U01DK060902 and U24DK060990). In addition, this work was supported in part by: the Perelman School of Medicine at the University of Pennsylvania Clinical and Translational Science Award NIH/NCATS UL1TR000003, Johns Hopkins University UL1 TR-000424, University of Maryland GCRC M01 RR-16500, Clinical and Translational Science Collaborative of Cleveland, UL1TR000439 from the National Center for Advancing Translational Sciences (NCATS) component of the National Institutes of Health and NIH roadmap for Medical Research, Michigan Institute for Clinical and Health Research (MICHR) UL1TR000433, University of Illinois at Chicago CTSA UL1RR029879, Tulane COBRE for Clinical and Translational Research in Cardiometabolic Diseases P20 GM109036, Kaiser Permanente NIH/NCRR UCSF-CTSI UL1 RR-024131, Department of Internal Medicine, University of New Mexico School of Medicine Albuquerque, NM R01DK119199. Dr. Bundy was supported by the National Institutes of Health/Eunice Kennedy Shriver National Institute of Child Health and Human Development career development grant (K12HD043451).
Footnotes
Disclosures: None
Contributor Information
Joshua D. Bundy, Department of Epidemiology, Tulane University School of Public Health and Tropical Medicine, New Orleans, LA; Tulane University Translational Science Institute, New Orleans, LA.
Katherine T. Mills, Department of Epidemiology, Tulane University School of Public Health and Tropical Medicine, New Orleans, LA; Tulane University Translational Science Institute, New Orleans, LA.
Amanda H. Anderson, Department of Epidemiology, Tulane University School of Public Health and Tropical Medicine, New Orleans, LA; Tulane University Translational Science Institute, New Orleans, LA; Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA.
Wei Yang, Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA.
Jing Chen, Department of Epidemiology, Tulane University School of Public Health and Tropical Medicine, New Orleans, LA; Tulane University Translational Science Institute, New Orleans, LA; Department of Medicine, Tulane University School of Medicine, New Orleans, LA.
Jiang He, Department of Epidemiology, Tulane University School of Public Health and Tropical Medicine, New Orleans, LA; Tulane University Translational Science Institute, New Orleans, LA; Department of Medicine, Tulane University School of Medicine, New Orleans, LA.
References
- 1.Levey AS, Coresh J. Chronic kidney disease. Lancet. 2012;379(9811):165–80. [DOI] [PubMed] [Google Scholar]
- 2.Levey AS, de Jong PE, Coresh J, et al. The definition, classification, and prognosis of chronic kidney disease: a KDIGO Controversies Conference report. Kidney Int. 2011;80(1):17–28. [DOI] [PubMed] [Google Scholar]
- 3.Coresh J, Turin TC, Matsushita K, et al. Decline in Estimated Glomerular Filtration Rate and Subsequent Risk of End-Stage Renal Disease and Mortality. JAMA. 2014;311(24):2518–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mills KT, Xu YY, Zhang W, et al. A systematic analysis of worldwide population-based data on the global burden of chronic kidney disease in 2010. Kidney Int. 2015. Nov;88(5):950–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Levey AS, Stevens LA, Schmid CH, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. 2009. May;150(9):604–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Inker LA, Schmid CH, Tighiouart H, et al. Estimating glomerular filtration rate from serum creatinine and cystatin C. N Engl J Med. 2012. Jul;367(1):20–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hsu C-Y, Yang W, Parikh RV, et al. Race, Genetic Ancestry, and Estimating Kidney Function in CKD. N Engl J Med. 2021. Sep; [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Vyas DA, Eisenstein LG, Jones DS. Hidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms. N Engl J Med. 2020. Aug;383(9):874–82. [DOI] [PubMed] [Google Scholar]
- 9.Levey AS, Titan SM, Powe NR, et al. Kidney Disease, Race, and GFR Estimation. Clin J Am Soc Nephrol. 2020. Aug;15(8):1203–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Diao JA, Inker LA, Levey AS, et al. In Search of a Better Equation - Performance and Equity in Estimates of Kidney Function. N Engl J Med. 2021. Feb;384(5):396–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Inker LA, Eneanya ND, Coresh J, et al. New Creatinine- and Cystatin C-Based Equations to Estimate GFR without Race. N Engl J Med. 2021. Sep; [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kidney Disease: Improving Global Outcomes (KDIGO) CKD Work Group. KDIGO 2012 Clinical Practice Guideline for the Evaluation and Management of Chronic Kidney Disease. Kidney Int Suppl. 2013;3(1):1–150. [Google Scholar]
- 13.Lok CE, Huber TS, Lee T, et al. KDOQI Clinical Practice Guideline for Vascular Access: 2019 Update. Vol. 75, American journal of kidney diseases : the official journal of the National Kidney Foundation. United States; 2020. p. S1–164. [DOI] [PubMed] [Google Scholar]
- 14.Tangri N, Stevens LA, Griffith J, et al. A predictive model for progression of chronic kidney disease to kidney failure. JAMA. 2011;305(15):1553–9. [DOI] [PubMed] [Google Scholar]
- 15.Tangri N, Grams ME, Levey AS, et al. Multinational Assessment of Accuracy of Equations for Predicting Risk of Kidney Failure: A Meta-analysis. JAMA. 2016. Jan;315(2):164–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Lash JP, Go AS, Appel LJ, et al. Chronic renal insufficiency cohort (CRIC) study: Baseline characteristics and associations with kidney function. Clin J Am Soc Nephrol. 2009;4(8):1302–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Denker M, Boyle S, Anderson AH, et al. Chronic renal insufficiency cohort study (CRIC): Overview and summary of selected findings. Clin J Am Soc Nephrol. 2015;10(11):2073–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Anderson AH, Yang W, Hsu CY, et al. Estimating GFR among participants in the Chronic Renal Insufficiency Cohort (CRIC) Study. Am J Kidney Dis. 2012;60(2):250–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Sumida K, Nadkarni GN, Grams ME, et al. Conversion of Urine Protein-Creatinine Ratio or Urine Dipstick Protein to Urine Albumin-Creatinine Ratio for Use in Chronic Kidney Disease Screening and Prognosis : An Individual Participant-Based Meta-analysis. Ann Intern Med. 2020. Sep;173(6):426–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the Performance of Prediction Models. Epidemiology. 2010;21(1):128–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Blanche P, Dartigues J-F, Jacqmin-Gadda H. Estimating and comparing time-dependent areas under receiver operating characteristic curves for censored event times with competing risks. Stat Med. 2013. Dec;32(30):5381–97. [DOI] [PubMed] [Google Scholar]
- 22.Pencina M, D’Agostino R Sr, D’Agostino R Jr, et al. Evaluating the added predictive ability of a newmarker: From area under the ROC curve to reclassification and beyond. Stat Med. 2008;27:157–72. [DOI] [PubMed] [Google Scholar]
- 23.Kattan MW, Gerds TA. The index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models. Diagnostic Progn Res. 2018;2:7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Chadban SJ, Ahn C, Axelrod DA, et al. KDIGO Clinical Practice Guideline on the Evaluation and Management of Candidates for Kidney Transplantation. Transplantation. 2020. Apr;104(4S1 Suppl 1):S11–103. [DOI] [PubMed] [Google Scholar]
- 25.Duggal V, Thomas I, Montez-Rath ME, et al. National Estimates of CKD Prevalence and Potential Impact of Estimating Glomerular Filtration Rate Without Race. J Am Soc Nephrol. 2021. Jun 1;32(6):1454–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Delgado C, Baweja M, Burrows NR, et al. Reassessing the Inclusion of Race in Diagnosing Kidney Diseases: An Interim Report from the NKF-ASN Task Force. J Am Soc Nephrol. 2021. Jun 1;32(6):1305–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Delgado C, Baweja M, Crews DC, et al. A Unifying Approach for GFR Estimation: Recommendations of the NKF-ASN Task Force on Reassessing the Inclusion of Race in Diagnosing Kidney Disease. J Am Soc Nephrol. 2021. Sep; [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Grams ME, Li L, Greene TH, et al. Estimating time to ESRD using kidney failure risk equations: results from the African American Study of Kidney Disease and Hypertension (AASK). Am J Kidney Dis. 2015. Mar;65(3):394–402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Major RW, Shepherd D, Medcalf JF, et al. The Kidney Failure Risk Equation for prediction of end stage renal disease in UK primary care: An external validation and clinical impact projection cohort study. PLoS Med. 2019. Nov;16(11):e1002955. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Braun L, Wentz A, Baker R, et al. Racialized algorithms for kidney function: Erasing social experience. Soc Sci Med. 2021. Jan;268:113548. [DOI] [PubMed] [Google Scholar]
- 31.Tangri N, Ferguson T, Komenda P. Pro: Risk scores for chronic kidney disease progression are robust, powerful and ready for implementation. Nephrol Dial Transplant. 2017. May;32(5):748–51. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.