Abstract
Acute kidney injury (AKI) is a frequent complication of cardiac surgery and increases morbidity and mortality. The identification of reliable biomarkers that allow earlier diagnosis of AKI in the postoperative period may increase the success of therapeutic interventions. Here, we conducted a prospective, multicenter cohort study involving 1219 adults undergoing cardiac surgery to evaluate whether early postoperative measures of urine IL-18, urine neutrophil gelatinase-associated lipocalin (NGAL), or plasma NGAL could identify which patients would develop AKI and other adverse patient outcomes. Urine IL-18 and urine and plasma NGAL levels peaked within 6 hours after surgery. After multivariable adjustment, the highest quintiles of urine IL-18 and plasma NGAL associated with 6.8-fold and 5-fold higher odds of AKI, respectively, compared with the lowest quintiles. Elevated urine IL-18 and urine and plasma NGAL levels associated with longer length of hospital stay, longer intensive care unit stay, and higher risk for dialysis or death. The clinical prediction model for AKI had an area under the receiver-operating characteristic curve (AUC) of 0.69. Urine IL-18 and plasma NGAL significantly improved the AUC to 0.76 and 0.75, respectively. Urine IL-18 and plasma NGAL significantly improved risk prediction over the clinical models alone as measured by net reclassification improvement (NRI) and integrated discrimination improvement (IDI). In conclusion, urine IL-18, urine NGAL, and plasma NGAL associate with subsequent AKI and poor outcomes among adults undergoing cardiac surgery. (Clinical Trials.gov number, NCT00774137).
Approximately 2 million cardiac surgeries are performed around the world each year. Acute kidney injury (AKI) is a frequent complication of cardiac surgery.1 Patients who double their serum creatinine or need acute dialysis have a 2- to 5-fold higher risk of death2,3 and patients who recover kidney function remain at increased risk of chronic kidney disease and premature death.4,5 Despite decades of research, no therapy has proven effective for human AKI. The failure of prior interventional trials has in part been attributed to delays in the diagnosis of AKI on the basis of early changes in serum creatinine.6 When the serum creatinine increases by 100% or more, it is obvious a patient has AKI, but this rise occurs days after the initial injury.7,8 In addition, serum creatinine is a measure of clearance rather than of direct kidney injury, and small increases may occur for reasons other than injury. Large consensus conferences have called for a better clinical paradigm that detects AKI earlier and more reliably using new plasma and urine biomarkers.9,10 Two of the most frequently studied new promising AKI biomarkers to date are IL-18 and neutrophil gelatinase-associated lipocalin (NGAL).11 Previous studies demonstrate that these biomarkers are released into the urine at the time of renal tubular injury.7,12,13 Plasma NGAL has also demonstrated encouraging early results in several studies.7,14 We conducted a large, prospective, multicenter cohort study of adult patients receiving cardiac surgery to evaluate whether early, postoperative measures of urine IL-18, urine NGAL, or plasma NGAL could identify which patients would develop AKI and other adverse patient outcomes.
RESULTS
Characteristics of the Study Cohort
We studied 1219 participants (Figure 1). The average age was 71 years and 68% were men. Most surgeries were elective (79%) and utilized cardiopulmonary bypass (90%). The mean preoperative estimated GFR (eGFR) was 67 ml/min per 1.73 m2 (Table 1). Sixty of 1219 participants (5%) developed AKI as defined by receipt of acute dialysis or doubling of serum creatinine at a median of 3 (interquartile range [IQR] 2 to 4) days after surgery. Eighteen patients (1.5%) received acute dialysis and 20 (1.6%) died before discharge. Patients who developed AKI were more likely to have a history of congestive heart failure, undergo combined coronary artery bypass graft (CABG) and valve surgery, have longer cardiopulmonary bypass time, and require a postoperative intra-aortic balloon pump and they had a higher preoperative serum creatinine (Table 1).
Figure 1.
Flow of study population.
Table 1.
Patient characteristics for those who did and did not develop AKI
| All (n = 1219) | Developed AKI |
P | ||
|---|---|---|---|---|
| Yes (n = 60) | No (n = 1159) | |||
| Age at the time of surgery, mean (SD) | 71 (10) | 71 (11) | 71 (10) | 0.72 |
| Age at the time of surgery, n (%) | 0.66 | |||
| <65 years | 267 (22%) | 17 (28%) | 250 (22%) | |
| 65 to 75 years | 455 (37%) | 20 (33%) | 435 (38%) | |
| 75 to 85 years | 438 (36%) | 20 (33%) | 418 (36%) | |
| >85 years | 59 (5%) | 3 (5%) | 56 (5%) | |
| Male gender, n (%) | 826 (68%) | 39 (65%) | 787 (68%) | 0.64 |
| White race, n (%) | 1141 (94%) | 55 (92%) | 1086 (94%) | 0.53 |
| Diabetes, n (%) | 511 (42%) | 29 (48%) | 482 (42%) | 0.30 |
| Hypertension, n (%) | 961 (79%) | 53 (88%) | 908 (78%) | 0.06 |
| Congestive heart failure, n (%) | 314 (26%) | 25 (42%) | 289 (25%) | 0.004 |
| Status of the procedure, n (%) | 0.02 | |||
| elective | 964 (79%) | 40 (67%) | 924 (80%) | |
| urgent or emergent | 255 (21%) | 20 (33%) | 235 (20%) | |
| Cardiac catheterization in the last 72 hours, n (%) | 123 (10%) | 6 (10%) | 117 (10%) | 0.93 |
| Operative characteristics | ||||
| Incidence, n (%) | 0.59 | |||
| first cardiovascular surgery | 1064 (87%) | 51 (85%) | 1013 (87%) | |
| redo cardiovascular surgery | 155 (13%) | 9 (15%) | 146 (13%) | |
| Surgery, n (%) | 0.004 | |||
| CABG | 585 (48%) | 20 (33%) | 565 (49%) | |
| valve | 355 (29%) | 16 (27%) | 339 (29%) | |
| CABG and valve | 279 (23%) | 24 (40%) | 255 (22%) | |
| CPB, n (%) | 0.002 | |||
| full | 1061 (87%) | 49 (82%) | 1012 (87%) | |
| none | 124 (10%) | 5 (8%) | 119 (10%) | |
| combination | 34 (3%) | 6 (10%) | 28 (2%) | |
| Perfusion time,a mean (SD) | 114 (60) | 177 (98) | 111 (55) | <0.0001 |
| Cross clamp time (minutes), mean (SD) | 78 (44) | 121 (66) | 76 (42) | <0.0001 |
| Cardioplegia, n (%) | 1077 (88%) | 55 (92%) | 1022 (88%) | 0.41 |
| Postoperative intra-aortic balloon pump, n (%) | 57 (5%) | 8 (14%) | 49 (4%) | 0.005 |
| Renal function | ||||
| Preoperative serum creatinine (mg/dl), median (IQR)b | 1.0 (0.9, 1.2) | 1.1 (0.9, 1.4) | 1.0 (0.9, 1.2) | 0.04 |
| Preoperative eGFR (ml/min per 1.73 m2), mean (SD) | 67 (19) | 63 (23) | 67 (19) | 0.17 |
| Preoperative eGFR, n (%) | 0.09 | |||
| eGFR ≥ 90 ml/min per 1.73 m2 | 146 (12%) | 6 (10%) | 140 (12%) | |
| eGFR 60 to 89 ml/min per 1.73 m2 | 649 (53%) | 29 (48%) | 620 (53%) | |
| eGFR 30 to 59 ml/min per 1.73 m2 | 387 (32%) | 20 (33%) | 367 (32%) | |
| eGFR 15 to 29 ml/min per 1.73 m2 | 37 (3%) | 5 (8%) | 32 (3%) | |
| Urine output | ||||
| Oliguria,c first postoperative day | 18 (2%) | 4 (7%) | 14 (1%) | 0.0007 |
| Oliguria,c first 3 days | 74 (6%) | 12 (20%) | 62 (5%) | <0.0001 |
| Urine output first postoperative day | ||||
| ml, median (IQR) | 1510 (1070, 2205) | 1050 (865, 1669) | 1530 (1095, 2210) | <0.0001 |
| ml/h, median (IQR) | 96 (68, 137) | 81 (57, 112) | 97 (69, 138) | 0.02 |
| Preoperative proteinuria by dipstickd | 205 (18%) | 13 (22%) | 192 (18%) | 0.39 |
| Preoperative medications | ||||
| β-blockers | 892 (73%) | 44 (73%) | 848 (73%) | 0.98 |
| ACE inhibitors or ARBs | 706 (58%) | 37 (62%) | 669 (58%) | 0.55 |
| aspirin | 889 (73%) | 46 (77%) | 843 (73%) | 0.50 |
| statins | 916 (75%) | 40 (67%) | 876 (76%) | 0.12 |
AKI defined by receipt of acute dialysis or a doubling in serum creatinine from the preoperative serum creatinine value. CPB, cardiopulmonary bypass; ARB, angiotensin receptor blockers; ACE, angiotensin-converting enzyme.
aPerfusion time is reported for the patients who had CPB.
bTo convert serum creatinine values to mmol/L, multiply by 88.4.
cOliguria defined as a patient who had <125 cc in 6 hours or <500 cc urine output in 24 hours.
dProteinuria by dipstick is defined as 30+, 100++, 300+++, or 2000 or more.
Traditional Biomarker Patterns and Performance
Median preoperative serum creatinine was 1.1 mg/dl (97 μmol/L) (IQR 0.9, 1.4 mg/dl) in those with AKI and 1.0 mg/dl (88 μmol/L) (IQR 0.9, 1.2 mg/dl) in those without (P = 0.04). The AUC of preoperative serum creatinine for AKI was 0.57 (SEM 0.03). Quintiles of preoperative serum creatinine were not associated with the risk of AKI. The first postoperative serum creatinine was equal to or lower than the baseline creatinine in 60% of the cohort. The proportion of patients with postoperative oliguria was higher in those with AKI; however, the incidence of oliguria was low (Table 1). Average urine output per hour on the day of surgery was not associated with the risk of AKI; the AUC of postoperative urine output for the diagnosis of AKI was 0.59 (SEM 0.04) and was 0.72 (SEM 0.04) for the first postoperative serum creatinine.
Biomarker Results and Postoperative Risk of AKI
The first postoperative samples (0 -to 6-hour time point) were collected at a median of 0.25 (IQR 0.1 to 0.5) hours after arrival to the intensive care unit (ICU). In patients with AKI, urine IL-18, urine NGAL, and plasma NGAL peaked at the first collection. Median levels of all three biomarkers were higher in those with AKI than those without AKI at each time point (Figure 2 and Supplementary Table 2).
Figure 2.
Urine IL-18, urine NGAL, and plasma NGAL peaked within 6 hours after surgery. Yellow bar indicates the IQR of the day to the first evidence of AKI in patients with AKI. Blue and green bars represent the IQR (25th to 75th percentiles) for AKI and non-AKI patients, respectively. The solid lines denote the median values. AKI was defined by receipt of acute dialysis or a doubling in serum creatinine during the hospital stay. Day 1 is the day of surgery, with time 0 representing the point when the patient arrived in the postoperative ICU.
Associations of the first postoperative biomarker measure (quintiles of urine IL-18, urine NGAL, plasma NGAL) with the risk of AKI are presented in Figure 3. Quintiles of urine IL-18, urine NGAL, and plasma NGAL had a graded relationship with the risk for AKI (P value for trend <0.001).
Figure 3.
Quintiles of urine IL-18, urine NGAL, and plasma NGAL had a graded relationship with the risk for AKI. Unadjusted P for trend is reported. Risk of AKI (%) is the percentage of AKI patients within each quintile. AKI was defined by receipt of acute dialysis or evidence of postoperative doubling of the preoperative serum creatinine value.
After adjustment for center and for clinical variables, the highest quintiles of urine IL-18 and plasma NGAL were associated with a 6.8-fold (95% confidence interval [CI] 1.9 to 24.3) and 5-fold (95% CI 1.6 to 15.3) adjusted odds when compared with the lowest quintile, respectively, for the development of AKI (Table 2). None of the quintiles of urine NGAL were independently associated with AKI.
Table 2.
Association of first postoperative biomarker quintiles with outcomes
| Quintile (cutpoints)b | Primary Outcome—AKIa |
Other Outcomes |
||||
|---|---|---|---|---|---|---|
| Percent AKI Patientsc | Unadjusted OR (95% CI) | Adjusted OR (95% CI)d | In-Hospital Death or Dialysis, % | Length of Stay in ICU, Mean (SD) | Length of Stay in Hospital, Mean (SD) | |
| Urine IL-18 (pg/ml) | ||||||
| Q1 (<3) | 1.3 | 1 (referent) | 1 (referent) | 0.4 | 2.8 (6) | 6.7 (6.7) |
| Q2 (3 to 8) | 2.1 | 1.7 (0.4, 7.1) | 1.6 (0.4, 6.7) | 2.1 | 2.5 (2.7) | 6.9 (3.7) |
| Q3 (8 to 17) | 3.3 | 2.7 (0.7, 10.4) | 2.3 (0.6, 9.3) | 1.3 | 2.7 (4.1) | 8 (8.4) |
| Q4 (17 to 60) | 4.6 | 3.8 (1, 13.8) | 2.8 (0.7, 10.5) | 2.1 | 3.2 (6.9) | 8.6 (8) |
| Q5 (>60) | 12.1 | 10.9 (3.3, 36) | 6.8 (1.9, 24.3) | 5.8 | 5.3 (15.3) | 12 (18) |
| unadjusted P for trend | <0.0001 | — | — | 0.0004 | 0.61 | <0.0001 |
| adjusted P for trendd | <0.0001 | — | — | 0.02 | 0.002 | <0.0001 |
| Urine NGAL (ng/ml) | ||||||
| Q1 (<3) | 2.5 | 1 (referent) | 1 (referent) | 0.4 | 2.7 (2.6) | 6.4 (3.1) |
| Q2 (3 to 7) | 2.9 | 1.2 (0.4, 3.5) | 1 (0.3, 3.0) | 0.8 | 2.4 (3.6) | 7.3 (7.4) |
| Q3 (7 to 16) | 2.9 | 1.2 (0.4, 3.5) | 0.9 (0.3, 2.9) | 1.7 | 2.7 (6.2) | 7.6 (7.3) |
| Q4 (16 to 102) | 4.2 | 1.7 (0.6, 4.7) | 1.1 (0.4, 3.3) | 3.3 | 3.1 (5.2) | 8.6 (6.7) |
| Q5 (>102) | 10.8 | 4.7 (1.9, 11.7) | 2.5 (0.9, 6.8) | 5.4 | 5.6 (15.9) | 12.2 (18.8) |
| unadjusted P for trend | <0.0001 | — | — | <0.0001 | 0.66 | <0.0001 |
| adjusted P for trendd | 0.03 | — | — | 0.01 | 0.01 | 0.0007 |
| Plasma NGAL (ng/ml) | ||||||
| Q1 (<105) | 1.7 | 1 (referent) | 1 (referent) | 0.4 | 2.1 (2.2) | 6.3 (3) |
| Q2 (105 to 158) | 2.1 | 1.3 (0.3, 4.7) | 1.1 (0.3, 4.1) | 0 | 2.2 (1.8) | 6.7 (4.2) |
| Q3 (158 to 218) | 4.6 | 2.8 (0.9, 9) | 2.3 (0.7, 7.4) | 1.3 | 2.9 (4.6) | 7.5 (5.3) |
| Q4 (218 to 292) | 3.4 | 2 (0.6, 6.9) | 1.4 (0.4, 5.1) | 2.1 | 3.2 (5) | 8.9 (9.8) |
| Q5 (>293) | 11.9 | 7.8 (2.7, 22.6) | 5.0 (1.6, 15.3) | 7.6 | 5.6 (16) | 12.4 (18.5) |
| unadjusted P for trend | <0.0001 | — | — | <0.0001 | <0.0001 | <0.0001 |
| adjusted P for trendd | 0.0005 | — | — | 0.0007 | 0.0002 | <0.0001 |
OR, odds ratio.
aAKI was defined by the receipt of acute dialysis or a doubling in serum creatinine during the hospital stay.
bThe number of patients per quintile: urine IL-18 n = 240, urine NGAL n = 240, plasma NGAL n = 236.
cPercent AKI patients within each quintile.
dAdjusted for age (per year), gender, white race, CPB time >120 minutes, nonelective surgery, preoperative eGFR, diabetes, hypertension, and center.
Biomarker Results and Nonrenal Outcomes
The average lengths of stay in the ICU and in the hospital for the entire cohort were 3.4 days (SD 8) and 8.5 days (SD 10), respectively. Urine IL-18, urine NGAL, and plasma NGAL were all linearly associated with a composite outcome of in-hospital death or dialysis and the length of hospital and ICU stay after adjustment for other prognostic factors (Table 2). Eighteen percent of patients with AKI died, and 1% of patients without AKI died (P < 0.001).
Diagnostic Testing (Area under the Curve)
The area under the receiver-operating characteristic curve (AUC) for the clinical model using pre- and intraoperative variables for AKI was 0.69 (SEM 0.04). The AUC for each biomarker for the detection of AKI is shown in Figure 4. When added to the clinical model, urine IL-18 improved the AUC to 0.76 (P = 0.03) and plasma NGAL also improved the AUC to 0.75 (P = 0.01). Urine NGAL did not improve the AUC above that of the clinical model (Table 3). The AUCs of the biomarkers at 0 to 6, 6 to 12, and 12 to 18 hours as well as at day 2 are shown in Supplementary Table 4a. Various combinations of the biomarkers were simultaneously able to increase the AUC to a maximum of 0.77 (Supplementary Table 4b).
Figure 4.
Diagnostic performance of the first postoperative value of urine IL-18, urine NGAL, and plasma NGAL for the detection of AKI. Receiver-operating characteristic (ROC) curves show the diagnostic performance of the four biomarkers and the table shows the performance of each biomarker at the fifth quintile.
Table 3.
AUC, NRI, and IDI of clinical model with first postoperative biomarkers
| AUC |
IDI |
NRI |
||||||
|---|---|---|---|---|---|---|---|---|
| Biomarker | Biomarker + Clinical Modela | Clinical Modela | Pb | IDI (SEM)c | P | NRI (SEM)c | P | |
| Urine IL-18 (pg/ml) | 0.74 (0.04) | 0.76 (0.03) | 0.69 (0.04) | 0.03 | 0.02 (0.007) | 0.002 | 0.25 (0.10) | 0.01 |
| Urine NGAL (ng/ml) | 0.67 (0.04) | 0.73 (0.04) | 0.12 | 0.01 (0.01) | 0.08 | 0.14 (0.09) | 0.12 | |
| Plasma NGAL (ng/ml) | 0.70 (0.04) | 0.75 (0.04) | 0.01 | 0.02 (0.007) | <0.0001 | 0.18 (0.09) | 0.05 | |
Values in parentheses are SEM.
aClinical model is comprised of age (per year), gender, white race, CPB time > 120 minutes, nonelective surgery, preoperative eGFR, diabetes, and hypertension.
bP value AUC of biomarker + clinical model compared with clinical model.
cIDI and NRI quantify the improvement of the biomarkers on predicting the risk of AKI. Comparing the clinical model to the biomarker + clinical model, NRI considers an improvement in reclassification as if an AKI patient moves up a risk category or if a non-AKI patient moves to a lower risk category. Similarly a worse reclassification occurs if an AKI patient moves down a risk category or if a non-AKI patient moves up a risk category. Overall, NRI is the difference in the proportion of improvements in reclassification and the proportion of worse reclassifications. The IDI formula quantifies the reclassification continuously instead of categorically.
Risk Prediction (Net Reclassification Improvement and Integrated Discrimination Improvement)
To evaluate the improvement of risk prediction with the addition of biomarkers to the clinical model, we determined the net reclassification improvement (NRI) and the integrated discrimination improvement (IDI) indices. We initially categorized all participants as being at low (<3%), intermediate (3% to 10%), or high (>10%) AKI risk on the basis of a prediction model that incorporated all clinical variables and risk categories.
Urine IL-18 provided improved risk prediction over the pre- and intraoperative clinical models alone: NRI 0.25 (P = 0.01), IDI 0.02 (P = 0.002) (Table 3 and Tables 4a–4c). Plasma NGAL also improved risk prediction: NRI 0.18 (P = 0.05), IDI 0.02 (P < 0.001). Urine NGAL did not improve risk classification after accounting for clinical variables.
Table 4a.
Risk classification for first postoperative urine IL-18 after addition to clinical model
| Clinical Model | Clinical Model + Urine IL-18 |
Total | ||
|---|---|---|---|---|
| <3% | 3% to 10% | >10% | ||
| AKI participants | ||||
| <3% | 8 (47.1%) | 9 (52.9%) | 0 (0%) | 17 |
| 3% to 10% | 3 (13.6%) | 5 (22.7%) | 14 (63.6%) | 22 |
| >10% | 0 (0%) | 6 (35.3%) | 11 (64.7%) | 17 |
| total | 11 | 20 | 25 | 56 |
| Non-AKI participants | ||||
| <3% | 484 (82.2%) | 105 (17.8%) | 0 (0%) | 589 |
| 3% to 10% | 147 (31.5%) | 231 (49.5%) | 89 (19.1%) | 467 |
| >10% | 4 (4.7%) | 43 (50%) | 39 (45.3%) | 86 |
| total | 635 | 379 | 128 | 1142 |
Clinical model is age (per year), gender, white race, CPB time > 120 minutes, nonelective surgery, preoperative eGFR, diabetes, and hypertension.
Table 4c.
Risk classification for first postoperative plasma NGAL after addition to clinical model
| Clinical Model | Clinical Model + Plasma NGAL |
Total | ||
|---|---|---|---|---|
| <3% | 3% to 10% | >10% | ||
| AKI participants | ||||
| <3% | 11 (64.7%) | 6 (35.3%) | 0 (0%) | 17 |
| 3% to 10% | 2 (9.1%) | 7 (31.8%) | 13 (59.1%) | 22 |
| >10% | 0 (0%) | 6 (35.3%) | 11 (64.7%) | 17 |
| total | 13 | 19 | 24 | 56 |
| Non-AKI participants | ||||
| <3% | 456 (79%) | 121 (21%) | 0 (0%) | 577 |
| 3% to 10% | 139 (30.1%) | 248 (53.7%) | 75 (16.2%) | 462 |
| >10% | 0 (0%) | 43 (51.8%) | 40 (48.2%) | 83 |
| total | 595 | 412 | 115 | 1122 |
Clinical model is age (per year), gender, white race, CPB time > 120 minutes, nonelective surgery, preoperative eGFR, diabetes, and hypertension.
Table 4b.
Risk classification for first postoperative urine NGAL after addition to clinical model
| Clinical Model | Clinical Model + Urine NGAL |
Total | ||
|---|---|---|---|---|
| <3% | 3% to 10% | >10% | ||
| AKI participants | ||||
| <3% | 12 (70.6%) | 5 (29.4%) | 0 (0%) | 17 |
| 3% to 10% | 1 (4.5%) | 10 (45.5%) | 11 (50%) | 22 |
| >10% | 0 (0%) | 6 (35.3%) | 11 (64.7%) | 17 |
| total | 13 | 21 | 22 | 56 |
| Non-AKI participants | ||||
| <3% | 509 (86.4%) | 80 (13.6%) | 0 (0%) | 589 |
| 3% to 10% | 94 (20.1%) | 288 (61.7%) | 85 (18.2%) | 467 |
| >10% | 0 (0%) | 42 (48.8%) | 44 (51.2%) | 86 |
| total | 603 | 410 | 129 | 1142 |
Clinical model is age (per year), gender, white race, CPB time > 120 minutes, nonelective surgery, preoperative eGFR, diabetes, and hypertension.
Supplementary Analyses
When we repeated the analyses using urine-creatinine-corrected values for urine IL-18 (pg/mg) and urine NGAL (ng/mg), there was no improvement in AUCs compared with the uncorrected results. We also examined the association between the first postoperative day biomarker value and the outcome of AKI within 5 days of surgery, but this analysis did not appreciably change the overall AUCs or biomarker associations. All of the results (quintile analyses, AUCs, NRI/IDI) were similar in magnitude and direction for a secondary outcome of mild AKI defined by a 50% increase in serum creatinine (data shown in Supplementary Tables 3 and 4). When we examined whether the discriminatory ability of biomarkers differed by patient characteristics (chronic kidney disease, oliguria, and diabetes), we found nonsignificant differences.
Finally, to study the utility of the biomarkers when postoperative variables were considered, we excluded the patients (n = 24) who already developed mild AKI on the basis of the first postoperative value. When postoperative variables (such as urine output and postoperative creatinine) were added to the model, we observed the following: (1) the upper quintile of urine IL-18 was still independently associated with AKI (odds ratio 7.3 [95% CI 1.5, 34.9]) (Supplementary Table 5), but quintiles of plasma and urine NGAL were not significantly associated with AKI; (2) the AUCs were not improved significantly from the clinical models by any of the biomarkers; and (3) only the NRI for urine IL-18 was still significant for severe AKI (NRI 0.20, P = 0.01).
DISCUSSION
For the past decade, there have been a host of investigations surrounding novel structural biomarkers of AKI because they may facilitate patient management and development of therapies. Most previous clinical studies of biomarkers for AKI were small in size, evaluated individual biomarkers, and were performed at single centers. The study presented here reports on three of the most frequently studied AKI biomarkers published to date with a broad participation of patients from many centers, and it evaluates several in-hospital outcomes after cardiac surgery. The findings of the previous studies over the last few years were mixed; thus, a study with large enough numbers to ascertain the utility of biomarkers has been uniquely needed. In this multicenter cohort study of adults undergoing cardiac surgery, we confirmed the paradigm that markers of tissue injury—urine IL-18 and plasma NGAL—were associated with the development of AKI. A postoperative level of urine IL-18 > 60 pg/ml denoted >6-fold risk of AKI compared with patients in the lowest quintile, whereas a postoperative level of plasma NGAL > 293 ng/ml denoted a 5-fold risk of AKI. These two biomarkers modestly improved the accuracy for the diagnosis of AKI as seen by AUC results; however, the biomarkers strongly improved the risk prediction by 25% and 18%, respectively, when added to clinical models using pre- and intraoperative variables. However, unlike the previous studies, these biomarkers performed only modestly with the outcome of mild AKI, and urine NGAL was not independently associated with AKI after consideration of clinical variables. The first postoperative levels of urine IL-18, urine NGAL, and plasma NGAL did demonstrate graduated relationships with important clinical outcomes, including length of stay in the hospital, dialysis, and death during hospitalization.
Our multicenter study is the largest AKI biomarker study performed to date and strengthens a new paradigm that immediate postoperative measurement of urine or plasma biomarkers of kidney injury can yield important prognostic information. However, similar to other validation experiences, the prognostic significance of these markers was less impressive than first reported.15,16 Although urine IL-18 and plasma NGAL were able to improve risk stratification for AKI, urine NGAL was not as effective in our cohort. Nonetheless, the ability of perioperative urine IL-18 and plasma NGAL to predict outcomes throughout hospitalization could lead to the novel clinical approach of detecting early structural kidney injury rather than the current clinical standard of relying on markers that reflect changes in glomerular filtration.
Urine IL-18 and NGAL are specific to kidney tubules and are biologic intermediates in the causal mechanisms of ischemia-reperfusion injury in the kidney. Both biomarkers are present at very low concentrations preoperatively, and their levels increase by severalfold in patients who develop AKI. IL-18, a known mediator of inflammation, is produced by proximal tubules and is activated by caspase-1; mice deficient in caspase-1 are protected from AKI because of impaired IL-18 processing.12,13 During ischemia-reperfusion injury in mice, the gene for NGAL is significantly upregulated in the kidney, and the protein is overexpressed in distal tubule cells.17 Experimental studies indicate that AKI also results in a dramatic increase in NGAL mRNA and protein expression in distant organs such as the lungs and liver.18 As an iron-transporting protein, NGAL may play a primary role in renal survival and recovery and has been used therapeutically in ischemia-reperfusion injury animal models.19 Additional nonrenal contributions to the plasma NGAL in AKI may include activated neutrophils, macrophages, and other immune cells, all of which are pertinent to humans undergoing surgeries with cardiopulmonary bypass.
The diagnostic performance of the individual urine and plasma biomarkers as well as their combinations did not exceed an AUC of 0.77. This performance may in part be explained by the use of a doubling of serum creatinine as part of the primary AKI definition—a feasible but imperfect method to assess true kidney injury. In truth, meticulous animal and human studies with nephrotoxic agents causing AKI have demonstrated that the association between elevations in serum creatinine and biopsy-proven kidney injury is modest.20 Rather, many novel biomarkers performed better than serum creatinine when histologic evidence of kidney injury was used as the reference standard.21 Hence, the U.S. Food and Drug Administration and European Medicines Agency have accepted multiple rodent urinary and plasma biomarkers as surrogates for renal injury on histology and accepted these for evaluation of newer therapeutic agents.22 We do not know with certainty whether the relative poor performance of the biomarkers in this study reflected the limitation of the biomarkers themselves or was again hampered by the nonspecificity of serum creatinine as a “gold standard” for AKI. As done with troponin and brain natriuretic peptide, proving associations with long-term outcomes such as chronic kidney disease and mortality will be useful to determine whether these new biomarkers should be adopted in routine clinical practice.23
Although these biomarkers are associated with AKI and other important outcomes, their potential value could be to identify individuals who might benefit from specific interventions for AKI to prevent complications. Many treatments that have proven beneficial for animal AKI have not translated to humans. This has been attributed in part to limitations of serum creatinine measurements that may not identify true injury or identify injury when it is too late to intervene. An expanded therapeutic window could potentially be achieved for prospective therapies to treat AKI by using early measurements of these new urine and plasma biomarkers as inclusion criteria with biomarker cutpoints that possess very high specificity. The improvement in risk prediction for AKI with biomarkers can also facilitate enrollment in new trials for AKI because the classification accuracy was not outstanding.
A strength of our study was that prospective complete specimen collection was performed under standardized conditions in consecutive patients undergoing cardiac surgery across multiple centers in the United States and Canada. AKI was defined according to modern AKI staging systems and as an end point useful for future therapeutic trials (RIFLE [Risk Injury Failure Loss End-Stage Kidney Disease]24 [Injury] or Acute Kidney Injury Network [AKIN] stage 2).25 This AKI end point that requires doubling of serum creatinine is also less prone to prerenal azotemia and fluctuations in baseline creatinine. However, there may have been patients with substantial tubular injury, but with a rise in serum creatinine < 100%, who were labeled and analyzed as no AKI (misclassification of outcome). In fact, although preclinical studies suggest strong biologic plausibility, we cannot be certain whether the associations between biomarkers and outcome are entirely attributable to kidney injury. We were unable to examine biomarker excretion rate because we did not have timed collections. Also, serum creatinine was not collected as frequently as other urine biomarkers after surgery. Finally, our cohort of adults was predominantly of Caucasian race; future studies should consider whether the results are the same other races.
In summary, early postoperative levels of urine IL-18 and plasma NGAL, biologic measures of kidney injury, were associated with AKI, mortality, and length of stay in the days after cardiac surgery. Although the biomarkers lacked outstanding classification for AKI, risk prediction may be a more relevant measure of performance for their clinical and research use.26 Thus, future studies will need to determine how best biomarkers of AKI can be utilized to guide prognostication, resource use, and enrollment into trials of therapeutic intervention above and beyond our current clinical measures.
CONCISE METHODS
Study Population
We prospectively enrolled adults undergoing cardiac surgery (CABG or valve surgery) who were at high risk for AKI at six academic medical centers in North America between July 2007 and December 2009. High risk for AKI was defined by the presence of one or more of the following: emergency surgery, preoperative serum creatinine > 2 mg/dl (>177 μmol/L), ejection fraction < 35% or grade 3 or 4 left ventricular dysfunction, age > 70 years, diabetes mellitus, concomitant CABG and valve surgery, or repeat revascularization surgery. We excluded patients with evidence of AKI before surgery, prior kidney transplantation, preoperative serum creatinine level > 4.5 mg/dl (>398 μmol/L), or end-stage renal disease. Participants with multiple surgeries could only be enrolled in the study once. All participants provided written informed consent, and each institution's research ethics board approved the study. The reporting of this study follows guidelines set out in the Strengthening the Reporting of Observational Studies in Epidemiology statement (Supplementary Table 1).
Sample Collection
We collected urine and plasma specimens preoperatively and daily for up to 5 postoperative days. The first postoperative samples were collected soon after admission to the ICU. For the first 24 hours postoperatively, urine samples were collected every 6 hours. The remaining daily blood and urine samples were obtained at the time of routine morning blood collection done for clinical care. Specimen collection was stopped on postoperative day 3 in subjects who had not yet had an increase in serum creatinine. We obtained fresh urine samples from the urimeter of the Foley catheter system and centrifuged the samples at 3000g for 10 minutes to remove cellular debris. Blood was collected in EDTA tubes and was centrifuged to separate plasma. We aliquoted urine supernatant and plasma into bar-coded cryovials and stored the samples at −80°C until biomarker measurement. No additives or protease inhibitors were added.
AKI Biomarker Measurements
We measured the two urine biomarkers with the ARCHITECT assay (Abbott Diagnostics, Abbott Park, IL). We measured urine creatinine by the modified Jaffe reaction. The intra-assay coefficient of variation (CV) for the urine creatinine assay was 5%, whereas the CVs for the NGAL and IL-18 assays were approximately 5% and 8%, respectively. We performed the measurements in two batches, approximately 1 year apart. We also confirmed a high correlation between the Abbott platform and the ELISA method used in previous studies7,18 in 189 randomly selected samples (urine NGAL r = 0.95; urine IL-18 r = 0.89; P < 0.001).
We measured plasma NGAL using the Triage NGAL immunoassay in conjunction with the Triage Meter (Biosite, Inc., San Diego, CA) in two batches 7 months apart. The Triage assay has a detection range of 60 to 1300 ng/ml with a CV of 10% to 15%.
Personnel performing the biomarker measurements were blinded to each patient's clinical information. All biomarkers were measured from frozen aliquots that did not undergo any additional freeze-thaw cycles. Repeat measurement of randomly selected samples between the batches confirmed high correlation without any assay drift for all of the assays measured in two batches. Urine NGAL and IL-18 and plasma NGAL concentrations are stable at −80°C for 2 years without any protease inhibitors with a variability of 2% to 8% (unpublished data).27,28
Outcome Definitions
The primary outcome was the development of AKI, defined as receiving acute dialysis during the entire hospital stay or a doubling in serum creatinine from the baseline preoperative value (RIFLE24 stage I or AKIN25 stage 2 AKI). We also analyzed outcome of mild AKI, defined by a 50% increase in serum creatinine (RIFLE stage “R”) or requirement of dialysis. All preoperative creatinine values were measured within 2 months before surgery. Pre- and postoperative serum creatinine levels were measured in the same clinical laboratory for each patient at all centers. Serum creatinine values were recorded for every patient throughout the hospital stay. Additional clinical outcomes were in-hospital mortality and length of in-hospital and ICU stay.
Variable Definitions
We collected preoperative characteristics, operative details, and postoperative complications using definitions of the Society of Thoracic Surgeons (http://www.ctsnet.org/file/rptDataSpecifications252_1_ForVendorsPGS.pdf). We recorded whether the patient had received cardiac catheterization within 72 hours before surgery. We estimated preoperative GFR (eGFR) using the Chronic Kidney Disease Epidemiology Collaboration equation.29
Statistical Analysis
We divided the population into quintiles using the first postoperative value of urine IL-18, urine NGAL, and plasma NGAL, respectively. We assessed unadjusted linear trends by the Cochran–Armitage test for dichotomous outcomes, the Jonckheere–Terpstra test for continuous outcomes, and adjusted linear trends using contrasts in logistic or linear regression, respectively. Continuous variables were compared with a two-sample t test or Wilcoxon rank sum test and dichotomous variables with the χ2 test or Fisher's exact test. We determined the adjusted odds ratios of AKI with mixed logistic regression with random intercepts for each center. We adjusted for important covariates that predict AKI in the cardiac surgery setting,30 including patient demographics (age [per year], gender, white race), clinical risk factors (preoperative eGFR, hypertension, diabetes) and operative characteristics (elective or urgent procedure, use and duration of cardiopulmonary bypass [>120 minutes]). We used AUC to determine the ability of the biomarkers to discriminate between patients with and without AKI. We used receiver-operating characteristic regression to determine if the separation of the biomarker values between patients with and without AKI differed by chronic kidney disease (defined by an eGFR < 60 ml/min/ per 1.73 m2), oliguria (urine output < 500 cc/24 h), or diabetes.31 We quantified the improvement of biomarkers on AKI risk prediction with the NRI and IDI indices.32 For NRI analyses, risk category definitions were based on clinical utility and the incidence of the outcomes in the study. For AKI, risk categories were classified as low (<3%), medium (3% to 10%), or high (>10%) and for mild AKI risk categories were classified as low (<10%), medium (10% to 20%), or high (10% to 20%) risk. We compared AUCs using the test developed by DeLong et al.33 We used logistic regression models to estimate biomarker combinations (up to three biomarkers in each model) with crossvalidation (3-fold).34 The discriminatory ability of the biomarker combinations was estimated with the AUC in the validation dataset.35 Urinary biomarkers were not corrected for urine creatinine in primary analyses, but they were corrected for creatinine in sensitivity analyses. We performed the analyses in SAS version 9.2 (SAS Institute, Cary, NC) and R 2.10.1 (R Foundation for Statistical Computing, Vienna, Austria). (Table 4)
Disclosures
C.R.P. and C.L.E. are named co-inventors on the IL-18 patent. P.D. is the co-inventor on the NGAL patents. C.R.P., S.G.C., and P.D. are consultants to Abbott Diagnostics. P.D. is a consultant to Biosite, Inc.
Acknowledgments
Members of Translational Research Investigating Biomarker Endpoints for Acute Kidney Injury (TRIBE-AKI) Consortium:
Yale-New Haven: Dr. Michael Dewar, Dr. Umer Darr, Dr. Sabet Hashim, Dr. Richard Kim, Dr. John Elefteriades, Dr. Arnar Geirsson, Dr. Susan Garwood, Ms. Elizabeth Dombrowski, Dr. Prakash Nadkarni, Dr. Simon Li, Ms. Laurie Cagnetta, Ms. Rowena Kemp, Ms. Laura Piscitelli, Dr. Madiha Koraishy, Ms. Maureen Legenos, Ms. Susan Collazo, Ms. Mary Ann Tranquilli
Danbury: Ms. Judy Nagy
London: Ms. Ellen Annett, Dr. Michael Chu, Ms. Stephanie Fox, Dr. Martin Goldbach, Dr. Lin Ruo Guo, Ms. Janice Hewitt, Ms. Anne Jackson, Dr. Bob Kiaii, Ms. Jodi Marshall, Ms. Elizabeth McEachnie, Dr. Neil McKenzie, Ms. Heather Motloch, Dr. Mary Lee Myers, Dr. Richard Novick, Ms. Kathy Pennell, Dr. Mac Quantz, Ms. Virginia Schumann, Ms. Crystal Watt, Ms. Laura Webster, Ms. Puiyan Wong, Mr. Sam Vijayan
Chicago: Dr. Patrick Murray, Dr. Shahab A. Akhter, Dr. Jai Raman, Dr. Valluvan Jeevanandam, Ms. Karah Herges
We would also like to thank the nursing and support staff of the preadmission clinic, anesthesia units, and the cardiac care units at all of the participating centers. The research reported in this article was supported by the American Heart Association Clinical Development award (grant R01HL-085757) from the National Heart, Lung, and Blood Institute. The study was also supported by a Clinical and Translational Science Award grant (UL1 RR024139) from the National Center for Research Resources. The urine biomarker assays were donated by Abbott Diagnostics, and plasma NGAL was donated by Biosite. The granting agencies, Abbott Diagnostics and Biosite, Inc., did not participate in the protocol development, analysis, or interpretation of the results.
Footnotes
Published online ahead of print. Publication date available at www.jasn.org.
See related editorial, “Blind Men and Elephants and the Biological Markers of AKI,” on pages 1578–1580.
Supplemental information for this article is available online at http://www.jasn.org/.
REFERENCES
- 1. Rosner MH, Okusa MD: Acute kidney injury associated with cardiac surgery. Clin J Am Soc Nephrol 1: 19–32, 2006 [DOI] [PubMed] [Google Scholar]
- 2. Chertow GM, Levy EM, Hammermeister KE, Grover F, Daley J: Independent association between acute renal failure and mortality following cardiac surgery. Am J Med 104: 343–348, 1998 [DOI] [PubMed] [Google Scholar]
- 3. Zanardo G, Michielon P, Paccagnella A, Rosi P, Calo M, Salandin V, Da Ros A, Michieletto F, Simini G: Acute renal failure in the patient undergoing cardiac operation. Prevalence, mortality rate, and main risk factors. J Thorac Cardiovasc Surg 107: 1489–1495, 1994 [PubMed] [Google Scholar]
- 4. Coca SG, Yusuf B, Shlipak MG, Garg AX, Parikh CR: Long-term risk of mortality and other adverse outcomes after acute kidney injury: A systematic review and meta-analysis. Am J Kidney Dis 53: 961–973, 2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Hobson CE, Yavas S, Segal MS, Schold JD, Tribble CG, Layon AJ, Bihorac A: Acute kidney injury is associated with increased long-term mortality after cardiothoracic surgery. Circulation 119: 2444–2453, 2009 [DOI] [PubMed] [Google Scholar]
- 6. Park M, Coca SG, Nigwekar SU, Garg AX, Garwood S, Parikh CR: Prevention and treatment of acute kidney injury in patients undergoing cardiac surgery: A systematic review. Am J Nephrol 31: 408–418 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Mishra J, Dent C, Tarabishi R, Mitsnefes MM, Ma Q, Kelly C, Ruff SM, Zahedi K, Shao M, Bean J, Mori K, Barasch J, Devarajan P: Neutrophil gelatinase-associated lipocalin (NGAL) as a biomarker for acute renal injury after cardiac surgery. Lancet 365: 1231–1238, 2005 [DOI] [PubMed] [Google Scholar]
- 8. Wagener G, Jan M, Kim M, Mori K, Barasch JM, Sladen RN, Lee HT: Association between increases in urinary neutrophil gelatinase-associated lipocalin and acute renal dysfunction after adult cardiac surgery. Anesthesiology 105: 485–491, 2006 [DOI] [PubMed] [Google Scholar]
- 9. American Society of Nephrology Renal Research Report. J Am Soc Nephrol 16: 1886–1903, 2005 [DOI] [PubMed] [Google Scholar]
- 10. Scirica BM, Morrow DA: Troponins in acute coronary syndromes. Prog Cardiovasc Dis 47: 177–188, 2004 [DOI] [PubMed] [Google Scholar]
- 11. Parikh CR, Lu JC, Coca SG, Devarajan P: Tubular proteinuria in acute kidney injury: A critical evaluation of current status and future promise. Ann Clin Biochem 47: 301–312, 2010 [DOI] [PubMed] [Google Scholar]
- 12. Melnikov VY, Ecder T, Fantuzzi G, Siegmund B, Lucia MS, Dinarello CA, Schrier RW, Edelstein CL: Impaired IL-18 processing protects caspase-1-deficient mice from ischemic acute renal failure. J Clin Invest 107: 1145–1152, 2001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Melnikov VY, Faubel S, Siegmund B, Lucia MS, Ljubanovic D, Edelstein CL: Neutrophil-independent mechanisms of caspase-1- and IL-18-mediated ischemic acute tubular necrosis in mice. J Clin Invest 110: 1083–1091, 2002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Haase M, Bellomo R, Devarajan P, Schlattmann P, Haase-Fielitz A: Accuracy of neutrophil gelatinase-associated lipocalin (NGAL) in diagnosis and prognosis in acute kidney injury: A systematic review and meta-analysis. Am J Kidney Dis 54: 1012–1024, 2009 [DOI] [PubMed] [Google Scholar]
- 15. Macdonald JS: Carcinoembryonic antigen screening: Pros and cons. Semin Oncol 26: 556–560, 1999 [PubMed] [Google Scholar]
- 16. Boerrigter G, Costello-Boerrigter LC, Burnett JC, Jr: Natriuretic peptides in the diagnosis and management of chronic heart failure. Heart Fail Clin 5: 501–514, 2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Mishra J, Ma Q, Prada A, Mitsnefes M, Zahedi K, Yang J, Barasch J, Devarajan P: Identification of neutrophil gelatinase-associated lipocalin as a novel early urinary biomarker for ischemic renal injury. J Am Soc Nephrol 14: 2534–2543, 2003 [DOI] [PubMed] [Google Scholar]
- 18. Devarajan P: Review: Neutrophil gelatinase-associated lipocalin: A troponin-like biomarker for human acute kidney injury. Nephrology (Carlton) 15: 419–428 [DOI] [PubMed] [Google Scholar]
- 19. Mishra J, Mori K, Ma Q, Kelly C, Yang J, Mitsnefes M, Barasch J, Devarajan P: Amelioration of ischemic acute renal injury by neutrophil gelatinase-associated lipocalin. J Am Soc Nephrol 15: 3073–3082, 2004 [DOI] [PubMed] [Google Scholar]
- 20. Hotchkiss RS, Swanson PE, Freeman BD, Tinsley KW, Cobb JP, Matuschak GM, Buchman TG, Karl IE: Apoptotic cell death in patients with sepsis, shock, and multiple organ dysfunction. Crit Care Med 27: 1230–1251, 1999 [DOI] [PubMed] [Google Scholar]
- 21. Bonventre JV, Vaidya VS, Schmouder R, Feig P, Dieterle F: Next-generation biomarkers for detecting kidney toxicity. Nat Biotechnol 28: 436–440 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Dieterle F, Sistare F, Goodsaid F, Papaluca M, Ozer JS, Webb CP, Baer W, Senagore A, Schipper MJ, Vonderscher J, Sultana S, Gerhold DL, Phillips JA, Maurer G, Carl K, Laurie D, Harpur E, Sonee M, Ennulat D, Holder D, Andrews-Cleavenger D, Gu YZ, Thompson KL, Goering PL, Vidal JM, Abadie E, Maciulaitis R, Jacobson-Kram D, Defelice AF, Hausner EA, Blank M, Thompson A, Harlow P, Throckmorton D, Xiao S, Xu N, Taylor W, Vamvakas S, Flamion B, Lima BS, Kasper P, Pasanen M, Prasad K, Troth S, Bounous D, Robinson-Gravatt D, Betton G, Davis MA, Akunda J, McDuffie JE, Suter L, Obert L, Guffroy M, Pinches M, Jayadev S, Blomme EA, Beushausen SA, Barlow VG, Collins N, Waring J, Honor D, Snook S, Lee J, Rossi P, Walker E, Mattes W: Renal biomarker qualification submission: A dialog between the FDA-EMEA and Predictive Safety Testing Consortium. Nat Biotechnol 28: 455–462 [DOI] [PubMed] [Google Scholar]
- 23. Antman EM, Tanasijevic MJ, Thompson B, Schactman M, McCabe CH, Cannon CP, Fischer GA, Fung AY, Thompson C, Wybenga D, Braunwald E: Cardiac-specific troponin I levels to predict the risk of mortality in patients with acute coronary syndromes. N Engl J Med 335: 1342–1349, 1996 [DOI] [PubMed] [Google Scholar]
- 24. Bellomo R, Ronco C, Kellum JA, Mehta RL, Palevsky P; Acute Dialysis Quality Initiative workgroup: Acute renal failure—Definition, outcome measures, animal models, fluid therapy and information technology needs: The Second International Consensus Conference of the Acute Dialysis Quality Initiative (ADQI) Group. Critical Care 8: R205–R212, 2004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Mehta RL, Kellum JA, Shah SV, Molitoris BA, Ronco C, Warnock DG, Levin A: Acute Kidney Injury Network: Report of an initiative to improve outcomes in acute kidney injury. Crit Care 11: R31, 2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Pencina MJ, D'Agostino RB, Sr, Steyerberg EW: Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med 30: 11–21, 2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Grenier FC, Ali S, Syed H, Workman R, Martens F, Liao M, Wang Y, Wong PY: Evaluation of the ARCHITECT urine NGAL assay: Assay performance, specimen handling requirements and biological variability. Clin Biochem 43: 615–620, 2010 [DOI] [PubMed] [Google Scholar]
- 28. Han WK, Wagener G, Zhu Y, Wang S, Lee HT: Urinary biomarkers in the early detection of acute kidney injury after cardiac surgery. Clin J Am Soc Nephrol 4: 873–882, 2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro AF, III, Feldman HI, Kusek JW, Eggers P, Van Lente F, Greene T, Coresh J: A new equation to estimate glomerular filtration rate. Ann Intern Med 150: 604–612, 2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Mehta RH, Grab JD, O'Brien SM, Bridges CR, Gammie JS, Haan CK, Ferguson TB, Peterson ED: Bedside tool for predicting the risk of postoperative dialysis in patients undergoing cardiac surgery. Circulation 114: 2208–2216; quiz 2208, 2006 [DOI] [PubMed] [Google Scholar]
- 31. Janes H, Pepe MS: Adjusting for covariates in studies of diagnostic, screening, or prognostic markers: An old concept in a new setting. Am J Epidemiol 168: 89–97, 2008 [DOI] [PubMed] [Google Scholar]
- 32. Pencina MJ, D'Agostino RB, Sr., D'Agostino RB, Jr, Vasan RS: Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Stat Med 27: 157–172; discussion 207–212, 2008 [DOI] [PubMed] [Google Scholar]
- 33. DeLong ER, DeLong DM, Clarke-Pearson DL: Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 44: 837–845, 1988 [PubMed] [Google Scholar]
- 34. Hastie T, Tibshirani R, Friedman J: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, New York, Springer-Verlag, 2007 [Google Scholar]
- 35. Baker SG, Kramer BS, McIntosh M, Patterson BH, Shyr Y, Skates S: Evaluating markers for the early detection of cancer: Overview of study designs and methods. Clin Trials 3: 43–56, 2006 [DOI] [PubMed] [Google Scholar]